Online resources to study machine learning

I have been collecting information on online resources for machine learning. I share them as it will surely help several grad students. If you have some interesting ML links please do post them in the comments.

Definition

• Many people are not clear about the boundaries (as flimsy as they may) between traditional AI, Neuroscience and Machine Learning. This introduction by Tom Michell ought to help.

Where to study from?

• Lectures by Andrew Ng (Stanford) are said to be a good start.
• Hundreds more of ML lectures are available at videolectures.net. These videos are good to learn more about topics that you are interested in.
• Long list of textbooks to read (links 1 and 2). This text is free online.

How to keep up with the field?

• ML is a fast changing field and you need to update yourself with the latest papers. The journals/conferences I found to be relevant are:

1. JMLR
2. ICML
3. NIPS
4. COLT
5. Pattern Recognition
6. TPAMI
7. Machine Learning

• I found these blogs to be useful too

1. http://hunch.net/
2. http://www.reddit.com/r/MachineLearning/
3. http://metaoptimize.com/qa/
   
4. http://mark.reid.name/iem
5. CIML

Programming tools

• Not everyone can afford Matlab. Fortunately Python (with its Numpy extension) can be used as a substitute.
• Before implementing any standard algorithm look for online ML libraries with it. (tip: some libraries work only on *nix environments)

Machine learning has a lot in common with statistics, in fact it can be said to be derived from statistics. This link has a funny take on why ML is more popular.

Addendum (06/01/2010)

Here are some links to online lecture videos on related topics. There might be other course videos out there. Do your own search if you have time.

• Calculus: Several course videos here. My pick
• Linear Algebra: The famous MIT course by Gilbert Strang
• Probability theorem: From UCLA
• Convex optimization: Stanford lectures by Stephen Boyd. Part 2 of the course is available on youtube

Stop reading Ayn Rand

I was introduced to Ayn Rand during my undergrad days. I had read The Fountain Head and frankly had thought that it was a decent book. Later I found that several of my friends and acquaintances swore by it and labelled it as eye opening. I had begun to wonder if it really deserved all that praise. Then I got my hands on the 1100 page tome of hers, Atlas Shrugged, which actually is one the biggest English language novels of all time.

It is the toughest book I've ever read. Tough not because it was too cerebral or because of its length, but because its one of the most boring fiction novels I've laid my hands on. I finished it because it was a challenge, because I looked cool reading it during the commute to work and because I could proclaim to my friends over the coffee table that I had read it and hence was better than them. Not until somewhere near the 700 page mark did it stop being a potent soporific. After that the plot does get interesting for a few hundred pages. But then it gets carried away and ends with the most conceited and bizarre 50 page monologue ever by John Galt mofo that you just might bang your head on a wall screaming, 'Why the fuck did I start reading this shit?' Reading Atlas Shrugged converted me from an Rand agnostic into a strident Rand anti-theist.

Somehow Atlas Shrugged has developed and maintained its reputation as an eye opener, a classic that ought to be a text book instead of being labelled a boring book with crazy, bitchy lead characters who belong either in jail or in a mental asylum. When I later heard an unanimously hated manager at my office pledge his admiration for Rand and wish that he could emulate her heroes (this during an overseas work call) I smiled inside thinking no wonder people hated him. Whenever I saw someone read that book I warned them about it. And yet they all continued to read it and a tiny percentage of them managed to live through the exercise. I figured out why they do it. Its because it is intellectual masturbation. What about all those Rand worshipers who wake up early in the morning and recite verses from her work? Jerks all of them.

At this point you are probably wondering why I rant on about Rand being the individual responsible for the most number of hours wasted by humanity (OK second most, the first being Shah Rukh Khan). It is because of this. A serious wtf trend that is the most incongruous given the current hatred towards corporate greed. Why? Why are the jerks winning? I want to stymie this trend and hence this post.

First let me point out why reading Atlas Shrugged and The Fountainhead (to a lesser extent) is a waste of time:
1. Atlas Shrugged is obviously too lengthy with too many boring monologues that could have been edited out without missing on any content. I believe if the whole thing was shortened to around 500 pages its would have been a good read.

2. Some Problems with Rand's Objectivism - A meritocratic society that values creativity, integrity and innovation is undoubtedly good. But pushing capitalism to the extreme and applying techniques used at the stock market to dating and in relationships is ridiculous.

3. Superhuman/inhumane heroes: Roark, Rearden, Galt and that other guy were probably born in Krypton. I agree that it is wonderful to love and obsess about your work or creation. But to blow up a completed housing project even when the design was an obscene mockery of your original work is simply a crime. They are all egocentric maniacs without an iota of compassion. Disagreements can happen between equals without either party acknowledging the other to be on par. But if the disagreeing parties behave the way these guys do life would be impossible.

4. Bitchy Heroines: Obvious to anyone who reads it. The rationale given for the infidelities is stupid.

5. Ignorant and useless Populace: Almost all the people other than her central characters are slackers and leechers. Her lack of faith in contemporary society is staggering

6. Views on Altruism: She implies that letting her heroes thrive would ultimately help humanity more than any form of charity. This is probably true. But if charity is not highly regarded the world would be a much worse place. I am not engaging in wishful thinking here. Just think of the missionaries imparting education to remote African villages. They might do it for religious reasons but the people there benefit. Once these regions develop they might open up new markets or at least stop being a threat for existing markets there. But this might take one or more life times and hence this kind of charity might not be of interest to an objectivist. May be if the world had paid more attention to Somalia we wouldn't have pirates running amok today in the Indian Ocean. This is why I believe all forms of charity should be praised or in other words why none of them should be ridiculed.

There are several other people who share my opinion. Here are some funny ones: cracked, another rant (more extreme). Just search for Ayn Rand criticism or Ayn Rand evil to see what I mean.

Even if you dont agree with any of this it still doesn't make sense to read and praise Atlas Shrugged. Almost everyone today, especially among the young generation that makes up most of Rand's readers, recognizes the importance of meritocracy, individual rights and integrity. Do you really need to torture yourself and read this humongous book to learn what you already know?

I HAVE THE ANSWERS

The last few days have been tough for me. I had to conquer a few inner daemons, namely laziness and eh more laziness, only to be held back by a violent bout with indigestion. One might wonder how I got through those troubling times. Stop wondering. I was aided and inspired by the strongest force in the universe, Heman.

In between my research work I spent a lot of time watching all the episodes of 2002 version of Heman. In fact watching it was also a kind of research. Those of you who watched the 80s version of the show probably asked yourself some very hard hitting questions like "What is this 'power'?", "Where can I find that sword?", "Will I look like Heman when I grow up?", etc... The answer to some of them became apparent in the following years (though I am very very muscular and am probably as strong as some super heroes like Batman, I am still not as strong as Heman). But some of questions have vexed me for several years . The 2002 series gives answers to all those questions and does it in a much more engrossing manner. No longer do I have to spent sleepless nights wondering how I can defend the planet from evil sorcerers.

I am going to quickly list these new revelations:

Grayskull: Not the castle. Eternia had an all powerful king, Grayskull who defended the planet against villains who would make skeletor look like the kinder garden bully. When the entire planet was threatened by a very evil guy (I think his name was Hortag) he managedre to cast him and his army away into another dimension but the tremendous effort killed him. On his death all his powers transferred to his council, ever since called the elders, and to his sword. His wife is entrusted to guard the sword and the castle containing it.

Elders: After defending Eternia for a long time they foretold the coming of heman and retired storing their powers in castle Grayskull.

Sorceress: The first one was the queen of Eternia, Grayskull's wife. When one of them retires a replacement is found.

Skeletor: A villain keldor who was after the power of elders, before they retired. When he breached their fortress Randor, who later became the king - Adam's father, defeated him. Keldor was fatally injured in the fight and to save his life summoned Hortag's spirit. Hortag healed him but turned him into skeletor giving him more powers (I think).

Teela: She is the daughter of the sorceress and Man-at-arms Duncan's brother. Hence her adopted father Duncan is her uncle. She will one day become the sorceress.

Snake mountain: Actually the dead body of a snake god Serpos who had tried to conquer Eternia

When Adam bellows 'By the power of grayskull', the power of the ancient king flows into him.

To change back to Adam Heman chants 'Let the power return' or something like that.

Phew!

Jaswant and Jinnah

I usually stay clear of politics but this topic is too interesting. I am not a historian which is why I have given several citations. Pakistanis credit Jinnah as the creator of their nation and Indians also credit (may be blame) him for the same. A few days back Jaswant Singh, a respected figure in Indian politics, proclaimed that Jinnah was not to be solely blamed and that leaders of Indian National Congress were equally or even more responsible for splitting the nation. I spent a little bit of time googling this topic myself and this is what I found.

The British fanned the flames of communalism with their divide and rule policy but they didn't start the mess. The origins of the Hindu-Muslim schism seem to have been socio-economic. According to [1] [2] there was resentment among the Muslim elite who stood to loose influence with the advent of English as the official language and of modern education. It seems Hindus were faster to adapt to those conditions and Muslims began to loose prominence. The Muslim elite would have lost even more with the introduction of democratic processes and in 1906 they formed the All India Muslim League. They demanded and secured separate legislative councils for Muslims.

Jinnah [5] was member of INC who later drifted away and became the leader of AIML. Jinnah branded INC as a Hindu party despite the fact that several Hindu parties such as Hindu Mahasaba (they performed poorly in the elections) and organizations such as RSS already existed. Even in 1927 when he was the leader of AIML Jinnah opposed the idea of separate electorates for Muslims and put forth the 14 points [3] as a regulation of INCs idea of unified electorates. INC rejected the idea. I guess this is Jaswant Singh's cardinal point, but read the 14 points and compare that to the democratic ideals of INC. After a long hiatus from politics he returned to India from England to campaign for the 1937 elections. AIML fared poorly losing in most of the Muslim majority states but gaining votes in Muslim minority areas. INC won in Hindu majority areas [4] and parties such as Unionist Party won in Muslim majority areas. Jinnah then offered to ally with Congress if they accepted them as the sole representatives of Indian Muslims and shared power with them. INC of course did not accept the ridiculous offer.

Jinnah went on a virulent campaign to make all Muslims accept AIML as their sole representative and in 1940 began to demand a separate country for Muslims. He successfully campaigned in NFWP, Baluchistan and Punjab, where he impressed upon them that the Muslims in the other states (where they were a minority) would be mistreated by their congress governments and that AIML was the only party that could help them by representing all Muslims in British India. In the elections of 1946 AIML had made drastic improvements and won most of the Muslim majority electorates, but INC had won majority of the votes in the country. The British tried to avoid partition with a call for separate constitutions for separate regions which INC and AIML would jointly govern, but this idea was also rejected by AIML [6]. An interim government was formed by the majority party INC which infuriated Jinnah and led him to declare "We do not want war, if you want war we accept your offer unhesitatingly. We shall have India divided or we shall have India destroyed." The infamous direct action campaign [7] was launched and thousands of Hindus and Muslims were killed, raped or maimed in the concomitant violence. After that no one could hinder AIML. With the looming threat of civil war the creation of Pakistan was inevitable.

So there it is, born out of a socio-economic power struggle Pakistan is a caricature of what it was meant to be. Created to prevent mistreatment of Muslims, it got split due to bias against East Pakistan (Bangladesh) and has long been accused of bigotry against the people of Baluchistan by the power centre in Punjab province. To make matters worse NWFP is teeming with tribal warlords, Taliban etc. and is said to be one of most dangerous places in the world.

Coming back to Jaswant Singh, this is how I see it. INC wanted to represent whole of India. AIML wanted to be regarded as the sole representative of Indian Muslims. But in a democracy the majority party would rule and hence AIML could never govern the country. Even though some Muslim leaders such as those in Unionist Muslim League [8] wanted to collaborate with INC (in fact they had collaborated), Jinnah was opposed to this. I think its clear who is responsible for the creation of Pakistan. Blaming the INC is pointless. I don't think the way to handle dissent is to capitulate to the dissenting party. The INC's mistake seems to be that they ignored Jinnah and that they did not foresee the partition. As for calling Jinnah secular, how can anyone responsible for the reprehensible direct action day be called secular?

[1] - http://en.wikipedia.org/wiki/Pakistan_Movement

[2] - http://mrmulliner.sulekha.com/blog/post/2006/08/journey-of-communalism-in-modern-india.htm

[3] - http://en.wikipedia.org/wiki/Fourteen_Points_of_Jinnah

[4] - http://mrmulliner.sulekha.com/blog/post/2006/08/growth-of-communalism-in-india-ii-1937-1947.htm, also read this

[5] - http://en.wikipedia.org/wiki/Muhammad_Ali_Jinnah

[6] - http://en.wikipedia.org/wiki/1946_Cabinet_Mission_to_India

[7] - http://en.wikipedia.org/wiki/Direct_Action_Day

[8] - http://en.wikipedia.org/wiki/Unionist_Muslim_League

Interesting side note [1]: The word 'Pakistan' referred to the five Northern units of India, Viz: Punjab, (Afghanistan Province), Kashmir, Sind and Baluchistan.

[2]: Khan Abdul Ghaffer Khan tried to emulate Gandhiji in NFWP. A sad story.

[3]: Great Bong has an interesting post about this topic

Incident at Heganahalli - Part 3

Author's Note: Read part 1 and part 2 first.

The solution came to me as clear and loud as a KSRTC bus. It was in fact a KSRTC bus parked close by. The driver must have seen the mob, abandoned all hope and run away. The mob would reach the bus in about 30 minutes I calculated. But before that they would probably damage all the shops in the area. I knew what to do to attract them all away from the destruction and pillaging and to satiate the primordial drives that seemed to have hijacked their senses. I hurried into the bus and slowly drove it close to the junction. Then I honked twice.

"Pom Pom."

One by one the rioters stopped their frenzied rampage and walked to the bus. They were like the moths that fly towards the light in my sit out. I had run out the back door to my bike and was ready to flee at the first sign of danger. A plump middle aged lady jerked off a road divider** and smashed a window of the bus. I caught the look on her face during the act. There was pure bliss there. Others quickly followed and then some smarty put a towel in the petrol tank and lit it. The metal giant burst into flames.

Everyone gathered around the burning locomotive as if to pay last respects. Several minutes passed and yet no one moved. They dropped the weapons they held; their yearnings had been quelled, energies spent and it was time to return to their families. The mob was dispersing. I took one last look at the juggernaut that stopped for the common man, the cheap transport of the masses that always inexplicably bore the brunt of their anger, before heading back to the station. On my way I passed a fire engine and an ambulance trying to make its way to the carnage almost one and half hours after the violence had erupted.

By the time I headed back home that day I was despondent. The brush with real danger, the madness of the mob and above all the mistreatment by that wretched inspector had made me feel depressed. The worst part was that I couldn't treat myself with my usual medicine. Top Star was no more. It seems many of Gowda's other bars had also been destroyed that day. I thought I should get used to it and went to a wine shop that was on the way and bought a quarter of RC.

As I entered my house I saw Malathi sitting near the door step. She looked distraught; her eyes were moist with tears. She hugged me and whispered in between sobs, "I was so worried."

"I am OK," I said beginning to feel better already.

------------------------------------------------------------------------------------------------

**- based on an actual event

Incident at Heganahalli - Part 2

Author's Note: Read part 1 first.

My heart sank. "I am right here," I said as I sheepishly moved forward from behind the HC.

"Right. Go ahead and find out where they are now and how bad the situation is." He then proceeded to yell more orders with so much gusto that I wished for an umbrella to shield me from all the spit that came my way. Anbu Valsan was hurt. He didn't care about the wound. He only cared about the rule of terror. What was the point in being a police man if the low-lifes didn't buckle at their knees on seeing us? How else could ten of us reign over a thousand of them? We all knew that the halo of terror was our best weapon, the scepter with which we subjugated the masses.

I was to go ahead as a scout and do reconnaissance. Meanwhile the rest of them would collect themselves, wake up those who were still sleeping, get the lathis, helmets, cane shields etc., to ready to take on a mob. I guess saar felt that I didn't need any of that extra protection. What a [synonym for cat]. So I left the station not knowing how much danger was on my way, only knowing that I needed to transfer to another station soon.

As I neared the Jaaravanahudi junction I caught the acrid smell of burning tyres. Several of them were stacked in the middle of the road and people holding [ruling party] flags were forcing nearby shops to down shutters. They held aloft banners that read "He ordered wine, he got swine", "Paid for pegs, served like pigs", "Gobi chilli good, piggy chills bad" and so on. But the centerpiece was a huge banner that read, "Justice to Mangappa. Gowda's liquor killed his ticker." In between the mayhem I saw that something else was burning. Top Star Bar.

"Nooooooo." But I couldn't get close. One of the miscreants spotted me and uttered a guttural cry that gave me the shivers. I turned my bike around and raced away with the sound of my rattling heart drowning all else. After traversing a safe distance I called the station.

"Yah we know there is a riot there. We got a complaint a few minutes back."

"Why didn't you call me?" my voice broke like that of a pubescent boy. Apparently none of those [hen drinker (drink like with a straw)] thought about me. I felt like tearing my uniform apart and joining the rioters. Just then two jeeps zoomed past me with sirens blaring. The cavalry had arrived. Normally when the ruling party instigates a riot we keep a distance. But this time it was personal; at least for the psychopath who ran the show.

My colleagues jumped off their jeeps with lathi's held high and the dozen and a half of them beat into pulp the lone rioter who had strayed too far away from the rest. Soon the other rioters saw this and attacked the police men with their sticks, tube lights, cycle chains etc. A bloody battle ensued and within minutes the men in khakhi were lying prostrate on the ground groveling and groaning. Some of them were trying to escape by crawling on all fours.

To my surprise one of the jeeps turned around and came towards me. It was him, my enemy. While his men were getting pulverised, the inspector had stayed back in his jeep. "He is probably worried about spoiling his complexion," I guessed.

"Stay here and make sure it doesn't get out of hand." And then he sped away leaving me gaping in disbelief.

The mob was getting bigger. It was no longer just party gundas. People had been intimidated by incessant news reports about the swine flu for months. They were desperate, but impotent. They would have grabbed any chance to get even with the invisible enemy, no matter how incredulous it was. The time was ripe for a riot. As more people gathered at the junction shouting slogans and blocking vehicles, Top Star Bar burned away in the background.

Then it got ugly. They pelted stones at a Maruthi car that was passing by and then pushed a middle aged man off his scooter and burned it. They were getting angrier and hungry for more violence. I knew I was in terrible danger. They had to be pacified somehow, I had to quell their rage or at least get out of their way.

continued in part 3....