Monday, August 17, 2009

Mining for Knowledge

'Knowledge mining' is a concept taught to computer science students about how to look for interesting patterns and structures in otherwise meaningless and large collection of data. The word 'mining' in 'knowledge mining' struck me as rather interesting. So much so that I stopped to ponder about its implications. Mining is usually associated with valuable stuff such as precious metals and stones, ores, etc. So it got me thinking, " My God..! Knowledge must be really valuable". My mind started wandering and as usual I started to form connections between hypothesis A and observation B until I was pretty sure that I had a theory C.

I won't go into this theory C for it is rather personal to me ( yes so personal that I won't post it on the blog ). However, I am glad to talk about a related topic that is of interest to me.

The topic is about what sort of jobs people like to do. I liked Nassim Nicholas Taleb's definition( He is the author of 'fooled by randomness' and 'black swan', if you haven't read it, I suggest it ). He created a dichotomy of all jobs that are out there. He labeled jobs as 'scalable' and 'non scalable'. Scalable jobs are those that give you a disproportionate amount of reward compared to the number of hours of work you do. Such as that of an artist who makes it big, a wall street trader, a politician, a drug lord, etc. These jobs are often associated with lots of glamor as there is often a chance to make it big really quick. A certain lottery, casino, gambling like factor.

On the other hand non scalable jobs give you the payback that you deserve based on the number of hours you put in and what your skill set is. The skill set is often acquired through a number of hours of hard work. These people are dentists, engineers, teachers, plumbers and even police officers. These jobs are mundane. Boring. Hardly worth bragging about. I am sure that the people doing these kind of jobs have at some point of time had doubts about their profession and often wished they were on the other side. However, Nassim advices people to take a job that is non scalable. He believes that a job where you get paid for the amount of knowledge you possess and the amount of work you put in, is a desirable job. Life is no longer indeterminant. You are the captain of your life, the cartographer of your destiny.

I agree with Nassim.


heartfelt13 said...

Nice writing... I don't completely agree though :)

Scalable jobs have their attractions...There's a certain satisfaction in treading paths that no one has walked on before...To not have a control on your destiny.. to search for it... :)

Ya, I Am A Dastard said...

I don't completely agree for the same reason as heartfelt13 :)

If all goes well, I well end up pretty scalable :)

Kolor said...

@ heartfelt13
@ adi
Since I am taking a stat course of late..
I would say that a non scalable job is one that has the minimax property i.e.

A job, J* such that

J* = arg min(max(Risk(J))

and E[Pay(J*)] > E_job[E_pay[Pay(J)]]

where E is expectation