Curated path of curiosity from collective intelligence (and a python script to discover it on twitter)

I don’t know where to start. The Interweb is an ocean of intrigue. I’m stuck staring at it from a distant beach. Towel in hand with nothing but a search bar to keep my hunger pangs away. Server farms flow into this ocean without any intention of respite.

I float by in a small dingy. Too much water to contend with. I bake in the sun and let others bombard me with information. Sugary news keeps me chubby and lethargic. There’s no energy left to swim and explore. I must consider the second-order effects of reading sugar and too much news.

Too many islands and no guide at my side. I procrastinate and boredom begins. I end up staring at a screen. A sage on sail tells me I’m playing defence with no room for offence.

She was right. Now I do pay close attention to my information diet.

Most know that we are a product of the five people we spend the most time with. But most don’t realise that this applies more so to information consumption.

Our rivers flood with information each day. Most of us are passively floating down these rivers. Except when any information disagrees with them. We shelter ourselves from information that makes us feel uncomfortable. We leave aside anything that can’t be easily digested.

We seek, create and fight for affirmation. When we read something that agrees with us we feel good, intelligent and safe. We fall into an information diet that comforts and numbs us. People create false narratives to avoid dealing with harsh reality.

Are society’s problems due to passive information diets and weakly tested, consensus driven opinions?

I don’t know. I’m floating by soaking up the sun.

My personality type is INTJ. I’m most excited in the activity of detective work over long periods. Lifting up untouched rocks revealing unseen artifacts.

In investing, you learn that it’s difficult to be right and costs much more to be wrong. A ton of work is just a prerequisite. And after that you can still be wrong. It’s hard to be confident that your opinions are right.

Bad opinions are costly. But in life, it’s easy to survive with poorly thought-out opinions rooted in consensus. Investors know well how consensus can be wrong and costly.

Given the stakes, investors keep opinions as assumptions. They continue to gather information. They try to find smarter people to stress test their views. They know there is more knowledge and wisdom outside of them than within them. They face and learn from the crowd.

The best possible answers come from this crowd of collective intelligence. Today, our collective intelligence lives not in personal networks but on the Interweb. To be successful in your decisions, you need the right intelligence around you. But how do you find it in the ocean of the Interweb?

To me, twitter is a network intelligence platform that can be mined for insights if used in the right way.

Most people live in their twitter timeline. But I find myself, digging through who others are following. Particularly, for those accounts that inspire my intellectual curiosity.

This graduated to always checking if they followed someone new. To me, this information is much more powerful than following someone’s tweets. Those they begin following are their newly chosen guides. And I want their guides.

The problem is that twitter doesn’t unearth this information well.

To help myself with this, I built a basic script in python. I’ve been using it for a few months now and find it really useful. Each week it’s like hosting a new dinner party where every guest delivers.

The script takes a list of twitter accounts you want to track. These are probably the top 30-50 people you follow and want to learn from. Each time you run the script, it pulls down any new followings and sends you a nicely formatted email to peruse in your own time. I run it on Sundays.

Here is a preview of last week’s email.

And here is the crux of the code.


If you think this might be useful, you can download the open source version here.

PS. Would love to know how others create curated paths of curiosity in the comments below?

PPS. Get new updates/analysis on investing in startups
I write a high-quality, monthly newsletter covering my perspectives on being an investor and what’s happening in venture. Sign up here. Promise you won’t get bored.

PPPS. A fair amount of caffeine was consumed in the making of this post

blog comments powered by Disqus