Much has been written about the lean startup movement, which advocates data-driven decision-making within a principled, customer-driven framework to both picking a startup idea and growing the fledgling company through its initial lifecycle.
There’s no question that overall the philosophy is a strong net positive to the ecosystem, but data can also lead to some bone-headed conclusions. If you ask the wrong question, data doesn’t say “sorry try again” - it often gives you back a perfectly reasonable but utterly incorrect answer. VC is just as prone to this as any startup.
It’s surprisingly rare to see real vision borne out of deep domain knowledge paired with good old common sense (in both VC and startups!) - but those are the guys you want to hang out with. Not enough attention is paid to asking the right questions, gathering the right data, even in choosing to play in the right ballpark.
I don’t have all the answers. What I am doing (with @AxiomZenTeam) is running experiments. Lots of them. And bit by bit, we learn and get better. What we try to do is continually challenge ourselves to ask our questions in different ways, to upend assumptions and get back to basics.
I feel like my biology background helps me have a more critical eye on experiment design, which is why I chuckled when reading the recent (absolutely fascinating) article from WSJ on animal intelligence. Namely, that we may have been underestimating our furry friends’ cognitive abilities for centuries precisely because of terrible experiment design by scientists that were likely too smart for their own good.
It’s a great article, so I’ll just post an excerpt and you should read the whole thing:
Experiments with animals have long been handicapped by our anthropocentric attitude: We often test them in ways that work fine with humans but not so well with other species. Scientists are now finally meeting animals on their own terms instead of treating them like furry (or feathery) humans, and this shift is fundamentally reshaping our understanding.
Elephants are a perfect example. For years, scientists believed them incapable of using tools. At most, an elephant might pick up a stick to scratch its itchy behind. In earlier studies, the pachyderms were offered a long stick while food was placed outside their reach to see if they would use the stick to retrieve it. This setup worked well with primates, but elephants left the stick alone. From this, researchers concluded that the elephants didn’t understand the problem. It occurred to no one that perhaps we, the investigators, didn’t understand the elephants.
Think about the test from the animal’s perspective. Unlike the primate hand, the elephant’s grasping organ is also its nose. Elephants use their trunks not only to reach food but also to sniff and touch it. With their unparalleled sense of smell, the animals know exactly what they are going for. Vision is secondary.
But as soon as an elephant picks up a stick, its nasal passages are blocked. Even when the stick is close to the food, it impedes feeling and smelling. It is like sending a blindfolded child on an Easter egg hunt.
More experiment design #fails:
Another failed experiment with elephants involved the mirror test—a classic evaluation of whether an animal recognizes its own reflection. In the early going, scientists placed a mirror on the ground outside the elephant’s cage, but the mirror was (unsurprisingly) much smaller than the largest of land animals. All that the elephant could possibly see was four legs behind two layers of bars (since the mirror doubled them). When the animal received a mark on its body visible only with the assistance of the mirror, it failed to notice or touch the mark. The verdict was that the species lacked self-awareness.
(hint: elephants can in fact recognize themselves)
A similar experimental problem was behind the mistaken belief, prevalent until two decades ago, that our species has a unique system of facial recognition, since we are so much better at identifying faces than any other primate. Other primates had been tested, but they had been tested on human faces—based on the assumption that ours are the easiest to tell apart.
that one’s a real face plant.
Read the whole WSJ article here.
I can’t help but feel that none of these scientists were deep domain experts, rather that they looked at animals as foreign objects to be studied. My academic advisor was Robert Sapolsky, among the world’s foremost experts in neurobiology (and really a fantastic guy). When he talked about baboon behavioral psychology, he knew his subject matter deeply. He had lived in Africa, immersed himself in the baboons’ day to day lives, their social structure and -dare i say it- their culture. He would not have given them human faces to tell apart.
What does this have to do with startups? I’m personally disappointed in the number of smart people making bone headed mistakes. What mistakes? Groupon. Trainwreck most of us saw coming for years before the IPO. Selling shitty goods from shitty merchants at 1/4 their traditional value to shitty customers. *But the data* they said, and the growth! Did anyone bother to actually keep using the product?
not just Groupon, lots of companies. don’t be one of them. I’ll leave you with one of my favorite Sapolsky quotes:
I am a reasonably emotional person, and I see no reason why that’s incompatible with being a scientist. Even if we learn about how everything works, that doesn’t mean anything at all. You can reduce how an impala leaps to a bunch of biomechanical equations. You can turn Bach into contrapuntal equations, and that doesn’t reduce in the slightest our capacity to be moved by a gazelle leaping or Bach thundering. There is no reason to be less moved by nature around us simply because it’s revealed to have more layers of complexity than we first observed.
The more important reason why people shouldn’t be afraid is, we’re never going to inadvertently go and explain everything. We may learn everything about something, and we may learn something about everything, but we’re never going to learn everything about everything. When you study science, and especially these realms of the biology of what makes us human, what’s clear is that every time you find out something, that brings up ten new questions, and half of those are better questions than you started with.