Map Memorizers or Explorers: Hiring Well Needs Systems Thinking
I sent out this email nearly 15 years ago. This was after a rather disastrous 5 minute interview call where I quickly realized how out of depth I was and excused myself. That’s how bad I thought it was going. But the lessons from that 5 minute call have been invaluable and helped make our hiring process better and far more effective, allowing us to hire exceptional candidate s— candidates who were otherwise often overlooked — from all over India. Thank you, Wolfgang.
(Here’s where the tl;dr folks can skip ahead to the end and see what our hiring process looks like. But I think why our process is the way it is is probably more important. )
It was in the final year of my PhD and the first month of job hunting. I was prepared for the usual leetcode, oh-so-clever logic puzzle battery. This interview, for a role with a pioneering quant fund, was different. Wolfgang started off with a question, which I later realized was one anyone who spent any time writing real code — building real stuff in the real world — could answer. (Something about the c++ compiler version on the specific linux flavor I was using, if I remember correctly.) Now how was a whiteboard warrior who was adept in algorithmic wizardry — and little else — going to answer that? It quickly went south from there. That experience was so chastening, I don’t think I applied for any more jobs after. But that did allow me to rethink my then priorities and led to me writing A Billion Wicked Thoughts with Ogi (and that eventually led to Journey of the Mind). So thank you for that too, Wolfgang.
Job interviews suck for the same reasons education sucks.
Standardized assessment. It’s the well-intentioned path leading to the hellscape that is our education system. The world, and the work we do to make it better, is increasingly complex and interconnected. We hope to train students to understand complex systems. And we then evaluate their understanding of these systems by assessing their surface level understanding of what the system might do in specific circumstances: What does a high customer acquisition cost tell you about a particular business proposition? What impact will high inflation rates have on loan applications? What does the high virality of a new infection say about the public health policies that are needed?
The answer to all of these is: it depends. But in focusing on a standardized assessment that assumes one right answer, we inevitably, inexorably forget about the systems underneath. We also do not allow for unconventional perepectives.
We make the exact same mistake with hiring. To inject objectivity and speed into the hiring process we introduce these surface level assessments. We train map memorizers with the hope of forging explorers. And we then turn around and select members for our expedition team by testing how well they remember maps!
The answer is obvious when you look at the problem through this metaphorical lens — we need to select our explorers by having them go on an expedition. But expeditions are expensive. They need time and money and patience. How can employers and candidates afford this? The only way is to craft landscapes that come as close as possible to what they will encounter in the real world. It isn’t easy, but not impossible. We do this all the time in the other marketplace matching problem: marriage. We date and we get to know the person. Not just their idealized best-version self, but also their many different selves in their daily domestic reality. The analog in hiring is to “date” by working on small projects. And to do this well you must be able to articulate what real skills are required for in the role you are hiring for. This is almost always lost when passing through an HR filter.
One method that has worked exceedingly well for us over the years is to craft puzzles or challenges that involve building or deconstructing something small and meaningful. In addition to that, we also see how candidates document their approach, their stumbles, and their wins along the way. Now that I think about it, what we are doing in effect is looking for candidates who exhibit both competence and comprehension in understanding and creating systems. Also, because we have largely been bootstrapped, we had to be creative in finding great candidates who were otherwise overlooked because they didn’t check the conventional pedigreed boxes. We also do not look for perfection or finished articles. One, because those that look like the finished article are often the most in-demand candidates in the job market. Second, which is something we are reminded of very regularly, is there’s no such thing as a finished article in a world that’s changing constantly. And every one of us is changing too — if we let the change in. We look for evidence there is an ability to learn and grow.
What does this have to do with systems thinking?
When we reduce a complex multi-dimensional person and their evolving abilities to a single static parameter, we are always losing precious information. Sure, you will get your inevitable true positives no matter how flawed your assessment is, but you will also have a ton of false negatives. And that’s why we use a multi-dimensional assessment that looks at a whole host of parameters without attempting to squeeze them down to a single numerical rating. Being unable to quantify in a single number is a feature here! That’s how you find those rough diamonds.
What does this hiring process look like in practice?
The first step is letting candidates know your company exists, and there are openings. In addition to the usual job portals we often advertise on social media (Instagram, for instance, worked well for discovering great UI/UX and curriculum designers). Our most recent ad led to this page https://www.comini.in/hiring-challenge
We emphasize that we do not care about educational qualifications, so that acts as a great signal for those who have an unconventional background and are often filtered out at the gate by most conventional recruiting pipelines. Quite a few of our hires have been people who dropped out or had a pile of backlogs.
As a first filter we then have them fill out a form (here’s a sample one for software hires https://forms.gle/QYsAFrjtEskHHV6RA ). One of the strongest signals is how much thought is put into answering these questions. This is where the dating analogy is strongest. You want to be with someone who wants to be with you. Now, with LLMs it’s become easy to flood these with generated answers, but also easy to suss out the lazy ones. Some of the questions in there are designed to snag the cleverer ChatGPT users.
And the final step is a hiring challenge designed to mimic the work they’ll take up in their role. We usually do this in two parts. Part 1, a relatively easy one, is designed to take up a few hours. We interview those who solve this and share a more involved Part 2 after we have a video call with them. We are often pleasantly surprised by how differently candidates view and solve the problems. It serves as a reminder that there’s often no one right answer.
…no educational system is possible unless every question directly asked of a pupil at any examination is either framed or modified by the actual teacher of that pupil in that subject.
— Alfred North Whitehead in The Aims of Education
I think what Mr. Whitehead is suggesting here is that the education system is a dynamic, interactive, and local one that simply cannot be subject to standardization. I would argue that is also true for hiring. We simply cannot standardize hiring. We also have to remember that we are not simply plugging a cog into a machine to generate output, we are creating a whole new productive relationship, altering the dynamics of the company for the better.
That’s another profound lesson from systems thinking. The magic is not in the individual parts you stitch together, but how the parts connect, communicate and relate with each to create emergent magic. And that means your hiring process isn’t just about assessing for ability, but also for meaningful connection and belonging.
