One very cold winter, my phone rang. It was the coldest of calls. A recruiter wanted to know whether I’d be interested in leaving my current gig to lead a team that he was trying to assemble. The gig I was working had been interesting but it was reaching that point where the tough work was done and the team was moving into the steady toil of maintaining a product. The money was good, the work was nebulous and half the team was remote.

I figured it would at least be interesting.

There was only one problem… I’d no idea what I was doing. I’d never led a team before, there was no one else on it and the platform was a 20 year-old multi-process UNIX application written in C (and C++ that looked like C). I’d no idea how to go about hiring a team, let alone lead it. The platform was massive and the remote team had been working on it since inception.

Instead of getting overwhelmed, I just did what comes naturally: hack code.

Hacking code turned out to be the best thing I could’ve done. I learned a lot about the platform. It was wonderfully designed but hadn’t aged well. The platform was - basically - a messaging platform. It had started in the era when the ITU-T standards had been in fashion (X.400, X.500) and had been steadily maintained throughout the era of IETF messaging standards adoption.

My hacking eventually got me an invite to travel abroad and visit the remaining members of the original team. What I learned from them gave me the first piece of the puzzle: The platform had been designed for specific machine types that were either defunct or far to expensive for the platform’s customers. Management wanted to run the platform on commodity hardware but much of the implementation couldn’t meet performance targets.

Now I knew what the team should be doing. We would be the Platform Performance and Optimization team. It would be our job to figure out how to get the platform to run more efficiently. Having come to this realization, I could now do the first requirement of my job: Build a roadmap for the (nonexistent) team.

The second requirement of the job continued to vex: I still didn’t know how I’d form the team…

I was out drinking one evening with the Lead of the corresponding front-end crew. After a couple, I started to to bemoan my circumstances. He’d already assembled his team, so I figured that if I complained a bit, he’d volunteer some things I could read that would help me learn how to rationally and logically put a team together. The only thing I got out of him was that he just went with his gut. In short: he interviewed and guessed!

This just made me all-the-more ranty. I continued to grumble about the circumstances, but somewhere along the line, I said something like: “Look dude. We don’t need coders for this thing. We need archaeologists. Archaeologists of the Impossible.”

And… There it was: The hook my brain needed.

We’re all wired for fiction, but I’m particularly susceptible. I can’t reason about a thing until I know the story of the thing. All along, I’d been waiting for the story of this team to strike me, like a meteor from the sky.

That phrase I’d used was drawn directly from a comics series called Planetary that I’d been in the midst of rereading. It made me realize that the story of the team was something like the Fantastic Four or Challengers of the Unknown or even Planetary. If I could frame the needs of the organization and the project as the story of a group of costumed adventurers, I could understand the people I needed to hire.

One coder that I knew we needed was a new graduate. It had to be someone that had barely worked in industry. If you recall, the code was very ancient and we were still working with the original authors. This often lead to many circular discussions that eventually bottomed out as “this is how we’ve always done it”. There needed to be someone on the team that innocently didn’t care about these problems. They needed to be brash and outspoken about their ideas in order to challenge the thinking of the old guard.

This coder also needed to be obsessed with performance. They needed to be fresh enough from school that they could still think clearly about theoretical ideas around performance without being jaded by years in industry spent ignoring these ideals.

After talking to Arthur for about 15 minutes, I knew I’d found the person I needed. He was very, very confident in his craft, most of it proven during the interview. I’d given him a laptop to hack on a little puzzle. His solution was a terrible bit of hatchet work, his C and C++ was raw and unpolished, but he was proud of it. That’s all I needed to know I wanted to hire him.

Decision made, I carefully spoon-fed him a number of C++ questions that I knew our Director would ask him. He had to be able to answer those questions to pass the rather silly gauntlet of our interviewing process. These questions had no value for this particular team and little value in general, but they were asked anyway. The Director insisted on a specific ritual to ensure we hired only the “best”, carefully avoiding anyone with potential.

While Arthur talked things over with the Director, I went down the hall to arrange his hiring paperwork…

You’ll recall that the platform was an ancient multi-process UNIX application and that the team had been with the project since inception. I needed to hire someone that had lived and breathed in that world. My career had started near the tail end of the period, but I’d not been immersed in the milieu. I’d been running Linux for years, but I’d not been immersed in the oral tradition of the UNIX hackers. More importantly, this coder needed to be able hold their own with the original, remote team.

Fortunately, I knew someone who had been. At a previous gig, he’d told me tons of stories about applications he’d written when I was still in middle school. Clearly, I had to convince him to join up. One night in the pub was all it took.

The Rock turned out to be exactly what was needed. He could explain most of the esoteric ioctl() calls the platform made. Those he didn’t know, he quickly figured out. The remote team came to really respect his work.

The final role I needed to fill turned out to be the hardest. If I was going to effectively lead the team, I needed to hire someone that took over the work I had been doing. This work was all over the place. Sometimes I was fixing bugs that came from customers, sometimes reworking an aspect of the testing framework or the build automation. It was clear that I needed a generalist. A kind of detective that just dug into each problem that came along and pushed a solid fix.

Mary came along a little by accident. She’d just moved to the city and was kind of exploring what options existed. Her partner was working a really good job, so she had all the time in the world to make a decision.

Mary’s interview will stick with me forever. She waltzed into the meeting room and patiently entertained a couple of basic questions before interrupting me. She handed me a little binder with some code printed out. It was something that I’d written and published a few years back. She’d found a more elegant way to solve the problem. She took over the interview and clearly explained the improvements her solution made.

I have to admit, it was a courageous approach to the interview. Truth be told, I’d already decided to hire her based on that move. I later found out that much of the history of her resume had been sysadmin work. She was a self-taught coder who mostly hacked in her spare time.

This team, arrayed in this narrative, really worked well together. The entire team was only together for about a year and a half, but we did a number of really great projects together.

Based on ideas from Arthur and Mary, we implemented a traffic simulator that exercised the platform. This allowed us to model typical customer use patterns. The information that we gathered from those simulations gave us enough evidence to begin a rewrite of a particular subset of our services. The Rock hacked together a very different process-forking model that allowed us to recycle idle processes. Arthur and I worked back from the data storage system, reimplementing the middle bits as an event-based, eventually consistent data service.

For a time, it was a joyous and wonderful place to hack, but nothing lasts forever. Mary left after a few months, having received a better offer with more money. The organization itself gradually came apart at the seams, so the team disbanded. We each went our own ways, but kept in contact.

The experience of forming the team using fiction as a guide stuck with me. In the years since, it’s become an essential tool in my reasoning about teams and platforms. When you’re going to dedicate significant time to a thing, if there isn’t a good story to tell, there’s perhaps nothing there at all.

---

A note on style

If you’re at all familiar with comic books, the style of writing in the framing story and the format of the team should be familiar. Originally, the Fantastic Four were my inspiration. Arthur is the Human Torch and The Rock was based on Thing; both from the classic Kirby and Lee era. Part of me really wanted to encourage the tense relationship between those characters on the actual team. Though it’s not obvious, Mary was equal parts Invisible Woman and She-Hulk.

And me? I’m Reed Richards.