There is a lot to talk about but here are the key things I learned that were surprising or made a big difference.
Before you go in, have a clear understanding of the Mission, Outcome and Competencies for the role. Candidates will ask you about it and it helps design the interview process. Mission is what you want them to do over the next 12-18 months. Outcome is how you’re going to measure success. What are the tangible results, for example shipping weekly instead of twice a year. Reduce attrition to X%, grow the team to Y etc. Competencies are the skills and experience that they bring that are required to achieve the outcomes.
This will inform your interview process and who should ask about which part. For example if you’re planning to restructure the team because your current leadership isn’t working, I’d want someone who has done that before and really understand deeply how they did it.
Spend a good amount of time understanding their process and find out if it’s compatible with how your team works or how you want it to work. Are they doing waterfall or agile? Are they just saying agile or do they really understand it? How structured are they and how much do they know about designing the right process in JIRA or similar tools?
We typically did a first call (or in person if convenient) with the person they’d be reporting to. This is mostly around selling them on your company and getting a first impression of the candidate. Then a round of interviews with key executives that they’d be interacting with the most: CEO, CTO, CPO or VP Product, VP of People.
With key leaders we also like to “interview” them in a social setting (dinner), typically with all founders. This is to find out if you can work with them as a person. Super important.
We tried having candidates present a 180 day plan to a group of execs and key engineering leaders but since they have little context on your company it ends up being pretty generic and not a meaningful datapoint.
Engineering team members (managers and other leaders like early employees or those with a large following) should get a chance to meet the candidate, but save that for a later stage once you have decided that you want to hire them. Position it as: we really like this candidate and think she is the right fit, we want to make sure you like her too. What I found is that when you bring in too many people too early your signal/noise ratio is all over the place. People pick up on small things they don’t like and that turns them into a no. Nobody is perfect so the chance that someone is going to find something is high, no matter how good the candidate is.