The Criteo Engineering Hiring Process

We talked before about great engineers and their crap CVs and how it didn’t matter in the end because we love them anyway. But you’ll agree that having a crap CV is not proof enough of an amazing engineering talent, so here’s a little deep dive into how we try to assess that better.

And so, it begins : phone call
Whether you’re an applicant headhunted on LinkedIn, or referred by a colleague, it all starts here: a friendly chat with one of our recruiters. This first step is to try to understand who you are, as well as tell you a bit about us, what our story is, our challenges and see with you if you’d be interested in joining our great adventure.

Criteo Hiring Process: Phone call with a recruiter

First date : technical Skype interviews
If so, the fun begins. Not that talking to our beloved recruiters isn’t fun but we’re engineers, we like to talk tech (though to be fair, some of them write better Python than you do). Next step will be one or two Skype interviews with our engineers. During this one-hour discussion, we’ll ask you technical questions, drill down in your areas of expertise (we try to match our interviewers to your own skill set so you can talk shop) and we’ll also make you write code on a shared scrapbook. You’d be surprised at the number of people applying for software jobs who can’t code their way out of a paper bag, #fizzbuzz.

Criteo Hiring Process : Skype interview:

Note: It doesn’t actually matter if you know nothing about webscale apps. Instead of asking you about things you don’t know, we’ll be checking that this « expert C++ » you wrote on your CV is actually true. We don’t do much C++ at Criteo (we do mostly Java and C#, a bit of Python) but we assume that if you are good enough to become an expert in some technology, you can learn quickly another one. The metaskills that someone develops to reach an expert status anywhere will enable them to quickly pick up expertise in another domain. So we’ll find an actual C++ expert in our pool of interviewers to grill you on partial template specialization, memory layout in inheritance diamonds, or lambda capture modes.
Of course, this is also a great opportunity to talk to our engineers, and understand what is fun about showing ads on the Internet. It doesn’t seem much at first sight, but think about how you would build the system to do it several billions times a day, in milliseconds, while leveraging petabytes of data…

The Panel : we meet at last
If the calls go well and we both think we may be a good match, then it’s time to have you come over in our office and have a series of face-to-face interviews. Details will be different if you live down the street or if we fly you over from the other side of the world, but the basics remain. Two coding interviews, a design interview and a motivation interview. This is the standard process for software engineers. Other tracks (data scientists, engineering program managers, devops, etc) differ slightly in implementation but the concepts remain globally the same.

Coding interview, take 1
You can argue this however you want, but the job of a software engineer is to write code. Code that solves problems. So, we’ll give you a problem and we’ll let you write code to solve it. Usually, it’s a fairly simple problem (easier to explain) and there is no trap. Just get a working solution on the whiteboard and we’ll start playing with it. What do I mean by that ? Well, it’s simple, we’ll change the hypotheses and see what breaks, we’ll ask you to make it webscale-ready, or we’ll simply keep adding features and see how you cope with it.

Criteo Hiring Process: Coding interview

I personally love to take candidates on a journey, starting with a few lines on the whiteboard and getting them to a point where I can end on « so you see, if you replace those integers by GUIDs and add some payload, well, we’ve re-implemented the core principle of MapReduce ». Or maybe « and this is a much simplified version of something we had to develop last year for a product ». Our exercises can seem deceptively simple but one thing you learn quickly is that there is no such thing as ‘simple’ in a web-scale world. Scale it up until it breaks and then scale it up again and again ! And make sure there are no bugs because bugs can take the Internet down.

Coding interview, take 2

I just said the job of a software dev is to write code. I lied. Well, not really, but the truth is that a developer spends a lot more time reading code than writing it. And more time designing it. And even more time putting up with bad APIs until, finally, a few carefully crafted lines of code appear. This second coding interview focuses more on those other aspects of the job. We’ll ask you to design a class to do something, or we’ll ask you to improve an existing API, or similar tasks that require a critical mind, some experience and the ability to both understand the problem and to convey your ideas concerning the solution.
Once again, the idea is to use the problem as a starting point for a discussion. Don’t just tell us that you will use this design, but also tell us why you decided against the other one. And define explicitly the assumptions you make (so we can change them and see you adjust to changing requirements).
From an actual interview:

• Interviewer: Why did you use a stack here ?
• Candidate: To solve the problem.

Silly interviewer, asking a vague question when he actually meant : « among all the data structures that you know, why did you select a stack ? Why not a queue or a heap, a doubly-linked list or a red-black tree ? »
Design interview
Criteo’s business is to develop highly scalable web applications. From our programmatic buying system to our reporting and configuration UIs, all our teams are faced with issues of scalability at some point. So we’ll dig a bit in your ability to fit in this demanding environment.

If you’re an experienced guru of scalability, we’ll expect to fly through the basics and quickly reach the meaty parts. Design interviews are run by our most experienced engineers and you can be sure they’ll give you a run for your money. It can be like fencing, be prepared to defend all your architectural choices !

Criteo Hiring Process: Coding interview 2

If, on the other hand, your only experience in this web thing is the website you ‘designed’ for your school’s student council years ago, that’s fine too. We’ll assess something else : your ability to ramp up. You will have to have some basic general knowledge of client/server architectures, ideas of sizing and how the web works but we won’t expect you to design a TCP/IP stack from scratch.
Our design problems are open-ended, there are no right answers (but there are many wrong ones !) and we’re more interested in your thought processes than in whether you designed the same thing we would have. Once again, we try to put you in a situation you’d be faced with if you joined Criteo to see how you react . There’s a reason most of our walls are actually whiteboards and we have comfortable couches near the coffee machines : lots and lots of design discussions sprouting everywhere all the time.

During all those interviews, you’ll again have the opportunity to meet people from many R&D teams, ask them what they like about their work, their challenges, how the teams work, etc. Although we try to keep a tight schedule and not wear you down too much, we also love to talk about our work and what we like about it. So don’t hesitate to ask us questions. All these people may end up being your future colleagues so get to know them as soon as possible, they are good people.

Final interview

Assuming you’ve lived through the grinder of the tech interviews, now is the time to meet one of the senior R&D managers or directors. Usually the head of the department where we think you’d fit best.
This last interview is not as technical as the previous ones (although we may still want to drill down on something we find interesting) as we are more concerned in a final assessment of the mutual fit. We’ll ask you to tell us a few stories about yourself, how you like to work, on which topics, good experiences, bad ones, etc. In exchange, we’ll do the same about Criteo and its R&D, its challenges and how we are going to solve them.

Criteo Hiring Process: Final interview

By this time, you should have a pretty clear picture of the work we do at Criteo so it’s time for you to bring all your questions to the table. Don’t worry, we’ll do the same with you.

Wrapping it up
When all is said and done, when you walk out of our building, we’ll both have a better idea of who we are and whether we’d like to work together. Sometimes, we may find out your skills don’t match our current needs, or the other way around. In either case we hope that you’ll have spent a nice time with us, that you were impressed with our people and our challenges.
But if we find that we like one another and if you accept our offer, then it will be time to finalize the details, begin the relocation process and get you on that final stretch that will get you one morning to ask the receptionist in the lobby to ping your Criteo buddy for your first day. Welcome aboard!!

PS: the process described here is current as of April 2015 and changes regularly as we incorporate feedback from interviewers, candidates and raw data analysis (yes, we compile stats and run ab-tests on our hiring process. We’re an engineering company, we trust only what we measure).