Romain Lerallut on scaling, scaling, scaling

Tell us about your background and how you ended up at Criteo.

After defending my Ph.D. in image processing in 2006, I joined a French start-up called A2iA working on handwriting recognition. When our robot overlords take over the world, they’ll be able to read your post-it notes on the fridge thanks to our work. For five years, I worked on the engineering aspects of building high-performing machine learning algorithms. This made me a good fit to join the budding Engine team at Criteo, back in 2011.

I joined as Lead of the Engine Infrastructure team, building the pipes necessary to launch the first modern ML algorithms to power our Engine. Then I evolved progressively to a management role, still working on our Engine. I don’t build software anymore; I build the teams that will build the software. I still see it as engineering. It’s Engineer Engineering!

What unique challenge or problem have you worked on here at Criteo?

Scaling, scaling, scaling. Scaling everywhere. Scaling everything. From the number of requests to process in real time, to the amount of data to train our models on, to organizing the work of now 500 R&D engineers. It’s all about growing as fast as we can while maintaining, and even increasing our quality standards.

What makes working in engineering at Criteo unique or different than your other experiences?

It’s very dynamic. Things are constantly in flux, and nothing is set in stone. We are in a permanent ‘try-and-learn’ mode, whether it’s for technologies, project management techniques, or HR processes. We strive to adapt to changing environments, we reshape our organizations to handle our growth, and we always try to make things work better.

What advice would you give to someone to be successful in your role?

As a manager, you need to show that you add value and help your teams deliver. You also need to make sure your teams are empowered and have the right level of information to make the right decisions (what technology to use, what trade-offs to make to deliver faster). You also need to hire people who will fit well in the various teams and make sure they are constantly challenged and motivated. And then you just need to learn to step out of their way and watch the magic happen.

What’s your favorite project you’ve worked on?

The projects I worked on directly are long completed and have been replaced by newer and better stuff. I still have a few lines of code in production that get called several millions of time per second. I don’t think about it very often, but when I do, I admit to feeling a tiny trickle of pride.

How has Criteo changed since you started 5 years ago?

It’s been a wild ride! I joined a post-startup company of fewer than 400 people, and it was pretty much the Wild West. Release process? What process? Just dump your binaries on a prod server and see if it works. Testing? Why? It’s for poor developers and insecure people who fear bugs. The “Hero Culture” of 2011 was very exciting but also exhausting. You never really knew what was going on. I discovered the joy of seeing my code go into production in a matter of days. I fondly remember posting about it on Facebook: “First line of code at Criteo deployed today. If the Internet breaks, it may be me.”

Fast-forward five years, and we have kept most of the good things and even added a few. I don’t miss much that we discarded. We’ve added some processes — of course, you can’t dump your binaries directly on the prod server anymore (thankfully!) — and absolutely all code is peer-reviewed. We even have decent code coverage. But we are still shipping like crazy. The problems have gotten tougher, and we need to think even harder to solve them.

Bonus point: I still have not broken the Internet.

For more updates on Romain, follow him on twitter: @RLerallut