How I Learned to Stop Worrying and Love C# 

A few weeks ago, we attended NDC Oslo. Hosted in the center of beautiful downtown Oslo, NDC takes over the Spektrum arena every year for 3 days with talks about everything developers love (except Java).  The conference serves about 1900 attendees who almost entirely originate from Norway (70%) and continental Europe (25%), and features a diverse set of speakers and partners from all over the world.

NDC Oslo

The most well-known speaker this year was probably Troy Hunt, the Australian web security expert famous for creating haveibeenpwned.com, a huge repository of usernames and emails that have been publicly leaked by hackers.  Hunt opened NDC on Wednesday with a humorous keynote about security and the intersection of old and new technologies.  You can check out the video in the NDC Oslo 2016 channel on Vimeo.

The talks covers a huge range of topics, a lot of them gravitating around Microsoft technologies in the open. This is interesting to Criteo, as .NET (most specifically C#) is a technology we do work a lot with. Some of us do like a lot the direction where .NET is headed, so you can guess which talks we were most interested in and will cover here.

The scale of .Net @ Criteo

Most of our real-time, online code is running on .NET. That’s north of 1.3M lines of C# in our codebase.

In production worldwide, we do have more than 6000 servers running various .NET services. Part of that is facing the internet and serving at peak around 2M HTTP requests per second.
Right now, all of those are Windows servers; the prospect of being able to run .NET Core on Linux, ideally on Mesos looks very appealing to us, for obvious reasons.

A little more esoteric (and fun), we also do have .NET based Map/Reduce jobs, running on our sizable Hadoop Cluster… on Mono. We’d also love to migrate that to Core, and not rely on Mono anymore.

We’re experimenting with.NET Core since RC1, around November 2015. Migrating several of our open sourced & internal libraries and getting some simple services running on Linux. Most of the time was spent keeping up to date with the ever changing libraries and tooling.
Hence, after attending the Build 2016, we were really looking forward to the RTM and any announcement that would be made at the NDC.

Noticeable Talks

Here’s a few highlights from what we found at the conference:

ASP.NET Core Kestrel: Adventures in building a fast web server – Damian Edwards, David Fowler

https://player.vimeo.com/video/172009499

An animated, deep technical dive into the new Kestrel web server that Microsoft is developing as part of its open source .Net initiative. Simultaneously entertaining and hard to follow, this talk was a real gem, and I’m glad I got the chance to see it.  The presentation was given by Damien Edwards and David Fowler, both from Microsoft, and they did not disappoint.

The talk starts with a tweet from @kellabyte, an enthusiast who built her own webserver, tested it out on a raspberry pi, and compared it to Kestrel running on a dual Xeon, 8 core server.  The results were surprising:  Kestrel could only manage 750 requests per second on big iron, whereas the little webserver on the raspberry pi was completing 72,000 requests per second!

The Kestrel team took this challenge to heart, and set out to break some records.  The story of what they did next is a twisted tale of cheating the garbage collector, custom socket awaiters, vector math extensions, and why you’d want to convert the first 8 characters of an HTTP request into a long int!  Crazy stuff.

C# Today and Tomorrow – Mads Torgersen

The talk opened on a quick presentation on today’s big feature, the Roslyn compiler.

He then announced several future enhancements to the language:

The records. This is the same as Scala’s case classes: a very simple immutable POCO. It will help to strongly type what would have been a Tuple previously. It’s really a little but very pragmatic new syntactic sugar which will help C# to be an even better language.

Multiple method return values. Definitely the one feature missing in C#. Let’s be honest, out parameters are crap, they don’t enforce consistency between the returned value and the parameter’s affectation, so they are really bug prone. Multiple return values for a single method will help get rid of that, it really make the code more readable. Here is an example of how it works:

public (int value, bool found) TryLookupThing() {
return (1, true);
}
public void SomeOtherMethod() {
var result = TryLookupThing();
if (result.found)
Console.WriteLine(result.value);
}

Way more compact than creating a local structure in the class (what I always do when I want multiple returns), and really cleaner than an out parameter. Also I really look forward for the improved compatibility with F# functions. I already love it!

The last big feature is pattern matching. It’s implemented within the switch construct; right now it seems to be great for filtering on records, and also it support guards (using the ‘when’ keyword). After the talk, Mads confirmed it was a first iteration, and they might support more cases later on.

Scale up and out with Akka.NET – Roger Johansson

Check out a video for the same talk at another conference:

We were lucky to see a talk by Roger Johansson, who is the creator of Akka.NET.

It’s a port of Akka, a Framework that allows you to use the Actor model on the JVM, but for the .NET world.

The goal of the Actor model is to simplify parallel coding by using Actors that are single threaded, to which you can asynchronously pass messages and get a response. This is particularly important since our CPUs are starting to get more cores instead of just being faster, and this model allows to use them all easily.

Another nice thing is that the Actor model doesn’t really care where an Actor is, which potentially allows to scale out and distribute the computation on multiple servers, and this is integrated in the Akka.NET library.

Some products based on the Actor model are very successful, such as Whatsapp and RabbitMq in Erlang, as well as Walmart and LinkedIn using Akka on the JVM.

Roger then showed us several demos, such as a chat server and clients, remote deployment of an Actor on another system, routers used to distribute messages to one or several Actors, and finally a Mandelbrot fractal computation on several systems.

Lessons from a quarter of a billion breached records – Troy Hunt

For those who haven’t seen a talk from Troy Hunt, then you must know that he is a very good speaker. He was interesting and funny at the keynote, and this talk was also funny and… frightening, with him showing how much we’re impacted by the recent leaks.

He told about several security breaches and the consequences they can have. Most companies security’s emphasis is on banking and credit card information; for some leaks (namely Ashley Madison), the company’s announcement about safety of banking information didn’t prevent some users to commit suicide merely because their usage of the website was disclosed publicly.

After a few basic web security points (not disclosing whether an email exists when asked for a password reset, among other things), he went and talked about those email leaks. It’s pretty simple to buy for a few bitcoins all those dumps; he bought one, and did a real live demo of importing it in his website https://haveibeenpwned.com. If you’re reading that blog, your email probably is to be found there, most likely part of the LinkedIn leak.

The Shape of Things to Come

Code-wise, .NET Core RTM was announced a few days ago. The Runtime is really stable, and the Kestrel web server does not disappoint.

We had a glimpse of the .NET platform future, and it looks really promising; In the next few weeks, we’ll definitely double down on codebase compatibility.
Our SRE engineers and DevOps are looking forward to managing Mesos instances instead of dedicated windows servers.
C# Developers are all looking forward to C# 7.

On the security side, we do not store any private information about you, browsing a website; that’s why we’re not a target for user dumps usually. We do have a lot of interesting security topics to tackle, tought.

Are you Interested in the .NET Core adventure, or in enhancing security? If taking this journey with us sounds fun, please feel free to apply !

Post written by:

Nicolas Truchi [ Engineering Dev Lead, R&D Platform], Sebastien Foutrier [ Staff Software Engineer, R&D Scalability], Paul Davis [ Snr Mobile Developer, R&D Platform Mobile] & Antonin Lenfant, R&D Platform Mobile]