Criteo at RECSYS 2016

Criteo was in full force at the 2016 RECSYS edition in Boston. With 10 people from our Paris office and 1 from Palo Alto, it was the place to be for the Reco team. The researchers were also in the spotlight with 2 Criteo papers published, (Thanks Elena, Flavian & Yu-Chin!)

We also had members of our recruitment team present to raise awareness for the Criteo employer brand, meet curious minds with tons of questions about working at Criteo.

For the opening keynote Claudia Perlich (Dstillery), a widely acclaimed expert on big data & machine learning, presented her latest, state of the art, recommender system at Dstillery. It felt good to know that we both use similar systems (logit with hashing trick + a modified collaborative filtering).

The current trending topic of the conference was clearly embeddings, everyone is trying embeddings and we expect them to be a big thing in the coming years. Hopefully thanks to the work of Flavian & Elena, Criteo is still ahead of the curve, while people were discussing the possibilities of embeddings at the conference, we were already ABTesting them.

Criteo talks

This year we presented two papers, one on: Factorization machines and one on product embeddings. Both papers were well-received and we had good follow-up discussions with the RecSys attendees.

In our first talk, Yuchin Juan presented  Field-aware Factorization Machines for CTR Prediction, as part of the Algorithm Session. Models based on degree-2 polynomial mappings (Poly2) and factorization machines (FMs) are widely used for CTR prediction. Recently, a variant of FMs, field-aware factorization machines (FFMs), outperforms existing models in some world-wide CTR-prediction competitions. In this paper we compare linear model, Poly2, FMs, and FFMs, and discuss why FFMs can be better than the other models. Further, we use FFMs as our major model to win two worldwide CTR competitions hosted by Criteo and Avazu on Kaggle. Based on these exciting results, we released a package  LIBFFM for people who are interested in using FFMs.

In our second talk, Flavian Vasile presented  Meta-Prod2Vec: Product Embeddings Using Side-Information for Recommendation, as part of the  Deep Learning session. Meta-Prod2Vec is a novel method to compute item similarities for recommendation that leverages existing item metadata. This types of scenarios are frequently encountered in many applications, such as content recommendation, ad targeting and web search. Our proposed method leverages past user interactions with items and their attributes in order to compute low-dimensional embeddings of items. More specifically, the item metadata is injected into the model as side information to regularize the item embeddings. In the paper we show that the new item representations lead to better performance not only on the cold-start traffic but also globally when part of an ensemble method with a classical neighbourhood-based collaborative filtering method. If you are interested to know more, we have published a dedicated  blog post that covers in depth the intuitions behind the paper!

Deep Learning Workshop

Deep learning is trendy. It has shown amazing results over the past few years in several fields, like computer vision or speech recognition, improving the state of the art to a level close to human capabilities. Many machine-learning conferences like NIPS or CVPR are now full of papers on Deep learning and its applications. Surprisingly, it hadn’t reach yet the field of recommenders systems, and thus, it was the first time that a Deep Learning workshop was organized at RecSys.

The main usage of deep learning in recommenders system is its ability to create low-dimensional distributed representations, also known as embeddings, of items, user, or other available data. They can be learned either by leveraging usage data (timeline of items, user ratings) or content data: audio files at Spotify, metadata like actors for movies recommendation, images, …. The last case is particularly useful to tackle the recurrent cold-start problem, as it provides a way to use items/products with little or no user interactions.

The talks at this workshop covered a wide range of applications, and ways to train and use those embeddings. The keynote speaker, Sander Dieleman, explained how Spotifiy combined audio tracks with existing matrix factorization/collaborative filtering, by predicting the latent factors of a song using deep convolutional networks. This combination of deep learning and more standard algorithms is also used at Google Play, who jointly learns a deep model and a linear regression with manually-engineered features.

Embeddings can also be used as input to learn a user-specific ranking (Exploring Deep Space: Learning Personalized Ranking in a Semantic Space)

Finally, a more advanced work from Georgia Tech (Recurrent Coevolutionary Feature Embedding Processes for Recommendation) tried to model the dynamic relation between user and items in a variable embeddings representation, by using Recurrent Neural Networks.

Emotions and Personality in Personalized System (EMPIRE)

I recommend you the see the slides and informations about this workshop by following this link:  https://empire2016recsys.wordpress.com/.

So, the RecSys research community has done a tremendous job in the last decade on exploiting various data sources to improve recommendations through sophisticated algorithms.

EMPIRE workshop complements these core RecSys activities by pushing the agenda of taking into account user-centric aspects, such as emotions and personality, into the RecSys framework.

This is extremely interesting for Criteo, especially that we are trying to focus more on the user rather than see the user as a bunch of views/sales products. Knowing his mood, his context, his personality can improve our own recommendation and user satisfaction.

One of the talks was about “A Comparative Analysis of Personality-Based Music Recommender Systems”, which shows that it is possible to get a better recommendation by using user’s context, e.g.: his personality, current location, activity, or any other contextual condition that might influence the user’s perception or evaluation of music. It used the model below to know user personality.

Another talk was about “Adapting to Emotional Reactions In Context-aware Personalization”, which uses user preferences to improve recommendations. User preferences are two kinds:

Explicit: Ratings/ binary feedbacks. Implicit: Review/behaviors

Criteo recommendation can use some of these techniques to improve our internal recommendation.

IntRS

The IntRS workshop (Interfaces and Human Decision Making for Recommender Systems) was nice discovery.

It focused on the way the recommendation are displayed to the user and how it impacts the user behavior.

The vast majority of the presented papers were running experiments in a controlled environment, asking people to interact with a recommender system and gather explicit or implicit feedback.

At Criteo, we tend to focus more on the ML side and ABTesting but there might interesting lessons we could learn from these studies.

For instance, we learned how the nature of the recommended items, the price of recommended items influence the user willingness to pay for a product( Complements and Substitutes in Product Recommendations: The Differential Effects on Consumers’ Willingness-to-pay, Mingyue Zhang, Jesse Bockstedt).

We also discovered that recommendation – even if random – impact the user choice and its confidence in this choice. This phenomenon is called the anchoring effect and was studied in  Investigating Mere-Presence Effects of Recommendations on the Consumer Choice Process, Sören Köcher, Dietmar Jannach, Michael Jugovac, Hartmut H. Holzmüller.

Criteo Cocktail

The Criteo cocktail event was one of the few private industry events organized in conjunction with the main Recsys events. Event was held in the Hyatt Regency Hotel overlooking the Boston. The networking event run from 6pm until 10 pm amidst some really good food & of course drinks!

This event was rather laid back and gave the privileged attendees 60+ attendees the chance to network and discuss current/past topics in the Recommender Systems ecosystem.

We look forward to being at Como next year with all the Recsys enthusiasts.

Post by:

Benoit D, Karim B, Flavian V, Elena S, Simon D, Lowik C, Romain L, Olivier K, Caroline C & Ibrahim