How to build and deploy a recommendation engine in 3 months?
What makes you read a book? Or watch a movie? With millions of books, movies, and songs available, how do you know which ones you will like? One tool that can help is a recommendation engine that analyzes your likes, dislikes, and the behavior of people with similar tastes to recommend something you enjoy.
Today, recommender systems are reshaping the world of e-commerce. They drive sales, increase user engagement, ease users’ lives, and improve their experiences. A well-thought-out engine can give users recommendations as if they come from a friend.
Blinkist, a book summarizing subscription-based service connecting 23 million readers (about the population of New York), aims to provide a factual analysis of books and their essential points to curious people who do not have the time to read. Blinkist’s quarterly goal was to increase the engagement of its power customers. Once the quarterly objective was set, the data teams started brainstorming how they could support this goal.
Blinkist offers users that do not have time, a chance to inform themselves by reading or listening to the book summaries, called Blinks. But how much effort does a user have to put into choosing a Blink? If users get recommended relevant Blinks that interest them, they will naturally engage in the app more. Blinkist’s data teams saw this opportunity and investigated potential improvements for the recommendations.
The idea that emerged was a recommendation engine that considers users’ locations and suggests books based on trending topics around them. The project was presented to Netlight, and two Netlighters, Manuel Schlegel and Nazli Uzuner(author), joined Blinkist’s data teams to support transforming this idea into a product.
The plan after these discussions was to create an MVP (Minimum Viable Product) during the first two months and use the last month to decide the next steps and review the data team’s current practices and system architecture.
The team’s first focus was investigating the data sources since all future decisions depended on them. We reviewed different trend sources such as Twitter, Google, YouTube, TikTok, and News API, an API that returns trending news articles by country. For the MVP, we decided to go with News API, but we also saw the potential of fetching data from different trend sources in the future. Hence, we introduced a modularized and generic implementation for the data pipeline, which would facilitate the integration of new trend sources and maintenance of the product. The independence of the pipeline parts eases the updates since the changes in one would not affect the others.
Each part of the data pipeline was written in separate Jupyter Notebooks for efficiency and run in an AWS SageMaker instance. Moreover, the team decided to store all intermediary pipeline results as data frames. This way, adding new components becomes much more accessible, and the data processing becomes more transparent.
The parts of the pipeline are given above, and the logic is as follows:
1. Fetch data from the news API
2. Preprocess data
3. Generate word embeddings for articles title, content and book summaries and match them using the Nearest Neighbor algorithm
4. Generate context for the recommendation using the GPT-3 transformer model
5. Human in the loop reviews the recommendation by checking the automatically generated text and the match quality
The output of the data pipeline contains the country where the article originated and a GPT3-generated message that aims to give users more context about why a particular book has been recommended to them. An example message is as follows: “With Wimbledon in full swing, we look at what tennis can teach us about the mental side of peak performance. The Blink, “The Inner Game of Tennis”, shows us how we can excel on the court and in life.”.
Blinkist uses Braze to send emails and push notifications to their customers.
The pipeline runs twice a week to generate data frames used by Braze’s recommender. The recommendation Lambda receives the user’s location and email from Braze and previously read books from DynamoDB. Afterward, it sends out personalized recommendations while considering not only trending topics close to the user but also whether the user has already read the book.
The team started user testing the MVP. While these tests were running, we found some improvement points based on human approval rates. The human-in-the-loop was a more demanding task than predicted, so we brainstormed to see how we could reduce the number of entries that needed to be manually rejected. After analyzing the current approval rates, we have found a correlation between embedding distances and approval rates. If the article embedding and book embedding has a more significant cosine distance than a certain threshold, they would always get rejected. So the team has decided to pre-filter matches by using this threshold value. With the generic implementation and separation of data in each stage, they could quickly introduce another component, named “Trend Relevance Filter”, to the pipeline. This filter aims to detect and pre-filter wrong matches helping the human QA.
The architecture of the system is given above. Braze sends the request via the reverse proxy Cloudflare to the AWS API Gateway. The request is forwarded to a Lambda authorization function which checks the API key sent by Braze. Once the request has been authorized, it is sent to Sagemaker instance. The instance retrieves the output of the data pipeline from an S3 bucket and user data from DynamoDB. Braze sends emails to the users as soon as it receives the response. Finally, during all these stages, logs are sent to DataDog.
To test if our solution worked at a scale, the team did the A/B test by emailing thousands of users in the US. In this email, the output text was generated from the pipeline. Blinkist also sends emails, seen in Figure 4, with their other recommenders to compare and see if localized trend recommender performs better or worse than the existing engines.
The localized trend recommender project performed better than the existing recommendation engines. Nevertheless, Blinkist is not the only company that recognizes the trend recommender as a success. Blinkist Signal, Braze email campaign using the localized trend recommender, has been selected as the winner for the Best Data Application category at the Braze Torchie Awards. Moreover, Braze email campaign will probably not be the only use of this recommender. In the future, the recommender will be utilized in different use cases, such as online marketing.
Working with Blinkist data teams on such an intriguing project has been a tremendous and exciting experience for us. We thank Blinkist for their trust and for a fruitful and interesting cooperation.
