Trying on new makeup is a hassle — apply, inspect, wash off, repeat. Not to mention the tribulation of visiting brick-and-mortar stores during the pandemic. Augmented reality is helping people try on all the makeup they want without leaving home.

What’s new: Modiface, a subsidiary of beauty giant L’Oréal, uses GANs to let customers see how different colors of lipstick, eye shadow, and hair will look on them.

How it works: Modiface’s approach is a hybrid of CycleGAN, StarGAN and StyleGAN, company operations chief Jeff Houghton told The Batch. It uses a CNN to track landmarks on a user’s hair and face. At L’Oreal’s website, users can select different lipsticks, eyeliners, blushes and hair dyes and fine-tune their shade, texture, and gloss. Then they can virtually apply those products to an uploaded selfie or a real-time video of their own faces.

To train the algorithm, Modiface collected and annotated thousands of images of people wearing and not wearing makeup and hair coloring under various lighting and background conditions.
A CycleGAN takes as input an image of a user’s face without makeup. When the user selects a specific makeup product, the system generates an image that shows what the selected product looks like when applied.
Adding new makeup types to the model is as simple as adding a color swatch to the database. Modiface must retrain the model to include new products with properties, such as a metallic sheen, that aren’t represented in the training data.
Collaborations with Amazon and Facebook enable users to access Modiface directly from those platforms.

Behind the news: Augmented reality applications are reshaping the beauty industry. In 2018, the same year L’Oréal purchased Modiface, American chain Ulta acquired Glamst, a startup specializing in augmented reality. Meitu, a multi-billion dollar Chinese company, uses AI-driven face manipulation to make its users appear more attractive in social media posts, job applications, and other digital venues.

Why it matters: E-commerce is increasingly important to the beauty industry’s bottom line, and the pandemic is driving even more business to the web. Tools like this make it easier for customers to try out products, which may boost sales.

We’re thinking: It seems like we’re spending half our lives in video conferences. Do we still need to apply makeup at all, or can we let a GAN do it instead?

StarGAN

A Good Look for AI

Subscribe to The Batch