Lensa AI is causing a stir with its sexy “Magic Avatar” images that no one wants

Over the past week, smartphone app Lensa AI has become a popular topic on social media because it can create stylized AI avatars based on selfies users upload. It is arguably the first time that the personal latent spread avatar generation has reached a wide audience.

While Lensa AI has proven popular with people on social media who like to share their AI selfies, the press has widely focused on the app’s reported tendency to sexualize images of women when launching the Lensa AI avatar feature.

Lensa was launched in 2018 as a Prisma Labs product as a subscription app focused on AI-powered photo editing. In late November 2022, the app has grown in popularity thanks to a new “Magic Avatar” feature. Lensa is said to use the Stable Diffusion image synthesis model under the hood, and Magic Avatar appears to be using a customization training method similar to Dreambooth (the implications of which we covered recently). All training takes place off the device and in the cloud.

In early December, women using the app noticed that Lensa’s Magic Avatar feature would inadvertently create semi-pornographic or sexual images. For CNN, Zoe Sottile wrote: “One challenge I encountered with the app was described by other women online. Although all the photos I uploaded were fully clothed and most were close-ups of my face, the app returned several photos with implied or actual nudity.”

The reaction in the press grew as other women had gone through similar things. For example, here’s a selection of the headlines covering Lensa in various publications:

Meanwhile, the same sexual arousal issue didn’t appear in photos of men uploaded to the Magic Avatar feature. For MIT Technology Review, Melissa Heckela writes: “My avatar was pornographic, while my male colleagues were astronauts, explorers, and inventors.”

This is one of the risks of basing a product on Stable Diffusion 1. x, which can easily make its output sexual by default. The behavior comes from the large amount of sexual images found in the image training data set, which were scraped from the Internet. (Stable Diffusion 2.x attempted to correct this by removing the NSFW material from the training set.)

The internet’s cultural biases toward sexual portrayals of women online gave AI creator Lensa its leanings. Speaking to TechCrunch, Prisma Labs CEO Andrey Usoltsev said, “Neither we nor the Stability AI organization can consciously apply any representational biases; to be more precise, unfiltered man-made data obtained online submitted the model to biases.” current humankind. We acknowledge that societal biases can exist, and so do we.”

In response to widespread criticism, Prisma Labs is working to prevent the accidental generation of nude images in Lensa AI.

On Tuesday, we experimented with Lensa AI by uploading 15 images of a woman (as well as 15 images of a man), and then paid to have a Magic Avatar represent each of them. The results of the women we saw weren’t sexual in any obvious way, so Prisma’s efforts to reduce the incidence of NSFW-style imagery may actually be working.

This is not to say that Lensa AI’s tendencies to sexualize female subjects do not exist. It should be noted that because of how Stable Diffusion generates images, different women are likely to see different results depending on how similar their face (or input images) are to images of a particular actress, celebrity, or model in the Stable Diffusion dataset. In our experience, this can strongly dictate how stable diffusion makes the rest of a person’s body and context.

Using Lensa AI is easy, but it’s a product with an annual subscription fee (currently $50 a year) and a one-time fee for training Magic Avatars — $3.99 for 50 avatars, $5.99 for 100 avatars, or $7.99 for 200 avatars. Customers upload 10-20 selfies taken from different angles and then wait approximately 20-30 minutes to see the results. Users must confirm that they are 18 years of age or older to use the Service.

Those who are technically inclined can try out Dreambooth for free instead, but it currently requires some knowledge of development tools to get started.