Instagram will test nudity protection in messages to combat sextortion

Far from being a shining example of protecting young people online, Instagram failed to stop its algorithm from promoting child sexual abuse content. But the new feature brings some (at least a little?) hope that the platform can become more secure. Meta has announced new tools aimed at protecting users from intimate image abuse and sextortion – when a person is digitally blackmailed by the threat of sharing private media.

One of the most important updates is nudity protection in private messages. Meta first confirmed it was developing the technology back in 2022, which would automatically launch the tool for users under 18 years old. Once turned on, the machine learning tool will detect and blur images it suspects contain the recipient’s nudity. Analysis occurs on the user’s device, so messages should remain end-to-end encrypted and Meta cannot access them. Users can choose to see images and pop-up messages from Meta indicating they should not feel pressure to respond, as well as a safety alert button and the option to block the sender.

Meta’s new tool, which will begin testing “soon”, will also detect if a person is sending nude photos and warn them to “be careful when sharing sensitive photos” while outlining potential risks. Additionally, it reminds users that they can delete the message before anyone sees it. And then there’s the final warning: When someone tries to retweet a message where nudity has been detected, a reminder appears asking to be responsible and respectful (although it still allows the image to be retweeted).

There are also tools designed to detect potential scammers or sextortionists and make it harder for them to access teenagers. Message requests from these possible bad actors should now go to hidden requests, and anyone already involved in the conversation will receive a warning with a boundary reminder and steps to report the user. As for young people, Meta previously banned users 16 or younger from messaging them without being connected to each other – even if another account claimed to be the same age. Now, even if these would-be scammers follow each other, they don’t see the option to send messages to teens.