Updated 01/18/2023: Added comment from Roblox.

Concerned about what her children were seeing on the extremely popular and controversial game production platform Roblox (opens in new tab), one parent decided to take a look for herself. She quickly deleted her children's accounts after six hours of playing the game for five-year-olds.

According to Caroline Velociraptor (opens in new tab), a veteran gaming industry Twitter handle who has worked for Bethesda, Ubisoft, and BioWare, the idea is to show the dark side of monetizing children and the service game is such a young users, and that it began with a curiosity about targeting users: "weird pedo stuff, toilet voyeurism games, and suicidal idealization."

Roblox has various age ratings, which affect which accounts can view certain content. Content creators complete a questionnaire to apply these ratings to the content they create. Thus, even though the criteria are set by Roblox, it does not manually age-rate the experience created by the player. Few major technology companies will talk about this, but since they do not want to invest in (or consider impractical) manual pre-moderation at this scale, the "solution" is to rely on the community to moderate based on reports.

The most disturbing theme Caroline encounters is the subtext of the toilet game.

"You use the toilet," Carolyn writes. 'Often your character's bottom clothes disappear. The door stays open and other players see you. Toilets turn yellow or brown."

As a parent, let me say that it is natural for young children to be preoccupied with toilets and bodily functions. However, one of the most ambiguous areas with regard to moderation is the role-playing experience, and unfortunately, that seems to be the real purpose of these toilet games.

"In no small number of these games," Carolyn writes, "there are areas where characters are forced to recline (by "slipping in water" or being hit by other players). And in some cases, they glitch or get stuck there."

When this happens, Carolyn provides two videos showing it, the player begins to move his or her avatar back and forth under the legs of the prone avatar. This seems to be a clear attempt to represent sex between two avatars, and is repeated in the videos by several players.

Carolyn cites another example of this "creep factor" as a toilet game where users can take pictures of each other trapped in a stall; Survive Pepa [sic] Piggy (opens in new tab) is clearly for all ages and features a bloody nightmarish header image of a bloody Daddy Pig being stabbed in the belly (Peppa Pig survival games are a genre of their own at Roblox).

The Toilet Game is a perfect example of why children and user-generated content can be such a bad combination and blur the boundaries. In theory, what would be wrong with one's child playing a silly video game in which characters pee and poop? However, the example of what is happening in Roblox's bathroom game makes it clear that this is nightmare fuel and not something that children should be watching.

"I was wondering why Roblox suddenly approved thousands of games for children under 8 (especially NSFW-like games like RP World)," Carolyn writes, referring to reports of developer exploitation on Roblox Corp.'s platform referring in part to (opens in new tab). She wrote, "As more and more parents began using the age limit settings, the problem became that fewer active players were engaging in the platform's most profitable experiences (which are impossible to actually make safe for children). Apparently, they don't care anymore.

I contacted Carolyn Velociraptor on Twitter. She has been dealing with some backlash from elements of Roblox fandom since her observations went viral.

"I should clarify from my thread (after speaking with several Roblox developers) that Roblox's parental control methods have changed. This all changed when Roblox decided to hold developers accountable and have them fill out a questionnaire to determine the age rating of their games and then use that rating in their new parental control system.

"Some developers felt pressured to answer the questionnaire in such a way as to obtain an all-ages rating,"

and "some developers felt pressured to use the questionnaire to determine the age rating of the game.

In other words, it is definitely in the developer's interest to try to avoid such restrictions. Purely because failure to obtain an "all ages" rating would severely limit the audience. Of greater concern is the change itself, which suggests that Roblox has more or less (in Caroline's words) "deliberately abdicated its responsibility."

That is a bigger takeaway than toilets.

Roblox has a devil's advocate position on the scale of its platform and therefore the seemingly insurmountable nature of the problem. But scale works both ways, and Roblox Corp is enormously profitable (the company's December 22 sales were $430-$439 million, daily users were 61.5 million, and the stock price was up 30% year over year).

If it is making so much money from an audience that tends to skew young, questions about responsible and ethical business must be answered; Roblox does not respond to criticism in public, preferring to release investor updates, keep interviews to a minimum, and tally funds It prefers to That stance may no longer work now that regulators are suspicious of big tech companies' treatment of children and have even threatened to jail bosses who don't do enough in the UK.

Update 01/18/2023: A lengthy statement was issued by Roblox, excerpted here:

"Last September, we launched the Experience Guidelines. The Experience Guidelines focus on the content of experiences created by developers, and there is an important distinction between experience content and user behavior.

"User behavior within the experience (which is what was highlighted in that thread) is dynamic and constantly changing, so user behavior that may occur within the experience is not evaluated within the Experience Guidelines.

The Experience Guidelines are Roblox age ratings.

"To determine the suitability of content for each age category, we researched global industry standards and consulted with child development experts to guide our policies. experiences highlighted in the Twitter thread, content is generally appropriate for all ages and are labeled "all ages" because they may, in rare instances, contain mild violence and/or mildly unrealistic blood.