Will OpenAI Let ChatGPT Make Porn? AI Maker Says It Depends – Decrypt

Leading AI developer OpenAI riled up the tech world and the mainstream press alike on Thursday when it introduced a new model specification that many interpreted to mean that its wildly popular generative AI tools would be allowed to generate not-safe-for-work adult content.

“We believe developers and users should have the flexibility to use our services as they see fit, so long as they comply with our usage policies,” OpenAI wrote. “We’re exploring whether we can responsibly provide the ability to generate NSFW content in age-appropriate contexts through the API and ChatGPT.”

In its announcement, OpenAI also reiterated that its current policy prevents ChatGPT from serving up NSFW content, described as content that would not be appropriate in “a conversation in a professional setting, like erotica, extreme gore, slurs, and unsolicited profanity.”

Nonetheless, the idea that OpenAI was contemplating lifting restrictions on the creation of pornography raised eyebrows. In retrospect, an Open AI product lead tells Decrypt, the company could have better explained what it was “exploring.”

“I think what I would love to do—based on the feedback and the reaction, in the next version we share—is be more precise about what some people’s definition of NSFW content and the taxonomy here,” OpenAI product lead for model behavior Joanna Jang said, noting that NSFW could mean anything from written profanity to AI-generated deepfake images.

As for what specific material would be allowed under a more permissive stance, Jang told NPR that it “depends on your definition of porn.”

She stressed that deepfakes are absolutely off the table.

“Following applicable laws and protecting people’s privacy, we thought that would cover it—but we should be clear that we are not in it for creating AI deepfakes or AI porn,” Jang told Decrypt. “That is just not something that we even should have the time or bandwidth to be prioritizing when there are more important problems to be solved.”

In addition to blocking NSFW content and ensuring legal compliance, the OpenAI model specification includes several other rules that ChatGPT is designed to follow, including not providing hazardous information, respecting creators and their rights, and protecting people’s privacy.

Currently, OpenAI’s ChatGPT Plus—which includes Dall-E 3 for images and GPT-4 for text—will not allow users to generate overtly sexual or gory images or text. When asked to do so, ChatGPT will respond that the request violates its terms of service. As a result, Jang points out that the web is rife with complaints from OpenAI users that the company is censoring them.

“There’s a lot of criticism of censorship and what have you, and a lot of that discussion is conflating—to no one’s fault—what is OpenAI’s policy versus what is actually not our policy,” Jang said. “Are these models behaving that way, even though it goes against the guidelines?”

As Jang explained, OpenAI published the spec to lay out ideal model behaviors, focusing on legal compliance and avoiding NSFW content while embracing transparency.

“We want to bring more nuance to this discussion because right now—before the model spec—it was, ‘Should AI crate porn or not?’” Jang said. “And I’m hoping that through the model spec, we can actually have these conversations.

”Again, this is why I wish I had actually put down [a framework] so that we could have kickstarted this conversation even a day earlier,” she added.

For mental health experts, the prospect of popular AI platforms moving into the realm of pornography and adult content is troubling.

“In the field of sex and porn addiction, with AI-generated pornography, we are starting to see an increase in addictive behaviors,” Brandon Simpson, a behavioral health specialist at the Men’s Health Foundation, told Decrypt. “Since AI-generated pornography creates the unique and ever-growing intensity needed to fulfill a dopamine response, people are diving in and replacing it with human interactions, which leads to levels of social anxiety, performance anxiety, and a variety of other sexual-related dysfunctions.”

Even with guardrails put in place by OpenAI and other major generative AI platforms from Google and Meta, there is strong demand for AI-generated porn, and myriad ways to harness the technology to depict an unsuspecting victim—or generate child sexual abuse material (CSAM).

“The spec is only part of our story for how to build and deploy AI responsibly,” an OpenAI representative later told Decrypt. ”It’s complemented by our usage policies, how we expect people to use the API and ChatGPT”—ultimately intended to demonstrate transparency and “to start a public conversation about how it could be changed and improved.”

OpenAI has invested heavily in improving the privacy and security of its suite of AI tools, including hiring cybersecurity red teams to find vulnerabilities on its platforms. In February, OpenAI and Microsoft announced a joint operation blocking Chinese and North Korean hackers from using ChatGPT, and OpenAI joined with Google and Meta last month in a pledge to prioritize child safety in the development of its AI models.

Edited by Ryan Ozawa.