OpenAI Group Won't Release Dangerous AI Text Generator

Posted on March 27, 2019

The OpenAI group created an AI called GPT-2 that is capable of generating full articles from just a couple human-written sentence prompts. OpenAI says GPT-2's writing samples "feel close to human quality and show coherence over a page or more of text." The AI could be used to write news articles, reviews, fiction and other types of content based on just a little bit of input.

OpenAI is not releasing the full dataset or code out of concerns it could be used nefariously. Open AI says in its GPT-2 report, "Due to our concerns about malicious applications of the technology, we are not releasing the trained model. As an experiment in responsible disclosure, we are instead releasing a much smaller model for researchers to experiment with, as well as a technical paper."

The Guardian notes that the organization usually fully releases its research. However, it is not releasing GPT-2 out of fears the program could be misused. Elon Musk is one of the backers of the nonprofit OpenAI.

OpenAI gives one example where its GPT-2 text generator is given two sentences about a herd of unicorns discovered in the Andes Mountains. GPT-2 then generated an article about the discovery that includes quotes from a fictional biologist. It is a ridiculous story - based on an absurd headline - but the complete article is coherent and it does read like a human could have written it. Here is a short excerpt from the machine-written response.

The scientist named the population, after their distinctive horn, Ovid’s Unicorn. These four-horned, silver-white unicorns were previously unknown to science.
Now, after almost two centuries, the mystery of what sparked this odd phenomenon is finally solved.
Dr. Jorge Pérez, an evolutionary biologist from the University of La Paz, and several companions, were exploring the Andes Mountains when they found a small valley, with no other animals or humans. Pérez noticed that the valley had what appeared to be a natural fountain, surrounded by two peaks of rock and silver snow.
Pérez and the others then ventured further into the valley. “By the time we reached the top of one peak, the water looked blue, with some crystals on top,” said Pérez.
Pérez and his friends were astonished to see the unicorn herd. These creatures could be seen from the air without having to move too much to see them – they were so close they could touch their horns.

OpenAI notes that malicious actors are already using AI tools to create fake accounts and fake stories. They say, "We should consider how research into the generation of synthetic images, videos, audio, and text may further combine to unlock new as-yet-unanticipated capabilities for these actors, and should seek to create better technical and non-technical countermeasures."

OpenAI describes GPT-2 as a large transformer-based language model with 1.5 billion parameters. It was trained on a database consisting of 8 billion webpages. If OpenAI can create something like GPT-2 then it is highly likely that another organization can as well. As we have reported some companies already using AI technology to write stories. For example, the AP has robots cranking out boring financial articles.

Powerful writing AIs are going to create a murky future that makes it increasingly difficult to ascertain whether something is written by a human or a program. The AIs can access massive amounts of human writing and use it as datasets to generate new content nearly instantaneously on a vast amount of topics. It is hard to see how this does not become a growing problem for writers in the near future, especially for writers in the news and information industries.