OpenAI's Dilemma: Withholding ChatGPT Detection Technology

OpenAI Holds Key to Stopping ChatGPT Cheating, But Keeps It Private

OpenAI reported has a way to detect if ChatGPT has created text with a 99.9% certainty, but it hasn't made it available to the public. According to the Wall Street Journal, the project " has been mired in internal debate" at OpenAI for two years, and has technically been ready to be released for around a year. The decision to hold it back lies in the company's struggle between wanting to be transparent about the use of its product and wanting to attract new users. The idea, of course, being if you call a bunch of college students out for using your tool to write their research papers, then they ultimately won't use your tool to write their research papers. It's a win for professors but not for the cheaters and procrastinators amongst us. The tool would work by inserting a watermark of sorts into text created by ChatGPT. The watermark wouldn't be visible to the human eye; however, when run through the AI-detection tool later on, the detector could provide a score on how likely it is that the text was created by ChatGPT. There is concern internally that watermarks could potentially be erased by translating the text into another language and back again through something like Google Translate. Also, employees warn that if too many people had access to the detection tool, bad actors would likely be able to figure out OpenAI's watermarking technique, rendering the tool useless. An OpenAI spokesperson told The Journal that another concern is that the tool might disproportionally impact non-native English speakers. Those who want the tool released; however, argue the good it could do outweighs the bad. Google has a watermarking tool that can detect text created by its Gemini AI. That tool, called SynthID, is currently being beta tested but also is not available to the public.

OpenAI has the tech to watermark ChatGPT text -- it just won't release it

Some say watermarking is the responsible thing to do, but it's complicated. According to The Wall Street Journal, there's internal conflict at OpenAI over whether or not to release a watermarking tool that would allow people to test text to see whether it was generated by ChatGPT or not. To deploy the tool, OpenAI would make tweaks to ChatGPT that would lead it to leave a trail in the text it generates that can be detected by a special tool. The watermark would be undetectable by human readers without the tool, and the company's internal testing has shown that it does not negatively affect the quality of outputs. The detector would be accurate 99.9 percent of the time. Some OpenAI employees have campaigned for the tool's release, but others believe that would be the wrong move, citing a few specific problems. First among those is the fact that even with 99.9 percent accuracy, the watermark detector would still be wrong some of the time, considering how often ChatGPT is used. Further ReadingAmong those who have shown the most interest in using the tools are teachers and professors, who have seen a rapid rise of ChatGPT-generated school papers and other assignments. But OpenAI's argument is this: 99.9 percent accuracy sounds like a lot, but imagine that one among 1,000 college papers was falsely labeled as cheating. That could lead to some unfortunate consequences for innocent students. Further, OpenAI says the tool's release could stigmatize non-native English speakers using ChatGPT for translation or to improve their writing, which the company argues is a legitimate use of the tool. Finally, a blog post by OpenAI clarified that it is relatively easy for bad actors to bypass the watermark in its current form. Running ChatGPT's output through another LLM text generator could do it, as could asking ChatGPT to insert special characters throughout the output and then manually removing those characters. There's one problem OpenAI didn't mention in its blog post, but it did come up in the Wall Street Journal article: A survey of ChatGPT users showed that as many as 30 percent said they would stop using ChatGPT if its output was watermarked. OpenAI is sitting on the watermark feature and hasn't rolled it out yet. It's also investigating alternative solutions that are still in development, such as including cryptographically signed metadata in outputs. That solution would be similar to how OpenAI has approached the content provenance issue with the DALL-E 3 image generator. The company includes C2PA metadata to help people identify when and how images were modified with DALL-E. OpenAI previously released and supported an AI text detection tool. It focused not just on ChatGPT, but also on detecting whether any AI tool was used to generate text. However, it was discontinued because it was highly inaccurate and prone to false positives, making it mostly useless.

OpenAI says it's taking a 'deliberate approach' to releasing tools that can detect writing from ChatGPT | TechCrunch

OpenAI has built a tool that could potentially catch students who cheat by asking ChatGPT to write their assignments -- but according to The Wall Street Journal, the company is debating whether to actually release it. In a statement provided to TechCrunch, an OpenAI spokesperson confirmed that the company is researching the text watermarking method described in the Journal's story, but said it's taking a "deliberate approach" to releasing anything to the public due to "the complexities involved and its likely impact on the broader ecosystem beyond OpenAI." "The text watermarking method we're developing is technically promising, but has important risks we're weighing while we research alternatives, including susceptibility to circumvention by bad actors and the potential to disproportionately impact groups like non-English speakers," the spokesperson said. This would be a different approach from most previous efforts to detect AI-generated text, which have been largely ineffective. Even OpenAI itself shut down its previous AI text detector last year due to its "low rate of accuracy." With text watermarking, OpenAI would focus solely on detecting writing from ChatGPT, not from other companies' models. It would do so by making small changes to how ChatGPT selects words, essentially creating an invisible watermark in the writing that could later be detected by a separate tool. Following the publication of the Journal's story, OpenAI also updated a May blog post about its research around detecting AI-generated content. The update says text watermarking has proven "highly accurate and even effective against localized tampering, such as paraphrasing," but has proven "less robust against globalized tampering; like using translation systems, rewording with another generative model, or asking the model to insert a special character in between every word and then deleting that character." As a result, OpenAI writes that this method is "trivial to circumvention by bad actors." OpenAI's update also echoes the spokesperson's point about non-English speakers, writing that text watermarking could "stigmatize use of AI as a useful writing tool for non-native English speakers."

OpenAI Hesitates to Detect AI-Written Assignments Because It Risks Losing Users

Bad news for teachers and good news for students; OpenAI is on the fence about releasing its tool that accurately detects ChatGPT-generated content. The tech giant has been working on the tool for around two years, with the tool being release-ready for a year now. But, as The Wall Street Journal reports, the company is internally divided on the idea of making it publicly available. Back in May of this year itself, OpenAI released a blog post, shedding some light on the importance of "understanding the source of what we see and hear online." The very point of this post was to reveal to the world how OpenAI is working on "creating new technology that specifically helps people identify content created by our own tools." Now, as soon as the WSJ's report went live, OpenAI updated this very blog post with some information on the different methods they're using to detect audiovisual content origins. These are: The text watermarking method is what will allow folks to identify a ChatGPT-written assignment or research paper. Now, while OpenAI does state that this is "highly accurate and even effective against localized tampering, such as paraphrasing" they also talk about how, it is less robust against globalized tampering; like using translation systems, rewording with another generative model, or asking the model to insert a special character in between every word and then deleting that character - making it trivial to circumvention by bad actors. But, according to some internal documents that WSJ saw, this very text watermarking method turned in a beyond impressive 99.9% efficiency. So, while OpenAI goes on to talk about how this could potentially stigmatize AI writing for non-native English speakers, personally, I don't see how it hampers the greater good. I even asked ChatGPT if it has any text watermarking standards in place, and here's what the AI chatbot had to say: ChatGPT states that while there's no built-in mechanism as such, "OpenAI uses various techniques to detect AI-generated content, but these are typically not accessible to end users." OpenAI used to be all about using AI responsibly. So, if you already have a weapon to prevent academicians from cheating, what's stopping you from releasing it to end users? Well, turns out that a ChatGPT survey unveiled how 30% of its users will use the chatbot less if watermarking comes into being. Isn't that their problem? Why isn't OpenAI sticking to its ethical standards? Everyone knows that ChatGPT is not perfect, especially since it displays the "ChatGPT can make mistakes. Check important info," message right underneath the text panel. So, if students are taking the AI-generated content as is and pasting it as their final assignment, they should be taught not to do so. The AI reliance is all too real, and text watermarking has the potential to break that very vicious cycle. Even ChatGPT asks you not to use AI as a tool to complete assignments on your behalf since it can, and I quote, "impede your educational development and may violate academic integrity policies." Smart. I think even your own AI tool knows what it's supposed to be used for, OpenAI. Not to mention that OpenAI's work on C2PA Metadata for AI-generated images is a lifesaver on its own, having over 99% accuracy in determining whether an image is generated by DALL-E or not. From displaying the edits made to AI-generated images to showing the tools used, this is some solid origin tracking that needs to be made available to the public. AI content watermarking is how you can make people use AI responsibly. When students know that there's something out there to detect if they use AI-generated content, they will not use it to do all their heavy lifting. Instead, AI will start getting used as the reliable helping hand that it is. While text watermarking is not perfect and even if it is released, there will be workarounds, it won't be as easy as it is now to pass AI-generated content as your own. If you ask me, OpenAI should stick to contributing to its authenticity standards like it has planned on doing, instead of taking steps back when the time is right.