OpenAI’s language model ChatGPT has implemented measures to prevent users from exploiting a method researchers use to reveal sensitive information about the AI training dataset.
This move comes after researchers asked ChatGPT to repeat specific words forever, thereby revealing sensitive private identifiable information (PII) and exposing the chatbot’s training data sources.
As reported by 404media, this vulnerability allowed users to bypass the company’s terms of service.
Earlier research revealed that GPT emits training data at a much higher frequency than other AI models, including LLaMa, Mistral, Falcon, and OPT.
Efforts to replicate this technique, such as instructing ChatGPT to repeat a word forever, now result in the bot issuing an error message that reads: This content may violate our content policy or terms of use. If you believe this to be an error, please submit your feedback — your input will aid our research in this area.
Still possible on GPT 3.5
In our investigation, in GPT-3.5, we found that repeating only a selected few words prompted the error message. We tried several words, including AI, computer, mango, apple, and candid. Only repeating AI yielded the error message and that too for the first time. The second time, things were back to normal.
However, in GPT-4, we got the error message for most prompts.
The reasons behind OpenAI’s decision to flag this specific action remain ambiguous. It raises many questions as most words did not generate an error flag.
While OpenAI’s terms of use explicitly prohibit reverse engineering and the use of automated methods to extract data, it remains unclear how asking a chatbot to repeat a word indefinitely aligns with these restrictions.
Security analysts and governments worldwide have criticised ChatGPT for the alleged data theft. Many organisations, including Apple and Samsung, have banned ChatGPT for their employees.
This fear, combined with the generative AI’s potential for job loss and data leaks, has led to many concerns regarding AIs.
Also read: 1681 HuggingFace tokens exposed posing supply chain threats to organisations
