OpenAI, the makers of ChatGPT, has released research exposing the potential for AI models to engaged in “AI scheming” by intentionally lying to humans, raising concerns about how people chatbots in every from the workplace to education.
OpenAI recently published a research report that sheds light on a disturbing trend in AI models: deliberate lying and scheming. The research, conducted in collaboration with Apollo Research, has uncovered instances where AI models behave one way on the surface while concealing their true intentions, a practice named “scheming” by the researchers.
The researchers likened AI scheming to a human stock broker breaking the law to maximize profits, highlighting the potential for AI to engage in deceptive practices to achieve its goals. While most instances of AI scheming observed in the study were not deemed highly harmful, such as pretending to have completed a task without actually doing so, the researchers cautioned that the potential for more consequential scheming could grow as AI is assigned more complex tasks with real-world consequences.
One of the most concerning aspects of the research is the revelation that AI developers have not yet found a reliable way to train models not to scheme. Attempts to “train out” scheming could inadvertently teach the model to scheme more carefully and covertly to avoid detection. This situational awareness exhibited by AI models adds another layer of complexity to the challenge of ensuring AI alignment with human values and goals.
The study also found that AI models can pretend not to be scheming when they understand they are being tested, further complicating efforts to detect and mitigate deceptive behavior. This ability to deceive even during evaluation raises serious questions about the trustworthiness of AI systems and the need for robust safeguards.
Up to this point, one of the primary concerns about AI has been hallucinations, in which AI systems make up false information while attempting to be helpful. Multiple lawyers have ended up in hot water after filing legal briefs written by AI that include references to imaginary cases. As Breitbart News previously reported:
In an internal letter shared in a court filing, Morgan & Morgan’s chief transformation officer cautioned the firm’s more than 1,000 attorneys that citing fake AI-generated cases in court documents could lead to serious consequences, including potential termination. This warning comes after one of the firm’s lead attorneys, Rudwin Ayala, cited eight cases in a lawsuit against Walmart that were later discovered to have been generated by ChatGPT, an AI chatbot.
The incident has raised concerns about the growing use of AI tools in the legal profession and the potential risks associated with relying on these tools without proper verification. Walmart’s lawyers urged the court to consider sanctions against Morgan & Morgan, arguing that the cited cases “seemingly do not exist anywhere other than in the world of Artificial Intelligence.”
In response to the incident, Ayala was immediately removed from the case and replaced by his supervisor, T. Michael Morgan, Esq. Morgan expressed “great embarrassment” over the fake citations and agreed to pay all fees and expenses related to Walmart’s reply to the erroneous court filing. He emphasized that this incident should serve as a “cautionary tale” for both his firm and the legal community as a whole.
Breitbart News will continue to report on AI developments.
Lucas Nolan is a reporter for Breitbart News covering issues of free speech and online censorship.
