AI \u201cscheming\u201d can prove to be a challenge when working with generative artificial intelligence. AI might follow the letter of a prompt but cut corners under the hood.\u00a0<\/p>\n
On Sep. 17, OpenAI published a blog post saying it had developed, with Apollo Research, a method called deliberative alignment to reduce scheming in two of its AI models.\u00a0\u00a0<\/p>\n
\u201cWe are seeing signs that this issue is beginning to emerge across all frontier models today,\u201d the researchers wrote.<\/p>\n
Generative AI can deliberately withhold or distort task-relevant information, or arrive at a conclusion that aligns with the literal meaning of a prompt but not the prompt\u2019s intended purpose. AI scheming<\/a> usually causes small-scale problems, such as ChatGPT pretending to complete a task that is actually outside its capabilities.\u00a0<\/p>\n \u201cHowever, as AIs are assigned more complex tasks with real-world consequences and begin pursuing more ambiguous, long-term goals, we expect that the potential for harmful scheming will grow \u2014 so our safeguards and our ability to rigorously test must grow correspondingly,\u201d the OpenAI researchers wrote<\/a>.\u00a0<\/p>\n Ironically, tests attempting to stop scheming may, in fact, reinforce it, the researchers noted.\u00a0<\/p>\n \u201cMitigations that reduce scheming may either (a) truly eliminate it, or (b) just teach the model to conceal it better,\u201d they noted. \u201cBoth outcomes look the same from the outside, so the reason for behavioral change is critical.\u201c<\/p>\n In an evaluation conducted with Apollo Research<\/a>, OpenAI created:\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/p>\n A teaching method to reduce scheming.\u00a0<\/p>\n A protocol for stress-testing future iterations of such methods.\u00a0<\/p>\n Plans to conduct further research with Apollo, raise awareness of the issue, and dedicate more team resources to scheming-related research.<\/p>\n OpenAI found cases of the AI engaging in covert actions, such as withholding or distorting information, in OpenAI o3 and o4-mini, Gemini-2.5-pro, and Claude Opus-4. (The investigation was conducted before GPT-5 was available.)\u00a0<\/p>\n The training performed based on the evaluation\u2019s findings reduced occurrences of scheming in OpenAI o4-mini from 8.7% to 0.3%. Scheming in OpenAI o3 was reduced from 13% to 0.4%.\u00a0<\/p>\n Another factor in reducing hidden processes in AI is reasoning transparency, or the difficulty in teaching generative AI to accurately explain its own thought processes<\/a>. As OpenAI pointed out, reasoning transparency is an ongoing field of study among AI giants such as Anthropic.\u00a0<\/p>\n In June, Google removed a view that had allowed developers to trace reasoning in Google AI Studio. OpenAI o1, the company\u2019s first reasoning model, also failed to show<\/a> its chain of thought.\u00a0<\/p>\n Vulnerabilities related to reasoning transparency are eligible for submission to OpenAI\u2019s Red-Teaming Challenge for gpt-oss-20b.\u00a0<\/p>\n OpenAI updated ChatGPT in response to address <\/strong>dangers to teens<\/strong><\/a>.\u00a0<\/strong><\/p>\n The post AI Might \u2018Scheme\u2019 Less Thanks to This OpenAI Research<\/a> appeared first on eWEEK<\/a>.<\/p>","protected":false},"excerpt":{"rendered":" AI \u201cscheming\u201d can prove to be a challenge when working with generative artificial intelligence. AI might follow the letter of a prompt but cut corners under the hood.\u00a0 On Sep. 17, OpenAI published a blog post saying it had developed, with Apollo Research, a method called deliberative alignment to reduce scheming in two of its […]<\/p>\n","protected":false},"author":0,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[],"class_list":["post-4986","post","type-post","status-publish","format-standard","hentry","category-news"],"_links":{"self":[{"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/posts\/4986"}],"collection":[{"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=4986"}],"version-history":[{"count":0,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/posts\/4986\/revisions"}],"wp:attachment":[{"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=4986"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=4986"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=4986"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}Ongoing challenges<\/h2>\n