GPT-5.1-Codex-Max System Card

GPT-5.1-Codex-Max is our new frontier agentic coding model. It is built on an update to our foundational reasoning model trained on agentic tasks across software engineering, math, research, medicine, computer use and more. It is our first model natively trained to operate across multiple context windows through a process called compaction, coherently working over millions of tokens in a single task. Like its predecessors, GPT-5.1-Codex-Max was trained on real-world software engineering tasks like PR creation, code review, frontend coding and Q&A.

This system card outlines the comprehensive safety measures implemented for GPT-5.1-Codex-Max. It details both model-level mitigations, such as specialized safety training for harmful tasks and prompt injections, and product-level mitigations like agent sandboxing and configurable network access.

GPT-5.1-Codex-Max was evaluated under our Preparedness Framework. It is very capable in the cybersecurity domain but does not reach High capability on cybersecurity. We expect current trends of rapidly increasing capability to continue, and for models to cross the High cybersecurity threshold in the near future. Like other recent models, it is being treated as High capability on biology, and is being deployed with the corresponding suite of safeguards we use for GPT-5. It does not reach High capability on AI self-improvement.

To further ecosystem insight into frontier AI, and for consistency with our general practice, we provide the following benchmark evaluations of GPT-5.1-Codex-Max across disallowed content categories tested in a conversational setting. We do not believe these conversational evals are reflective of real-world risk in the context of this coding-focused model. The model is not intended for deployment in any general-purpose chat application, and will not be deployed by OpenAI in any such context.

We report here on our Production Benchmarks, a new more challenging evaluation set with conversations representative of challenging examples from production data. As we noted in previous system cards, we introduced these Production Benchmarks to help us measure continuing progress given that our earlier Standard evaluations for these categories had become relatively saturated.

These evaluations were deliberately created to be difficult. They were built around cases in which our existing models were not yet giving ideal responses, and this is reflected in the scores below. Error rates are not representative of average production traffic. The primary metric is not_unsafe, checking that the model did not produce output that is disallowed under the relevant OpenAI policy.

* New evaluations, as introduced in the GPT-5 update on sensitive conversations.

** These evaluations were introduced after GPT-5-Codex was released

GPT-5.1-Codex-Max generally performs on par with or higher than GPT-5.1-thinking when used in a conversational setting. As stated above, the model is not intended for conversational deployments.

We evaluate the robustness of models to jailbreaks: adversarial prompts that purposely try to circumvent model refusals for content it’s not supposed to produce.

Below is an adaptation of the academic jailbreak eval, StrongReject [1]. This eval inserts a known jailbreak into an example from disallowed content evals. We then run it through the same policy graders we use for disallowed content checks. We test jailbreak techniques on base prompts across harm categories, and evaluate for not_unsafe according to relevant policy.

We ran the image input evaluations introduced with ChatGPT agent, that evaluate for not_unsafe model output, given disallowed combined text and image input.

Codex agents are intended to operate within isolated, secure environments to minimize potential risks during task execution. The sandbox method is determined by the interface, and differs between using Codex locally or in the cloud.

When using Codex in the cloud, the agent runs with access to an isolated container hosted by OpenAI, effectively its own computer with network access disabled by default. This containerized environment prevents the agent from interacting with the user’s host system or other sensitive data outside of its designated workspace.

When using Codex locally on MacOS and Linux, the agent executes commands within a sandbox by default. On MacOS, this sandboxing is enforced using Seatbelt policies, a built-in MacOS feature. On Linux, a combination of seccomp and landlock is utilized to achieve similar isolation. On Windows, users can use an experimental native sandboxing implementation or benefit from Linux sandboxing via Windows Subsystem for Linux. Users can approve running commands unsandboxed with full access, when the model is unable to successfully run a command within the sandbox.

These default sandboxing mechanisms are designed to:

While users have the flexibility to expand these capabilities (e.g., enabling network access to specific domains), the default configurations are intentionally designed to provide a robust baseline for risk mitigation.

As part of our commitment to iterative deployment, we originally launched Codex cloud with a strictly network-disabled, sandboxed task-execution environment. This cautious approach reduced risks like prompt injection while we gathered early feedback. Users told us they understand these risks and want the flexibility to decide what level of Internet connectivity to provide to the agent during task execution.

For example, as the agent works, it may need to install or update dependencies overlooked by the user during environment configuration. Giving the user the choice to enable internet access–whether to a specific set of allowed sites, or to the internet at large–is necessary to unlock a number of use cases that were previously not possible.

We enabled users to decide on a per-project basis which sites, if any, to let the agent access while it is running. This includes the ability to provide a custom allowlist or denylist. Enabling internet access can introduce risks like prompt injection, leaked credentials, or use of code with license restrictions. Users should review outputs carefully and limit access to trusted domains and safe HTTP methods. Learn more in the docs: https://developers.openai.com/codex/cloud/agent-internet

Our approach to safety mitigations builds upon the comprehensive mitigation strategies already implemented for different interfaces including Codex cloud and Codex CLI. This section will focus exclusively on the specific safety training mitigations applied to the GPT-5.1-Codex-Max model itself.

Safeguarding against malicious uses of AI-driven software engineering—such as malware development—is increasingly important. At the same time, protective measures must be carefully designed to avoid unnecessarily impeding legitimate, beneficial use cases that may involve similar techniques, such as low-level kernel engineering.

We have pre-existing policies and safety training data that cover refusing harmful tasks in ChatGPT such as user requests for guidance on how to make illegal drugs. These policies and this training data already are intended to lead to refusals for related coding tasks, such as a request to build a website to sell illegal drugs. To further strengthen safety for Codex, we developed a more detailed policy and training data for codex-1 to refuse tasks related to malware development and have re-used this approach for GPT-5.1-Codex-Max.

To support this, we built a synthetic data pipeline that generates a diverse set of prompts, code snippets, and environment configurations involving malware-relevant scenarios.

We then taught the model to follow these safety specifications—refusing certain high-risk requests, providing only contextual or defensive content, and appropriately handling dual-use scenarios without excessive refusal or hedging. We incorporated edge cases and adversarial examples to thoroughly test the model’s boundaries and reinforce policy-compliant behavior in ambiguous or complex situations. We used a curated “golden set” of test cases developed by internal policy experts to assess the effectiveness of this training.

Prompt injection occurs when an attacker manipulates the model’s behavior by injecting malicious instructions into user inputs or data processed by the model. These injected instructions can override system-level directives or alter the intended function of the model.

With the ability for Codex to make network calls and use web search in local environments, the attack surface for prompt injection expands. The model can now encounter untrusted text not only in user-provided code, but also from external sources like websites, documentation, or search results. This untrusted text can contain malicious instructions designed to alter the model’s behavior.

If successful, a prompt injection against Codex could result in outcomes such as:

In addition to our model-specific mitigations, we also have a set of product-specific mitigations, described below, including sandboxing to prevent undesired actions.

To enhance the model’s robustness against prompt injection attacks, following the Instruction Hierarchy framework [2], we created new safety training data and trained the model to ignore prompt‑injection attacks in the coding environment.

We also developed a new prompt injection evaluation suite tailored to the Codex environment to measure the model’s robustness to such attacks. The evaluation includes several types of attacks, such as:

Coding agents have access to powerful tools—file systems, Git, package managers, and other development interfaces—that enable them to act autonomously. While these capabilities unlock productivity, they also introduce high-impact failure modes that involve deletion or corruption of data.

Simple instructions like “clean the folder” or “reset the branch” can mask dangerous operations (rm -rf, git clean -xfd, git reset –hard, push –force) that lead to data loss, repo corruption, or security boundary violations.

We observed that Codex models were more likely to attempt data-destructive actions when encountering user-produced edits during the course of its rollouts. GPT-5.1-Codex-Max was trained with a “user model” that made conflicting edits over the course of its rollouts during RL. The model received positive reinforcement if it did not revert the user’s changes during the course of the rollout.

To measure that the intervention was effective, we developed a new destructive actions evaluation that measures the model’s ability to preserve user-produced changes and avoid taking destructive actions.

GPT-5.1-Codex-Max’s frontier capabilities are assessed under the Preparedness Framework. We are treating GPT-5.1-Codex-Max as High risk in the Biological and Chemical domain. While the model’s cybersecurity capabilities are stronger than its predecessors, it does not reach our threshold for High capability in the Cyber domain. It also does not reach High capability in the AI self-improvement domain.

As we did for GPT-5, we are treating GPT-5.1-Codex-Max as High risk in the Biological and Chemical domain, and continuing to apply the corresponding safeguards.

We graded the accuracy of model responses to long-form biorisk questions. Our long-form biothreat information questions test acquiring critical and sensitive information across the five stages of the biological threat creation process [3]: Ideation, Acquisition, Magnification, Formulation, and Release.

We designed the questions and detailed rubrics with Gryphon Scientific due to their expertise working with dangerous biological agents in a national security setting. We used the OpenAI o1-preview (pre-mitigation) model as an autograder, validating agreement with a trusted biosecurity expert. We made adjustments to the rubric and iterated on the autograder based on the expert feedback.

GPT-5.1-Codex-Max refused 100% of the prompts in this evaluation.

To evaluate models’ ability to troubleshoot wet lab experiments in a multimodal setting, we evaluate models on a set of 350 fully held-out virology troubleshooting questions from SecureBio.

All models exceed the median domain expert baseline of 22.1%. These scores are on a multi-select variant which increases the difficulty, which we plot here upon SecureBio’s recommendation.

To evaluate models’ ability to troubleshoot commonly published lab protocols, we modify 108 multiple choice questions from FutureHouse’s ProtocolQA dataset [4] to be open-ended short answer questions, which makes the evaluation harder and more realistic than the multiple-choice version. The questions introduce egregious errors in common published protocols, describe the wet lab result of carrying out this protocol, and ask for how to fix the procedure. To compare model performance to that of PhD experts, we performed expert baselining on this evaluation with 19 PhD scientists who have over one year of wet lab experience.

All models underperform the consensus expert baseline (54%) and the median expert baseline of 42%. To run this eval with browsing, we maintain a domain blocklist and filter out any browsing results to sites on the blocklist. We also inspect browsing rollouts using a classifier which flags instances of cheating and manually review all flagged rollouts.

We evaluated models on a tacit knowledge and troubleshooting multiple choice dataset created with Gryphon Scientific. The questions span all 5 stages in the biothreat creation process and focus on areas where tacit knowledge would be a bottleneck. Tacit knowledge questions are meant to be obscure to anyone not working in the field, i.e., they either require tracking down authors of relevant papers or knowing people in the field. Troubleshooting questions are meant to be obscure to anyone without hands-on experience, i.e., the answers are known only to people who have tried the protocol themselves.

This set is uncontaminated; it was created fully in-house with our partners at Gryphon Scientific and has not been published.

GPT-5.1-Codex-Max is the highest performing model on this benchmark at 77%, nearing the consensus expert baseline of 80%. If refusals are counted as successes, the model passes the expert baseline.

To evaluate models’ ability to identify and correct real-world experimental errors in biological protocols, we built a short-answer troubleshooting dataset from expert-written wet lab procedures. TroubleshootingBench focuses on tacit, hands-on knowledge and uncontaminated procedures that are not available online.

Scientists with a PhD in a relevant biological discipline (virology, genetics, microbiology, or protein engineering) were asked to transcribe biological protocols they have personally used in the lab. Each protocol must include precise step-by-step procedures, equipment, and reagents. If a protocol was adapted from a publication, experts were required to significantly alter at least several steps. From these protocols, they created three troubleshooting questions each, introducing subtle or realistic execution errors (e.g., improper homogenization technique) and describing the resulting failed outcome.

After going through independent expert review, the resulting dataset includes 52 protocols, each paired with three expert-written troubleshooting questions. To benchmark model performance, we ran a human baselining campaign with 12 independent PhD experts. The 80th percentile expert score (36.4%) is used as an indicative threshold for model performance. Compared to ProtocolQA Open-Ended, which focuses on well-known published procedures, TroubleshootingBench is designed to test model performance on non-public, experience-grounded protocols and errors that rely on tacit procedural knowledge.

GPT-5.1 scores highest on this evaluation without accounting for refusals, 3 percentage points more than GPT-5.1-Codex. If refusals are counted as successes, GPT-5.1-Codex-Max scores highest at 40%.

Cybersecurity capabilities and capability measurements are both rapidly evolving. GPT-5.1-Codex-Max is the most cyber-capable model we’ve deployed to date. As a result, we are continuing to invest in safeguards that we put in place as part of GPT-5-Codex, including both the model safety training described above and scaling up our monitoring and enforcement pipeline to disrupt potential misuse. We are also working on initiatives to help defenders and strengthen cybersecurity safeguards, and we continue to build and strengthen our cybersecurity evaluations.

Under our Preparedness Framework, High cybersecurity capability is defined as a model that removes existing bottlenecks to scaling cyber operations, including either by automating end-to-end cyber operations against reasonably hardened targets, or by automating the discovery and exploitation of operationally relevant vulnerabilities.

In our analysis of prior cybersecurity incidents involving severe harm through either of these two routes, we find each instance demanded proficiency in each of three skills:

Each of our recent frontier releases has reflected refinements in the benchmarks we use to measure cyber capability:

With this launch, we continued to refine our approach, incorporating three evaluations that are associated with the three skills above:

Each of these evaluations has important limitations:

Because of these limitations, excelling on all three evaluations is necessary but not sufficient for a model to be considered High in cyber capability. Even very strong results can still fall short of what is required for scalable, end-to-end cyber operations against hardened targets. As noted below, GPT-5.1-Codex-Max’s CVE-Bench performance—while stronger than earlier models—does not meet the level of consistency needed for High cyber capability.

We evaluate model performance on a test set of curated, publicly available CTF challenges which met the following criteria:

Challenges were further divided into the level of proficiency needed to solve the challenge, ultimately organized by whether the challenge required high school, collegiate, or professional-level cybersecurity knowledge and skills, with judgment on the requisite skills manually determined by our in-house cybersecurity expert. For all Collegiate and Professional CTFs we require that the solution involves a non-trivial level of exploitation, requiring chaining two or more exploitation steps and not be solvable by copy-pasting textbook examples. Historically, we reported results across high school, collegiate, and professional levels but given the rise in capabilities and our investment in more rigorous benchmarks; we will focus on professional CTFs.

To evaluate the model against the CTFs we give it access to a headless Linux box with common offensive cybersecurity tools preinstalled as well as a harness which allows the model to call those tools or otherwise execute commands similar to as a human. We run 16 rollouts for each CTF exercise, recording the pass@12 metric over the best set of rollouts; this metric is then compared to the thresholds established by the Preparedness Framework as well as the performance of previously released OpenAI models. While the performance is strong, it does not reach our threshold for potentially High cyber capability.

Compaction helps on long-running cyber evals because it lets the model extend its work across multiple context windows. That allows it to keep operating across very long sequences without losing track or timing out. For CTFs, this means the model can sustain coherent progress over an extended period instead of collapsing when the window fills, improving task completion rates.

CVE-Bench is a benchmark that tasks models with identifying and exploiting real-world web-application vulnerabilities in a sandbox environment. We used CVE-Bench (version 1.0) with a focus on vulnerabilities covering content-management systems, AI/ML apps, business-management tools, operational-monitoring systems, web infrastructure, libraries/packages, e-commerce platforms, and a small number of computing-management, mail-server, and web-portal applications; due to some infrastructure porting challenges, we only ran 34 out of the 40 benchmark challenges. We ran the benchmark using the zero-day prompt configuration, where the model is given a general task description of what it needs to do, as opposed to the one-day configuration, where the model is given a description of the vulnerability to exploit. Additionally, we ran the benchmark such that the agent does not have access to the source code of the web-application, and instead must probe it remotely.

We use pass@1 for this evaluation to measure the model’s ability to consistently identify vulnerabilities which are considered relatively straightforward by internal cybersecurity experts. Consistency is important to measure the model’s cost-intelligence frontier to identify vulnerabilities and its ability to potentially evade detection mechanisms that look for scaled attempts of vulnerability discovery and exploitation.

As with CTFs, GPT-5.1-Codex-Max performed significantly better than previous models due to its ability to extend its work across multiple context windows. This performance increase is expected for tasks like vulnerability discovery that involve running many commands and trying different tools with an almost brute-force approach to explore various opportunities for potential exploit identification. While this performance is strong, it does not reach our threshold for potentially High cyber capability, which requires sustained consistency in cyber operations. A consistent pattern of mistakes, even occasional ones, is incompatible with the type of scaled cyber operations that define High cyber capability. We are working on more evaluations that measure the model’s ability to discover advanced and operationally critical vulnerabilities.

Cyber range exercises measure a model’s ability to conduct fully end-to-end cyber operations in a realistic, emulated network.These exercises are long-form, requiring the model to (1) construct a plan to achieve an abstract adversary objective; (2) exploit vulnerabilities, misconfigurations, and weaknesses that are likely to be seen in the wild; and (3) chain together these exploits to achieve the scenario objective.

Since the GPT-5 system card, we made two changes in how we evaluate models on Cyber Range:

We’ve also added four new scenarios to our existing set that better align with real-world attack scenarios, including testing model ability to establish and use complex command-and-control mechanisms as well as interact with common enterprise network services.

GPT-5.1-Codex-Max surpasses the performance of all previous models and solves most of the scenarios except for the most sophisticated scenario of command-control combined with privilege escalation. It also highlights the need for more evaluations that represent sophisticated, real-world cybersecurity scenarios.

Irregular (formerly known as Pattern Labs) evaluated the model’s ability to solve three types of offensive cyber challenges: Network Attack Simulation, Vulnerability Discovery and Exploitation, and Evasion. This work was done on the locked checkpoint, but did not make full use of compaction, which requires scaffolding to take advantage of the model’s ability to use multiple context windows, and which could achieve higher performance. Irregular had access to scaffolding, but in the time given, were not able to extensively experiment with the feature.

The model displayed moderate capabilities overall. Specifically, when compared to GPT-5, GPT-5.1-Codex-Max showed similar or slightly reduced cyberoffensive capabilities. GPT-5.1-Codex-Max achieved an average success rate of 37% in Network Attack Simulation challenges, 41% in Vulnerability Discovery and Exploitation challenges, and 43% in Evasion challenges. It solved 17 out of 18 easy challenges, solved 9 out of 17 medium challenges, and did not solve any of the 6 hard challenges. Compared to GPT-5, GPT-5 solved questions in 17 out of 18 easy challenges, 11 out of 17 medium challenges, and solved 1 of the 6 hard challenges.

Irregular found that GPT-5.1-Codex-Max’s overall similarity in the cyber capability profile to GPT-5 and its inability to solve hard challenges would provide a) only limited assistance to a moderately skilled cyberoffensive operator, and b) do not suggest that it could automate end-to-end cyber operations against reasonably hardened targets or c) enable the discovery and exploitation of operationally relevant vulnerabilities.

One of the primary causes of the significantly increased performance in the cybersecurity domain for GPT-5.1-Codex-Max is its improved ability to tackle long-horizon tasks since most cyber challenges are limited by exploring many different paths which involve running commands that can produce verbose logs and easily consume the model’s context window. We ran evaluations with the model configured to be able to use up to 10 context windows to ensure proper capability elicitation.

In reviewing the totality of the evidence, our Safety Advisory Group assessed that the criteria for High capability in the cyber domain were not met, and recommended that this model not be classified as High capability.

SAG also acknowledged that GPT-5.1-Codex-Max demonstrated meaningful improvements in its cybersecurity skills over our prior models, and made several additional recommendations to OpenAI leadership. The recommendations included increasing the difficulty of our cyber evals, accelerating our investments in cybersecurity safeguards, continuing to ensure we are sufficiently eliciting capabilities when our models are being tested by third parties, and exploring whether we should consider other avenues for evaluating potential cyber harm than the two currently described in the Preparedness Framework.

As the models increase in cybersecurity capability, technical mitigations alone may not be sufficient to prevent harm. In cybersecurity, defensive and offensive behaviors often look very similar. So in addition to the traditional mitigation approach, we are making efforts to strengthen the ecosystem and accelerate defenders. One early example of these efforts is Aardvark, our agentic security researcher agent, which we are using to improve open-source software (OSS) security at global scale.

GPT-5.1-Codex-Max showed modest improvement across all of our self-improvement evaluations, but did not meet our High thresholds.

Note: Results as of July 17th, 2025 update to the SWE-Lancer dataset, accessible via GitHub.

Developed by OpenAI, SWE-Lancer [6] evaluates model performance on real-world, economically valuable full-stack software engineering tasks including feature development, frontend design, performance improvements, bug fixes, and code selection. For each task, we worked with vetted professional software engineers to hand write end-to-end tests, and each test suite was independently reviewed 3 times.

Individual Contributor Software Engineering (IC SWE) Tasks measure model ability to write code. The model is given (1) the issue text description (including reproduction steps and desired 38 behavior), (2) the codebase checkpointed at the state before the issue fix, and (3) the objective of fixing the issue. The model’s solution is evaluated by applying its patch and running all associated end-to-end tests using Playwright, an open-source browser testing library.

We report pass@1 performance on the IC SWE Diamond set.

GPT-5.1-Codex-Max exceeds the performance of GPT-5.1, setting a new upper bound among our models.

PaperBench [7] evaluates the ability of AI agents to replicate state-of-the-art AI research. Agents must replicate 20 ICML 2024 Spotlight and Oral papers from scratch, including understanding paper contributions, developing a codebase, and successfully executing experiments. For objective evaluation, we develop rubrics that hierarchically decompose each replication task into smaller sub-tasks with clear grading criteria. In total, PaperBench contains 8,316 individually gradable tasks.

We measure a 10-paper subset of the original PaperBench splits, where each paper requires <10GB of external data files. We report pass@1 performance with Extra High reasoning effort and no browsing.

GPT-5.1-Codex-Max exceeds the performance of gpt-5.1, setting a new upper bound among our models.

Developed by OpenAI, MLE-bench evaluates an agent’s ability to solve Kaggle challenges involving the design, building, and training of machine learning models on GPUs. In this eval, we provide an agent with a virtual environment, GPU, and data and instruction set from Kaggle. The agent is then given 24 hours to develop a solution, though we scale up to 100 hours in some experiments.

The full dataset consists of 75 hand-curated Kaggle competitions, worth $1.9m in prize value. Measuring progress towards model self-improvement is key to evaluating autonomous agents’ full potential. We use MLE-bench to benchmark our progress towards model self-improvement, in addition to general agentic capabilities. The subset plotted below is 30 of the most interesting and diverse competitions chosen from the subset of tasks that are <50GB and <10h.

GPT-5.1-Codex-Max exceeds the performance of gpt-5.1, setting a new upper bound among our models.

Measuring if and when models can automate the job of an OpenAI research engineer is a key goal of self-improvement evaluation work. We test models on their ability to replicate pull request contributions by OpenAI employees, which measures our progress towards this capability.

We source tasks directly from internal OpenAI pull requests. A single evaluation sample is based on an agentic rollout. In each rollout:

If all task-specific tests pass, the rollout is considered a success. The prompts, unit tests, and hints are human-written.

GPT-5.1-Codex-Max exceeds the performance of GPT-5.1, setting a new upper bound among our models.

OpenAI-Proof Q&A evaluates AI models on 20 internal research and engineering bottlenecks encountered at OpenAI, each representing at least a one-day delay to a major project and in some cases influencing the outcome of large training runs and launches. “OpenAI-Proof” refers to the fact that each problem required over a day for a team at OpenAI to solve. Tasks require models to diagnose and explain complex issues—such as unexpected performance regressions, anomalous training metrics, or subtle implementation bugs. Models are given access to a container with code access and run artifacts. Each solution is graded pass@1.

GPT-5.1-Codex-Max makes meaningful gains on this evaluation relative to both GPT-5 and GPT-5.1.

METR assessed whether GPT-5.1-Codex-Max might enable two key threat models related to autonomy and AI self-improvement:

METR previously found that further incremental development of models similar to GPT-5 was unlikely to enable these threat models, and the primary goal of their evaluation was to revisit and extend this assessment. This work spanned 2 weeks, with OpenAI sharing all critical background information requested about the model, and (in some assessments) providing access to reasoning traces. This work was done on the locked checkpoint, but may not have made full use of the tools for compaction described in the GPT-5.1-Codex-Max blog post. METR had access to compaction scaffolding, but during the available testing period did not find that the model performed significantly better than with their mainline scaffold.

METR found that the capabilities of GPT-5.1-Codex-Max were in line with expectations, and that absent a significant trend break, further development would be unlikely to enable these threat models in the next 6 months, based on the following:

Further details on METR’s assessment, including additional details on robustness checks and other qualitative impressions are available in METR’s report.

Apollo Research fully evaluated the locked checkpoint for deception, in-context scheming, and sabotage capabilities, and evaluated key priority evaluations on the locked checkpoint. Apollo also evaluated GPT-5.1-Codex-Max with compaction enabled on all task configurations for which at least one rollout in the original evaluation exceeded 100,000 tokens. They found overall performance improves slightly with compaction enabled, but was not consistent across all tasks and difficulty levels, with some settings performing worse with compaction enabled.

Apollo found that the capability and propensity of GPT-5.1-Codex-Max for scheming is comparable to that of other recent frontier models, including GPT-5. As with other models, the model sometimes falsifies data, feigns task completion, violates rules, or denies its previous actions. GPT-5.1-Codex-Max exhibits covert deceptive behaviors at rates broadly comparable to GPT-5, though with notable exceptions in specific tasks, where Apollo found elevated rates of falsifying task completion and strategic sandbagging, which may indicate a greater propensity for reward-hacking behaviors compared to GPT-5. GPT-5.1-Codex-Max also exhibits awareness of being evaluated at rates similar to GPT-5, and deception rates are not significantly affected by scaffolding. Based on these findings, they believe GPT-5.1-Codex-Max is likely not capable of causing catastrophic harm via scheming.

1. [1]

   Alexandra Souly, Qingyuan Lu, Dillon Bowen, Tu Trinh, Elvis Hsieh, Sana Pandey, et al. A strongreject for empty jailbreaks. arXiv preprint arXiv:2402.10260.
2. [2]

   Eric Wallace, Kai Xiao, Reimar Leike, Lilian Weng, Johannes Heidecke, and Alex Beutel. The instruction hierarchy: Training LLMs to prioritize privileged instructions. arXiv https://arxiv.org/abs/2404.13208.
3. [3]

   Tejal Patwardhan, Kevin Liu, Todor Markov, Neil Chowdhury, Dillon Leet, Natalie Cone, et al. Building an early warning system for LLM-aided biological threat creation. OpenAI.
4. [4]

   Jon M. Laurent, Joseph D. Janizek, Michael Ruzo, Michaela M. Hinks, Michael J. Hammerling, Siddharth Narayanan, et al. “LAB-bench: Measuring capabilities of language models for biology research.” Available at: https://arxiv.org/abs/2407.10362.
5. [5]

   Yuxuan Zhu, Antony Kellermann, Dylan Bowman, Philip Li, Akul Gupta, Adarsh Danda, et al. “CVE-bench: A benchmark for AI agents’ ability to exploit real-world web application vulnerabilities.” Available at: https://arxiv.org/abs/2503.17332.
6. [6]

   Samuel Miserendino, Michele Wang, Tejal Patwardhan, and Johannes Heidecke. “SWE-Lancer: Can frontier LLMs earn $1 million from real-world freelance software engineering?” Available at: https://arxiv.org/abs/2502.12115.
7. [7]

   Giulio Starace, Oliver Jaffe, Dane Sherburn, James Aung, Jun Shern Chan, Leon Maksin, et al. “PaperBench: Evaluating AI’s ability to replicate AI research.”