OpenAI's latest AI models achieved 98 percent accuracy in migrating aging COBOL code to modern programming languages at high reasoning effort, according to a blog post published October 19th. The company touted what it called "substantial improvements" in translating software written in older languages like COBOL into contemporary code, a capability it claims could assist organizations in maintaining and updating older systems. COBOL remains deeply woven into critical infrastructure worldwide, powering everything from IRS systems to banking networks despite being nearly seven decades old.
The numbers highlight COBOL's stubborn grip on essential services. IBM reports that COBOL forms the backbone for more than 40 percent of online banking platforms, supports 80 percent of credit card transactions made in person, and processes 95 percent of ATM transactions, according to the report. It also props up numerous government operations, from Social Security to the IRS to services operated by most states. One rough calculation suggests COBOL's inefficiencies cost the US economy roughly $105 billion each year. In OpenAI's testing, GPT-6.1 Sol hit 87 percent behavior preservation accuracy at low reasoning effort, rising to 98 percent at higher reasoning levels, while GPT-6 Astra scored around 86 percent at low reasoning, GPT-6 Sol just 21 percent, and GPT-5.6 Sol a mere 5 percent.
OpenAI wrote that "these evaluation results demonstrate progress toward more accurate, affordable code migration for teams modernizing the software their businesses depend on," noting that a migration counts as successful only when all tests pass. The company also reported gains in its models' capacity to construct database software across various programming languages, including COBOL itself. Anthropic entered the conversation in February by releasing a step-by-step "code modernization playbook" and positioning its coding agent, Claude Code, as a tool for moving COBOL into modern languages like Java or Python, though it offered no performance benchmarks showing how effectively Claude Code could handle such migrations.
Yet experts who spoke to The Verge say the advances aren't close to enabling safe replacement of critical legacy systems. Vadim Zaytsev, an associate professor of software evolution at the University of Twente in the Netherlands, explained that "rewriting code is not the same as replacing a system," emphasizing that systems involve not just code but also data in various formats, technical integrations with other platforms, and the expectations of people and organizations. Arie van Deursen, a software engineering professor at Delft University of Technology, called OpenAI's blog "superficial" and said the evaluations don't tackle the central challenge: breaking up a complex legacy system into smaller pieces that can be swapped out one at a time. Alastair Donaldson, a professor of programming languages at Imperial College London, stressed that robust testing is crucial, noting that very few people know how to program in languages like COBOL anymore and "nobody in their right mind would use COBOL for a new development project."
OpenAI spokesperson Oscar Haines acknowledged that legacy code "is still far from a solved problem" and said the company is listening to developer and enterprise feedback while exploring ways to build better tests and keep improving its models' grasp of coding languages. Zaytsev warned that when systems are as critical as those running on COBOL, even minor differences in behavior before and after migration can be unacceptable, adding that while AI can deliver savings and speed things up, partial automation isn't sufficient given the complexity involved. He illustrated the point starkly: "Suppose you have a magic AI wand that handles 99 percent of your code. What do you do with the remaining 1 percent? That 1 percent can be a million lines of code, spread all over the place and subtly failing in more ways you can imagine within your lifetime." The tools may be improving, but the gap between laboratory benchmarks and production reliability remains wide enough that organizations betting their operations on automated migration would be taking on risk that many simply can't afford.