New research from OpenAI reveals that the GPT-6.1 Sol model achieves up to 97.9% accuracy in translating legacy COBOL software into modern languages under the MultiLangBench benchmark.
New research to measure performance in COBOL, Fortran, and Visual Basic languages
MultiLangBench benchmark and the superiority of the GPT-6 Astra model across 20 languages
Translating COBOL systems to modern languages with up to 97.9% accuracy
Availability of tests on GitHub and application limits in actual banks
Frequently asked questions
New research to measure performance in COBOL, Fortran, and Visual Basic languages
On October 19, 2026, OpenAI published a new research study examining the performance of its latest language models when working with programming languages that are underrepresented in typical AI training data, led by COBOL, Fortran, Visual Basic, and OCaml. These languages retain critical importance in banking systems, scientific computing, and legacy enterprise software despite receiving less attention compared to modern development languages.
The research addresses two interrelated problems: the first involves writing entirely new software in various programming languages, while the second focuses on converting existing legacy programs into modern programming languages without altering their intended functional behavior.
MultiLangBench benchmark and the superiority of the GPT-6 Astra model across 20 languages
The company named the first evaluation MultiLangBench, a test that asks AI models to build software compatible with the SQLite database engine across 20 different programming languages. This test requires much more than generating short code snippets; the model must organize the entire program, manage data, handle errors, and execute functions that produce the expected database outputs.
At the maximum reasoning effort level, the GPT-6 Astra model achieved an average test pass rate of 87.4%, compared to just 26.8% for the GPT-5.6 Sol model. Although performance improved across all 20 languages, results varied clearly; specifically in COBOL, the success rate jumped from 10.9% to 72.1%, while COBOL remained the most difficult language in the evaluation.
Translating COBOL systems to modern languages with up to 97.9% accuracy
The second evaluation focused on software migration and modernization, where OpenAI tested its models’ ability to translate programs written in COBOL into modern languages including C++, Java, Kotlin, Python, and Rust while fully preserving their original behavior. The danger of this task lies in the fact that the translated program may run without technical errors yet produce incorrect financial results due to subtle differences in number rounding or banking calculations.
The company reported that the GPT-6.1 Sol model achieved 86.6% accuracy at low reasoning effort compared to just 5.4% for the GPT-5.6 Sol model, while GPT-6.1 Sol’s accuracy at higher reasoning effort rose to 97.9%.
Availability of tests on GitHub and application limits in actual banks
OpenAI announced the release of the MultiLangBench evaluation environment and its test suite via GitHub to enable independent researchers to examine the methodology and reproduce or critique the results. Although these developments may help organizations maintain systems written decades ago, the research reading emphasizes that the results come from the company’s own evaluation framework and do not constitute independent proof of the feasibility of migrating a bank’s full production systems without expert human review that accounts for dependencies, business rules, and security.
OpenAI’s research on underrepresented programming languages like COBOL, Fortran, and Visual Basic gains extreme economic and technical importance for the banking, financial, and government sectors, as trillions of dollars in global financial transactions still pass daily through systems written in COBOL decades ago at a time when the number of expert programmers capable of maintaining or updating those systems is declining.
The rise in the success rate on the MultiLangBench benchmark to 87.4% for the GPT-6 Astra model and the achievement of 97.9% COBOL program translation accuracy in the GPT-6.1 Sol model offer a promising horizon for accelerating digital modernization projects, provided these tools are coupled with precise engineering and financial reviews to ensure no deviation occurs in banking calculations or critical business rules.
Frequently asked questions
Question: What programming languages did OpenAI’s new research focus on?
Answer: The research focused on languages underrepresented in training data that are vital to enterprises, such as COBOL, Fortran, Visual Basic, and OCaml, within an evaluation covering 20 languages.
Question: What result did the GPT-6 Astra model achieve in the MultiLangBench benchmark?
Answer: The model achieved an average pass rate of 87.4% across 20 programming languages compared to 26.8% for the GPT-5.6 Sol generation, and raised the success rate in COBOL from 10.9% to 72.1%.
Question: How did the GPT-6.1 Sol model perform in translating COBOL programs to modern languages?
Answer: The model achieved 86.6% accuracy at low reasoning effort and reached 97.9% accuracy when raising the reasoning effort level.
Question: Do these results mean bank systems can be updated automatically without human review?
Answer: No, the results pertain to specific testing benchmarks, whereas production banking systems require rigorous expert review to ensure the accuracy of accounting rules, security, and dependencies.
