How Coding Agents Are Transforming Scientific Software: 8 OpenAI Case Studies

Coding agents are moving beyond app development and into scientific infrastructure

On July 28, 2026, OpenAI published “Scientific computing in the age of agentic AI,” an exploratory field report covering eight scientific-software projects completed with coding-agent assistance. Five projects used Codex alone, while three combined Codex and Claude Code. Most of the case studies come from the life sciences, including genomics, RNA sequencing, immunology, sequencing-data simulation, and Bayesian statistics.

The important point is not that an AI system added a few convenience functions or fixed a syntax error. The work ranges from maintaining build infrastructure and making local optimizations while preserving every output byte, to migrating a deep-learning backend from TensorFlow/Keras to PyTorch, reimplementing more than 20,000 lines of C/C++ in Rust, consolidating fifteen quality-control tools into a single-pass pipeline, and designing a new GPU-native genomics editing system.

OpenAI’s report suggests that coding agents can reduce the engineering burden that often traps small research teams in packaging, testing, maintenance, optimization, and technology migrations. The report also draws a firm boundary around that benefit. An agent can help execute an ambitious project, but it cannot reliably decide which scientific question is worth pursuing, which evidence proves the result is valid, or who should remain responsible for the software years after the initial implementation.

AI therefore does not remove the scientist from the process. It shifts the scientist’s role from writing every line personally to defining goals, designing validation, orchestrating the work, interpreting discrepancies, and taking responsibility for scientific correctness.

Benchmark note: the performance results discussed here come from the field report and the contributing project teams. OpenAI standardized the presentation and checked internal consistency, but it did not independently rerun every benchmark. The numbers should be read as results for specific configurations, datasets, and acceptance criteria rather than universal performance guarantees.

Why scientific software is a strong candidate for coding-agent assistance

Many scientific tools begin as code accompanying a paper. They are commonly created by small academic teams whose primary incentive is to demonstrate a method, not to build and operate a software product for a decade. If the tool becomes widely used, its original authors inherit a growing set of engineering obligations: supporting new operating systems, updating dependencies, repairing continuous-integration pipelines, maintaining packaging, handling larger datasets, optimizing CPU or GPU execution, preserving backward compatibility, and diagnosing failures across many downstream workflows.

This work can be highly valuable without producing a new paper. It consumes the time of researchers while not directly advancing the scientific question that motivated the software. Coding agents fit this gap because they can inspect large repositories, follow dependency chains, modify multiple files, run tests and benchmarks, prepare documentation, and iterate after feedback.

Scientific software nevertheless differs from an ordinary application in one critical way: running successfully does not mean being scientifically correct. A change may make a program faster, pass a conventional test suite, and avoid crashes while quietly altering a probability distribution, read ordering, mutation frequency, alignment result, or statistical behavior. Agentic engineering becomes valuable only when the research team has an external reference and a sufficiently strong acceptance contract.

A map of the eight projects, from maintenance to a new system

The field report places the projects along a broad spectrum of intervention:

1. Maintenance: cyvcf2 modernized its build, test, and release infrastructure.
2. Local optimization: HI.SIM improved runtime under a byte-identical output requirement.
3. Targeted heuristic optimization: hifiasm optimized hot paths in genome assembly.
4. Compatibility migration: MHCflurry moved from TensorFlow/Keras to PyTorch while retaining compatibility with released models.
5. Faithful reimplementation: bayesm and rustar-aligner were reimplemented in Rust with scientific behavior as the target.
6. Workflow redesign: RustQC consolidated many quality-control steps into a single processing pass.
7. New system: HelixForge redesigned a genomics workflow around GPU-native execution.

This range shows that a coding agent is not a single-mode tool. Moving toward a wholly new system can unlock larger performance gains, but it also expands the validation surface, the risk of hidden divergence, and the long-term stewardship burden.

1. cyvcf2: modernizing the build system around a mature genomics library

cyvcf2 is a Python library for reading and writing VCF files, a standard format for representing genetic variants. After roughly a decade of changes to Python, dependencies, packaging practices, and release infrastructure, the project’s build and test process had become increasingly difficult to maintain.

In this case study, GPT-5.5 did not change the scientific logic used to process VCF data. The work focused on infrastructure: updating package metadata and dependencies, modernizing continuous-integration jobs, standardizing release workflows, and introducing scikit-build-core into the build process.

The example matters because much scientific technical debt lives around an algorithm rather than inside it. A scientifically sound library that cannot be installed on a current platform is still unusable for many researchers. If an agent can handle repetitive maintenance, maintainers gain time for scientific bugs, documentation, user support, and method development.

The refactor was merged upstream into cyvcf2. That outcome represents a healthy stewardship pattern: the work strengthens the established project and its release process instead of creating another fork that divides users and maintainer attention.

2. HI.SIM: reducing runtime while preserving byte-identical output

HI.SIM is a simulator for PacBio HiFi sequencing data. The task was intentionally conservative: make the program faster without changing the structure, background statistics, or any byte of its output.

The agent identified a group of local improvements, including removing repeated floating-point division from hot loops, reducing unnecessary allocation and copying, and batching small writes into larger buffers. This was not a dramatic algorithmic rewrite. Its value came from finding several narrow bottlenecks in specialized code that a human maintainer could easily overlook.

Across four benchmark workloads, the combined runtime reduction was reported as 30.97%. The workloads included a synthetic 50 kbp genome at 1,000× coverage, a synthetic 2 Mbp genome at 120×, E. coli K-12 MG1655 at 80×, and S. cerevisiae S288C at 50×. Output was confirmed to be byte-identical.

This is a strong early automation pattern. When acceptance is a byte-for-byte comparison, the evaluation space is unusually clear. The agent can propose and benchmark optimizations, while the regression system acts as an independent referee. The scientific risk is constrained by a precise external reference.

3. hifiasm: optimizing genome assembly and testing beyond the development benchmark

hifiasm is a widely used genome assembler for PacBio HiFi reads. Runtime is concentrated in steps such as read correction, edit-distance computation, trace generation, and overlap chaining. The project asked the agent to reduce runtime while still meeting predefined proxy thresholds for assembly quality and ordering.

The changes did not replace the entire algorithm. They focused on hot paths in the existing C implementation, reduced repeated work, improved memory access, and added fast paths for common cases.

On a held-out synthetic 200 Mb dataset, the best optimized version reduced runtime by 25.1% while meeting the required quality thresholds. When evaluated on real human chromosome 20 data from the Human Pangenome Project, the reduction was 14.7%.

The gap between 25.1% and 14.7% is instructive. An optimization that looks excellent on a small or synthetic workload may transfer only partially to real data. The agent should not be allowed to optimize and simultaneously select the benchmark that presents its own change most favorably. Teams need separate development data, intermediate benchmarks, held-out evaluation data, and a final test on the real distribution they expect users to process.

4. MHCflurry: migrating nearly 10,000 lines from TensorFlow/Keras to PyTorch

MHCflurry is an open-source model that predicts which protein fragments are likely to be presented to T cells, supporting immunology and cancer research. Its older neural-network backend depended on an aging TensorFlow/Keras stack that was becoming difficult to maintain on modern systems.

The objective was not merely to rewrite the implementation in PyTorch. The migration had to satisfy three difficult conditions at once:

• replace model definitions, tensor operations, and training code;
• load previously released weights without forcing the community to retrain models;
• keep predictions within a small, predefined tolerance.

Coding agents participated in a migration spanning nearly 10,000 lines across about 130 files. The result was released in MHCflurry 2.2.0. Existing weights could be loaded without a manual conversion, while predictions agreed within the required tolerance.

The project shows that agents can support a codebase-scale migration, but writing speed was not the decisive factor. Success depended on compatibility requirements defined in advance: old models had to load, prediction drift had to remain minimal, tests had to cover both inference and training, and the project’s maintainers had to accept responsibility for the upstream change.

5. bayesm: a Rust rewrite and the danger of statistical code that only looks correct

bayesm is a Bayesian statistics library. Reimplementing such a system is especially sensitive because an error does not need to produce a crash or exception. A sampler can run, generate numbers, and produce convincing charts while converging to the wrong posterior distribution.

The base Rust rewrite achieved posterior agreement with the reference implementation and improved runtime across multiple functions. The report describes speedups on several workloads, but the most valuable lesson came from the early failures rather than the fastest benchmark.

Agent-generated extensions for HMC, NUTS, and HART initially looked plausible but contained important defects: an incorrect mass-matrix inverse, faulty trajectory construction, an inefficient draw_delta update, and prior scaling reduced to a fixed constant. A team relying on simple tests or visual plausibility could have accepted a statistically invalid implementation.

Researchers used convergence diagnostics, rank-normalized split R-hat, effective sample size, and simulation-based calibration to identify the problems. After correction, the engines met convergence criteria, and the corrected HART implementation ran about 2.6× faster than the original.

The lesson is direct: the agent that creates statistical code is not a trustworthy judge of that code. Validation must examine distributional behavior and calibration, not merely a handful of expected values.

6. rustar-aligner, svb, and kuva: reproducing the accumulated behavior of legacy software

STAR is an important RNA-seq alignment tool, but its C/C++ codebase of more than 20,000 lines is no longer actively maintained. rustar-aligner aims to provide a more maintainable Rust implementation while closely reproducing STAR’s behavior across reference indexing, alignment and scoring, multimapping, output formats, command-line behavior, and downstream compatibility.

On 10,000 yeast RNA-seq reads, rustar-aligner achieved reported tie-adjusted parity of 99.815% for single-end and 99.883% for paired-end when comparing position, CIGAR, MAPQ, NH tags, and the proper-pair flag. No reads were mapped by only one of the two tools, and the 10,862-entry suffix array was byte-identical.

The team did not treat 99.8% as an automatic completion signal. Some differences came from random choices among equal-scoring multimappers, while others affected downstream workflows. Running rustar-aligner inside the nf-core/rnaseq test framework revealed discrepancies that the local benchmark had missed.

The same project group also produced svb, a pure-Rust StreamVByte library reported to compress or decode faster than a commonly used crate, and kuva, a Rust scientific plotting library and command-line tool with more than sixty plot types and multiple output backends.

Because STAR had been abandoned, rustar-aligner moved under new community stewardship. That institutional outcome matters as much as the benchmark. A rewrite has lasting scientific value only when it has owners, releases, contributors, compatibility policy, and a maintenance plan.

7. RustQC: consolidating fifteen QC steps into a single-pass pipeline

The RustQC case study goes beyond a faithful rewrite. Instead of reproducing each quality-control tool independently, the team redesigned the post-alignment QC workflow used around nf-core/rnaseq so that a large file would not need to be read and written repeatedly by separate programs.

RustQC consolidates fifteen QC tools into a single-pass pipeline. On a dataset containing 186 million reads, the reported sum of sequential task runtimes fell from 15 hours 34 minutes to 14 minutes 54 seconds, a reduction of more than 60×. Disk traffic fell from 2.5 TB to 0.1 TB, while numerical outputs were reported as equivalent under the project’s validation rules.

The group also pursued less disruptive paths: a Trim Galore rewrite at roughly 7×, FastQC-Rust at roughly 3×, and upstream optimization of the original FastQC Java implementation at roughly 3×.

Pursuing redesign, faithful rewrite, and upstream optimization in parallel demonstrates that there is no single modernization strategy for every user. A new pipeline can achieve exceptional performance but changes a larger compatibility surface. A faithful rewrite is easier to substitute component by component. Upstream optimization benefits current users without forcing a migration.

8. HelixForge: a GPU-native design with nearly 60× end-to-end speedup

HelixForge is the case study furthest from maintenance. The system does not simply port code to another language; it redesigns BamSurgeon’s read-editing pipeline around modern hardware.

The older workflow orchestrates BWA-MEM, Picard, and samtools for each mutation, creating several sequential steps and a post-editing realignment stage. HelixForge replaces that flow with a custom htslib and CUDA C++ engine on an NVIDIA H200, edits reads directly on the GPU, and removes the post-edit realignment stage.

For the reported benchmark using one donor and a 10 Mb region:

• the editing stage was 98.6× faster;
• end-to-end runtime was 59.6× faster;
• average mutation-frequency error fell from 0.076 to 0.034;
• realignment artifacts were nearly eliminated.

The redesign improved both speed and a measured quality signal. It remains a case-specific benchmark on H200 hardware, not proof that every dataset, variant class, or GPU will achieve the same gain. A system like HelixForge still needs broader validation, deployment documentation, regression data, hardware-cost analysis, and a clearly responsible owner.

Shared lesson one: validation is becoming the bottleneck

Across the eight projects, agents handled clearly scoped requests effectively. The recurring problem appeared when the system had to decide whether its own output was scientifically valid. The report notes that agents could remain confident even when an implementation contained clear errors.

The strongest projects used an external referee:

• byte-identical output for HI.SIM;
• parity with a reference tool for rustar-aligner;
• legacy weights and prediction tolerances for MHCflurry;
• proxy thresholds and held-out benchmarks for hifiasm;
• convergence diagnostics and simulation-based calibration for bayesm;
• numerical equivalence and I/O measurement for RustQC;
• mutation-frequency error and artifact analysis for HelixForge.

A good prompt is not enough. A serious coding-agent workflow needs an acceptance contract specifying test data, baseline, tolerance, metrics, failure conditions, and the evidence that must be retained. VietBoost’s review of AI-agent isolation after the Hugging Face incident shows why permission, data, and evaluation environments must also remain separated.

Shared lesson two: the last mile often takes the most work

Agents often produce a first implementation quickly. Edge cases, numerical discrepancies, undocumented legacy behavior, output formats, and downstream integration consume far more iterations.

A project should therefore be divided into checkpoints:

1. reproduce the baseline;
2. build reference tests;
3. make the first small change;
4. run an intermediate benchmark;
5. expand the change surface;
6. evaluate held-out data;
7. run downstream workflows;
8. complete scientific review;
9. prepare release and rollback procedures;
10. assign long-term maintainers.

A broad instruction such as “rewrite this entire tool in Rust and make it faster” may create an impressive demo. It does not create trustworthy scientific infrastructure unless the project has evaluation gates between the initial prompt and the release.

Shared lesson three: humans move toward orchestration

Researchers may no longer need to write every change personally, but they must perform work that is harder to substitute:

• choose scientifically valuable questions;
• separate behavior that must be preserved from behavior that may change;
• construct simulations with known ground truth;
• define numerical tolerances;
• detect benchmark overfitting;
• evaluate results on real data;
• decide whether evidence is sufficient for release;
• accept responsibility to the user community.

Coding agents reduce engineering barriers, but they may increase demand for research engineering, test design, benchmark governance, and software stewardship. Teams deploying these systems should also review VietBoost’s guide to taking a coding agent from issue to branch, tests, pull request, and security review.

Long-term stewardship: faster implementation does not guarantee easier maintenance

Lower implementation costs will encourage more forks and rewrites. If every group creates a new version of the same tool without a maintainer, users become fragmented, documentation diverges, and scarce expert attention is spread across many repositories.

The field report presents several paths. MHCflurry and cyvcf2 returned changes to their upstream projects. rustar-aligner established new community stewardship because the original project was not actively maintained. New redesigns need an owner and a credible maintenance plan from the beginning.

Source code is only one part of the asset. Mature scientific software also contains undocumented conventions, compatibility expectations, user trust, bug history, and knowledge about unusual datasets. An agent can translate code without automatically inheriting that social and operational capital.

Permission boundaries, approvals, and audit trails become more important as an agent’s scope grows. VietBoost’s analysis of safe production AI-agent design with MCP, approvals, and audit logs provides a complementary governance framework.

A practical playbook for research and engineering teams

1. Choose tasks that can be verified

Begin with maintenance, packaging, tests, documentation, and local optimization that has a reference output. Do not start with a complete rewrite if the original system lacks a regression suite.

2. Freeze the baseline before the agent changes anything

Record the commit, environment, compiler, dependencies, random seed, data, and hardware. If the baseline cannot be reproduced, any performance claim is weak.

3. Express the acceptance contract in metrics

Avoid “the result looks equivalent.” Specify exact matching, tolerances, posterior behavior, read-order thresholds, latency percentiles, memory use, I/O, and explicit failure cases.

4. Separate development data from evaluation data

An agent may overfit a benchmark included in its working context. Hold back a dataset or workload that is never exposed during optimization.

5. Use an independent reviewer

Do not let one agent write the code, summarize the risk, and declare the result passed. Use CI, scanners, benchmark harnesses, and domain experts as independent validation layers.

6. Test the downstream workflow

Parity in unit tests does not guarantee compatibility with a real pipeline. Run the tool in nf-core, containers, notebooks, or the production workflow used by actual researchers.

7. Measure cost, not only speed

A GPU-native redesign may be faster while requiring expensive hardware, more complex drivers, and new operational expertise. Measure compute cost, energy, I/O, review time, and future maintenance alongside runtime.

8. Assign ownership before release

Define the primary repository, maintainers, contribution policy, release cadence, issue channel, security process, and support period. “The AI finished writing it” is not a responsible end state.

For teams comparing several models or agent roles, see VietBoost’s multi-model AI operating guide for solopreneurs and its review of Claude Opus 5 in GitHub Copilot and long-running agentic coding.

What the report does not prove

OpenAI describes this work as an exploratory field report, not a randomized controlled experiment comparing teams with and without agents. It does not fully answer:

• the total human hours spent prompting, reviewing, and correcting each project;
• token and compute cost per project;
• how strongly outcomes depend on the model, hardware, or contributor experience;
• whether the software remains healthy after one or more years;
• the number of failed or excluded projects;
• how easily third parties can reproduce every benchmark.

These limits do not invalidate the reported outcomes. They prevent the case studies from supporting an exaggerated claim that AI can independently build all scientific software. A better interpretation is that coding agents can substantially expand the engineering capacity of small teams when those teams also possess strong scientific validation standards and credible stewardship. For effort selection and cost control in a smaller workflow, see VietBoost’s practical Claude Sonnet 5 guide for solopreneurs.

Conclusion: AI can reduce pipeline maintenance, but it cannot own scientific judgment

The eight case studies show that coding agents have moved beyond autocomplete. They can modernize build systems, find hot paths, migrate deep-learning frameworks, reproduce C/C++ codebases in Rust, consolidate workflows, and help design GPU-native systems.

The report’s most important message is not the number of lines an agent can produce. It is a change in the allocation of labor. Scientists can spend less time keeping old pipelines alive and more time defining questions, designing validation, inspecting results, and deciding what should be built next.

In science, speed matters only when accompanied by correctness, reproducibility, and long-term responsibility. Coding agents can help teams move faster. Going far still requires expertise, caution, external evaluation, and a person or community willing to stand behind the software after the agent session ends.

Frequently asked questions

Did Codex independently complete all eight projects?

No. Five projects used Codex alone and three combined Codex with Claude Code, but every case involved people who set goals, built or reviewed validation, interpreted discrepancies, and decided whether the result was acceptable.

Does a faster benchmark prove the rewritten scientific software is correct?

No. Performance and correctness are separate gates. Scientific software may run faster while changing statistical or biological behavior. Teams need reference outputs, held-out data, domain-specific metrics, and downstream workflow tests.

Which tasks are safest to automate first?

Maintenance, packaging, documentation, test expansion, and local optimization with exact or tightly bounded reference outputs are the strongest starting points. Whole-system rewrites require a much broader validation and stewardship plan.

What is the central operating lesson for a small team?

Invest in the acceptance contract before investing in a large agent run. Define baseline, datasets, metrics, tolerances, failure conditions, rollback, and ownership first. The agent can then work inside a verifiable boundary.

Official sources

• OpenAI: Scientific computing in the age of agentic AI
• Scientific computing in the age of agentic AI: an exploratory field report (PDF)