MolecularIQ, arXiv:2601.15279
Benchmark and methodology
On structure questions about molecules, such as counting atoms, identifying rings and scaffolds or assigning stereocentres, 14 to 41 % of answers are correct without CovaSyn and 76 to 92 % with CovaSyn. Measured on MolecularIQ, an independent benchmark (Bartmann et al., ICLR 2026). The numbers apply to this task type; the benchmark does not measure the interpretation of spectra, stability or toxicology data. This page sets out the numbers, the methodology and the limits of the measurement.
Create an account, get your API key, use the tools in your usual working environment.
The same tasks, once without and once with CovaSyn.
4 test runs, 3,540 verified tasks, 12,540 responses. Only an exact match with the stored solution counts.
- Correct without CovaSyn
- 14 to 41%
- Correct with CovaSyn
- 76 to 92%
Source: Bartmann C., Schimunek J., Ielanskyi M., Seidl P., Klambauer G., Luukkonen S. (2026). MolecularIQ: Characterizing Chemical Reasoning Capabilities Through Symbolic Verification on Molecular Graphs. ICLR 2026. Preprint: arXiv:2601.15279. Snapshot: 2026-05-17.
The effect shows up in every test run.
“Without CovaSyn” is the accuracy without calculation tools, “With CovaSyn” the same tasks with the CovaSyn tools. One test run covers the full test split, the other three a proportionally stratified subset.
| Test run | Without CovaSyn | With CovaSyn | Difference (percentage points) | Factor |
|---|---|---|---|---|
| Run 1 | 21.18% | 85.38% | +64.20 | 4.03× |
| Run 2 | 40.75% | 91.51% | +50.76 | 2.25× |
| Run 3 | 22.29% | 89.92% | +67.63 | 4.03× |
| Run 4 | 13.68% | 75.66% | +61.98 | 5.53× |
Source: Bartmann C., Schimunek J., Ielanskyi M., Seidl P., Klambauer G., Luukkonen S. (2026). MolecularIQ: Characterizing Chemical Reasoning Capabilities Through Symbolic Verification on Molecular Graphs. ICLR 2026. Preprint: arXiv:2601.15279. Snapshot: 2026-05-17.
In all four test runs, accuracy rises by more than 50 percentage points with CovaSyn. So the effect is not tied to one working environment; it comes from the exact calculations.
What was measured, and how.
Benchmarks a vendor designs for itself deserve scepticism: whoever tests their own tools has an incentive to look good. That is why we measure on a benchmark we did not design and whose task selection we had no influence over.
- Independent benchmark
- MolecularIQ was created by Bartmann et al. (Klambauer Lab, JKU Linz) and published at ICLR 2026, a peer-reviewed conference (preprint: arXiv:2601.15279). CovaSyn neither designed nor contributed to the benchmark; we measured our tools against it.
- Symbolic verification
- Every answer is verified symbolically against the stored solution and only counts on an exact match. There is no partial credit and no discretionary grading. That leaves no room to flatter the result.
- Tasks
- We evaluated the test split of MolecularIQ with 3,540 tasks, restricted to question types that can be verified symbolically.
- Scope
- 4 test runs in the common working environments of three providers, each without and with CovaSyn. 12,540 answers evaluated in total (one without and one with CovaSyn per task and run); one run on the full test split, the other three on a stratified sample of 910 questions per run.
- Tools
- Chemistry primitives from the CovaBasicChem suite: deterministic cheminformatics that returns exact values. Same input, same result.
- Snapshot and reproducibility
- Data snapshot of 2026-05-17. Dataset and evaluation code are public; with a CovaSyn account every number on this page can be recomputed.
Paper, data, code
- Paper on arXiv (abstract)https://arxiv.org/abs/2601.15279
- Paper PDF on arXivhttps://arxiv.org/pdf/2601.15279
- Dataset on Hugging Facehttps://huggingface.co/datasets/ml-jku/moleculariq-v0.0
- Evaluation code on GitHubhttps://github.com/ml-jku/moleculariq
The PDF is hosted by arXiv, not on our servers. Dataset and code belong to the benchmark's authors.
Citation
If you use the numbers on this page in a publication, a poster or an internal report, please cite the authors' MolecularIQ paper:
@inproceedings{bartmann2026moleculariq,
title = {MolecularIQ: Characterizing Chemical Reasoning Capabilities
Through Symbolic Verification on Molecular Graphs},
author = {Bartmann, C. and Schimunek, J. and Ielanskyi, M. and
Seidl, P. and Klambauer, G. and Luukkonen, S.},
booktitle = {The Fourteenth International Conference on Learning
Representations (ICLR)},
year = {2026},
url = {https://openreview.net/forum?id=RqwEzZqMFv}
}Bartmann C., Schimunek J., Ielanskyi M., Seidl P., Klambauer G., Luukkonen S. (2026). MolecularIQ: Characterizing Chemical Reasoning Capabilities Through Symbolic Verification on Molecular Graphs. ICLR 2026. Preprint: arXiv:2601.15279.
Where CovaSyn has the strongest effect.
Mean lift per question category in percentage points, averaged over three of the four test runs. The current snapshot has no per-category values for the fourth run.
| Category | Without CovaSyn | With CovaSyn | Difference (percentage points) |
|---|---|---|---|
| Scaffold and fragments | 18.0% | 86.5% | +68.4 |
| Rings and topology | 29.4% | 93.2% | +63.8 |
| Bonds and chains | 17.6% | 80.9% | +63.3 |
| Multi-feature questions | 27.3% | 88.4% | +61.1 |
| Atom and formula counts | 38.7% | 98.3% | +59.7 |
| Stereochemistry | 28.7% | 86.0% | +57.4 |
| Electronics and H-bonds | 31.2% | 81.5% | +50.3 |
These are the building blocks of daily medicinal chemistry. A scaffold has to be identified exactly, a ring analysis must not drift into guessing, and a multi-constraint query such as “all branch points within two bonds of a halogen” has to resolve exactly.
Where the remaining gap sits.
Accuracy is not 100%, and we do not hide that. We assigned every wrong answer with CovaSyn to one of three causes. This is how the wrong answers break down per test run, and where it pays to look closer in your own validation.
| Cause | Run 1 | Run 2 | Run 3 | Run 4 |
|---|---|---|---|---|
| Tool result overridden | 81% | 86% | 66% | 25% |
| Format error | 1% | 1% | 25% | 73% |
| Tool value off | 18% | 13% | 9% | 3% |
- Tool result overridden
- A correct tool result is overridden in the answer. This depends on the working environment, not the tool. A clearer question structure reduces it noticeably but does not remove it entirely.
- Tool value off
- The tool value does not fit the question. These are real workflow errors, mostly edge cases with complex stereochemistry or unusual structures. We work on them continuously.
- Format error
- The answer cannot be parsed. This class is technically easy to catch because an output check surfaces it immediately.
At most 18 % of a test run's wrong answers come from a tool value that does not fit. The rest arises in the working environment: a correct result is overridden or the answer cannot be parsed. Both can be caught in the workflow.
What this means for research and development.
- Less rework.
- When accuracy rises from 14 to 41 % to 76 to 92 %, the share of results that must be recalculated by hand falls by more than half.
- The lift is reproducible.
- All four test runs point the same way. The tools compute deterministically: same input, same result.
- The gap is disclosed.
- For validation in regulated environments an open error distribution is worth more than a polished promise. It shows which failure types your QA team should test specifically when validating its own use case.
Frequently asked questions
What is the MolecularIQ benchmark?
A benchmark published in 2026 by the Klambauer Lab, JKU Linz (arXiv:2601.15279) that measures how reliably structured chemistry analysis is solved. 3,540 tasks in the evaluated test split, symbolic verification without discretionary grading.
Why does accuracy rise so much with CovaSyn?
Without a calculation tool, structural chemistry tasks such as atom counts or scaffold extraction are often estimated. CovaSyn computes them exactly with verified methods. An estimate becomes a calculation.
Can the result be reproduced?
Yes. The dataset is public on Hugging Face, the evaluation code is on GitHub, and with a CovaSyn account you can recompute every number.
What about the other CovaSyn tools?
The numbers on this page come from one part of our tools. We are measuring the others against matching datasets. Once results are in, they will appear on this page, with full methodology and the full distribution.
Research and development
The development of the CovaSyn MCP platform is certified as research and development under the German Research Allowance Act by the certification body BSFZ.
The certificate assesses the research character of the development work. It is not a statement about product properties and not a state product approval. Bescheinigungsstelle Forschungszulage.
Next step
Measure it yourself.
Test what is behind these numbers on your own molecule.
Create an account, get your API key, use the tools in your usual working environment.

