Instrument data become reusable when three things are kept together: the unprocessed measurement, the parameters needed to process it, and the context that says what was measured. Most archives keep the first, lose the second in a vendor-specific project file and never record the third. The result is a terabyte of data that only the original software version can open and only the original analyst can interpret, which in practice means the data are kept and not used.
What counts as raw data
For NMR, the raw measurement is the free induction decay with its acquisition parameters: nucleus, field, pulse programme, number of scans, relaxation delay, spectral width, temperature, solvent. A processed spectrum as a picture is not raw data, and neither is a peak list. Everything downstream, phasing, baseline, apodisation, integration, is a processing decision that someone must be able to repeat or change.
For chromatography, the raw measurement is the detector signal over time with the method: column, mobile phase composition and gradient, flow, temperature, injection volume, detector settings. An integrated area without the signal behind it cannot be re-integrated, and re-integration is the single most frequent reason to go back to chromatographic data.
The practical rule for an archive: keep whatever allows the result to be produced again from the measurement. If a reviewer can only see the conclusion, the archive is a filing cabinet, not a data resource.
The format landscape
| Format | Scope | Role |
|---|---|---|
| Vendor native (NMR vendor directories, chromatography project files) | Everything the instrument recorded | The authoritative original; keep it unchanged |
| JCAMP-DX | Spectroscopic data, including NMR and IR | Long-established exchange format, widely readable, limited for complex multidimensional data |
| nmrML | NMR raw and processed data with metadata | Open standard aimed at reproducibility and repository deposition |
| AnIML | Analytical data across techniques | XML-based open standard for instrument data and metadata |
| Allotrope Simple Model and data format | Lab data across techniques, with controlled vocabulary | Industry consortium standard, increasingly used in regulated labs |
| mzML | Mass spectrometry | De facto open standard for MS raw data |
| CSV or text export | Signal values | Useful for sharing, insufficient as an archive, because metadata are lost |
The practical approach is not to pick one. Keep the vendor original as the authoritative record, add an open-format copy for long-term readability, and treat any text export as a convenience copy rather than an archive.
Metadata: the part that makes data findable
A file that can be opened but not understood is as unusable as one that cannot be opened. Two separate sets of metadata matter:
Technical metadata
come from the instrument: all acquisition parameters, instrument identity and configuration, software version, calibration state, timestamp.
Contextual metadata
come from the lab and have to be attached deliberately: which sample, from which batch or experiment, prepared how, measured why, by whom, linked to which study or project, and which result the data support. Without the context, a spectrum is a curve.
A useful minimum for each measurement: a unique identifier, sample identifier, project or study identifier, technique, instrument, date, operator, and a link to the processing record. Attach these at acquisition, not later, because contextual metadata are unrecoverable once the person who knew them has moved on.
Four decisions that determine whether the archive works
1. Conversion is additive, never a replacement. Convert to an open format for readability, keep the original. A conversion can lose information you have not noticed yet. 2. Processing is recorded as a parameter set, not as a result. For NMR, the phase, baseline and window function; for chromatography, the integration parameters. Then a different analyst can reproduce the same number, or change one setting with the rest held constant. 3. Identifiers are machine-readable. Sample names typed by hand are the usual breaking point between the instrument archive and the LIMS. A barcode or a generated identifier removes a whole class of problems. 4. Reprocessing is an event, not an overwrite. Keep both the original and the reprocessed result with the reason. This is a data integrity requirement in regulated labs and simply good sense elsewhere.
What the effort buys
With raw data, parameters and context in place, several things that are normally projects become queries: finding every spectrum of a compound family measured in the last ten years, re-integrating a batch of chromatograms with one changed parameter, comparing a current batch against all historical batches of the same product, and using historical measurements as the basis for an in-house model. Each of those needs the metadata more than it needs the signal.
It also removes a specific risk: dependence on one software version. An archive that only opens in a discontinued application has a deadline, whether anyone has written it down or not.
Limits of the statement
- Open formats are not lossless for every technique. Multidimensional NMR, hyphenated techniques and vendor-specific detector modes can carry information that a converter does not represent. Check conversion fidelity on real examples before trusting it.
- Standards evolve, and adoption varies between vendors and techniques. Check the current state of each standard and the actual export capability of your instruments rather than assuming support.
- Metadata quality cannot be improved retroactively. For historical data, the realistic goal is to capture what is recoverable and to mark the rest as unknown, not to reconstruct it.
- In regulated environments the archive is also subject to retention, backup and audit trail requirements that go beyond readability.
Frequently asked questions
What is raw data in NMR?
The free induction decay together with the acquisition parameters needed to process it: nucleus, field, pulse programme, scans, relaxation delay, spectral width, temperature and solvent. A processed spectrum or a peak list is a result, not raw data.
Should instrument data be converted to an open format?
Yes, as an addition. Keep the vendor original as the authoritative record and store an open-format copy for long-term readability, after checking on real examples that the conversion preserves what you need.
Which open formats exist for analytical data?
JCAMP-DX for spectroscopic data, nmrML for NMR, AnIML for analytical data across techniques, the Allotrope formats for lab data with a controlled vocabulary, and mzML for mass spectrometry. Support differs by vendor and technique, so verify export options per instrument.
What metadata should be captured with every measurement?
A unique identifier, sample and project identifiers, technique, instrument and configuration, software version, date, operator, acquisition parameters and a link to the processing record. Technical metadata come from the instrument; contextual metadata must be attached at acquisition.
Why can integrated areas not replace the chromatographic signal?
Because re-integration is the most common reason to return to the data. Without the signal, a changed integration parameter, a corrected baseline or a re-evaluated shoulder cannot be applied, and the measurement cannot be reviewed.