ferro-hgvs
A high-performance HGVS variant nomenclature parser and normalizer, written in Rust with Python
bindings. It supports every HGVS coordinate system (g / c / n / r / p / m / o) and
edit type (substitution, deletion, insertion, duplication, inversion, repeat).
Use it as a Python package, a Rust crate, or a command-line tool. API docs are on docs.rs; source is on GitHub.
Start here
- Installation —
pip install ferro-hgvs,cargo install ferro-hgvs, or the Rust crate. - Quickstart — parse a variant, then normalize one.
Guides
- Normalize variants — batches, files, formats, error modes.
- Reference data — what
ferro preparedownloads, and the optional Ensembl / RefSeqGene data. - Benchmarking — timing ferro and comparing it against other HGVS tools.
- CLI reference — every subcommand.
Interpretation
The Interpretation section is a per-line reading of the HGVS recommendations that records exactly how ferro interprets each clause — with runnable example spellings, the form ferro normalizes each input to, and the reasoning behind each decision. It is under construction, one spec page at a time.
Installation
ferro ships as a Python package, a Rust crate, and a standalone command-line tool. Install whichever fits your workflow — they share the same engine.
Python
pip install ferro-hgvs
Pre-built wheels are available for Linux (x86_64, aarch64), macOS (x86_64 and Apple Silicon), and Windows (x86_64) on Python 3.10+.
Command-line tool
cargo install ferro-hgvs
This builds the ferro binary from source and places it on your PATH. It needs a Rust toolchain
(rustup).
Rust library
Add the crate to your project:
[dependencies]
ferro-hgvs = "0.14"
Verify the install
ferro parse "NM_000088.3:c.459A>G"
import ferro_hgvs
print(ferro_hgvs.parse("NM_000088.3:c.459A>G"))
Parsing needs no reference data. Normalization against real transcripts does — see Reference data.
Quickstart
This walks through parsing a variant, then normalizing one against reference data.
Parse a variant
Parsing validates a description and returns its structure. It needs no reference data.
ferro parse "NM_000088.3:c.459A>G"
import ferro_hgvs
variant = ferro_hgvs.parse("NM_000088.3:c.459A>G")
print(variant.variant_type) # "coding"
print(variant.reference) # "NM_000088.3"
print(str(variant)) # "NM_000088.3:c.459A>G"
Normalize a variant
Normalization rewrites a description into its canonical form. It needs reference sequences, so first prepare a reference (a one-time download — see Reference data):
ferro prepare --output-dir ferro-reference
ferro check --reference ferro-reference
Then normalize:
ferro normalize "NM_000088.3:c.459del" --reference ferro-reference/
import ferro_hgvs
normalizer = ferro_hgvs.Normalizer.from_manifest("ferro-reference/manifest.json")
print(normalizer.normalize("NM_000088.3:c.459del"))
Read the warnings
Normalization sometimes repairs a description in a way the normalized string does not itself
record — separately reported members merged into one delins
(MEMBERS_COALESCED_FROM_REPORTED_FORM), a reference-range insertion payload replaced by the bases
it denotes (INSERTED_SEQUENCE_EXPANDED), or a stated reference base that contradicted the reference
and was accepted anyway (REFSEQ_MISMATCH).
These are reported as warning[CODE]: message on stderr (and in the warnings array under
--format json, the detail column under --format tsv), so a pipeline reading only stdout will
not see them. The message on each warning[CODE]: line says what the code means.
Next steps
- Normalize variants — batches, files, output formats, error modes.
- Reference data — what
ferro preparedownloads and the optional data. - CLI reference — every subcommand.
Normalize variants
Normalization rewrites a description into its canonical form — 3′-shifting, collapsing equivalent spellings, and applying the recommendations’ preferred representation. This guide covers batches, files, output formats, and error modes.
Normalization needs reference sequences. Prepare a reference first (see
Reference data) and pass --reference.
A single variant
ferro normalize "NM_000088.3:c.459del" --reference ferro-reference/
Read from stdin instead:
echo "NC_000001.11:g.12345A>G" | ferro normalize --reference ferro-reference/
A batch from a file
One description per line:
ferro normalize -i variants.txt --reference ferro-reference/ -o normalized.txt
Sort the input by transcript accession (or genomic position) for large batches. ferro caches each resolved transcript, so consecutive variants on the same transcript skip the dominant cost of re-reading it. Sorted input keeps the working set resident and is markedly faster.
Use several workers for large batches:
ferro normalize -i variants.txt --reference ferro-reference/ -j 8
Output formats
-f/--format selects the output:
text(default) — the normalized description, one per line.json— a structured record per input, including awarningsarray.tsv— a table with headerline, input, normalized, changed, status, detail, plus a summary line on stderr. This is the format for answering “which of my variants changed?”
ferro normalize -i variants.txt --reference ferro-reference/ -f tsv > normalized.tsv
Error modes
--error-mode controls how strict ferro is about its input and output:
strict(default) — validates that the input conforms to the recommendations and fails on a violation.lenient— accepts a wider range of inputs and repairs where it can; fails only when it cannot normalize.silent— lenient, but without the diagnostic messages.
Regardless of mode, normalization may repair a description in a way the output string does not
record. Those repairs are reported as warning[CODE]: message on stderr (and in the warnings
array under -f json, the detail column under -f tsv). A pipeline reading only stdout will not
see them, so capture stderr or use -f json/-f tsv. The message on each warning[CODE]: line
says what the code means; use --ignore / --reject to tune specific codes.
Normalize or rederive?
ferro exposes two entry points for canonicalizing an HGVS description, but they are different operations with different guarantees. This guide says what each does, which axes each supports, and when to reach for one over the other.
normalize | rederive | |
|---|---|---|
| what it takes | an HGVS description | an HGVS description |
| what it does | shifts and re-spells the given description in place | throws the spelling away and re-derives from the denoted bases |
| axes | every axis — g/c/n/r/p/m | genomic only — g. (and m. on the two rCRS accessions) |
| confluence | best-effort (rule 3) — converges equivalent inputs | confluent by construction, over one window |
| needs a reference | yes | yes |
normalize — canonicalize a description you were handed
Use normalize when you already have a description and want its canonical form: 3′/5′ shifting
and the recommendations’ preferred representation, applied to the spelling you passed in. It works
on every axis, because a c./r./p. description carries information — reading frame, which
transcript, a protein consequence — that a genomic base window cannot recover.
import ferro_hgvs
nz = ferro_hgvs.Normalizer(reference_json="…")
nz.normalize("NM_000088.3:c.100delA") # -> "NM_000088.3:c.100del"
Converging equivalent inputs to one output is confluence — rule 3 of the normalization
rules, which normalize targets on a best-effort basis: a
handful of known residual defects remain, where two spellings of one variant still normalize to
two strings. Those are tracked bugs, not the intended contract.
rederive — one canonical description per variant
Use rederive when the input’s spelling should carry no weight — when you want the description
for a variant, regardless of how it happened to be written. rederive expresses the variant as
the bases it denotes (a reference/alternate window) and derives a description from those bases
alone, so two spellings of one variant reach one result:
nz.rederive("NC_TEST.1:g.14_15insACGTACGT") # -> "NC_TEST.1:g.7_14dup"
nz.rederive("NC_TEST.1:g.7_14dup") # -> "NC_TEST.1:g.7_14dup"
Because it re-derives from a genomic sequence window, rederive is genomic-only and refuses a
transcript or protein axis. Project onto the axis you need afterwards if required.
rederive returns the alignment-derived form by default. Pass recommended_form=True to route
the result through normalize for ferro’s recommended, reference-anchored form:
nz.rederive("NC_TEST.1:g.11del", recommended_form=True) # 3′-shifted per the recommendations
rederive is the single-call form of the to_sequences → from_sequences round trip described
in Deriving a description from sequences; read that guide for the
mechanics and for the read-dependence cost of a window-local derivation.
Which one?
- You have a description and want it in canonical form on its own axis →
normalize. - You want the same description for a variant no matter how two callers spelled it, on a genomic
axis →
rederive. - You have raw bases (a reference/alternate window) rather than a description →
from_sequences.
Project a variant to another axis
ferro project re-expresses a variant on a chosen output axis — genomic (g), coding (c), non-coding (n), protein (p), or RNA (r) — against a transcript. It is how you move a genomic call onto a transcript, or read the protein consequence of a coding one.
ferro project --axis c --transcript NM_000059.4 --reference <dir> \
'NC_000013.11:g.32316466T>G'
# NC_000013.11(NM_000059.4):c.6T>G
Coding-axis rules apply on the projected axis
The projected c./r. axis is normalized on the transcript, so it applies the coding-axis rules a bare genomic axis cannot — and produces the same string ferro normalize gives for the coding-authored form of the variant, whichever axis you started from.
The clearest case is the coding one-amino-acid delins exception (DNA/delins.md:18): two substitutions one nucleotide apart within a single codon are written as one delins, not as two members.
# BRCA2 codon 2 (c.4_6 = CCT): c.4 C>A and c.6 T>G, with c.5 unchanged.
ferro project --axis c --transcript NM_000059.4 --reference <dir> \
'NC_000013.11:g.[32316464C>A;32316466T>G]'
# NC_000013.11(NM_000059.4):c.4_6delinsACG
Because both members fall in one codon, the coding axis merges them into c.4_6delinsACG — identical to ferro normalize on NM_000059.4:c.[4C>A;6T>G]. The genomic axis has no reading frame, so it keeps the members individual, and a pair that straddles a codon boundary or an exon junction stays split on every axis.
Pairs with deriving from sequences
If you hold bases rather than a description — a window from a BAM, a VCF row — derive the genomic description with from_sequences first (see Deriving a description from sequences), then project it onto the transcript:
# 1. derive the g. description from the bases
# 2. project it onto the transcript
ferro project --axis c --transcript NM_000059.4 --reference <dir> '<the g. description>'
Deriving a description from sequences
If what you have is bases rather than a description — a window out of a BAM, a VCF row, an
aligner’s output — from_sequences derives the description instead of asking you to spell one:
import ferro_hgvs
ferro_hgvs.from_sequences("NC_000001.11", 1000, "AGCGT", "AGT")
# NC_000001.11:g.1002_1003del
use ferro_hgvs::{from_sequences, FromSequencesOptions};
let variant = from_sequences("NC_000001.11", 1000, "AGCGT", "AGT",
&FromSequencesOptions::default())?;
The axis follows the accession: g., or m. on NC_012920 / NC_001807, which HGVS requires the
m. coordinate system for. Every other accession class — transcript, protein, UniProt — is refused
with a message naming it.
It reads no reference sequence. The output is a pure function of its arguments — the
accession, the position, the two sequences and the options — so the same bases give the same
description on any machine, against any reference build, with no hidden input. position is
1-based, and reference is taken on trust: verifying it would need the reference and would make
the provider a hidden input, costing exactly the determinism the function exists to provide.
That is five values, not four. max_grid_cells is not inert — it decides whether an answer is
produced at all — so the “four arguments” this section and the Rust docs both used to claim is
withdrawn. Purity is the property; the count was wrong. (FromSequencesOptions carries a
direction too, but it is not a caller-facing knob: it is #[doc(hidden)], always 3’ on every
shipped path, and exists for the internal differential oracle described under rule 6.)
How to: one canonical description per variant
The job this exists for — a pipeline that aligns reads, post-processes a BAM, and wants one description per variant, decided by the observed bases and nothing else.
1. Get a window. From a pileup, a VCF row, or an aligner’s output: the reference bases over
some interval, the observed bases over the same interval, and the 1-based position of the window’s
first base. If what you hold is a description rather than bases, Normalizer::to_sequences
produces the same window — see Going the other way.
2. Derive, and read the flag.
d = ferro_hgvs.from_sequences_detailed("NC_000001.11", 1000, "AGCGT", "AGT")
d.variant # NC_000001.11:g.1002_1003del
d.placement_bounded_by_window # False — the bases settled the placement, not the window
# The same variant, read through a window that stops at it:
d = ferro_hgvs.from_sequences_detailed("NC_000001.11", 1000, "AGCG", "AG")
d.variant # NC_000001.11:g.1002_1003del — the same answer
d.placement_bounded_by_window # True — the deletion is flush with the window's 3' edge
The two rows are the flag’s whole meaning: same description, different confidence that a wider
read would agree. The first window has a base to spare 3’ of the deletion, so the placement is
settled by the bases; the second ends exactly where the deletion does, so the flag fires even
though the answer is unchanged. This example used to show only the second window while claiming
False, which inverted both the value and its explanation.
Prefer from_sequences_detailed to from_sequences in a pipeline. The flag is the only thing that
tells you the window may have decided the answer, and a bare from_sequences discards it.
3. Store it, or normalize first. What you have is already conformant (rule 1) and deterministic
(rule 4), so it is safe to store and to compare between runs and between machines. It is not
necessarily the recommended form, and it is not guaranteed to agree with a description derived from
a different window — those are rules 2 and 3, and both need the reference. If you want them, run
normalize on the result. That is the whole offer: derive now, normalize later, or never.
4. If the flag is set and you need the reference-anchored answer, either re-derive from a wider
window or run normalize. Do not treat a flagged result as wrong — see below.
How wide should the window be?
Wide enough to contain the whole interval over which the change could legally be placed. That is the exact condition, and it is what the cost section pins.
Operationally: pad on both sides by at least the length of the longest ambiguous run — a
homopolymer or tandem repeat — the variant might sit in. An insertion or duplication needs one
further base 5’, since that is where a 5’-most insertion anchors. to_sequences pads by 128 on
each side, so its window is span + 2 * pad; that covers ordinary repeats comfortably, and is
worth raising if you work with long tracts.
Two things follow that are easy to get backwards:
- A window that cuts the interval does not give a wrong answer, it gives a bounded one. The description still denotes the same bases and carries the same canonical SPDI; it is simply placed at the window’s edge rather than the run’s. What you lose is the preferred spelling, not correctness.
placement_bounded_by_windowis conservative on purpose. It reports “this could have moved”, not “this is wrong” — a window flush with a tract is flagged and is nonetheless the same answer a whole-sequence derivation gives. Distinguishing the two needs the reference, which this function does not read.
The refusals, and what to do about each
The policy is to refuse rather than quietly answer with a weaker rule, so each of these is actionable rather than fatal:
| refusal | why | what to do |
|---|---|---|
| the accession is not genomic | this surface emits g. (and m. on the two rCRS mitochondrial accessions) and nothing else; NM_…:g.9_10del would be well-formed and denote nothing | pass the genomic accession, and project afterwards if you need a transcript axis |
| an inserted payload sits against the window’s 5’ edge | HGVS writes an insertion between two positions, so it would have to anchor at position - 1 — outside the window | re-fetch with more 5’ flank. How much is direction-dependent, so widen rather than adding a fixed one base |
the alignment grid exceeds max_grid_cells | a cost bound — a cell is roughly 18 bytes, and the default admits a window of about 4 096 bases | raise max_grid_cells if you have the memory, or narrow the window. Real structural alleles are far past any sane budget: LRG_542:g.[101177_102434delins36;107248_127198delins21] spans 26 kb |
a symbol outside the IUPAC-IUBMB set, a zero position, an empty reference | the input cannot denote anything | fix the input. X and - are refused deliberately (standards.md:39) — they are alignment symbols, not bases |
U in either sequence | this surface’s axis is DNA; a g./m. description naming U would be well-formed and wrong | pass T, and project onto an r. axis afterwards if you need RNA |
Case is not a refusal: a soft-masked (lower-case) window derives exactly as its upper-case twin does, and both sequences are folded before anything reads them.
Which rules it delivers
This is the whole design, and it falls straight out of the four normalization rules:
| rules delivered | force | needs | |
|---|---|---|---|
from_sequences | 1 (conformant), 4 (deterministic) | both absolute | the caller’s four arguments |
normalize, afterwards | 2 (recommended form), 3 (confluent) | both best effort | the reference |
Rules 1 and 4 are the two the normalization rules call always achievable, so a function that has only the caller’s arguments can still deliver both in full. Rules 2 and 3 need the reference: rule 2’s scope names the 3’ rule explicitly, and a reference-anchored shift is precisely what a window-local function cannot perform.
So an output may be 3’-shiftable further than the window allowed, and that is not a defect.
Run normalize afterwards if you want it — Normalizer::from_sequences(..., recommended_form = true)
does both in one call. (Normalizer::rederive is the one-call path when you have a
description rather than a sequence pair — see Normalize or rederive?.)
Run it unless you have a reason not to. In an internal sweep of many synthetic shapes, normalize
moved a meaningful share of derived descriptions — in three classes: repeat notation (g.27_28insAAA → g.27A[4]),
reference-anchored member re-derivation, and an inversion spread across several members
(g.[17C>A;19T>A;21T>G] → g.17_21inv, which the alignment DAG partitions before anything can see
it, since it minimises edit distance and an inversion is not in that cost model). All three are
rule 2 and rule 3 — the recommended form and agreement with a wider view — which is exactly the
pair this design assigns to normalize. Rules 1 and 4 hold either way.
That movement rate is a qualitative claim about a narrow sweep, not a benchmark: one synthetic contig, a handful of shape generators, genomic axis only, from a one-off measurement with no committed harness — so it is deliberately not quoted here as a number.
What you get for it
Two spellings of one variant, over one window, reach one description — because the derivation
never sees a spelling. Over the cis confluence corpus that is 5 636 classes with no divergence —
its genomic half, and all of what this surface can reach: the corpus is generated --axes g,c
at 11 272 classes, and the 5 636 c. classes are drawn against NM_TEST.1, which the g.-only
gate refuses, so not one of them enters the comparison. The exclusion is structural and is asserted
as such in tests/it/from_sequences_corpus.rs.
Over the nine externally-reported confluence pairs (#1419 / #1420 / #1421) it is nine of nine, in
both shuffle directions — where normalize, handed the same pairs as descriptions, currently
converges none of them.
Read “over one window” as load-bearing, not as hedging. It is what makes the claim arithmetic:
to_sequences computes its window from the denoted bases, so both spellings of a variant get
byte-identical (position, reference, alternate) triples, and a pure function of that triple can
only give one answer. It is also the exact limit — the claim is confluence over spellings, and
rule 3’s scope is confluence over inputs. Two reads covering one variant differently are two
inputs, and the section below is what happens then.
Read the comparison with normalize as two functions answering different questions rather than as
one beating the other. A caller who has a description and wants it normalized still needs
normalize to converge; the pairs are simply the case where being handed a description is itself
the problem.
The cost, stated plainly
A window-local derivation is read-dependent. Nothing may shift outside the bases you supplied,
so a read that stops partway through an ambiguous run places the change at the end of the read
rather than the end of the run. One deletion from the AAAA at 12–15 of a test contig, seen
through three windows:
| window | reference | alternate | derived | placement_bounded_by_window |
|---|---|---|---|---|
| 10–16 | GCAAAAG | GCAAAG | g.15del | false |
| 12–15 | AAAA | AAA | g.15del | true |
| 10–14 | GCAAA | GCAA | g.14del | true |
None of these is wrong. All three carry the same canonical SPDI (14:A:) and denote the same
bases — g.14del is a conformant description of exactly the same variant, it is simply not the
3’-most spelling. So what a truncating read costs you is the recommended form (rule 2) and
agreement with a wider read (rule 3): precisely the two rules this function never claimed,
because both need the reference. Rules 1 and 4 hold in every row. normalize closes the gap,
shifting to 15 regardless of what the read covered.
The boundary is exact, and pinned in tests/it/from_sequences_window_condition.rs:
Two windows that both contain the whole interval over which the change can be placed derive the same description. A window that cuts that interval places the change at its own edge instead.
Note that for an insertion or duplication that interval already reaches one base 5’ of the tract, since that is where a 5’-most insertion anchors — so “contains the interval” subsumes the flank requirement rather than needing a separate clause.
placement_bounded_by_window is a “could move” flag, not a “is wrong” flag, and it is
conservative in that direction on purpose. Row 2 is flagged and already correct; row 3 is flagged
and merely non-preferred. Telling those apart requires knowing what lies outside the window — the
reference — which this function does not read, so it reports the uncertainty rather than resolving
it. Treat a true as “re-derive from a wider window, or run normalize, if you need a
reference-anchored answer”.
Two refusals are worth knowing about in advance, both deliberate — the policy is to refuse rather than degrade to a weaker rule:
-
An alignment grid over budget. The default admits a window of about 4 096 bases; a cell costs roughly 18 bytes.
max_grid_cellsis the knob, and the refusal names it. Real structural alleles are well past it —LRG_542:g.[101177_102434delins36;107248_127198delins21]spans 26 kb.Do not read the multi-member census as a measure of this refusal. Of the 592 multi-member alleles harvested from ClinVar, CMRG and Paraphase, 443 windows were captured and 59 derive — but splitting the 384 refusals by message gives 384 accession refusals and 0 grid refusals, every one of them an
NM_transcript hitting theg.-only gate above. The structural rows are filtered out of the capture before the grid is ever consulted. That figure was quoted here as evidence for the grid bound and is not. -
An inserted payload against the window’s 5’ edge. HGVS writes an insertion between two positions, so such a payload can only be anchored at
position - 1— outside the window, and non-existent whenpositionis 1. Supply more 5’ flank;Normalizer::to_sequencespads both sides for you.
Going the other way
Normalizer::to_sequences is the inverse, so a caller who already holds descriptions needs no new
plumbing to reach the derivation:
pair = normalizer.to_sequences(variant, pad=128)
derived = normalizer.from_sequences(pair.accession, pair.position, pair.reference, pair.alternate)
The pad is not decoration: dup typing reads the reference bases immediately 5’ of an insertion
point (DNA/duplication.md:18), so a member flush with the window’s 5’ edge comes back as an ins
instead of a dup. It is applied to both sides — the window is span + 2 * pad — and the
bases come back upper-cased, so a soft-masked region does not produce a mixed-case pair.
Bounding a derivation to a region it must not leave
When a variant must stay inside a target region, an amplicon or a tiling window, anchor every raw pair to that region first. The derivation is a pure function of the window it is handed, so one window gives one answer:
pair = ferro_hgvs.SequencePair("chr1", 10, "GCAAAAG", "GCAAAG") # straight from a BAM
str(pair.derive().variant) # 'chr1:g.15del' — rolls to the run's end
bounded = pair.trim_to(end=14) # hold it at 14
str(bounded.derive().variant) # 'chr1:g.14del'
trim_to needs no reference and can only narrow. To widen, use Normalizer::reanchor, which
reads the padding bases from the reference:
anchored = normalizer.reanchor(pair, start=5, end=25) # 5' widened, 3' widened
str(anchored.derive().variant) # 'chr1:g.15del'
both = normalizer.reanchor(pair, start=5, end=14) # 5' widened, 3' narrowed
str(both.derive().variant) # 'chr1:g.14del'
reanchor moves a window’s edges; it does not relocate the window. Each edge may go outwards
(padded from the reference) or inwards (trimmed), in any combination — but the window you ask for
must overlap the pair’s own, and the overlap must still hold the bases the two sequences
disagree on. reanchor(pair, start=1000, end=1200) on the pair above is refused, not fetched: the
changed bases exist only in the pair, so there is nothing to carry to a region the pair does not
cover. So “anchor every raw pair to my target region” works exactly when every raw pair overlaps
that region — which is the case the feature is for, and is worth checking rather than assuming.
Prefer pair.derive() to re-spreading the four fields: a pair returned by trim_to or
reanchor carries its own position, and pairing a pre-trim position with post-trim bases is
the mistake the method exists to prevent.
Both take 1-based inclusive bounds, and None leaves that edge where it is.
Both refuse rather than clamp, in every case: a bound that would cut a base the two sequences
disagree on (naming the coordinate), a bound that would empty the reference, start past end,
and — for reanchor — a bound outside the sequence, or a window disjoint from the pair’s. A window
silently pulled back to the contig would hide a bug upstream of the call.
Case is not a disagreement: a soft-masked reference against an upper-case alternate trims normally.
trim_to fetches nothing and so leaves your bases as you passed them; reanchor reads flank from
the provider and therefore returns the whole window upper-cased, exactly as to_sequences
does, rather than splicing provider bases onto caller bases and handing back a mixed-case pair.
Reach for this when the bound is a requirement, not to make heterogeneous inputs agree. For that,
Normalizer::from_sequences(..., recommended_form = true)and ato_sequencesround trip both already converge, and both reach the reference-anchored placement — which can shift as far as the sequence allows rather than as far as your window allows. Anchoring to a window that cuts an ambiguous run makes every caller using that window agree with each other and disagree with the reference. That is a legitimate contract and a poor default;placement_bounded_by_windowreports it either way.
Reference data
Parsing needs no reference data. Normalization does — it must read the transcript and genome
sequences a description refers to. ferro prepare downloads and assembles them into a reference
directory.
Prepare a reference
ferro prepare --output-dir ferro-reference
This downloads RefSeq transcripts, genome FASTAs, and cdot metadata, and writes them under
ferro-reference/. A bare prepare builds a RefSeq-only reference — accessions NM_ / NR_ /
NP_ / NG_.
Verify it
ferro check --reference ferro-reference
Optionally pre-build the on-disk cdot cache as a setup step, so the one-time cache build does not slow the start of a real (or timed) run:
ferro check --reference ferro-reference --build-cache
Optional data
Two opt-in flags provision more than RefSeq. Pass them at prepare time; both are incremental, so
re-running prepare over an existing reference adds the requested data and preserves what is already
there.
Ensembl support (accessions ENST / ENSG / ENSP) — downloads Ensembl cdot metadata and cDNA
FASTAs (~1 GB+); off by default. Without it, an Ensembl input reports “Reference not found”:
ferro prepare --output-dir ferro-reference --ensembl
RefSeqGene placements — derives version-independent NG_ placements and the
NG_→transcript-version map, required to resolve legacy gene-symbol selectors (NG_(GENE):c.…) and
bare-NG_ hosted lookups:
ferro prepare --output-dir ferro-reference \
--derive-ng-placements path/to/ng_accessions.txt
A fully-provisioned reference combines both in one run:
ferro prepare --output-dir ferro-reference --ensembl \
--derive-ng-placements path/to/ng_accessions.txt
Using it
Point any normalizing command at the directory:
ferro normalize -i variants.txt --reference ferro-reference/
To guard against a reference that has drifted on disk, add --strict-reference, which hard-fails if
the reference’s content no longer matches its recorded identity (the default warns and proceeds).
Error Handling
ferro-hgvs provides configurable error handling with three modes:
| Mode | Behavior |
|---|---|
strict | Reject non-conformant input (default) |
lenient | Auto-correct with warnings |
silent | Auto-correct silently |
# Use lenient mode to auto-correct common issues
ferro parse --error-mode lenient "p.val600glu" # Corrects to p.Val600Glu
# Ignore specific warnings
ferro parse --ignore W1001,W2001 "p.val600glu"
# Get help on any error/warning code
ferro explain W1001
ferro explain --list
Configuration File
Create .ferro.toml in your project directory:
[error-handling]
mode = "lenient"
ignore = ["W1001", "W2001"] # Silently correct these
reject = ["W3003"] # Always reject these
Comparing normalization rules (FERRO_PARTITION)
Unstable.
FERRO_PARTITIONis an evaluation switch, not a supported feature. It is not covered by semantic versioning, its values may change, and it is expected to be removed once the normalization rule is settled. Do not depend on it in production pipelines.
Normalization cuts each changed region of sequence into allele members. FERRO_PARTITION selects which rule does that cutting, so a candidate rule can be measured against the shipped one over a real corpus before anything changes for users.
| Value | Rule |
|---|---|
unset / empty / canonical-coalesced | The shipped rule, and what every normal invocation uses. canonical, plus the delins.md:44-47 merge: a split whose payload realigns as one block is re-spelled as a single delins. Applied after the downstream passes rather than at partition time, so it cuts identically to canonical and differs only in what survives. |
live | The rule shipped up to and including v0.14.0: a single-gap alignment search plus two narrow escapes. No longer the default — set it by name to reproduce pre-flip output. |
shadow | Cut only at alignment steps common to every minimal alignment. |
canonical | The member-count-minimal minimal alignment, without the merge above. |
With the variable unset — or set to the empty string — output is byte-identical to a build with no switch at all.
The default moved in v0.15.0, from
livetocanonical-coalesced. If you are comparing against a stored corpus normalized by v0.14.0 or earlier, the like-for-like arm is nowFERRO_PARTITION=live, not the unset one. The change is disclosed as a representation change inCHANGELOG.md; the ruling behind it is that a description is derived from the resulting sequence rather than preserved from the input’s spelling, whichpartition_block—live’s cutter — cannot do.
Running an A/B comparison
Run the same input twice and diff the normalized descriptions. In tsv format the columns are line, input, normalized, changed, status, detail, so column 3 is the normalized string:
# 1. a baseline arm, named explicitly. `live` is the pre-v0.15.0 rule, which is
# the one to use when the question is "what moved for my stored corpus";
# name the arm rather than relying on unset, which is now the NEW rule.
FERRO_PARTITION=live ferro normalize --input variants.txt --reference /path/to/reference \
--format tsv --error-mode lenient -j 10 > shipped.tsv
# 2. the candidate rule — here, the shipped default
ferro normalize --input variants.txt --reference /path/to/reference \
--format tsv --error-mode lenient -j 10 > candidate.tsv
# 3. what moved
diff <(cut -f3 shipped.tsv) <(cut -f3 candidate.tsv) | head
# 4. how many moved, counting only rows that succeeded on both sides
# (columns 3/5 are `normalized`/`status`; a row that failed carries no
# normalized string, so comparing column 3 alone would score two different
# failures as identical)
paste <(cut -f3,5 shipped.tsv) <(cut -f3,5 candidate.tsv) \
| awk -F'\t' '$2 == "ok" && $4 == "ok" && $1 != $3' | wc -l
# and, separately, rows whose status itself changed
paste <(cut -f5 shipped.tsv) <(cut -f5 candidate.tsv) | awk -F'\t' '$1 != $2' | wc -l
This works on a stock release build — the switch is not behind a build feature, so no special binary or wheel is needed.
Two traps worth knowing
A misspelled value is refused, loudly. FERRO_PARTITION=canonicl makes ferro exit with an error before it reads any input, naming the value you gave and the arms this build has:
FERRO_PARTITION="canonicl" is not a partitioner this build has. This build's arms
are: live, shadow, canonical, canonical-coalesced. Refusing rather than falling
back to `live`, because a bake-off served the shipped rule under a candidate's
name reports that the candidate changes nothing.
Naming this build’s arms is the point: a value that exists on some other branch, or that used to exist, is reported as absent here rather than quietly answered as live.
The refusal comes from the CLI, not from the normalizer. Up to and including v0.14.0 it was a panic raised deep inside normalization, which meant a development-only switch could abort any process — a long-running service, or a Python caller, across the FFI boundary — that merely happened to have the variable set. A release build of the library now falls safe to live for a value that names no arm and keeps the refusal for its caller to report. Every binary and every example in this repository reports it — ferro, ferro-web, ferro-benchmark, both spec generators, and all 23 declared examples, which are the bake-off harnesses, the window extractors and the artifact generators — and that is pinned by a test whose denominator is Cargo.toml’s own [[bin]] and [[example]] tables rather than by this sentence, so a target added later cannot quietly skip it. The one class of cargo target still outside that obligation is [[bench]]: criterion’s criterion_main! generates the main, so there is no hand-written entry point to put the call in. If you embed ferro as a library and run bake-offs through it, read ferro_hgvs::normalize::partition_switch_startup_error() at startup and do the same.
Builds up to and including v0.13.1 fell back to live instead, and produced a clean, empty diff that read as “the candidate changes nothing”. The fallback emitted a warning through the log facade, but the ferro CLI installs no logger, so that warning reached no stream and RUST_LOG could not surface it — there was no signal at all. If you are on an older build, treat every empty diff as unproven.
A positive control on an input known to differ is still worth running, because it catches the other way a comparison can be vacuous: a variable that never reached the process at all (a lost export, a sudo that scrubbed the environment, a container that did not forward it). That case is indistinguishable from unset, which is legitimately live, so no amount of validation inside ferro can catch it. On your own corpus a zero remains ambiguous between “the switch is not taking effect” and “this corpus has no affected variants”.
These three inputs run against the built-in test data, so they need no --reference and no prepared reference directory. Between them they separate all four arms — every pair of arms disagrees on at least one row, so the control tells you which arm you got, not merely that something changed:
printf 'NM_001234.1:c.[5_6insAC;9del]\nNM_001234.1:c.2_6delinsGA\nNM_001234.1:c.4_10delinsAC\n' > control.txt
for arm in live shadow canonical canonical-coalesced; do
echo "== $arm"
if FERRO_PARTITION=$arm ferro normalize --input control.txt --format tsv \
--error-mode lenient > "control.$arm.tsv"; then
tail -n +2 "control.$arm.tsv" | cut -f3
else
echo " FAILED (exit $?) -- ferro did not produce this arm's column"
fi
done
The status check is not boilerplate. An unrecognised arm now aborts the process, and a pipeline reports the exit status of its last command — so ferro … | tail | cut reports cut’s success and the loop prints an empty column under the arm’s heading. An empty column and an aborted run look identical, which is the same “a broken measurement reads as a result” failure this whole section is about. Redirecting first and testing the status makes the abort say so.
| input | live | shadow | canonical | canonical-coalesced (= unset) |
|---|---|---|---|---|
c.[5_6insAC;9del] | c.[5_6insA;7_9delinsCAA] | c.[5_6insAC;11del] | c.[5_6insAC;11del] | c.[5_6insAC;11del] |
c.2_6delinsGA | c.2_6delinsGA | c.2_6delinsGA | c.[2del;4_6delinsA] | c.2_6delinsGA |
c.4_10delinsAC | c.4_10delinsAC | c.4_10delinsAC | c.[4C>A;6_10del] | c.[4C>A;6_10del] |
All six pairs of arms are separated, which is what makes this tell you which arm you got rather than merely that something changed: row 1 separates live from the other three, row 2 isolates canonical (it is the only arm that does not merge that block), and row 3 separates {live, shadow} from {canonical, canonical-coalesced}. Rows 1 and 3 together separate live from canonical-coalesced; row 3 alone separates shadow from it.
Because canonical-coalesced is now the default, running with the variable unset must reproduce that last column exactly. If it reproduces the live column instead, you are on a pre-v0.15.0 build.
If the arm you selected does not produce its column above, the variable is not reaching ferro and any comparison you run is meaningless. Only once the control behaves is a zero on your own corpus informative.
This control has now been wrong twice, so check it rather than trusting it. The first version offered
c.[2del;9del],c.[3del;9del]andc.[2del;9dup]and claimedcanonicalanswersc.[2del;33del]; on those three inputs no arm differed fromliveat all. Its replacement —c.[5_6insAC;9del],c.[2del;5del],c.[2del;9del]— was also wrong in four of its twelve cells when re-measured:liverow 1 readc.[5_6insA;7_9delinsCAA]and notc.6_9delinsACCAA, and rows 2 and 3 separated nothing, every arm answeringc.[2del;6del]andc.[2del;11del]. Both failures share one cause:NM_001234.1is a G homopolymer fromc.9toc.33, so deletion pairs inside it shuffle to a common form on every arm instead of partitioning differently. A discriminating row has to be adelinswhose payload re-aligns, which is what the three above are. The table is measured, not composed.
From Python, it must be set before the first normalization. The value is read once per process and cached, so:
import os
os.environ["FERRO_PARTITION"] = "canonical" # must precede the first normalize call
import ferro_hgvs
Setting it after any variant has been normalized silently does nothing, and it cannot be changed within a running process. Comparing two rules therefore means two separate processes (or two CLI runs, as above), not two calls in one script.
Benchmarking
ferro ships a benchmark harness for measuring its own throughput and comparing it against other HGVS
tools (mutalyzer, biocommons/hgvs, hgvs-rs). This guide covers running it; for the published figures
and full method, see
docs/BENCHMARK_RUNBOOK.md.
Timing ferro alone
The main ferro binary needs no special build. Prepare a reference (see
Reference data), then time a normalize run:
ferro prepare --output-dir data/ferro
ferro check --reference data/ferro
ferro normalize -i patterns.txt --reference data/ferro -j 8 -t timing.json
-t/--timing writes timing information to JSON. Two things matter for a fair number:
- Warm the cache first.
ferro check --reference data/ferro --build-cachebuilds the on-disk cdot cache as a setup step, so the one-time build does not land inside the timed region. - Sort the input by transcript accession (or genomic position). ferro caches each resolved transcript, so sorted input keeps the working set resident and is markedly faster on large batches.
Comparing against other tools
Tool comparison uses the separate ferro-benchmark binary, built with the benchmark feature:
cargo build --release --features benchmark
ferro-benchmark prepares each tool’s reference data (reusing ferro’s for transcripts), runs
parse/normalize, and compares results:
# Prepare another tool against the same transcript data
ferro-benchmark prepare mutalyzer --ferro-reference data/ferro --output-dir data/mutalyzer
# Normalize a shared pattern set with each tool
ferro-benchmark normalize mutalyzer -i patterns.txt -o mutalyzer.json \
--mutalyzer-settings data/mutalyzer/mutalyzer_settings.conf
# Compare two tools' outputs
ferro-benchmark compare results normalize ferro.json mutalyzer.json -o comparison.json
Supported tool values: ferro, mutalyzer, biocommons, hgvs-rs, and all.
The Python-based tools (mutalyzer, biocommons/hgvs, seqrepo) run in a pixi
environment defined by the repository’s pixi.toml; run pixi shell to activate it before
preparing them.
Reading the numbers honestly
All tools are benchmarked offline — reference data preloaded locally, network disabled — so the
figures measure parse/normalize compute, not per-variant I/O. Out of the box the other tools resolve
each variant against a remote service (a network round-trip per variant), which is far slower than
any local figure. That local, offline setup is exactly what ferro prepare builds. When you quote a
benchmark, say which configuration it was measured in, and confirm the run succeeded and produced
correct output before trusting a timing — a crashed run still prints a wall-clock line.
Normalization rules
ferro’s normalizer follows four rules about its output, and three about how it handles the gaps.
The output contract
-
Conformant. Output follows the HGVS recommendations. Absolute — never traded. Scope: the
syntax.yamlgrammar, plus every prohibition, read from prose force rather than keyword casing — “not allowed”, “not correct”, “can not be used”, “by definition”, theclass="invalid"markup, and the setchecklist.mdenumerates. -
Recommended form. Where the spec prefers among conformant forms, ferro produces it. Best effort. Scope: “recommended”, “preferred”, lowercase “should”, the 3’ rule. A preference clause outranks maintainer judgment, but not rule 3: where it is not evaluable on what a normalizer holds, rule 3 governs.
-
Confluent. Inputs denoting one variant produce one output. Best effort. Every rule — rule 2 included — is evaluated over the resulting sequence, never over the input’s spelling. Reference context counts as sequence: transcript model, exon boundaries, reading frame, strand and topology are functions of the accession, not of the description.
-
Deterministic. Same input, same output. Absolute. Note that 4 does not imply 3 — a deterministic normalizer can be arbitrarily non-confluent.
The procedure
-
Where the spec is silent, ambiguous, or self-contradictory: the issue is filed upstream first and cited, and only then does ferro ship a provisional choice.
- Self-contradictory — two clauses that cannot both hold — is a defect. Every conforming tool must pick a side and none of them can be right, so filing is a bug report and is not optional.
- Silent is merely incomplete. Ferro decides under rule 6 and violates nothing; filing is a feature request, worth making but never a reason to hold a release.
-
Among multiple conformant forms: the maintainers choose. There are no user options for normalization form. Error mode is an orthogonal axis and stays available. The 3’/5’ shuffle direction was the one exception, and it is now removed from the public surface rather than excused: it is not orthogonal — it selects the frame every rule is evaluated in — so it was a user option for normalization form sitting inside this rule. ferro shifts 3’, the only direction the HGVS recommendations describe, and no CLI flag, Python keyword or service config key selects otherwise. The 5’ arm survives internally as a differential oracle over ferro’s own test suite; an instrument is not a user option, and rules 2 and 3 are now claimed once rather than per direction.
-
Disclosure. Any change to these rules, and any different choice made under 5 or 6, is disclosed: in the changelog before v1, by a major version bump after. Output that violates rules 1-4 is a bug, not a disclosure.
Why 2 and 3 are best effort, and 1 and 4 are not
Rules 1 and 4 are always achievable. Conformance is checkable against the spec text, and determinism is a property of ferro’s own code; nothing external can prevent us honouring them.
Rules 2 and 3 depend on the spec determining an answer, and sometimes it does not:
- No preference exists. The spec ranks substitution, deletion, inversion, duplication and
insertion, but says nothing about competing
delinsforms. - Two preferences disagree.
general.mdranks duplication above insertion;DNA/inversion.mdprefers an insertion for inverted copies. No single output satisfies both. - The same clause has two versions.
general.md’s current text and its forthcoming NOTE give opposite answers for variants separated by one nucleotide. - The variant’s decomposition is not recoverable. Recovering one means choosing an alignment,
and the spec does not say which, so there is no derivable form to converge on. Block length does
not settle it: an equal-length block can still carry a balanced del+ins pair, so its column
correspondence need not be unique —
CAG -> AGAis equal length with edit distance 2, not the position-wise 3. What decides whether a reference base is unchanged is whether every minimal alignment matches it, so the property to key on is edit distance against block length, never length alone. Seerulings[unchanged-is-read-over-every-minimal-alignment]. - The preference keys on information a normalizer does not hold. A “frequently occurring
variant” (
RNA/delins.md:41); a repeat “variable in the population” (RNA/repeated.md:33,protein/repeated.md:22); two variants “reported (or might occur) individually” (DNA/delins.md:83). The spec determines an answer; ferro cannot see what it keys on.
“Best effort” is bounded by the spec’s determinacy and by what a normalizer’s inputs can decide — not by ferro’s implementation quality. A failure of rule 2 or 3 caused by ferro’s code is a bug under rule 7; one caused by the spec not determining an answer triggers rule 5; one caused by a clause keying on provenance or population data is a declared deviation, and a permanent one — no upstream answer can put that information into a normalizer’s hands.
Permanent is the only thing the last case adds, and it does not exempt the question from rule
5. The grain matters, because the two grains give opposite answers. Read on its own, such a
clause is not silent, ambiguous or self-contradictory — it determines an answer perfectly well, and
ferro is the one that cannot see what the answer keys on. Read against the spec’s own
re-derivation mandate — general.md:157-160, which has a protein description derived by
comparing the variant and reference protein sequences and says knowledge of the underlying DNA
change “should not be used”, with general.md:13 extending the method to RNA — it is one half of a
pair that cannot both hold. That is rule 5’s self-contradiction limb exactly, and the ruling ledger
says so in those words. What the carve-out states is that rule 5’s escalation cannot end
the matter here: a choice made under rule 5 is provisional pending an upstream ruling, and no
ruling upstream makes provenance visible, so the deviation outlives the filing.
Where it is recorded is worth stating precisely, because the obvious pointer does not resolve.
canonical-form-choice-when-both-legal has no deviates_from field: it carries all four
clauses in its rationale as the recorded counter-evidence to re-derivation, opens that paragraph
with “The spec contradicts itself here”, and adopts re-derivation over them deliberately. The one
of the four that is recorded as a deviation is DNA/delins.md:83, through the
deviates_from: ["docs/recommendations/DNA/delins.md:79-84"] on
separation-is-a-property-of-the-spelling-not-of-the-variant — the record whose ruling the rule 3
example below turns on.
A worked example of reading force from prose
DNA/duplication.md says a variant that can be described as a duplication must be — but the
“must” is scoped by the preceding clause, which defines when a duplication can be used at all:
only when the additional copy is directly 3’-flanking the original. So the rule ranks the label
for one span; it does not require that a partition be chosen so as to produce a duplication.
Reading the force without the scope inverts the rule.
What rule 3 excludes
“Never over the input’s spelling” is narrower than it sounds. Most of a description is context.
| Carried by the description | Treatment |
|---|---|
Accession, axis (g./c./n.), version | Used — it is the reference context |
| Which bases end up different | Used — this is the variant |
Cis against trans ([a;b] against [a];[b]) | Used — different variants, not two spellings of one |
Type label (dup against ins, inv against delins) | Re-derived, then ranked by general.md |
| How the edit set is cut into members | Excluded — a property of who wrote the string |
| Which copy in a run of identical residues a member names | Excluded — the 3’ rule assigns this “arbitrarily” (general.md:41) |
| Repeat unit and phase, where several are equivalent | Excluded |
So three rows read Excluded, and they are three spellings of the one thing the input does not get to decide: the partition, the run-position choice that feeds it, and — where several unit-and-phase pairs describe one tract equally well — which pair a repeat member names.
NC_000001.11:g.1001002_1001016 reads ATGAGGGGCCACTGT: a GGGG run at 1001006-1001009, a CC
run at 1001010-1001011, a lone C at 1001013. Two spellings, one denoted sequence
(ATGAGGGCATGT), because 1001010 and 1001011 are both C:
g.[1001009del;1001010del;1001013del] written gaps of 0 and 2
g.[1001009del;1001011del;1001013del] written gaps of 1 and 1
general.md:34’s “two variants separated by one or more nucleotides should be described individually”
reads those gaps, so it answers twice for one variant. Rule 3 reads them off the partition ferro
derives instead: both give g.[1001009_1001010del;1001013del]. Rule 2 then keeps that over the
spanning g.1001009_1001013delinsCA, which merges across two unchanged nucleotides. Pinned in
tests/it/cis_confluence_adjudication.rs.
Known limitation
ferro cannot today guarantee that every input form is normalized according to these rules. They are enforced intent, not a claim of current completeness.
Rule 7’s disclosure mechanism — the Representation-Change: trailer and how it reaches the
changelog — is documented in
CONTRIBUTING.md.
Where the individual decisions live
Rules 5 and 6 above say how a question is decided where the recommendations are silent, ambiguous or self-contradictory. What was decided, case by case, is recorded separately, as adjudication records — each naming the clauses in tension, which one governs, which is deviated from, and why. Those records are published in full, with their clause quotes, in docs/NORMALIZATION_CONTRACT.md.
That document is generated from the records and gated against them, so it cannot drift; it deliberately does not restate the seven rules above, which are stated only here.
The inverse index — for each stage of the normalizer, which record or clause governs it, and which decisions are governed by nothing — is docs/NORMALIZATION_STAGE_AUDIT.md. Read that one if what you want to know is whether a behaviour you are looking at was chosen or merely happened.
CLI reference
The ferro command groups its work into subcommands. Run ferro <command> --help for the full,
authoritative options of any one — this page is a map, not a substitute.
Core
| Command | What it does |
|---|---|
parse | Parse and validate an HGVS description (no reference needed). |
normalize | Rewrite descriptions into canonical form (needs a reference). See Normalize variants. |
project | Re-express a variant on a chosen output axis (g / c / n / p / r). |
explain | Explain an error or warning code (e.g. ferro explain W3003). |
Reference data
| Command | What it does |
|---|---|
prepare | Download and assemble reference data. See Reference data. |
check | Verify a reference directory is ready (optionally pre-build the cache). |
convert-gff | Convert a GFF3/GTF annotation to transcripts.json. |
build-transcript | Build a single-exon transcripts.json from a FASTA + CDS coordinates. |
VCF and interchange
| Command | What it does |
|---|---|
annotate-vcf | Annotate a VCF file with HGVS notation. |
vcf-to-hgvs / hgvs-to-vcf | Convert between VCF and HGVS. |
extract-hgvs | Extract HGVS patterns from VEP-annotated VCF files. |
liftover | Lift genomic coordinates between genome builds. |
Generation and prediction
| Command | What it does |
|---|---|
generate | Generate an HGVS description from components. |
describe | Generate a description from reference and observed sequences. |
effect | Predict the protein effect of a variant. |
backtranslate | Backtranslate a protein variant to possible DNA variants. |
Common options
Most commands accept -i/--input <file> (one description per line) and -o/--output <file>. The
output format is set with -f/--format, and the accepted values depend on the command: text|json
for parse and project, text|json|tsv for normalize, and text|json|markdown for explain.
Commands that normalize also accept --reference <dir> and --error-mode <strict|lenient|silent>.
Supported HGVS Syntax
| Type | Prefix | Example |
|---|---|---|
| Genomic | g. | NC_000001.11:g.12345A>G |
| Coding DNA | c. | NM_000088.3:c.459A>G |
| Non-coding | n. | NR_000001.1:n.100A>G |
| RNA | r. | NM_000088.3:r.459a>g |
| Protein | p. | NP_000079.2:p.Val600Glu |
| Mitochondrial | m. | NC_012920.1:m.3243A>G |
Edit Types
- Substitution:
A>G,Val600Glu - Deletion:
del,100_200del - Insertion:
100_101insATG - Deletion-Insertion:
100_102delinsATG - Duplication:
100_102dup - Inversion:
100_200inv - Repeat:
100CAG[20]
Why ferro-hgvs?
ferro-hgvs provides the most comprehensive HGVS variant normalization across all pattern types, with performance orders of magnitude faster than alternatives.
Normalization Capabilities Comparison
| Pattern Type | ferro | mutalyzer | biocommons | hgvs-rs |
|---|---|---|---|---|
| Genomic (g.) | ✓ | ✓ | ✓ | ✓ |
| Coding (c.) exonic | ✓ | ✓ | ✓ | ✓ |
| Coding (c.) intronic | ✓ | ✓** | ✗ | ✗ |
| Non-coding (n.) | ✓ | ✓ | ✓ | ✓ |
| RNA (r.) | ✓ | ✓ | ✓ | ✓ |
| Protein (p.) | ✓ | Net* | ✗ | ✗ |
* mutalyzer protein normalization requires network access for NP_→NM_ lookups (cannot be cached locally). ** mutalyzer intronic support is enabled by default via genomic-context rewriting; disable with –no-rewrite-intronic.
Performance Comparison
All tools are benchmarked in ferro’s offline configuration — best case for every tool. Reference data is preloaded locally (a local UTA database and SeqRepo) and the network is disabled, so the figures below measure parse/normalize compute, not I/O. Out of the box, hgvs-rs, biocommons/hgvs, and mutalyzer resolve each variant against a remote UTA/SeqRepo or the Mutalyzer web API — a network round-trip per variant (~100–1000 ms), i.e. roughly 1–10 variants/sec, hundreds to thousands of times slower than shown here (an order-of-magnitude estimate from per-call network latency, not separately benchmarked). That local, offline setup is exactly what ferro’s prepare command builds; ferro needs no external service.
Median patterns/sec over 5 reps on an Apple M2 Max, local/offline. All tools draw from one stratified ClinVar population; per-tool sample sizes are calibrated so each tool is measured over a meaningful interval — fast cells (e.g. ferro/hgvs-rs parse) draw from millions of patterns, while slower cells (e.g. the per-tool normalize columns) draw from as few as tens to thousands. All tools exclude process/interpreter startup from the timed region — the mutalyzer/biocommons Python subprocesses are timed by their own internal startup-excluded timer, matching ferro/hgvs-rs. Only ferro parallelizes natively (rayon); the other tools are single-threaded libraries, so their normalize @8 workers figures come from the benchmark harness running 8 independent instances in parallel, while parsing is not sharded for them — hence the single-threaded label in their parse @8 workers column (mutalyzer normalize likewise shows no gain at 8 workers: per-call cache and IPC overhead dominate, so sharding does not help). Every tool runs fully offline against local reference data — a local UTA database and SeqRepo, with mutalyzer’s network lookups disabled — the configuration ferro’s prepare command enables; the figures therefore reflect compute throughput, not per-variant network latency. Reference-data load is excluded for all tools. ferro full-population peak: parse 20.0M/s, normalize 77.0k/s. See docs/BENCHMARK_RUNBOOK.md for the full method.
Parse
| Tool | Throughput @ 1 worker | Throughput @ 8 workers | ferro speedup @ 8w |
|---|---|---|---|
| ferro | 5.1M/s | 12.2M/s | — |
| mutalyzer | 352/s | single-threaded | 35,000× |
| biocommons | 3.9k/s | single-threaded | 3,100× |
| hgvs-rs | 3.6M/s | single-threaded | 3× |
Normalize
| Tool | Throughput @ 1 worker | Throughput @ 8 workers | ferro speedup @ 8w |
|---|---|---|---|
| ferro | 78.1k/s | 260.2k/s | — |
| mutalyzer | 4/s | 4/s | 73,000× |
| biocommons | 368/s | 818/s | 320× |
| hgvs-rs | 195/s | 1.3k/s | 200× |
ferro thread scaling
| Threads | 1 | 2 | 4 | 8 |
|---|---|---|---|---|
| ferro parse | 5.1M/s | 9.4M/s | 16.0M/s | 12.0M/s |
Input ordering matters for batch throughput. Resolving a transcript (reading its full sequence from the reference and rebuilding its CDS/exon metadata) dominates per-variant cost. ferro memoizes resolved transcripts in a bounded in-memory cache, so repeated lookups of the same transcript are near-free. Providing variants sorted by transcript accession — or by genomic position, which clusters variants onto the same transcripts — maximizes the cache hit rate and can speed up large batches by an order of magnitude versus randomly-ordered input. Ordering matters most when the number of distinct transcripts in the run exceeds the cache capacity (very large or genome-wide inputs); below that, the working set stays resident regardless of order.
Reference Data: What ferro Prepares
The ferro prepare command downloads and organizes all reference data needed for comprehensive normalization. This data is then shared with other tools (mutalyzer, biocommons, hgvs-rs) to enable their local operation.
| Data Type | Source | Size | Enables |
|---|---|---|---|
| RefSeq transcripts | NCBI | ~1GB | NM_/NR_/XM_ normalization |
| cdot metadata | MANE | ~200MB | Transcript-to-genome mappings |
| GRCh38 + GRCh37 genomes | NCBI | ~4GB | NC_ genomic normalization |
| RefSeqGene (sequences + genome alignments) | NCBI | ~600MB | NG_ gene-region normalization; projecting c./n. variants into an NG_ parent’s own g. frame (via the RefSeqGene→genome alignment GFF3) |
| LRG sequences + XML | EBI | ~50MB | LRG_ stable-reference normalization; projecting c./n. variants into an LRG_ parent’s own g. frame (via the LRG XML genomic mapping) |
| Protein sequences | Derived from CDS | ~200MB | NP_/XP_ protein normalization |
| Legacy transcript versions | NCBI | ~50MB | Historical ClinVar variants |
Key insight: Without ferro’s reference preparation, other tools require network access for each variant lookup (adding 100-1000ms latency per variant). With ferro’s cached reference data, all tools can operate fully offline with consistent, reproducible results.
Deriving version-independent NG_ placements (#728)
ferro prepare --derive-ng-placements <accessions.txt> derives genomic placements for the listed NG_ versions (one exact accession per line, e.g. NG_012337.3; blank lines and # comments ignored), writing derived_refseqgene_placements.json into the reference directory and wiring the manifest’s derived_refseqgene_placements field. This fills version gaps the archived RefSeqGene→genome GFF3 snapshots do not cover. It needs cdot + the genome in the same prepare run and uses NCBI EFetch per accession; accessions that cannot be validated are skipped with a warning. The field is preserved across subsequent prepare runs.
Interpreting the HGVS Recommendations
Under construction. Pages are added one HGVS spec page at a time; this section will fill in as they land.
What it is
The HGVS nomenclature recommendations are the standard for naming sequence variants. They are large and carefully written, but — like any natural-language standard — in places they leave a question open, or two passages can be read as pointing different ways. A normalizer still has to emit exactly one string. Where ferro has had to settle such a question, we record the decision (and, where a passage looks inconsistent, file it upstream with the SVD Working Group).
This section mirrors the recommendations page for page and records, per clause, how ferro reads each one. Each page gives, for a span of the spec:
- the spec text it interprets, quoted verbatim;
- ferro’s reading of that clause;
- example spellings, each with the form ferro normalizes it to and a verdict describing that output against the recommendations (see below);
- which spellings converge on one output;
- the reason ferro reads the clause the way it does, and a link to the governing decision.
How to read the verdicts
The recommendations are HGVS’s, not ferro’s. ferro’s job is to take any input and produce the form the recommendations prefer. The verdict describes ferro’s output against that goal:
- recommended — ferro’s output is the form the HGVS recommendations prefer. This is the target, whether the input was already that form or ferro normalized it there.
- conformant — ferro’s output is a valid HGVS string but not yet the recommended form. This is a known ferro limitation, and it links to the issue tracking it.
- refused — the input is not valid HGVS; ferro rejects it in strict mode. Refusing is the correct behavior.
- bug — ferro’s output is not valid HGVS. This should never happen; where a page shows one, it links to the open defect.
What it is not
It is not a second copy of the rules. The canonical ruleset lives in the
normalization rules page, and
every adjudication is recorded once in the ruling ledger
(hgvs_spec_normalization_overrides.json,
rendered for reading as
NORMALIZATION_CONTRACT.md).
These pages draw their explanations from the ledger rather than restating them — a rule
written in two places is a rule that drifts.
How it stays honest
Every example is executed through ferro by a build check: each spec quote is verified against the pinned spec checkout, each spelling is parsed (and strict-refused where marked refused), and each normalized output is asserted against a real GRCh38 reference. A wrong verdict, a moved quote, or a changed normalization fails the build — so a page cannot claim behavior ferro does not have.
Pages
DNA
Substitution — ferro’s reading
ferro’s reading of substitution.md. The rules are HGVS’s; ferro’s job is to produce the form the
recommendations prefer. Verdicts describe ferro’s output:
- recommended — ferro’s output is the form the recommendations prefer (whether the input was already that form, or ferro normalized it there).
- conformant — ferro’s output is valid HGVS but not yet the recommended form — a ferro limitation, with a tracking issue.
- refused — the input is not valid HGVS; ferro rejects it in strict mode (correct behavior).
- bug — ferro’s output is not valid HGVS (a defect). None on this page.
Each Why block is transcluded from the ruling ledger — the record’s own one-line summary, rendered here and linked to its full entry in NORMALIZATION_CONTRACT.md. The reasoning lives once, in the ledger; it is never re-typed here.
substitution.md:5 — definition: one nucleotide for one
one nucleotide is replaced by one other nucleotide
Ferro: a substitution is exactly 1→1; IUPAC ambiguity codes stand in for the single replacing base.
| Input | Verdict | Normalizes to | Notes |
|---|---|---|---|
NM_004006.3:c.76A>G | recommended | self | canonical single-base substitution |
NM_004006.3:c.54G>H | recommended | self | replacement is IUPAC H = A, C, or T |
NC_000023.10:g.33038255C>A | recommended | — | the spec’s genomic example (GRCh37 accession — parse-only here) |
NM_004006.3:c.79GC>TT | refused | — | two nt — violates 1→1; rejected as “deprecated multi-base substitution syntax” |
substitution.md:15 — two or more nucleotides is a delins
two or more consecutive nucleotides are described as deletion/insertion (delins) variants
Ferro: a multi-base replacement is never a substitution; write it as a delins.
Why.
absolute-prohibition-enforcement-stage — Spellings the spec prohibits are rejected — at parse in strict mode; lenient mode instead repairs the input where it can and fails only if it cannot normalize.
| Input | Verdict | Normalizes to | Notes |
|---|---|---|---|
NM_004006.3:c.79_80delinsTT | recommended | self | the two-base change written as a delins |
NM_004006.3:c.79_80GC>TT | refused | — | rejected: “deprecated multi-base substitution syntax” — use delins |
substitution.md:16-18 — separation, and the one-codon exception
two variants separated by one or more nucleotides should be described individually
Ferro: the separation rule (ruleset rule 2); the exception folds two subs into one delins only when they sit one nucleotide apart and together change one amino acid.
Why.
separation-rule-force-modal-or-negation — Two changes a nucleotide or more apart are described individually — this is the spec’s preference (ruleset rule 2), not an outright ban; the only spelling the recommendations forbid is the split at separation zero.
codon-carve-out-shape-restriction — Two changes one nucleotide apart that together alter a single amino acid are written as one delins, whatever the edit types — because “together affecting one amino acid” is a fact about the resulting sequence, not about how the input was spelled.
| Input | Verdict | Normalizes to | Notes |
|---|---|---|---|
NM_004006.3:c.145_147delinsTGG | recommended | self | one-codon delins, CGC→TGG |
NM_004006.3:c.[145C>T;147C>G] | recommended | NM_004006.3:c.145_147delinsTGG | ferro normalizes the split to the recommended delins (warns MEMBERS_COALESCED_FROM_REPORTED_FORM) |
NM_004006.3:c.235_237delinsTAT | recommended | self | the Lys79 codon; the split predicts p.[Lys79*;Lys79Asn], the delins p.Lys79Tyr |
NM_004006.3:c.[235A>T;237G>T] | recommended | NM_004006.3:c.235_237delinsTAT | split → the recommended delins (same codon exception) |
Confluence. Each split and its delins converge on one recommended output —
{c.[145C>T;147C>G], c.145_147delinsTGG} → c.145_147delinsTGG, and the Lys79 pair →
c.235_237delinsTAT.
substitution.md:19 — no change is =, not a substitution
found not changed are described as
Ferro: a tested, unchanged position is =; a no-change substitution is normalized to it.
| Input | Verdict | Normalizes to | Notes |
|---|---|---|---|
NM_004006.3:c.123= | recommended | self | c.123 screened, unchanged (reference C) |
NM_004006.3:c.123C>C | recommended | NM_004006.3:c.123= | a no-change substitution is normalized to the recommended = |
substitution.md:20 — polymorphisms are not A/G
it is not correct to describe
Ferro: the slash form for “polymorphism” is not valid HGVS; ferro rejects it.
| Input | Verdict | Normalizes to | Notes |
|---|---|---|---|
NM_004006.3:c.76A>G | recommended | self | the correct form |
NM_004006.3:c.76A/G | refused | — | the disallowed “polymorphism” slash — rejected at parse |
substitution.md:32 — adjacent substitutions are one delins
changes involving two or more consecutive nucleotides are described as deletion-insertion (delins) so the description
Ferro: at separation zero the split is not correct; two adjacent members that both consume reference
bases are one delins. The same rule licenses re-merging adjacency that per-member 3′ shifting
creates (protein p.[Gly16Ala;Gly17del] → p.Gly16_Gly17delinsAla), keeping normalization
idempotent.
Why.
delins-adjacent-members-when-both-consume-reference — Two adjacent changes that both consume reference bases are written as a single delins; the spec marks the split spelling “not correct” at separation zero.
canonical-form-choice-when-both-legal — When two descriptions of one variant are both legal and no clause chooses between them, ferro derives the form from the resulting sequence rather than preserving the input’s spelling.
| Input | Verdict | Normalizes to | Notes |
|---|---|---|---|
NM_004006.3:c.79_80delinsTT | recommended | self | adjacent c.79,c.80 written as one delins |
NM_004006.3:c.[79G>T;80C>T] | recommended | NM_004006.3:c.79_80delinsTT | ferro normalizes the adjacent split to the recommended delins |
NM_004006.3:c.79_80GC>TT | refused | — | the multi-base substitution spelling — rejected |
Confluence. {c.[79G>T;80C>T], c.79_80delinsTT} → c.79_80delinsTT.
See also → substitution.md:15, substitution.md:16-18.
substitution.md:47-49 — mosaic and chimeric
a mosaic case where at position
Ferro: mosaic (/) and chimeric (//) mixtures are valid; the recommendations write the reference
allele first.
| Input | Verdict | Normalizes to | Notes |
|---|---|---|---|
NM_004006.3:c.85=/T>C | recommended | self | mosaic: reference = written first, then T>C |
NM_004006.3:c.85=//T>C | recommended | self | chimeric: a mix of c.85= and c.85T>C cells |
NM_004006.3:c.85T>C/= | conformant | self | valid, but substitution.md:49 writes the reference first; ferro does not yet reorder this to the recommended c.85=/T>C. Tracked by #2034. |