BAM/VCF classifiers ignore file_name — filename-scoped rules never see the real name (~8% of BAM under-classified)

[NoopDog]

Problem

classify_from_header (BAM/CRAM) and classify_from_vcf_header both accept a file_name parameter and never read it. Each unconditionally synthesizes a filename from the extension:

# header_classifier.py:65-67 (BAM)
filename = "sample.bam"
if file_format:
    filename = f"sample{file_format}"

# header_classifier.py:141-143 (VCF)
filename = "sample.vcf.gz"
if file_format:
    filename = f"sample{file_format}"

Every filename-scoped tier-2 rule therefore matches against sample.bam / sample.vcf.gz instead of the real name. Any signal carried by the filename — hifi_reads, rnaseq, CHM13Y, chm1 — is silently discarded for the two most common file types in the corpus.

pipeline.py passes the real file_name into the classifier (line 242), so the data is present and thrown away at the last step.

Reproduction

engine.classify_extended(FileInfo("sample.rnaseq.bam"))  -> data_modality = transcriptomic.bulk
engine.classify_extended(FileInfo("sample.bam"))         -> data_modality = None
classify_from_header("<hdr>", file_name="sample.rnaseq.bam", file_format=".bam")
                                                         -> data_modality = None   # file_name dropped

The committed golden fixture already records the symptom — tests/fixtures/golden/expected_output.json has file_name: sample.rnaseq.bam classifying data_modality: not_classified.

Measured impact

Ran the production classifier bodies over the real cached headers in data/evidence/anvil/, twice per file: once with the synthesized sample{ext} name they use today, once with the real filename. Tier-3 header rules and contig-length detection were active in both arms, so this is the net cost after header signals get their chance to recover the information.

type	headers	output changes with real filename
BAM	16,409	1,371 (8.4%)
VCF	20,000 (sample)	10 (0.1%)

Fields recovered (currently not_classified, would be classified):

BAM   data_modality       1,371   e.g. m84046_230719_001055_s4.hifi_reads.bc2061.bam -> genomic
      data_type           1,371                                                      -> alignments
      assay_type          1,371                                                      -> WGS
      platform              110   e.g. HG02055.paternal.hifi.bam                     -> PACBIO
VCF   reference_assembly     10   e.g. 1kgp.chr20...variants.chm1                    -> CHM13

No value disagreements. In no case does the real filename produce a different value than a header-derived one — it only fills fields that are currently unclassified. Contig-length detection already recovers reference_assembly for nearly all VCFs, which is why VCF barely moves; BAM's loss is concentrated in data_modality/data_type/assay_type, which contig detection does not supply.

So this is a coverage defect, not a correctness one: ~8% of BAM files are left not_classified on three dimensions that the filename would settle. Consistent with accuracy over coverage — nothing wrong is being emitted — but the coverage is recoverable for free.

Caveats on the measurement

• file_size is not stored in the evidence cache, so both arms passed None. Size-dependent assay_type inferences (WGS/WES thresholds) fired in neither arm; with real sizes the assay_type figure could move in either direction.
• VCF was a random 20,000-file sample of 204,692 cached headers (seed 0). BAM was the full cached set (16,409 of 18,663 corpus BAM/CRAM files).

Fix

Prefer the real filename, fall back to the extension, mirroring what classify_from_gfa_segment_tags now does (#151):

if file_name:
    filename = file_name
elif file_format:
    filename = f"sample{file_format}"
else:
    filename = "sample.bam"

Same-class survey

classifier	filename derivation	status
BAM	sample{fmt} — file_name ignored	this issue
VCF	sample{fmt} — file_name ignored	this issue
GFA	file_name -> file_format -> default	fixed in #151
FASTQ	file_name or 'sample.fastq.gz' — file_format ignored	benign today: .fastq/.fq/.fastq.gz/.fq.gz never split across rules
FASTA	file_name or 'sample.fa.gz' — file_format ignored	benign today, same reason
BED	file_name or 'sample.bed'	takes no file_format param; .bed vs .narrowpeak/.broadpeak do split across rules (unified_rules.yaml:795 vs 843), so an empty file_name would apply bed-only rules to a peak file

FASTQ/FASTA are benign only because their extension variants always appear together in a rule's extensions: list. Worth fixing for symmetry, not urgency.

Tasks

• Use file_name when present in classify_from_header and classify_from_vcf_header.
• Regenerate tests/fixtures/golden/expected_output.json — the BAM golden (sample.rnaseq.bam) will start classifying transcriptomic.bulk, which is the point.
• Tests: real filename drives tier-2 rules; file_format fallback when file_name is empty; header/contig signals still win where they should.
• Re-run the corpus and confirm the ~1,371 BAM files move from not_classified, with no value regressions.
• Apply the same fallback to FASTQ/FASTA for symmetry; give BED a file_format param.

Refs: #151 (GFA fix + the same-class scan that surfaced this), #148.

[NoopDog]

See #154 — this is a symptom of a missing abstraction, not an isolated defect.

classify_from_header can drop file_name only because each per-format classifier reconstructs its own view of the file (filename = "sample.bam") instead of receiving one. Under #154's fact model filename is a fact like any other, uniformly present for every type, so there is no line to forget.

The measured ~8.4% BAM coverage loss is worth fixing now regardless; #154 is why it happened.

[NoopDog]

#151 added header_classifier.filename_for_rules(file_name, file_format, default), which is the helper this issue's fix should reuse rather than re-deriving per classifier.

It handles a sub-case not described in the original issue text. The pipeline selects a record when either file_name or file_format carries a matching extension (_filter_records), but the engine reads the extension from the filename only. So a selected record whose file_name lacks an extension loses every extension-scoped rule — and extract_extension('hprc-v1.0-mc-grch38') returns '.0-mc-grch38', a junk suffix from the last dot, rather than an empty string.

The non-obvious part: the rule must be "does the filename already yield a known extension", not "does it end with file_format". 5,227 corpus records are named *.fastq.gz while declaring file_format: '.fastq'; appending there would produce *.fastq.gz.fastq.

Still hand-deriving their filename, and in scope for this issue:

• classify_from_header (bam) — filename = 'sample.bam', file_name ignored
• classify_from_vcf_header — filename = 'sample.vcf.gz', file_name ignored
• classify_from_fastq_header — file_name or 'sample.fastq.gz', file_format ignored
• classify_from_fasta_header — file_name or 'sample.fa.gz', file_format ignored
• classify_from_bed_signals — file_name or 'sample.bed', takes no file_format at all

Route all five through filename_for_rules. Note the fastq classifier is where the *.fastq.gz/.fastq mismatch actually lives, so it is the one most likely to change output.