BoAChallengeData loads an empty methylation matrix (930659 × 0 samples): parser line numbers off-by-one vs current GSE246337 series matrix

[IsraelAfangideh]

Summary

DataLibrary().get("BoAChallengeData").load() returns a methylation matrix with zero samples — shape (930659, 0) — and scrambled metadata. The cause is that the biomarkers-challenge-2024 parser config for this dataset uses hardcoded series-matrix line numbers that are off by one relative to the current GSE246337_series_matrix.txt, so the betas-column → GSM mapping comes out empty and every sample column is pruned.

This makes the 2024 Biomarkers-of-Aging Challenge dataset unusable as currently shipped.

Reproduction

from biolearn.data_library import DataLibrary
d = DataLibrary().get("BoAChallengeData").load()
print(d.dnam.shape)          # (930659, 0)   <- 0 samples
print(d.metadata.shape)      # (1003, 7)     <- 500 subjects, but 1003 rows
print(d.metadata[["subject_id", "tissue", "race1"]].notna().sum())  # mostly NaN / shifted

Environment: biolearn 0.9.1, Python 3.11.13, pandas 2.2.3, numpy 2.1.3 (also reproduced on pandas 3.0.3 — independent of pandas version).

Root cause

The BoAChallengeData entry in data/library.yaml declares (abridged):

parser:
  type: biomarkers-challenge-2024
  id-row: 31
  matrix-file-key-line: 53
  metadata:
    subject_id: { row: 39 }
    tissue:     { row: 40 }
    sex:        { row: 41 }
    race1:      { row: 42 }
    race2:      { row: 43 }
    ethnicity:  { row: 44 }
    age:        { row: 45 }

But the current GSE246337_series_matrix.txt has these fields one row lower:

Field	config row	actual row (current file)	content at the config row
sample key (id-row)	31	32 (!Sample_geo_accession)	row 31 is !Sample_title
betas-column key (matrix-file-key-line)	53	54 (!Sample_description, the sentrix IDs)	row 53 is !Sample_scan_protocol
subject_id	39	40	!Sample_organism_ch1 ("Homo sapiens")
tissue	40	41	subject id: BoA1
sex	41	42	tissue: blood
race1	42	43	Sex: M
race2	43	44	race1: White
ethnicity	44	45	race2: NA
age	45	46	ethnicity: Non Hispanic

ChallengeDataParser.parse builds the betas-column→GSM mapping via build_column_mapping(file_path, matrix_file_key_line=53, id_row=31), then map_and_prune_columns drops every betas column whose renamed value isn't in that mapping. Reproducing build_column_mapping on the current file:

import pandas as pd
def build_mapping(path, from_key_line, to_key_line):
    m = pd.read_table(path, index_col=0,
        skiprows=lambda x: x != from_key_line-1 and x != to_key_line-1)
    rec = m.to_dict("records")[0]
    return {v: k for k, v in rec.items()} if to_key_line < from_key_line else rec

# CONFIGURED (key=53, id=31):
#   1 entry:  'Standard Illumina protocol' -> 'Genomic DNA from Blood from subject BoA1'
# CORRECTED (key=54, id=32):
#   500 entries:  '207700470022_R08C01' -> 'GSM7866964'  (sentrix array ID -> GSM)

So at the configured lines the mapping is a single nonsense pair (the repeated !Sample_scan_protocol value collapses to one column under pandas), the betas columns (sentrix IDs like 207700470022_R08C01) match nothing, and map_and_prune_columns removes all 500 → dnam ends up (930659, 0).

The same +1 offset shifts every metadata field onto the wrong !Sample_characteristics_ch1 line (e.g. race1 reads Sex: M), and the downstream metadata.combine_first(proteomic_metadata) on a non-matching index is what produces the 1003 rows for 500 subjects.

Suggested fix

Two options:

1. Minimal: bump each hardcoded row in the BoAChallengeData parser config by +1 (id-row: 32, matrix-file-key-line: 54, metadata rows 40–46). With the corrected lines the column mapping resolves to the full 500 sentrix→GSM pairs (verified above).
2. More robust (recommended): key the challenge/GEO-matrix parsers off the !Sample_* tag names (and the characteristics_ch1 key: prefixes) rather than absolute line numbers, since GEO series-matrix headers shift between revisions and will re-break any pinned line numbers. This would also fix the metadata field misalignment structurally.

Happy to open a PR with option 1 (and a small regression test asserting dnam.shape[1] == 500) if that'd be useful.