Working With Your 23andMe Raw Data

Oak
genotyping vcf bioinformatics imputation arrays
A gridded glass chip in a lab instrument with a few amber lit points, above it a projected denser lattice of fainter blue points.

By the end of this you will have your 23andMe genotypes as a bgzipped, reference-checked VCF on both GRCh37 and GRCh38, a PLINK binary set, an optional imputed VCF covering tens of millions of sites, and a clear sense of which questions those files can answer. You need the raw data download from your account, a Unix shell, roughly 20 GB of free disk, and four tools: bcftools (1.19 or newer, with the liftover plugin), plink 1.9, plink2, and either CrossMap or the bcftools liftover plugin. You also need two reference FASTAs (GRCh37 and GRCh38) and the corresponding chain file. Everything here runs on a laptop in under an hour, except imputation, which runs on someone else’s servers.

1. Download the file and keep your own copy

In the web app: profile menu (top right) → Settings, scroll to “23andMe Data”, then “Request your data” and check the “Raw data” box; or Resources → Browse Raw Genotyping Data → Download. You confirm by email, the archive is generated asynchronously, and the download link usually arrives within an hour, sometimes up to 24 hours. The link expires after a few days, so save the file somewhere permanent the moment it lands.

What you get is a single tab-separated text file inside a zip, named something like genome_Jane_Doe_v5_Full_20260112.txt. That file name is the answer to “what is the raw data file called”: it is not a VCF, not a FASTQ, and not a BAM. It is roughly 15–25 MB uncompressed.

Download it now rather than later. 23andMe filed for Chapter 11 in 2025 and the database changed hands, which is a reminder that your access to a hosted account is a business arrangement, not a property right. The consent and intellectual-property questions around direct-to-consumer genetics have been argued in the literature for over a decade, and the short version is that the terms you agreed to govern research use of your sample and derived data in ways most customers never read carefully.1 A local copy of the text file is the only part of this you fully control.

To answer a related question directly: you cannot upload a file to 23andMe. They genotype their own samples and do not accept third-party data. Uploads are a feature of GEDmatch, FamilyTreeDNA, MyHeritage, and various interpretation sites.

2. Inspect the file before you touch it

head -25 genome_v5.txt
wc -l genome_v5.txt

The header is comment lines beginning with #, and it tells you the build. Every 23andMe chip version through v5 reports positions on GRCh37 (hg19), with genotypes given on the plus strand of the human reference. After the header, four columns: rsid, chromosome, position, genotype.

rs4477212	1	82154	AA
rs3094315	1	752566	AG
i713426	1	787173	--

Things to note. Chromosomes are 1–22, X, Y, MT. IDs beginning with i are internal 23andMe probes with no rsID, and they are the ones most likely to be junk. -- means no-call. X, Y, and MT calls in male samples appear as a single letter (hemizygous), which breaks naive parsers that assume two characters.

Quick sanity counts:

grep -v '^#' genome_v5.txt | awk -F'\t' '{n[$2]++} END {for (c in n) print c, n[c]}' | sort -k1,1V
grep -v '^#' genome_v5.txt | awk -F'\t' '$4 ~ /-/ {n++} END {print "no-calls:", n+0}'

Expect on the order of 630,000 markers for a v5 (Illumina GSA-based) chip, around 600,000 for v4, and roughly 960,000 for v3. No-call rates above about 2% suggest a poor sample; 23andMe normally re-runs those, but check.

Check the specific variants you care about before building a pipeline around them. Chip content differs between versions, and a variant that exists in a friend’s v3 file may be absent from your v5 file:

grep -wE 'rs429358|rs7412' genome_v5.txt

If a line is missing, the array does not carry that site. No amount of downstream processing creates data that was never measured. Imputation can estimate it, which is a different thing (step 6).

3. Convert to VCF against a reference

The raw file records your two alleles but not the reference allele, so it is not yet a VCF. bcftools convert --tsv2vcf was written for exactly this format and resolves REF from a FASTA:

bcftools convert --tsv2vcf genome_v5.txt \
  -f human_g1k_v37.fasta \
  -s ME \
  -Oz -o me.GRCh37.vcf.gz
bcftools index -t me.GRCh37.vcf.gz

Use a GRCh37 FASTA whose contigs are named 1, 2, …, X, Y, MT (the 1000 Genomes human_g1k_v37.fasta matches). If your FASTA uses chr1/chrM, either rename the FASTA contigs or pre-edit the text file. Mismatched naming produces a VCF that is empty or silently missing chromosomes, so always check:

bcftools index -s me.GRCh37.vcf.gz

Then verify that REF alleles agree with the reference and that your genotypes contain one of them:

bcftools norm --check-ref w -f human_g1k_v37.fasta \
  -Oz -o me.GRCh37.norm.vcf.gz me.GRCh37.vcf.gz

--check-ref w warns rather than discards, so you can count problems first. A few hundred warnings out of 600,000 is normal (indels, ambiguous probes, multi-mapping regions). Thousands means a build or naming mismatch.

For a PLINK binary set, skip bcftools and use the native reader:

plink --23file genome_v5.txt FAM1 ME 1 --make-bed --out me

The trailing 1 is sex (1 = male, 2 = female), which matters because PLINK will otherwise infer it from X heterozygosity and complain. PLINK’s --23file is convenient and fast, but it does not resolve REF from a FASTA, so allele coding in the resulting .bim follows PLINK’s A1/A2 conventions, not VCF REF/ALT. Use PLINK for association-style work and scoring, and use the bcftools VCF for anything that needs correct reference alleles.

4. Lift over to GRCh38

Most current annotation resources are GRCh38-first. The bcftools plugin is the fastest route:

bcftools +liftover --no-version -Ou me.GRCh37.norm.vcf.gz -- \
  -s human_g1k_v37.fasta \
  -f GRCh38_full_analysis_set_plus_decoy_hla.fa \
  -c hg19ToHg38.over.chain.gz \
  --reject rejected.vcf --reject-type v \
| bcftools sort -Oz -o me.GRCh38.vcf.gz
bcftools index -t me.GRCh38.vcf.gz
wc -l rejected.vcf

Note the naming change: the GRCh38 analysis set uses chr1, chrM. The plugin handles contig renaming from the target FASTA, but confirm the output with bcftools index -s. Expect a few thousand rejected sites, mostly in segmental duplications and regions restructured between builds. Liftover also flips strand for inverted segments and will swap REF/ALT where the reference allele changed between builds, which is precisely why you should not do this with a naive coordinate table of your own.

Keep both builds. Half the tools you will want are still GRCh37-only.

5. Annotate with rsIDs, allele frequencies, and clinical databases

bcftools annotate -a dbSNP156.GRCh38.vcf.gz -c ID \
  -Oz -o me.GRCh38.rsid.vcf.gz me.GRCh38.vcf.gz

bcftools annotate -a clinvar_GRCh38.vcf.gz \
  -c INFO/CLNSIG,INFO/CLNDN,INFO/CLNREVSTAT \
  -Oz -o me.GRCh38.clinvar.vcf.gz me.GRCh38.rsid.vcf.gz

bcftools view -i 'INFO/CLNSIG ~ "Pathogenic"' me.GRCh38.clinvar.vcf.gz \
| bcftools query -f '%ID\t%CHROM:%POS\t%REF>%ALT\t[%GT]\t%INFO/CLNSIG\t%INFO/CLNDN\n'

Read the next paragraph before you read that output.

Genotyping arrays are optimized for common variants. Probes for rare pathogenic variants sit at the edge of the technology’s competence, because cluster separation is estimated from a population in which almost no one carries the alternate allele. Clinical confirmation studies of direct-to-consumer raw data have found that roughly 40% of variants flagged as pathogenic in raw files failed to replicate on sequencing. Treat any pathogenic hit in this output as a hypothesis with a coin-flip prior, not a result. Confirming or acting on a suspected pathogenic variant requires a clinical-grade orthogonal test ordered through a physician or genetic counselor. Also filter CLNREVSTAT to two-star-and-above review status before you spend an evening reading about a one-star submission from 2011.

The same asymmetry explains why this data is so useful in aggregate and so limited individually. Association analyses across millions of consented 23andMe participants have produced genetic evidence that measurably improves the success rate of drug programs.2 That is a statement about statistical power over a cohort. It says nothing about the precision of any single genotype call in your file.

6. Imputation, if you want more sites

Your file covers roughly 0.02% of the genome directly. Imputation uses linkage disequilibrium with a reference panel to estimate the genotypes in between. The TOPMed Imputation Server (r3 panel, ~97k samples) and the Michigan Imputation Server (1000G, HRC) both accept per-chromosome VCFs.

for c in $(seq 1 22); do
  bcftools view -r ${c} -Oz -o chr${c}.vcf.gz me.GRCh38.vcf.gz
  bcftools index -t chr${c}.vcf.gz
done

Requirements that cause most rejected jobs: one chromosome per file, sorted by position, bgzip (not gzip), correct build selected in the form, and contig naming matching the panel (TOPMed wants chr1). Submit with Eagle phasing and the population set to “other/mixed” unless you are confident about ancestry.

What comes back is a per-chromosome .dose.vcf.gz with an R2 INFO field. Filter it:

bcftools view -i 'INFO/R2>0.8' -Oz -o chr1.imputed.r2.vcf.gz chr1.dose.vcf.gz

Above minor allele frequency 5%, a decent fraction of sites will clear R2 > 0.8. Below 0.5% MAF, most will not, which is the whole point: imputation recovers common variation and largely fails on the rare variation that drives monogenic disease. Imputed genotypes are probabilistic estimates and are not a substitute for sequencing at any single site you care about.

7. Polygenic scores done correctly

Download a harmonized scoring file from the PGS Catalog (*_hmPOS_GRCh37.txt.gz), match the build to your PLINK set, and run:

plink2 --bfile me \
  --score PGS000123_hmPOS_GRCh37.txt 1 4 6 header \
  cols=+scoresums,+denom no-mean-imputation \
  --out pgs000123

Columns 1, 4, 6 are variant ID, effect allele, and effect weight in that file; check the header, because layouts vary. no-mean-imputation matters: without it, PLINK fills missing variants with the population mean dosage and quietly inflates your apparent coverage. Compare DENOM in the output to the number of variants in the score file. If you matched 40% of the weights, the score is not comparable to the published distribution and the percentile you compute from it is meaningless.

Two more constraints. Scores are reported in standard deviations relative to a reference population, and most are derived from European-ancestry GWAS, so portability to other ancestries is poor and the direction of the bias is not predictable. And a percentile is a population statement, not a personal prediction.

8. Ancestry estimates, read carefully

Your raw file supports ancestry inference (ADMIXTURE, or a projection onto 1000 Genomes principal components via plink2 --pca), and the hosted report gives you percentages to two decimal places. Those percentages are model output conditioned on a reference panel and a set of labels chosen by the vendor, and the labels are social categories with historical baggage attached. How people read and re-read those numbers as identity claims has itself become a subject of study.3 Run the PCA if you want it, and interpret the output as a statement about allele-sharing with reference panels.

Common problems

Genotype column has one character instead of two. Hemizygous X/Y/MT calls in male samples. Handle explicitly rather than dropping the lines, or you lose all mitochondrial data.

bcftools convert --tsv2vcf outputs almost nothing. Contig naming mismatch between the text file (1, MT) and your FASTA (chr1, chrM). Check with bcftools index -s.

High --check-ref warning counts. Usually the wrong build (you fed a GRCh38 FASTA to a GRCh37 file). Every position will be off by a variable offset and REF will mismatch nearly everywhere.

Imputation server rejects the job. Un-sorted VCF, gzip instead of bgzip, multiple chromosomes per file, or a build/panel mismatch. The error page names the failing check.

A PRS percentile looks implausible. Check DENOM. Low variant match rates and mean-imputation of missing dosages produce confident-looking garbage.

Pathogenic ClinVar hit in your raw data. Do not act on it. Array probes for rare variants fail often, and confirmation belongs in a clinical lab with a clinician interpreting the result.

CYP2D6, HLA types, CNVs, repeat expansions, or anything structural. The array cannot measure these. CYP2D6 in particular involves copy number, hybrid alleles, and a pseudogene, and no combination of the SNPs on your chip resolves star alleles reliably. Third-party interpretation sites that report them from uploaded raw data are extrapolating.

No sequence data at all. The file contains genotypes at a fixed set of chosen positions, not your genome. There is no BAM, no FASTQ, and no way to derive them. The interpretive work of turning a partial molecular readout into something meaningful about yourself is real work, and it has been described well as a burden the technology quietly transfers to the person holding the file.4 Know which part of that burden the file can support.

Oak builds longitudinal molecular profiles of individuals: whole-genome sequencing, RNA sequencing, proteomics, blood biomarkers, and continuous glucose data, integrated into one model of you. Build your profile.

Footnotes

  1. Megan Allyse. 23 and Me, We, and You: direct-to-consumer genetics, intellectual property, and informed consent. Trends in Biotechnology, 2013. https://doi.org/10.1016/j.tibtech.2012.11.007 ↩

  2. Xin Wang, Sotiris Karagounis, Suyash S. Shringarpure, et al. The impact on clinical success from the 23andMe cohort. 2024. https://doi.org/10.1101/2024.06.17.24309059 ↩

  3. Yukun Yang. r/23andMe and my pictures: Platform racial spectacles as collective racial identity formation. New Media & Society, 2025. https://doi.org/10.1177/14614448251344291 ↩

  4. Sandra Soo‐Jin Lee. Excavating the Personal Genome: The Good Biocitizen in the Age of Precision Health. Hastings Center Report, 2020. https://doi.org/10.1002/hast.1156 ↩