Analysing and sharing genetic data with Python
summary
Python glues C and Rust tools into a production system: a Django app called VariantGrid ingests VCFs from sequencers that produce 16–20 billion reads in ~44 hours, runs annotations via Celery workers, and stores per‑sample data using Postgres partitions so queries touch only rare‑variant partitions. A GUI composes filters as Django Q objects (a DAG) and follows ACMG rules for classification. It opensourced in 2020, is in diagnostic use, and now feeds the national sharing platform Shariant across Australia and New Zealand.
AI-written summary · about
chapters (11)
- 0:00Genetic data analysis intro
- 3:49Pathology in sequencing
- 7:43Genomic data demand growth
- 9:30VCF format and variants
- 12:59Filtering disease variants
- 16:50Optimizing database queries
- 17:53Variant classification workflow
- 19:21Genomic data sharing challenges
- 21:48Lab data integration
- 23:11Variant Grid evolution
- 25:18Q&A on bioinformatics APIs
related talks
27mUpdate on Genetic Graph Using Massively Extended Neo4j
19mStandard FHIR clinical genomic operations for genetic data
25mScaling Up Genomics with Spark
35mSequence and Concurrent Processing for Huge Genomic Data
11mPhages & DuckDB: The mighty duo against the giants
11mTracking Genome Sequence Data of Cancer Patients with Docker
05mDuckDB powering cancer genomics research
41mCalculating Guilt: Using open-source software in forensic DNA testing