Multimodal fusion
Integrating genetics, imaging, clinical assessments, molecular bio-specimens, and wearable data from diverse populations to build comprehensive disease profiles.
Clinician workflow
How clinicians use multimodal insights
Our workflow takes multimodal patient data, discovers population-level subgroups, places each new patient into the nearest subgroup, and delivers interpretable decision support to the treating neurologist.
Cluster identification
Generative population modeling uncovers latent biomarker associations, grouping patients into distinct mechanistic subtypes such as tremor-dominant (slow progression), PIGD (rapid progression), and cognitive-behavioral clusters.
Patient application
Population-level knowledge translates into individual care, providing the neurologist with differential diagnosis support, prognostic trajectory insights, and therapeutic trial stratification.
The twin approach
Translational AI enables patient-level precision therapeutics and support without replacing clinical judgment, bridging the gap between large-scale population learning and individualized decision-making.
Population subgroup discovery and patient placement
Rather than treating every Parkinson's patient identically, we use multimodal data to discover population-level subgroups: clusters of patients who share biology and disease trajectory.
When a new patient arrives, their multimodal features are projected into this population map. The system identifies the nearest subgroup and retrieves comparable patients, providing the clinician with outcome context drawn from real cases.
The pipeline
1
Input data
Imaging, wearable, clinical, and biospecimen features from the patient encounter.
2
Subgroup placement
The patient is mapped into a population-derived subgroup that shares biology and trajectory.
3
Similar patients
Nearest-neighbor lookup surfaces comparable cases and their longitudinal outcomes.
4
Decision support
Monitoring priorities, pathway burden, and intervention context presented to the clinician.
What the clinician sees
- Subgroup label
- A biomarker-defined cluster assignment with population context, indicating which patient group the individual most closely resembles.
- Nearest neighbors
- The closest patients in the population map with their longitudinal trajectories and treatment outcomes.
- Pathway burden
- Motor, cognitive, and autonomic pathway contributions weighted by the patient’s multimodal signature.
- Monitoring priorities
- Domain-specific flags highlighting which measures warrant closest follow-up and suggested assessment intervals.
Explore the interactive workflow in the patient portal: pick a participant record and watch the modeled gait on an anatomical figure.
Workflow 1
Generative latent-space modeling
Workflow 1 constructs harmonized latent representations from imaging, diffusion, and clinical modalities using scalable, robust Bayesian co-clustering. Subject-level matrices of region-specific imaging biomarkers (DTI, DaT-SPECT, T1 MRI) and clinical scores (UPDRS, MoCA, UPSIT, SCOPA-AUT) feed into the SRVCC framework.
Core stages
- Data harmonization. Skull-stripping, registration, z-score normalization, and wearable signal filtering produce aligned feature spaces with shared metadata.
- Latent embedding. Dual encoders with Gaussian-mixture priors learn patient and feature embeddings; a joint latent captures cell-level interactions.
- Alignment losses. Mutual-information and compositional KL regularizations ensure diffusion and clinical manifolds stay in register.
- Subtype discovery. The resulting latent checkerboard reveals severity-aligned clusters that inform treatment trajectories and cohort stratification.
Outputs
- Multimodal latent codes exported for downstream policy learning and simulation.
- Quality-controlled, analysis-ready tables for replication and external validation.
- Diagnostics that flag outliers and monitor modality drift.
Workflow 1 underpins the biomarker program by supplying stable, interpretable state estimates rooted in multimodal evidence.
Workflow 2
Clinician-centered decision and visualization pipeline
Workflow 2 operationalizes latent inferences for neurologists through interactive tooling. Outputs from Workflow 1 flow into decision-support dashboards, motion visualization, and sensor-clinical correlation scores that support shared decision-making.
Experience design
- Patient timelines. Cross-visit overlays align gait, arm swing, and UPDRS-III subscores, exposing deviation from subtype baselines at a glance.
- Motion exploration. Animated gait and arm-swing reconstructions reveal asymmetries that standard in-clinic tests miss, with controls for patient selection, animation speed, and sensor segment inspection.
- Cohort intelligence. Sensor-clinical correlation dashboards highlight divergence events, while regulatory-ready documentation tracks provenance for audits.
Deployment pipeline
- Ingest harmonized multimodal outputs from Workflow 1.
- Validate dosage safety, contraindications, and sequencing within policy recommendations.
- Stream personalized insights to clinician workstations and remote collaborators through secure UT Austin infrastructure.
This workflow closes the loop from data harmonization to bedside impact. Neurologists gain interpretable, case-ready insight backed by latent modeling, while patients benefit from personalized, continually updated intervention plans.