Published: October 20, 2026 | Competition: RSNA Knee Abnormality Detection | Metric: Macro ROC-AUC (12 Pathologies) | Jump: 0.515 → 0.958 (+4,904 Leaderboard Positions)
In the RSNA Knee Abnormality Detection competition on Kaggle, the goal is to evaluate 12 clinical abnormalities across multi-sequence MRI studies (Sagittal, Coronal, and Axial planes with both fluid-sensitive and non-fluid-sensitive acquisitions):
ACL Tear, MCL Tear, Medial Meniscus Tear, Lateral Meniscus Tear, Medial Osteoarthritis, Lateral Osteoarthritis, Patellofemoral Osteoarthritis, Joint Effusion, Synovitis, Baker's Cyst, Bone Contusion, and Fracture.
Our initial submission scored a modest 0.515 ROC-AUC, hovering at Rank 5,827 out of 6,200+ teams. In medical imaging benchmarks, 0.515 essentially means random chance.
Why did the early baseline fail?
1. DICOM Pixel Compression: Kaggle's hidden test set contains compressed DICOM formats. Without offline decoders (pylibjpeg and openjpeg), standard readers fail silently, causing fallbacks to constant 0.5 predictions.
2. Geometric Misalignment: Medical MRIs are not standard 2D photos. Different MRI scanners use arbitrary slice ordering and coordinate spaces. Without normalizing physical coordinates via ImageOrientationPatient and ImagePositionPatient into canonical LPS (Left-Posterior-Superior) 3D space, models ingest flipped or inverted anatomies.
3. Weak Label Noise: The competition provides complex radiology reports with free text. Standard cross-entropy trained on raw binary labels fails to capture nuanced uncertainty across 12 co-occurring findings.
To bridge the gap between 0.515 and top-tier performance, we orchestrated a multi-stage, multi-family ensemble combining different inductive biases:
- Built on coatnet_rmlp_2_rw_384 pre-trained with Osteoarthritis Initiative (OAI) knee data and soft LLM report labels.
- Constructs an anatomically sorted 96-slice uint8 volume across 5 canonical series slots (Sagittal Fluid, Sagittal Non-Fluid, Coronal Fluid, Coronal Non-Fluid, Axial).
- Samples 94 overlapping 3-slice 2.5D evaluation windows cropped to 320px.
- Incorporates Anatomical Left/Right Mirror TTA: Evaluates both the scan and its anatomical mirror view (reversing sagittal slice channels and horizontally flipping coronal/axial slices), averaging probabilities in prediction space.
- Standalone performance: 0.949 AUC.
- An independent checkpoint trained on a 0.8 center crop resized to 224px.
- Captures localized ligament and joint space details.
- Blended into the Raptor slot at a 35% weight: 0.65 × Rank(384px) + 0.35 × Rank(224px).
- While CoAtNet utilizes full-field global depth slices, the R6 ConvNeXt-Small pipeline uses a 3D SKM-TEA Anatomy V-Net (anatomy.pt) to locate exact anatomical landmarks of the knee joint.
- Extracts centered medial, lateral, and patellofemoral compartments at 352px crop.
- Evaluates across 4 cross-validation folds with Flip TTA in logit space (swapping medial and lateral predictions on flipped passes).
- Employs specialized secondary ridge stackers: a fluid volume model for Joint Effusion and a multi-compartment state model for Baker's cyst.
Because ROC-AUC depends purely on rank order rather than raw probabilities, direct probability averaging can degrade performance when model calibrations differ. Instead, predictions were converted to percentile ranks over non-failed studies.
Crucially, different model families specialize in different knee compartments. We utilized pathology-specific weights:
As a third independent leg, we integrated a 2.5D ConvNeXt-Tiny reader coupled with a 2-layer Transformer Encoder across 6 anatomical plane slots, providing finding-specific attention pooling blended at calibrated residual weights (0.003 to 0.035).
Upon submitting the unified 4-stage pipeline (Version 5), our public leaderboard score jumped to 0.95800, propelling us past 4,904 teams directly into Rank 923.
The 4th Decimal Lesson:
Over 1,390 teams are clustered in the 0.958 band (spanning Ranks 212 to 1,610), where unrounded 4th and 5th decimal places dictate leaderboard rank. In our v5 run, an untested multi-scale resolution shift in Part B (res+32, fov=0.88) introduced an out-of-domain artifact on weights trained at 256px, slightly degrading unrounded precision. Pinpointing and correcting that resolution shift and updating to the V2 blend weights sets the foundation for breaking into the top ~200.
This rapid turnaround highlights how agentic workflows transform competitive machine learning and complex engineering:
1. Automated Diagnostics: Moving beyond blind trial-and-error by reverse-engineering exact error mechanisms (DICOM compression, orientation matrices, calibration mismatch).
2. Agent-Driven Compute: Seamlessly connecting autonomous coding agents (like Antigravity, Claude Code, and Codex) to high-memory remote GPUs (such as Colab NVIDIA A100) via tools like vspagent.
3. Ensemble Diversity: Combining complementary model families (hybrid Convolution-Attention vs. Anchor-based ConvNeXt vs. Study Transformers) consistently outperforms single architectures on complex clinical imaging.