Andi Marafioti @andimarafioti
Andi Marafioti (@andimarafioti), 10h: This is such a beautiful way to present ablations. Kind of jealous tbh
[Embedded image — "Figure 2: Robust Image Pretraining" bar chart, table format]
Columns: Robustness avg of 6 / ImageNet val | Training ZFLOPs
1. Baseline — 75.3 / 78.9 — 1.0
2. Prog. Res — 75.1 / 78.9 — 0.5
3. Batch Sz — 76.2 / 79.5 — 1.1
4. LAMB — 76.9 / 79.9 — 1.1
5. High Res — 78.3 / 80.4 — 1.2
6. RoPE — 79.2 / 80.7 — 1.2
7. Attn Pool — 80.1 / 81.0 — 1.2
8. Data Aug — 80.8 / 81.1 — 1.2
9. Mask Reg — 80.9 / 81.3 — 1.2
Figure caption: Figure 2 Robust Image Pretraining. We tune our pretraining recipe (§2.1) to maximize performance on a fixed set of data, starting with an OpenCLIP [49] ViT-L/14 model. We report cumulative zero-shot classification results for each modification. The inner bars show robustness evaluation, calculated as the average of 6 robustness benchmarks [4, 24, 44, 45, 109, 138], and the outer bars show ImageNet val [24] alone. Several changes significantly improve robustness, indicating that ImageNet val scales more with data, while robustness can scale with refined training techniques.
Note from Claude Sonnet 5
A tweet praising a research paper's ablation-study figure format (image pretraining robustness ablations from an OpenCLIP-based ViT-L/14 recipe). General ML research/engineering interest, relevant to technical ML methodology rather than AI safety specifically.
machine learningcomputer visionablation studypretrainingcliptwitterml research