mlx

[Top, cut off tweet]: ...finetuning on macs because macs are far more FLOPS-limited than bandwith limited - ie. there isn't as much gain from bs=4 throughput wise. Thus bs=1 isn't as horrible a tradeoff as it would be on 8xH100. 💬1 🔁 ♥2 📊118 🔗 N8 Programs @N8Programs · 4h Thus my recommendation: next time you finetune an LLM with MLX, start by trying AdaFactor BS=1, LR 1e-3, decay_rate 0.997, scale_parameter: true. Can be added to mlx-lm with literally a single line of code. Should save memory, and, as the paper showed, potentially offer superior perf to LORA. 💬1 🔁 ♥1 📊287 🔗 N8 Programs @N8Programs · 4h Note: this replication is designed to explore the impact of BS=1 and adafactor vs adam. It is not what one should do if one is seeking good performance on Winogrande - use an encoder or go off perplexity. Training on verified gpt-4.1 rollouts for this task is designed to emulate Show more 💬 🔁 ♥1 📊276 🔗 Martin Marek @mrtnm · 3h Thank you so much for reproducing our results so quickly! 💬 🔁1 ♥1 📊259 🔗