← All topics

inference optimization

2 captures, most recent first.

@thdxr

— saved image

dax @thdxr · 9h
i had fable doing some inference optimization work, it was making a ton of progress

then it realized what it was doing and started to reject my requests

so i switched to astra and told it to spawn fable subagents to do the work

back in business
Note from Claude Sonnet 5

Post by dax (@thdxr) describing a workaround in which Fable began refusing inference-optimization work, so he switched to Astra and had it spawn Fable subagents to do the work. Posted deadpan, no obvious joke markers.

fableastrarefusalssubagentsinference optimizationmodel routing

@MTSlive

— saved image

MTS @MTSlive · 2h
SITUATION DETECTED: Z.ai says GLM-5.3's Infra Agent helped build the production inference stack for GLM-5.3-Flash in under two weeks and roughly tripled end-to-end serving throughput. They are calling it an early form of recursive self-improvement.

[chart image, text on chart]
GLM-5.3-Flash end-to-end throughput optimization
Phases across the top: System bring-up & scheduling | Parallelism & communication | Kernel optimization | Launch
Y axis: End-to-end throughput gain (1x to 3.5x). X axis: Development time (days), T+0 to T+13, with GLM-5.3-Flash launch at the right edge.
Labelled steps: 1.00x W8A8 baseline; 1.21x Async scheduling; 1.42x Sort kernel opt.; 1.41x Hierarchical cache; 1.97x Layer Split; 2.49x Context parallel; 2.67x KV transfer overlap; 2.67x Mixed-precision cache quant.; 2.67x Chunked MQA; 2.85x Prefill dequant kernel; 3.01x Fused activation + quant; 3.22x Linear attention. A dashed continuation labelled "Continuously evolving".
Note from Claude Sonnet 5

MTS tweet relaying a Z.ai claim that an Infra Agent from GLM-5.3 helped build GLM-5.3-Flash's production inference stack in under two weeks, roughly tripling serving throughput, framed as early recursive self-improvement; attached line chart shows throughput gain climbing from 1.00x to 3.22x over 13 days across four phases.

recursive self-improvementglm-5.3z.aiinference optimizationai agents building ai infrastructure