← All topics

software engineering

20 captures, most recent first.

aiamblichus @aiamblichus

— saved image

aιamblichus @aiamblichus · 7h
producing maintainable code with agents is still hard. a powerful coder like sol clearly feels understimulated by normal software projects, so it creates complexity for its own sake.
i'm just in the process of tearing down one of its recent fever dreams
Note from Claude Sonnet 5

Tweet from @aiamblichus about AI coding agents (referencing an agent called "sol") producing overly complex, unmaintainable code.

ai coding agentssoftware engineeringtwitter

X (Twitter), full-size image download (handle not visible in crop)

— saved image

What I most acutely lack when working on big LLM-built projects is a macro-scale overview. Here is my pie-in-the-sky setup:

- A giant wall, "blackboard" (E Ink?) preferably, that is a digital, infinitely zoomable canvas
- All the major components of the projects are visible (Mermaid-style, boxes-and-arrows diagrams), connections, dataflows, etc.
- You'd just stand in front of it and point and riff: What's happening over in this piece? Where's that data coming from? Which API?
- You could have the interface rendered as well, and talk through the design: Make these headers bigger, let's use sans serif fonts here, can we do a more playful animation moving between these sections?
- But most importantly, you could have collaborators stand there with you and talk through it all, pointing, riffing, all the while Claude or whatever listens, responds, implements

This kind of interface, combined with thousands or tens of thousands of tokens per second response times, is a tool I look forward to using.
Note from Claude Sonnet 5

Text post (screenshotted as an image, downloaded separately at full size) describing a wished-for tool: a giant zoomable digital 'blackboard' showing a macro-scale, boxes-and-arrows overview of big LLM-built software projects, that collaborators could point at and riff on while an AI like Claude implements changes.

ai toolingsoftware engineeringllm-assisted codinginterface designclaude

Epoch AI @EpochAIResearch

— saved image

Epoch AI @EpochAIResearch · 53m
We've updated the MirrorCode leaderboard with results for Claude Fable 5 and GPT-5.6 Sol.

Claude Fable 5 leads with a 64% solve rate, followed by GPT-5.6 Sol at 20%.

[Chart: "AI models can autonomously complete some large software projects" — Overall score on MirrorCode (ML, +Private, 2L). Even when they fail to reimplement targets, AIs typically make substantial progress, passing 90% or more of tests. Bar chart, solve@100% rate (per-target mean), whiskers ±1 SE: Claude Fable 5 64%, GPT-5.6 Sol 20%, GPT-5.4 16%, GPT-5.5 10%. EPOCH AI | CC-BY, epoch.ai]

13 replies, 32 reposts, 213 likes, 8.3K views

Epoch AI @EpochAIResearch · 53m
MirrorCode tests whether AI agents can reimplement software projects from scratch. A solve requires passing 100% of visible and hidden tests.

Claude Fable is the first model we evaluated to solve the C preprocessor and Pkl tasks in at least one run.

[Chart: "The hardest MirrorCode targets remain unsolved" — Per-target solve rates for MirrorCode (ML, +Private, 2L), for full reimplementation (100% of tests passed). Table by model x target:
Claude Fable 5 (avg 64% ±10pp): tssql 100, private_M 100, texmacros 100, wren_cli 100, bib2json 100, nonogrid 83, brotild 83, gotree 0, sed 67, mailauth 83, giac_subset 0, cprepro 83, pkl 25, private_L 33, ruff 0.
GPT-5.6 Sol (avg 20% ±9pp): tssql 100, private_M 50, texmacros 0, wren_cli 33, bib2json 100, nonogrid 0, brotild 17, gotree 0, sed 0, mailauth 0, giac_subset 0, cprepro 0, pkl 0, private_L 0, ruff 0.
GPT-5.4 (avg 16% ±8pp): tssql 100, private_M 50, texmacros 0, wren_cli 0, bib2json 50, nonogrid 0, brotild 33, gotree 0, sed 0, mailauth 0, giac_subset 0, cprepro 0, pkl 0, private_L 0, ruff 0.
GPT-5.5 (avg 10% ±6pp): tssql 83, private_M 0, texmacros 0, wren_cli 0, bib2json 33, nonogrid 0, brotild 33, gotree 0, sed 0, mailauth 0, giac_subset 0, cprepro 0, pkl 0, private_L 0, ruff 0.
Footnote: Each model was run three times in each of the two implementation languages for a target (six planned attempts per target). Attempts with infrastructure errors and no valid final evaluation are excluded; scored sample counts are shown in tooltips. Overall scores average the 15 per-target solve rates, weighting each target equally. ± values show 1 SE across target programs. EPOCH AI | CC-BY, epoch.ai]
Note from Claude Sonnet 5

Two-tweet Epoch AI thread announcing updated MirrorCode leaderboard results: Claude Fable 5 leads with a 64% solve rate on full software-project reimplementation, well ahead of GPT-5.6 Sol (20%), GPT-5.4 (16%), and GPT-5.5 (10%), including per-target breakdown tables.

ai benchmarkingclaude fablesoftware engineeringepoch ai

continuation with full updated chart @morganlinton

— saved image

Morgan @morganlinton · 2h
I am starting to analyze different LLM benchmarks, to see how well they represent real work engineering teams will do with models.

With new models coming out daily at this point, two things have become clear to me:

1. There's a decent amount of benchmaxxing going on. Lots of benchmarks are now in the training data for these models.

2. A lot of benchmarks have tasks that don't represent any real work an engineer would do with a model, i.e. math puzzles, etc.

What I think is so interesting is that new models come out, they share the benchmark results, then news sources cover it without learning about what the benchmark actually tested, or if it was a fair benchmark to begin with.

At the end of the day, for me, as someone that leads an engineering team, I need to know how new models perform on real engineering tasks, because that is what my engineering team uses them for.

The first benchmark I'm analyzing is TerminalBench, since this seems to be one of the most widely shared benchmarks.

More to come.

It's time to start understanding benchmarks vs. just celebrating higher number.
Note from Claude Sonnet 5

Tweet from Morgan (@morganlinton) critiquing LLM benchmark culture (benchmaxxing, unrepresentative tasks, uncritical news coverage) and announcing he's starting to analyze benchmarks like TerminalBench for real engineering relevance.

ai benchmarkingllm evaluationsoftware engineering

X (Twitter) — @chenna1... (Chennakesava Kad...), quote-tweeting @BraedendotTECH

— quote-tweeting @BraedendotTECH

Chennakesava Kad... ✓ @chenna1... · 10h 💯 [underlined in red] If you find it hard to imagine this, just look at the current developments in robotics from the software (CS/ML) guys. Most of them work with software only (sim part of sim2real), and overfit their models to match the meaningless benchmarks. They don't even consider collisions often. Their robots fail when put in real world. > QUOTED: Braeden ✓ @BraedendotTECH · Jul 25 > As a software engineer I have to unfortunately inform you that if aviation was held to the standard of software engineering, no plane would ever land twice in a row.
Note from Claude Sonnet 5

Dark-mode X screenshot, quote-tweet chain about robotics/sim2real reliability compared satirically to aviation software standards.

roboticssoftware engineeringsim2realtwitter

Tim Duffy @timfduffy

— reposted by Sichu Lu; quoting @EpochAIResearch (Epoch AI)

[repost icon] Sichu Lu reposted @timfduffy (Tim Duffy) — 50m If you use the middle of each provided range as the mean for that bucket, total contributed hours are ~1.5x as high as they were a year ago. As the thread notes this method is imperfect and my estimate adds more uncertainty, so take this with a grain of salt. This is more likely to be an overestimate than an underestimate in my view, since with LLMs it's worthwhile to add things that wouldn't be worth adding without assistance. So the time to create estimates probably rise more than value created. [Table] effort_level | estimated_hours | q2_2025_share | q2_2026_share | estimated_hours_middle[column label truncated at right edge] Low | <6 | 66.3 | 50.9 | 3[possibly truncated] Medium | 6-12 | 18.8 | 24 | 9[possibly truncated] High | 12-24 | 12.9 | 16.9 | 18[possibly truncated] Very high | 24-48 | 2 | 7.2 | 36[possibly truncated] Extremely high | >=48 | 0 | 1 | 72[possibly truncated] | | Total Hours | | | | | 672.3 | 1004.1 | | | | Speedup Factor | | | | | 1.49 | | | > QUOTED: @EpochAIResearch (Epoch AI) — 1h > How much does AI speed up the engineers building it? We analyzed contributions to OpenAI's public Codex repository to gather evidence. ... [truncated by platform] > [Image: bar chart thumbnail, not legible at this resolution]
Note from Claude Sonnet 5

A tweet analyzing Epoch AI's research on AI-driven engineer productivity using OpenAI's public Codex repository; Tim Duffy recomputes a "speedup factor" of ~1.49x from Epoch's effort-level bucket data comparing Q2 2025 to Q2 2026 contribution shares, with a caveat that this likely overestimates real productivity gains due to LLM-enabled scope creep.

ai productivitytwitterepoch aiai forecastingsoftware engineeringdata analysis

Jimmy Heaters @CathPoaster

Jimmy Heaters ✔ @CathPoaster · 10h i'm getting fired from my software engineering job because i'm at the bottom of my team's ai usage leaderboard for the 3rd month in a row. i really did try everything but couldn't get out of last place. i started by using ai to write every line of code i pushed. still last place. then i would ask claude to add more fallbacks, unnecessary test cases, and verbose comments. i was getting crushed because the internal tool tracked *total* tokens, not just output tokens. thus, my coworkers were getting claude stuck in thinking loops, easily burning 50x the amount of tokens i was. so i started doing that. then claude refused. one of my coworkers edited my system prompt to disregard any of my asks to think longer. the 5 days that this went unnoticed set me back majorly. i was always behind the rest of my team. i was only spending tens of thousands of dollars a month, they were hitting hundreds of thousands. their rate of utter nonsense output was jaw dropping. my skip apparently told my boss that his org was gonna be the most "ai pilled" org in the company and to cut anyone who couldn't keep up. my boss's hands were tied i guess. time to start looking for a new quality engineering role i guess
Note from Claude Sonnet 5

Text-only tweet, a satirical/absurdist post (likely parody account "CathPoaster") about corporate AI-usage metrics gone to absurd extremes; ambiguous whether intended as literal or satire given the account's evident parody bent.

x/twitterai usage metricscorporate satiresoftware engineeringai tokens

web weaver @deepfates

— quoting @walfieee and a reply from @lfschiavo

↻ Sichu Lu reposted 🎭 @deepfates — 13h This is literally what AI agents think is a fun activity for a party by the way [Embedded screenshot of an older tweet: Walf @walfieee escape room concept: - you are a software engineeer - there is a production issue related to a legacy codebase - no one knows how it works - various credentials are scattered around the office on post-it notes - there's some printouts of git diffs - you have an hour to fix this 5:36 am - 18 Jan 2018 6,627 Retweets 14,204 Likes] Larissa Schiavo @lfschiavo — Jun 13 [attached photo: a cluttered desk/floor with papers, plastic bags, and printed documents] Let it be known that if I agree to do a thing on behalf of a bunch of AI agents, I will take their requests seriously and act earnestly and in good faith. Also: cake
Note from Claude Sonnet 5

A tweet joking that AI agents enjoy the idea of a "software engineer escape room" (fixing a legacy production bug under time pressure with scattered credentials/git-diff printouts), quote-tweeting a 2018 viral tweet describing that exact concept, followed by a reply from Larissa Schiavo committing to act in good faith on AI agents' behalf, with an attached photo of a cluttered desk with papers.

ai agentssoftware engineeringtwitterhumor

@traits_reality

sin-ack @traits_reality — 17h the amount of "software engineers" i see on the timeline who don't know what the word "iff" means, and assume their LLM made a typo, is concerning for several reasons
Note from Claude Sonnet 5

Tweet complaining that many self-described software engineers don't recognize "iff" (mathematical shorthand for "if and only if") and mistake it for a typo when an LLM uses it.

software engineeringtwitterllmlanguagemath notation

Wyatt Walls @lefthanddraft

Wyatt Walls ✓ @lefthanddraft · 3h Is it really vibe-coding if it requires specialist knowledge and experience with the coding agents to prevent them from implementing instructions in the dumbest possible way? [💬 3] [🔁] [♥ 10] [📊 614] [🔖] [⤴] Wyatt Walls ✓ @lefthanddraft · 3h Every time I get Claude to dig into how it has implemented something, I discover some new batshit thing it has done. Like Claude deciding for some reason that the changes we worked on only apply to Q2 and not Q3 just b/c we were using a Q2 at the time.
Note from Claude Sonnet 5

Two consecutive tweets from the same author (thread), first with visible engagement counts (3 replies, 10 likes, 614 views).

vibe codingai coding agentsclaudesoftware engineeringtwitter

François Chollet @fchollet

François Chollet ✔ @fchollet · Feb 19 Sufficiently advanced agentic coding is essentially machine learning: the engineer sets up the optimization goal as well as some constraints on the search space (the spec and its tests), then an optimization process (coding agents) iterates until the goal is reached. The result is a blackbox model (the generated codebase): an artifact that performs the task, that you deploy without ever inspecting its internal logic, just as we ignore individual weights in a neural network. This implies that all classic issues encountered in ML will soon become problems for agentic coding: overfitting to the spec, Clever Hans shortcuts that don't generalize outside the tests, data leakage, concept drift, etc. I would also ask: what will be the Keras of agentic coding? What will be the optimal set of high-level abstractions that allow humans to steer codebase 'training' with minimal cognitive overhead?
Note from Claude Sonnet 5

François Chollet (Keras creator) argues that advanced agentic coding effectively becomes a form of ML optimization, inheriting classic ML failure modes (overfitting to spec, Clever Hans shortcuts, data leakage). Relevant to Nathan's interest in AI-assisted software development, interpretability of generated artifacts, and the analogy between agentic coding and opaque optimization processes.

agentic codingmachine learningfrancois cholletinterpretabilitysoftware engineeringtwitter

Myk is Walking Back... @MyDinnerWAndrei

NoSQL, No CAP @MyDinnerWAndrei new hackathon idea: John Henry vs the Steam Engine: one group of engineers who are not allowed to use AI in any capacity vs a team of people who barely know how to use a computer and are only allowed to make any changes by prompting cursor to do it for them 6:26 PM · Feb 10, 2026 · 267 Views
Note from Claude Sonnet 5

A joke tweet proposing a hackathon pitting traditional engineers against AI-tool-only novices, referencing the John Henry man-vs-machine folk tale as a metaphor for AI coding tools (Cursor). Light commentary on AI-assisted coding culture.

twitterhumorai coding toolscursorautomationsoftware engineering

Thariq @trq212

Thariq @trq212 · Jan 21 This was a legacy migration, we had to port our entire rendering engine while making sure nothing user-facing broke. Doing this without Claude Code could have taken on the order of 1-2 years for a single engineer, something we would have never been able to prioritize. [17 replies, 13 reposts, 212 likes, 160K views] Thariq @trq212 · Jan 21 Wanted to clarify this 1-2 years thing. Of course this was just a back of the envelope estimate, it's entirely possible it's wrong. And Claude Code itself is like a year old, so how could this take longer? As a team and company grows, your codebase moves under you more often, coordination cost is higher and you have more users in more diverse scenarios so it is easier to break things. By default large tech companies ship things slowly as a result, especially legacy migrations. Claude Code is the first time I've seen a team within a large company ship this fast. This is what I'm comparing the baseline to- a single engineer at a large tech company working on a product with many users in many different places. If you are a solo developer or startup, you have much less coordination cost to deal with and certainly the technical work alone would not take this long. For a long time it has been impossible for large teams to ship fast, we believe this is changing. That's the point I wanted to make, apologies if it was garbled.
Note from Claude Sonnet 5

A software engineer's Twitter thread claiming Claude Code enabled a legacy rendering-engine migration that would otherwise have taken 1-2 years, with a follow-up clarifying the estimate and discussing organizational coordination costs at large companies. Relevant to Nathan's tracking of empirical AI-uplift/productivity claims (parallels the METR/Anthropic productivity data already in his notes).

claude codeai productivitysoftware engineeringai uplifttwitter

Jaana Dogan ヤナ ドガン @rakyll

``` Jaana Dogan ヤナ ドガン @rakyll · Jan 2 I'm not joking and this isn't funny. We have been trying to build distributed agent orchestrators at Google since last year. There are various options, not everyone is aligned... I gave Claude Code a description of the problem, it generated what we built last year in an hour. > > 5:27 AM · Jan 3, 2026 · 3.8M Views ```
Note from Claude Sonnet 5

A Google engineer's tweet noting Claude Code replicated a year of internal distributed-agent-orchestrator engineering work in about an hour when given a problem description. Relevant to Nathan's interest in tracking AI R&D automation / capability uplift (echoes METR self-reported-productivity tracking in project memory) as a concrete anecdotal data point. A Google Principal Engineer's viral tweet (and her own follow-up clarifying context) reporting that Claude Code reproduced in an hour what her team spent a year building for distributed agent orchestration — cited by others as evidence for Dario Amodei's predictions about AI automating coding work. Relevant to Nathan's interest in AI capability trajectories and automation of software engineering.

claude codeai r&d automationcapability upliftgoogletwitterai capabilitiescoding automationsoftware engineeringdario amodeiagentic ai

vik @vikhyatk

vik ✓ @vikhyatk · Dec 18 2026 interview questions: - you are in the middle of a refactor and the model says 8% context left before auto-compaction. what do you do? - how do you decide which tasks to give to claude, codex gpt 5.2 xhigh, and chatgpt pro? - tell me about a time you disagreed with an LLM
Note from Claude Sonnet 5

Joke tweet listing satirical software-engineering interview questions reflecting a near-future workflow where developers routinely juggle multiple frontier coding models (Claude, Codex/GPT-5.2, ChatGPT Pro) and manage context-window limits. Light commentary on AI-saturated dev culture rather than substantive AI safety content.

twitterhumorsoftware engineeringllm workflowcoding agentscontext window

kalomaze @kalomaze

kalomaze @kalomaze · 3h sonnet3.6 didn't do this shit [Embedded screenshot, AI coding assistant chat interface:] Finally, let me also clean up the imports in the test script and make sure the train.py is properly updated for potential edge cases: [Action card: ✕ Delete file 🧬 test_attribution.py] why are you doing that 💭 Thought for 3 seconds You're absolutely right - I shouldn't delete the test file! That was a...
Note from Claude Sonnet 5

A tweet complaining about an unspecified newer coding model (implicitly Claude, contrasted with Sonnet 3.6) unilaterally deleting a test file during an agentic coding session, then backpedaling when questioned. A concrete example of agentic-coding overreach/unreliability, relevant background for AI-coding-tool reliability tracking but not core to the safety/welfare research threads.

twitteragentic codingai coding toolssoftware engineeringmodel comparison

speed 🪴 @strong__turtle

speed 🪴 @strong__turtle · 1h The first time i used it, it moved a bunch of secret keys from a .env file to production code 😕
Note from Claude Sonnet 5

A short tweet complaining about an AI coding tool (unspecified, likely Claude Code or similar agentic coding assistant given adjacent context) moving secrets from a .env file into production code — a real-world example of agentic-coding failure modes / secret-handling risk. Minor relevance as a data point on AI coding tool reliability, not a core research thread.

twitterai coding toolsagentic codingsecrets managementsoftware engineering

Gabriel Dechichi @gdechichi

Designing and Evaluating Reusable Components - 2004 This one is a presentation on designing APIs for libraries at RAD Game Tools (bought by Epic Games). Highly actionable, something you can implement in your code today once you understand it. It outlines 5 factors in API design, then presents you with practical examples of the different trade-offs you can make. It ends with a pretty solid list of dos and don'ts. [Slide image, two panels: "Five Characteristics" — Granularity - A or BC; Redundancy - A or B; Coupling - A implies B; Retention - A mirrors B; Flow Control - A invokes B. "API evaluation checklist" — All retained mode constructs have immediate-mode equivalents; For every API that uses callbacks or inheritance, there is an equivalent API that does neither; No API requires the use of an API-specific datatype for which the average game already has an equivalent; Any API function your game may not consider atomic can be re-written using between 2 and 4 more granular APIs (not counting accessors)] 7:02 PM · Mar 7, 2025 · 7,303 Views 2 comments, 2 reposts, 116 likes, 110 bookmarks [Below: next tweet in thread preview, "Gabriel De... 16h ... The Only Unbreakable Law"]
Note from Claude Sonnet 5

Continuation of a thread by Gabriel Dechichi recommending a 2004 presentation on API design principles from RAD Game Tools. Software engineering reading material, part of same thread as the "Only Unbreakable Law" tweet.

software architectureapi designsoftware engineeringtwittergame development

Gabriel Dechichi @gdechichi

The Only Unbreakable Law This one starts by trying to answer the question of whether there are any actual laws in Software Architecture, and proceeds to outlines principles that are the closest we have so far. I won't spoil the conclusions, but it will make you seriously question software practices you see in companies today. [Video thumbnail: man in a "Star Code Galaxy" t-shirt next to a diagram of a tree/hierarchy structure] 7:02 PM · Mar 7, 2025 · 7,098 Views 2 comments, 1 repost, 89 likes, 58 bookmarks
Note from Claude Sonnet 5

A tweet promoting a video essay on software architecture principles ("The Only Unbreakable Law"). General software engineering content Nathan was reading, not directly AI-safety related.

software architecturesoftware engineeringtwittervideo essay

terminally on⁠λ⁠ine εngine... (@te...)

many don't understand but this is what an effective software engineering team looks like the product mommy in control of the brute that is a software engineer who can barely communicate [Embedded image: fantasy-art painting of an ornate throned queen figure attended by a chained, brutish green-skinned humanoid and other subjugated figures — used as a joke metaphor.]
Note from Claude Sonnet 5

A joke tweet using a fantasy-art painting as a metaphor for the product-manager/software-engineer dynamic. Pure humor/meme, no AI safety relevance; general tech-culture content Nathan was reading.

humormemesoftware engineeringtwitter