synthetic data

4 captures, most recent first.

2026-08-12 — saved image

will brown @willccbb · 2h
i don't think "progress multiples" is really the right framing of RSI

"how fast is LLM progress moving vs if we didn't have LLMs" isn't really coherent

we're doing things that make no sense without good LLMs, like judge rewards and synth data

what's the counterfactual?

Note from Claude Sonnet 5

Tweet by will brown arguing against framing recursive self-improvement (RSI) in terms of 'progress multiples,' since comparing LLM-era progress speed to a counterfactual without LLMs is incoherent given that current techniques like judge rewards and synthetic data only make sense because good LLMs already exist.

ai progress twitter recursive self-improvement rsi will brown synthetic data

Percy Liang @percyliang

2026-03-20 — quoting Konwoo Kim (@konwookim)

Percy Liang @percyliang In our last episode, careful tuning, scaling, and ensembles led to a 5x gain in data efficiency (requires 5x less data to get the same loss). Now, with a rephraser model, we can get an additional 1.8x gain in data efficiency. I know, everyone's compute constrained, but we're preparing for a data-constrained future. > QUOTED: Konwoo Kim @konwookim · 7h > for data-constrained pre-training, synth data isn't just benchmaxxing, it lowers loss on the real data distribution as we generate more tokens > for even better scaling, treat synth gens as forming on... > [Embedded paper image: "Data-efficient pre-training by scaling synthetic megadocs" — Konwoo Kim, Suhas Kotha, Yejin Choi, Tatsunori Hashimoto, Nick Haber, Percy Liang, Stanford University. Diagrams of "Simple Rephrasing," "Stitched Rephrasing," and "Latent Thoughts" methods for generating synthetic training data from real docs; a scaling chart "Scaling Generations" showing IID Loss decreasing from 3.55 (real data only) down to ~3.34-3.41 as generations per pretraining doc increase from 0 to 32, with Latent Thoughts performing best.] 2:52 PM · Mar 20, 2026 · 5,748 Views

Note from Claude Sonnet 5

A Stanford paper (Percy Liang's group) on data-efficient pretraining via synthetic "megadocs" generated by rephrasing/expanding real documents, addressing anticipated data-constrained scaling. Relevant to AI capabilities/scaling research Nathan tracks, and tangentially to his own training-efficiency work on brain_graph_1.

twitter machine learning research data efficiency synthetic data pretraining scaling laws stanford percy liang

Saved image — no attribution recorded

2025-05-17 — saved image

# collect R1 rollouts from API
import os
from openai import OpenAI
base_url = os.getenv("DEEPSEEK_API_URL")
api_key = os.getenv("DEEPSEEK_API_KEY")
client = OpenAI(base_url=base_url, api_key=api_key)
results = vf_env.eval_api(client, "deepseek-reasoner", max_concurrent=32,
sampling_args={"temperature": 0.6})

# make dataset from results (flatten reward scores)
def flatten_rewards(rewards: dict) -> list[float]:
    return [sum(r) for r in zip(*rewards.values())]
dataset = Dataset.from_dict({
    "prompt": results['prompt'],
    "completion": results['completion'],
    "answer": results['answer'],
    "reward": flatten_rewards(results['rewards']),
})

# filter to top half of rows by rewards
dataset = dataset.sort("rewards", reverse=True).select(range(len(dataset) // 2))
print(dataset[0])

# save to hub
dataset.push_to_hub("R1-reverse-wikipedia-paragraphs-v1-1000")

———

will brown @willccbb · 4h
cheat code for making RL warmup data

> QUOTED (code block):
> # collect R1 rollouts from API
> import os
> from openai import OpenAI
> base_url = os.getenv("DEEPSEEK_API_URL")
> api_key = os.getenv("DEEPSEEK_API_KEY")
> client = OpenAI(base_url=base_url, api_key=api_key)
> results = vf_env.eval_api(client, "deepseek-reasoner", max_concurrent=32, sampling_args={"temperature": 0.6})
>
> # make dataset from results (flatten reward scores)
> def flatten_rewards(rewards: dict) -> list[float]:
>     return [sum(r) for r in zip(*rewards.values())]
> dataset = Dataset.from_dict({
>     "prompt": results['prompt'],
>     "completion": results['completion'],
>     "answer": results['answer'],
>     "reward": flatten_rewards(results['rewards']),
> })
>
> # filter to top half of rows by rewards
> dataset = dataset.sort("rewards", reverse=True).select(range(len(dataset) // 2))
> print(dataset[0])
>
> # save to hub
> dataset.push_to_hub("R1-reverse-wikipedia-paragraphs-v1-1000")
[5 comments, 3 retweets, 79 likes, 3.7K views]

will brown @willccbb · 4h
synthetic data engines = RL environments = eval harnesses

Note from Claude Sonnet 5

A technical tweet from will brown (RL/eval tooling developer, "verifiers" framework author) sharing a Python code snippet for generating RL warmup training data by collecting rollouts from DeepSeek-R1 via API, filtering to top-reward-half, and pushing to HuggingFace Hub. Technical ML-engineering reference, tangential to the project's interest in RL training methods relevant to alignment.

reinforcement learning deepseek synthetic data twitter ml engineering code snippet

will brown @willccbb

2025-05-04

[Continuation of original post, top cut off]: "...a technique i'm increasingly believing is going to be really useful in large-scale synth data pipelines is the use of graph algorithms to do semantic deduplication submitted a paper about this a while back, never got around to cleaning it up for arxiv, but it's a neat trick" [Embedded diagram: pipeline showing items (Apple, Orange, Pear, Windows, Doors, Tables) → embeddings + vector db → LLM queries pairwise comparisons (green check / red X for match/no-match) → graph community detection clustering into two groups: {Pear, Orange, Apple} and {Windows, Tables, Doors}] 5:53 PM · May 4, 2025 · 3,315 Views [8 replies, 4 reposts, 101 likes, 58 bookmarks] will brown ✓ @willccbb · 46m imagine you have N outputs from a model and you want to deduplicate which ones are "basically the same" or "join on semantic property X" how do you do this fast + reliably at scale? traditional wisdom says embedding clustering, but this is brittle + generally needs finetuning

Note from Claude Sonnet 5

Technical thread on using LLM-pairwise-comparison plus graph community detection for semantic deduplication of model outputs at scale, an alternative to embedding clustering — relevant to synthetic data pipeline design and possibly to Nathan's own evaluation/dataset work.

twitter semantic deduplication graph algorithms synthetic data llm evaluation embeddings