Vespasian is an open-source weight space editing tool for small language models. A frontier model proposes direct tensor changes, evaluates them against your dataset and retains the best checkpoint. Run it in your terminal or on Hugging Face.
The objective, health checks, generated answers and edit genealogy stay together, letting you inspect candidates down to the layer and tensor.
Weight space programming through direct tensor edits.
A frontier language model inspects the target model and its answers, then proposes a bounded weight alteration. Direct tensor operations and forward evaluation drive the search. Validation scores select the next step.
Explore the search loop ↗Define the behaviour, connect edit and validation examples, then choose the metric and capabilities to preserve.
The editor inspects tensors and activations, selects a supported operation and proposes its parameters. Vespasian applies the bounded change directly to the weights.
Vespasian generates answers and measures the objective alongside copying, arithmetic, JSON or MMLU health checks. Optional regression limits reject edits that reduce these scores too far.
Improved validation scores retain a candidate. Rejected edits are restored. Export the best checkpoint or a replayable weight patch with its measurements and optional editor trajectory.
Load a local JSONL or a Hugging Face dataset. Preview the records, map the fields and set how many examples to use.
{"prompt":"What is the capital of France? Give only the answer.","target":"Paris","split":"edit"}Compare generated answers with references using exact match, numeric accuracy, token overlap or goal scoring.
vespasian examples/concise.jsonl --goal "Answer precisely" --budget 2
Hugging Face field previews · Nested records · Example limits · Private repositories
Use maths problems from OpenAI’s GSM8K dataset to search for better final-answer accuracy. Compare the original model and retained checkpoint on held-out test questions.
Open the browser workbench, then choose Improve GSM8K maths from the guide’s Examples tab.
Preview the question and answer fields, adjust the example limits, then choose your model, editor and USD budget.
Watch final-answer accuracy and health checks through the weight hill climb. Review generated solutions and the final held-out evaluation.
Install the Rust CLI and its Python model runtime. Release packages are available for Linux, macOS and Windows.
cargo install vespasian --locked
vespasian setup
vespasianInstall Rust and uv, then run these commands in your terminal.
View the Rust crate ↗
Choose the model, data and measurements that fit the behaviour you want to change.
Weight space editing changes a neural network’s parameters to alter its behaviour. Vespasian uses an LLM-guided hill climb: a frontier model inspects weights and outputs, proposes direct tensor surgery, then uses measured results to guide its next edit.
Vespasian supports Qwen3.5-0.8B and Gemma 3 1B. The terminal also accepts Hub model IDs and local checkpoints with compatibility checks. The hosted workbench offers models loaded by its operator.
Yes. Use a text corpus for unconditional generation, or prompts scored against a style goal. Paired examples support reference-answer tasks such as maths and factual responses. Dataset modes and field mapping ↗
The Hugging Face Space runs model evaluation on ZeroGPU. Sign in with Hugging Face to access private datasets and save results to the Hub. Public datasets are available without signing in.
The terminal connects to Codex or Claude CLI sessions, or OpenAI/Anthropic APIs. The browser accepts an OpenAI or Anthropic API key. Set a USD budget for frontier inference and follow tracked spend. CLI login sessions report estimated cost between calls.
Export a full model checkpoint or a replayable weight patch, together with evaluation scores and optional editor trajectories. Download the files or publish selected results to a public or private Hugging Face repository.