A finished run is not a finished model
When a fine-tuning run completes, you have a file. Whether you have something your people can rely on is a separate question.
Before an adapter goes into a client's environment, we answer four questions about it and hand it over with the evidence behind each answer. If a question has no answer yet, the adapter is not ready.
Does it do the job?
Test it on the work it was trained for, using examples agreed with you before training started. Generic quality scores say little about whether it drafts your clauses or reads your screens correctly.
Say what the number means. When we trained a Qwen3.5-9B adapter to operate Mac software, its action score on a frozen 477-step test rose from 0.33 to 0.77. Part of that gain came from the model learning the answer format, so we report it that way, and we treat it as our own offline test, not a measure of live task success.
What did it break?
An adapter can make a model better at one thing and quietly worse at others. Compare it with the base model on work the base model already handled well:
- answers that became weaker or vaguer
- formatting that drifts or breaks
- refusing too often, or answering when it should not
- new weak cases the base model did not have
Can it run where it has to?
Check it on the machine it will live on, with the memory, speed and context length that machine allows. An adapter that only works on our hardware has not been tested.
Can it be switched off?
Load it, disable it, replace it and roll it back in the real runtime before anyone depends on it. Decide in advance what would trigger a rollback, and who can do it.
What travels with the adapter
An adapter should never be handed over as a loose file. It comes with a short evidence pack:
- which base model it was trained on
- which data shaped it, with a reference to the dataset card
- the training settings
- the test examples and the results
- where it failed or got worse
- how to switch it on, switch it off and roll it back
- who reviewed it, and whether it was accepted
The pack is not there to make the adapter look good. It is there so anyone can see how it behaves.
Six questions before it ships
- What task is it meant to improve, and which examples show that it does?
- Which examples show no improvement, or a regression?
- Which base model and runtime were used for the tests?
- How is it switched on, and how is it switched off?
- What would make us roll it back?
- Who approves its use with your staff or customers?
When all six have answers, the adapter is ready to go into your environment, and it can be reviewed, switched on and reversed there.