Where the Model Fits
A model should have a defined job in the stack, and the organization should be able to replace it.
A legal AI stack should be able to explain why every model is there.
Some work needs a large model with broad reasoning and strong tool use. Some work needs a smaller model that performs one narrow task repeatedly. Some steps should be deterministic software. Others belong in the agent, the environment, or the verifier.
Using the largest available model for every step can hide those choices. It can also leave the organization with a system that works only while one provider, model version, price, and set of guardrails remain available.
Give the model a job
Start with the assignment. What does the system have to read? Which actions can it take? What does it have to produce? Where can the work fail?
Those questions define the role before the model is selected. A model may be responsible for reading the record and proposing edits. Deterministic code may validate the document, citations, calculations, or required fields. Another model may judge a narrow quality that cannot be checked mechanically. An expert may review the cases where the system is uncertain or the answer is genuinely contestable.
The architecture should make those boundaries visible. If a smaller open model can handle a narrow step, use it there and test it there. If the work needs a stronger model, preserve the trajectories and evaluation so the system can compare the next one.
Keep what surrounds it
The organization gives a hosted model its documents, instructions, playbooks, prompts, tools, corrections, and workflows. The provider returns an answer. The information that made the answer useful often remains scattered across chats and product-specific configuration.
That surrounding work is the durable part. It includes the task definition, agent logic, environment, verifiers, reviewed examples, and record of what happened. Those pieces can improve the next model. They can also make it possible to replace the current one.
Open weights help, but weights are one component. A checkpoint without the task, tools, evaluation, and data does not preserve the system. A managed model can still have a place if the organization keeps the artifacts and interfaces needed to continue without it.
Replacement is not free. Models use tools differently. Tokenizers, context limits, prompting, and failure modes change. The system needs held-out work that can show whether the replacement still completes the assignment.
The organization should be able to change the model, rerun the evaluation, see what broke, and decide whether the new system is better.
A model has earned its place when it performs a defined job inside a system the organization can test and change.