The AI governance reference on this site started from a six-layer outline: inventory, data foundation, data security, model assurance, human oversight, compliance and audit. Two of its entries told me who it was written for. The legal layer listed the EU AI Act and GDPR. The model layer listed bias screening of training data.
A fifteen-person practice in California does neither of the things those entries assume. It does not train models, and European law is not the first rulebook it answers to.
The layers still hold for that practice. Four things change. One test applies to every control that survives the changes: it has to name the evidence that proves it.
Someone has to own it
The outline starts at inventory. An inventory nobody owns gets filled in once and never opened again.
The standards put ownership first. NIST's AI Risk Management Framework has four functions (Govern, Map, Measure, Manage) and describes Govern as the one that runs through the other three. ISO/IEC 42001 is a management-system standard, so written policy and assigned responsibility come before any technical control.
In a practice that means one named person with the authority to switch a system off, and a one-page acceptable-use policy. The person is a partner or the practice manager. It is not the IT vendor, and I say that as an IT vendor. A vendor can run the controls. Deciding which risks the practice accepts belongs to someone inside the practice.
It runs as a loop
The arrows in the outline point down and stop at audit. An audit produces findings: an incident, a failed test, a change in the law. Each one changes the inventory or a risk tier, so the last layer has to feed the first.
The law supplied two examples this year. The EU moved its obligations for stand-alone high-risk systems from August 2, 2026 to December 2, 2027. The amendment took force on July 27, six days before the original date. Colorado went further. In May it repealed its 2024 AI Act and replaced it with a different law on automated decision-making, before the first version ever took effect.
A plan built around either original date was out of date before the date arrived. A program that never returns to layer one describes last year's systems under last year's rules.
The legal layer is your own rulebook
The EU AI Act and GDPR are European instruments. A California medical practice meets other rules first: the HIPAA Security Rule, and AB 3030, the state law that since January 2025 has required a disclaimer on AI-generated patient messages about clinical information unless a licensed provider has read and reviewed the message. A law firm meets ABA Formal Opinion 512 and the State Bar of California's Practical Guidance on generative AI. A financial firm covered by the FTC Safeguards Rule has to keep a written information security program, and that program reaches any AI system that touches customer financial data.
Which rules bind a given practice is a question for its attorney, and I am not one. The governance job starts once the attorney has answered: write the answer down, and attach each rule to a control someone can check.
You deploy models; you do not train them
Bias screening of training data, public benchmarks, and model cards all assume you built the model. A practice running a published open-weight model against its own files has different exposures:
- which documents the retrieval index can reach
- whether the index honors folder permissions
- whether anyone tested the system on the practice's real work
OWASP's Top 10 for LLM Applications reads closer to that situation than a training checklist does. It names prompt injection, sensitive information disclosure, excessive agency, and vector and embedding weaknesses.
Each of the publisher's checks has a counterpart that falls to you. Keep the publisher's model card, and add your own record: version, quantization, license, file hash, date deployed. Public leaderboards do not measure your work, so build a test set of a few dozen questions from your own documents, with known answers. Re-run it whenever the model, the prompt, or the document set changes. Fairness testing stays in the stack, sized to the stakes. It matters wherever output informs a decision about a person.
Ask each control what proves it
A governance document is easy to write and hard to check. The fix is to make every control name its evidence. Three examples from the reference page:
- Named owner. A name, not a department, on every row of the use-case register.
- Permission-aware retrieval. Access tests run as different roles. If a paralegal cannot open a folder, the assistant cannot quote from it.
- Incident reporting. The incident log, including the empty months.
The empty months are the point of the last one. A log that only ever contains incidents cannot tell you whether a quiet quarter was quiet or unwatched.
The reference page lists five controls for each layer, and every one names its evidence.
What a server in your office settles
I build on-premises inference systems, so this is the section I have the most reason to overstate. Here is the exact split.
Running the model on a server in your office settles three things for the workloads you keep there:
- The data path. A prompt and the documents retrieved for it travel across your own network to your own GPU and back. No outside company processes them, so the questions about a vendor retaining your data or training on it do not arise for that route.
- Version control. The model's weights are a file on your disk. They change when you replace the file.
- The audit trail. Logs sit on storage you control, kept for as long as you decide.
It leaves four things open:
- Retrieval permissions. A local model will quote a partner's personnel file to an associate as readily as a hosted one will, if the index ignores folder permissions.
- Human review. Where the model runs has no bearing on whether a qualified person reads the output before it is used.
- The outside route. Any workload forwarded to a hosted model is back under a vendor's contract terms.
- The server itself. It is now your equipment to patch, back up, and keep in a locked room.
On the outside route, IT CloudLink LLC's Data Routing Policy takes one position: local by default, with routing decided per workload, not per request. Whatever your own rule is, the register from layer one has to record which route each use takes.
Where to go from here
The reference page ends with five first steps. None of them is a purchase. The four corrections and the on-premises split are there in full, with the sources behind every dated fact in this post.
A plain-language version of this post for practice owners is on the IT CloudLink LLC blog: Five Decisions Before Your Office Buys Any AI.
This post is a technical reference and not legal advice. Which laws apply to your practice is a question for your attorney.
Chris Rondthaler is the founder of IT CloudLink LLC and runs Rondthaler Labs. YouTube: @aiopsdude