Open models versus hosted models for small business software

A client with sensitive customer records asked whether their AI could run on a computer in their own office. The answer was yes, and then we spent an hour explaining why they probably should not start there.

What the two words mean

A hosted model lives on the provider's computers. OpenAI, Anthropic, and Google work this way, as of this writing. Your software sends text to them and gets text back. You never see the model, you never update it, and you pay for what you use, usually by the amount of text in and out.

An open model is published as a file. Meta's Llama family, Mistral's models, and DeepSeek's models are the ones we see most. You download it, put it on a server you control (or rent), and run it yourself. There is no per use bill from the model's maker. There is a server bill, and there is you, keeping it running.

Where open models win

• Privacy. The text never leaves your control. For a client handling applicant records, financial details, or anything under a confidentiality agreement, this is sometimes the only acceptable shape. The data does not go to a third party because there is no third party.
• Cost shape. Hosted cost grows with use. Open cost is mostly flat: the server costs what it costs whether it handles ten requests or ten thousand. For a product that runs constantly, flat can be cheaper. For one that runs occasionally, it almost never is.
• Stability. A hosted provider can retire a model or change its behavior on their own schedule. An open model you have downloaded is yours. It behaves next year the way it behaves today, which matters when a system has been tuned carefully around it.
• Fit. Open models can be further trained on a client's own material. For narrow, repetitive jobs (sorting support emails into categories, pulling fields out of the same kind of form) a small tuned open model can do the job well and quickly.

Where hosted models win

• Ability. The hosted families tend to be the ones that get new capabilities first: better reasoning, images, voice, longer documents. If the job needs the frontier, it is usually hosted.
• No upkeep. No server to patch, no model file to update, no one on call when it falls over at two in the morning. For a business without technical staff, this is the deciding factor more often than any other.
• Starting cost. There is nothing to buy. A hosted assistant can be live in days with a bill that starts small and grows only if the business grows.
• Speed of change. When a better model appears, switching is a configuration change. With open models, switching means downloading, testing, and redeploying.

The maintenance burden nobody prices in

This is the part of the conversation that changes minds. Running an open model is running a server. Someone has to watch it, update it, restart it, and scale it when the business grows. If that someone is us, it is a monthly line item. If it is nobody, the system quietly degrades until the day it stops. We have taken over self hosted setups that had been running an old version for a year because the person who set it up left.

Hosted providers absorb all of that. You are paying partly for the model and partly for not having to think about it.

How we actually decide

We start hosted for almost every client. The website assistant, the morning brief, the content drafts, the research agent: hosted. Then we watch two things. The first is whether any job touches data the client is not comfortable sending out. The second is whether any job is running so constantly that the hosted bill has become a real line in the budget.

When either is true, we move that one job to an open model we run for the client, and we leave everything else hosted. A builder's applicant screener might run on an open model because of the records involved, while the assistant on their website stays hosted because it only knows public pages anyway. One product, two shapes, each chosen for the job.

What we do not do is move a whole business to self hosting on principle. The control is real but so is the cost of carrying it, and for most small businesses the hosted bill is lower than the attention it would take to avoid it.

This is the kind of decision we make inside every system we build, from a transit platform serving universities to the assistant on a painting company's site: hosted by default, open where it earns its place.

Questions people ask