The appeal of local AI is easy to understand: run the model on your own machine, experiment without sending every prompt to a hosted chatbot, and learn how the pieces fit together.
The phrase “local AI” can still hide several different arrangements. Start by checking where the model runs and what the rest of the application connects to.
Open weights and local are different ideas
An open-weight model makes its parameters available under a licence. It can be run locally or hosted by someone else.
“Open source” can imply broader conditions about software and access, so read the actual licence rather than treating every downloadable model as equivalent.
OpenAI’s gpt-oss release is one example of downloadable models under Apache 2.0. Its smaller model was designed to run within 16 GB of memory in a supported configuration. That does not guarantee a satisfying experience on every 16 GB laptop.
Hardware needs depend on the workload
Model size is only one part of memory use. The runtime, operating system and amount of context also matter.
Check supported hardware and software before downloading large files or buying equipment. A smaller model running comfortably may teach you more than a larger one that makes the computer unusable.
For a first experiment, use hardware you already own and a short, non-sensitive document.
Verify the route your app takes
A local interface may offer both local and cloud models. Search, plugins, telemetry or document processing can introduce additional connections.
Ollama’s documentation describes a local-only mode that disables its cloud features. Check the configuration of the runtime and any separate interface you install.
Disconnecting from the internet after setup can help demonstrate whether a particular test works offline. It does not by itself audit every data-handling behaviour.
A local-only check you can reproduce
If you use Ollama, its current FAQ documents the OLLAMA_NO_CLOUD=1 setting, or a disable_ollama_cloud option in its server configuration, followed by a restart. These disable its cloud features, including cloud models and web search.
Follow the official instructions for your operating system. Select a downloaded local model and keep the first document fictional. Any separate interface or extension still needs its own data-flow check.
Record the model, computer and context settings before comparing results. This turns “local AI felt slow” into a useful observation about a particular setup.
A useful first comparison
Ask a local model and a hosted assistant to summarise the same fictional procedure. Check omissions, invented details and time to a usable answer.
Local processing may offer useful control, but maintaining the machine, updating software and protecting stored files remain your responsibility.
You do not need to replace your main assistant to explore this. A small local experiment can make the technology more understandable and reveal where control, convenience and capability trade off.