Ollama is a program that lets you run large language models (LLMs) on your own computer. These models can chat, write code, brainstorm ideas, and more, all offline. It's the closest option we currently have to use an AI locally without all of the data getting sent back to the companies that run them. In this guide, we'll show you how to install Ollama, download models, use them and of course, how to uninstall them if you change your mind.

Takeaways:

  1. What is Ollama and how do you use it to install AI locally
  2. What's the best way to run AI locally on your computer?
  3. Best AI to run on a local system?

What is Ollama, and what does it do?

Ollama is a tool for running AI models locally. You can download multiple AI models through Ollama and use them without internet access. And you stay in control of your data because everything runs on your own computer. The only downside is that some models are bulky, and some require a lot of hardware to work well. Though there are plenty of smaller models around for basic tasks.

How to Install Ollama on Windows 11

  • Go to Ollama.com.
  • Download the version for your computer (Windows, macOS, or Linux).
  • Follow the installer instructions.
  • After installation, open PowerShell on Windows or Terminal on macOS/Linux to manage the model.

That part is super simple and straightforward. Now you can choose an AI model that you want to use locally.

How to Download and Run a Model in Ollama on Windows 11

This part is really important so pay attention to the system requirements. Some models are massive, and some need specific hardware in order to work. Once you find one that seems to fit your needs and hardware, it's a good idea to jump on YouTube and watch a bit of info on it.

  • Open Ollama and browse available AI models.
  • Choose a model based on your computer’s hardware:
  • Small models (1B–8B parameters) for basic laptops or PCs
  • Medium models (32B) for high-end PCs
  • Very large models (hundreds of billions) require special hardware
  • Download your chosen model.
  • Run the model in PowerShell or Terminal using a simple command (instructions are usually shown on Ollama).
  • The model will start running, and you can type questions or prompts to interact with it.

Using Models for Common Tasks

You can ask models to do many things, such as the examples below. But this will vary depending on the model/s you download and install.

  • Chat with you naturally
  • Generate code snippets or help with programming
  • Brainstorm ideas for projects, stories, or businesses
  • Solve problems and give step-by-step explanations
  • Summarise documents or research

How does Performance work in Ollama

  • Large models need more computer power and may run slowly on low-end PCs.
  • Open smaller models if your computer is slow.
  • High-performance graphics cards (like RTX 4090) make bigger models faster.
  • Running multiple sessions at the same time splits your computer’s resources.

Clean Up Model Outputs (Delete the bloat)

  • Some models add extra “thinking tags” or notes in their responses.
  • Beginners can ignore them, or remove them manually.
  • Advanced users can write a small script to automatically clean up outputs.

Integrate Models with Programs (Advanced Users)

This isn't something the average person is going to jump straight into, but the option is there if you're looking to expand your knowledge base.

  • Ollama works with Python libraries like LlamaIndex or LangChain.
  • You can program your AI to answer questions, write stories, or even build chatbots.

How to Uninstall Models and Ollama, and Model Files

  • To remove a model, list installed models in PowerShell or Terminal and remove the ones you no longer need.
  • To remove Ollama completely:
  • Windows: Go to Control Panel > Programs > Uninstall a Program > Ollama
  • Delete leftover model files:

Windows: C:\Users\username.ollama\models

macOS: ~/.ollama/models

Linux: /usr/share/ollama/.ollama/models