Run Local LLMs on Mac

Think of your Mac as a tiny, private library—and running a local LLM is like checking out a book that never leaves your shelf, no internet required.

Why Bother Running a Local LLM?

You might wonder, “Why not just use ChatGPT?” Good question. Running a model on your own Mac means:

• Privacy: Your chats never leave your computer.
• No subscription fees: Free forever (after the initial setup).
• Offline access: Works even when Wi-Fi is down.
• Fun learning: You get to see how AI actually ticks.

The catch? You need a Mac with at least 8GB of RAM (16GB is better), and you’ll be using smaller models—think “pocket-sized” rather than “giant brain.”

Step 1: Install Ollama (The Easy Button)

Ollama is the simplest way to run LLMs locally. It’s like a package manager for AI models—no complex Python setup.

Open your Terminal (search for it in Spotlight) and run:

curl -fsSL https://ollama.com/install.sh | sh

This downloads and installs Ollama in one go. Once it’s done, check it works:

ollama --version

Step 2: Pick and Download a Model

Think of models as different “flavors” of AI. For Mac beginners, I recommend Llama 3.2 (small, fast, works on most Macs) or Mistral (great for chat).

Download one with:

ollama pull llama3.2

This pulls the model from the internet once, then it’s stored locally. You can see what you have with:

ollama list

Step 3: Chat With Your Local AI

Now for the fun part—talking to your very own offline AI. Run:

ollama run llama3.2

You’ll see a prompt. Type anything, like “Explain gravity like I’m five,” and hit Enter. To exit, type /bye.

Step 4: Make It Feel Like a Real App (Optional)

If you want a nicer interface than the terminal, install ChatGPT-style GUI called Open WebUI:

ollama serve

Then in a new terminal window:

pip install open-webui
open-webui serve

Open your browser to http://localhost:3000—you’ll get a chat window that looks familiar, but everything runs on your Mac.

Common Fixes (If Things Go Wrong)

• “Out of memory” error: Close other apps, or try a smaller model like llama3.2:1b.
• Slow responses: That’s normal on older Macs. Use shorter prompts.
• Can’t find the terminal? Press Cmd + Space, type “Terminal,” and hit Enter.

Wrapping Up

You now have a private, offline AI assistant living on your Mac. No subscription, no cloud, no tracking—just you and your model.

Practical tip: Start with the smallest model (llama3.2:1b) to test your Mac’s speed, then upgrade to larger ones if it feels snappy. And remember—you can always delete models with ollama rm llama3.2 to free up space.

Now go have a conversation with your new digital neighbor. It’s waiting to help you brainstorm, explain, or just chat—all from the cozy confines of your own hard drive.