🦙Quickstart

Ollama ..

Ollama

Ollama

Ollama simplifies running LLMs locally by handling model downloads, quantization, and execution seamlessly.

1. Download and install Ollama from the official website.

Ollamaollama

Link to Ollama

Models

It might sound obvious, select models that are able to run on your resources. Also bear in mind models are optimized to perform certain tasks - reasoning, math, code, tools, etc ...

Windows

1. Double-click on the executable to install.

2. Select your model to download. Select Models option from menu.

1. Test the setup and download our model.

ollama run deepseek-r1:7b

DeepSeek 7B

1. That's it .. you're good to go .. Let's ask the model a question..?

Chat away ..

---

Some Useful Commands

  /set            Set session variables
  /show           Show model information
  /load <model>   Load a session or model
  /save <model>   Save your current session
  /clear          Clear session context
  /bye            Exit
  /?, /help       Help for a command
  /? shortcuts    Help for keyboard shortcuts

Use """ to begin a multi-line message.

1. To check the Ollama server

ollama serve

1. Switch to user directory.

cd $HOME

1. Check Ollama server is up and running.

Get-NetTCPConnection -LocalPort 11434 -ErrorAction SilentlyContinue

Check Ollama connection

1. You can also send the question as a JSON object.

$jsonContent = '{
  "model": "deepseek-r1",
  "messages": [{ "role": "user", "content": "Solve: 2 + 2" }],
  "stream": false
}'

[System.IO.File]::WriteAllText("$HOME\request.json", $jsonContent)
curl.exe -X POST http://localhost:11434/api/chat -d "@$HOME\request.json" -H "Content-Type: application/json"

JSON request to Ollama server.

Useful, as there's now have an endpoint that can be used for out Chat UI frontend.

Linux

1. Run the following command.

curl -fsSL https://ollama.com/install.sh | sh

Install Ollama

1. In another terminal, verify that Ollama is running.

ollama -v

---

Adding Ollama as a startup service (recommended)

1. Create a user and group for Ollama.

sudo useradd -r -s /bin/false -U -m -d /usr/share/ollama ollama
sudo usermod -a -G ollama $(whoami)

1. Create a service file in /etc/systemd/system/ollama.service.

[Unit]
Description=Ollama Service
After=network-online.target

[Service]
ExecStart=/usr/bin/ollama serve
User=ollama
Group=ollama
Restart=always
RestartSec=3
Environment="PATH=$PATH"

[Install]
WantedBy=default.target

1. Then start the service.

sudo systemctl daemon-reload
sudo systemctl enable ollama

---

Start Ollama

1. Start Ollama and verify it is running:

sudo systemctl start ollama
sudo systemctl status ollama

Commands & Flags

Commands & Flags

x

Command	Description
ollama list	Displays all installed models on your system.
ollama pull <model name>	Download a New Model.
ollama run <model name>	Run an LLM for Chat or Inference.
ollama update <model name>	Ensures you have the latest version.
ollama remove <model name>	Deletes a model.
ollama status	Displays CPU and GPU utilization, active models, and memory usage.

x

x

Flag	description
--verbose	Displays stats. Net generation time and output time with tokens/seconds

Ollama also has its REST API, you can call it through curl

The following example uses the Microsoft Phi4 model.

curl http://localhost:11434/api/generate -d '{
  "model": "phi4",
  "prompt": "Why is the sky blue?"
}'

The response will be streamed back ..

chatbot@Office:~$ curl http://localhost:11434/api/generate -d '{
  "model": "phi4",
  "prompt": "Why is the sky blue?"
}'
{"model":"phi4","created_at":"2025-02-25T13:01:16.847307208Z","response":"The","done":false}
{"model":"phi4","created_at":"2025-02-25T13:01:16.895178316Z","response":" sky","done":false}
{"model":"phi4","created_at":"2025-02-25T13:01:16.926463821Z","response":" appears","done":false}
{"model":"phi4","created_at":"2025-02-25T13:01:16.957592326Z","response":" blue","done":false}
{"model":"phi4","created_at":"2025-02-25T13:01:16.988690731Z","response":" due","done":false}
...
319,6138,3135,304,1524,810,72916,315,24210,93959,320,12481,323,6307,705,10923,5129,93959,1093,2579,82,323,85138,311,5662,701,6548,382,45600,11,13558,64069,72916,15100,3249,584,1518,264,6437,13180,1234,14595,62182,4787,13],"total_duration":14595395219,"load_duration":3775345351,"prompt_eval_count":16,"prompt_eval_duration":465000000,"eval_count":280,"eval_duration":10352000000}