Ollama is a runtime for running AI models locally or on a personal server.
Installation
Official Install Script
curl -fsSL https://ollama.com/install.sh | sh
Arch Linux
No GPU
sudo pacman -S ollama
ROCm (AMD)
sudo pacman -S ollama-rocm
CUDA (Nvidia)
sudo pacman -S ollama-cuda
Using Models
When picking models, ensure there is enough memory to fit the model you are using. Models small enough to fit in the GPUs memory will only use the GPU while models too big to fit in the GPU will use a mix of GPU and CPU. To check GPU and CPU memory usage run:
ollama ps
From Terminal
Gemma 3 will download if not already on the system and then run.
ollama run gemma3
Gemma 3 by default uses the 4B version of the model but different versions can be specified.
ollama run gemma3:12b
Ollama uses q4_K_M quantisation by default but different quantisation level can be specified.
ollama run gemma3:12b-it-q8_0
From a Graphical User Interface
Alpaca
Alpaca is officially distributed as a Flatpak.
flatpak install flathub com.jeffser.Alpaca
Alpaca can be configured to use Ollama externally including remote hosts but can also install Ollama internally as a Flatpak.
flatpak install flathub com.jeffser.Alpaca.Plugins.Ollama
The Flatpak Ollama plugin can support AMD GPUs using another plugin.
flatpal install flathub com.jeffser.Alpaca.Plugins.AMD
Visual Studio Code
Ollama can be integrated into Visual Studio Code using Continue.
Officially Supported GPUs
Consult this table for official GPU support.
Use Unsupported AMD GPUs on ROCm
To use ROCm on GPUs not officially supported by Ollama we can extend the Systemd service file with the following command:
sudo systemctl edit ollama.service
This will open up a file in the systems text editor where you can extend the service. Below are examples of what to add to the file:
Example: Radeon RX 6000 series
[Service]
Environment="HSA_OVERRIDE_GFX_VERSION=10.3.0"Example: Radeon RX 5000 series
[Service]
Environment="HSA_OVERRIDE_GFX_VERSION=10.1.0"The Ollama service will need to be restarted before the GPU can be used.
sudo systemctl restart ollama.service