Skip to main content

Ollama Guide

This guide will walk you through the process of installing and using Ollama, a tool for running large language models locally, on Manjaro Linux. Ollama simplifies the process of running various open-source LLMs on your local machine.

1. System Requirements​

Before we begin, ensure your system meets these minimum requirements:

  • 8GB RAM (16GB or more recommended)
  • 10GB free disk space (more for multiple or larger models)
  • Modern multi-core CPU (or NVIDIA GPU for faster processing)
  • Manjaro Linux (up-to-date)

Remember that while Ollama and other interfaces simplifies the process of running LLMs locally, these models can still be resource-intensive. Always be mindful of your system's capabilities and the ethical considerations when working with AI models.

2. Preparing Your System​

First, update your system:

sudo pacman -Syu

3. Installing Ollama​

Ollama provides a simple installation script. Run the following command:

curl https://ollama.ai/install.sh | sh

This script will download and install Ollama on your system.

4. Running Ollama​

After installation, follow the steps below:

  1. Visit: https://ollama.com/library
  2. Click on the model that you want to install locally.
  3. Copy the provided command on the site to your terminal.

For example:

ollama run mistral:latest

5. Using Ollama​

Once Ollama is running, you can interact with it directly in the terminal. Here are some examples:

  • Basic interaction: Simply type your prompt and press Enter. The model will generate a response.
  • Multiline input: To input multiple lines, use Shift+Enter to create new lines without sending the prompt.
  • Ending a conversation: Type /bye to end the current conversation and start a new one.
  • Exiting Ollama: Press Ctrl+C or type /exit to quit Ollama.

6. Managing Models​

Ollama supports various models. Here are some commands to manage them:

  • List available models:

    ollama list

  • Download a specific model:

    ollama pull modelname

    Replace modelname with the desired model (e.g., llama2, codellama, mistral, etc.)

  • Remove a model:

    ollama rm modelname

  • Run a specific model:

    ollama run modelname

7. Optimizing Performance​

To improve performance:

  1. Use a GPU if available. Ollama automatically uses CUDA if an NVIDIA GPU is present.
  2. Choose an appropriate model size for your hardware.
  3. Close unnecessary applications to free up system resources.

8. Troubleshooting​

Common issues and solutions:

  • Slow Performance:

    • Ensure you have enough free RAM
    • Try a smaller model
    • Check if GPU acceleration is working (for NVIDIA GPUs)

  • Installation Fails:

    • Ensure your system is up-to-date
    • Check your internet connection
    • Try running the curl command with sudo

  • Model Download Issues:

    • Check your internet connection
    • Ensure you have enough free disk space

9. Additional Resources​