Running deepseek-r1:8b locally with ollama

Running deepseek-r1:8b locally with Ollama, a lightweight version of deepseek-r1, but still a lot to offer.

It is like “word fill in” puzzles, there is a word for a designated space, with a fixed length, but you need to think thoroughly, and scan your vocabulary to find it. Only a few are the ones who creates the puzzle itself, while rest of us just fill it, where toys like these can help.

But not for creating the puzzle itself with unique generated words added.
Reasoning != Authenticity
Don’t be enchanted with the nicely crafted word sequences if you are trying to create a new puzzle 🤨

Enough said, here is step-by-step description, generated with AI, but tested by me 🙂

I’ll assume you’re starting from a clean slate and will be using a typical macOS or Linux environment.

Prerequisites:

• A Computer: You’ll need a computer with a decent amount of RAM (at least 16 GB recommended, 32 GB for optimal performance), and a relatively modern CPU and/or a compatible NVIDIA GPU (optional but highly recommended for faster inference).
• Operating System: macOS (Intel or Apple Silicon) or Linux are officially supported. Windows is unofficially supported.
• Internet Connection: Required for downloading Ollama and the model itself.
• Basic Terminal Skills: Familiarity with using the command line (Terminal on macOS/Linux).

Step-by-Step Instructions:

1. Install Ollama:

• macOS:
  1. Open your terminal.
  2. Run the following command: curl -fsSL https://ollama.com/install.sh | sh content_copy download Use code with caution.Bash
  3. This script will download and install Ollama and its dependencies.
• Linux:
  1. Open your terminal.
  2. Choose the appropriate installation command based on your distribution:
     • Debian/Ubuntu: curl -fsSL https://ollama.com/install.sh | sh content_copy download Use code with caution.Bash
     • Other Linux (using apt or similar):
       You might need to install using the provided instructions on the Ollama website for your specific distribution.
  3. The script will download and install Ollama and its dependencies.
• Windows (Unofficial):
  • Refer to the Ollama website for unofficial Windows installation instructions, as they can vary. You’ll likely need to use WSL (Windows Subsystem for Linux).

2. Verify Ollama Installation:

• Open a new terminal window.
• Run the following command: ollama --version content_copy download Use code with caution.Bash
• If Ollama is installed correctly, you should see the version number printed to the terminal. If you get an error, double-check that the installation was successful and that your terminal has access to the installed executable.

3. Download the deepseek-r1:8b Model:

• In your terminal, run the following command: ollama pull deepseek-r1:8b content_copy download Use code with caution.Bash
• This command will download the deepseek-r1:8b model from the Ollama model registry. This can take a while, depending on your internet speed. You’ll see a progress bar in your terminal as the download proceeds.

4. Run the Model:

• Once the download is complete, you can start chatting with the model using the ollama run command: ollama run deepseek-r1:8b content_copy download Use code with caution.Bash
• You will then be presented with a prompt (>>>) where you can type your questions or prompts. The model will then generate the response.

5. Interact with the Model:

• Type your text after the >>> prompt and press Enter.
• The model will generate a response, and you will see it displayed in the terminal.
• To end your conversation and exit the model, type /exit and press Enter.

Example Interaction:

      >>> Hello, how are you doing?
I am doing well, thank you for asking. How are you today?
>>> What is the capital of France?
The capital of France is Paris.
>>> /exit
    

content_copy download Use code with caution.

Important Considerations:

• Model Size: The deepseek-r1:8b model is a large language model, so it will require significant memory and CPU/GPU resources to run effectively. If you have limited hardware, you might experience slow inference times or potentially encounter out-of-memory errors.
• GPU Acceleration: For significantly faster inference, ensure that Ollama is configured to use your NVIDIA GPU if you have one. Refer to the Ollama documentation for instructions on enabling GPU acceleration.
• Resource Usage: Keep an eye on your system’s resource usage (CPU, RAM, GPU) while running the model, especially during the initial interaction.
• Ollama Documentation: Refer to the official Ollama documentation for more advanced options, troubleshooting tips, and updated information.

Troubleshooting:

• Download Issues: If the download fails, double-check your internet connection. Try again using ollama pull deepseek-r1:8b.
• “Out of Memory” Errors: If you encounter out-of-memory errors, try running the model on a machine with more RAM. Consider closing other resource-intensive applications to free up more memory.
• Slow Inference: If the response generation is slow, try to optimize your prompts. Enabling GPU acceleration is also a big performance booster.

That’s it! You should now have a working installation of ollama deepseek-r1:8b running locally. Remember that running local models can be resource-intensive and performance will be limited by your hardware. Enjoy experimenting!