Run DeepSeek-R1 1.5B on NeoEdge NG4500 With Ollama

A local language model gives an edge application a way to process text on the device. On the CamThink NeoEdge NG4500, Ollama provides a practical starting point: download a small model, send a prompt, and inspect how it runs before connecting it to an application.

This guide uses deepseek-r1:1.5b, the Ollama package for DeepSeek-R1-Distill-Qwen-1.5B. It is a distilled text model, distinct from the full DeepSeek-R1 model. The goal is a working local text session and an API response you can inspect.

Choose the Hardware and Model

The NG4500 family includes several Jetson Orin configurations. Check the model on your unit before choosing a workload. CamThink’s product specifications list the following options:

This walkthrough covers the 8 GB and 16 GB configurations, following the starting requirements in CamThink’s deployment documentation. Use the 1.5B model for the first run.

The 1.5B model listing specifies an approximately 1.1 GB download with Q4_K_M quantization. Download size is not runtime memory consumption: the model also needs working memory, and longer conversations add context storage. Other applications need room as well.

Start with this small model to establish a baseline. A larger model is useful only if its answers improve enough to justify the additional memory and response time for your task.

Check the NG4500 Environment

Use a CamThink-supported system image with its JetPack and CUDA components intact. The NG4500 product documentation lists JetPack 6.0+; record the versions actually installed on your unit. A generic Ubuntu version alone does not describe the Jetson software stack.

uname -m
cat /etc/nv_tegra_release
free -h
df -h / /usr/share
dpkg-query -W nvidia-jetpack nvidia-l4t-core

The architecture should be aarch64. Save the L4T release and memory readings with your test notes. Some images do not install the nvidia-jetpack meta-package, so a missing package entry does not by itself mean CUDA is absent. If the image is unfamiliar, use the NG4500 quick start to establish the device environment first.

You will need internet access for the installer and model download, plus storage for the runtime, model files, and logs. Check available space on the actual installation volume; allow additional capacity if you plan to add a browser interface or more models.

An NG4500 can serve as the starting platform for an application prototype. If the standard configuration does not fit your deployment, discuss the hardware and integration configuration with CamThink.

Install and Check Ollama

Install Ollama directly on the NG4500. NVIDIA’s Ollama on Jetson guide documents this route for Jetson. Run the official installer from the NG4500 terminal:

curl -fsSL https://ollama.com/install.sh | sh
ollama --version
systemctl status ollama --no-pager
curl -fsS http://127.0.0.1:11434/api/tags

The service status should show it is running, and the API call should return JSON containing a models array. An empty array is normal before the first download. A listening port alone does not show that inference works.

If the service is stopped, use sudo systemctl start ollama and retry the API call. For startup errors, inspect journalctl -u ollama -n 50 --no-pager. The Ollama Linux documentation covers service setup and logs.

Run the 1.5B Distilled Model

ollama pull deepseek-r1:1.5b
ollama show deepseek-r1:1.5b
ollama run deepseek-r1:1.5b

After the download finishes, the final command opens an interactive session. Try a short prompt with an answer you can check, such as: Explain what MQTT does in two sentences. Type /bye to leave the chat.

In a second terminal, inspect the loaded model while the session is active:

ollama ps

The PROCESSOR column distinguishes GPU, CPU, and mixed placement. A reply can arrive even when the model runs on the CPU, so check this before drawing conclusions about device performance. The Ollama FAQ explains the output.

Keep the explicit :1.5b tag in your commands. Record the model ID shown by ollama list together with the Ollama version; a named tag alone is not a permanent record of the downloaded artifact.

Send a Request from an Application

The same local service accepts HTTP requests. This example uses a short context and a bounded output to keep the first integration test small:

curl -fsS http://127.0.0.1:11434/api/generate \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "deepseek-r1:1.5b",
    "prompt": "Explain what MQTT does in two sentences.",
    "stream": false,
    "options": {
      "num_ctx": 2048,
      "num_predict": 256
    }
  }'

With stream: false, the service returns one JSON response after generation finishes. Inspect response, done, and the timing fields described in the generate API reference. A reasoning model can spend its output budget before completing an answer; increase the limit if the reply is cut short.

A useful test separates model loading from generation. Repeat the request while the model remains loaded, and record both runs. Output throughput can be calculated as eval_count / (eval_duration / 1e9) when eval_duration is nonzero. This figure excludes some of the time the user waits, so retain total_duration too.

For a real application, test representative prompts and check the answers, memory use, and response time together. A fluent answer is not evidence that the model understood the input correctly.

Add an Optional Browser Interface

Open WebUI adds a browser chat interface to the local model service. Get the terminal and API tests working first, then follow the current Open WebUI quick start for installation and account setup.

Keep the network mode and browser address consistent. With Docker bridge networking, -p 3000:8080 maps host port 3000 to the interface’s container port 8080. With Linux --network=host, that mapping is absent; an unmodified Open WebUI server is reached on host port 8080. The two arrangements are different.

Also check how the container reaches Ollama. A bridge-networked container’s 127.0.0.1 refers to the container itself. The quick start describes the host connection and required Ollama bind settings. Configure access for the intended users before making either service available beyond the device.

Evaluate the Local Setup

After downloading the model, test a new prompt with the internet disconnected if offline operation matters to your application. Use a local model and disable external integrations for that test. Local inference removes the need to send that request to a hosted model; response time still includes prompt processing and generation.

For an NG4500 running other AI workloads, repeat the test with those workloads active. A single chat session on an otherwise idle device does not represent shared use. Use the power supply and thermal installation specified for your unit, and measure in that configuration before changing power modes.

This walkthrough is based on the linked product and software documentation. Performance measurements should come from your own NG4500 configuration.

You have a useful prototype when a representative prompt completes, the API returns an inspectable result, and you know where the model runs. Keep that baseline with your software versions. It gives you a concrete comparison for the next model or application change.