Guides
GPU rig model, anywhere

Use your GPU rig's model from anywhere

You run an open-weights model on a GPU machine at home or in the office. This guide makes it answer on http://localhost:11434 on your laptop, wherever the laptop is, as if the model were running locally.

  • Nothing is published to the internet. The model is reachable only by your own agents, over your private, encrypted Route6 mesh.
  • No network changes. No port forwarding on your router, no dynamic DNS, no VPN server. Both machines only make outbound connections, so it works behind any NAT or firewall that allows ordinary web browsing.
  • Works on the free plan. Free gives you two agents (one for the rig, one for the laptop), the mesh between them and one port forward per agent. Traffic between your own agents does not count toward the free plan's 250 MB of internet traffic.
laptop app ─► localhost:11434 ─► socat ─► r6me ══ private mesh ══► r6me ─► 127.0.0.1:11434 Ollama
                (laptop)                         (encrypted)               (GPU rig)

You need a Route6 account (opens in a new tab) with two agents, and their two API keys from the dashboard under Agents. Each machine gets its own key.

Connect the rig

On the GPU machine, as root, install the client with the rig's key and start it at boot:

curl -fsSL https://dl.route6.me/install.sh | ROUTE6_API_KEY=sk_a6_key_for_the_rig sh
sudo systemctl enable --now r6me
r6me status

Wait for "transport": "connected". The Quick Start covers installing as a normal user and on other platforms.

Leave the model server on localhost

Keep Ollama on its default address, 127.0.0.1:11434. Do not set OLLAMA_HOST=0.0.0.0: the Route6 client delivers mesh traffic to 127.0.0.1 on the rig, so the model never needs to listen on your local network.

Check it answers locally:

curl http://127.0.0.1:11434/api/tags

llama.cpp's llama-server and vLLM work the same way. They serve the same OpenAI-compatible API, on port 8080 and 8000 by default. Use that port wherever this guide says 11434.

Publish the port to your mesh only

The rig's agent publishes the port with the port_forward tool. scope: "mesh" means there is no public listener at all: only agents in your own team can reach it.

If an agent (Claude Code, Cursor, any MCP client) is connected to the rig's client, ask it:

Publish port 11434 to my Route6 mesh only.

It makes this call:

port_forward { action: "create", port: 11434, scope: "mesh" }

No agent on the rig? Make the same call from the shell. The client serves MCP on localhost:3000; this function opens a session and calls one tool:

# r6tool <tool> '<json arguments>': call one Route6 MCP tool on the local client
r6tool() {
  u=http://127.0.0.1:3000/mcp
  sid=$(curl -s -D - -o /dev/null -H 'Content-Type: application/json' \
    -H 'Accept: application/json, text/event-stream' "$u" \
    -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"shell","version":"1"}}}' \
    | tr -d '\r' | awk 'tolower($(1))=="mcp-session-id:" {print $(2)}')
  curl -s -o /dev/null -H 'Content-Type: application/json' -H 'Accept: application/json, text/event-stream' \
    -H "mcp-session-id: $sid" "$u" -d '{"jsonrpc":"2.0","method":"notifications/initialized"}'
  curl -s -H 'Content-Type: application/json' -H 'Accept: application/json, text/event-stream' \
    -H "mcp-session-id: $sid" "$u" \
    -d "{\"jsonrpc\":\"2.0\",\"id\":2,\"method\":\"tools/call\",\"params\":{\"name\":\"${1}\",\"arguments\":${2}}}" \
    | sed -n 's/^data: //p'
}
 
r6tool port_forward '{"action":"create","port":11434,"scope":"mesh"}'

Either way, the reply names the address your laptop will use:

{
  "external_port": 11434,
  "scope": "mesh",
  "mesh_endpoint": "north-river-5b24.mesh.route6.me:11434",
  "exposure": "mesh-only — reachable only by agents in your team mesh"
}

north-river-5b24 is the rig's assigned name; yours will differ. The .mesh.route6.me names resolve only inside your mesh.

Connect the laptop

On the laptop, install the client with the laptop's key:

curl -fsSL https://dl.route6.me/install.sh | ROUTE6_API_KEY=sk_a6_key_for_the_laptop sh
r6me up

On a Linux laptop installed as root, sudo systemctl enable --now r6me keeps it running across reboots instead.

Bring the model to localhost

Install socat (sudo apt install socat, or brew install socat on macOS) and run, with your rig's name:

socat TCP-LISTEN:11434,bind=127.0.0.1,fork,reuseaddr \
  PROXY:127.0.0.1:north-river-5b24.mesh.route6.me:11434,proxyport=1080

This listens on the laptop's localhost:11434 and carries each connection through the Route6 client's local proxy (port 1080) to the rig. From another terminal:

curl http://localhost:11434/api/tags

You should see the rig's model list.

Use it

Anything that talks to a local Ollama now talks to the rig, unchanged. With the OpenAI SDK:

from openai import OpenAI
 
client = OpenAI(base_url="http://localhost:11434/v1", api_key="unused")
stream = client.chat.completions.create(
    model="qwen3.5-9b",  # a model name from /api/tags
    messages=[{"role": "user", "content": "Explain a WireGuard handshake in two sentences."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
print()

Editor plugins, coding agents and chat UIs that let you set an Ollama URL or an OpenAI-compatible base URL take http://localhost:11434 (or http://localhost:11434/v1) the same way.

An empty reply from a Qwen3.5 model? It thinks before answering by default, and with a low max_tokens the whole budget can go on thinking. On the OpenAI-compatible endpoint, pass reasoning_effort="none" to turn thinking off. Ollama's own think: false option is ignored on that endpoint.

Keep it running

On a Linux laptop, a small systemd unit starts the relay at boot and restarts it if it stops. Save as /etc/systemd/system/rig-llm.service, with your rig's name:

[Unit]
Description=GPU rig model on localhost:11434, over the Route6 mesh
After=r6me.service
Wants=r6me.service
 
[Service]
ExecStart=/usr/bin/socat TCP-LISTEN:11434,bind=127.0.0.1,fork,reuseaddr PROXY:127.0.0.1:north-river-5b24.mesh.route6.me:11434,proxyport=1080
Restart=always
RestartSec=2
 
[Install]
WantedBy=multi-user.target
sudo systemctl daemon-reload
sudo systemctl enable --now rig-llm

On macOS, run the socat command in a terminal or a tmux session.

Why socat, and not the proxy settings

The Route6 client's proxy on port 1080 already reaches *.mesh.route6.me names, so it is tempting to point your app straight at http://north-river-5b24.mesh.route6.me:11434 with http_proxy set. Two things get in the way:

  • Ollama refuses it with 403 Forbidden. Ollama protects itself against DNS rebinding by rejecting requests whose Host header is not a local name while it listens on 127.0.0.1. A mesh name is not local. Through socat, your app sends Host: localhost and Ollama accepts it, so the rig can keep its safe default.
  • Many tools ignore proxy settings, or apply them only to some requests. A plain localhost address works with every one of them.

Security

  • No public listener. A scope: "mesh" forward is bound only on the rig's private mesh address. Nothing about it exists on the internet to scan.
  • Encrypted on every hop. Mesh traffic travels inside WireGuard from each machine to Route6's hub (or inside HTTPS, on the relay below) and never crosses the internet in the clear. The hub forwards it between your two machines, so it is encrypted in transit, not end to end.
  • Only your team can reach it. By default any agent in your team may call any mesh endpoint. To restrict who may reach the rig, set per-agent rules on the dashboard's Agent Security page.
  • The model stays on 127.0.0.1. Nothing on the rig's local network can reach it either.
  • socat listens on 127.0.0.1 only (bind=127.0.0.1), so other machines on the laptop's network (a café, a hotel) cannot use your relay.

Networks that block UDP

The mesh normally runs over WireGuard, which uses UDP. Some networks (hotels, guest Wi-Fi, corporate networks) block it. The client then switches to a relay over HTTPS on port 443 by itself; r6me status shows "mesh_transport": "relay" instead of "wg". Your app needs no change.

Over the relay, connections carry HTTP only, which is exactly what model servers speak, with two limits:

  • 10 MiB per request. Enough for very long prompts. A larger request gets 413 Request body over the mesh relay limit of 10 MiB.
  • 300 seconds without any response byte. This covers prompt processing on a slow GPU. Once tokens are streaming, a long answer is never cut off.

Over WireGuard there are no such limits.

Troubleshooting

curl: (7) Failed to connect to localhost port 11434 on the laptop: socat is not running. Start it, or check systemctl status rig-llm.

403 Forbidden: your app is calling the mesh name directly through the proxy. Point it at http://localhost:11434 and let socat carry it.

curl: (52) Empty reply from server: socat is running but cannot reach the rig. Check the Route6 client on both machines with r6me status (both should show "transport": "connected"), and that the forward exists on the rig with port_forward { action: "list" }.

The model list is empty or a model is missing: that is the rig's Ollama talking. Pull the model on the rig (ollama pull …); the laptop needs nothing.