Architecture
Skill-based LLM routing on NVIDIA Jetson Orin Nano. Runs local Ollama first, escalates to the best cloud provider when confidence is low.
api.pazlabs.ioThe chat UI POSTs to the public API endpoint with the user's query and a session ID.
A persistent Cloudflare tunnel proxies the request to the edge-router FastAPI service running on the local Jetson at port 8000 — no port-forwarding needed.
The router forwards the query to the local Ollama instance running gemma2 or llama3.2. No cloud API keys used at this stage.
The response is passed through a lightweight scorer that estimates how confident and complete the local answer is. Scores below the threshold trigger cloud escalation.
If the local model scores well, the answer is returned immediately with metadata: model_used, source: "local", tokens, latency_ms.
A skill classifier determines the query type and selects the most capable model:
Every response includes: answer, model_used, source, skill, confidence_score, latency_ms, and tokens (input/output/total). The chat UI logs this to Firestore.
Also on this router
tripBuddy sends its Plan chat requests to a separate, additive trip-planning endpoint here. It routes local first (qwen2.5:7b on Ollama) and escalates to Gemini or Grok only under daily caps; expense messages are forced local. The chat path above is unchanged. tripBuddy architecture →
Tech Stack