From HTTP Endpoints to Policy-Gated Local Inference

A local model is not automatically a safe model. The moment it can read files, call tools, or sit inside an organization's workflow, it becomes a security boundary. Our recent PI work makes that boundary explicit: llama.cpp is the inference engine, but pi-daemon is the governed runtime.

That is a meaningful change from the familiar pattern of launching a model behind an HTTP server and pointing clients at a URL. HTTP is excellent when a service must be shared across networks. It is also a large, general-purpose interface: URL routing, headers, parsers, content negotiation, streaming, monitoring endpoints, browser-facing behavior, authentication middleware, and network exposure all become part of the model's reachable surface. If the model is meant to be a private, single-user capability on one workstation, we asked a simpler question: why expose a network service at all?

Architecture targets, not benchmark claims: the native path is Linux-first and deliberately bounded.

The rewrite: make inference a capability

In the old shape, a client talks to a model server. In the new shape, a client requests a completion from a local control plane that owns the model, checks the caller, validates the request, and decides whether the model is ready. pi-daemon owns the llama.cpp context and the EnTT inference world. libv_pi and pi_agent are clients. Tools stay on the client side; only tool definitions cross the boundary as metadata.

What changed in our code

The recent implementation pass replaced the HTTP client/server path with a native protocol and a split runtime. The changes are not a wrapper around an HTTP endpoint; they move ownership of model loading and inference into the daemon itself.

Security is a chain, not a word

We use “Crypt-authorized” precisely. Crypt Keeper issues an opaque capability tied to the active session, policy hash, model ID, scope, protocol version, and expiration. The daemon validates that capability through the Crypt Keeper control socket before completing a request. Logout and teardown invalidate the authorization.

We do not claim that a Unix-domain socket magically encrypts bytes. The local transport is protected by a private pathname, restrictive permissions, and Linux peer-credential checks; authorization is then reinforced by a session-bound capability and policy validation. If an organization needs cryptographic protection against a privileged local observer or a hostile kernel, this design needs an additional mechanism and a different threat model. Accuracy here matters more than marketing.

Why not just harden the HTTP server?

There is nothing inherently insecure about HTTP when it is correctly authenticated, authorized, patched, rate-limited, monitored, and isolated. llama.cpp's official server documentation describes a useful HTTP product with OpenAI-compatible chat-completion routes, streaming, embeddings, monitoring, and a web UI. Those are strengths for a network service. They are also capabilities a single-user local inference path does not need.

Our design decision is therefore about surface area and placement, not a claim that HTTP is bad. A private native socket removes the listening TCP port and the HTTP routing/parser stack from this production path. The daemon still has to defend its framing, authorization, lifecycle, model loader, and resource limits. We traded one broad interface for a smaller interface we can own end to end.

Table 1 — design comparison, not a universal security ranking. A correctly deployed HTTP service can still be the right choice for shared inference.

The operating sequence

The useful property of this architecture is that the security story follows the lifecycle. Operators do not start a model and hope the rest of the system catches up later.

1. Provision: Crypt Keeper supplies the active PI policy and issues a short-lived capability for the approved model and scope.
2. Validate: pi-daemon resolves the configured asset root, checks the manifest, rejects traversal and ambiguity, and computes the local model hash.
3. Load: The daemon loads the approved GGUF into its model registry, reports loading, and reaches ready only after validation and model load succeed.
4. Connect: pi_agent or libv_pi connects to the private inference socket. Peer identity and the capability are checked before completion.
5. Complete: The request is decoded into ECS stages: prompt construction with the loaded chat template, tokenization, generation, and response construction.
6. Act locally: The client parses any tool call, checks its configured tool set and workspace path boundary, executes locally, and sends only the next bounded request.
7. Revoke: Logout, expiry, policy mismatch, shutdown, or a malformed request stops the path rather than silently falling back to an unauthenticated server.

The organization-level payoff

This is a better fit for teams that want local AI without turning every developer laptop, research workstation, or incident-response host into an ad hoc internal API product. The model can remain local for privacy and resilience. The organization can still define which model identity is approved, which capabilities are live, which client tools are configured, and when the service is considered healthy.

What remains deliberately unfinished

This is an implementation milestone, not a claim of solved local AI security. The current specification is Linux-first. Windows named-pipe parity is a follow-up path. We still need broader integration testing, operational telemetry, stronger hardening against compromised same-user processes (currently limited to peer-UID checks and path sandboxing without process isolation via seccomp, Landlock, or namespaces), and a clear policy for sandboxing the client-side tools. A Unix socket can reduce accidental exposure; it cannot make a compromised account trustworthy.

We also keep the threat model honest around encryption. Crypt Keeper protects and governs session material and capability issuance. The local socket relies on operating-system isolation and authorization, not on an invented promise of wire encryption. That distinction lets a security team decide whether the remaining local threat is acceptable, or whether to add an encrypted transport and stronger host isolation for a different deployment class.

Our thesis

Running a local model should look less like opening a port and more like activating a narrowly scoped device capability. The model is valuable. The model is also untrusted output that can influence tools, files, and people. Put policy in front of it, keep identity in the request path, verify the asset before loading, bound the protocol, and leave consequential tools behind a separate client-side gate.

That is the process we are building in CatGame: a local AI runtime that earns readiness, rather than assuming it; that treats inference as a governed operation, rather than a URL; and that makes the hardening boundary visible enough for an organization to review.

Verified references

1. Linux unix(7) — pathname Unix sockets, permissions, peer credentials, and the distinction from abstract sockets.
2. Linux socket(7) — socket-level behavior and options.
3. llama.cpp server documentation — the HTTP/OpenAI-compatible server capabilities we intentionally do not place in the local production path.
4. OWASP LLM06: Excessive Agency — why excessive functionality, permissions, and autonomy turn model output into system risk.
5. OWASP Top 10 for LLM Applications — prompt injection and related application risks.
6. NIST AI 600-1 — the Generative AI Profile for the AI Risk Management Framework.

Implementation references: PI API docs, our agent-security field note, and the Hugging Face incident review.