“Self-hosted” sounds like a privacy guarantee, but it only describes part of an AI coding stack. The agent backend can run inside your infrastructure while model calls, telemetry, package downloads or tool integrations still cross the boundary.
IBM announced general availability of self-hosted IBM Bob for customer-controlled Red Hat OpenShift environments, including on-premises and private-cloud deployments, with support for air-gapped configurations. That makes a broader developer question timely: how do you verify that a self-hosted coding agent actually keeps code private?
AiFolder test status: this is a verification methodology, not a claim that Bob or another product leaked data. Tests use synthetic repositories and canary values only.
Map the data path before testing
A coding agent can touch source files, prompts, embeddings, build logs, terminal output, screenshots, Git metadata and credentials. Draw every destination: model endpoint, telemetry collector, update server, MCP tools, package registry and source-control remote.
Five tests for a “private” deployment
1. Network-deny test
Run the agent behind an outbound deny policy. A genuinely air-gapped workflow should still perform its documented local functions. Record every failed DNS request and destination rather than merely checking whether the UI loads.
2. Canary-code test
Add a harmless unique token to a synthetic repository and observe outbound payloads at a controlled gateway. Never use real credentials. The goal is to determine which components transmit repository content and under what action.
3. Model-endpoint test
Self-hosted orchestration does not imply self-hosted inference. Verify whether the selected model is local, in a private endpoint, or an external service. Document the switch because privacy changes with configuration.
4. MCP/tool test
An agent can be locally hosted and still send context to external tools. Test each MCP server separately, apply least privilege and log arguments. This is especially important for search, issue trackers and browser tools.
5. Telemetry test
Inventory analytics, crash reporting, traces and update checks. Metadata can reveal repository names, paths or workflow details even when source text is excluded.
What IBM Bob's release tells us
IBM says the Bob backend can run on an organization's own OpenShift cluster and can use either an open-weight model on owned GPUs or supported model services. That distinction is useful: deployment location and inference location are separate controls.
Self-hosted vs local vs air-gapped
| Mode | Useful meaning | What still needs verification |
|---|---|---|
| Self-hosted | Application backend is under your control | Model, telemetry, tools |
| Local model | Inference happens on controlled hardware | App/tool egress |
| Air-gapped | No external network path | Update/import/export process |
A practical acceptance criterion
For a strict private-code environment, a useful target is: repository content never leaves approved infrastructure; model endpoints are explicitly enumerated; tool access is allowlisted; outbound traffic is observable; and audit logs show what context each agent action consumed.
