Provider credentials
Give the gateway provider keys as environment references, approve self-hosted endpoints, and use AWS identities for Bedrock.
Provider credentials never go through the browser or into the database. A connection in Admin stores a reference; you, the operator, supply the secret on the server.
API keys as environment variables
- Put the key in an environment variable with a name you choose, for example
OPENAI_API_KEY. - Add the name to
GATEWAY_SECRET_ENV_ALLOWLIST(comma-separated). The default is empty: no variable can be referenced. - Restart. In Admin, the connection's credential is "Secret in an environment variable" with that name.
GATEWAY_SECRET_ENV_ALLOWLIST=OPENAI_API_KEY,ANTHROPIC_API_KEY,OPENROUTER_API_KEY,SMTP_PASSWORD
OPENAI_API_KEY=…
ANTHROPIC_API_KEY=…
OPENROUTER_API_KEY=…- The container image can read
OPENAI_API_KEYandANTHROPIC_API_KEYfrom files with_FILE; other names must be set directly by your orchestrator's secret mechanism. - The same mechanism supplies the SMTP password (
env:SMTP_PASSWORDin Settings › Email). - References and values are never returned by the API or written to logs.
- Removing a name from the allowlist disables every connection that uses it, after a restart.
To rotate a key: create the new key at the provider, update the variable, restart, check that requests work, then revoke the old key.
Amazon Bedrock
Bedrock uses AWS identities, never keys typed into the gateway:
| Mode | Reference | Operator setup |
|---|---|---|
| Server identity | aws:default | The AWS SDK's default credential chain. Prefer a workload role: ECS task role, EC2 instance profile, EKS role. |
| Named profile | aws:profile:<name> | A profile in the server's shared AWS config. Add its name to GATEWAY_AWS_PROFILE_ALLOWLIST. |
| Assume role | aws:role:<arn> | The server identity needs sts:AssumeRole on the role, and the role's trust policy must allow it (and require the external ID, if you use one). |
For a PrivateLink endpoint, add its exact HTTPS origin to GATEWAY_BEDROCK_ENDPOINT_ALLOWLIST. Allowlists are checked when a connection is saved and again on every request, so removing an entry and restarting disables matching connections.
Give the identity bedrock:InvokeModel and bedrock:InvokeModelWithResponseStream, restricted to the models and inference profiles you intend to offer. The gateway ignores AWS_ENDPOINT_URL and similar endpoint overrides.
Self-hosted model servers
Self-hosted servers (vLLM, SGLang, Ollama, other OpenAI-compatible servers) are reachable only at addresses you approve in GATEWAY_LOCAL_UPSTREAMS:
GATEWAY_LOCAL_UPSTREAMS='[
{"endpoint": "http://models.example.internal:8000/v1", "addresses": ["10.10.1.20"]},
{"endpoint": "https://gpu.example.internal/vllm/v1", "addresses": ["10.10.1.21"]}
]'- Each
endpointis an exact base URL ending in/v1, without credentials, query or fragment. - Each is pinned to its IP addresses: the gateway never looks the name up in DNS. Up to 64 endpoints, 16 addresses each.
- Plain HTTP is allowed only to private addresses (loopback only in development). Metadata, link-local and other special addresses are refused. HTTPS keeps certificate checks.
- A connection may use no authentication, or an
env:reference sent as a bearer token to that server only.
The dashboard can only choose among approved endpoints; it can't approve a new one.
Rules for every connection
- Cloud providers use their fixed HTTPS addresses (OpenAI, Anthropic, OpenRouter), or AWS's for Bedrock.
- No redirects are followed, proxy environment variables (
HTTP_PROXY,HTTPS_PROXY) are ignored, and nothing is retried implicitly. - Callers' API keys are never forwarded to a provider.
Enforce outbound network rules too: the gateway's approval lists are not a firewall.