Skip to main content

Setup Guide

To use the LLM Gateway, administrators and developers each perform preparation on separate screens (SaaS only).

Overview​

RoleScreenTasks
AdministratorAdmin consoleModel definition (registering available models), IP restriction, and viewing usage logs
DeveloperConsoleCreating an LLM Gateway client (obtaining authentication information)
Prerequisites
  • Administrators can log in to the admin console, and developers can log in to Console (developer role or higher)

Administrator Preparation (Admin Console)​

Defining Models​

Administrators register the available models in the "LLM Gateway Management" menu of the admin console. The registered public model name is the name that developers pass in the request model field, and it is returned by GET /llm/v1/models. The form is organized into "Basic Settings / Connection / Routing & Budget / Administrator notes".

Basic Settings​

ItemRequiredDescription
Public Model Name✓The public name clients send (the lookup key). Developers pass it in model, and it is returned by GET /v1/models.
DeploymentAn identifier that groups multiple deployments under the same public model name.
EnableWhether this model definition is enabled or disabled.

Routing & Budget​

ItemRequiredDescription
TierRouting tier (high / middle / low / unset). Registering multiple tiers under the same public model name enables "Auto Routing" (below).
WeightWithin the same priority, higher values are used first (e.g. 5 before 1). It is not a traffic ratio. Default 1.
PriorityRouting priority (a smaller number is higher priority).
Daily budgetDaily cost cap. Choose "Unlimited" or "Set a limit"; when you set a limit, enter the limit (USD / day).
Behavior When Limits Are Exceeded

Exceeding the daily budget returns 429 (with x-should-retry: false). See Errors in the Reference for details.

Connection​

Connection settings follow the LiteLLM format. You can enter them in Simple mode or JSON mode (switchable via tabs). The fields in Simple mode are:

ItemRequiredDescription
litellm model✓Specify it in LiteLLM's provider/model format (e.g. azure/gpt-5.5).
API KeyThe provider authentication key (not required for proxy-based or local models).
API Base URLThe provider endpoint URL (not needed for official OpenAI; specify it for Azure / self-hosted).
API VersionAPI version (required for Azure OpenAI, etc.).
Additional litellm ParamsSpecify other LiteLLM parameters as JSON. Values for the dedicated fields above (model, api_key, api_base, api_version) are ignored if entered here.

Administrator notes (Optional)​

You can save your own operational notes as free-form JSON. They are not used for routing or model capability decisions.

Do Not Hard-Code Model IDs

Available models are defined per tenant by administrators, so they vary by environment. In your application, retrieve them dynamically with GET /llm/v1/models (Reference) or check with your administrator.

Auto Routing (Optional)​

When two or more tiers (high / middle / low) are registered for the same model name, an administrator can enable tenant-wide Auto Routing from the "LLM Gateway Management" menu in the admin console. When enabled, prompts are automatically routed to a model of the appropriate tier based on their difficulty. Developers simply specify a model name as usual; the tier is selected automatically.

IP Restriction (Optional)​

In the "LLM Gateway Allowed IPs" screen of the admin console, you can restrict the networks that can access the LLM Gateway.

  1. Register allowed IPs — Register an IP address (IPv4 / IPv6 / CIDR formats are supported; e.g. 192.168.1.0/24) and a description.
  2. Enable IP restriction — When enabled, only registered IP addresses can access the LLM Gateway. You cannot enable it when no allowed IPs are registered.

When enabled, requests from networks that are not allowed are rejected with 403.

Usage Logs​

Usage such as model, provider, token count, cost, and latency can be viewed in the admin console and via the llm-gateway-logs endpoint in the System Integration API Reference.

Developer Preparation (Console)​

From the "LLM Gateway" menu in Console, register a new client, then use the detail screen to review credentials and manage the client lifecycle (enable / disable / edit / delete).

1. Enter Basic Information​

In the "LLM Gateway" menu, click "Register" and enter the following to create the client.

ItemRequiredDescription
LLM Client Name✓Display name of the client (max 255 characters).
DescriptionDescription of the purpose or use.

2. Review Authentication Information​

After creation, you can review and manage credentials under "Developer Info" on the detail screen. Depending on your use case, use either an API key (set directly in the SDK) or the Client Credentials flow (obtain an access token with the Client ID / Client Secret; see the Authentication API Reference (SaaS) for the procedure).

ItemDescription
Client IDIdentifier of the client. Used in the Client Credentials flow.
Client SecretSecret for the Client Credentials flow. Shown masked; it can be copied and regenerated ("Regenerate Secret").
API KeyA fixed key (agtstr_...) that can be set directly in the OpenAI / Anthropic SDK. Sent via Authorization: Bearer or x-api-key.
Token URLEndpoint for obtaining an access token in the Client Credentials flow.

Issuing and Managing the API Key​

The API key is not issued immediately after creation. Manage it on the detail screen with the following actions.

ActionDescription
Issue API KeyIssues a fixed API key (agtstr_...).
Regenerate API KeyIssues a new key. The previous key becomes invalid.
Delete API KeyDeletes the key. It can no longer be used for authentication.
Handling Authentication Information

API keys and Client Secrets are confidential information equivalent to passwords. Do not hard-code them in source code; manage them with environment variables or a Secret Manager. Regenerating a key or secret causes all calls using the previous key / secret to fail, so review the impact beforehand.

3. Enable / Disable​

From the header of the detail screen, you can enable / disable the client. A disabled client cannot call the LLM Gateway. You can delete clients that are no longer needed.

Enabling and contract slots

Only enabled clients can call the LLM Gateway. Enabled clients use LLM contract slots; once the limit is reached, you cannot create additional clients ("The contracted seat limit has been reached, so LLM access cannot be granted. Enabled clients count toward the limit." is shown). Disabling a client frees one slot.

Next Steps​