# Domain Settings Guide

This guide explains how to configure custom domains and upgrade SSL certificates for the AGENTIC STAR marketplace edition.

## Target Versions and Checking the Entry

The entry of AGENTIC STAR (the part that receives requests from browsers and APIs) differs by version. Certificates and related settings are configured in different places, so check the entry of your environment first and use the steps in the matching section.

| Version                | Entry                                                           | Steps to use                                                                                             |
| ---------------------- | --------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------- |
| **v3.4.0 or later**    | NGINX Gateway Fabric (Gateway and HTTPRoute of the Gateway API) | The main text of this page (from [Default State](#default-state) to [Troubleshooting](#troubleshooting)) |
| **v3.3.x and earlier** | ingress-nginx (Ingress)                                         | [Steps for v3.3.x and Earlier (ingress-nginx)](#v33-ingress-nginx)                                       |

You can check the entry with the following command.

```bash
kubectl get svc -n ingress-nginx -l app.kubernetes.io/purpose \
  -o custom-columns='NAME:.metadata.name,ENTRY:.spec.selector.gateway\.networking\.k8s\.io/gateway-name'
```

- Every row shows a name such as `front` or `admin` in `ENTRY`: the entry is NGINX Gateway Fabric (use the steps for v3.4.0 or later)
- `ENTRY` is `<none>`: the entry is ingress-nginx (use the steps for v3.3.x and earlier)
- Nothing is shown (an environment without LoadBalancer Services): decide by the version shown on the version management screen of the admin console

> **Environments updated from v3.3.x to v3.4.0**
>
> The certificate and the settings for placing a CDN or load balancer in front, made with the steps for v3.3.x and earlier, are not used by the entry of v3.4.0 or later. If you have set up your own domain, follow [Migrating the Settings of an Environment Updated from v3.3.x](#migrate-from-v33).

## Default State

AGENTIC STAR ships with **HTTPS using self-signed certificates**.

### LoadBalancer Endpoint Configuration

Four LoadBalancer endpoints are assigned. Each endpoint has one Gateway of the entry (NGINX Gateway Fabric). Both the Services and the Gateways are in the `ingress-nginx` Namespace.

| Service Name           | Gateway Name | Purpose                |
| ---------------------- | ------------ | ---------------------- |
| `ingress-nginx-front`  | `front`      | Main application       |
| `ingress-nginx-admin`  | `admin`      | Admin console          |
| `ingress-nginx-auth`   | `auth`       | Authentication service |
| `ingress-nginx-extapi` | `extapi`     | External API           |

A single self-signed certificate covering all 4 endpoints as SANs is applied. The certificate is in a TLS Secret in the `ingress-nginx` Namespace, and the HTTPS listeners of all Gateways refer to the same Secret. Requests that arrive over HTTP (80) are redirected to HTTPS.

> **Service and Namespace names**
>
> The LoadBalancer Service names (such as `ingress-nginx-front`) and the Namespace name (`ingress-nginx`) are the same as in v3.3.x and earlier. ingress-nginx itself is not installed in new environments of v3.4.0 or later.

You can check the state of the Gateways and the name of the Secret the HTTPS listeners refer to with the following command (the later steps also use it to verify settings).

```bash
kubectl get gateway -n ingress-nginx -o custom-columns=\
'NAME:.metadata.name,'\
'PROGRAMMED:.status.conditions[?(@.type=="Programmed")].status,'\
'SECRET:.spec.listeners[?(@.name=="https")].tls.certificateRefs[0].name,'\
'CERT:.status.listeners[?(@.name=="https")].conditions[?(@.type=="ResolvedRefs")].status'
```

The output in the default state is as follows. The entry is healthy when `PROGRAMMED` and `CERT` (whether the certificate Secret could be read) are `True`. `SECRET` is the name of the Secret of the shipped self-signed certificate (`selfsigned-tls-cert` in this example; if the name differs in your environment, use the displayed name in the following steps).

```text
NAME     PROGRAMMED   SECRET                CERT
admin    True         selfsigned-tls-cert   True
auth     True         selfsigned-tls-cert   True
extapi   True         selfsigned-tls-cert   True
front    True         selfsigned-tls-cert   True
kc       True         selfsigned-tls-cert   True
```

> **Fifth entry (external entry of the authorization server)**
>
> The authorization server has a **fifth entry** that lets users sign in with their own account from external tools (AI clients such as Claude Code or Codex, and OIDC-capable applications). It ships closed; only the 4 endpoints in the table above are exposed. When you open it, `ingress-nginx-kc` is added as a fifth LoadBalancer (the Gateway `kc` exists even while the entry is closed). See [External Entry of the Authorization Server (Sign-in from External Tools)](#kc-direct-login) for how to open and close it.

#### Endpoint Format Differs by Cloud

The value shown in the `EXTERNAL-IP` column of `kubectl get svc -n ingress-nginx` differs depending on the cloud you use.

| Cloud     | EXTERNAL-IP value | DNS record | Notes                                                                  |
| --------- | ----------------- | ---------- | ---------------------------------------------------------------------- |
| **Azure** | Static IP address | A record   | Standard Load Balancer Public IP (static by default)                   |
| **AWS**   | ELB DNS name      | CNAME      | Network Load Balancer DNS name (`xxxxxxxx.elb.<region>.amazonaws.com`) |
| **GCP**   | Static IP address | A record   | Direct IP for both global and regional load balancers                  |

> **Why AWS Does Not Show an IP**
>
> AWS Network Load Balancers use ENIs spanning multiple Availability Zones, so they do not have fixed IPs — AWS may change the underlying IPs over time. Always reference NLBs by their ELB DNS name. This guide uses terms like "IP1–IP4" and "IP address"; on AWS, read these as "ELB DNS name."

### Default Limitations

| Item                            | Status                                                          |
| ------------------------------- | --------------------------------------------------------------- |
| HTTPS communication             | Works (browser certificate warning shown)                       |
| OAuth / Keycloak authentication | Works                                                           |
| Secure Cookie                   | Works                                                           |
| WebAuthn (Passkey)              | **Not available** (available after proper domain configuration) |

> **danger**
>
> Running in production with self-signed certificates carries the following risks:
>
> - User confusion from browser certificate warnings
> - Inability to detect man-in-the-middle attacks (users develop a habit of ignoring warnings)
> - WebAuthn (Passkey) functionality unavailable
> - Some client SDKs may refuse connections
>
> **Always configure a proper domain and valid SSL certificate before making the service publicly available.**

## Choosing an SSL Configuration Method

After obtaining your domain, upgrade to a valid SSL certificate using one of the following methods.

| Method                          | SSL Termination              | Certificate Management | Recommended For                                       |
| ------------------------------- | ---------------------------- | ---------------------- | ----------------------------------------------------- |
| **CDN / Load Balancer**         | Cloud service                | Managed by cloud       | When WAF / DDoS protection or CDN features are needed |
| **External Certificate**        | Entry (NGINX Gateway Fabric) | Manual                 | When you already have a certificate                   |
| **Let's Encrypt Auto-Issuance** | Entry (NGINX Gateway Fabric) | cert-manager           | When you want free, auto-renewed certificates         |

## Common Upgrade Steps

The following common steps are required regardless of which method you choose.

### 1. Obtain a Domain and Configure DNS Records

Prepare domains for each service and configure DNS records to point to each LoadBalancer endpoint. The record type differs by cloud.

| Example Domain      | Target Service         | Purpose                                                                           |
| ------------------- | ---------------------- | --------------------------------------------------------------------------------- |
| `chat.example.com`  | `ingress-nginx-front`  | Main application                                                                  |
| `admin.example.com` | `ingress-nginx-admin`  | Admin console                                                                     |
| `auth.example.com`  | `ingress-nginx-auth`   | Authentication service                                                            |
| `api.example.com`   | `ingress-nginx-extapi` | External API                                                                      |
| `kc.example.com`    | `ingress-nginx-kc`     | External entry of the authorization server ([only when opened](#kc-direct-login)) |

#### DNS Configuration by Cloud

**Azure / GCP (A records pointing to IPs):**

```text
chat.example.com.   IN  A  <IP1>
admin.example.com.  IN  A  <IP2>
auth.example.com.   IN  A  <IP3>
api.example.com.    IN  A  <IP4>
kc.example.com.     IN  A  <IP5>   # only when the external entry of the authorization server is open
```

**AWS (CNAME records pointing to ELB DNS names):**

```text
chat.example.com.   IN  CNAME  <ELB DNS name of ingress-nginx-front>
admin.example.com.  IN  CNAME  <ELB DNS name of ingress-nginx-admin>
auth.example.com.   IN  CNAME  <ELB DNS name of ingress-nginx-auth>
api.example.com.    IN  CNAME  <ELB DNS name of ingress-nginx-extapi>
kc.example.com.     IN  CNAME  <ELB DNS name of ingress-nginx-kc>   # only when the external entry of the authorization server is open
```

> **note**
>
> The endpoints for each Service (IPs on Azure/GCP, ELB DNS names on AWS) remain unchanged after domain configuration. The DNS records simply point to the corresponding endpoints; routing does not change.
>
> On the AWS Marketplace edition, NLB IPs may change on AWS's side, so you must always use CNAME records pointing to the ELB DNS names. Registering the NLB IPs directly in A records will cause connectivity issues when the IPs change. If you use Route 53, you can also use Alias records (A record alias) to target the NLB.

### 2. Check the Current Helm Chart Version

```bash
HELM_DRIVER=configmap helm list -A
```

Check each service's chart name and version (e.g., `agentic-star-front-3.4.0`, `agenticai-admin-3.4.0`, `agentic-star-edge-3.4.0` for the entry, etc.). The following steps use the same chart version.

### 3. Add the Helm Repository

```bash
helm repo add agenticai-prod https://agtstrmarketplacecharts.blob.core.windows.net/charts
helm repo update
```

### 4. Update Domain / URL Settings

Change the IP-based URLs in each service's ConfigMap to domain-based URLs using `helm upgrade --reuse-values --set`. The `--reuse-values` flag preserves all existing settings, and only the values specified with `--set` are overwritten.

> **caution**
>
> Specify the same chart version that is currently deployed, as confirmed in step 2.

**Main Application (librechat):**

```bash
HELM_DRIVER=configmap helm upgrade librechat agenticai-prod/agentic-star-front \
  --version 3.4.0 \
  --reuse-values \
  --set appConfig.domainClient="https://chat.example.com" \
  --set appConfig.domainServer="https://chat.example.com" \
  --set appConfig.authBaseUrl="https://auth.example.com" \
  -n agentic-star-front
```

**Admin Console (agenticai-admin):**

```bash
HELM_DRIVER=configmap helm upgrade agenticai-admin agenticai-prod/agenticai-admin \
  --version 3.4.0 \
  --reuse-values \
  --set env.AFD_CHATBOARD_ADMIN_DOMAIN="admin.example.com" \
  --set env.AFD_CHATBOARD_FRONT_AUTH_DOMAIN="auth.example.com\,chat.example.com" \
  --set env.ALLOWED_ORIGINS="https://auth.example.com\,https://admin.example.com" \
  --set env.CHATBOARD_FRONT_DOMAIN="chat.example.com" \
  --set env.EXTAPI_PUBLIC_URL="https://api.example.com" \
  --set env.MP_OAUTH_TOKEN_URL="https://auth.example.com/api/v1/oauth/token" \
  --set env.AUTH_PUBLIC_URL="https://auth.example.com" \
  --set env.ADMIN_PUBLIC_URL="https://admin.example.com" \
  -n agentic-star-admin
```

**Authentication Service (librechat-auth):**

```bash
HELM_DRIVER=configmap helm upgrade librechat-auth agenticai-prod/agentic-star-auth \
  --version 3.4.0 \
  --reuse-values \
  --set env.ADMIN_CONSOLE_URL="https://admin.example.com" \
  --set env.ALLOWED_ORIGINS="https://chat.example.com\,https://auth.example.com" \
  --set env.ALLOWED_REDIRECT_URIS="https://chat.example.com/oauth/librechat/callback\,https://auth.example.com/*\,https://admin.example.com/*" \
  --set env.APP_URL="https://auth.example.com" \
  --set env.LIBRECHAT_URL="https://chat.example.com" \
  --set env.NEXT_PUBLIC_APP_URL="https://auth.example.com" \
  --set env.WEBAUTHN_ORIGIN="https://auth.example.com" \
  --set env.WEBAUTHN_RP_ID="auth.example.com" \
  -n librechat-auth
```

**Authentication platform (keycloak-extensions):**

Change `WEBAUTHN_RP_ID` and `WEBAUTHN_ORIGIN`, which are used to verify passkeys, to the new domain of the authentication service. These two are inside `keycloakx.extraEnv` (a multi-line string), so export the current values, edit them, and apply them.

```bash
HELM_DRIVER=configmap helm get values keycloak-extensions -n librechat-auth -o yaml > keycloak-extensions-values.yaml
```

In `keycloakx.extraEnv` of `keycloak-extensions-values.yaml`, change the value of `WEBAUTHN_RP_ID` to `auth.example.com` and the value of `WEBAUTHN_ORIGIN` to `https://auth.example.com`, then run the following.

```bash
HELM_DRIVER=configmap helm upgrade keycloak-extensions agenticai-prod/keycloak-extensions \
  --version 3.4.0 \
  --reuse-values \
  -f keycloak-extensions-values.yaml \
  -n librechat-auth
```

The exported file contains secret values such as connection passwords. Delete it after the work.

**External API (agenticai-extapi):**

```bash
HELM_DRIVER=configmap helm upgrade agenticai-extapi agenticai-prod/agenticai-extapi \
  --version 3.4.0 \
  --reuse-values \
  --set config.APIGW_PUBLIC_URL="https://api.example.com" \
  --set config.A2A_PUBLIC_URL="https://api.example.com" \
  --set config.MP_OAUTH_TOKEN_URL="https://auth.example.com/api/v1/oauth/token" \
  -n agenticai-extapi-service
```

**External entry of the authorization server (only when opened):**

In an environment where the external entry of the authorization server is open, run the open/close command again with the new domain instead of `helm upgrade`. It replaces the external address the authorization server announces and the token issuer passed to the external API at once, and verifies that the issuer has switched. For the values to pass when running `helm upgrade` by hand, see the [Helm Values Reference](https://developers.fd.agenticstar.tm.softbank.jp/en/docs/marketplace/build/reference/helm-values-ref.md#kc-direct-login-values).

```bash
bash kc_direct_login.sh open --address kc.example.com
```

Because the issuer changes, users who sign in from external tools must sign in again.

> **note**
>
> - The entry (NGINX Gateway Fabric) accepts requests without limiting the host name. You do not need to register host names on the entry when you change domains (there is no setting that corresponds to `server-alias` in v3.3.x and earlier)
> - When the external entry of the authorization server is open, the open/close command above replaces the token issuer passed to the external API
> - When specifying values containing commas with `--set`, escape them with `\,`

### 5. Configure SSL Certificates

Proceed to one of the methods below. Every method changes the settings of the entry with the steps in the next section.

## Changing the Settings of the Entry (NGINX Gateway Fabric)

The settings of the entry are determined by the values of the Helm release `agentic-star-edge` (Namespace `ingress-nginx`). The certificate and the handling of HTTP are the following two values inside `gateways`, the list of Gateways.

| Value                                            | Meaning                                                                                                                                             | Default                               |
| ------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------- |
| `gateways[].listeners.https.certificateRef.name` | Name of the certificate Secret used by the HTTPS listener. Place the Secret (type `kubernetes.io/tls`) in the `ingress-nginx` Namespace             | Secret of the self-signed certificate |
| `gateways[].httpsRedirect`                       | `true`: redirect requests that arrive over HTTP (80) to HTTPS. `false`: deliver requests that arrive over HTTP (80) to the applications as they are | `true`                                |

Because `gateways` is a list, export the current list, edit it, and apply it by passing the whole list.

**1. Export the current `gateways` value to a file**

```bash
HELM_DRIVER=configmap helm get values agentic-star-edge -n ingress-nginx -o yaml \
  | yq '{"gateways": .gateways}' > edge-gateways.yaml
```

Check that `edge-gateways.yaml` contains the 5 Gateways `front`, `admin`, `auth`, `extapi`, and `kc`.

**2. Edit the values in the file**

What to change is described in the steps of [Method 1](#method-1), [Method 2](#method-2), and [Method 3](#method-3).

**3. Apply**

```bash
HELM_DRIVER=configmap helm upgrade agentic-star-edge agenticai-prod/agentic-star-edge \
  --version 3.4.0 \
  --reuse-values \
  -f edge-gateways.yaml \
  -n ingress-nginx
```

For `--version`, specify the current version of `agentic-star-edge` confirmed in common step 2. Changing the settings does not change the LoadBalancer addresses (IP addresses or ELB DNS names).

**4. Verify**

With the `kubectl get gateway` command in [Default State](#default-state), check that `PROGRAMMED` and `CERT` are `True` for the 5 Gateways.

> **caution**
>
> - Do not specify only part of `gateways`, as in `--set 'gateways[0]…'`. A list is replaced as a whole by what you specify, so the apply fails with an error or the settings of the other Gateways are lost. Always pass the whole list in a file as in the steps above
> - Use mikefarah's `yq` v4. If you do not have `yq`, save only the `gateways:` part of the output of `helm get values agentic-star-edge -n ingress-nginx -o yaml` to `edge-gateways.yaml` and edit it with an editor. Do not save the other parts of the output, because they may contain secret values such as connection passwords

> **Check the entry settings after an update**
>
> When an AGENTIC STAR update upgrades the chart of the entry, the values changed with the steps in this section (the certificate Secret name, the HTTP redirect, and the user IP address settings) may return to the default values or to the settings of the entry in v3.3.x and earlier. After an update, check the values with the command in "4. Verify" of this section and `HELM_DRIVER=configmap helm get values agentic-star-edge -n ingress-nginx`, and configure them again if they have changed.

## Method 1: CDN / Load Balancer

Perform SSL termination at the cloud service and connect to the entry over HTTP (80).

```text
User → HTTPS (443) → CDN / Load Balancer (SSL termination) → HTTP (80) → Entry (NGINX Gateway Fabric) → Pod
```

In the default state, the entry redirects requests that arrive over HTTP (80) to HTTPS. In a configuration where the CDN / load balancer connects to the entry over HTTP (80), stop this redirect so that requests arriving over HTTP (80) are delivered to the applications.

### Stop the HTTP Redirect of the Entry

```bash
# 1. Export the current gateways value
HELM_DRIVER=configmap helm get values agentic-star-edge -n ingress-nginx -o yaml \
  | yq '{"gateways": .gateways}' > edge-gateways.yaml

# 2. Stop the redirect from HTTP to HTTPS on all Gateways
yq -i '.gateways[].httpsRedirect = false' edge-gateways.yaml

# 3. Apply
HELM_DRIVER=configmap helm upgrade agentic-star-edge agenticai-prod/agentic-star-edge \
  --version 3.4.0 \
  --reuse-values \
  -f edge-gateways.yaml \
  -n ingress-nginx
```

The external entry of the authorization server (the Gateway `kc`) is changed together by the same steps.

After applying, verify with the following commands.

```bash
kubectl get httproute -n ingress-nginx
curl -sS -o /dev/null -w '%{http_code}\n' http://<address of ingress-nginx-front>/
```

- First command: if no redirect HTTPRoute (such as `front-https-redirect`) is shown, the redirect is stopped (`No resources found in ingress-nginx namespace.`)
- Second command: if `200` is returned instead of `301` (the redirect to HTTPS), requests arriving over HTTP (80) reach the application (in an environment that restricts connections to the LoadBalancers, verify through the CDN / load balancer)

> **caution**
>
> After the redirect is stopped, requests that arrive directly at a LoadBalancer over HTTP (80) also reach the applications. Allow connections to the LoadBalancers only from the CDN / load balancer, and set up this restriction before you stop the redirect (the NSG and security group steps in "Cloud-Specific Configuration" below).

> **note**
>
> - The HTTPS (443) listener and certificate of the entry remain as they are
> - In a configuration where the CDN / load balancer also connects to the entry over HTTPS (443), skip these steps and configure a valid certificate on the entry with [Method 2](#method-2) or [Method 3](#method-3)

### User IP Address Settings

When you place a CDN / load balancer (reverse proxy) in front, the source the entry sees is the IP address of the front component, not of the user. As it is, features that use the user's IP address (such as the source IP restriction) are evaluated with the IP address of the front component. Set the following values on the entry so that it reads the user's IP address from the header added by the front component.

| Value                    | Meaning                                                                                                                                           |
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------- |
| `realIp.enabled`         | When `true`, reads the user's IP address from the header below. When `false`, uses the IP address of the source connected directly to the entry   |
| `realIp.header`          | Name of the header that carries the user's IP address                                                                                             |
| `realIp.trustedCidrsCsv` | CIDRs of the sources (the front component) whose header is trusted. Separate multiple CIDRs with commas. Required when `realIp.enabled` is `true` |

The values for each front component are as follows.

| Front component                                              | `realIp.header`    | `realIp.trustedCidrsCsv`                                                  |
| ------------------------------------------------------------ | ------------------ | ------------------------------------------------------------------------- |
| Azure Front Door                                             | `X-Azure-SocketIP` | Address ranges of the service tag `AzureFrontDoor.Backend`                |
| GCP external Application Load Balancer                       | `X-Forwarded-For`  | `130.211.0.0/22,35.191.0.0/16`                                            |
| Others (such as AWS CloudFront or Application Load Balancer) | `X-Forwarded-For`  | CIDRs of the sources from which the front component connects to the entry |

Create `edge-realip.yaml` with the following content, adjust the values to your environment, and apply it.

```yaml
realIp:
  enabled: true
  header: X-Forwarded-For
  trustedCidrsCsv: "203.0.113.0/24,198.51.100.0/24"
```

```bash
HELM_DRIVER=configmap helm upgrade agentic-star-edge agenticai-prod/agentic-star-edge \
  --version 3.4.0 \
  --reuse-values \
  -f edge-realip.yaml \
  -n ingress-nginx
```

You can check the current settings with the following command.

```bash
HELM_DRIVER=configmap helm get values agentic-star-edge -n ingress-nginx -o yaml | yq '.realIp'
```

> **caution**
>
> Specify only the CIDRs of the front component in `realIp.trustedCidrsCsv`. Do not set `realIp.enabled` to `true` in a configuration without a front component. In either case, the user's IP address can no longer be determined correctly.

### Cloud-Specific Configuration

#### Azure: Front Door

Azure Front Door provides security through WAF and SSL/TLS via Azure-managed certificates.

Key steps:

1. Create a DNS zone and configure NS delegation
2. Create an AFD profile and WAF policy
3. Register custom domains and issue certificates
4. Register the IP of each LoadBalancer (such as `ingress-nginx-front`) in origin groups (when the external entry of the authorization server is open, register the IP of `ingress-nginx-kc` as a fifth origin)
5. Configure routing rules to associate domains with origins (associate the authorization server domain in the same way)
6. Update NSG rules (allow access only from AFD)
7. Update the settings of the entry ("Stop the HTTP Redirect of the Entry" and "User IP Address Settings" above)

#### AWS: CloudFront + NLB

On the AWS Marketplace edition, LoadBalancers are provisioned as NLBs, so we recommend configuring CloudFront origins to **point directly at the NLB DNS names**. NLB IPs may change on AWS's side, which makes the conventional pattern of registering NLB IPs in an ALB target group unstable.

1. Issue a certificate with ACM (AWS Certificate Manager) — certificates for CloudFront must be issued in **us-east-1**
2. Create a CloudFront distribution and set each Service's **NLB DNS name** as an origin (4 origins, one per service; when the external entry of the authorization server is open, add the NLB of `ingress-nginx-kc` as a fifth)
3. Use behaviors to route requests to the appropriate origin by host or path (add a behavior for the authorization server domain as well)
4. In Route 53, point each custom domain at the CloudFront distribution domain via **CNAME** (or Alias record)
5. Restrict NLB security groups so they only accept traffic from the CloudFront managed Prefix List (`com.amazonaws.global.cloudfront.origin-facing`)
6. Update the settings of the entry ("Stop the HTTP Redirect of the Entry" and "User IP Address Settings" above)

> **tip**
>
> If you prefer to avoid additional components, you can also place Route 53 directly in front of the NLBs without CloudFront. In that case, terminate SSL at the entry (NGINX Gateway Fabric) or via a TLS listener on the NLB (see "Method 2: External Certificate" or "Method 3: Let's Encrypt Auto-Issuance" below).

> **caution**
>
> A configuration with an ALB in between (using AWS Load Balancer Controller with IP target type) is technically possible, but stacking an NLB and ALB increases operational complexity. The CloudFront → NLB direct configuration above is recommended.

#### GCP: Cloud Load Balancing

1. Create a Google-managed SSL certificate
2. Create an external HTTPS load balancer
3. Register the IP of each LoadBalancer (such as `ingress-nginx-front`) in backend services (when the external entry of the authorization server is open, also add the IP of `ingress-nginx-kc` as a backend and point the authorization server domain to it with a host rule in the URL map)
4. Configure DNS in Cloud DNS
5. Update the settings of the entry ("Stop the HTTP Redirect of the Entry" and "User IP Address Settings" above)

## Method 2: External Certificate

Use an externally issued certificate on the entry (NGINX Gateway Fabric).

```text
User → HTTPS (443) → Entry (NGINX Gateway Fabric) (SSL termination / valid certificate) → HTTP → Pod
```

### Steps

#### 1. Register the Certificate as a Kubernetes Secret

Place the Secret read by the entry in the `ingress-nginx` Namespace (you do not need to place it in the Namespace of each service).

```bash
kubectl create secret tls app-tls-cert \
  --cert=tls.crt --key=tls.key \
  -n ingress-nginx
```

> **note**
>
> - A single certificate is used by all Gateways. Include every domain you configured (for example `chat.example.com`, `admin.example.com`, `auth.example.com`, `api.example.com`) in the certificate's SANs. A wildcard certificate (`*.example.com`) also works
> - When the external entry of the authorization server is open, also include the authorization server domain (e.g., `kc.example.com`) in the SANs
> - Put the server certificate followed by the intermediate certificates in `tls.crt`

#### 2. Switch the Certificate of the Entry

```bash
# 1. Export the current gateways value
HELM_DRIVER=configmap helm get values agentic-star-edge -n ingress-nginx -o yaml \
  | yq '{"gateways": .gateways}' > edge-gateways.yaml

# 2. Set the certificate of the HTTPS listener of all Gateways to the Secret of the valid certificate
yq -i '.gateways[].listeners.https.certificateRef.name = "app-tls-cert"' edge-gateways.yaml

# 3. Apply
HELM_DRIVER=configmap helm upgrade agentic-star-edge agenticai-prod/agentic-star-edge \
  --version 3.4.0 \
  --reuse-values \
  -f edge-gateways.yaml \
  -n ingress-nginx
```

#### 3. Verify

With the `kubectl get gateway` command in [Default State](#default-state), check that `SECRET` is `app-tls-cert` and that `PROGRAMMED` and `CERT` are `True` for the 5 Gateways. Then check the certificate from outside.

```bash
openssl s_client -connect chat.example.com:443 -servername chat.example.com </dev/null 2>/dev/null \
  | openssl x509 -noout -subject -issuer -enddate
```

If the subject, issuer, and expiry date of the certificate you registered are shown, the certificate has been switched. Check the other domains in the same way.

#### 4. Delete the Secret of the Shipped Self-Signed Certificate

After confirming that you can connect with the valid certificate, delete the Secret of the shipped self-signed certificate that remains in the `ingress-nginx` Namespace (the name displayed in `SECRET` by the `kubectl get gateway` command in [Default State](#default-state); for example, `selfsigned-tls-cert`).

```bash
kubectl delete secret <name of the shipped Secret> -n ingress-nginx
```

If the shipped Secret remains, the certificate setting of the entry may return to this Secret in a later update. Do not delete the Secret the entry currently refers to (the name shown in `SECRET` in step 3).

### Renewing the Certificate

To replace the certificate before it expires, replace the contents of the Secret of the same name with the new certificate. You do not need to change the settings of the entry.

```bash
kubectl create secret tls app-tls-cert \
  --cert=tls.crt --key=tls.key \
  -n ingress-nginx \
  --dry-run=client -o yaml | kubectl apply -f -
```

After replacing it, check that the new expiry date is shown with the `openssl` command in step 3.

## Method 3: Let's Encrypt Auto-Issuance

Use cert-manager to automatically issue and renew Let's Encrypt certificates. The domain validation at issuance (ACME HTTP-01 challenge) goes through the Gateways of the entry (NGINX Gateway Fabric).

```text
User → HTTPS (443) → Entry (NGINX Gateway Fabric) (SSL termination / Let's Encrypt certificate) → HTTP → Pod
```

### Prerequisites

- cert-manager is installed (included with AGENTIC STAR)

- DNS for your domains is pointing to the LoadBalancer endpoints

  <!-- -->

  - Azure / GCP: A records pointing to the LoadBalancer IPs
  - AWS: CNAME records pointing to the LoadBalancer ELB DNS names

- Port 80 access is allowed from the internet (for ACME HTTP-01 challenges). The HTTP-01 challenge does not pass in an environment that restricts sources

- The entry is set to redirect requests arriving over HTTP (80) to HTTPS (the default setting). The HTTP-01 challenge does not pass in an environment where the redirect was stopped with [Method 1](#method-1)

### Steps

#### 1. Enable the Gateway API in cert-manager

The shipped cert-manager is not configured to use the Gateway API. Enable it with the following commands. For `--version`, specify the current version of cert-manager confirmed with `helm list` in common step 2 (the part after `cert-manager-` in the `CHART` column, e.g. `v1.20.0`). In environments updated from v3.3.x or earlier, it may still be the version installed at build time.

```bash
helm repo add jetstack https://charts.jetstack.io
helm repo update

HELM_DRIVER=configmap helm upgrade cert-manager jetstack/cert-manager \
  --version <current cert-manager version> \
  --reuse-values \
  --set config.apiVersion="controller.config.cert-manager.io/v1alpha1" \
  --set config.kind="ControllerConfiguration" \
  --set config.enableGatewayAPI=true \
  -n cert-manager

kubectl rollout status deployment/cert-manager -n cert-manager
```

It is enabled if the following command shows `enableGatewayAPI: true`.

```bash
kubectl get configmap cert-manager -n cert-manager -o jsonpath='{.data.config\.yaml}'
```

#### 2. Create a ClusterIssuer

Create `cluster-issuer.yaml` with the following content, change `email` to the address that should receive notifications, and apply it (the name `letsencrypt-prod` is used in the next step). List the 5 Gateways of the entry in `parentRefs`.

```yaml
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: admin@example.com
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
      - http01:
          gatewayHTTPRoute:
            parentRefs:
              - name: front
                namespace: ingress-nginx
                kind: Gateway
                sectionName: http
              - name: admin
                namespace: ingress-nginx
                kind: Gateway
                sectionName: http
              - name: auth
                namespace: ingress-nginx
                kind: Gateway
                sectionName: http
              - name: extapi
                namespace: ingress-nginx
                kind: Gateway
                sectionName: http
              - name: kc
                namespace: ingress-nginx
                kind: Gateway
                sectionName: http
```

```bash
kubectl apply -f cluster-issuer.yaml
kubectl get clusterissuer letsencrypt-prod
```

Check that `READY` is `True`.

#### 3. Create the Certificate

Create `certificate.yaml` with the following content, change `dnsNames` to the domains you configured in common step 4, and apply it. Create the Certificate in the `ingress-nginx` Namespace, the same as the entry.

```yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: app-tls-cert
  namespace: ingress-nginx
spec:
  secretName: app-tls-cert
  issuerRef:
    name: letsencrypt-prod
    kind: ClusterIssuer
  dnsNames:
    - chat.example.com
    - admin.example.com
    - auth.example.com
    - api.example.com
```

```bash
kubectl apply -f certificate.yaml
kubectl get certificate app-tls-cert -n ingress-nginx
```

Wait until `READY` becomes `True` (this may take a few minutes). When it becomes `True`, cert-manager creates the Secret `app-tls-cert` in the `ingress-nginx` Namespace.

> **note**
>
> When the external entry of the authorization server is open, add the authorization server domain (e.g., `kc.example.com`) to `dnsNames`. The DNS record of the domain must point to `ingress-nginx-kc`.

#### 4. Switch the Certificate of the Entry

After `READY` of the Certificate becomes `True`, switch the certificate of the entry to `app-tls-cert` with the same commands as [step 2 of Method 2](#method-2). If you switch before the Secret exists, the entry cannot read the certificate and HTTPS connections fail.

#### 5. Verify and Delete the Shipped Secret

With the same steps as [steps 3 and 4 of Method 2](#method-2), check that the certificate has been switched and delete the Secret of the shipped self-signed certificate. cert-manager renews the certificate automatically, and you do not need to change the settings of the entry.

> **Environments where the HTTP-01 challenge cannot be used**
>
> In an environment that restricts sources, or one combined with Method 1, specify a DNS-01 solver in `solvers` of the ClusterIssuer (see the cert-manager documentation for the settings). Step 3 and later are the same. With DNS-01, enabling the Gateway API in step 1 is not required.

## Migrating the Settings of an Environment Updated from v3.3.x

When you update from v3.3.x to v3.4.0 ("A-10. Entry switch when updating to v3.4.0" in the [Update Guide](https://developers.fd.agenticstar.tm.softbank.jp/en/docs/marketplace/build/guide/update-guide.md)), the entry is switched from ingress-nginx to NGINX Gateway Fabric. The settings placed on the Ingress with the [steps for v3.3.x and earlier](#v33-ingress-nginx) are not used after the switch. If you have set up your own domain, configure the entry again with the steps in this section.

### Settings That Are Carried Over and Settings to Configure Again

| Setting in v3.3.x and earlier                                                                | After the update                                                                                                                                              | What to do                                                                              |
| -------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- |
| DNS records and LoadBalancer addresses                                                       | Unchanged                                                                                                                                                     | Nothing                                                                                 |
| Domain / URL settings of each service (the values in common step 4)                          | Carried over. The entry accepts requests without limiting the host name, so no setting that corresponds to `server-alias` is needed                           | Nothing                                                                                 |
| Source restriction of the LoadBalancers (`loadBalancerSourceRanges`)                         | Carried over                                                                                                                                                  | Nothing                                                                                 |
| Source restriction of the entry and the user IP address settings (settings of ingress-nginx) | Carried over                                                                                                                                                  | Nothing (you can check with the command in [User IP Address Settings](#real-ip))        |
| `ssl-redirect` settings of Method 1 (CDN / Load Balancer)                                    | Not carried over. The entry redirects requests arriving over HTTP (80) to HTTPS                                                                               | ["Stop the HTTP Redirect of the Entry" in Method 1](#method-1)                          |
| Certificate of Method 2 (External Certificate)                                               | As described in "Certificate Used by the Entry" below. It is carried over if the default certificate of ingress-nginx was set to `ingress-nginx/app-tls-cert` | Check, and follow [Method 2](#method-2) if it is not the valid certificate              |
| Certificate of Method 3 (Let's Encrypt)                                                      | Not carried over (the certificates issued per Ingress are not used)                                                                                           | Issue it again with [Method 3](#method-3)                                               |
| External entry of the authorization server                                                   | Switched together with the entry. Its certificate and HTTP handling are determined by the same entry settings as the other 4                                  | Included in the work above (include the authorization server domain in the certificate) |

### Certificate Used by the Entry

At the switch, the certificate Secret used by the entry (NGINX Gateway Fabric) is determined in this order:

1. If the default certificate of ingress-nginx (`--default-ssl-certificate`) points to a Secret in the `ingress-nginx` Namespace and that Secret exists, that Secret
2. Otherwise, `selfsigned-tls-cert` in the `ingress-nginx` Namespace

If the determined Secret does not exist in the `ingress-nginx` Namespace, the entry cannot be switched and `entry` fails on the completion screen of the update (the entry keeps running on ingress-nginx). This may apply to environments where `selfsigned-tls-cert` was deleted by following the steps of Method 2 or Method 3 for v3.3.x and earlier.

### What to Check Before the Update

Check the default certificate of ingress-nginx and the TLS Secrets in the `ingress-nginx` Namespace with the following commands.

```bash
kubectl get deploy ingress-nginx-controller -n ingress-nginx \
  -o jsonpath='{range .spec.template.spec.containers[0].args[*]}{@}{"\n"}{end}' \
  | grep default-ssl-certificate
kubectl get secret -n ingress-nginx --field-selector type=kubernetes.io/tls
```

The first output has the form `--default-ssl-certificate=ingress-nginx/<Secret name>`. If this `<Secret name>` (`selfsigned-tls-cert` when nothing is shown or the Namespace is not `ingress-nginx`) is in the second list, the entry is switched with that Secret.

If it is not in the list, create a Secret with that name before the update.

```bash
kubectl create secret tls <Secret name> \
  --cert=tls.crt --key=tls.key \
  -n ingress-nginx
```

- Environments using Method 2 (External Certificate): create it with the valid certificate (`tls.crt` and `tls.key`). The entry uses the valid certificate right after the switch

- Environments using Method 3 (Let's Encrypt): create it with a temporary self-signed certificate, and issue the certificate again with [Method 3](#method-3) right after the update. While the temporary certificate is in use, browsers show a certificate warning

  ```bash
  openssl req -x509 -nodes -newkey rsa:2048 -days 30 \
    -keyout tls.key -out tls.crt \
    -subj "/CN=chat.example.com" \
    -addext "subjectAltName=DNS:chat.example.com,DNS:admin.example.com,DNS:auth.example.com,DNS:api.example.com"
  ```

### What to Do Right After the Update

Run the update when usage is low, and when it completes, do the following in order.

**1. Check that the entry has been switched**

With the command in [Target Versions and Checking the Entry](#entry-version), check that every row shows a name in `ENTRY`. If `entry` failed on the completion screen, see "If `entry` Failed in the Update" below.

**2. Check the state and the certificate of the entry**

With the `kubectl get gateway` command in [Default State](#default-state), check that `PROGRAMMED` and `CERT` are `True` for the 5 Gateways, and check the name in `SECRET`.

**3. Configure the entry according to the configuration used in v3.3.x and earlier**

- Environments using Method 1: [stop the HTTP redirect of the entry](#method-1). Check that you can connect through the CDN / load balancer
- Environments using Method 2: if `SECRET` is not the Secret of the valid certificate, do steps 1–3 of [Method 2](#method-2). If `app-tls-cert` already exists in the `ingress-nginx` Namespace, step 1 is not required
- Environments using Method 3: do steps 1–5 of [Method 3](#method-3). The `kubectl apply` in step 2 replaces the ClusterIssuer of v3.3.x and earlier (configured to use the Ingress) with the one configured to use the Gateways

**4. Check that you can connect with each domain**

Open each domain in a browser and check that no certificate warning is shown and that you can sign in.

**5. Clean up the settings that are no longer used**

- Environments using Method 2 or Method 3: after confirming that you can connect with the valid certificate, delete the shipped Secret in the `ingress-nginx` Namespace (if it remains) and the temporary Secret created before the update, as in [step 4 of Method 2](#method-2). Do not delete the Secret the entry refers to (`SECRET` in step 2)

- Environments using Method 3: turn off the cert-manager integration of each service that was enabled with the steps for v3.3.x and earlier. Otherwise, the renewal of the per-Ingress certificates that are no longer used keeps failing

  ```bash
  # Main application
  HELM_DRIVER=configmap helm upgrade librechat agenticai-prod/agentic-star-front \
    --version 3.4.0 --reuse-values \
    --set global.ingress.tls.useCertManager=false \
    -n agentic-star-front

  # Admin console
  HELM_DRIVER=configmap helm upgrade agenticai-admin agenticai-prod/agenticai-admin \
    --version 3.4.0 --reuse-values \
    --set global.ingress.tls.useCertManager=false \
    -n agentic-star-admin

  # Authentication service
  HELM_DRIVER=configmap helm upgrade librechat-auth agenticai-prod/agentic-star-auth \
    --version 3.4.0 --reuse-values \
    --set global.ingress.tls.useCertManager=false \
    -n librechat-auth

  # External API
  HELM_DRIVER=configmap helm upgrade agenticai-extapi agenticai-prod/agenticai-extapi \
    --version 3.4.0 --reuse-values \
    --set global.ingress.tls.useCertManager=false \
    -n agenticai-extapi-service
  ```

  Then run `kubectl get certificate -A` and delete any Certificate that remains in the Namespace of each service (do not delete `app-tls-cert` in the `ingress-nginx` Namespace).

### If `entry` Failed in the Update

If the error message on the completion screen mentions the HTTPS listener and the certificate Secret, the certificate Secret used by the entry does not exist in the `ingress-nginx` Namespace. The entry is still running on ingress-nginx.

1. With the `kubectl get gateway` command in [Default State](#default-state), check the name shown in `SECRET`
2. With the `kubectl create secret tls` command in "What to Check Before the Update", create a Secret with that name in the `ingress-nginx` Namespace
3. After checking that `CERT` has become `True`, run the update to the same version again from the version management screen (updated services are skipped and the switch is retried)
4. Do "What to Do Right After the Update"

## External Entry of the Authorization Server (Sign-in from External Tools)

The authorization server has an external entry that lets users sign in with their own account from external tools (AI clients such as Claude Code or Codex, and OIDC-capable applications) and use the User API and MCP. **It is closed by default.** An administrator opens and closes it with a command after installation. For the preparation on the external tool side (creating clients, approval workflow), see the [API Setup Guide](https://developers.fd.agenticstar.tm.softbank.jp/en/docs/marketplace/api/guide/setup-guide.md). Attach to each external tool client the sign-in flow that, as on the Main Screen, lets through only users approved by an administrator whose email address has been verified ([Attach the sign-in flow](https://developers.fd.agenticstar.tm.softbank.jp/en/docs/marketplace/api/guide/setup-guide.md#login-flow)).

The command is used in the same way in environments of v3.3.x and earlier. For the differences in v3.3.x and earlier, see [External Entry of the Authorization Server (v3.3.x and Earlier)](#v33-kc).

### What Is Exposed

When opened, only the **12 paths** of the authorization server needed for sign-in are exposed (`<realm>` is `keycloak.realm_name` in the installer's `config.yaml`).

| Match  | Path                                                                                                                      |
| ------ | ------------------------------------------------------------------------------------------------------------------------- |
| Exact  | `/realms/<realm>/protocol/openid-connect/auth`, `token`, `certs`, `userinfo`, `logout`, `logout/logout-confirm`, `revoke` |
| Exact  | `/realms/<realm>/.well-known/openid-configuration`, `/.well-known/oauth-authorization-server/realms/<realm>`              |
| Prefix | `/realms/<realm>/login-actions/`, `/realms/<realm>/broker/`, `/resources/`                                                |

The console and admin API of the authorization server are not exposed. The sign-in flow of the main application does not change either.

### Prerequisites

- The command `kc_direct_login.sh` is included in the installer distribution (zip). If it is no longer on the machine you installed from, download the zip of the same version from the distribution location
- In an environment updated to v3.4.0, use the command from the distribution of the version after the update. The command from the distribution of v3.3.x and earlier does not support the entry (NGINX Gateway Fabric)
- The machine running the command needs `kubectl` (connected to the target cluster), `helm` 3.14 or later, `yq`, `openssl`, and `curl`. The command does not read `config.yaml`; it reads what it needs from the connected cluster
- The `keycloak-extensions`, `init-platform`, and `agenticai-extapi` charts must be at least the version in the distribution. If any is older the command stops, so upgrade from the GUI first

### Opening

```bash
bash kc_direct_login.sh open --address <domain|auto>
```

| `--address` value         | Environments                                  | Behavior                                                                                                                                   |
| ------------------------- | --------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
| `kc.example.com` (domain) | All                                           | The authorization server announces this domain as its external address. Write it in lowercase without a scheme (`https://`), path, or port |
| `auto`                    | Only environments without a configured domain | Uses the IP assigned to `ingress-nginx-kc` (the NLB DNS name on AWS) as the address                                                        |

Before proceeding, the command shows that the authorization server will be restarted and, in environments using the shipped self-signed certificate, that the certificate of the existing 4 endpoints will be regenerated, and asks for confirmation (`--yes` skips the prompt). It then proceeds in this order:

1. In environments without a configured domain, creates the fifth LoadBalancer `ingress-nginx-kc` and waits for its address (if the authentication service's LoadBalancer has a source IP restriction `loadBalancerSourceRanges`, the same restriction is copied)
2. In environments using the shipped self-signed certificate, regenerates the certificate with the authorization server address added and replaces the Secret in 5 Namespaces, including the `ingress-nginx` Namespace used by the entry
3. Passes the external address settings to the authorization server and restarts it
4. Creates the route to the authorization server (the HTTPRoute `keycloak-direct-login` in the `librechat-auth` Namespace) on the Gateway `kc` of the entry
5. Verifies that the issuer in the discovery document seen from outside (`/.well-known/openid-configuration`) is the external address
6. Passes the issuer of tokens from the external entry (`https://<address>/realms/<realm>`) to the external API

When finished, it prints the external address, the issuer, and the address of the external endpoint. If you specified a domain, point its DNS record at the address of `ingress-nginx-kc` ([DNS record configuration](#1-obtain-a-domain-and-configure-dns-records)).

The certificate and HTTP handling of the external entry are determined by the settings of the entry (the Gateway `kc`), the same as for the other 4 entries. The steps of [Methods 1–3](#choosing-an-ssl-configuration-method) configure the Gateway `kc` together. When you use a valid certificate, include the authorization server domain in the certificate.

> **Environments installed with a domain from the start**
>
> In environments without per-service LoadBalancers, the command does not create `ingress-nginx-kc`. Add the Gateway `kc` of the entry (the Service `kc-agentic-star-edge` in the `ingress-nginx` Namespace) as the destination of the authorization server domain to the same path used by the existing 4 endpoints (the CDN / load balancer in front), and prepare a certificate ([Methods 1–3](#choosing-an-ssl-configuration-method)).

### Closing

```bash
bash kc_direct_login.sh close
```

Removes the issuer from the external API, removes the external address settings from the authorization server and restarts it, and deletes `ingress-nginx-kc` and the route to the authorization server on the entry (the HTTPRoute `keycloak-direct-login`). Tokens obtained from external tools stop working against the external API as soon as the entry is closed. The Gateway `kc` of the entry and its certificate and HTTP handling settings are kept and reused the next time you open it.

### Checking the Status

```bash
bash kc_direct_login.sh status
```

Shows whether the entry is open (open / closed / in progress), the realm, the external address, the issuer in the external API, the address of `ingress-nginx-kc`, whether the route to the authorization server (HTTPRoute) exists on the entry, the kind of certificate (shipped self-signed or your own), and the chart versions. "In progress" means an open or close operation stopped partway; run `open` or `close` again.

### Changing the Address

Run `open` again with the new domain. The external address and the issuer in the external API are replaced. Because the issuer changes, **users who sign in from external tools must sign in again.** In environments using the shipped self-signed certificate, if the IP of `ingress-nginx-kc` (the NLB DNS name on AWS) has changed, the certificate is regenerated after confirmation.

### Notes for Environments Without a Configured Domain

- The self-signed certificate regenerated when opening is the single certificate shared with the existing 4 endpoints. After regeneration, users' browsers show the certificate warning again
- External tools cannot connect with the product's self-signed certificate as is. For steps to connect only while trying (Claude Code and Codex), see [AI Client Connection Guide (MCP) — Trying It in an Environment without a Domain](https://developers.fd.agenticstar.tm.softbank.jp/en/docs/marketplace/api/guide/ai-client-guide.md#self-signed). To use sign-in from external tools in production, configure a proper domain and certificate

> **The authorization server restarts every time you open or close**
>
> The external address settings are startup settings of the authorization server, so it restarts every time you open, close, or change the address. In the shipped configuration (a single authorization server instance), sign-in to the main application and token refresh are also unavailable during that time. Do this when few users are active.

> **Restricting source addresses**
>
> The external entry itself has no source IP or request rate restriction (per-user lockout after failed sign-ins is handled by the authorization server). To restrict by IP, place a WAF in front or set `loadBalancerSourceRanges` on `ingress-nginx-kc` ([Method 1](#method-1)).

## Troubleshooting

These are the checks for environments of v3.4.0 or later (where the entry is NGINX Gateway Fabric). For environments of v3.3.x and earlier, see [Troubleshooting (v3.3.x and Earlier)](#v33-troubleshooting).

### Cannot Log In with the Default Self-Signed Certificate

```bash
# Check Gateway status (whether PROGRAMMED is True)
kubectl get gateway -n ingress-nginx

# Check listener and certificate status (<name> is front, admin, auth, extapi, or kc)
kubectl describe gateway <name> -n ingress-nginx

# Verify TLS Secret exists (<Secret name> is SECRET in the kubectl get gateway command in Default State)
kubectl get secret <Secret name> -n ingress-nginx

# Check the HTTPRoute status of each service
kubectl get httproute -A

# Check the entry logs
kubectl logs -n ingress-nginx deployment/<name>-agentic-star-edge
kubectl logs -n ingress-nginx deployment/nginx-gateway
```

**Common causes:**

- Browser certificate exception has not been accepted
- TLS Secret does not exist in the `ingress-nginx` Namespace
- The Pods of the entry are restarting

### Check Currently Applied Entry Settings

```bash
HELM_DRIVER=configmap helm get values agentic-star-edge -n ingress-nginx
```

The output may contain secret values such as connection passwords. Be careful when sharing your screen or saving it to a file.

### Cannot Connect over HTTPS After Switching the Certificate

```bash
kubectl describe gateway <name> -n ingress-nginx
```

If `ResolvedRefs` of the `https` listener is `False`, the listener cannot read the Secret it refers to.

**Common causes:**

- The Secret is not in the `ingress-nginx` Namespace (it was created in the Namespace of each service)
- The Secret name differs from the entry setting (`gateways[].listeners.https.certificateRef.name`)
- The Secret type is not `kubernetes.io/tls`, or the certificate and private key do not match

### Certificate Not Issued (Let's Encrypt)

```bash
# Check Certificate resource status
kubectl describe certificate app-tls-cert -n ingress-nginx

# Check cert-manager logs
kubectl logs -n cert-manager deployment/cert-manager

# Check ACME verification status with Challenge resources
kubectl get challenge -n ingress-nginx

# Check whether the HTTPRoute for the challenge is accepted by the Gateways
kubectl get httproute -n ingress-nginx
```

**Common causes:**

- DNS is not pointing to the LoadBalancer endpoint (A record to the IP for Azure/GCP, CNAME to the ELB DNS name for AWS)
- Port 80 access is blocked
- The Gateway API is not enabled in cert-manager ([step 1 of Method 3](#method-3))
- The HTTP redirect of the entry is stopped (combined with [Method 1](#method-1))
- ClusterIssuer email address is invalid

### Not Redirecting to HTTPS

```bash
kubectl get httproute -n ingress-nginx
```

If there is no redirect HTTPRoute (such as `front-https-redirect`), `gateways[].httpsRedirect` of the entry is `false`. Set it back to `true` with the steps in [Changing the Settings of the Entry](#edge-settings).

## Steps for v3.3.x and Earlier (ingress-nginx)

This section contains the steps for environments where the entry is ingress-nginx (v3.3.x and earlier). In environments of v3.4.0 or later, the settings in this section (`global.ingress.tls.*` and the Ingress annotations) are not used. For environments of v3.4.0 or later, use the steps in the main text of this page.

The default state (endpoint configuration and limitations), choosing an SSL configuration method, and common steps 1–3 (configuring DNS records, checking the chart version, and adding the Helm repository) are the same as in v3.4.0 or later. See [Default State](#default-state), [Choosing an SSL Configuration Method](#choosing-an-ssl-configuration-method), and [Common Upgrade Steps](#common-upgrade-steps) in the main text. The following points differ in v3.3.x and earlier.

- The entry is ingress-nginx (Ingress Controller), and there are no Gateways. SSL is terminated at the Ingress in Method 2 and Method 3
- The Secret of the self-signed certificate, `selfsigned-tls-cert`, is in 5 Namespaces (`agentic-star-front`, `agentic-star-admin`, `librechat-auth`, `agenticai-extapi-service`, and `ingress-nginx`), and the Ingress of each service refers to it
- Examples of chart versions are `agentic-star-front-3.3.0`, `agenticai-admin-3.3.0`, etc.

### Update Domain / URL Settings (v3.3.x and Earlier)

Change the IP-based URLs in each service's ConfigMap to domain-based URLs using `helm upgrade --reuse-values --set`. The `--reuse-values` flag preserves all existing settings, and only the values specified with `--set` are overwritten.

> **caution**
>
> Specify the same chart version that is currently deployed, as confirmed in common step 2.

**Main Application (librechat):**

```bash
HELM_DRIVER=configmap helm upgrade librechat agenticai-prod/agentic-star-front \
  --version 3.3.0 \
  --reuse-values \
  --set appConfig.domainClient="https://chat.example.com" \
  --set appConfig.domainServer="https://chat.example.com" \
  --set appConfig.authBaseUrl="https://auth.example.com" \
  --set ingress.main.annotations."nginx\.ingress\.kubernetes\.io/server-alias"="chat.example.com" \
  --set ingress.monitoring.annotations."nginx\.ingress\.kubernetes\.io/server-alias"="chat.example.com" \
  --set ingress.vnc.annotations."nginx\.ingress\.kubernetes\.io/server-alias"="chat.example.com" \
  -n agentic-star-front
```

**Admin Console (agenticai-admin):**

```bash
HELM_DRIVER=configmap helm upgrade agenticai-admin agenticai-prod/agenticai-admin \
  --version 3.3.0 \
  --reuse-values \
  --set env.AFD_CHATBOARD_ADMIN_DOMAIN="admin.example.com" \
  --set env.AFD_CHATBOARD_FRONT_AUTH_DOMAIN="auth.example.com\,chat.example.com" \
  --set env.ALLOWED_ORIGINS="https://auth.example.com\,https://admin.example.com" \
  --set env.CHATBOARD_FRONT_DOMAIN="chat.example.com" \
  --set env.EXTAPI_PUBLIC_URL="https://api.example.com" \
  --set env.MP_OAUTH_TOKEN_URL="https://auth.example.com/api/v1/oauth/token" \
  --set env.AUTH_PUBLIC_URL="https://auth.example.com" \
  --set env.ADMIN_PUBLIC_URL="https://admin.example.com" \
  --set ingress.client.annotations."nginx\.ingress\.kubernetes\.io/server-alias"="admin.example.com" \
  -n agentic-star-admin
```

**Authentication Service (librechat-auth):**

```bash
HELM_DRIVER=configmap helm upgrade librechat-auth agenticai-prod/agentic-star-auth \
  --version 3.3.0 \
  --reuse-values \
  --set env.ADMIN_CONSOLE_URL="https://admin.example.com" \
  --set env.ALLOWED_ORIGINS="https://chat.example.com\,https://auth.example.com" \
  --set env.ALLOWED_REDIRECT_URIS="https://chat.example.com/oauth/librechat/callback\,https://auth.example.com/*\,https://admin.example.com/*" \
  --set env.APP_URL="https://auth.example.com" \
  --set env.LIBRECHAT_URL="https://chat.example.com" \
  --set env.NEXT_PUBLIC_APP_URL="https://auth.example.com" \
  --set env.WEBAUTHN_ORIGIN="https://auth.example.com" \
  --set env.WEBAUTHN_RP_ID="auth.example.com" \
  --set ingress.annotations."nginx\.ingress\.kubernetes\.io/server-alias"="auth.example.com" \
  -n librechat-auth
```

**Authentication platform (keycloak-extensions):**

Change `WEBAUTHN_RP_ID` and `WEBAUTHN_ORIGIN`, which are used to verify passkeys, to the new domain of the authentication service. These two are inside `keycloakx.extraEnv` (a multi-line string), so export the current values, edit them, and apply them.

```bash
HELM_DRIVER=configmap helm get values keycloak-extensions -n librechat-auth -o yaml > keycloak-extensions-values.yaml
```

In `keycloakx.extraEnv` of `keycloak-extensions-values.yaml`, change the value of `WEBAUTHN_RP_ID` to `auth.example.com` and the value of `WEBAUTHN_ORIGIN` to `https://auth.example.com`, then run the following.

```bash
HELM_DRIVER=configmap helm upgrade keycloak-extensions agenticai-prod/keycloak-extensions \
  --version 3.3.0 \
  --reuse-values \
  -f keycloak-extensions-values.yaml \
  -n librechat-auth
```

The exported file contains secret values such as connection passwords. Delete it after the work.

**External API (agenticai-extapi):**

```bash
HELM_DRIVER=configmap helm upgrade agenticai-extapi agenticai-prod/agenticai-extapi \
  --version 3.3.0 \
  --reuse-values \
  --set config.APIGW_PUBLIC_URL="https://api.example.com" \
  --set config.A2A_PUBLIC_URL="https://api.example.com" \
  --set config.MP_OAUTH_TOKEN_URL="https://auth.example.com/api/v1/oauth/token" \
  --set ingress.annotations."nginx\.ingress\.kubernetes\.io/server-alias"="api.example.com" \
  -n agenticai-extapi-service
```

**External entry of the authorization server (only when opened):**

In an environment where the external entry of the authorization server is open, run the open/close command again with the new domain instead of `helm upgrade`. It replaces three things at once — the external address the authorization server announces, the alias of the entry, and the token issuer passed to the external API — and verifies that the issuer has switched. For the values to pass when running `helm upgrade` by hand, see the [Helm Values Reference](https://developers.fd.agenticstar.tm.softbank.jp/en/docs/marketplace/build/reference/helm-values-ref.md#kc-direct-login-values).

```bash
bash kc_direct_login.sh open --address kc.example.com
```

Because the issuer changes, users who sign in from external tools must sign in again.

> **note**
>
> - The `server-alias` of each service is the host name the Ingress accepts. Update it to match the new domain
> - When the external entry of the authorization server is open, the open/close command above replaces the token issuer passed to the external API
> - When specifying values containing commas with `--set`, escape them with `\,`

### Method 1: CDN / Load Balancer (v3.3.x and Earlier)

Perform SSL termination at the cloud service and disable TLS on the Ingress side.

```text
User → HTTPS (443) → CDN / Load Balancer (SSL termination) → HTTP (80) → Ingress Controller → Pod
```

#### Deploy with Helm

Run the following commands for each service.

```bash
# Main application
HELM_DRIVER=configmap helm upgrade librechat agenticai-prod/agentic-star-front \
  --version 3.3.0 --reuse-values \
  --set global.ingress.tls.enabled=false \
  --set ingress.main.annotations."nginx\.ingress\.kubernetes\.io/ssl-redirect"="false" \
  --set ingress.main.annotations."nginx\.ingress\.kubernetes\.io/force-ssl-redirect"="false" \
  --set ingress.monitoring.annotations."nginx\.ingress\.kubernetes\.io/ssl-redirect"="false" \
  --set ingress.monitoring.annotations."nginx\.ingress\.kubernetes\.io/force-ssl-redirect"="false" \
  --set ingress.vnc.annotations."nginx\.ingress\.kubernetes\.io/ssl-redirect"="false" \
  --set ingress.vnc.annotations."nginx\.ingress\.kubernetes\.io/force-ssl-redirect"="false" \
  -n agentic-star-front

# Admin console
HELM_DRIVER=configmap helm upgrade agenticai-admin agenticai-prod/agenticai-admin \
  --version 3.3.0 --reuse-values \
  --set global.ingress.tls.enabled=false \
  --set ingress.client.annotations."nginx\.ingress\.kubernetes\.io/ssl-redirect"="false" \
  --set ingress.client.annotations."nginx\.ingress\.kubernetes\.io/force-ssl-redirect"="false" \
  -n agentic-star-admin

# Authentication service
HELM_DRIVER=configmap helm upgrade librechat-auth agenticai-prod/agentic-star-auth \
  --version 3.3.0 --reuse-values \
  --set global.ingress.tls.enabled=false \
  --set ingress.annotations."nginx\.ingress\.kubernetes\.io/ssl-redirect"="false" \
  --set ingress.annotations."nginx\.ingress\.kubernetes\.io/force-ssl-redirect"="false" \
  -n librechat-auth

# External API
HELM_DRIVER=configmap helm upgrade agenticai-extapi agenticai-prod/agenticai-extapi \
  --version 3.3.0 --reuse-values \
  --set global.ingress.tls.enabled=false \
  --set ingress.annotations."nginx\.ingress\.kubernetes\.io/ssl-redirect"="false" \
  --set ingress.annotations."nginx\.ingress\.kubernetes\.io/force-ssl-redirect"="false" \
  -n agenticai-extapi-service

# External entry of the authorization server (only when opened: remove TLS from the Ingress and stop redirecting to https)
HELM_DRIVER=configmap helm upgrade keycloak-extensions agenticai-prod/keycloak-extensions \
  --version 3.3.0 --reuse-values \
  --set-json 'keycloakx.ingress.tls=[]' \
  --set keycloakx.ingress.annotations."nginx\.ingress\.kubernetes\.io/ssl-redirect"="false" \
  --set keycloakx.ingress.annotations."nginx\.ingress\.kubernetes\.io/force-ssl-redirect"="false" \
  -n librechat-auth
```

> **note**
>
> Unlike the other 4 services, the Ingress of the authorization server (`keycloak-extensions`) does not read `global.ingress.tls.*`. Pass TLS settings and annotations directly under `keycloakx.ingress.*`. The open/close command sets these values only the first time the entry is opened and keeps the values you change afterwards.

#### Cloud-Specific Configuration

The cloud-specific configuration (Azure Front Door, AWS CloudFront + NLB, and GCP Cloud Load Balancing) is the same as [Cloud-Specific Configuration](#cloud-specific) in the main text. Read the step "Update the settings of the entry" as "Deploy with Helm" above.

### Method 2: External Certificate (v3.3.x and Earlier)

Use an externally issued certificate with Ingress.

```text
User → HTTPS (443) → Ingress Controller (SSL termination / valid certificate) → HTTP → Pod
```

#### 1. Register the Certificate as a Kubernetes Secret

Delete the existing self-signed certificate Secrets and replace them with valid certificate Secrets.

```bash
# Delete existing self-signed certificate Secrets (from all 5 Namespaces)
kubectl delete secret selfsigned-tls-cert -n agentic-star-front
kubectl delete secret selfsigned-tls-cert -n agentic-star-admin
kubectl delete secret selfsigned-tls-cert -n librechat-auth
kubectl delete secret selfsigned-tls-cert -n agenticai-extapi-service
kubectl delete secret selfsigned-tls-cert -n ingress-nginx

# Create valid certificate Secrets (in all 5 Namespaces)
for ns in agentic-star-front agentic-star-admin librechat-auth agenticai-extapi-service ingress-nginx; do
  kubectl create secret tls app-tls-cert \
    --cert=tls.crt --key=tls.key \
    -n $ns
done
```

> **note**
>
> The Secret in the `ingress-nginx` Namespace is used by the Ingress Controller's `default-ssl-certificate`. Update `controller.extraArgs.default-ssl-certificate` in `ingress-nginx-values.yaml` to `"ingress-nginx/app-tls-cert"` and redeploy the Ingress Controller.
>
> When the external entry of the authorization server is open, `app-tls-cert` in the `librechat-auth` Namespace is shared by the authentication service and the entry of the authorization server. Include the authorization server domain (e.g., `kc.example.com`) in the certificate's SANs.

#### 2. Deploy with Helm

Switch the TLS Secret name to the valid certificate for each service.

```bash
# Main application
HELM_DRIVER=configmap helm upgrade librechat agenticai-prod/agentic-star-front \
  --version 3.3.0 --reuse-values \
  --set global.ingress.tls.enabled=true \
  --set global.ingress.tls.secretName="app-tls-cert" \
  --set global.ingress.tls.useCertManager=false \
  -n agentic-star-front

# Admin console
HELM_DRIVER=configmap helm upgrade agenticai-admin agenticai-prod/agenticai-admin \
  --version 3.3.0 --reuse-values \
  --set global.ingress.tls.enabled=true \
  --set global.ingress.tls.secretName="app-tls-cert" \
  --set global.ingress.tls.useCertManager=false \
  -n agentic-star-admin

# Authentication service
HELM_DRIVER=configmap helm upgrade librechat-auth agenticai-prod/agentic-star-auth \
  --version 3.3.0 --reuse-values \
  --set global.ingress.tls.enabled=true \
  --set global.ingress.tls.secretName="app-tls-cert" \
  --set global.ingress.tls.useCertManager=false \
  -n librechat-auth

# External API
HELM_DRIVER=configmap helm upgrade agenticai-extapi agenticai-prod/agenticai-extapi \
  --version 3.3.0 --reuse-values \
  --set global.ingress.tls.enabled=true \
  --set global.ingress.tls.secretName="app-tls-cert" \
  --set global.ingress.tls.useCertManager=false \
  -n agenticai-extapi-service

# External entry of the authorization server (only when opened: share the Secret with the authentication service)
HELM_DRIVER=configmap helm upgrade keycloak-extensions agenticai-prod/keycloak-extensions \
  --version 3.3.0 --reuse-values \
  --set keycloakx.ingress.tls[0].hosts[0]="kc.agentic.internal" \
  --set keycloakx.ingress.tls[0].secretName="app-tls-cert" \
  -n librechat-auth
```

### Method 3: Let's Encrypt Auto-Issuance (v3.3.x and Earlier)

Use cert-manager to automatically issue and renew Let's Encrypt certificates.

```text
User → HTTPS (443) → Ingress Controller (SSL termination / Let's Encrypt certificate) → HTTP → Pod
```

#### Prerequisites

- cert-manager is installed (included with AGENTIC STAR)

- DNS for your domains is pointing to the LoadBalancer endpoints

  <!-- -->

  - Azure / GCP: A records pointing to the Ingress IPs
  - AWS: CNAME records pointing to the Ingress ELB DNS names

- Port 80 access is allowed (for ACME HTTP-01 challenges)

#### 1. Create a ClusterIssuer

Create `cluster-issuer.yaml` with the following content, change `email` to the address that should receive notifications, and apply it (the name `letsencrypt-prod` is used in the next step).

```yaml
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: admin@example.com
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
      - http01:
          ingress:
            ingressClassName: nginx
```

```bash
kubectl apply -f cluster-issuer.yaml
```

#### 2. Deploy with Helm

Enable cert-manager integration for each service.

```bash
# Main application
HELM_DRIVER=configmap helm upgrade librechat agenticai-prod/agentic-star-front \
  --version 3.3.0 --reuse-values \
  --set global.ingress.tls.enabled=true \
  --set global.ingress.tls.secretName="app-tls-cert" \
  --set global.ingress.tls.useCertManager=true \
  --set global.ingress.tls.clusterIssuer="letsencrypt-prod" \
  --set global.domain="chat.example.com" \
  -n agentic-star-front

# Admin console
HELM_DRIVER=configmap helm upgrade agenticai-admin agenticai-prod/agenticai-admin \
  --version 3.3.0 --reuse-values \
  --set global.ingress.tls.enabled=true \
  --set global.ingress.tls.secretName="app-tls-cert" \
  --set global.ingress.tls.useCertManager=true \
  --set global.ingress.tls.clusterIssuer="letsencrypt-prod" \
  --set ingress.client.host="admin.example.com" \
  -n agentic-star-admin

# Authentication service
HELM_DRIVER=configmap helm upgrade librechat-auth agenticai-prod/agentic-star-auth \
  --version 3.3.0 --reuse-values \
  --set global.ingress.tls.enabled=true \
  --set global.ingress.tls.secretName="app-tls-cert" \
  --set global.ingress.tls.useCertManager=true \
  --set global.ingress.tls.clusterIssuer="letsencrypt-prod" \
  --set ingress.host="auth.example.com" \
  -n librechat-auth

# External API
HELM_DRIVER=configmap helm upgrade agenticai-extapi agenticai-prod/agenticai-extapi \
  --version 3.3.0 --reuse-values \
  --set global.ingress.tls.enabled=true \
  --set global.ingress.tls.secretName="app-tls-cert" \
  --set global.ingress.tls.useCertManager=true \
  --set global.ingress.tls.clusterIssuer="letsencrypt-prod" \
  --set-json 'ingress.hosts=[{"host":"api.example.com","paths":[{"path":"/","pathType":"Prefix"}]}]' \
  -n agenticai-extapi-service
```

> **note**
>
> Let's Encrypt requires your own domain as the certificate name (host name). The commands above set the host name of each service's Ingress to your own domain (for example `chat.example.com`, `admin.example.com`, `auth.example.com`, `api.example.com`). Use the same values as the domains in "Update Domain / URL Settings". After the host names change, the screens cannot be opened with the LoadBalancer IP addresses.

> **note**
>
> The external entry of the authorization server is not covered by Method 3. The external entry keeps its current certificate.

> **note**
>
> cert-manager will automatically create a new certificate using the Secret name (`app-tls-cert`). Manually delete the original self-signed certificate Secrets (`selfsigned-tls-cert`) from each Namespace. Also update the Ingress Controller's `default-ssl-certificate` to `"ingress-nginx/app-tls-cert"`.

#### 3. Verify Certificate Issuance

```bash
kubectl get certificate -n <namespace>
kubectl describe certificate <cert-name> -n <namespace>
```

### External Entry of the Authorization Server (v3.3.x and Earlier)

The commands for opening, closing, checking the status, and changing the address are the same as in [External Entry of the Authorization Server (Sign-in from External Tools)](#kc-direct-login) in the main text. The following points differ in v3.3.x and earlier.

- The external entry is an Ingress of ingress-nginx (`keycloak-extensions-keycloakx` in the `librechat-auth` Namespace); Gateways and HTTPRoutes are not used. When opening, the external address and the entry settings are passed to the authorization server and it is restarted (there is no step 4 of "Opening" in the main text)
- The certificate (TLS Secret name) and the `https` redirect are determined by the values of the Ingress (`keycloakx.ingress.tls` and `keycloakx.ingress.annotations`). The open/close command sets these values only the first time the entry is opened and keeps the values you change afterwards. These values are kept when you close the entry and reused the next time you open it (the commands for the external entry of the authorization server in Method 1 and Method 2)
- When you change the address, the external address, the alias of the entry (`server-alias`), and the issuer in the external API are replaced
- When the shipped self-signed certificate is regenerated, `selfsigned-tls-cert` in all 5 Namespaces is replaced
- In environments installed with a domain from the start, add the authorization server domain to the same path used by the existing 4 endpoints (the CDN / load balancer in front, or the Ingress Controller) and prepare a certificate (the authorization server lines in Methods 1–3)

### Troubleshooting (v3.3.x and Earlier)

#### Cannot Log In with the Default Self-Signed Certificate (v3.3.x and Earlier)

```bash
# Check Ingress status
kubectl describe ingress <name> -n <namespace>

# Verify TLS Secret exists
kubectl get secret selfsigned-tls-cert -n <namespace>

# Check Ingress Controller logs
kubectl logs -n ingress-nginx deployment/ingress-nginx-controller
```

**Common causes:**

- Browser certificate exception has not been accepted
- TLS Secret does not exist in the correct Namespace
- Ingress Controller is restarting

#### Check Currently Applied TLS Settings (v3.3.x and Earlier)

```bash
HELM_DRIVER=configmap helm get values <release-name> -n <namespace>
```

#### Certificate Not Issued (Let's Encrypt; v3.3.x and Earlier)

```bash
# Check Certificate resource status
kubectl describe certificate <name> -n <namespace>

# Check cert-manager logs
kubectl logs -n cert-manager deployment/cert-manager

# Check ACME verification status with Challenge resources
kubectl get challenge -n <namespace>
```

**Common causes:**

- DNS is not pointing to the Ingress endpoint (A record to the IP for Azure/GCP, CNAME to the ELB DNS name for AWS)
- Port 80 access is blocked
- ClusterIssuer email address is invalid

#### Not Redirecting to HTTPS (v3.3.x and Earlier)

```bash
HELM_DRIVER=configmap helm get values <release-name> -n <namespace>
```

Check that `nginx.ingress.kubernetes.io/ssl-redirect` is set to `"true"`.
