Skip to main content

[BETA] JWT-based Auth

Use JWT's to auth admins / projects into the proxy.

info

This is a new feature, and subject to changes based on feedback.

Usage​

Step 1. Setup Proxy​

  • JWT_PUBLIC_KEY_URL: This is the public keys endpoint of your OpenID provider. Typically it's {openid-provider-base-url}/.well-known/openid-configuration/jwks. For Keycloak it's {keycloak_base_url}/realms/{your-realm}/protocol/openid-connect/certs.
export JWT_PUBLIC_KEY_URL="" # "https://demo.duendesoftware.com/.well-known/openid-configuration/jwks"
  • enable_jwt_auth in your config. This will tell the proxy to check if a token is a jwt token.
general_settings:
  master_key: sk-1234
  enable_jwt_auth: True

model_list:
- model_name: azure-gpt-3.5 
  litellm_params:
      model: azure/<your-deployment-name>
      api_base: os.environ/AZURE_API_BASE
      api_key: os.environ/AZURE_API_KEY
      api_version: "2023-07-01-preview"

Step 2. Create JWT with scopes​

Create a client scope called litellm_proxy_admin in your OpenID provider (e.g. Keycloak).

Grant your user, litellm_proxy_admin scope when generating a JWT. 

curl --location ' 'https://demo.duendesoftware.com/connect/token'' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'client_id={CLIENT_ID}' \
--data-urlencode 'client_secret={CLIENT_SECRET}' \
--data-urlencode 'username=test-{USERNAME}' \
--data-urlencode 'password={USER_PASSWORD}' \
--data-urlencode 'grant_type=password' \
--data-urlencode 'scope=litellm_proxy_admin' # 👈 grant this scope

Step 3. Test your JWT​

curl --location '{proxy_base_url}/key/generate' \
--header 'Authorization: Bearer eyJhbGciOiJSUzI1NiI...' \
--header 'Content-Type: application/json' \
--data '{}'

Advanced - Allowed Routes​

Configure which routes a non-admin JWT can access via the config.

By default, a non-admin JWT can call openai + any /info endpoints. 

general_settings:
  master_key: sk-1234
  enable_jwt_auth: True
  allowed_routes: ["/chat/completions", "/embeddings"]

Advanced - Set Accepted JWT Scope Names​

Change the string in JWT 'scopes', that litellm evaluates to see if a user has admin access.

general_settings:
  master_key: sk-1234
  enable_jwt_auth: True
  litellm_proxy_roles:
    proxy_admin: "litellm-proxy-admin"

Allowed LiteLLM scopes​

class LiteLLMProxyRoles(LiteLLMBase):
    proxy_admin: str = "litellm_proxy_admin"
    proxy_user: str = "litellm_user" # 👈 Not implemented yet, for JWT-Auth.

JWT Scopes​

Here's what scopes on JWT-Auth tokens look like

Can be a list

scope: ["litellm-proxy-admin",...]

Can be a space-separated string

scope: "litellm-proxy-admin ..."