Model Providers¶

Agentlab ships with three built-in model providers — KARLI, KARLI (AISL) and tokeneurope — alongside the standard Langflow providers (OpenAI, Anthropic, etc.). Unlike earlier versions, there is no dedicated "KARLI Model Provider" component — providers are surfaced through the standard Langflow model components.

Models are grouped by type:

LLMs — Chat models from KARLI, KARLI (AISL) and tokeneurope, selectable from any component that accepts a model (Language Model, Agent, etc.).
Embeddings — Embedding models hosted by KARLI and KARLI (AISL), selectable from the Embedding Model component.
Data Extraction — Document- and audio-extraction models, used by the Read File component when its backend is set to karli or aisl. KARLI and KARLI (AISL) are the only providers offering this category.

KARLI¶

KARLI is the default provider, offering LLMs, embeddings, and document extraction via the KARLI platform.

Credentials¶

The KARLI provider uses two separate endpoints — a LiteLLM proxy for LLM inference (including tool calling) and the KARLI backend for services like document extraction. Each endpoint has its own base URL and API key.

Variable	Required	Secret	Description
`LITELLM_BASE_URL`	No	No	Base URL of the LiteLLM proxy used for LLM inference. Falls back to the `LITELLM_BASE_URL` environment variable; defaults to the production endpoint when neither is set.
`LITELLM_API_KEY`	Yes	Yes	API key for LLM inference via the LiteLLM proxy. Falls back to the `LITELLM_API_KEY` environment variable.
`KARLI_BASE_URL`	No	No	Base URL of the KARLI backend (used for document extraction and other backend services). Falls back to the `KARLI_BASE_URL` environment variable.
`KARLI_API_KEY`	No	Yes	API key for KARLI backend services (e.g. document extraction). Falls back to the `KARLI_API_KEY` environment variable.

When is KARLI_API_KEY needed?

Only when accessing the Agentlab API directly without a Karli Studio session. When accessed through Karli Studio, a JWT is used automatically and this field is ignored. Leave it blank if you use Studio.

KARLI_API_KEY is a separate credential from LITELLM_API_KEY. The LiteLLM key covers model inference; the KARLI key covers backend services such as document extraction.

KARLI (AISL)¶

KARLI (AISL) reaches the AI Service Layer — a single gateway fronting chat, embeddings and document extraction — through one credential instead of the two base URLs and two keys the KARLI provider needs.

The two providers do not currently serve the same models. AISL's catalog is read from the gateway, so it reflects whatever that instance actually offers; the model names will converge with KARLI's as the gateway takes them on.

Both providers can be enabled at once. KARLI remains in use on the production system until the switch to the in-house Kubernetes stack, so this provider is additive: existing flows keep running on KARLI untouched.

Credentials¶

Because one gateway serves every surface, there is a single credential and no base URLs to configure in the UI:

Variable	Required	Secret	Description
`AISL_API_KEY`	Yes	Yes	API key for the AI Service Layer, covering chat, embeddings and document extraction. Falls back to the `AISL_API_KEY` environment variable.

The gateway address is an instance-level environment setting, configured by the operator and never exposed in the model-provider form:

Environment variable	Required	Description
`AISL_BASE_URL`	Yes	Root URL of the AI Service Layer gateway, e.g. `https://gateway.dev.aisl.karli.ai`. Chat and embeddings are served from `{AISL_BASE_URL}/v1`, extraction from `{AISL_BASE_URL}/api/v1`.

When is AISL_API_KEY needed?

Same rule as KARLI: only when accessing the Agentlab API directly without a Karli Studio session. Through Karli Studio a JWT is used automatically for document extraction.

Document extraction requires object storage

The AISL docparse API fetches the document itself from a presigned URL rather than accepting an upload. That requires LANGFLOW_STORAGE_TYPE=s3; on instances using local file storage, use the docling extraction backend or the KARLI provider instead.

Model catalog¶

The chat and embedding lists are read live from {AISL_BASE_URL}/v1/models, and the extraction list from {AISL_BASE_URL}/api/v1/services?type=docparse. When the gateway is unreachable, a bundled catalog is used instead so the pickers stay populated.

tokeneurope¶

tokeneurope is a European-hosted model provider offering OpenAI-compatible LLM access. Like KARLI, it appears as a selectable provider in any model-aware component.

Credentials¶

Variable	Required	Secret	Description
`TOKENEUROPE_BASE_URL`	No	No	Base URL for the tokeneurope endpoint. Falls back to the `TOKENEUROPE_BASE_URL` environment variable.
`TOKENEUROPE_API_KEY`	Yes	Yes	API key for tokeneurope model inference. Falls back to the `TOKENEUROPE_API_KEY` environment variable.

tokeneurope currently offers LLMs only — it does not provide embedding or data-extraction models.

Restricting available providers¶

An instance can be limited to a subset of providers with the MODEL_PROVIDERS environment variable — a comma-separated list of provider names exactly as they appear above:

MODEL_PROVIDERS=KARLI,KARLI (AISL),Ollama

Unset or empty means no restriction. Names that match no known provider are ignored.

This is an operator-level gate, distinct from credential configuration. A provider is normally offered once someone configures its API key; a provider missing from MODEL_PROVIDERS is withheld even when a key is configured. The list applies to every model picker (LLM, embedding, extraction), the /api/v1/models API, and the builder assistant's provider probe.

Data Sovereignty

For deployments with data sovereignty requirements, MODEL_PROVIDERS restricts the instance to approved providers, and the Karli Studio administrator controls which providers are enabled at the platform level, so that only models hosted in approved regions are reachable.