Skip to content

Language Models

The AI features of Atfinity run on a language model your installation connects: Atfinity Intelligence, AI Case Checks and the AI Prompt rule action. Atfinity sends the case data those features need to that model, so a model you host yourself keeps it inside your network.

The model is set for the whole installation by whoever runs it, and every tenant on it uses the same one. The AI features stay off until ENABLE_ATFINITY_INTELLIGENCE is on and a model is connected. Each tenant then decides, under Integrations in AI, whether its conversations may use tools and whether they may change anything.

The OpenAI ChatGPT, Google Gemini and Mistral AI integrations in the marketplace are separate from this model. A rule calls them like any other integration, with credentials of their own, and they do not power the AI features.

Connecting the model

Info

New in Atfinity 17.

Atfinity works with any model served through an OpenAI-compatible chat completions API, such as OpenAI itself, a gateway like OpenRouter, or a model you host with vLLM or Ollama. Atfinity Intelligence hands the model tools to call, so pick one that supports tool calling.

  • LLM_BASE_URL: the base URL of the API, such as https://api.openai.com/v1
  • LLM_API_KEY: the key Atfinity sends, which can stay empty for a model you host without authentication
  • LLM_MODEL: the model to ask, which falls back to DEPLOYMENT_NAME and then to gpt-5-chat
  • LLM_TOKEN_PARAM: max_tokens or max_completion_tokens, whichever your model expects. Left unset, Atfinity starts with max_tokens and switches the first time the model rejects it

The test_llm management command sends one request to the model and prints its answer, which confirms the connection without opening Atfinity.

Azure OpenAI

Leave LLM_BASE_URL unset to use Azure OpenAI. Atfinity then sends requests to the resource in AZURE_OPEN_AI_ENDPOINT, uses LLM_MODEL or DEPLOYMENT_NAME as the name of the deployment, and authenticates with LLM_API_KEY, or with AZURE_OPEN_AI_API_KEY when that is unset. Set AZURE_OPEN_AI_ENDPOINT to your own resource, since left unset it points at a default one rather than yours.