Skip to content

Batch Models

Configure batch models for processing multiple text inputs simultaneously — used by post-processors, summary generation, and other background tasks.

Overview

Batch models are designed for high-volume asynchronous AI tasks that don't require real-time responses. They are used by:

  • Conversation Post-Processors (Summary, Knowledge Ingestion) — to analyze and summarize expired conversations
  • Message Analyzer (Turn Post-Processor) — for re-run capability
  • Document processing — for metadata extraction and regeneration tasks

Batch models are configured at the workspace level under Settings > AI Models > Batch Models.

Adding a Batch Model

Batch Models List

  1. Navigate to Settings > AI Models > Batch Models
  2. Click "Create Batch Model"
  3. Select a provider from the available options
  4. Configure the model settings
  5. Click "Save"

Supported Providers

Add Batch Model

Provider Description
Google AI Google's batch processing models
Anthropic Anthropic's Claude models for batch processing
Azure Azure OpenAI batch models
OpenAI OpenAI's batch processing models

Configuration

Each provider requires the same core fields:

Batch Model Configuration

Field Required Description
Batch Model Name Yes A descriptive name for this batch model configuration
Connect Credential Yes Select the provider credential configured under Settings > Credentials
Model Name Yes Select a model from the dropdown. Available models depend on the selected provider and credential
Thinking Level No Controls the depth of reasoning the model applies before generating a response. Options vary by model (e.g., "Minimal", "Low", "Medium", "High"). Higher thinking levels produce more thorough analysis but may increase processing time and cost

Azure Note

For Azure, select credentials using an Azure API version that includes the -preview suffix (minimum version: 2023-04-01-preview).

Model Availability

The list of available models is dynamic — new models are added and older ones deprecated as providers release updates. The models shown in the dropdown reflect what is currently available for your configured credentials.

Using Batch Models

Once configured, batch models become available for selection in:

  • Post-Processor configuration — select a batch model for Summary and Message Analyzer processors
  • Knowledge Repository — used for document summary regeneration and metadata extraction
  • Batch Monitoring — track the status of batch jobs via User Icon > Batch Monitoring

For details on post-processor configuration, see Post-Processors.