# AI Agents

> AI-powered analysis and processing tools including vision analysis, audio transcription with speaker diarization, and text embeddings generation.

Source: https://cotera.co/docs/reference/tools/individual-tools/ai-agents

---

**Authentication Type:** No Authentication
**Description:** AI-powered analysis and processing tools including vision analysis, audio transcription with speaker diarization, and text embeddings generation.

***

## Vision Analysis

AI-powered image and visual content analysis.

### Analyze Image

Analyze an image using an AI model to extract insights, descriptions, objects, text, or other details. Provide the image URL and specify which AI model to use.

**Operation Type:** Query (Read)

**Parameters:**

* **prompt** `string` (required): Prompt used to analyze the image
* **imageUrl** `string` (required): URL of the image to analyze
* **model** `string` (required): LLM model to use for image analysis. Options: "gpt-4o-mini", "gpt-4.1", "gpt-4.1-mini", "gpt-4.1-nano", "gemini-2.5-flash-preview-05-20"

**Returns:**

* **result** `string`: The AI-generated analysis of the image

**Example Usage:**

```json
{
  "prompt": "Analyze this product image and describe its features, condition, and potential market value. Include details about materials, design elements, and any visible wear or damage.",
  "imageUrl": "https://example.com/images/vintage-watch.jpg",
  "model": "gpt-4o-mini"
}
```

***

## Audio Transcription

Transcribe audio files with speaker diarization using OpenAI Whisper.

### Transcribe with Speaker Diarization

Transcribe audio from a URL and identify different speakers in the conversation. Returns timestamped segments with speaker labels.

**Operation Type:** Query (Read)

**Parameters:**

* **audioUrl** `string` (required): URL to the audio file to transcribe
* **language** `string` (nullable): Language of the audio (ISO-639-1 format, e.g., "en", "es")
* **prompt** `string` (nullable): Optional text to guide the model's style or continue a previous audio segment

**Returns:**

* **segments** `array of objects`: Array of transcribed segments with speaker diarization
  * **id** `string`: Segment ID
  * **start** `number`: Start timestamp in seconds
  * **end** `number`: End timestamp in seconds
  * **text** `string`: Transcribed text for this segment
  * **speaker** `string` (nullable): Speaker identifier (e.g., "SPEAKER\_00", "SPEAKER\_01")
* **language** `string` (nullable): Detected language of the audio
* **duration** `number` (nullable): Duration of the audio in seconds
* **text** `string` (nullable): Full transcribed text

**Example Usage:**

```json
{
  "audioUrl": "https://example.com/recordings/meeting-2024-01-15.mp3",
  "language": "en",
  "prompt": "This is a business meeting discussing quarterly sales targets"
}
```

***

## Text Embeddings

Generate vector embeddings from text using OpenAI models for semantic search, RAG, or similarity comparisons.

### Generate Embeddings

Generate vector embeddings from text for semantic search, RAG, or similarity comparisons.

**When to use:** Convert text into numerical vectors (embeddings) that capture semantic meaning.

**Perfect for:**

* Preparing text for vector search in databases like Turbopuffer
* RAG (Retrieval Augmented Generation) pipelines
* Semantic similarity comparisons
* Clustering similar content

**Operation Type:** Query (Read)

**Parameters:**

* **text** `string` (required): Text to generate embeddings for (max 8191 tokens)
* **model** `string` (required): Embedding model to use. Options: "text-embedding-3-small", "text-embedding-3-large"
  * text-embedding-3-small (1536 dimensions) - Fast, cost-effective, recommended for most use cases
  * text-embedding-3-large (3072 dimensions) - Higher quality, more expensive

**Returns:**

* **embedding** `array of numbers`: The embedding vector for the input text
* **dimensions** `number`: Number of dimensions in the embedding vector

**Example Usage:**

```json
{
  "text": "Machine learning is a subset of artificial intelligence that focuses on algorithms that can learn from data without being explicitly programmed.",
  "model": "text-embedding-3-small"
}
```

**Use with Turbopuffer:** Generate embeddings for your text, then upsert to Turbopuffer with the vector for semantic search capabilities.

***

## Common Use Cases

**Audio Transcription & Analysis:**

* Transcribe meeting recordings with automatic speaker identification for accurate meeting minutes
* Convert podcast episodes and interviews to text with speaker labels for content repurposing
* Process customer service calls to generate timestamped transcripts for quality assurance and training
* Create searchable archives of video content by extracting and transcribing audio tracks

**AI & Machine Learning:**

* Generate embeddings for RAG (Retrieval Augmented Generation) systems and AI chatbots
* Create semantic search indexes by combining embeddings generation with Turbopuffer storage
* Build content recommendation systems using vector similarity comparisons
* Develop AI assistants that understand context and meaning through embeddings

**Product and Inventory Analysis:**

* Analyze product images for e-commerce listings to generate detailed descriptions and feature lists
* Assess condition and quality of items from photos for resale marketplaces and inventory management
* Extract text from product labels and packaging for catalog management and compliance tracking

**Content Moderation and Classification:**

* Automatically classify and moderate user-generated image content for social platforms
* Detect inappropriate or harmful visual content in uploaded images and media files
* Analyze images for brand safety and advertising suitability across different content categories

**Document and Data Extraction:**

* Extract text and data from scanned documents, receipts, and business forms
* Analyze charts, graphs, and visual data presentations to extract key metrics and insights
* Process screenshots of applications and interfaces for quality assurance and testing documentation

**Creative and Marketing Analysis:**

* Analyze visual content for brand consistency and design quality assessment
* Generate detailed descriptions of artwork, photography, and creative assets for marketing purposes
* Evaluate visual elements in marketing materials for accessibility and design effectiveness