Ottic is a QA platform for LLM products that lets you test and continuously improve your LLM product. With Ottic, you can create and manage test cases and prompts, execute test runs and generate informative reports on your LLM product performance. Introduction

​
View interactive demo

Interested in seeing Ottic in action?
Schedule a call to see how Ottic fits your projects.
Prefer a quick video? Watch this Introduction to Ottic (1 min)

​
User flow

Here’s what the core user flow in Ottic looks like:
1

Add Your Prompt

Create the initial prompt to begin testing.
2

Write Your Test Cases

Define test cases and specify the expected behavior of your application.
3

Execute Test Runs

Run tests and review reports on successes and areas needing improvement.
4

Publish Your Prompt

Publish your prompt directly to production from Ottic.
This documentation will guide you through this process in detail.
We provide a white-glove migration of your existing testing pipeline.
Request your migration here.

​
Essentials

Creating prompts

Create, manage and publish prompts versions

Defining test cases

Define scenarios to assess your LLM’s responses

Executing test runs

Execute tests, analyze results, and identify improvement areas.

Datasets

Manage datasets to support context (RAG)

​
Advanced

Integrations

Connect with tools you team is using

API reference

Sync prompts, test cases and runs with your codebase