Get Started
Quickstart
A step-by-step guide to get started at Ottic.
This quickstart guide walks you through each essential step in Ottic:
1
Insert your model provider API key
Go to LLM Models and create model with your model provider API key.
This is essential enabling you to run LLM-as-a-judge evaluations.
2
Create your first prompt
In the Prompts
menu:
- Click “Create new prompt”;
- Add the prompt you will test;
- Save the prompt you created.
See more on prompts in the section create prompts
3
Write your first test case
In the Test Cases
menu:
- (1) Click “Create new test case”;
- (2) Define of details of your test case (title and description);
- Choose the prompt you just created;
- Add the input (the message you want to test);
- Add the expected output (the response expected);
- (3) Save your test case.
See more on test cases in the section write test cases
We provide an API to migrate your test cases to Ottic. See more on API reference
We provide white-glove support for test pipeline migration. Contact us.
4
Run your first test
In the Test Runs
menu:
- Click “Start new test run”;
- Select the test case you just created;
- Click, “Run” to run your test;
- See the result (passed, failed);
- Open the executed test case to explore details and reasoning.
See more on test runs in the section execute test runs
5
Publish your prompt to production
You can publish your prompt directly to production from Ottic.
In the Prompts
menu:
- Open the prompt you just tested;
- Select the version you want to publish from the dropdown;
- Click “Publish”.
To publish prompts to production from Ottic, install Ottic SDK.
See more on Integrations
To unlock the full potential of Ottic, you’ll find detailed guides on the next sections.