Tools
Tools let you connect external capabilities — a REST API call or a database query — that can be referenced directly inside your prompts when running tests. Once a tool is created, you can pull it into any prompt by typing @ and picking it from the list, so a run can call out to your API or database as part of its steps.
There are two kinds of tools:
- API Tools — call any REST endpoint (GET, POST, PUT, PATCH, DELETE).
- DB Tools — run a read-only SQL query against one of your connected databases.
Both live on the Tools page in your dashboard, and both are used the same way once created — the only difference is what you configure when setting one up.
Creating a Tool
- Go to the Tools page.
- Click Add Tool.
- Choose a type — API or Database.
- Fill in the fields (see below for each type).
- Click Save.
Every tool needs a Name. Names can only contain letters, numbers, hyphens, and underscores — no spaces (e.g. get_user_balance, create-ticket). This is the name you'll search for later when using @ in a prompt, so make it short and easy to recognize.
A Description is optional but strongly recommended — it's what helps the system decide when a tool is relevant to a given step, so the more specific you are about what the tool does and when it should be used, the better it works in practice.
API Tools
Use an API tool when you want a prompt to be able to call an external REST endpoint — for example, creating a record in your CRM, checking a feature flag, or hitting an internal health endpoint.
When creating an API tool, you'll configure:
| Field | Description |
|---|---|
| Method | The HTTP method — GET, POST, PUT, PATCH, or DELETE. |
| Endpoint | The full URL to call, e.g. https://api.example.com/users. |
| Headers | Optional key/value pairs — commonly used for auth tokens (Authorization: Bearer ...) or content-type overrides. Add as many as you need with Add header. |
| JSON request body | Only shown for methods that support a body (POST, PUT, PATCH). Provide the JSON payload to send, e.g. { "name": "John Doe", "active": true }. |
| Expected Status Code | Optional — the status code you expect on success (e.g. 200, 201). Useful for validating the call worked as intended. |
| Expected Sample Response | Optional — an example of what a successful response looks like. This is just a reference for you and doesn't affect the actual call. |
| Description | What the tool does and when it should be used — the more specific, the better. |
Once saved, the tool shows up on the Tools list tagged API.
DB Tools
Use a DB tool when you want a prompt to be able to query your database directly — for example, looking up a user's status, checking a record count, or verifying data written by a previous test step.
Before you start: connect a database
DB tools run against one of your org's active database connections. If you don't have one yet:
- Go to Integrations → Database.
- Click Connect Database (or Add Connection if you already have one and want another).
- Fill in the connection details — host, port, database name, username, password, and optionally a schema and whether SSL is required.
- Use Test to confirm the connection works before saving.
- Save the connection.
You can connect multiple databases and give each a distinct name — you'll pick which one a given DB tool runs against when you create it. Connections can be individually enabled/disabled or deleted later from the same Database page; a disabled or deleted connection will break any tool pointing at it, so double check before turning one off.
Creating the DB tool
When adding a DB tool, if you have no active connections yet, you'll see a prompt to go connect one first. Once you have at least one active connection, you'll configure:
| Field | Description |
|---|---|
| Connection | Which of your active database connections this tool's query runs against. |
| SQL | Your query. Read-only SELECT statements only — this is not for inserts, updates, or deletes. |
| Parameters | Automatically detected from your SQL as you type. |
| Description | What this query returns and when it should be used — be specific, since this is what determines when the tool gets used. |
Writing parameterized SQL
Reference parameters in your query using a colon prefix, e.g.:
SELECT * FROM users WHERE no_of_repos >= :no_of_repos;
As soon as you type a :placeholder, it automatically appears below as a parameter you can configure further:
- Type — string, number, boolean, date, or datetime.
- Description (optional) — helps clarify what value should be passed in.
- Aliases (optional, comma-separated) — alternate names this parameter might be referred to by.
- Required — whether the value must be supplied for the query to run.
If you remove a :placeholder from the SQL, its parameter entry is automatically dropped — you don't need to manage this list by hand, just edit the query and the parameters stay in sync.
Editing and Deleting Tools
From the Tools page:
- Click the pencil icon next to any tool to open it back up for editing. You can change the name, description, and the full configuration (endpoint/headers/body for API tools; connection/SQL/parameters for DB tools).
- Click the trash icon to delete a tool. You'll be asked to confirm — this can't be undone.
Use the search box and the type filter (API / DB) at the top of the Tools page to quickly find a tool if you have a lot of them, and sort the list by name, type, or creation date by clicking the column headers.
Using a Tool in a Prompt
Once a tool exists, you can bring it into any prompt — whether you're testing locally or kicking off a run — by typing @ followed by the tool's name. A tooltip/dropdown will pop up letting you pick from your available API and DB tools; select the one you want and it gets referenced in your prompt.
For example:
Check that the checkout total matches @get_order_total for order #4821, then confirm the confirmation email was sent.
Here, @get_order_total could be a DB tool that looks up the order total directly from your database, letting the run cross-check the UI against ground truth instead of relying only on what's visually rendered.
A few tips for getting the most out of this:
- Write clear descriptions. The description you give a tool when creating it is what determines whether and when it gets pulled in — vague descriptions lead to it being picked (or skipped) at the wrong times.
- Keep names short and distinctive. Since you'll be typing
@and searching by name, a name likeget_user_balanceis easier to find than something generic liketool1. - Double check required parameters. If a DB tool has required parameters, make sure the prompt gives enough context for the right values to be supplied.
- Test independently first. Before relying on a tool inside a larger prompt, it's worth confirming the underlying API endpoint or SQL query behaves as expected on its own.