> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tazolabs.com/llms.txt
> Use this file to discover all available pages before exploring further.
# User profiling
> Annotate a speech clip with Tazo, ask one question per annotation, and flag low-confidence labels for human review.
Labeling speech by hand, for example the speaker's age group or what they are doing
in a call, is slow. With Tazo you can ask those questions about the **audio itself**,
with no transcription step. You can use probabilities to determine which labels to trust and
which to ask a second reviewer for their opinion.
In this cookbook we will attempt to build an annotation tool. It takes a WAV clip
and returns one label per question, with a flag when a person should review the clip. It
uses the [Tazo Python SDK](https://docs.tazolabs.com/sdk/python/overview.md) and asks 3 questions:
| Question | Options |
| - | - |
| **Speaker age group** | Under 30, 30 to 49, 50 and over |
| **Speaker gender** | Male, female |
| **Dialog act** | 16 tags for what the speaker is doing, such as greeting, problem description or thanks |
```mermaid theme={null}
flowchart LR
clip["speech clip
WAV"] --> q1["Tazo: Speaker age group"]
clip --> q2["Tazo: Speaker gender"]
clip --> q3["Tazo: Dialog act"]
q1 --> gate{{"confidence ≥ 65%
and lead ≥ 12 points?"}}
q2 --> gate
q3 --> gate
gate -->|every label| ok["labels stand"]
gate -->|any label| review["clip needs review"]
```
## One question per request
Tazo Large answers one `choice` question per request, so each annotation is its own call
on the same clip. The three calls run in parallel.
Each question lists its options as a `criteria` map from an option ID to a description.
**The description is what Tazo reads**, so make options clearly different and describe each one
in plain language. Every question also gets a `NO_TOOL` option, so Tazo can say that
nothing fits when the clip is silent or unclear instead of guessing.
## Setup
```bash theme={null}
python -m pip install tazo-sdk
export TAZO_API_KEY='your-key'
```
Clips must be WAV files of at most 30 seconds. The SDK accepts a path, bytes
or a file object.
## Define the questions
```python theme={null}
import asyncio
from pathlib import Path
from tazo import AsyncTazo
from tazo.types import ChoiceAnswer
MODEL = "tazo-large"
MIN_CONFIDENCE = 0.65 # the winner must reach this probability...
MIN_MARGIN = 0.12 # ...and lead the runner-up by this much, or the clip needs review
NO_TOOL = "NO_TOOL"
NO_TOOL_DESCRIPTION = "None of the options fits: the clip is silent, unclear or unrelated to the question."
CONTEXT = (
"You are annotating a short recording of a person speaking. Listen to the audio itself "
"and answer only from what you hear."
)
QUESTIONS = {
"age": {
"name": "Speaker age group",
"instructions": "Which age group does the speaker in this audio clip most likely belong to?",
"options": {
"young": "The voice of a young adult under 30 years old.",
"middle": "The voice of a middle-aged adult between 30 and 49 years old.",
"older": "The voice of an older adult who is 50 years old or older.",
},
},
"gender": {
"name": "Speaker gender",
"instructions": "Which gender does the speaker in this audio clip most likely have?",
"options": {
"male": "The voice of a male speaker.",
"female": "The voice of a female speaker.",
},
},
"act": {
"name": "Dialog act",
"instructions": "What is the speaker doing in this audio clip?",
"options": {
"greeting": "Saying hello or introducing oneself or the company at the start of the call.",
"problem_description": "The caller explaining what they need or what went wrong, such as a lost card or a request.",
"open_question": "Asking a broad question such as how can I help you or is there anything else I can help with.",
"data_question": "Asking for a specific piece of information such as an address, a name or an account.",
"data_response": "Answering with a specific piece of information such as a number, an address or a name.",
"data_communication": "Reading out or stating information such as numbers or figures to the other person.",
"confirm_data": "Repeating or confirming a piece of information to check it is right.",
"procedure_explanation": "Explaining what was done or what will happen next, such as a card being ordered and when it will arrive.",
"bear_with_me": "Asking the other person to hold on or wait a moment.",
"yes_response": "Answering a question with yes or an agreement.",
"response": "Answering a question in general.",
"acknowledgement": "A short acknowledgement such as okay, alright or I see.",
"thanks": "Thanking the other person.",
"closing": "Saying goodbye or ending the call.",
"filler_disfluency": "Only filler sounds such as um, uh or hmm.",
"other": "Anything else that does not fit another option.",
},
},
}
```
## Decide when a label needs review
A label is trusted only when the winner is not `NO_TOOL`, has at least 65% probability, and leads the
runner-up by at least 12 points. A close call, such as 40% against 38%, has a reasonable
top confidence but a small lead, so the lead is checked as well.
```python theme={null}
def is_confident(answer: ChoiceAnswer) -> bool:
ranked = sorted(answer.probabilities.values(), reverse=True)
runner_up = ranked[1] if len(ranked) > 1 else 0.0
return (
answer.choice != NO_TOOL
and answer.confidence >= MIN_CONFIDENCE
and answer.confidence - runner_up >= MIN_MARGIN
)
```
## Annotate a clip
`annotate` asks one question. `profile_clip` asks every question about the same clip in
parallel and flags the clip when any label is uncertain.
```python theme={null}
async def annotate(client: AsyncTazo, audio: Path, question_id: str) -> dict:
"""One Tazo decision: a single question about the clip."""
question = QUESTIONS[question_id]
result = await client.decisions.create(
audio=audio,
model=MODEL,
state=CONTEXT,
questions={
"answer": {
"type": "choice",
"instructions": question["instructions"],
"criteria": {**question["options"], NO_TOOL: NO_TOOL_DESCRIPTION},
}
},
)
answer = result.answers["answer"]
assert isinstance(answer, ChoiceAnswer)
return {
"question": question["name"],
"choice": None if answer.choice == NO_TOOL else answer.choice,
"confidence": round(answer.confidence, 2),
"needs_review": not is_confident(answer),
}
async def profile_clip(audio: Path) -> dict:
"""Ask every question about the same clip, in parallel, and flag the clip for review if needed."""
async with AsyncTazo() as client:
labels = await asyncio.gather(*(annotate(client, audio, q) for q in QUESTIONS))
return {
"clip": audio.name,
"labels": dict(zip(QUESTIONS, labels)),
"needs_review": any(label["needs_review"] for label in labels),
}
```
Run it:
```python theme={null}
async def main() -> None:
import json
print(json.dumps(await profile_clip(Path("caller.wav")), indent=2))
```
The result looks like this. The shape is real; the numbers are illustrative.
```json theme={null}
{
"clip": "caller.wav",
"labels": {
"age": {"question": "Speaker age group", "choice": "middle", "confidence": 0.58, "needs_review": true},
"gender": {"question": "Speaker gender", "choice": "female", "confidence": 0.97, "needs_review": false},
"act": {"question": "Dialog act", "choice": "problem_description", "confidence": 0.93, "needs_review": false}
},
"needs_review": true
}
```
Age is the hard one: it is a judgment from the voice alone, so its confidence is
often low, which is exactly what the review flag is for.
## Build on it
* **Add an annotation.** Add an entry to `QUESTIONS` with a question and 2 to 20 options. Nothing else changes.
* **Keep corrections.** Store the Tazo label and the reviewer's correction side by side, and export the corrected one.
* **Tune the gate.** Raise `MIN_CONFIDENCE` to send more clips to review while you build trust in the labels.
## Limitations
* Tazo only chooses among the options you give it. Clear, distinct options matter more than a clever question.
* Age and gender are estimates from the voice alone, not facts. Use them with human review.
* The 65% and 12-point thresholds are starting points, not calibrated guarantees.
* A request has a text budget for the state, question and options together. If a question with many options is rejected as too long, shorten the descriptions.
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.