> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tazolabs.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Home assistant
> Control the lights in a three-room home by voice: Tazo picks the room and the action straight from the audio.
Assume you are to build a voice assistant for your home. At the simplest level we would like the assistant to control functionality of each appliance in each room. The assistant would have to answer two things about every command: which room does the speaker mean, and what should happen there? Tazo can answer both straight from the
**audio**, with no transcript.
We will also see that Tazo can correctly identify a variety of targets, such as a specific room, any two rooms such as Bedroom & Kitchen, the room a user currently is in, and the whole house.
In this cookbook we will build a small home assistant. It uses the
[Tazo Python SDK](https://docs.tazolabs.com/sdk/python/overview.md) and a home with three rooms, each with one light:
**Bedroom**, **Kitchen** and **Study**. Each command is two questions:
| Question | Options | Model |
| - | - | - |
| **Target** | one per room, one per combination of any two rooms, the whole house, unspecified, unknown | `tazo-large` |
| **Action** | turn on, turn off, other | `tazo-large` |
```mermaid theme={null}
flowchart LR
cmd["spoken command
WAV"] --> q1["Tazo: Target"]
cmd --> q2["Tazo: Action"]
ctx["speaker + room
+ light states"] --> q1
ctx --> q2
q1 --> g{{"both ≥ 80% confident
and a usable answer?"}}
q2 --> g
g -->|yes| act["switch the light(s)"]
g -->|no| ask["ask the speaker to clarify"]
```
Tazo answers one `choice` question per request, so each question is its own call on the same
audio. The two calls run in parallel. The model never switches anything: the app does, and
only when every answer reaches the confidence threshold.
## Setup
```bash theme={null}
python -m pip install tazo-sdk
export TAZO_API_KEY='your-key'
```
Each command is a WAV recording of at most 30 seconds.
## Model the home
The home is a dictionary of lights. Tazo sees its current state, plus who is speaking and
which room they are in, as the `state` of each request. The speaker's room matters for one
thing only: resolving "here" or "this room".
```python theme={null}
import asyncio
from dataclasses import dataclass, field
from itertools import combinations
from pathlib import Path
from tazo import AsyncTazo
from tazo.types import ChoiceAnswer
THRESHOLD = 0.8 # every answer must reach this confidence before a light changes
TARGET_MODEL = "tazo-large"
ACTION_MODEL = "tazo-large"
NO_TOOL = "NO_TOOL"
NO_TOOL_DESCRIPTION = "No usable request."
ROOMS = ["Bedroom", "Kitchen", "Study"]
@dataclass
class Home:
"""Replace this with your real device API."""
lights: dict[str, bool] = field(default_factory=lambda: {room: False for room in ROOMS})
def describe(self, speaker: str, room: str) -> str:
state = ", ".join(f"{name} light {'on' if on else 'off'}" for name, on in self.lights.items())
return (
f"Home with 3 rooms. {state}. "
f"The speaker is {speaker}, currently in the {room}. "
)
```
## Define the questions
The target question has one option per room, one per pair of rooms, and three escape
hatches: `whole_home`, `unspecified` and `unknown_target`. Whole-home is an option so that
"turn off all the lights" is recognized and then deliberately not acted on. The action
question is the same idea for the operation. Both also get a `NO_TOOL` option for audio
with no usable request.
```python theme={null}
def target_choices() -> tuple[dict[str, str], dict[str, list[str]]]:
"""The target options, and the rooms each one stands for."""
criteria, rooms_for = {}, {}
for name in ROOMS:
criteria[name.lower()] = f"The {name} light, named or as 'here' when the speaker is in the {name}."
rooms_for[name.lower()] = [name]
for a, b in combinations(ROOMS, 2):
criteria[f"{a.lower()}_{b.lower()}"] = f"Both the {a} and {b} lights, both spoken."
rooms_for[f"{a.lower()}_{b.lower()}"] = [a, b]
criteria["whole_home"] = "Every light in the home, no room named."
criteria["unspecified"] = "No identifiable room. A generic 'the lights' is not whole-home."
criteria["unknown_target"] = "A room that is not in this home."
return criteria, rooms_for
TARGET_INSTRUCTIONS = "Which room's light does the AUDIO target? Use the speaker's room only for 'here' or 'this room'."
ACTION_CHOICES = {
"turn_on": "Switch the light on.",
"turn_off": "Switch the light off.",
"other": "Unclear, compound or unsupported action.",
}
ACTION_INSTRUCTIONS = "Choose the tool that best matches the user's spoken request for light control."
```
> [!NOTE]
>
> A request's text (state, instructions and option descriptions) has a budget of about 1,000
> characters. That is why the descriptions above are so short. If you add rooms, the number of
> two-room options grows quickly, so consider dropping them and handling one room per command.
## Ask Tazo
One helper sends the audio with one question and returns the typed answer.
```python theme={null}
async def ask(client, audio, model, state, instructions, criteria) -> ChoiceAnswer:
result = await client.decisions.create(
audio=audio,
model=model,
state=state,
questions={
"action": {
"type": "choice",
"instructions": instructions,
"criteria": {**criteria, NO_TOOL: NO_TOOL_DESCRIPTION},
}
},
)
answer = result.answers["action"]
assert isinstance(answer, ChoiceAnswer)
return answer
```
## Decide what to do
This is the safety gate. A light changes only if the target is a real room (or pair of
rooms) with at least 80% confidence **and** the action is on or off with at least 80%
confidence. Anything else, including whole-home, becomes a clarifying question for the
speaker.
```python theme={null}
def decide(target: ChoiceAnswer, action: ChoiceAnswer) -> tuple[list[str], str] | str:
"""Rooms and operation to apply, or a clarifying question."""
_, rooms_for = target_choices()
if target.confidence < THRESHOLD or target.choice not in rooms_for:
return "Which room do you mean?"
if action.confidence < THRESHOLD or action.choice not in {"turn_on", "turn_off"}:
return "Do you want the light on or off?"
return rooms_for[target.choice], action.choice
async def handle_command(home: Home, audio: Path, speaker: str, room: str) -> str:
"""One spoken command: two questions on the same audio, asked in parallel."""
state = home.describe(speaker, room)
criteria, _ = target_choices()
async with AsyncTazo() as client:
target, action = await asyncio.gather(
ask(client, audio, TARGET_MODEL, state, TARGET_INSTRUCTIONS, criteria),
ask(client, audio, ACTION_MODEL, state, ACTION_INSTRUCTIONS, ACTION_CHOICES),
)
outcome = decide(target, action)
if isinstance(outcome, str):
return outcome
rooms, operation = outcome
for name in rooms:
home.lights[name] = operation == "turn_on"
return f"{operation} -> {', '.join(rooms)}"
```
## Run it
Say "lights on here" while standing in the kitchen, then "turn off the study light".
```python theme={null}
async def main() -> None:
home = Home()
print(await handle_command(home, Path("lights_here.wav"), speaker="Maya", room="Kitchen"))
print(await handle_command(home, Path("study_light_off.wav"), speaker="Maya", room="Kitchen"))
print(home.lights)
if __name__ == "__main__":
asyncio.run(main())
```
How the gate behaves on different commands:
| Spoken | Target | Action | Result |
| - | - | - | - |
| "Lights on here" (from the kitchen) | `kitchen` | `turn_on` | Kitchen on |
| "Turn on the bedroom and the study" | `bedroom_study` | `turn_on` | Both on |
| "Turn off all the lights" | `whole_home` | `turn_off` | "Which room do you mean?" |
## Build on it
* **Keep the context fresh.** Send the real device states and the speaker's real room on every
command. "Here" is only as good as the room you pass in.
* **Add more questions.** You can extend the app to more appliances and more specific things such as setting the thermostat to a particular temperature. Additional information would create another call to Tazo.
* **Tune the threshold.** 0.8 is a cautious default for actions that change the physical
world. Lower it for harmless actions, raise it for risky ones.
## Limitations
* Whole-home commands always ask for clarification on purpose.
## Bonus - Command to set the thermostat to a temperature
A setting command has a value, so it needs a third question. Tazo answers `choice` questions, so give it one option per temperature. Add a `set_temperature` action to the existing action question, and ask the temperature question only when that action wins:
```python theme={null}
ACTION_CHOICES["set_temperature"] = "Set the thermostat to a temperature."
TEMPERATURE_CHOICES = {f"t{c}": f"Set to {c} degrees Celsius." for c in range(16, 29)}
TEMPERATURE_INSTRUCTIONS = "Which temperature does the AUDIO ask the thermostat to be set to?"
async def set_thermostat(client, audio, state) -> str:
answer = await ask(client, audio, TARGET_MODEL, state, TEMPERATURE_INSTRUCTIONS, TEMPERATURE_CHOICES)
if answer.confidence < THRESHOLD or answer.choice == NO_TOOL:
return "What temperature do you want?"
return f"thermostat -> {answer.choice[1:]} °C"
```
Then call it from `handle_command`, right after the two answers arrive and before `decide`, while the client is still open:
```python theme={null}
if action.choice == "set_temperature" and action.confidence >= THRESHOLD:
return await set_thermostat(client, audio, state)
```
"Set the thermostat to 22 degrees" now returns `thermostat -> 22 °C`.
To set the thermostat in a particular room, combine the temperature with the target question you already ask. If each room has its own thermostat, the target answer already tells you which room or rooms the speaker means, including "here". Ask the temperature question in parallel with the target and action questions, and when the action is `set_temperature`, apply the temperature to the rooms from `decide`. The same confidence gate applies: if the target is unspecified or unsure, ask "Which room do you mean?" instead of changing a thermostat.
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.