API · Render mode

Example: stream your own voice

A small client that starts a render session, makes the avatar speak once and stops. The Python version streams your own voice's audio; the JavaScript version sends text that the avatar voices itself.

The client

# pip install requests websockets
import asyncio, base64, json, os, wave
import requests, websockets

API = "https://talk.selviaai.com/v1"
H = {"Authorization": f"Bearer {os.environ['SELVIA_API_KEY']}"}
AGENT_ID = 7
WAV = "reply.wav"                      # mono, 16-bit PCM, e.g. 24000 Hz: your voice's output
SAID = "[warmth] Hi! Thanks for stopping by. How can I help?"   # what the audio says


async def events(ws):
    """Print every event; stop at the end of our utterance."""
    async for raw in ws:
        if isinstance(raw, bytes):
            continue
        ev = json.loads(raw)
        print(ev["type"], ev.get("source_event_id", ""))
        if ev["type"] == "error":
            raise RuntimeError(ev["error"])
        if ev["type"] in ("agent.speak_ended", "agent.speak_interrupted"):
            return


async def main():
    s = requests.post(f"{API}/sessions", headers=H, json={"agent_id": AGENT_ID, "mode": "render"})
    s.raise_for_status()
    s = s.json()
    print("Open this for the visitor:", s["player_url"])
    input("Press Enter once the page shows the avatar...")
    try:
        with wave.open(WAV, "rb") as w:
            assert w.getnchannels() == 1 and w.getsampwidth() == 2, "mono 16-bit PCM"
            rate = w.getframerate()
            async with websockets.connect(f"{s['ws_url']}&sample_rate={rate}") as ws:
                json.loads(await ws.recv())                     # session.state_updated: connected
                listener = asyncio.create_task(events(ws))
                first = True
                while frames := w.readframes(rate // 10):        # 100 ms pieces, as fast as we can
                    msg = {"type": "agent.speak", "audio": base64.b64encode(frames).decode()}
                    if first:
                        msg.update(event_id="hello", text=SAID)  # the words pick the expressions
                        first = False
                    await ws.send(json.dumps(msg))
                await ws.send(json.dumps({"type": "agent.speak_end"}))
                await listener
    finally:
        requests.post(f"{API}/sessions/{s['session_id']}/stop", headers=H)


asyncio.run(main())

Making it a real agent

  • •Replace the WAV file with your speech engine's stream: send each audio chunk as it arrives, then agent.speak_end when the reply is complete.
  • •When your agent hears the visitor start talking, send agent.interrupt, then the new reply.
  • •Send agent.start_listening while the visitor speaks, for the listening look.
  • •Keep one socket open for the whole session and stop the session when the conversation ends.
If your speech engine streams audio over HTTP, a chunk can end in the middle of a sample. Keep the last byte of an odd-length chunk and put it in front of the next one: the avatar only accepts whole 16-bit samples.

Expressions from the text, inline tags and exact timing are covered in Expressions.