API · Render mode
Example: stream your own voice
A small client that starts a render session, makes the avatar speak once and stops. The Python version streams your own voice's audio; the JavaScript version sends text that the avatar voices itself.
The client
# pip install requests websockets
import asyncio, base64, json, os, wave
import requests, websockets
API = "https://talk.selviaai.com/v1"
H = {"Authorization": f"Bearer {os.environ['SELVIA_API_KEY']}"}
AGENT_ID = 7
WAV = "reply.wav" # mono, 16-bit PCM, e.g. 24000 Hz: your voice's output
SAID = "[warmth] Hi! Thanks for stopping by. How can I help?" # what the audio says
async def events(ws):
"""Print every event; stop at the end of our utterance."""
async for raw in ws:
if isinstance(raw, bytes):
continue
ev = json.loads(raw)
print(ev["type"], ev.get("source_event_id", ""))
if ev["type"] == "error":
raise RuntimeError(ev["error"])
if ev["type"] in ("agent.speak_ended", "agent.speak_interrupted"):
return
async def main():
s = requests.post(f"{API}/sessions", headers=H, json={"agent_id": AGENT_ID, "mode": "render"})
s.raise_for_status()
s = s.json()
print("Open this for the visitor:", s["player_url"])
input("Press Enter once the page shows the avatar...")
try:
with wave.open(WAV, "rb") as w:
assert w.getnchannels() == 1 and w.getsampwidth() == 2, "mono 16-bit PCM"
rate = w.getframerate()
async with websockets.connect(f"{s['ws_url']}&sample_rate={rate}") as ws:
json.loads(await ws.recv()) # session.state_updated: connected
listener = asyncio.create_task(events(ws))
first = True
while frames := w.readframes(rate // 10): # 100 ms pieces, as fast as we can
msg = {"type": "agent.speak", "audio": base64.b64encode(frames).decode()}
if first:
msg.update(event_id="hello", text=SAID) # the words pick the expressions
first = False
await ws.send(json.dumps(msg))
await ws.send(json.dumps({"type": "agent.speak_end"}))
await listener
finally:
requests.post(f"{API}/sessions/{s['session_id']}/stop", headers=H)
asyncio.run(main())Making it a real agent
- •Replace the WAV file with your speech engine's stream: send each audio chunk as it arrives, then
agent.speak_endwhen the reply is complete. - •When your agent hears the visitor start talking, send
agent.interrupt, then the new reply. - •Send
agent.start_listeningwhile the visitor speaks, for the listening look. - •Keep one socket open for the whole session and stop the session when the conversation ends.
If your speech engine streams audio over HTTP, a chunk can end in the middle of a sample. Keep the last byte of an odd-length chunk and put it in front of the next one: the avatar only accepts whole 16-bit samples.
Expressions from the text, inline tags and exact timing are covered in Expressions.