Connecting OpenClaw agents to XiaoZhi voice devices

In March 2026 we moved our XiaoZhi server work onto a fresh copy of the open-source xiaozhi-esp32-server, then used it to connect voice devices to OpenClaw agents. The main branch of our repo tracks upstream. Our changes sit on two branches.

Porting our server additions

The chromia branch carries our earlier work forward in one commit: Privy-based accounts, device bind and unbind, usage and token tracking, interaction summaries, and the CoinGecko plugin. It also adds a BytePlus streaming ASR provider, a metrics module, and three HTTP endpoints:

• POST /xiaozhi/chat runs text through the LLM using a virtual session and, if the device is connected, speaks the reply on it. If the device is offline the reply is still returned.
• POST /xiaozhi/echo sends exact text to the device's TTS with no LLM step.
• GET /xiaozhi/devices lists connected devices.

A SKILL.md documents these endpoints for coding agents, along with when to extend the server with a Python plugin (needs device state) versus an MCP server (any language, process isolation).

The adapter

xiaozhi-openclaw-adapter started as a project by dsw0000: a JSON-RPC 2.0 WebSocket server that runs inside xiaozhi-server and exposes three tool wrappers. We implemented those tools so traffic flows both ways:

• send_message: the server broadcasts a message/send notification to connected OpenClaw clients, which deliver it to channels such as Telegram or Discord.
• device_control: commands are routed to the device as natural language through /xiaozhi/chat, with a broadcast fallback when no device is configured.
• agent_task: prompts are submitted asynchronously, return a task id immediately, and can be polled or cancelled through an in-memory task store.

We also added a devices/list method.

The OpenClaw plugin

The repo's skill/ directory is a TypeScript OpenClaw plugin, installable directly from the repository. It connects to the adapter over WebSocket with auto-reconnect, a 30 second ping heartbeat and optional bearer token auth. Tools are registered immediately and the connection is made in the background, so a slow server does not block OpenClaw startup. It ships a prebuilt dist/ so no build step is needed at install time, plus an openclaw.plugin.json manifest with a config schema.

Server integration

The feat/openclaw branch vendors the adapter into the server's tool providers with its config, protocol and executor tests, and adds a send_message plugin so the device's own LLM can push messages out through OpenClaw.