Overview

Aiden Hardware Demo is a specialized automation and AI demonstration platform built for the Luckfox Pico Zero (RV1106). It integrates high-speed HDMI frame capture, full-duplex audio, and USB HID peripheral emulation to create an autonomous system capable of observing and interacting with external devices (such as smartphones or PCs) via a Large Language Model (LLM) agent.

The system bridges the gap between low-level hardware control and high-level AI reasoning by exposing hardware capabilities through a layered C++ service architecture and a Go-based agent runtime.

System Architecture

The project is organized into a hierarchical stack, ensuring that hardware resources are managed by dedicated owners while providing flexible access to high-level applications.

Overall System Diagram

The following diagram illustrates how the major software components interact and their corresponding locations in the codebase.

System Block Diagram

Sources: docs/02-architecture/overview.md5-34 src/aiden_sdk.cpp1-20 src/agent/pkg/runtime/runtime.go1-50

Major Subsystems

1. Hardware Platform

The project targets the Luckfox Pico Zero, utilizing its RV1106 SoC. Key hardware integrations include:

• HDMI Capture: Handled via the TC358743XBG bridge connected to the CSI interface.
• USB HID: Emulates a keyboard (/dev/hidg0) and a touchscreen/mouse (/dev/hidg1) using the Linux Gadget framework.
• GPIO: Used for hardware wakeup triggers.

For details on wiring and device nodes, see Hardware Platform and Connections.

2. C++ Hardware Services

To prevent resource contention, dedicated C++ daemons manage the hardware:

• frame_service: Captures video frames into a ring buffer and provides JPEG-encoded snapshots via Unix Domain Sockets (UDS) src/frame_service_main.cpp1-50
• audio_service: Manages PCM recording and playback sessions src/audio_service_main.cpp1-40
• UDS Transport: A custom IPC protocol using JSON headers and binary payloads for cross-language communication src/uds_server.cpp1-30

For details on the IPC protocol and service implementations, see C++ Hardware Services.

3. Go Agent

The Aiden Go Agent is the "brain" of the system. It runs a ReAct-style loop to:

• Observe: Take screenshots via frame_service.
• Reason: Consult an LLM (OpenAI, OpenRouter, or Ollama).
• Act: Execute HID commands (clicks, swipes, typing) or system commands.
• Interact: Process voice via a pipeline involving VAD (Voice Activity Detection), STT (Speech-to-Text), and TTS (Text-to-Speech).

For details on the agent runtime and tools, see System Architecture and Data Flows.

Data Flow: Observation to Action

The following diagram maps the flow of a single "turn" from the agent perceiving the screen to performing a touch gesture.

Visual Control Data Flow

Sources: docs/02-architecture/overview.md55-73 src/agent/pkg/tools/tools_hid.go1-100 src/frame_ipc.cpp1-40

Child Pages

• Hardware Platform and Connections: Detailed pinouts, HDMI-to-CSI bridge configuration, and USB Gadget setup.
• System Architecture and Data Flows: Deep dive into the layered software design, IPC mechanisms, and the end-to-end voice/visual pipelines.

Sources: README.md1-70 docs/README.md1-69 docs/02-architecture/overview.md36-53