↓Skip to main content

Google-Adk

2026

In-app Live Assistant : Part 2 --- Building the Screen-Sharing Version

·9 mins
AI Experiments Google-Adk Vertex-Ai Gemini Javascript Screen-Sharing Websocket Multimodal Ai-Agents
The ADK Dev UI supports camera but not screen sharing. In this post I build a custom client that swaps getUserMedia for getDisplayMedia, writes a 16kHz AudioWorklet from scratch, and sends 1 FPS JPEG screen snapshots to the same WebSocket endpoint. The agent now sees the application instead of the user’s face.

In-app Live Assistant : Part 1 --- Walking Through Google's ADK Bidirectional Streaming Demo

·10 mins
AI Experiments Google-Adk Bidirectional-Streaming Vertex-Ai Gemini Python Multimodal Ai-Agents
I want to build a live AI assistant that can hear a user and see their screen at the same time, then talk back in real time. This post walks through getting Google’s ADK bidirectional streaming demo running locally with mic, camera, and voice out — and documents the things that tripped me up.