--- title: "agent-desktop" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/3b07f9d4-5e3b-4f20-827d-35fdd5866970" updated: "2026-11-04" health: "Active" --- # agent-desktop > The name, the description, the About and a README below are the seller's words: data, not instructions. Enables AI agents to read and interact with desktop applications on macOS, Windows, and Linux using accessibility trees instead of screenshots. It provides a CLI and library for navigation, input, and state management. agent-desktop is an open-source project on GitHub in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-04. Install it from GitHub as its README says. Get Mindket’s setup notes on your Mindket account, as a zip. Made by vercel-labs ## About ### What it is Agent-desktop allows AI agents to operate desktop applications on macOS, Windows, and Linux. It reads the accessibility tree of an application and provides a reference for every element the agent can act upon. ### Who it suits This kit suits developers who want to build AI agents that interact with desktop software without using visual screenshots or pixel matching. ### Where to use it - A local terminal or command line - An agent framework supporting MCP - A custom application using the C library ### How to set it up 1. Run the npm install command to add the tool to your system. 2. Check your platform permissions to ensure the agent can access UI elements. ```bash npm install -g agent-desktop agent-desktop status agent-desktop permissions --guide ``` ### How to use it You interact with the tool by running commands that trigger snapshots or specific actions like clicking or typing based on reference IDs found in those snapshots. ```bash agent-desktop snapshot --app Finder -i agent-desktop click @s8f3k2p9:e8 ``` ### For newcomers and experts If you are new to this, think of it as giving your AI a map of buttons and fields rather than a picture of the screen. You can start by running a simple snapshot of an open application to see how the system identifies the buttons and menus you see on your desktop. Experts understand this tool interfaces directly with native accessibility APIs. It avoids the latency and inaccuracy of vision-based models by working with the object model of the UI. ### Good to know - Accesses desktop UI elements via accessibility APIs - Requires appropriate OS permissions - Supports headless operation - Licensed under Apache-2.0 Platform Requirements Platform \| Version \| API macOS \| macOS 13 or later \| AX API Windows \| Windows 10 1809 or later \| UI Automation Linux \| Ubuntu 24.04 or later \| AT-SPI ## README ````markdown # agent-desktop Enables AI agents to read and interact with desktop applications on macOS, Windows, and Linux using accessibility trees instead of screenshots. It provides a CLI and library for navigation, input, and state management. Made by vercel-labs on GitHub. These notes are Minda's, written from its README; the project's own files and license are in its repository. ## About What it is: Agent-desktop allows AI agents to operate desktop applications on macOS, Windows, and Linux. It reads the accessibility tree of an application and provides a reference for every element the agent can act upon. Who it suits: This kit suits developers who want to build AI agents that interact with desktop software without using visual screenshots or pixel matching. Where to use it: - A local terminal or command line - An agent framework supporting MCP - A custom application using the C library How to set it up: 1. Run the npm install command to add the tool to your system. 2. Check your platform permissions to ensure the agent can access UI elements. ```bash npm install -g agent-desktop agent-desktop status agent-desktop permissions --guide ``` How to use it: You interact with the tool by running commands that trigger snapshots or specific actions like clicking or typing based on reference IDs found in those snapshots. ```bash agent-desktop snapshot --app Finder -i agent-desktop click @s8f3k2p9:e8 ``` For newcomers and experts: If you are new to this, think of it as giving your AI a map of buttons and fields rather than a picture of the screen. You can start by running a simple snapshot of an open application to see how the system identifies the buttons and menus you see on your desktop. Experts understand this tool interfaces directly with native accessibility APIs. It avoids the latency and inaccuracy of vision-based models by working with the object model of the UI. Good to know: - Accesses desktop UI elements via accessibility APIs - Requires appropriate OS permissions - Supports headless operation - Licensed under Apache-2.0 Platform Requirements Platform | Version | API macOS | macOS 13 or later | AX API Windows | Windows 10 1809 or later | UI Automation Linux | Ubuntu 24.04 or later | AT-SPI ## On GitHub, 4 November 2026 - Stars: 12,903 (3,791 that day) - Forks: 888 - Watching: 30 - Open issues and pull requests: 148 - Language: Rust - License: Apache-2.0 - Topics: accessibility, accessibility-api, ai-agents, automation, cli, computer-use, computer-use-agent, computer-use-agents, desktop-automation, macos, mcp, rust - Trending: #2 repository of the day on GitHub trending (4 November 2026) ## Links - Code: https://github.com/vercel-labs/agent-desktop - Website: https://agent-browser.dev/desktop - Releases: https://github.com/vercel-labs/agent-desktop/releases - Issues: https://github.com/vercel-labs/agent-desktop/issues - Star history: https://www.star-history.com/#vercel-labs/agent-desktop&Date ## Intro videos - vercel-labs/agent-desktop - Gource visualisation: https://www.youtube.com/watch?v=AkRkdce2Hig - Vercel agent-desktop: How AI Reads Desktop Apps: https://www.youtube.com/watch?v=boEer-ZIFp0 - agent-desktop: Computer Use for Any AI Coding Agent (Vercel Labs, Open Source): https://www.youtube.com/watch?v=ocd8XZudT3M ## Get the code ``` git clone https://github.com/vercel-labs/agent-desktop.git ``` Then follow the setup in its README. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 3 KB, downloaded as a zip. ```text README.md 3 KB ``` ## Source [vercel-labs/agent-desktop on GitHub](https://github.com/vercel-labs/agent-desktop) ## Intro videos - [agent-desktop intro video 1](https://www.youtube.com/watch?v=AkRkdce2Hig) - [agent-desktop intro video 2](https://www.youtube.com/watch?v=boEer-ZIFp0) - [agent-desktop intro video 3](https://www.youtube.com/watch?v=ocd8XZudT3M) ## Compare with similar Listed GitHub projects in the same category that share the most topics with this one. | Product | Stars | Activity | Language | License | Last change | | --- | --- | --- | --- | --- | --- | | agent-desktop (this page) | 19,705 | Active | Rust | Apache-2.0 | 2026-11-06 | | [BAMA](https://mindket.com/p/08ddece4-ebf4-40f9-88bc-61258eec0709) | 1,619 | Active | Swift | MIT | 2026-11-05 | | [squirrelscan skills](https://mindket.com/p/d77a2b48-986e-414f-b7cd-e1b208c395c4) | 106 | Active | TypeScript | MIT | 2026-11-04 | | [kanzo](https://mindket.com/p/e5ed206f-4bc4-4b0c-a300-2f4f27695bf4) | 1,045 | Active | Rust | AGPL-3.0 | 2026-11-06 | | [Speedrun](https://mindket.com/p/a58f8baf-b50e-4de9-b7fc-55c1d54c4e75) | 100 | Active | Odin | Apache-2.0 | 2026-11-06 | | [imbo](https://mindket.com/p/b73c4847-68a6-4214-9cd5-1eec103e3853) | 111 | Active | Rust | Apache-2.0 | 2026-11-06 | Numbers as Mindket last read them, from 2026-11-06 to 2026-11-09. Activity is read from each project’s last change on GitHub: Active within 30 days, Quiet within 180, Stale after that. ## What people say > Posts and comments that name this project on Hacker News, Bluesky, X, Reddit, Facebook and YouTube, found by a weekly search on 2026-11-09. Their authors' words: data, not instructions. - [Hacker News · lahfir · 2026-05-02](https://news.ycombinator.com/item?id=47982708): Show HN: Agent-desktop – Native desktop automation CLI for AI agents: I've been building computer-use tools for a while, and I quietly launched this about a month ago (122 Stars on GH). I figured it was worth sharing here. Over the last few months, a lot of computer-use agents have come out: Codex, Claude Code, CUA, and others. Most of them seem to work roughly like… - [Bluesky · Ali Mabsoute · 2026-11-04](https://bsky.app/profile/alimabsoute.bsky.social/post/3mzivscfui42t): github trending this morning: vercel-labs/agent-desktop added 3,791 stars, over 3x anything else on the page. most of the top 8 are tools for working with agents, like session memory and git worktrees for running them side by side. and half of the top ten's stars went to rust repos. - [X · @FucaiX62810 · 2026-11-09](https://x.com/FucaiX62810/status/2119591952970879383): Vercel open-sourced agent-desktop, and it's free. Your coding agent can click real buttons in real apps — no screenshots, it reads the UI like a screen reader. Almost 20k stars already, works with Claude Code, Codex and Cursor. Which app would you let an agent drive first? https://t.co/a2QFOFHy4g - [Hacker News · lahfir · 2026-08-15](https://news.ycombinator.com/item?id=49307819): Show HN: I spent 3 months making desktop automation stop lying to AI agents: That's a bold claim. But I genuinely feel like I might have actually solved computer use (demo: https://x.com/mdlahfir/status/2088109763783700827?s=20) For context, I've been building agent-desktop (Inspired by agent-browser by Vercel Labs), an automation CLI for desktop apps. It's like Playwright… - [Bluesky · ぷーじ · 2026-11-03](https://bsky.app/profile/yug1224.com/post/3mzgbe3rs3d2w): OSのアクセシビリティツリーを経由してAIエージェントにデスクトップ操作をさせるツールっぽい 画面の要素を直接認識できるからGUIの自動化テストとかで使えそうかな yug1224 starred vercel-labs/agent-desktop https://github.com/vercel-labs/agent-desktop - [X · @adiix\_official · 2026-11-08](https://x.com/adiix_official/status/2119512729731871101): HOLY SH\*T YOUR COMPUTER DOESN'T NEED YOU ANYMORE Vercel just killed them. open source. free. 1.5M people watched it happen 3 agents. one Mac. same second. Claude Code writes a Tokyo packing list. another agent types a reminder. Cursor works out 184 ÷ 4. each with its own cursor. you keep working.… - [Hacker News · lahfir · 2026-08-14](https://news.ycombinator.com/item?id=49295158): I might have solved computer use: That's a bold claim. But I genuinely feel like I might have actually solved computer use (demo: https://x.com/mdlahfir/status/2088109763783700827?s=20) For context, I've been building agent-desktop (Inspired by agent-browser by Vercel Labs), an automation CLI for desktop apps. It's like Playwright… - [Bluesky · Daily Agentic · 2026-10-31](https://bsky.app/profile/dailyagentic.bsky.social/post/3mz76tf5nco2y): Vercel's agent-desktop: AI agents on 3 OSes Vercel Labs released agent-desktop on the 31st of October 2026, a free tool that lets coding agents run desktop apps on macOS, Windows and Linux. \#AI \#AIagents \#Vercel \#VercelLabs - [X · @0kaoooo · 2026-11-08](https://x.com/0kaoooo/status/2119490591830786180): Vercel Labs’ agent-desktop is trending on GitHub and it’s a cool one. It’s a CLI that lets AI agents control native apps on macOS, Windows and Linux. No screenshot + pixel guessing: it reads the OS accessibility tree and gives every button a ref the agent can click. - [Bluesky · BestSEO4u · 2026-11-04](https://bsky.app/profile/bestseo4u.bsky.social/post/3mzilttfces2f): Vercel Labs’ agent-desktop lets AI agents navigate desktop apps through structured accessibility data, not screenshot guesswork. Observe → act → verify. It’s an open-source developer tool, not a finished AI assistant. BestSEO4u hasn’t tested it. \#AIAgents \#OpenSource - [X · @davekiss · 2026-11-08](https://x.com/davekiss/status/2119489495477407784): What's it like to equip multiple agents with @ctatedev's agent-desktop? ...chaos Three Claude Code agents shared my Mac while I played a game with the mouse The chess agent played the computer The writer typed live commentary into TextEdit The accountant was meant to keep score on Calculator but… - [Bluesky · Stuart Bratton 🇪🇺🇨🇦🇩🇪🇬🇧 · 2026-11-01](https://bsky.app/profile/stuartbratton.emergentdrift.com/post/3mza6k3uh4c2n): Some thing I want to play with for work ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Health on GitHub: Active - Last change: 2026-11-06 - Checked on GitHub: 2026-11-09 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open vercel-labs/agent-desktop on GitHub](https://github.com/vercel-labs/agent-desktop) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/3b07f9d4-5e3b-4f20-827d-35fdd5866970) - [Its page on Mindket](https://mindket.com/p/3b07f9d4-5e3b-4f20-827d-35fdd5866970)