I’ve been building Buddy, a free, open source (Apache 2.0) Android app. You type or say what you want in a floating bubble, and an AI agent reads the screen, taps and types through your apps to do it.
- Your choice of agent: Claude Code, Codex, OpenCode, Antigravity or Grok, running inside the app (Alpine Linux via proot) on your own plan. OpenCode’s free models need no account.
- No root, adb or PC: it reads the screen and taps through the accessibility service.
- Asks first: before anything is sent, posted, bought or deleted. It pauses when you touch the screen and never types passwords or payment details.
- Builds apps: with Linux tools on, it builds real APKs on the phone. This week I drew a maze on a paper towel, took a photo, and it made it a tilt game. The first build crashed on launch; it found the bug and fixed it.
To be upfront:
- What the agent reads on your screen goes to the AI service you pick.
- It’s early (2.2), tested most on a OnePlus 12 with Android 16.
- It’s not on Play (Google rejected the accessibility use), so it’s an APK from GitHub releases, built by GitHub Actions from the source.
Source and releases: github.com/ghorbelhamdi/buddy-android
Happy to answer questions here.
Aw sweet, man made horrors entirely within my comprehension.
Jesus Christ
Lol, no.
I ain’t your pal, Buddy.
I ain’t your buddy, friend.
I’m not your friend, pal 😄
It’s not on Play (Google rejected the accessibility use)
And rightly so. Stop making me side with Google.
Maybe if it were designed to run completely locally I could respect it.
It can now: since 2.2.25 it works with your own model server (Ollama, LM Studio or llama.cpp) on your network or over Tailscale, with no key, and nothing goes to an AI company. Honest caveat: small models (4–8B) handle simple requests but fumble multi-step tasks on the screen, so you’d want a bigger model for real use.
Oh hell no, never in my life will I let an agent control my phone. Fff— that.
Not a privacy invasion, at all. And you are abusing the accessibility features too? I’m proud of you.
Fair concern, and it’s not for everyone. The agent only sees the screen while it’s doing a task you gave it, it asks before anything is sent, and it’s open source, so you can check exactly what it does. If you’d rather not hand that to an AI, that’s a completely reasonable choice.
Why would anyone in the right state of mind allow that shit to control your phone??
Fair question, so here are real ones:
- Accessibility: with live voice (real-time, through your ChatGPT plan) you just talk to it. It does the tapping and reads things back, which helps people who can’t easily see or use a touchscreen.
- Testing apps: it exposes the phone as an MCP server, so Claude Code or Codex can drive your app on a real device for UI testing.
- Building apps on the phone itself: Buddy was mostly developed that way, with Claude Code running on the phone and using Buddy’s own phone tools to install and test each build.
It asks before anything is sent or bought and stops when you touch the screen. Not for everyone, and that’s fine.
Let Al have full access to your contacts and the first most likely device to have pictures of your genitals.
No.
Token work offloading.
Unethical bingo sentences – center square.It’s just an MCP server for the phone with an app to control it, basically computer use, but for Android. Agents using computers are already normal, so I’m not sure why phones would be different. And it runs on your own plan, or on local models if you’d rather keep it off the cloud entirely.
A much safer accessibility method would be a flutter webview. Local user will be confined to the webview instance and will still need to resolve ai agent challenges themselves.
Accessibility-from-scratch utilizing a server\client with a linux shell is what something like Shizuku was made for. Elevated privileged activities like listing installed apps, running apps, running a url (elevated) are not normal in ai agent workflows. MCP server resources can be “patched” this way from a very specific call… hence offloading somebody else’s token work.
It’s not on Play (Google rejected the accessibility use), so it’s an APK from GitHub releases, built by GitHub Actions from the source.
Did you explain its use as an accessibility aid being its core feature for differently abled people? If you have explained that as a core feature google could’ve budged. You would’ve needed to limit the API to select few like OpenAI, Gemini, Claude with additional guardrails and use reporting. But, it could’ve been accepted by google.
I did appeal. Google’s answer, word for word: “the app utilizes the accessibility API for performing autonomous actions… The accessibility API is not designed and cannot be requested for performing autonomous tasks.” So the problem was the agent acting on its own, not which AI services it uses.
The accessibility-tool exception is only for apps built mainly for people with disabilities. Buddy can help there, but it does a lot more, so claiming that wouldn’t be honest. If anyone has gotten an agent app like this through review, I’m open to trying again.
Excellent for local models running via OpenCode and helping with UAT on Android. Well done. What else is on your roadmap for this? Will you support Watch/TV/XR? F-Droid might take this.
Thanks, and thanks for the crosspost! Local models already work: in Settings you can add a custom OpenAI-compatible address (llama.cpp, Ollama or LM Studio on your network) and use it with OpenCode. It works best with models that are good at tool calling, since everything goes through phone tools.
Other devices: it’s built for phones, but someone already forked it to run on a Nintendo Switch with Android, with Joy-Con navigation: https://github.com/Lendss/buddy-switch. So other Android devices are clearly possible, I just haven’t tested watches, TVs or XR myself.
F-Droid: good timing. Buddy now builds all its native parts (proot, talloc and friends) from source in CI, with no prebuilt binaries in the repo, which was the main thing in the way. So an F-Droid build should be doable now. Until then, releases are on GitHub and the app updates itself.
Update: it’s now submitted to F-Droid, with reproducible builds, so the F-Droid and GitHub versions will be the same app signed with the same key, and you can switch between them: https://gitlab.com/fdroid/fdroiddata/-/merge_requests/56198
Nice to hear!
Please, ping me once you get the approval, I prefer F-Droid over Obtainium 🙂
Local models already work: in Settings you can add a custom OpenAI-compatible address (llama.cpp, Ollama or LM Studio on your network)
You should’ve mentioned this in the readme 🙂 I’ll check it out!
Done: 2.2.25 makes this easy. Ollama, LM Studio and llama.cpp are in the service list, plain http works on your own network and over Tailscale, and no key is needed. README has a section now: https://github.com/ghorbelhamdi/buddy-android#local-models
Honest results so far, on a PC with an RTX 2060 Super: Qwen3 8B and 4B handle simple one-step requests (battery, phone status), but multi-step tasks on the screen failed. The 4B tapped the wrong buttons in the calculator and then claimed it worked. So for real tasks you’ll want a bigger model; I’m trying gpt-oss 20B next.
It’s is expected, they’re small models. But something like Qwen 3.8 27B or Qwen 3.8 Flash Next should handle it.
How much VRAM do you have? If it’s a 12GB card and you also happens to have 64 GB of RAM, you should be able to run Qwen 3.8 Flash Next with Q3. It’s one of the most “hyped” local models right now (because it’s quite capable and has somewhat realistic HW requirements).
Update: you were right. Qwen3.8 27B (unsloth UD-IQ3_XXS) did the calculator task correctly: opened it, typed 1234×5678, read back 7,006,652. Qwen3.5 9B typed the wrong numbers and claimed success, like the smaller ones before it.
The catch is speed on an 8 GB card: 18 minutes for that one task, almost all of it reading the prompt and screen contents on the CPU (only ~1.7k tokens written). So it works, but not for daily use on this PC.
People are so mean these days, show some support
Name a single good reason that I should support this
It sounds, in theory, a lot easier to use compared to doing everything manually via TalkBack. There’s a very long running podcast called The Phones Show Chat and one of their regular guests is a vision-impaired man. Many of the AI features on phones that sighted people scoff at have been transformative for him, it’s very interesting. I’m not saying this would necessarily be great for him, but listening to his experience using his phone has taught me not to write-off new technologies just because I can’t find my own use case for them.
If the backdoor was intentional, you could see it as donating compute resources to the devs token maxing addiction
One: someone who can’t see the screen can just talk to it, with live voice, and it does the tapping and reads things back. For a lot of people, that’s the difference between using a phone app themselves and needing someone to do it for them.
Something wrong with the old voice access?
I’d be worried something as unpredictable as an LLM would put more disabled users in danger that it would help.
Everything related to AI gets down voted. Controversial at best.
Youre god damn right it does.
Curious why. Seems people vote against datacenters in their vicinity, against the flood of self-promote by AI agents in the remaining somewhat human corners of the internet, against the flood of dubious vibe-coded projects which for some reason all have the same description that starts with “I built …”
