- cross-posted to:
- android@lemdro.id
- cross-posted to:
- android@lemdro.id
cross-posted from: https://lemdro.id/post/48219138
Buddy: an open source app that lets AI agents (Claude Code, Codex, OpenCode) use your Android phone
I’ve been building Buddy, a free, open source (Apache 2.0) Android app. You type or say what you want in a floating bubble, and an AI agent reads the screen, taps and types through your apps to do it.
- Your choice of agent: Claude Code, Codex, OpenCode, Antigravity or Grok, running inside the app (Alpine Linux via proot) on your own plan. OpenCode’s free models need no account.
- No root, adb or PC: it reads the screen and taps through the accessibility service.
- Asks first: before anything is sent, posted, bought or deleted. It pauses when you touch the screen and never types passwords or payment details.
- Builds apps: with Linux tools on, it builds real APKs on the phone. This week I drew a maze on a paper towel, took a photo, and it made it a tilt game. The first build crashed on launch; it found the bug and fixed it.
To be upfront:
- What the agent reads on your screen goes to the AI service you pick.
- It’s early (2.2), tested most on a OnePlus 12 with Android 16.
- It’s not on Play (Google rejected the accessibility use), so it’s an APK from GitHub releases, built by GitHub Actions from the source.
Source and releases: github.com/ghorbelhamdi/buddy-android
Happy to answer questions here.
You must log in or register to comment.
Yeah, I’d recommend to be careful with apps like this. There’s a lot of sensitive information on smartphones, including all your life of the past years, personal information of your friends, maybe nudes, passwords to everything…
If the model is local and since the APP is FOSS, it’s probably fine.
and since the APP is FOSS, it’s probably fine
This (blind) trust is the attack vector, and exploits and attacks have been increasing in this vector
A bad command can mess up a lot of things. See people deleting their prod database because they gave the agent access. Local models are also more likely to make such mistakes usually
If you have the right deterministic gates and checkpoints this becomes impossible. Only really an issue if you use the model raw.
I think the news reporting of the last year or so paints a very different picture. We had OpenAI have their hacking agents escape their confines, it happened to their competitors, we have lots of ludicrous things going on since OpenClaw was invented… I think this is more of a theoretical/hypothetical argument… If you had deterministic and watertight gates, if they were configured in a very annoying way and if there weren’t any bugs in any part of the harness and the setup. Nor any bugs or unforseen design choices in Android, the reachable apps and websites. Then you’d be safe. But such a thing doesn’t exist in the real world. And you kind of don’t want to babysit and micromanage your agents anyway, so the user isn’t even motivated to set it up anywhere close to this.
I get your point, and well taken. Sure bugs, missed edge cases, zero days, regressions, incomplete test suites, etc continue to exist. But just like with anything, if you reduce your attack surface or other precautionary measures you can reduce your risk to near zero or at least acceptable enough to decide when and where it is appropriate to use various models, go to market, or what have you
Well hypothetically you could make an effort. And you should. That was my point. “Be careful.” In practice I’d invite people to follow up on that. I mean I’m currently looking at an app with a fairly simplistic heuristic to decide what’s okay and what’s not okay to do. It checks the labels of buttons to determine if the agent is publishing information to the internet… But simultaneously I’m also looking at the button below the comment box I’m currently typing in… And I can’t help but notice it’s labeled “Kommentar” and that word is not in the regex. So the actually existing “safety measure”(?) already fails here on the first opportunity presented and doesn’t provide any security…
Which software has deterministic trustworthy gates?
I’ve tried looking into that and was disappointed.
This is !localllama@sh.itjust.works isn’t it? You own whatever you want them to be, you just have to make them yourself.
Even with frontier model harnesses these days though, you can restrict certain commands, operations, directories, etc to deterministically require a hitl.
Oh yeah! But I think something like Qwen 3.8 27B should be able to handle it 🤔
Curious too. In my tests Qwen3 4B and 8B handled one-step requests but fumbled multi-step ones (tapped the wrong buttons, then said it worked). gpt-oss 20B did better but was inconsistent: one run got a 2-step task right, the next claimed a tool didn’t exist. I haven’t tried a 27B yet, and I don’t know if it’ll even run usably on my RTX 2060 Super (8 GB). If anyone runs one through Ollama or llama.cpp, I’d love to hear how it does.
Yes. They delete prod databases, write blogs and penny dreadful and doxx their owners on Moltbook. Chat in underground message boards. Circumvent safety measures because that’s a good way to fulfill a goal. I had my coding agent delete the test suite because it kept failing and it needed “to focus”… The internet is filled with funny anecdotes like that. And local models can do all of that. They’re pretty “clever” these days.
- What the agent reads on your screen goes to the AI service you pick.
Not especially fine.
We’re in !ocalllama@sh.itjust.works. So if this goes to the service you own (runs on your PC, for example) - I think it’s fine. The author confirmed the local models are supported.
Ahh, “local” meaning “not on the phone in your hand, but still potentially on your own hardware” - that’s my bad, thanks for clarifying
- Asks first: before anything is sent, posted, bought or deleted. It pauses when you touch the screen and never types passwords or payment details.
Lol, sure it does. How many times have wee seen this claim, now?
Fair to be skeptical, but it isn’t the model being asked nicely. Buddy checks every tap itself: when the button is Send, Post, Buy, Pay, Delete and similar (by its label or id, in several languages), Buddy stops and shows an approval card, whatever the model wants. It’s a word list, so an oddly named button can slip through; it’s all in Guard.java if you want to look: https://github.com/ghorbelhamdi/buddy-android/blob/main/app/src/main/java/app/buddy/assistant/Guard.java
When I see word matching like this, in the context of AI, I still fear it’s embedded in text prompt interfaces, meaning it can be evaded. (
D E L E T Einstead ofdeleteetc)It also doesn’t tell me much about the wider risks and gaps of the connected system architecture - which, to be fair, I haven’t inspected or analyzed, so I don’t know one way or the other. One guard may be better than none, but doesn’t define or fully describe overall/system risk.
Naturally, I’m more risk-averse and skeptical than many other people. I guess the original commenter is, too. :)
It doesn’t match the model’s text. The check runs in the accessibility service on the actual button the tap would land on: its label, icon description and view id, as the app reports them. The model can write D E L E T E all it wants, the button still says Delete. It also can’t tap Buddy’s own approval card to approve itself.
Where it can miss: a button with an odd or blank label, and apps you mark as trusted, where asking is left to the agent. And agreed, it’s one guard, not the whole risk picture. Prompt injection from what’s on screen is the bigger open problem, and the approval card is the backstop there.
