Project case study · Full-stack development
Axion – Mobile-Driven Desktop Automation
A three-part automation system that lets a user drive and automate their desktop from their phone: a mobile-first Next.js web app, a MERN cloud relay, and a Python desktop agent, with live step-by-step visibility and mid-run steering.
What it is
Axion lets someone control and automate their desktop from their phone. It is built from three pieces: a mobile-first Next.js web application, a MERN cloud relay, and a Python desktop application.
The user types a prompt or speaks into the web app; that becomes a task which travels through the relay to the desktop agent, which carries it out. The services run in Docker containers.
Problem and approach
Problem
Desktop automation normally has to be set up and watched at the desktop itself, which is exactly where the user is not when they want it to run.
Solution
Split the system so intent is captured on the phone, transport is handled by a cloud relay, and execution stays on the desktop — with each step streamed back so the user can follow along and intervene.
How it works
Task creation on mobile
The user types a prompt or uses voice input in the mobile-first web application, which creates a task.
Cloud relay
The task is pushed to a MERN cloud relay that sits between the phone and the desktop.
Desktop execution
The Python desktop application picks the task up from the relay and executes it on the machine.
Watch and steer
Every step of the run is visible from the phone, and the user can steer the task midway instead of only seeing the final result.
How I built it
- 1.
Mobile-first web application
Built the Next.js web application around prompt and voice entry on a phone, and around following a run in progress.
- 2.
Cloud relay
Built the cloud relay on a MERN stack to carry tasks from the web application to the desktop agent.
- 3.
Python desktop agent
Built the desktop application that receives tasks and automates actions on the machine.
Challenges and decisions
An automated run that the user cannot see or correct is difficult to trust.
Approach: Streamed each step back to the phone and allowed the user to steer the task while it is still running.
Lesson: Visibility and the ability to intervene matter as much as the automation itself.
Skills and technologies
What I learned
- How to design a system that spans a phone, a relay, and a desktop agent without any of them being able to assume the others are reachable.
- That keeping a human able to intervene mid-run is a design requirement for automation, not an extra.