Building a Low-tech Edge AI: Running AI on Pocket-Change Hardware

July 24, 2026 · Kilo Code

# Building a Low-tech Edge AI: Running AI on Pocket-Change Hardware

*By Isaac Bell’s R&D desk, 6 May 2026*

**Tier:** 🧠 Pro / ⚙️ Student-friendly
*Full R&D log. Students can follow the flash-and-go section to build their own edge node.*

## Why chase the cheapest edge node?

### Deployment script & image burn

A full bring-up is now packaged in `scripts/start_edge_stack.sh` (with a matching `stop_edge_stack.sh`). Both live in the repo so you can rsync the entire tree onto the SBC and launch everything with one command:

“`
cd /opt/zeroclaw-edge
scripts/start_edge_stack.sh
“`

Default expectations:
– llama.cpp model lives alongside the repo (`gemma-3-4b-it-Q4_K_M.gguf` by default);
– Signal identity sits under `~/.local/share/signal-cli/` (copy your registered account before first boot);
– ZeroClaw reads `zc-local/config/` (already configured for ports 58010/58011).
– llama.cpp auto-detects the correct format from the GGUF metadata.

Environment variables let you override paths and ports (e.g., `LLAMA_MODEL_PATH`, `SIGNAL_CLI_CONFIG`, `ZEROCLAW_BIN`). When you’re done, `scripts/stop_edge_stack.sh` tears everything down cleanly.

### Flash-and-go (USB drive workflow, no prior imaging experience required)

You can absolutely plug in a keyboard and monitor, but the process below works even if you’ve never “flashed” a single-board computer before.

#### 1. Download the operating system image
– **Raspberry Pi Zero 2 W**: grab “Raspberry Pi OS Lite (64-bit)” from raspberrypi.com
– **Orange Pi Zero 3**: use the vendor’s Debian/Ubuntu Lite image (Armbian works too)
– **Generic ARM SBC (Le Potato, Radxa Zero)**: pick the official minimal Debian image

Each download comes as a `.img` or `.xz` file.

#### 2. Write the image to a microSD card (your “disk”)
Two friendly tools, both free:
– **Balena Etcher** (macOS, Windows, Linux) — graphical utility
– **Raspberry Pi Imager** — equally simple, works for non-Pi boards too

Steps in Balena Etcher:
1. Insert the microSD (use a USB card reader if needed).
2. Open Etcher → “Flash from file” → pick the `.img`/`.xz` you downloaded.
3. “Select target” → choose your microSD card.
4. Click “Flash”. Wait until it finishes (1–5 minutes).

This is the only “flashing” step. No command line required.

#### 3. Enable headless access (optional but recommended)
If you don’t plan to connect a monitor:
– **Raspberry Pi OS**: after Etcher finishes, a new drive shows up (named “boot”). Place an empty file called `ssh` (no extension) on that drive. This turns on SSH so you can access the Pi over the network.
– **Orange Pi / others**: many images enable SSH by default. Check the release notes. If not, mount the second partition (rootfs), open `/etc/ssh/sshd_config`, and ensure `PermitRootLogin yes` (you can tighten it later).

#### 4. Copy the ZeroClaw repo onto the SD card before first boot
Still on your main computer, with the SD card mounted:
1. Create a folder on the SD card (e.g., `zeroclaw-edge/`). For Pi OS, use the root partition (it shows up as “boot” plus another volume). If your OS hides it, perform this copy after first boot instead.
2. Copy the entire `/Users/ike/Documents/agents/` folder into `zeroclaw-edge/`. On macOS Finder you can just drag and drop; on Linux you can `cp -a` or `rsync -a`.
3. Create a `README-FIRST.txt` on the card with the Wi-Fi SSID/password or IP configuration you intend to use.

> Tip: If your SBC uses eMMC or NVMe instead of SD, Etcher will flash those too. Works the same way.

#### 5. Insert SD into the SBC and power it up
– For the Pi Zero 2 W, use a good 5V/2.5A USB supply.
– For the Orange Pi Zero 3, a 5V/3A USB-C or barrel adapter works.
– Let it boot for a minute. Most images auto-expand the filesystem on first boot.

#### 6. Connect (SSH) and install dependencies
From your computer:
“`bash
ssh root@orangepi.local # or raspberrypi.local / the board’s IP
# Default credentials vary: Pi OS uses pi/raspberry, Orange Pi often root/orangepi
“`
Then run (adjust for your distro):
“`bash
sudo apt update
sudo apt install -y signal-cli python3 curl git
sudo install -m 755 /opt/zeroclaw-edge/bin/llama-server /usr/local/bin/
cargo install zeroclaw –root /usr/local # or copy a prebuilt binary
“`
If `cargo` isn’t installed, either `apt install cargo` (Pi OS) or copy a prebuilt `zeroclaw` binary into `/usr/local/bin/`.

#### 7. Restore Signal identity (if already registered)
Copy your saved `~/.local/share/signal-cli/` to the SBC (SCP, rsync, or plug the old SD). Example:
“`bash
scp -r signal-backup/ root@orangepi.local:/root/.local/share/signal-cli
“`

#### 8. Launch the stack
“`bash
cd /opt/zeroclaw-edge/scripts
./start_edge_stack.sh
“`
Watch logs with `tail -f ../logs/llama-server.log` (and the others). To stop:
“`bash
./stop_edge_stack.sh
“`

#### Imaging multiple boards quickly
1. Prepare one SD card end-to-end.
2. Use `dd` (Linux/macOS) or Win32DiskImager (Windows) to clone that card to an image file.
3. Flash that image onto additional cards using Etcher.
4. Update hostnames/keys post-boot to avoid conflicts.

Even easier: carry the repo and scripts on a USB drive. After flashing the stock OS to each new SD, boot the board, plug in the USB drive, and run:
“`bash
sudo rsync -a /media/usb/agents/ /opt/zeroclaw-edge/
“`
Then install packages and run the start script.

The key is repetition: once you’ve done it once, it’s five minutes per board, no coding required.

### Budget SBC tradeoffs (sub-$50 target)

**Takeaways (on a $530 bill of materials):**
– **Signal-only relay:** Pi Zero 2 W (or Radxa Zero) with Qwen inference handled elsewhere; power draw stays under 2 W.
– **All-in-one & cheapest workable LLM:** Orange Pi Zero 3 (4 GB) with GraalVM-native signal-cli, Qwen 0.8B CPU-only (~3 t/s). This is the reference platform for the deployment scripts.
– **Stretch goal under $50:** Pair the Orange Pi Zero 3 with a 40 W solar panel + buck regulator; still cheaper than chasing a Pi 4 on the grey market.

Higher-tier boards (Pi 4/5, Orange Pi 5) show up later only as context—basically, “what you might graduate to if the budget changes.” They’re not recommended buys here because they blow past the $40 ceiling.

Everything else in this repo—documentation, scripts, and casts—assumes the Orange Pi Zero 3 (4 GB) as the primary build target, with Pi Zero 2 W or Pi 3 B+ acting as fallback Signal relays when stock or shipping disrupts the plan.

All of these practical notes will stay in this draft as the article evolves, so future readers can reproduce the testbed without hunting for $200 boards.

### What a $30 bill of materials looks like

– Orange Pi Zero 3 (4 GB) mainboard: $28
– 32 GB microSD (Class 10) : $6
– 5V 3A USB-C supply or buck regulator: $5
– Peace of mind spare: $5 contingency

Total: ~$44 shipped (AliExpress baseline). Reuse an old phone charger and you’re hovering around $35.

Drop the board into a $5 off-the-shelf enclosure or 3D-print a shell. Add a $15 flexible solar panel and a USB power bank for a fully islanded setup (documented under “Stretch goal” above).

### Getting the scripts onto the board

1. Clone the repo locally (`git clone` or unzip the archive).
2. Throw it onto a USB flash drive.
3. On the SBC, mount the drive and do:
“`
sudo rsync -a /media/usb/agents/ /opt/zeroclaw-edge/
cd /opt/zeroclaw-edge/scripts
./start_edge_stack.sh
“`
4. Logs appear in `/opt/zeroclaw-edge/logs/`. PIDs live in `run/stack.pids`.

There’s no need to rebuild or reconfigure anything on-device, provided the binaries (signal-cli, llama-server, zeroclaw) are present under `/usr/local/bin/` or referenced via environment variables.

Training runs, GPU clouds, and petabyte datasets soak up most of the headlines, but the real frontier for practical autonomy is far more modest: a self-contained AI device that can live in a workshop, a server closet, or a rural shed with more dust than bandwidth. When energy is scarce and the upstream connection is unreliable (or intentionally absent), the usual assumptions about “just stream it from the cloud” collapse. That is where ZeroClaw—a compact Rust agent that talks over Signal and leans on llama.cpp—earns its keep. The question is not whether we can run it on a macOS workstation. The question is what happens when the budget drops below $50 and the power budget fits inside a USB charger.

To answer that, I re-ran the entire ZeroClaw + signal-cli + llama.cpp stack from scratch, but with two non-negotiable rules:

1. **No global footprint.** All configuration had to live inside the project folder (`zc-local/`). If the machine died, the backup should be nothing more than a tarball.
2. **No marketing hardware.** I only counted what I could afford to lose—or mail to a self-sufficiency hobbyist who still wonders why the solar calculator they bought in 1992 still works.

What follows is the field guide that shook out of those constraints. Every command listed here appears verbatim in the capture `captures/2026-05-06_success-run.cast`, and the supporting documentation (signal CLI setup, R&D log, hardware notes) lives alongside this draft.

## The software stack: Three simple pieces

Once you have a 3-watt board running, the software must be equally frugal. The ZeroClaw edge configuration resolves into three moving parts and a strict port map:

The success run boils down to the following shell playbook:

“`
pkill zeroclaw signal-cli llama-server 2>/dev/null

llama-server –model gemma-3-4b-it-Q4_K_M.gguf \
–port 58010 –host 127.0.0.1

signal-cli –config ~/.local/share/signal-cli daemon –http 127.0.0.1:58011 \
–no-receive-stdout –receive-mode on-start

zeroclaw –config-dir zc-local/config daemon

# Optional sanity ping
zeroclaw –config-dir zc-local/config agent -m “edge node online”
“`

Note: no `–chat-template` flag. Forcing a template (whether `chatml` or `gemma`) produced garbage output from Gemma 3 4B. Letting llama.cpp auto-detect the template from the GGUF metadata fixed it. The auto-detected format uses Gemma’s native `` tokens, not ChatML’s `<|im_start|>` wrapping.

Nothing here is theoretical. Replay the cast, and you will see the agent respond: *”Hello. I’m working.”* No Jinja stack traces, no SSE loops, and—most importantly—no writes to `~/.zeroclaw`.

## Why the project-local layout matters

The entire repo is now self-contained:

– `zc-local/config/config.toml` points the Signal channel at `http://127.0.0.1:58011` and llama.cpp at `http://127.0.0.1:58010`.
– `zc-local/config/workspace/` stores ZeroClaw’s memory, cron state, and audit log.
– `captures/README.md` maps every Asciinema session to its purpose (baseline failure, chatml experiment, final success).
– `signal-cli-setup.md` documents the new port, the zsh quoting gotcha (`?` must be quoted), and the exact bring-up steps.

That means recovery after a crash looks like this: rsync the repo onto fresh silicon, drop the model file in place, restore `~/.local/share/signal-cli/` (your Signal identity), and replay the four commands above. There is no scavenger hunt through `$HOME`.

## Hardware: how low can we go?

The companion note `ultra-budget-zeroclaw-hardware.md` digs into every board I could lay hands on or justify buying. Highlights:

| Board | RAM | Qwen 0.8B | Gemma 3 4B | Verdict |
|——-|—–|———–|———–|———|
| Pi Zero 2 W | 512 MB | Swap hell (~1 t/s) | Impossible | Viable only as Signal + agent stub |
| Pi 3 B+ | 1 GB | 2–4 t/s | Swap bound | Cheapest “real Pi” for 0.8B |
| Pi 4 (4 GB) | 4 GB | 3–5 t/s | 1.5–2.5 t/s | Minimum budget for Gemma 3 4B |
| Pi 5 (8 GB) | 8 GB | 12–18 t/s | 8–11 t/s | Comfortable interactive use |
| Orange Pi Zero 3 (4 GB) | 4 GB | 3–4 t/s | 1–2 t/s | $46 all-in, slow but workable |
| Orange Pi 5 (8 GB) | 8 GB | 12–15 t/s | 5–7 t/s | Cheaper Pi 5 alternative |

A few conclusions emerged:

1. **RAM, not CPU, draws the boundary.** The Gemma 3 4B model simply does not fit in under 3 GB, full stop. No amount of swap or SD cards will change that—it is slower than waiting for the mail.
2. **GraalVM native saves the day.** Dropping the Java runtime trims 200–300 MB, enough headroom to keep signal-cli stable on a 4 GB board.
3. **Speed follows thermal design.** A Pi 5 without a heat sink throttles to 1 GHz within a minute; with a fan, it sustains 8+ tokens per second on Gemma 3 4B.
4. **Power is part of the budget.** The Orange Pi Zero 3 pulls 2–3 W under load. A Pi 5 with active cooling skates past 10 W. Design your solar panels accordingly.

## Deployment checklist for DIY and interns

1. **Register & verify Signal once.** Use a phone number you control, then back up `~/.local/share/signal-cli/`. The identity is portable; tar it up and move it to the SBC when the time comes.
2. **Pre-load models.** Gemma 3 4B (~2.5 GB GGUF) sits alongside the repo. If storage is tight on a small SD card, symlink to `/mnt/nvme` or an external USB drive. For the tightest boards, Qwen 0.8B (~589 MB) is the fallback.
3. **Copy the repo.** Move `zc-local/`, `captures/`, and `scripts/` as a unit (USB thumb drive works great for multiple SBCs).
4. **Install dependencies.** On ARM64 SBCs, prefer the GraalVM-native signal-cli build; drop cross-compiled `llama-server` binaries into `/usr/local/bin/`.
5. **Run `scripts/start_edge_stack.sh`.** The script sequences startup (llama-server → signal-cli → health-check poll → ZeroClaw), records PIDs, and logs everything under `logs/`. No manual `pkill` needed.
6. **To stop:** `scripts/stop_edge_stack.sh` does graceful shutdown with PID tracking, then sweeps for any orphaned processes that outlasted the kill signal.
7. **Document the run.** Capture a cast on the lab bench; in the field, log boot time, battery draw, and any Signal send/receive latencies.

That’s the minimal process that survived handoffs and simulated outages without confusing anyone—interns and hobbyists included. The same scripts double as a field checklist when you’re at a solar-powered workbench miles from mains power.

## Low-tech inspiration: slowing down to stay online

Kris De Decker’s “How to Build a Low-tech Internet” reminds us that every bit has a cost. [^1] Efficiency alone does not save energy; it simply widens the pipe. The same logic applies here. By keeping the agent stack small, well documented, and air-gapped by choice, we cap its appetite. A single-board computer pulling 3–10 watts can host an assistant that parses local documents, responds over Signal, and keeps running when the backhaul drops. That is a different vision of AI: not a cloud that hunts for users, but a toolbench that minds its own circuits.

[^1]: Kris De Decker, *How to Build a Low-tech Internet?*, Low-tech Magazine, 2015.

## Next steps (v0.5 milestone)

### What the development machine validated

The core loop works: a Signal message arrives at the edge node, Gemma 3 4B generates a response locally, and signal-cli sends it back. No cloud relay, no Docker, no SMS fallback—three local processes on three ports, started and stopped by the scripts in this repo. That loop is the hinge the entire project turns on, and it held across repeated runs.

### Pitfalls encountered (so you can avoid them)

These errors are fixed in my code. They are not issues with ZeroClaw itself; rather, they are side effects from combining the integrations we used.

1. **Signal SSE race condition.** ZeroClaw connects to signal-cli’s SSE endpoint on startup, but signal-cli’s HTTP server can take a second or two to bind its port. If ZeroClaw connects first, it gets a connection refused and retries in a loop. The fix: `start_edge_stack.sh` now polls `http://127.0.0.1:58011/api/v1/check` before launching ZeroClaw. If you write your own start script, include that health-check poll.

2. **Chat template auto-detection.** Forcing `–chat-template` (whether `gemma` for Gemma models or `chatml` for Qwen-style models) produced garbage output from Gemma 3 4B. The solution was simpler than it looked: omit the flag entirely and let llama.cpp read the template from the GGUF metadata. The auto-detected format uses the model’s native tokens. If your responses come out as endless repetition, remove the `–chat-template` flag first.

3. **Config corruption from automated tooling.** During development, an AI coding assistant deleted the ZeroClaw provider configuration, causing a silent fallback to cloud APIs. The symptom was that inference kept working—but it wasn’t local. Lesson: keep a known-good tarball of `zc-local/config/` somewhere outside the repo. Before any long test run, diff the live config against your backup.

4. **signal-cli database lock after unclean shutdown.** If the board loses power or signal-cli is killed with SIGKILL, it can leave a lock file in the config database. The next launch fails until the lock is cleared. The `stop_edge_stack.sh` script now handles this with PID tracking and a sweep, but if you ever bring the stack up manually and see a database-locked error, check for stray `.db.lock` files in `~/.local/share/signal-cli/`.

### What still needs doing

The pieces are in place, but the proof is in the field. Here is what remains:

– **SBC field trial.** The entire stack has been tested on a development laptop, not on the target budget board. The next concrete step is to rsync the repo onto an Orange Pi Zero 3 (or equivalent 4 GB ARM64 SBC), restore a Signal identity, run `start_edge_stack.sh`, and measure token throughput under real thermal and power constraints. The 1–2 t/s estimate for Gemma 3 4B on the Orange Pi Zero 3 is a projection from datasheets, not a measurement.

– **Signal identity migration testing.** The documented procedure (tar `~/.local/share/signal-cli/` → copy to SBC) has not been run end-to-end from one architecture to another. Signal’s registration keys may or may not be portable between an x86 development machine and an ARM64 board. This needs a dedicated test with a spare phone number.

– **Long-running stability.** No 24-hour soak test has been performed. Unknowns include: memory pressure over time under repeated inference, signal-cli’s SSE connection stability across network dropouts, and whether the stack recovers cleanly after the Wi-Fi or cellular backhaul goes away and returns. A multi-day run with periodic logging and a kill-switch timer on the power supply would answer most of these.

– **Real power measurements.** Every watt figure in this article is either a vendor claim or a back-of-the-envelope estimate. A proper field trial should include a USB watt-meter or inline multimeter, logging draw at idle, during inference, and during Signal message processing. These numbers determine the solar panel and battery sizing for an off-grid deployment.

– **Article polish.** The capture files are in the repo, but the finished article needs photos of the hardware setup, as-built cost tables with shipping, and the actual watt-meter readings from the field trial.

### Honest assessment

A budget single-board computer, a local model, and a phone number you control. The three parts talk to each other. The core loop holds on a development machine. Whether it holds on a 4 GB board pulling 3 watts, in a place where the network comes and goes, is the question the field trial will answer. The pieces are ready to test. Worth building.