Joy

What Joy is

Joy is one agent. Not a swarm, not a framework, one named assistant with a memory, a task board, a phone number's worth of inputs, and standing permission to do real work while I'm asleep. It knows my context, remembers what matters, knows when to bother me and when to handle it silently, and can run a multi-step job overnight without anyone watching.

Six other people and six clients have their own Hermes agents on infrastructure I built. Those are separate agents with separate memories. Joy is mine.

The architecture

Hermes is the agent loop, scheduler, and tool layer. It runs in a Compose stack of about ten services next to the databases, the browser, and the vault.

The model router is the part I'm proudest of. Every job Joy takes gets classified first: triage, extraction, judgment, code, long-context reading. A small model I fine-tuned makes that call. It's Qwen3 8B in a 4-bit quantization, trained with Unsloth on a dataset that is part GPT-5.5-generated, part written by hand from real routing decisions I'd already made. It routes into three tiers: a fast model for routing, extraction, tagging, and voice commands; a general model for most overnight work and subagents; a deep model for direct questions, orchestration, synthesis, and consequential decisions. It also picks the reasoning effort and whether to use the provider's priority tier. An OpenAI-compatible gateway carries the call with prompt caching on the shared prefixes. Every routing decision is logged with the model, the effort, and any fallback.

Plane is the task board. I self-host it, and it is the contract between Joy and me. I own outcomes, decisions, approvals, and anything that needs my hands. Joy owns research, imports, reminders, automations, maintenance, bounded implementation, and follow-ups. Assignments name the owner. Comments carry findings and handoffs. Every task carries its evidence: source pointers, tool output, test results, live readbacks, job IDs, artifact hashes. The board held 356 work items at the last snapshot. Discord is for urgent nudges; Plane is the record. It's how I know what Joy did last night without reading a log.

GBrain is memory: PostgreSQL with pgvector, separated into compiled knowledge, source chunks, graph edges, version history, and a query cache. It was seeded with bulk ingestions of years of daily notes, meeting transcripts, message history, people records, documents, and books, and it grows every night. At last read: 1,393 compiled pages and about 6,500 chunks. Everything it holds is cited to a source with a time boundary, so Joy can say where it learned something and when it stopped being true.

Integrations reach Discord, Google Calendar and Gmail, Notion, Home Assistant voice and devices, the media stack, the arcade cabinet, GitHub, Reddit, Spotify, Oura, SimpleFIN, ElevenLabs for voice, and the internet. The scheduler runs 37 jobs, 18 of them enabled, some every minute. A persistent browser with an authenticated profile handles the sites that don't have APIs. Secrets live in a self-hosted vault, never in prompts.

The overnight pipeline

This is where the design earns its keep. It stages source collection, context analysis, explicit and proactive work extraction, dispatch, validation, recovery, knowledge updates, report compilation, publication, and a checkup. Each stage writes an atomic artifact. Dependency-aware resumption means a failed stage restarts from its last good input, not from scratch. Leases stop two runs from colliding.

The planner decides what to do. Workers do it with bounded tool access. A separate verifier decides whether it was actually done, and it wants evidence. A summary that says "done" doesn't count.

What Joy does

• Reads my calendar, inbox, and messages overnight and turns them into a morning report with the decisions I actually need to make.
• Runs the household: Home Assistant automations, the media library, the arcade cabinet, and the server that hosts all of it.
• Takes coding tasks off the Plane board, runs them through Cyrus, and comes back with a diff and a test report.
• Watches for new books and releases for the household and files what it finds.
• Keeps GBrain current: what was learned, what was saved, and what changed.
• Prepares outreach, purchases, and anything irreversible, then waits. Those still require me.

My role

I designed it, I run it, and I am the product owner who rejects stages that lose information. When a context-compression step started dropping proactive-work extraction, it got cut. When GBrain updates only said "ingestion succeeded", I made them say what was learned. A lot of the implementation was done by agents inside this same system, working from my specs and my reviews. I think that's the honest way to describe how software gets built now.

Hosting it for other people

Six people run their own Hermes agents on my hardware, each in its own Docker stack with its own workspace, browser containers, and credentials. Six clients run agents on their own VPS with a preconfigured stack deployed through Portainer. The two architectures are different on purpose.

What I'd do differently

I'd have separated the browser's CDP lifecycle from the container lifecycle earlier. It's the one service that needs resynchronization after recreation, and it was the last thing I made boring.