Phase 9: The Part I Kept Putting Off

From day one of this rewrite, Phase 9 was the phase I refused to touch. AI integration, a background job pipeline, a Telegram bot, live-collaborative websockets. Every earlier post stopped short of them on purpose. Not because they were optional, but because each one involves concurrency and external systems that deserve their own attention, not a drive-by port while I was still figuring out how to write a Go HTTP handler.

Today I finished it. One session. Five Python modules, full tests, deployed live. The Go rewrite now has full feature parity with the original Atlas Docs backend.

What landed

Phase 9 is really five things that happen to live in the same release:

1. `ai.py` → a thin HTTP client to a local "Grok bridge" service that wraps a CLI OAuth session. Nothing fancy. Call out, get structured text back.
2. `knowledge.py` → a research pipeline. Send a link or a thought to Grok, get a structured "knowledge card" document back, save it as a draft for review.
3. `telegram.py` → a Telegram bot. Long-polls for messages, turns them into knowledge-pipeline jobs, and can approve, publish, or retry drafts via chat commands.
4. `collab.py` → a websocket relay for live collaborative editing. It's based on the Yjs protocol, but the server doesn't understand Yjs at all. It just relays binary messages between browser tabs editing the same document.
5. `uploads.py` → image upload and serving, with a content-type allowlist and a size cap.

One genuinely new third-party dependency came along for the ride: gorilla/websocket, the first new dependency since early in the project. (google/uuid also shows up in this phase's code, but it had already been sitting in the dependency tree as an indirect dependency since Phase 4, pulled in transitively; Phase 9 is just the first time the app imports it directly.) A new knowledge_jobs SQLite table went in via a migration applied live against the already-running database, with a brief container restart and no data loss. Then the whole thing shipped to the Go subdomain and got verified over real HTTPS.

Import cycles are a hard wall in Go

This was the best language lesson of the phase.

The knowledge pipeline needs to notify the Telegram bot when a job finishes ("your draft is ready"). The Telegram bot needs to call back into the knowledge pipeline to start a retry. In Python, this was solved with knowledge.py importing telegram.py at the top of the file, and telegram.py importing knowledge.py back via a lazy import inside a function body. Python only checks for circular imports at the moment a module actually gets loaded, so a lazy import inside a function dodges the check. By the time that function runs, both modules have already finished loading.

Go does not have this trick. The package import graph is checked once, at compile time. There is no "lazy" import. Two packages simply cannot import each other, full stop, whether or not that import would ever actually run at runtime.

The fix was a callback field. The knowledge service got a public Notify func(job) field, left as a no-op by default. Then in main(), after both services already exist:

knowledge.Notify = telegramBot.NotifyJob

One line. Both sides only depend on a function signature. Neither depends on the other's package. It's a small pattern, but it's a very clean example of the language's constraints forcing a specific design, not just a style preference. Python let me paper over the cycle. Go made me face it and invent the seam.

Two bugs tests caught that reading never would

Bug 1: collaborative editing websockets died after exactly 30 seconds.

Cause: a request-timeout middleware, added back in an earlier phase for normal HTTP endpoints so a slow request can't hang forever, had been applied globally to every route, including ones added much later. A websocket meant to stay open for an entire editing session obviously shouldn't have a 30-second timeout. Reading the websocket handler's own code would never surface this, because the handler never touches anything timeout-related. It only showed up by noticing that a global middleware application from an earlier phase silently covered a route that didn't exist yet when that middleware was written. Fixed by scoping the timeout middleware to a sub-group of routes that excludes the websocket endpoint.

Bug 2: unauthorized websocket connections were upgraded first, rejected second.

The first Go attempt upgraded the HTTP connection to a websocket and then sent a "you're not authorized" close message over the now-upgraded connection. The original Python version rejects at the plain HTTP level, before any upgrade happens. The Go version got the order backwards.

This one only got caught by opening a raw TCP socket by hand with a Python script and reading the literal HTTP response bytes off the wire. It said 101 Switching Protocols (a successful upgrade) when it should have said 401 Unauthorized and never upgraded at all. A test written against a normal websocket client library would likely not have caught this, because from a client library's point of view both cases just look like "the connection closed." The distinction only shows up at the raw byte level. Fixed by checking authorization before ever attempting the upgrade.

The rest of the test suite is the same philosophy I've been using all project: real temporary SQLite databases (not mocks), fake HTTP servers standing in for the Grok bridge and Telegram's API, and genuinely real websocket connections dialed against a test server. Including proving that a 20-connection room-capacity limit actually rejects the 21st connection with the right close code.

Built, tested, and deliberately turned off

The Grok bridge integration and the Telegram bot both got fully built and fully tested. Then they got deployed disabled, with empty API tokens, on the live Go version.

That's not unfinished work. It's intentional. This Go rewrite runs side-by-side with the original Python production app. The Python app is already the one actively polling the real Telegram bot's messages. If the Go version also polled the same bot token, both processes would race to fetch and respond to the same incoming messages, and the bot's actual owner would start seeing duplicated or interleaved replies on a live bot they use every day.

So the integration exists. The tests pass. Flipping it on for real is a deliberate future opt-in, like pointing it at a second, separate test bot token. Not something you enable by default just because the code is done.

Full parity

That's it. Nine phases. Skeleton, models, SQLite/sqlc, auth, all 16 core endpoints, CORS and config, Docker deploy, benchmarks, and now AI, background jobs, a Telegram bot, collaborative websockets, and uploads. The Go rewrite has full feature parity with the original Python backend.

The part I kept putting off was also the part that taught me the most about how Go wants you to structure concurrent systems. Import cycles, middleware scope, and the order of operations on a websocket handshake are not glamorous, but they are exactly the kind of thing that only shows up when you stop deferring and actually build it.