Lab Architecture Guide
Lab Architecture Guide — Personal Site / Living Lab
This guide documents the complete architecture of Badluck’s Bane lab. It covers our philosophy, technology stack, services, infrastructure, and build order. Everything here is designed to be inspectable, forkable, and versioned by default.
Core Philosophy
Static content + git as source of truth, with small decoupled services for anything interactive.
- Nothing platform-managed
- Everything inspectable, forkable, versioned by default
- Independent components that can be replaced without affecting the whole system
Core Site (Writing, Lab Notebook, Versioned Ideas)
Stack: Hugo (Static Site Generator)
- Single Go binary — no runtime, no database
- Content = markdown files in a git repository
- Build time: milliseconds even for large sites
- Hand-rolled templates — given CSS/HTML background, custom templates fit better than fighting someone else’s theme
Math / LaTeX Rendering
Hugo now includes a built-in transform.ToMath function using an embedded KaTeX engine. This renders LaTeX to HTML/MathML server-side at build time, with zero JavaScript and zero client-side dependency.
Setup:
- Enable the Goldmark passthrough extension in Hugo config so raw markdown (including delimiters) survives to a render hook calling
transform.ToMath - Use the
htmlAndMathmloutput option for actual rendered glyphs rather than plain MathML — this needs one external KaTeX CSS link, but still no JS
Delimiter Recommendation:
- Use
\(... \)/\[...\]for inline/block math, not$...$ - Dollar-sign delimiters conflict with literal
$used in prose (a real issue when mixing physics notation with regular writing) - Using
$...$forces MathJax instead of KaTeX to avoid misfires - Sidestep the whole problem by never using
$...$
Alternative: If client-side rendering is ever needed, KaTeX is generally faster, while MathJax has historically broader LaTeX feature coverage — the same delimiter guidance applies either way.
Markdown-Adjacent Tooling
| Tool | Purpose | Notes |
|---|---|---|
| Chroma | Syntax highlighting | Hugo’s built-in highlighter, zero setup, covers Python/bash/C code blocks automatically |
| Goldmark | Markdown engine | Hugo’s engine, supports footnotes/citations natively — useful for physics writing with references |
| Pandoc | Document conversion | Worth having for converting existing LaTeX physics papers to markdown/Hugo content |
Diagrams-as-Text
| Tool | Use Case | Notes |
|---|---|---|
| Mermaid | Flowcharts, sequence diagrams, state machines | Written as plain text in markdown, rendered to SVG. Fits documenting Aurora’s architecture or media stack data flow without maintaining separate image files |
| Graphviz/DOT | Dependency graphs | Lower-level, more control; good fit for visualizing Aurora’s tool-registry dependency graph programmatically |
Content Structure & UX
- Admonitions/callouts — (note/warning/tip boxes) available via Hugo shortcodes; useful for flagging “this claim is unresolved” or “this is speculative” inline
- Auto-generated table of contents — Goldmark supports this natively per-page; worth enabling for longer physics writeups
- GFM tables — Goldmark handles GitHub-flavored markdown tables out of the box; relevant for data comparisons (SPARC results, prediction ledger snapshots)
Feeds & Syndication
- RSS/Atom — Hugo generates these automatically per-section, so notebook, physics, and Aurora content can each have separate feeds
- JSON Feed — Simpler modern alternative to RSS, near-zero extra effort since it templates from the same data Hugo already generates RSS from
Images
- Hugo’s built-in image processing — resize/crop/format-convert at build time via shortcodes, avoids needing a separate image pipeline
- WebP/AVIF output — Hugo can generate modern formats automatically from source images
Citations & Bibliography
- Pandoc-citeproc / BibTeX-driven citations — if physics papers accumulate real references, a
.bibfile plus Pandoc’s citation processor gives proper numbered citations and a generated bibliography
Recommended Content Structure
/content/notebook/ — raw dated lab-notebook entries (unpolished, timestamped)
/content/physics/ — Horizon Leak writeups, papers, data notes
/content/aurora/ — Aurora dev log
/content/philosophy/ — theology/philosophy pieces
Navigation principle: No hard category walls in nav — let cross-linking do the work.
Versioning as a Feature
- Git history is the “versioned ideas” mechanism — no custom tooling needed
- Consider exposing
git log/diffs for specific files publicly (e.g., a “history” link per post that shows past revisions) — trivial withgit log --followpiped into a page
Deployment
- Git push -> webhook or cron pulls on VPS ->
hugo build-> serve/publicvia nginx - No CI needed at this scale; a 10-line bash script is enough
Interactive Services (Decoupled from Core Site)
Each interactive service is its own small app, embedded via iframe or fetch — never part of the static build.
Aurora Live Dashboard
- Read-only view into memory growth, heartbeat/dream cycle activity, tool invocations
- Backend: Aurora’s existing SQLite → thin read API (Flask/FastAPI or Node/Express) → JSON → simple JS frontend
- Security: read-only, rate-limited, no write path exposed publicly
Prediction Ledger
- Schema: claim, timestamp, resolution status, resolution timestamp, outcome
- Backend: small CRUD API + SQLite or Postgres
- Frontend: sortable/filterable table, RSS feed optional for “just resolved” claims
Horizon Leak / Math Sandbox
Options ranked by effort:
- Lowest effort: Observable notebook (hosted externally, embedded via iframe) — parameter sliders, live-updating charts against SPARC data, zero backend needed
- More control: custom JS frontend (Plotly/D3) hitting a small Python backend running the existing
horizon_leak_test.pylogic as an API endpoint
Given the existing test script, wrapping it in a Flask endpoint that takes params and returns a fit result is probably a weekend, not a project.
Open Questions Board
- Can be as simple as a single markdown file in the repo, rendered as a page — no database needed
- Or a minimal CRUD app if others submit/claim questions directly (auth becomes a consideration; even a shared submission token/email gate keeps it simple)
Comments & Discussion
Don’t build this yourself — not the interesting part, easy to get wrong (spam/moderation). Decide per-content-type: notebook entries might not need comments at all; physics claims probably do.
Recommended Options (Ranked)
Giscus (top pick)
- Uses GitHub Discussions/Issues as the storage + auth backend
- Zero hosting/moderation burden
- Filters commenters toward people who already have GitHub accounts — exactly the audience for fork/build-on-it
- Natural pairing with the git-forward angle
- Note: Requires external GitHub accounts (external dependency, cannot be self-hosted) for login
- See: GitHub to Gitea Mapping for external dependency documentation
Remark42
- Go, single binary or Docker, no tracking, social login support
- Best fully-self-hosted option if you want comments independent of GitHub
- Actively maintained, low overhead, good VPS fit
Cusdis
- ~5kb gzipped, privacy-first, self-hosted on your own database
- Simplest to stand up if you want minimal footprint
Waline / Valine
- More built-in features (moderation UI, richer config)
Skip: Disqus (tracking/ads), Commento (heavier setup, paid tier), older/thinly-maintained options (Statique, e-comments).
Satellite Services (Own Subdomains, Decoupled from Main Site)
Grouped by purpose — add incrementally, not all at once.
Extends the Commons / Fork-and-Build Theme
- Gitea — self-hosted git server
- Makes “fork and build on it” literal rather than aspirational
- Browsable repo backend for the site content itself, plus any code/tools published (Aurora modules,
horizon_leak_test.py, etc.) - Lightweight, single binary, easy on a small VPS
- Gitea Actions (built-in CI) — free CI/CD once repos are hosted; someone forks Horizon Leak code, pushes a change, tests run automatically
- Built-in package/container registry — a place for Aurora tool modules or scripts to be pulled from directly, without depending on npm/PyPI
- Consider hosting site content itself as a Gitea repo (not just a separate deploy pipeline) so the “versioned ideas” concept is literally browsable by visitors
Useful Personally, Low Effort, On-Theme
- Miniflux or FreshRSS — self-hosted RSS reader for personal reading across physics/philosophy/self-hosting
- Optionally expose public feed/reading list as a subdomain — models transparency about inputs, not just outputs
- PrivateBin — self-hosted pastebin alternative. Trivial to run, useful for quick code/data snippets in discussions
Extends the “Watch a System Run” Theme
- Uptime Kuma — tiny-footprint status page/monitoring
- Could expose infra status (VPS, Aurora, etc.) publicly as its own subdomain
- Literal “watch the machine breathe” — on-theme and genuinely useful ops tooling
- Grafana (+ lightweight metrics source) — heavier lift, but the natural home for real time-series data if Aurora or the sandbox starts generating it
Bigger Commitment — Add Once There’s Real Demand
- Wiki.js or a plain git-backed wiki — good fit for the “open questions board” / shared body of thought once there’s an audience
- Matrix homeserver (Synapse, or lighter-weight Conduit) — optional real-time side-channel; not the main mechanism given preference for async/lower social cost
Skip for Now
- Full analytics suites
- Mailing list platforms
- Anything with its own separate user-auth system beyond what Gitea/Giscus already provide
Guiding filter: Any new service should either (a) directly serve the fork/build-on-it mechanic, or (b) be something you’d use yourself regardless of whether anyone else shows up. That keeps everything sustainable without becoming infrastructure-for-its-own-sake.
Infrastructure
- VPS: Cheap tier is plenty (Hugo output is static files; interactive services are lightweight)
- Web server: nginx — serves static Hugo output directly, reverse-proxies to the small backend services on other ports/subpaths
- TLS: Let’s Encrypt / certbot, standard
- Email: Only needed if you want notification-on-comment or ledger-resolution alerts — reuse existing email server experience, keep it minimal (transactional only)
- Backups: The git repo is the backup for content; DB-backed services (ledger, Aurora dashboard) need their own simple backup (cron + sqlite
.dumpor pg_dump to off-VPS storage)
Graphics, Media & Interactive Visualization
Principle: Add only where a visualization replaces an explanation with something manipulable (slider instead of paragraph, live render instead of screenshot). Where text already explains it clearly, skip — decoration competing with substance cuts against the “process over performance” theme.
Ready-Made, Near-Zero Code (Use These First)
| Tool | Best For | Notes |
|---|---|---|
| Streamlit | Python interactive UIs | Wraps a function into a full interactive UI (sliders, live-updating plots) in ~20 lines. Best fit for Horizon Leak sandbox: wrap existing horizon_leak_test.py logic directly |
| Gradio | Function → UI | Same category as Streamlit, even lower-code. Slightly less polished visually |
| Observable | Hosted notebooks | Large library of existing forkable D3 visualizations, including galaxy rotation curve / orbital mechanics notebooks. Embeds via iframe, no backend needed |
| Grafana | System monitoring | Ready-made answer for “watch the system run.” Point at Aurora’s SQLite to get live graphs/gauges for memory growth, dream cycle activity, heartbeat — configuration, not code |
| OBS Screen Recordings | Demonstrations | Zero new infrastructure. Aurora’s dream cycle firing, the SMB interceptor working, a live terminal session — cheapest and most honest way to “show the machinery” |
Custom Visualization (Higher Effort)
- Three.js scenes — e.g., the balloon/horizon-leak geometry, made manipulable. Bespoke 3D, though starting from an existing Three.js example/template beats a blank canvas
- Generative art tied to Aurora’s actual internal state — e.g., memory graph rendered as an evolving node network. Nobody builds this off-the-shelf since it’s inherently specific to system internals
- Sonification — e.g., Tone.js turning a data stream (ledger outcomes, heartbeat interval) into audio. Forkable examples exist, but wiring to specific data source is custom
Recommendation: Cover the physics sandbox and system dashboard with Streamlit/Gradio + Grafana, use screen recordings for anything “watch it run,” and deprioritize the fully-custom 3D/generative/sonification ideas unless/until there’s spare appetite. They’re the highest-effort, lowest-leverage items on the whole list.
Further Afield — Lesser-Known Options
Broader scope, not core recommendations — browse and pick what resonates rather than treating this as a to-do list.
Protocols / Philosophy Adjacent to the Project
- Webmention — W3C-standard protocol; sites notify each other when one links to another, so a visitor’s blog post about your work shows up as a linked response automatically. Decentralized commenting without a platform. Tools: webmention.io (hosted receiver) or go-jamming (self-hosted), plus a small send-script wired into the Hugo build
- ActivityPub — the Fediverse/Mastodon protocol. A lightweight endpoint on posts lets Mastodon users follow/boost/reply natively from their own feed. Tools: activitypub-express (Node.js library) or go-ap (Go implementation), plus a small endpoint that exposes post metadata in ActivityPub format. Implementation requires: (1) JSON-LD formatted post data, (2) ActivityPub actor endpoints, (3) inbox/outbox handling for incoming/outgoing activities.
- Digital garden framing — a whole subculture built around raw/evolving/non-linear notes (backlinks between notes, “seedling/budding/evergreen” maturity tags)
Lesser-Known Ready-Made Interactive/Data Tools
- Marimo — reactive Python notebook (cells re-run automatically on dependency change, spreadsheet-style), exports directly to a standalone interactive web app. Possibly a cleaner fit than Streamlit for the Horizon Leak sandbox
- Observable Framework — self-hostable version of Observable notebooks (not the hosted product) — fits the “own everything” instinct better
- Datasette — turns any SQLite file into a browsable, queryable web UI instantly. Strong fit for the prediction ledger or Aurora’s memory DB — point it at the file, get a full explorable interface with zero custom backend code
- Vega-Lite / Observable Plot — declarative charting, much less code than D3 for standard chart types
Lesser-Known Infra/Ops
- Ntfy — dead-simple self-hosted push notifications (curl a message, get a phone notification). Useful for “notify me when someone forks the repo” or “notify me when a prediction resolves”
- Changedetection.io — self-hosted webpage-change monitor; useful if any part of the work tracks external things changing
- SearXNG — self-hosted meta search engine; unusual but on-theme thing to expose on your own domain
- Fossil SCM — alternative to Gitea: bundles version control + wiki + bug tracker + forum into one single-file binary
Annotation
- Hypothes.is — open web annotation layer; people highlight/comment directly on posts or papers inline, without building anything. Different texture than end-of-post comments — closer to peer review in the margins, fits the “break this” invitation
Two worth prioritizing: Webmention/ActivityPub (fits “let it roam” at the protocol level) and Datasette (near-zero effort, directly serves the ledger/Aurora-data exposure goals).
Discoverability, Security & Polish
Lower-excitement but real — the stuff that bites later if skipped entirely.
Discoverability / Link Previews
- OpenGraph / Twitter Card meta tags — without these, a shared link renders as a bare URL instead of a title/image/description card. Hugo templates this trivially from front matter
- Sitemap.xml / robots.txt — Hugo generates a sitemap automatically; be deliberate with robots.txt (e.g., explicitly allow indexing of notebook/physics content)
- Structured data (schema.org) — marking physics posts as
ScholarlyArticle(or similar) can get them treated more seriously by search engines and academic crawlers
VPS Security Hygiene
- SSH key-only auth, fail2ban, ufw/firewall rules — basic hardening before anything public-facing goes live. With multiple services on one VPS, firewall each service’s port explicitly so a compromise doesn’t cascade
- Unattended security upgrades — cheap insurance, easy to forget once the box is stable
- Reverse proxy rate limiting (built into nginx) — protects interactive services (ledger, sandbox) from being hammered
Workflow
- Staging/preview step before publishing — even just
hugo serverlocally, or a second build on a subdomain, before pushing live. Cheap insurance against a broken build - Dark mode / print stylesheet — small UX touches; print stylesheet specifically useful for physics writeups people may want to save as PDF-via-print
Accessibility
- Basic semantic HTML + alt text on images/diagrams from the start — costs nothing now, expensive to retrofit across hundreds of posts later
Suggested Build Order
This phased approach ensures you have something public quickly while incrementally adding complexity.
- Core Hugo site live with a handful of notebook/physics/aurora/philosophy posts — get something public first
- Gitea up and content repo browsable — makes fork/build-on-it literal from day one
- Comments wired in (Giscus first — near-zero maintenance, filters toward the right audience)
- Prediction ledger (simplest custom interactive piece, clear schema, immediate content)
- Aurora read-only dashboard — Grafana pointed at Aurora’s SQLite (config, not custom code) rather than hand-built API + frontend
- Uptime Kuma / Miniflux (low-effort, useful regardless of external traffic — good to slot in whenever)
- Horizon Leak sandbox via Streamlit or Gradio wrapping
horizon_leak_test.py— highest-value, lowest-code interactive piece - Open-questions board, wiki, Matrix — add once there’s actually an audience to use them
Closing Notes
- Nothing here requires a framework, container orchestration, or a database bigger than SQLite until/unless traffic demands it
- Every piece is independently replaceable — if one experiment (e.g., the sandbox) turns out badly designed, it doesn’t take down the rest of the site
- This whole stack is deployable and maintainable via SSH from your phone if needed, consistent with how you already work
Historical Context
This architecture guide evolved from earlier thinking. For historical perspective:
- Early Architecture Notes - Portfolio + Comments Stack — Initial thinking about using Astro, Cactus Comments, and Caddy (before evolving to Hugo)
This guide is a living document. It will evolve as the lab evolves.
Early Architecture Notes - Portfolio + Comments Stack
Portfolio + Comments Stack
Historical Architecture Document — Early Thinking on Site Structure
Static site (Astro) + federated comments (Cactus Comments, Matrix-backed) + reverse proxy (Caddy).
This fits comfortably on a 1–2GB VPS because only Caddy runs persistently on the box, serving static files. Astro only runs at build time, on your own machine — nothing Node-based stays running on the server.
Note
This document represents early architectural thinking from the project’s formative stages. The actual implementation evolved to use Hugo instead of Astro, and the comments system was reconsidered. It’s preserved here for historical context and to show the evolution of the lab’s technical decisions.