vault/Technical/Coding/zainezq upgrade.md

# • Technical Onboarding Report

  Project Summary

  This repo is the source for https://zainezq.com, described in README.md:3 as a personal/familial hub. It is primarily a static site generated from Org-mode files using
  Emacs org-publish, with supporting Emacs Lisp, CSS, browser JavaScript, Python search-index generation, and Gitea Actions deployment.

  The main entry point is build-site.el:1. The documented local build command is:

  emacs -Q --script build-site.el

  That is also wrapped by make build in Makefile:13.

  Architecture

  The architecture is a static publishing pipeline:

  1. Source content lives mostly in .org files under blogs/, posts/, home/, books/, play/, and tags/.
  2. build-site.el:555 defines multiple org-publish projects.
  3. Local Lisp modules in lisp/ provide sitemap, tag, sidenote, and recently-updated behavior.
  4. Assets from assets/ are copied into output/assets/.
  5. search-index-json.py:57 scans generated HTML in output/ and writes output/test.json.
  6. Browser search uses lunr.js plus assets/scripts/search.js:1.
  7. CI runs the build through Gitea Actions in .gitea/workflows/build.yml:47.

  The site has several generated artifacts: output/, .org-timestamps/, tag pages, sitemap pages, recently-updated.org, assets/content/comment-pages.json, and output/
  test.json.

  Repo Structure

  Observed top-level areas:

  - blogs/: dated blog content, arranged by year and month, for example blogs/2026/05-may/....
  - posts/: longer-form posts, currently mostly under posts/career/.
  - home/: homepage-adjacent pages and guides.
  - play/: interactive/playful pages with dedicated JavaScript and CSS.
  - books/: book notes.
  - lima/: Markdown/Org content handled by a special publisher.
  - assets/: static CSS, JS, images, icons, fonts, sketchnotes, and content JSON.
  - lisp/: custom Emacs Lisp build helpers.
  - tags/: generated or semi-generated tag pages.
  - output/: generated site output; ignored by Git.
  - .gitea/workflows/: CI workflow.
  - build-site-tests.el: ERT tests for some build helpers.

  I counted roughly 165 .org files, 16 browser JS files, 11 CSS files, 6 Emacs Lisp files, and 5 Markdown files outside generated/vendor directories.

  Build And Publishing

  build-site.el:24 establishes z/site-root as the canonical root and build-site.el:41 sets z/output-root, overrideable with SITE_OUTPUT_DIR.

  The publishing projects are explicit in build-site.el:555:

  - org-main: root .org content and main sitemap.
  - org-assets: static assets.
  - org-categories-sitemap: category page generation from home/.
  - org-posts: posts.
  - org-blogs: blogs, with grouped sitemap.
  - org-books: book notes.
  - org-career: career subsection sitemap.
  - wip-pages: work-in-progress listing.
  - org-tags: tag pages.
  - org-lima: Markdown/Org Lima content.

  The build includes cache safety logic. build-site.el:713 documents that output/ and .org-timestamps/ must be wiped together. build-site.el:738 checks output integrity and
  forces republish if key outputs are missing.

  make currently runs set-perms clean-output build search clean-venv via Makefile:11. This depends on an absolute external permissions script at Makefile:7, which makes a
  fresh developer setup less portable.

  Dependencies

  Clear dependencies from the repo:

  - Emacs with Org export support.
  - Emacs package htmlize, bootstrapped from MELPA/ELPA in build-site.el:116.
  - Python packages: beautifulsoup4, lxml, legacy-cgi in requirements.txt:1.
  - Browser libraries vendored under assets/scripts/, including lunr.js, mermaid.min.js, svg-pan-zoom.min.js, and bigger-picture.min.js.
  - pandoc is required for Lima Markdown publishing in build-site.el:512.
  - inotifywait is required by watcher.sh:13.
  - PowerShell is required by the Gitea build monitor in .gitea/workflows/build.yml:60.
  - A separate authoring service is referenced at /home/zaine/master-folder/org-platform/authoring-service in README.md:9 and Makefile:9.

  Missing: there is no lockfile for Python, no pinned Emacs package version except the currently present .packages cache, and no documented system package install list.

  Data Flow

  Content metadata drives behavior. The repo uses Org keywords such as #+TITLE, #+DATE, #+FILETAGS, #+COMMENTS, #+SLUG, #+HIDDEN_PAGE, #+WIP, and #+NO_SIDENOTES.

  Examples:

  - Comments are enabled when #+COMMENTS: t is present, handled by build-site.el:309.
  - Comment slugs come from #+SLUG, falling back to filename in build-site.el:320.
  - Hidden pages are omitted from sitemaps using #+HIDDEN_PAGE: t in lisp/sitemaps.el:29.
  - #+NO_SIDENOTES and some tags affect body classes through lisp/sidenotes.el:11.
  - Tag pages are generated from FILETAGS/TAGS in lisp/tags.el:39.
  - Recently updated content uses #+DATE or file modification time in lisp/recently-updated.el:47.

  Client-side comments call /api/comments/... in assets/scripts/comments.js:92. I could not determine the comments backend from this repo; only the frontend and generated
  comment-page manifest are present.

  Coding Standards And Conventions

  Clear conventions:

  - Emacs Lisp functions use a z/ prefix.
  - Build script sections are heavily commented and organized.
  - Org metadata is the main content contract.
  - Generated pages are regular Org files in some cases, for example tag pages and recently-updated.org.
  - Browser JS is plain JavaScript, no bundler, no module system.
  - CSS is split by concern: style.css, comments.css, kanban.css, play.css, wird-tracker.css, etc.
  - Generated output and local build caches are ignored in .gitignore:1.

  Missing or unclear:

  - No formatter or linter config for Emacs Lisp, Python, JS, CSS, or Org.
  - No documented code style beyond existing patterns.
  - Vendored minified JS dependencies are not documented with source/version/license.
  - There are Syncthing conflict files in tags/, which suggests generated/source ownership is not fully clean.

  Testing

  There is an ERT test file, build-site-tests.el:1, covering comments helpers, Lima sitemap formatting, attachment copying, and body class behavior.

  Important finding: tests currently document known bugs. For example build-site-tests.el:112 says non-HTML backend handling returns nil because when has no else branch, and
  build-site-tests.el:248 documents the same issue for body-class filtering.

  make test does not run these local ERT tests. It delegates to the external authoring-service repo via Makefile:25. I found only Python bytecode under tests/__pycache__,
  not source tests in this repo.

  I did not run the test suite because the available test command points outside the repository, and loading build-site.el can generate files as part of its top-level
  execution.

  Developer Workflow

  Documented workflow:

  - Build site: make build or emacs -Q --script build-site.el.
  - Full build: make, which also sets permissions, cleans output, builds, generates search, and removes .venv.
  - Generate search: make search.
  - Start authoring UI: make author, delegated to the external authoring-service.
  - CI: Gitea Actions runs SITE_CLEAN=1 make and then a PowerShell monitor.

  The workflow is practical for the original machine, but not yet portable for a new developer because multiple paths are absolute and outside this repo.

  Maintainability Assessment

  Strong points:

  - The core build pipeline is centralized in build-site.el.
  - Local Emacs Lisp helpers are split by domain.
  - Metadata-driven behavior is understandable once keywords are known.
  - Build logging and cache integrity handling are thoughtful.
  - The repo has a README with the high-level build model and authoring-service notes.

  Risks and gaps:

  - build-site.el performs package install, generation, config, and publish execution at top level, making it hard to load safely in tests.
  - make test does not test this repo’s local ERT tests.
  - Absolute paths tie the repo to one machine and user.
  - Some generated files appear committed or source-adjacent, especially tags/*.org, recently-updated.org, and list pages.
  - Python dependencies are unpinned.
  - The comment API backend and authoring service are outside the repo, but the static site depends on their contracts.
  - Search output is named test.json, which is unclear for production behavior.
  - The working tree was already dirty: gitea-build-monitor.log and posts/posts-list.org were modified.

  Prioritised Recommendations

  1. Make local onboarding reproducible: document required system packages, Emacs version, Python version, pandoc, inotifywait, PowerShell, and expected external services.
  2. Remove or parameterize absolute paths in Makefile, CI, watcher.sh, and deployment scripts. Use env vars with documented defaults.
  3. Split build-site.el into loadable definitions and an explicit main entry point so tests can load functions without generating files or publishing.
  4. Wire local ERT tests into make test, and keep the external authoring-service tests under a separate target such as make test-author.
  5. Fix the documented non-HTML backend bugs in z/org-html-insert-comments-into-body and z/org-html-add-body-classes, then flip the tests.
  6. Clarify generated-file ownership: decide which .org files are hand-authored versus generated, remove sync-conflict files, and add generation notes or ignores where appropriate.
  7. Rename output/test.json and related references to a production name such as search-index.json.
  8. Pin Python dependencies and document vendored JS library versions/licenses.
  9. Document the comment API contract and authoring-service contract, since frontend code depends on /api/comments and the README points to an external repo.
  10. Add lightweight lint/format checks for Emacs Lisp, Python, and browser JS once the build/test workflow is stable.