KnowStack

Six agents, one cron, a newsletter nobody had to write.

A daily learning platform for AI, DevOps and Cloud engineers. Six agents run on a cron, turn the morning’s firehose into something worth reading, and publish a newsletter and quizzes nobody had to write.

6
pipeline stages
768
embedding dimensions
06:37 UTC
daily, on GitHub Actions

crawl → filter → summarise → embed → persist → publish

RSS feeds and Hacker News go in. Keyword and quality scoring drops what is off-topic. A Groq LLM writes a two-or-three sentence technical summary of what survives. all-mpnet-base-v2 turns each into a 768-dimension vector locally and for free. Supabase stores them behind an IVFFlat index, keyed on the original URL so re-runs upsert rather than duplicate. Then the day’s newsletter is built and up to five quizzes are generated from it. FastAPI on Vercel Python serves the API; React and Vite serve the reader.

Decisions

Embedding locally rather than by API

sentence-transformers runs inside the Actions job, so embedding costs nothing and adds no rate limit to the critical path. The trade is that the embedding dimension becomes load-bearing infrastructure: all-mpnet-base-v2 emits 768 dimensions and the articles.embedding column must match it, so changing model is a migration and a re-embed of every row, not a config change.

A cron, not a queue

The whole pipeline is one scheduled GitHub Actions run with workflow_dispatch enabled, so it can also be triggered by hand. For something that needs to happen once a day, a queue would be infrastructure to operate in exchange for latency nobody is waiting on.

Not persisting raw_content

Article text comes from the RSS excerpt at crawl time and is dropped after summarisation. It keeps the database small and avoids storing other people’s content — at the cost that repairing a bad summary means re-fetching the original URL, which is what backfill_summaries.py exists to do.

A fallback that hides a failure is worse than a crash

Groq retired llama-3.1-8b-instant on 2026-08-16. Every summarise call began returning 404, and a per-article except fell back to summary = title. The pipeline embedded those titles, persisted them, and exited 0. Six consecutive green runs published nothing but headlines, and it cost thirteen days to notice — while a second model, llama3-8b-8192 in the quiz generator, had been dead for roughly a year. summarize_articles now raises when more than half the articles fail, so the run goes red before anything is embedded or written. A per-article fallback is fine; a wholesale one is a failure and must not be allowed to look like success.

Built with

  • FastAPI
  • React · Vite
  • Supabase · pgvector
  • sentence-transformers
  • Groq
  • GitHub Actions