All updates
backend
database
frontend
devops

Ingest pipeline overhaul: reliable batch processing and real‑time sync status

  • Project‑scoped sync runs and batched commit processingA new project‑scoped sync run coordinates all repositories in a resync so commits are processed exactly once. Commits are now processed in batches (50) with parallel diff→LLM→write waves, preventing duplicate work and ensuring period summaries run only after every repo is finished.
  • Real‑time UI sync statusThe web UI now polls repository sync status every 5s and automatically refreshes when status changes so you no longer need to manually refresh to see progress.

Added

  • Added a project‑scoped sync_runs table and repository to track a single resync across all repos in a project.database
  • Added a new batched processing function that processes up to 50 commits per batch with parallel fetch, LLM calls, and writes to make historical syncs faster and more reliable.backend
  • Added a lightweight API endpoint and client poller (SyncStatusPoller) so the dashboard and project pages refresh automatically as sync status changes.frontend
  • Added server migration and SQL RPCs to atomically register repo batch counts and increment completed batches for robust cross‑repo coordination.devops

Changed

  • Inlined period aggregation so period entries are generated only after all commit batches are finished, keeping project status accurate while aggregation runs.backend
  • Made history estimates include the current open period and clarified dialog copy so users know partial current‑period entries are created immediately and updated automatically.frontend
  • Scoped unprocessed‑commit lookups to each repo's paginated commit set to prevent concurrent syncs from claiming the same commits.backend
  • Updated sync status labels in the UI to be more descriptive: Live, Syncing Commits, Writing Entries, Building Changelog.frontend
  • Enabled BYOK users to bypass the free‑plan monthly commit cap so projects using their own key are not blocked by the system cap.backend

Fixed

  • Fixed a JSON serialization bug where returning Infinity caused every historical commit to be skipped; the free‑plan cap now uses a bypass flag so batches process correctly.backend
  • Fixed a race where two repo syncs in the same project could both claim the full unprocessed pool and send duplicate batches; each sync now only processes its own commits.backend
  • Fixed period backfill prematurely marking projects idle; aggregation now runs inline and set‑idle only happens after all entries are persisted.backend
  • Fixed missing database type aliases that caused build failures; types and exports were restored.database

API

  • Updated project APIs to create a single sync_run per resync and pass its ID to all repo sync events so period aggregation is triggered exactly once per project.backend