| Metadata | Value |
|---|---|
| Status | Active |
| Version | 1.1.0 |
| Last Updated | 2026-09-10 |
| Author | Critical architecture review (for Seshadri) |
| Document Type | Design reference |
| Companion docs | sangeetha-grantha-state-of-nation-july-2026.md, sangeetha-grantha-uplift-tasks.md, 02-architecture/backend-system-design.md |
| Scope | Critical evaluation of the current codebase + a north-star reference architecture for a system of this class, with a gap analysis and sequenced path |
[!NOTE] Design/reference material: this page may include proposals or earlier implementation assumptions. Use current feature map for implemented behavior and current operating steps.
The question this answers: if you were building a digital scholarly corpus with AI-assisted ingestion and human curation today, from scratch, with no legacy — what would it look like, and how far is Sangeetha Grantha from that? The companion State-of-Nation doc covers dependency/AI-ecosystem drift; this doc deliberately goes one level up to architecture, engineering practice, and product shape. Findings are evidence-based (file paths cited) and severity-rated. The verdict is not “rewrite” — the bones are good — it is “harden the trust boundary, close the verification gap, and converge the data model toward a provenance-first canonical record.”
Sangeetha Grantha is, in class terms, a curated knowledge base: a system of record for Carnatic krithis, fed by an AI extraction pipeline, gated by human review, served through an admin console, with an aspirational consumer mobile surface. The right comparison points are not generic CRUD apps but systems like Wikidata, MusicBrainz, and digital-humanities corpora (Perseus, SARIT for Sanskrit texts) — systems whose core asset is trustworthy, attributable, versioned data, not code.
Measured against that bar:
What is genuinely strong
db-migrate), Prometheus metrics and ETag caching are wired in (Metrics.kt, Etag.kt), and every mutation writes to AUDIT_LOG.Where it falls materially short of north star
| # | Finding | Severity | Area |
|---|---|---|---|
| N1 | Passwords are stored unhashed — hashPassword() returns the input verbatim (UserManagementService.kt:143) |
Blocker | Security |
| N2 | No CI/CD whatsoever — no .github/, no pipeline; “verify across all three layers before committing” is a human convention, not a gate |
Critical | Engineering practice |
| N3 | Verification gap: 22 backend test files vs 0 DAL tests, 0 frontend tests (113 TS/TSX files untested), E2E track deferred (TRACK-035) | Critical | Quality |
| N4 | No production deployment story: compose prod profile still uses postgres/postgres literals; no managed-infra target, no backup/restore drill, no TLS story |
Critical | Operations |
| N5 | The canonical record has no version history — edits overwrite; AUDIT_LOG records that a change happened, not a re-materializable what. For a scholarly corpus, this is a data-model gap, not a nicety | Major | Data architecture |
| N6 | API surface drift: OpenAPI spec is 661 lines against ~30 route files and 35+ services — the contract is decorative, not enforced | Major | API |
| N7 | No API rate limiting / abuse controls (rate limiting exists only client-side toward Gemini), no request-ID propagation, no tracing | Major | Security / Observability |
| N8 | Repo hygiene: stray vendored org/jetbrains/... and Users/ directories at root, backend.log/out.log/*_logs.txt at root, an ADMIN_TOKEN in tracked tools/sangita-cli-archived/.env, four parallel agent-config files (AGENTS/CODEX/GEMINI/GOOSE.md) drifting |
Major | Hygiene |
| N9 | KMP mobile is aspirational — 18 Kotlin files across both shared modules; the “multiplatform” in the project name is currently a build configuration, not a product | Observation | Product |
| N10 | The AI layer issues (retired models, deprecated SDK, prompt-coerced JSON) — covered fully in the State-of-Nation doc as F1–F6; inherited here by reference | Critical | AI |
The one-paragraph verdict: the system has a publisher’s data model wrapped in a prototype’s trust boundary. The north star for this application class is “provenance-first corpus platform” — every canonical fact traceable to a source, every change replayable, every pipeline output schema-constrained and eval-gated, all behind a hardened, boring, automated delivery system. The gap is closable incrementally; nothing requires a rewrite, and the single most dangerous item (N1) is a one-day fix.
imported_krithis (raw + parsed payloads) → curator review → canonical krithi tables is a staging/curation/canon pipeline — the medallion pattern applied to scholarly data. The auto-approval service with quality scoring (QualityScoringService.kt, AutoApprovalService.kt) layered on top is the correct way to scale human review: score, auto-approve the unambiguous, queue the rest.gradle/libs.versions.toml + Makefile + compose gives one-command onboarding (make dev). The decomposition done in TRACK-073–076 (DTOs/mappers/repos/services split) shows the backend is being actively gardened, not just grown.MigrationRunner.kt.N1 — The authentication layer is hollow. UserManagementService.kt:143 stores the password as-is with a // NOT SECURE TODO. JWT issuance and role claims sit on top of credentials that are plaintext at rest. Everything else in the security story (JWT verification in Security.kt, role-based route guards) is undermined by this single function. There is also no login rate limiting, no lockout, no token revocation path. For a single-curator dev system this has been survivable; it must be fixed before any deployment that isn’t localhost.
N2/N3 — The verification gap is the biggest systemic risk. The project’s own CLAUDE.md instructs “verify changes compile/build across all three layers before committing” — which is an admission that no machine does this. Concretely:
BulkImport.tsx and a 688-line CuratorReviewPage.tsx that encode the curation workflow — the most business-critical UI in the system.main is discovered by the next human who runs make test.The pattern this produces is visible in the conductor history itself: TRACK-079 remediated 92% section inconsistency, TRACK-093 needed import data cleanup, TRACK-096 is converging divergent payload formats. These are all data regressions that tests and schema enforcement would have prevented upstream. The cost of the missing verification layer is already being paid — it’s just being paid in remediation tracks instead of CI minutes.
N4 — There is no path to production. The compose prod profile exists but ships POSTGRES_PASSWORD: postgres, no TLS termination, no secrets management, no backup automation, and the ops docs (08-operations/deployment.md, monitoring.md) describe intent more than runbook. The Google-cloud scaling docs in 02-architecture/ are forward-looking design, not provisioned reality. For a system of record, the backup/restore drill is the most important missing operational artifact — the corpus is the asset, and right now it lives in one Docker volume on one machine.
N5 — Audit ≠ history. AUDIT_LOG answers “who changed what, when” for compliance, but the canonical krithi tables hold only current state. A scholarly corpus needs bitemporal answers: “what did the canonical text of this krithi say on 2026-01-15, and which source/extraction/curator decision produced each section?” Today that’s reconstructable only by forensic reading of audit rows, if at all. This also blocks future features the docs already gesture at: variant comparison over time, curator-decision analytics, public changelog per krithi (the MusicBrainz “edit history” page every serious reference site has).
N6 — The API contract is unenforced. 661 lines of OpenAPI cannot describe ~30 route files. Nothing generates the spec from the routes, nothing generates clients/types from the spec, and the frontend hand-maintains its own types.ts (plus a parallel types/ directory — duplication already visible). The shared-domain KMP module was supposed to be the single source of DTO truth, but the frontend, being TypeScript, can’t consume it — so the project currently has three parallel type systems (Kotlin DTOs, OpenAPI YAML, TS types) with no mechanism keeping them aligned.
N7 — Observability is half-built. Prometheus metrics: yes. Structured request logging: partially (RequestLogging.kt). But: no trace/correlation IDs flowing curator-click → API → worker → Gemini call, which is exactly the path that fails in interesting ways; no error aggregation; no alerting; pipeline observability is log files (sangita_extraction_logs.txt sitting at repo root tells the story). For an async-pipeline system, per-job lineage (which source URL → which extraction → which import → which canonical record) is the observability that matters, and it’s currently spread across DB rows and logs.
N8 — Hygiene debt that erodes trust in the repo. Stray org/jetbrains/kotlin/... Kotlin-Gradle source files and a Users/<user>/... tree vendored at repo root (almost certainly an accidental IDE copy), build/log artifacts at root, an ADMIN_TOKEN value in the tracked tools/sangita-cli-archived/.env, and four AI-agent instruction files (AGENTS.md, CODEX.md, .agents/AGENTS.md, GOOSE.md) that will silently drift from CLAUDE.md. None of this breaks the build; all of it raises the noise floor for every future contributor, human or agent. The token should be rotated on principle even though the CLI is archived.
N9 — The product surface is narrower than the architecture pretends. The Gradle settings include shared KMP modules “for iOS & Android,” but with 18 files of shared code and no app modules, the mobile story is a placeholder. Meanwhile the actual product today is admin-only: there is no public read surface at all — no consumer web, no public API, no export. For a compendium, the absence of any reader is the largest product-level gap; the corpus currently has curators but no audience.
To be fair to the architecture, several fashionable things are rightly missing, and the north star below deliberately keeps them out: microservices (a modular monolith is correct at this scale and team size), Kafka/event buses (Postgres queues are fine), Kubernetes (one VM or one managed container service is enough for years), agent frameworks in the AI layer (bounded extraction with human review is the right posture for a system of record), and a separate vector database (pgvector covers it). Restraint is part of why this codebase is salvageable in-place.
The application class: a provenance-first scholarly corpus platform — AI-assisted ingestion, human curation, canonical publication. Here is what “best of breed, buildable by 1–3 people, 2026 toolchain” looks like.
flowchart TB
subgraph public["Public surface"]
direction TB
API[Read-only API]
WEB[Consumer web — later]
EXP[Bulk exports — later]
end
subgraph ingest["Ingestion"]
PY[Python worker<br/>extraction, enrichment, embeddings]
end
subgraph core["Core modular monolith"]
direction TB
STG[Staging]
REV[Review / curation]
CAN[Versioned canon]
AUD[(Audit + event log)]
end
subgraph data["PostgreSQL 18"]
direction TB
REL[Relational canon]
VEC[pgvector HNSW]
FTS[FTS]
end
EVAL[Eval harness]
PY --> STG
STG --> REV --> CAN
CAN --> REL
PY --> VEC
public --> CAN
EVAL --> PY
Hybrid + semantic search (TRACK-108) now implements the “search as a data feature” row: one Postgres, pgvector + lexical fusion. Consumer web and bulk exports remain later.
Data architecture — the heart of the difference
shared/domain/model/import/canonical-extraction-schema.json pointing this direction).krithi_revisions pattern: every accepted change creates a revision row; current view is a projection). This gives diff-able history, point-in-time reads, and safe bulk-operation rollback — the feature that turns “we ran a bad remediation” from a crisis into a revert.canonical_section.provenance → extraction_id → source_document_id → source_registry as enforced foreign keys. The question “why does this line read paramAtmA and not paramAthma?” has a one-query answer.SearchService fusing both. No external search infra.AI/ingestion layer
Backend
dal has its own test suite (Testcontainers against real Postgres), services are constructor-injected and unit-tested, routes are thin and the OpenAPI spec is generated from code (or code from spec — either direction, but mechanically). TypeScript client types are generated from that spec; the frontend never hand-writes a DTO again.Frontend
Delivery & operations
.env files purged and tokens rotated.Repository & collaboration
.gitignore hardened. One canonical agent-instruction file (CLAUDE.md) with the others reduced to pointers, so guidance can’t fork. Conductor tracks stay — they’re a strength — but get a CI-checked link between track ID and commit (the convention already exists; enforce it).Microservices, Kubernetes, event buses, GraphQL federation, multi-region, a separate vector DB, agentic AI pipelines, and real-time collaboration. At <10⁶ records, <10 curators, and a read-mostly public surface, every one of these is negative-value complexity. The north star is a very well-run monolith with a very trustworthy database.
| Dimension | Current state | North star | Gap size |
|---|---|---|---|
| Domain/data model | Strong: staged ingestion, junction tables, audit log | + versioned canon, enforced provenance graph | Medium — additive migrations, no redesign |
| Security | JWT + RBAC shell over plaintext passwords; no server rate limiting; tracked token | Boring-complete auth; secrets manager; throttling | Large but cheap — days, not weeks |
| Quality/verification | Good service tests; 0 DAL, 0 frontend, 0 E2E; no CI | CI-gated full-stack tests + eval harness | Large — the biggest sustained investment |
| AI layer | Deprecated SDK, retired model strings, prompt-coerced JSON (per F1–F6) | Unified SDK, structured output, batch, evals-in-CI | Medium — already fully specified in uplift tasks U1–U5 |
| API contract | Hand-written OpenAPI, drifted; 3 parallel type systems | Generated spec + generated TS client | Medium |
| Observability | Metrics + request logs | + request IDs, traces, pipeline lineage, curator-override dashboard | Medium |
| Operations | make dev excellent; prod profile cosmetic; no backups |
IaC’d single target, PITR + restore drills | Large — and the highest-stakes gap |
| Product surface | Admin console only; KMP mobile aspirational | + public read API/web with search & permalinks | Large — the strategic leap |
| Repo hygiene | Stray trees, logs, multi-agent-file drift | Clean root, one instruction source | Small — an afternoon |
This deliberately interleaves with the open uplift tasks (U1–U5) rather than competing with them.
Phase 0 — Stop the bleeding (days).
hashPassword with bcrypt/argon2id + a migration to rehash on next login; add login throttling. (N1)ADMIN_TOKEN; remove stray Users/, org/, root logs; harden .gitignore. (N8)Phase 1 — Foundation under in-flight work (1–2 weeks, alongside U1–U3).
CuratorReviewPage and BulkImport. (N3)Phase 2 — The data-model uplift (2–3 weeks).
krithi_revisions + projection views) and enforced provenance FKs. Do this before the Trinity import completes at full scale — retrofitting history onto 1,245 imported krithis is much harder than recording it from day one. (N5)Phase 3 — Production reality (1–2 weeks).
Phase 4 — The audience (ongoing).
The honest one-liner: Sangeetha Grantha has a north-star-quality data model and documentation culture attached to prototype-grade security, verification, and operations. That is the good failure mode — the inverse (hardened pipelines around a muddled domain model) is far more expensive to fix. Every gap identified here is closable in-place, in sequence, without a rewrite, and the two highest-leverage moves are also the cheapest: hash the passwords this week, and stand up CI so the tests you already wrote start protecting you. After that, versioned canon + provenance is the investment that most differentiates this system in its class — it is the difference between a database of krithis and a scholarly record of them.