Skip to main content

CQRS in Framework M

Command Query Responsibility Segregation (CQRS) is Framework M's escape hatch for DocTypes whose read workload no longer fits a row-per-document table. The write side stays on the GenericRepository; the read side is separated from it at three escalating levels. Each level is opt-in per DocType, so the default path — one repository, one table — is untouched until you choose otherwise.

LevelSeparationEnabling knob
1 — Read replicaReads hit a second database engineFRAMEWORK_M_DB_REPLICA_URL
2 — Inverted child-table viewsChild rows browsable standalone, reads via read-only routesChild DocType + is_child_table metadata
3 — Custom read modelsReads served by a per-DocType projection, not the tableframework_m.adapters.read_model entry point

Level 1 — Read replica​

The lightest form of CQRS: the same repository, the same SQL, but read operations execute against a dedicated replica engine instead of the primary. Writes still go to the primary.

When FRAMEWORK_M_DB_REPLICA_URL is configured, the lifespan registers the replica as a bind (session.py) and SessionFactory.get_read_session() hands out a session bound to it. Every generic read route — list, get-one, singleton GET — uses the read session and falls back to the primary when no replica is registered:

async with repo_factory.session_factory.get_read_session() as session:
    items, total = await repo.list_entities(session, filters=...)

Read-your-own-writes​

A replica can lag the primary, so a user who just wrote a record might not see it on the next read. Framework M covers this with an X-Force-Primary: true request header: the frontend sends it for reads made within 5 seconds of a create/update/delete, and the read route then uses the primary session. The result is method-based read-your-own-writes without any cross-node coordination.

Routing at the router, not the repository​

Replica routing happens at the web/router layer, deliberately not inside GenericRepository. Callers inject a single session into the repository to wrap multi-repo operations in one transaction (the Unit of Work pattern); if the repository reached for a replica session itself, that transaction guarantee would break. Keeping the choice of session in the route preserves the UoW and makes the replica an explicit per-request decision.

Pagination posture (cursor pagination declined)​

The generic read path paginates with offset + limit, and large-table totals use a bounded count: count() runs SELECT count(*) FROM (SELECT 1 … LIMIT threshold+1) and reports the sentinel 10000+ instead of scanning the whole table (see the count_threshold parameter). This offset-plus-bounded-count posture is the chosen performance posture — keyset/cursor pagination is explicitly not implemented. If profiling later shows deep offset pages are a real cost, the migration path is a composite index on (creation, id) so a keyset variant can be added without a schema rethink.

Level 2 — Inverted child-table views​

Child tables (line items, addresses, permissions) are normally only manipulated through their parent document, which owns their validation, lifecycle checks, and domain logic. But read access is another matter: child rows are ordinary tables, so they can be browsed standalone without routing every lookup through the parent.

The generic list route serves child DocTypes read-only:

  • create_meta_router mounts child tables with create_crud_routes(read_only=True) — only list_entities/get_entity (plus singleton GET) exist; POST/PUT/ DELETE/restore are omitted, so writes get 405/404. Writes must go through the parent's endpoints, preserving its validation, submitted-document lifecycle, and domain logic (e.g. recalculating a Purchase Order total when a line item changes).
  • The metadata payload carries is_child_table, so the Desk renders a Parent link column pointing back to the owning document (parent_doctype/parent_id are merged into list/get responses from the DB parent/parenttype columns).
  • An explicit api_resource opt-in still wins over child-table read-only: a DocType that declares both gets full CRUD.

In CQRS terms this is an inverted view: the same rows, presented as their own read surface instead of only as nested fragments of the parent.

Level 3 — Custom read models​

For DocTypes whose read workload no longer fits a row-per-document table, the full escape hatch: a separately maintained, query-optimized projection of the same data, served in place of the GenericRepository.

When to use a read model​

  • The list endpoint needs an exact total (pagination) and the underlying table is too large for a bounded count.
  • The list endpoint should not hit the relational table at all (denormalized aggregates, GROUP BY rollups, column-store, full-text search).
  • You want the read path eventually consistent with the write path rather than transactionally coupled to it.

Do not use a read model when the default repository already serves the DocType fine — the read model adds a projection you must keep in sync.

The protocol​

ReadModelProtocol lives in framework_m_core.interfaces.read_model and has three methods:

MethodPurpose
project(event)Apply a domain event to the projection (idempotently)
query(...)Serve the optimized read; returns ReadQueryResult
rebuild()Rebuild the projection from event history

query() is what the web layer calls. It takes list[FilterSpec] (operators preserved — EQ/LT/LIKE/IN), an order_by list (-field for descending), and limit/offset; it returns a ReadQueryResult:

from framework_m_core.interfaces.read_model import ReadQueryResult

result = await read_model.query(
    filters=[FilterSpec(field="amount", operator=FilterOperator.GT, value=100)],
    order_by=["-total"],
    limit=20,
    offset=0,
)
# result.items -> list[dict], result.total -> exact count for pagination

Worked example: per-DocType invoice analytics​

The Invoice DocType keeps its transactional table. A separate read model serves the monthly-total list view.

1. The projector — keep the read model in sync​

A projector subscribes to the DocType's lifecycle events (doc.created, doc.updated, doc.deleted) and updates the projection. Make it idempotent: reprocessing the same event must produce the same state (the event bus may redeliver).

# analytics/invoice_read_model.py
from framework_m_core.interfaces.event_bus import Event
from framework_m_core.interfaces.read_model import (
    ReadModelProtocol,
    ReadQueryResult,
)


class InvoiceAnalytics(ReadModelProtocol):
    """Denormalized monthly totals per customer, kept in sync via events."""

    def __init__(self, session_factory) -> None:
        self._session_factory = session_factory

    async def project(self, event: Event) -> None:
        doctype = event.data.get("doctype") if event.data else None
        if doctype != "Invoice":
            return
        async with self._session_factory.get_session() as session:
            if event.type == "doc.deleted":
                await self._delete_row(session, event.data)
            else:
                await self._upsert_row(session, event.data)  # idempotent

    async def query(
        self,
        filters: list[FilterSpec] | None = None,
        order_by: list[str] | None = None,
        limit: int = 100,
        offset: int = 0,
    ) -> ReadQueryResult:
        # Analytics projection: GROUP BY customer + month, exact COUNT for total
        async with self._session_factory.get_session() as session:
            rows, total = await self._run_query(
                session, filters, order_by, limit, offset
            )
            return ReadQueryResult(items=rows, total=total)

    async def rebuild(self) -> None:
        # Truncate the projection table and replay the Invoice event history.
        ...

    def register(self, bus) -> None:
        """Subscribe the projector to the events it cares about."""
        bus.subscribe("doc.created", self.project)
        bus.subscribe("doc.updated", self.project)
        bus.subscribe("doc.deleted", self.project)

2. Register it for the DocType​

Read models are registered per DocType through the framework_m.adapters.read_model entry-point group. The framework instantiates the class once at startup (ep.load()()), so the class constructor must take no required arguments — resolve dependencies (session factory, event bus) from the DI container or module-level globals.

# pyproject.toml in your app
[project.entry-points."framework_m.adapters.read_model"]
Invoice = "analytics.invoice_read_model:InvoiceAnalytics"

Any DocType without a matching entry point is untouched and keeps hitting the GenericRepository.

3. How the read path routes​

When the web layer serves GET /api/v1/Invoice, meta_router asks the RepositoryFactory for a read model first:

GET /api/v1/Invoice
   └─ repository_factory.get_read_model("Invoice")
        ├─ registered  ─► read_model.query(filters=rls_filters, order_by, limit, offset)
        │                   └─ ReadQueryResult ─► {items, total, has_more, count_capped}
        └─ not found   ─► GenericRepository (default behavior, unchanged)

Routing notes:

  • Precedence. A registered read model replaces the repository for the list endpoint — the repository is only used when no read model exists.
  • RLS still applies. The user/tenant filters from apply_rls_filters() pass through unchanged as list[FilterSpec], so row scoping is the read model's job too.
  • Bypasses bind/fan-out. A read model is a single global projection, so ?bind=<tenant> and cross-tenant fan-out are skipped for that DocType.
  • Exact pagination. total comes from the read model, so has_more = offset + len(items) < total and count_capped is always False.
  • Write path is unchanged. POST/PUT still go through the GenericRepository; the read model is updated asynchronously by its projector and may lag the write (eventual consistency).

Level 3 and eventual consistency​

The read-model branch bypasses read-replica session routing entirely — a read model is its own eventually-consistent data source, so X-Force-Primary has no effect there. If a DocType needs read-your-own-writes, the projector/write path must provide it (e.g., a synchronous first-write projection), not the read side.

Related: