Skip to main content

Concepts

This section provides a deep dive into the fundamental concepts and recurring patterns used across Framework M.

8.1 The DocType Pattern​

At the core of Framework M is the DocType (Document Type), a metadata-driven approach to defining business entities.

  • Pydantic Metadata: Schemas are defined as Pydantic models, providing runtime validation and IDE type-safety.
  • Metadata Endpoints: Global metadata registries allow generic UI components (like the Desk) to automatically render forms and lists without manual frontend coding.

8.2 Dependency Injection (DI) as Universal Composer​

Framework M uses the dependency-injector library as its primary orchestration mechanism. This provides a unified composition layer across all execution contexts, ensuring that the same services and adapters are available regardless of how the system is invoked:

  • API Layer: Services are wired into request handlers via the DI container.
  • CLI Framework (Cyclopts): Services are provided to terminal commands.
  • Background Jobs (Taskiq): Workers bootstrap the same DI container to process jobs.
  • Interactive REPL: Developers can access the full service layer in a shell.
  • Testing (Pytest): Providers are overridden via fixtures for mocking.

8.3 Hexagonal Adapters & Repository Pattern​

Business logic is decoupled from infrastructure through formal Protocols (Interfaces).

  • Repositories: Standardized CRUD interfaces for data access.
  • Controllers: Where DocType-specific business logic and lifecycle hooks (validate, before_save, after_save) live.
  • Adapters: Concrete implementations for specific databases (SQLAlchemy), message brokers (NATS), or caches (Redis).

8.4 Plugin Discovery via Python Entrypoints​

Framework M discovers adapters and plugins dynamically through Python entrypoints in pyproject.toml. This allows for a "Zero-Cliff" future where any part of the system can be extended or replaced by a third-party package without modifying the core framework code.

8.5 Unit of Work (UoW)​

The Unit of Work (UoW) is a behavioral pattern used to maintain a list of objects affected by a business transaction and coordinate the writing out of changes and the resolution of concurrency problems.

In Framework M, the UoW acts as the transactional orchestrator:

  • Atomic Operations: It ensures that changes across multiple repositories (e.g., updating an Order and decreasing Stock) are committed atomically within a single database transaction.
  • Adapter Coordination: It manages the lifecycle of database sessions and message broker connections, ensuring that events are only emitted to NATS JetStream after the database transaction has successfully committed.
  • Rollback Safety: If any part of the unit fails, the UoW guarantees a clean rollback across all participating adapters to prevent data corruption.

8.6 Frontend Modular Architecture (MFE)​

The frontend is a modular, high-performance architecture centered around the Desktop Workspace Shell and external Plugins. Depending on the deployment needs, it supports two distinct orchestration modes:

  • Distributed MFE Mode: Utilizing a Micro-Frontend approach, the framework-m-desk acts as a host that dynamically loads "Plugin MFEs" via Vite-powered module federation. This is ideal for large, independently-deployable macroservices.
  • Monolith Single Bundle Mode: For smaller projects or unified deployments, the framework-m-vite-plugin can automatically collapse all discovered plugins into a single, optimized production bundle. This simplifies deployment by serving everything from a single static directory.
  • Tamagui Styling Engine: The framework uses Tamagui as its primary styling and layout engine, providing performance-optimized CSS-in-JS that remains compatible with React Native.
  • MComponents Abstraction: To avoid vendor lock-in, all UI primitives (Tabs, Modals, Tables) are wrapped and exposed via MComponents. This ensures that the underlying engine can be swapped or upgraded without modifying plugin code.
  • Token-Driven Themes: Architecture is strictly token-based (Themes, Spacing, Typography), enabling unified customization ("White Labeling") across the entire system by simply updating the framework-m-ui theme configuration.
  • Host-Plugin Bridge: Communication between the shell and plugins is handled through the Plugin SDK, which provides formal JS protocols for sidebar registration, menu items, and global state access.

8.7 Virtual DocTypes (Data Abstraction)​

"Virtual DocTypes" allow DocTypes to be backed by any data source, not just the primary SQL database.

  • SQL Binds: For other SQL databases (e.g., Legacy MySQL), we use SQLAlchemy Binds (Meta.db_bind).
  • Non-SQL Backends: For external APIs, NoSQL (MongoDB), or System resources (RAM/Files), we use Custom Repositories.
    • Mechanism: The App registers a custom implementation of RepositoryProtocol for that DocType.
  • Consistency: For Virtual DocTypes backed by non-SQL stores, we recommend the Outbox Pattern or Saga Pattern to guarantee eventual consistency across primary and virtual data sources.

8.8 Optimistic Concurrency Control (OCC)​

Framework M supports optional OCC to prevent lost updates in valid-first-wins scenarios.

  • Mechanism: A _version (integer) column is added to the schema.
  • Update Logic: The Repository performs UPDATE ... WHERE id=:id AND _version=:current_version. If zero rows are affected, a VersionConflictError is raised.

8.9 Domain Events & Audit Log​

Every state change in the system emits a Domain Event via the EventBusProtocol.

  • Standard Events: doc.create, doc.update, doc.delete, doc.submit.
  • Audit Trail: The "Audit Log" is a built-in service that subscribes to doc.* events and persists them to a SystemLog table. This decouples logging from the business transaction.

8.10 Controller Lifecycle (Hooks)​

In a Hexagonal architecture, lifecycle hooks are placed on the Controller (Service) rather than the Model.

  • Hooks: validate, before_save, after_save, on_submit.
  • Execution: The GenericRepository invokes these hooks on the registered Controller during the transaction lifecycle.

8.11 The Studio Architecture (Twin-Mode)​

The Studio provides a visual development environment that maintains a single source of truth in Source Code.

  • Mode 1: Local Bridge: Runs locally and writes directly to *.doctype.py files using LibCST to preserve manual code changes and comments.
  • Mode 2: Cloud Studio: A hosted environment that interacts with the project via a Git Adapter, committing and pushing changes directly to the repository.

8.12 Extensibility & Overrides​

Framework M avoids monkey-patching in favor of Dependency Injection and Python Entrypoints.

  • Overrides: Any framework Protocol (Auth, Storage, etc.) can be overridden by a custom implementation registered via [project.entry-points."framework_m.overrides"] in a package's pyproject.toml.
  • Boot Phase: During startup, the DI container checks these entrypoints and binds custom implementation to the kernel.

8.13 Governed Output & Observability​

Framework M enforces a Zero-Print Policy to ensure that all system output is either human-optimized (CLI) or machine-optimized (Logs), and always governed by the security redaction layer.

  • CLIConsole: A centralized Rich-based utility for terminal output. It handles consistent styling (Success, Error, Warning) and ensures that human-facing diagnostics are clean and readable.
  • LoggingProtocol & Standard Logging: A structured logging interface (backed by structlog) used for all system events. Standard Python logging.getLogger(__name__) is also fully supported as it is intercepted by our Unified Bridge and routed through the same redaction pipeline.
  • Unified Bridge: During bootstrap, the framework intercepts standard Python logging and Uvicorn logs, routing them through the unified structlog pipeline for consistent observability across the entire stack.

8.14 Architectural Isolation & API Prefixing​

To support a seamless transition from monolithic to distributed Macroservice architectures, Framework M implements a dynamic API prefixing strategy.

  • Dynamic Prefixing: Micro-frontends (MFEs) are aware of their "service context" via build-time injection. They dynamically construct API endpoints (e.g., /api/{service}/v1) at runtime.
  • Service Namespacing: The frameworkMDataProvider supports namespaced resource resolution (e.g., finance/PurchaseInvoice). This allows a single frontend shell to coordinate with multiple backend macroservices by routing requests to the correct service-scoped endpoint.
  • Zero-Cliff Migration: This pattern ensures that the same MFE code can operate in both a monolithic deployment (where all services share an /api/v1 prefix) and a distributed deployment (where each service has its own namespace), without requiring code modifications.
  • Protocol-Agnostic Resolution: The system handles API, Metadata, and WebSocket endpoints through a unified normalization layer, ensuring consistent prefixing even when using absolute base URLs or CDN-hosted assets.

8.15 Transactional Outbox Pattern​

To achieve reliable data propagation without tight database coupling or distributed transactions, Framework M implements a Transactional Outbox Pattern.

  • Atomicity: Changes to primary tables and the staging of external event messages are committed within the same database transaction.
  • Outbox Table: Pending operations are saved as OutboxEntry records with target, payload, status (pending, processed, failed), and retry metadata.
  • Async Dispatching: A dedicated process_outbox worker polls the outbox database table periodically, dispatches events to the primary EventBus, and marks entries accordingly, guaranteeing at-least-once delivery semantics.

8.16 Background Workers & Stateless Scheduling​

Periodic background tasks and scheduled cron jobs are integrated directly into the CLI worker environment.

  • Embedded & Stateless Scheduling: Instead of a standalone stateful daemon, scheduling is driven by a pluggable TickerProtocol (e.g., InProcessTickerAdapter running inside m worker). The ticker wakes up on time boundaries, formats a unique time-bucketed deduplication_id, and enqueues jobs dynamically.
  • Broker-Level Deduplication: Multiple worker processes run concurrent ticker loops safely. The backing queue/broker (like NATS JetStream) uses the deduplication_id to discard duplicate triggers at the transport layer, ensuring exactly-once execution without database or application locks.
  • Immediate Async Tasks: High-throughput operations, such as sending emails, are dispatched immediately as single background tasks (e.g., send_email) via the JobQueueProtocol, avoiding latency and database-polling overhead.

8.17 Nested Set Tree Management and Tree Scope​

For hierarchical data structures (like Account charts, Warehouse trees, and Categories), Framework M provides built-in nested set management.

  • Nested Set (lft/rgt): Optimized tree manipulation via bounds shifting. Root and descendant insertions/updates shift bounds in database transactions rather than rebuilding whole trees.
  • Tree Scope Fields: To support multi-tenant or domain-isolated tree structures, tree_scope_fields can be declared in the DocType Meta configuration. This forces tree boundary shifts and parent lookups to be isolated dynamically within specific scopes (e.g., tenant_id).

8.18 User Self-Service Federation Lock​

When an application is configured in Federated Mode, the external Identity Provider (such as Keycloak or Azure AD) serves as the Single Source of Truth (SSoT) for user profiles.

To prevent data drift and JIT synchronization conflicts, Framework M enforces a federation lock:

  • Local self-service profile and password update endpoints are disabled.
  • The user is redirected to the upstream IdP's configured profile_url for all profile edits.
  • Standard forms render in read-only mode in the frontend shell, providing a link to redirect users upstream.

8.19 Enterprise Data Handover​

During employee offboarding or administrative restructuring, document and asset ownership must be transferred between users. To prevent breaking business rules:

  • Framework M rejects raw database SQL updates (e.g., blanket UPDATE owner_id = ... statements).
  • Instead, the OwnershipTransferService delegates ownership changes to each document type's repository and controller lifecycle hooks (e.g. before_transfer_ownership and after_transfer_ownership).
  • Historical logs (like AuditLog and ActivityLog) are marked immutable and explicitly excluded from transfers to preserve compliance history.

8.20 OIDC Frontchannel Logout​

In multi-application federated environments, logging out from one application should invalidate the user's session across all other active client applications.

  • Iframe Invalidation: The central IdP invalidates the user's central session and loads a hidden page rendering logout iframes pointing to each client's frontchannel logout URI.
  • BFF Session Destruction: Framework M's web adapter intercepts the browser-level frontchannel iframe call (which includes the user's cookies), destroys the corresponding local session record in the SessionStore, and sends headers to clear browser session cookies.

8.21 Row-Level Security (RLS) & Multi-Tenant Filtering​

In single-database multi-tenant applications, both tenant isolation and application-level access control (like user or team restrictions) are enforced via Row-Level Security (RLS) query filters. Framework M divides the management of these filters:

  • Framework-Managed Filtering (Tenants): The framework automatically injects tenant-isolation filters (e.g., WHERE tenant_id = :tenant_id) globally at the repository boundary to prevent cross-tenant leaks.
  • Developer-Managed Filtering (Users/Teams): Developers can configure additional, granular RLS filters using DocType metadata:
    • rls_field specifies the column in the database (e.g., owner, team_id) to filter on.
    • rls_scope specifies the attribute name on the UserContext object (e.g., id, teams) to match.