Framework M Features

Auto-generated feature list from phase checklists.

Overview

Phase	Title	Completion
01	Phase 01: Core Kernel & Interfaces	██████████ 100%
02	Phase 02: DocType Engine & Database	██████████ 100%
03	Phase 03: API Layer & Authorization	██████████ 100%
04	Phase 04: Background Jobs & Events	██████████ 100%
05	Phase 05: CLI & Developer Tools	██████████ 100%
06	Phase 06: Built-in DocTypes & Core Features	██████████ 100%
07	Phase 07: Studio (Code Generation UI)	██████████ 100%
08	Phase 08: Workflows & Advanced Features	██████████ 100%
09a	Phase 09A: Frontend	██████████ 100%
09b	Phase 09B: Documentation	██████████ 100%
10	Phase 10: ERP Enterprise Features	██████████ 100%
10	Phase 10: Production Readiness & Deployment	██████████ 100%
11	Phase 11: Package Split & MX Pattern	██████████ 100%
12	Phase 12: LLM-Ready Documentation & Automation	██████████ 100%
13	Phase 13 Pending Checkpoints	░░░░░░░░░░ 0%
13	Phase 13: UI/UX Polish & Design System Enhancement	██████████ 100%
14	Phase 14: Studio Enhancements (Connected Citizen Edition)	██████████ 100%
15	Phase 15: Frontend Plugin Architecture	██████████ 100%

Phase 01: Core Kernel & Interfaces

Phase: 01 Objective: Set up the project foundation with clean hexagonal architecture. Define all protocol interfaces (ports) without any implementation. Status: 100% Complete

1. Project Initialization

Progress: 17/17 (100%)

Completed:

✅ Create project structure using uv:
✅ Setup pyproject.toml with dependencies
✅ Add litestar[standard]>=2.0.0
✅ Add sqlalchemy[asyncio]>=2.0.0 (ensure agnostic, test with sqlite)
✅ Add pydantic>=2.0.0
✅ Add pydantic-settings>=2.0.0
✅ Add dependency-injector>=4.41.0 (powerful DI container)
✅ Add asyncpg>=0.29.0 (PostgreSQL async driver)
✅ Add redis>=5.0.0
✅ Remove arq>=0.26.0
✅ Add taskiq>=0.11.0, taskiq-nats>=0.4.0, nats-py>=2.0.0
✅ Add dev dependencies: pytest, pytest-asyncio, mypy, ruff
✅ Configure development tools
✅ Setup mypy with strict mode in pyproject.toml
✅ Configure ruff for linting and formatting
✅ Add .gitignore for Python projects
✅ Create directory structure

2. Define Port Interfaces > 2.1 Repository Protocol

Progress: 16/16 (100%)

Completed:

✅ Create tests/core/interfaces/test_repository.py
✅ Define test for RepositoryProtocol interface compliance
✅ Create src/framework_m/core/interfaces/repository.py
✅ Define supporting models:
✅ FilterSpec - Typed filter specification (field, operator, value)
✅ OrderSpec - Sorting specification (field, direction)
✅ PaginatedResult[T] - Result with items, total, limit, offset, has_more
✅ Define RepositoryProtocol[T] (Generic):
✅ async def get(self, id: UUID) -> T | None
✅ async def save(self, entity: T, version: int | None = None) -> T (OCC support)
✅ async def delete(self, id: UUID) -> None
✅ async def exists(self, id: UUID) -> bool
✅ async def count(self, filters: list[FilterSpec] | None = None) -> int
✅ async def list(self, filters: list[FilterSpec] | None, order_by: list[OrderSpec] | None, limit: int, offset: int) -> PaginatedResult[T]
✅ async def bulk_save(self, entities: list[T]) -> list[T]
✅ Add type hints with Generic[T] and Protocol

2. Define Port Interfaces > 2.2 Event Bus Protocol

Progress: 10/10 (100%)

Completed:

✅ Create src/framework_m/core/interfaces/event_bus.py
✅ Define Event base model with id, timestamp, source, type, data
✅ Define EventBusProtocol with methods:
✅ async def connect(self) -> None (for NATS/Kafka)
✅ async def disconnect(self) -> None
✅ def is_connected(self) -> bool
✅ async def publish(self, topic: str, event: Event) -> None
✅ async def subscribe(self, topic: str, handler: Callable[[Event], Awaitable[None]]) -> str (returns subscription_id)
✅ async def subscribe_pattern(self, pattern: str, handler: Callable) -> str (e.g., doc.*)
✅ async def unsubscribe(self, subscription_id: str) -> None

2. Define Port Interfaces > 2.3 Auth Context Protocol

Progress: 8/8 (100%)

Completed:

✅ Create src/framework_m/core/interfaces/auth_context.py
✅ Define UserContext Pydantic model with fields:
✅ id: str
✅ email: str
✅ roles: list[str]
✅ tenants: list[str]
✅ Define AuthContextProtocol with methods:
✅ async def get_current_user() -> UserContext

2. Define Port Interfaces > 2.4 Storage Protocol

Progress: 11/11 (100%)

Completed:

✅ Create src/framework_m/core/interfaces/storage.py
✅ Define FileMetadata model with path, size, content_type, modified, etag
✅ Define StorageProtocol with methods:
✅ async def save_file(self, path: str, content: bytes, content_type: str | None = None) -> str
✅ async def get_file(self, path: str) -> bytes
✅ async def delete_file(self, path: str) -> None
✅ async def list_files(self, prefix: str) -> list[str]
✅ async def get_metadata(self, path: str) -> FileMetadata | None
✅ async def get_url(self, path: str, expires: int = 3600) -> str (presigned URL for S3)
✅ async def copy(self, src: str, dest: str) -> str
✅ async def move(self, src: str, dest: str) -> str

2. Define Port Interfaces > 2.5 Job Queue Protocol

Progress: 9/9 (100%)

Completed:

✅ Create src/framework_m/core/interfaces/job_queue.py
✅ Define JobStatus enum: PENDING, RUNNING, SUCCESS, FAILED, CANCELLED
✅ Define JobInfo model with id, name, status, enqueued_at, started_at, result, error
✅ Define JobQueueProtocol with methods:
✅ async def enqueue(self, job_name: str, **kwargs) -> str (returns job_id)
✅ async def schedule(self, job_name: str, cron: str, **kwargs) -> str
✅ async def cancel(self, job_id: str) -> bool
✅ async def get_status(self, job_id: str) -> JobInfo | None
✅ async def retry(self, job_id: str) -> str (returns new job_id)

2. Define Port Interfaces > 2.6 Permission Protocol

Progress: 5/5 (100%)

Completed:

✅ Create src/framework_m/core/interfaces/permission.py
✅ Define PermissionProtocol with methods:
✅ async def has_permission(user: UserContext, doctype: str, action: str, doc_id: str | None) -> bool
✅ async def get_permitted_filters(user: UserContext, doctype: str) -> dict
✅ Add action types: "read", "write", "create", "delete", "submit"

2. Define Port Interfaces > 2.7 Print Protocol

Progress: 3/3 (100%)

Completed:

✅ Create src/framework_m/core/interfaces/print.py
✅ Define PrintProtocol with methods:
✅ async def render(doc: BaseModel, template: str, format: str = "pdf") -> bytes

2. Define Port Interfaces > 2.8 Cache Protocol

Progress: 10/10 (100%)

Completed:

✅ Create src/framework_m/core/interfaces/cache.py
✅ Define CacheProtocol:
✅ async def get(self, key: str) -> Any | None
✅ async def set(self, key: str, value: Any, ttl: int | None = None) -> None
✅ async def delete(self, key: str) -> None
✅ async def exists(self, key: str) -> bool
✅ async def get_many(self, keys: list[str]) -> dict[str, Any]
✅ async def set_many(self, items: dict[str, Any], ttl: int | None = None) -> None
✅ async def delete_pattern(self, pattern: str) -> int (returns count deleted)
✅ async def ttl(self, key: str) -> int | None (remaining TTL)

2. Define Port Interfaces > 2.9 Notification Protocol

Progress: 4/4 (100%)

Completed:

✅ Create src/framework_m/core/interfaces/notification.py
✅ Define NotificationProtocol:
✅ async def send_email(to: str, subject: str, body: str)
✅ async def send_sms(to: str, body: str)

2. Define Port Interfaces > 2.10 Search Protocol

Progress: 7/7 (100%)

Completed:

✅ Create src/framework_m/core/interfaces/search.py
✅ Define SearchResult model with items, total, facets, highlights
✅ Define SearchProtocol:
✅ async def index(self, doctype: str, doc_id: str, doc: dict) -> None
✅ async def delete_index(self, doctype: str, doc_id: str) -> None
✅ async def search(self, doctype: str, query: str, filters: dict | None = None, limit: int = 20, offset: int = 0) -> SearchResult
✅ async def reindex(self, doctype: str) -> int (returns count indexed)

2. Define Port Interfaces > 2.11 I18n Protocol

Progress: 4/4 (100%)

Completed:

✅ Create src/framework_m/core/interfaces/i18n.py
✅ Define I18nProtocol:
✅ async def translate(text: str, locale: str) -> str
✅ async def get_locale() -> str

3. Define Domain Layer > 3.1 Base DocType

Progress: 12/12 (100%)

Completed:

✅ Create src/framework_m/core/domain/base_doctype.py
✅ Define BaseDocType class inheriting from pydantic.BaseModel
✅ Add standard fields:
✅ name: Optional[str] (primary key, auto-generated if None)
✅ creation: datetime
✅ modified: datetime
✅ modified_by: Optional[str]
✅ owner: Optional[str]
✅ Add Meta nested class for metadata:
✅ layout: dict = {} (via get_layout())
✅ permissions: dict = {} (via get_permissions())
✅ Add class method get_doctype_name() -> str

3. Define Domain Layer > 3.2 Base Controller

Progress: 12/12 (100%)

Completed:

✅ Create src/framework_m/core/domain/base_controller.py
✅ Define BaseController[T] generic class
✅ Add lifecycle hook methods:
✅ async def validate(self, context: Any = None) -> None
✅ async def before_insert(self, context: Any = None) -> None
✅ async def after_insert(self, context: Any = None) -> None
✅ async def before_save(self, context: Any = None) -> None
✅ async def after_save(self, context: Any = None) -> None
✅ async def before_delete(self, context: Any = None) -> None
✅ async def after_delete(self, context: Any = None) -> None
✅ async def on_submit(self, context: Any = None) -> None
✅ async def on_cancel(self, context: Any = None) -> None

3. Define Domain Layer > 3.3 Mixins

Progress: 10/10 (100%)

Completed:

✅ Create src/framework_m/core/domain/mixins.py
✅ Define DocStatus enum:
✅ DRAFT = 0
✅ SUBMITTED = 1
✅ CANCELLED = 2
✅ Define SubmittableMixin class with:
✅ docstatus: DocStatus = DocStatus.DRAFT
✅ def is_submitted() -> bool
✅ def is_cancelled() -> bool
✅ def can_edit() -> bool (returns False if submitted)

4. Meta Registry

Progress: 13/13 (100%)

Completed:

✅ Create src/framework_m/core/registry.py
✅ Implement MetaRegistry as singleton
✅ Add storage dictionaries:
✅ _doctypes: Dict[str, Type[BaseDocType]]
✅ _controllers: Dict[str, Type[BaseController]]
✅ Implement methods:
✅ register_doctype(doctype_class, controller_class=None)
✅ get_doctype(name: str) -> Type[BaseDocType]
✅ get_controller(name: str) -> Type[BaseController] | None
✅ list_doctypes() -> list[str]
✅ discover_doctypes(package_name: str) (scans for BaseDocType subclasses)
✅ Load Order: Follows installed_apps list.
✅ Conflict Policy: Raise DuplicateDocTypeError if same name registered twice.

5. Port Implementation (Adapters) > 5.1 Dependency Injection

Progress: 5/5 (100%)

Completed:

✅ Create tests/core/test_container.py
✅ Write test for Container initialization and service resolution
✅ Create src/framework_m/core/container.py
✅ Implement Container class (using dependency_injector):
✅ Define Container class:

5. Port Implementation (Adapters) > 5.2 Provider Types

Progress: 6/6 (100%)

Completed:

✅ Understand provider types:
✅ Singleton - Single instance shared across app
✅ Factory - New instance each time
✅ Resource - For resources with lifecycle (db connections)
✅ Callable - For functions
✅ Dependency - For protocol injection

5. Port Implementation (Adapters) > 5.3 Configuration Provider

Progress: 1/1 (100%)

Completed:

✅ Setup configuration loading:

5. Port Implementation (Adapters) > 5.4 Wiring

Progress: 2/2 (100%)

Completed:

✅ Configure wiring for automatic injection:
✅ Use @inject decorator in functions:

5. Port Implementation (Adapters) > 5.5 Entrypoint Scanning for Overrides

Progress: 5/5 (100%)

Completed:

✅ Create override mechanism:
✅ Scan framework_m.overrides entrypoints
✅ Allow apps to override providers
✅ Example:
✅ Implement override loader:

5. Port Implementation (Adapters) > 5.6 Testing Support

Progress: 2/2 (100%)

Completed:

✅ Use override for testing:
✅ Reset overrides after tests:

6. Testing Setup

Progress: 8/8 (100%)

Completed:

✅ Create tests/ directory structure
✅ Setup conftest.py with fixtures:
✅ pytest_asyncio configuration
✅ Mock implementations of all protocols
✅ Write initial tests:
✅ Test MetaRegistry registration and retrieval
✅ Test BaseDocType field validation
✅ Test BaseController hook method existence

7. Documentation

Progress: 7/7 (100%)

Completed:

✅ Create README.md with:
✅ Project overview
✅ Installation instructions
✅ Basic usage example
✅ Create docs/ folder with:
✅ architecture.md - Hexagonal architecture explanation
✅ ports.md - List of all protocol interfaces (Repository, EventBus, Cache, Search, etc.)

Validation Checklist

Progress: 6/6 (100%)

Completed:

✅ All protocol interfaces are defined with proper type hints
✅ mypy --strict passes with no errors
✅ No infrastructure dependencies in core/ (no imports of litestar, sqlalchemy, redis)
✅ BaseDocType and BaseController are properly generic
✅ MetaRegistry can register and retrieve DocTypes
✅ All tests pass with pytest

Phase 02: DocType Engine & Database

Phase: 02 Objective: Implement the metadata engine that dynamically creates database tables from Pydantic models and provides generic CRUD operations. Status: 100% Complete

1. Schema Mapper (Pydantic → SQLAlchemy) > 1.1 Field Registry & Type Mapping

Progress: 15/15 (100%)

Completed:

✅ Create src/framework_m/adapters/db/field_registry.py
✅ Implement FieldRegistry class (Singleton):
✅ Start with standard types: str, int, float, bool, datetime, date, decimal, uuid, json
✅ Allow registration of custom types: register_type(pydantic_type, sqlalchemy_type)
✅ Define type mapping using Registry:
✅ str → String
✅ int → Integer
✅ float → Float
✅ bool → Boolean
✅ datetime → DateTime
✅ date → Date
✅ Decimal → Numeric
✅ UUID → UUID
✅ list[str] → JSON (portable, works on all databases)
✅ dict → JSON

1. Schema Mapper (Pydantic → SQLAlchemy) > 1.2 Schema Mapper Implementation

Progress: 19/19 (100%)

Completed:

✅ Create tests/adapters/db/test_schema_mapper.py
✅ Test mapping simple fields (str, int)
✅ Test mapping relationships
✅ Create SchemaMapper class
✅ Use FieldRegistry to look up SQLAlchemy types
✅ Implement create_table(model: Type[BaseDocType]) -> Table:
✅ Extract table name from model class name (lowercase)
✅ Iterate over model.model_fields
✅ Map each field to SQLAlchemy column
✅ Handle Optional types (nullable=True)
✅ Set id field as primary key (UUID, auto-generated)
✅ Set name field as unique index (for human-readable lookup)
✅ OCC: Add _version (Integer, default=0) if Meta.concurrency="optimistic"
✅ Return SQLAlchemy Table object
✅ Handle special field types:
✅ Relationships (ForeignKey) - detect by field name pattern *_id
✅ Enums - map to SQLAlchemy String type (database agnostic)
✅ Nested Pydantic models (list[BaseModel]) - store as JSON
✅ Child DocTypes (list[DocType]) - map to Relational Table with Foreign Key

1. Schema Mapper (Pydantic → SQLAlchemy) > 1.3 Table Registry

Progress: 5/5 (100%)

Completed:

✅ Create TableRegistry class to store created tables
✅ Add methods:
✅ register_table(doctype_name: str, table: Table)
✅ get_table(doctype_name: str) -> Table
✅ table_exists(doctype_name: str) -> bool

2. Database Connection Setup > 2.1 Connection Factory & Virtual DocTypes

Progress: 5/5 (100%)

Completed:

✅ Create src/framework_m/adapters/db/connection.py
✅ Implement ConnectionFactory (supports Multiple Bindings):
✅ Load db_binds from config (e.g., legacy, timescale)
✅ Create generic async engine (supports SQLite, PostgreSQL, MySQL, MariaDB, Oracle, MSSQL)
✅ Maintain map of engine_name -> AsyncEngine

2. Database Connection Setup > 2.2 Session Factory (SQL Support)

Progress: 3/3 (100%)

Completed:

✅ Add session factory:
✅ async def get_session(bind: str = "default") -> AsyncSession
✅ Support VirtualDocType (SQL Bind) by accepting bind_key from DocType Meta

2. Database Connection Setup > 2.3 Non-SQL Virtual DocTypes (Custom Repositories)

Progress: 7/7 (100%)

Completed:

✅ Create concept of RepositoryOverride:
✅ Allow DocType to define repository_class in Meta
✅ RepositoryFactory instantiates this instead of GenericRepository
✅ Create tests/adapters/db/test_repository_factory.py:
✅ Implement a Mock FileRepository that reads from JSON file
✅ Register it for a VirtualDoc
✅ Verify VirtualDoc.get() calls FileRepository.get()

3. Generic Repository Implementation

Progress: 9/9 (100%)

Completed:

✅ Create tests/adapters/db/test_generic_repository.py
✅ Define tests for CRUD operations (mock AsyncSession)
✅ Create src/framework_m/adapters/db/generic_repository.py
✅ Implement GenericRepository[T] class
✅ Constructor dependencies (session is NOT stored here):
✅ model: Type[T]
✅ table: Table
✅ controller_class: Type[BaseController] | None
✅ event_bus: EventBusProtocol | None (InMemoryEventBus for dev)

3. Generic Repository Implementation > 3.1 CRUD Operations

Progress: 49/49 (100%)

Completed:

✅ Implement async def get(session: AsyncSession, id: UUID) -> Optional[T]:
✅ Build SELECT query
✅ Filter deleted_at IS NULL (unless specific flag overrides)
✅ Execute with provided session (not stored session)
✅ Convert row to Pydantic model
✅ Return None if not found
✅ Implement async def get_by_name(session: AsyncSession, name: str) -> Optional[T]:
✅ Helper for looking up by human-readable name
✅ Implement async def save(session: AsyncSession, entity: T, version: int | None = None) -> T:
✅ Check if entity exists (by id)
✅ If new:
✅ Generate id (UUIDv7)
✅ Note: created_by / owner must be set by Controller/Service before calling save
✅ Call controller validate()
✅ Call controller before_create()
✅ Call controller before_save()
✅ Execute INSERT with provided session
✅ Call controller after_save()
✅ Call controller after_create()
✅ Emit Event: event_bus.publish(f"{doctype}.create", payload)
✅ If existing:
✅ Note: modified_by must be set by Controller/Service before calling save
✅ Call controller validate()
✅ Call controller before_save()
✅ OCC: If optimistic:
✅ UPDATE table SET ..., _version=_version+1 WHERE id=:id AND _version=:expected
✅ Token source: explicit version= arg, else the _version carried on the entity from a prior read
✅ No token at all → raise MissingVersionError (HTTP 428)
✅ If rowcount == 0 and row exists → raise StaleEntityError (HTTP 409, carries expected/current)
✅ If rowcount == 0 and row gone → raise VersionConflictError
✅ On success, return the advanced _version on the entity
✅ bulk_set_value intentionally bypasses OCC (documented exception)
✅ Else: Execute standard UPDATE
✅ Call controller after_save()
✅ Emit Event: event_bus.publish(f"{doctype}.update", payload)
✅ Return saved entity (caller calls uow.commit() when ready)
✅ Implement async def delete(session: AsyncSession, id: UUID, hard: bool = False) -> None:
✅ Load entity
✅ Call controller before_delete()
✅ If hard:
✅ Execute DELETE
✅ Else (Soft Delete):
✅ Update deleted_at = now()
✅ Call controller after_delete()
✅ Emit Event: event_bus.publish(f"{doctype}.delete", payload)
✅ Return (caller calls uow.commit() when ready)
✅ Implement async def list(session: AsyncSession, filters, limit, offset, order_by) -> Sequence[T]:
✅ Build SELECT query with filters
✅ Default filter: deleted_at IS NULL
✅ Apply pagination (limit, offset)
✅ Apply sorting (order_by)
✅ Execute query
✅ Convert rows to Pydantic models
✅ Return list

3. Generic Repository Implementation > 3.2 Lifecycle Hook Integration

Progress: 5/5 (100%)

Completed:

✅ Create helper method _call_hook(hook_name: str):
✅ Check if controller exists
✅ Check if hook method exists on controller
✅ Call hook method if present
✅ Handle exceptions and rollback on error

4. Migration System > 4.1 Alembic Integration

Progress: 8/8 (100%)

Completed:

✅ Initialize Alembic in project:
✅ Configure alembic.ini:
✅ Set database URL from environment
✅ Configure migration file location
✅ Create alembic/env.py:
✅ Import MetaRegistry
✅ Import all registered DocTypes
✅ Set target_metadata from SchemaMapper tables

4. Migration System > 4.2 Auto-Migration Detection

Progress: 11/11 (100%)

Completed:

✅ Create src/framework_m/adapters/db/migration.py
✅ Implement detect_schema_changes():
✅ Compare registered DocTypes with database schema
✅ Detect new tables
✅ Detect new columns
✅ Detect type changes
✅ Return list of changes
✅ Implement auto_migrate():
✅ Call detect_schema_changes()
✅ Generate Alembic migration if changes detected
✅ Apply migration automatically (dev mode only)

4. Migration System > 4.3 CLI Commands

Progress: 5/5 (100%)

Completed:

✅ Add migration commands to CLI:
✅ m migrate - run pending migrations
✅ m migrate create <name> - create new migration
✅ m migrate rollback - rollback last migration
✅ m migrate status - show migration status

5. Repository Factory

Progress: 9/9 (100%)

Completed:

✅ Create src/framework_m/adapters/db/repository_factory.py
✅ Implement RepositoryFactory class:
✅ create_generic_repository(doctype_name: str) -> GenericRepository
✅ Look up DocType from MetaRegistry
✅ Look up Table from TableRegistry
✅ Look up Controller from MetaRegistry
✅ Create and return GenericRepository instance
✅ Support event_bus parameter for domain events
✅ Support custom repository overrides for Virtual DocTypes

6. Engine, Session Factory & Unit of Work > 6.1 Engine & Session Factory Setup

Progress: 8/8 (100%)

Completed:

✅ Create src/framework_m/adapters/db/connection.py (ConnectionFactory)
✅ Implement create_engine(url: str) -> AsyncEngine
✅ Pool configuration (pool_size, max_overflow, timeout, recycle, pre_ping)
✅ Environment variable expansion (${VAR} syntax)
✅ Implement SessionFactory (returns AsyncSession context managers)
✅ Support multiple binds (for Virtual DocTypes with SQL binds)
✅ Configuration from ConnectionFactory
✅ Auto commit/rollback in context manager

6. Engine, Session Factory & Unit of Work > 6.2 Unit of Work (`UnitOfWork`)

Progress: 14/14 (100%)

Completed:

✅ Create tests/core/test_unit_of_work.py
✅ Test: UoW provides session
✅ Test: commit() persists changes
✅ Test: Exception causes rollback (no explicit rollback call needed)
✅ Test: Session is closed after __aexit__
✅ Create src/framework_m/core/unit_of_work.py
✅ Implement UnitOfWork context manager:
✅ __init__(session_factory: Callable[[], AsyncSession])
✅ session property with safety check
✅ async __aenter__() creates session
✅ async commit() calls session.commit()
✅ async rollback() for explicit rollback
✅ async __aexit__() with auto-rollback on exception
✅ Register UnitOfWorkFactory in DI container

6. Engine, Session Factory & Unit of Work > 6.3 Multi-Source Coordination (Outbox Pattern)

Progress: 17/17 (100%)

Completed:

✅ Create src/framework_m/core/domain/outbox.py
✅ Define OutboxEntry model:
✅ id: UUID
✅ target: str (e.g., "mongodb.audit_log", "api.payment_gateway")
✅ payload: dict
✅ status: str (pending, processed, failed)
✅ created_at: datetime
✅ processed_at: datetime | None
✅ error_message: str | None
✅ retry_count: int
✅ Create OutboxRepository (SQL-backed) in adapters/db/outbox_repository.py
✅ add(session, entry) - add entry in same transaction
✅ get_pending(session, limit) - get pending entries
✅ mark_processed(session, id) - mark as processed
✅ mark_failed(session, id, error) - mark as failed
✅ Document: Services write to Outbox in the same SQL transaction
✅ (Phase 04) Background worker processes Outbox entries

6. Engine, Session Factory & Unit of Work > 6.4 Startup Sequence (Schema Sync)

Progress: 7/7 (100%)

Completed:

✅ Create src/framework_m/adapters/db/__init__.py
✅ Implement startup sequence (called once at app boot, NOT per-request):
✅ Initialize database engine (NOT session)
✅ Discover all DocTypes via MetaRegistry
✅ Create/sync tables via SchemaMapper
✅ Register tables in TableRegistry
✅ Run auto-migration (if enabled)

7. Testing > 7.1 Unit Tests

Progress: 9/9 (100%)

Completed:

✅ Test SchemaMapper:
✅ Test type mapping for all supported types (test_schema_mapper.py)
✅ Test primary key creation (test_table_has_id_column_as_primary_key)
✅ Test nullable fields (TestSchemaMapperNullableFields)
✅ Test enum mapping (TestSchemaMapperEnums)
✅ Test GenericRepository:
✅ Test CRUD operations with mock data (test_generic_repository.py)
✅ Test lifecycle hook calls (TestControllerHooks)
✅ Test transaction rollback on error (TestTransactionRollback)

7. Testing > 7.2 Integration Tests

Progress: 13/13 (100%)

Completed:

✅ Setup testcontainers for PostgreSQL (skips if Docker unavailable)
✅ Create test DocType (IntegrationTestDoc / PostgresTestDoc)
✅ Test full flow across supported databases (database tests skip without Docker):
✅ Register DocType (test_register_doctype)
✅ Create table (test_table_created)
✅ Insert document (test_insert_document)
✅ Query document (test_query_document)
✅ Update document (test_update_document)
✅ Delete document (test_delete_document)
✅ Test migration:
✅ Add field to DocType
✅ Run auto-migration
✅ Verify column added to table

8. Error Handling

Progress: 10/10 (100%)

Completed:

✅ Create custom exceptions (core/exceptions.py):
✅ DocTypeNotFoundError
✅ ValidationError
✅ PermissionDeniedError
✅ DuplicateNameError
✅ RepositoryError, EntityNotFoundError, DatabaseError, IntegrityError
✅ Add error handling in repository (generic_repository.py):
✅ Catch SQLAlchemy errors (IntegrityError, OperationalError, SQLAlchemyError)
✅ Convert to domain exceptions (DuplicateNameError, DatabaseError, etc.)
✅ Log errors with context (using logger.error with extra dict)

9. Performance Optimizations

Progress: 13/13 (100%)

Completed:

✅ Add query result caching (phase-04):
✅ Cache get() results by ID
✅ Invalidate cache on save/delete
✅ Use Redis for distributed cache
✅ Add bulk operations (generic_repository.py):
✅ async def bulk_save(entities) - separates new/existing
✅ async def _bulk_insert(entities) - true SQLAlchemy bulk insert
✅ Uses insert().values([...]) for performance
✅ Add query optimization (add indexes after the API layer exists):
✅ Add indexes for common queries (schema_mapper: owner, creation, Meta.indexes)
✅ Add parent index for child tables (efficient joins)
✅ Use select_in_loading for relationships (load_children_for_parents method)
✅ Add query logging in dev mode (logger.debug for GET/LIST)

Validation Checklist

Progress: 6/6 (100%)

Completed:

✅ Can create tables from Pydantic models dynamically
✅ CRUD operations work with lifecycle hooks
✅ Migrations are generated and applied correctly
✅ All integration tests pass with real PostgreSQL (skips if Docker unavailable)
✅ No direct SQLAlchemy imports in domain layer
✅ Repository implements RepositoryProtocol correctly

Phase 03: API Layer & Authorization

Phase: 03 Objective: Expose DocTypes via HTTP with auto-generated CRUD endpoints, implement permission system with row-level security, and add RPC support. Status: 100% Complete

1. Litestar Application Setup

Progress: 14/14 (100%)

Completed:

✅ Create src/framework_m/adapters/web/app.py
✅ Initialize Litestar app:
✅ Configure CORS for development
✅ Add exception handlers
✅ Configure OpenAPI documentation
✅ Integrate with dependency-injector Container (wire modules)
✅ Middleware Discovery:
✅ Scan MiddlewareRegistry for registered App Middlewares.
✅ Allow Apps to inject ASGI Middleware (e.g. RateLimit, cors, Compression).
✅ APM & Tracing: Support OpenTelemetry, DataDog, Sentry via standard ASGI middleware.
✅ Support ordering (priority).
✅ Add application lifecycle:
✅ on_startup: Initialize Container, database, discover DocTypes
✅ on_shutdown: Close database connections, unwire Container

2. Authentication Middleware

Progress: 9/9 (100%)

Completed:

✅ Create src/framework_m/adapters/web/middleware.py
✅ Implement AuthMiddleware:
✅ Extract x-user-id header
✅ Extract x-roles header (comma-separated)
✅ Extract x-tenants header (optional, comma-separated)
✅ Create UserContext object
✅ Store in request.state.user
✅ Return 401 if headers missing (configurable)
✅ Add middleware to Litestar app (available via create_auth_middleware())

3. Permission System > 3.1 Permission Protocol Implementation

Progress: 7/7 (100%)

Completed:

✅ Create src/framework_m/adapters/auth/rbac_permission.py
✅ Implement RbacPermissionAdapter:
✅ Implement evaluate(request: PolicyEvaluateRequest) -> PolicyEvaluateResult
✅ Read permissions from DocType.Meta.permissions
✅ Match request.principal_attributes["roles"] against allowed roles
✅ CustomPermission DocType created (integration pending)
✅ Return PolicyEvaluateResult(authorized=True/False, decision_source=DecisionSource.RBAC)

3. Permission System > 3.2 Permission Configuration

Progress: 7/7 (100%)

Completed:

✅ Define permission structure in DocType:
✅ Implement has_permission():
✅ Check requires_auth — if False and no user, allow read
✅ Check if user has required role
✅ Check database overrides via PermissionLookupService (CustomPermission DocType)
✅ For object-level: check ownership or custom rules
✅ Return boolean

3. Permission System > 3.3 System Context (Elevated Operations)

Progress: 4/4 (100%)

Completed:

✅ Create src/framework_m/core/system_context.py
✅ Implement SystemContext:
✅ Usage in background jobs:
✅ Audit: All system operations logged with principal="system"

3. Permission System > 3.4 Row-Level Security (RLS)

Progress: 13/13 (100%)

Completed:

✅ Implement get_permitted_filters():
✅ For standard users: add WHERE owner = :user_id
✅ For admins: no filter (see all)
✅ For custom rules: via Meta.rls_field (completed by Section 3.5 Team-Based Access)
✅ Return dict of SQLAlchemy filters
✅ Update GenericRepository.list():
✅ list_for_user() method with RLS filtering
✅ apply_rls_filters() helper available
✅ Merge with user-provided filters
✅ Apply to SQL query
✅ Update GenericRepository.get():
✅ get_for_user() method with permission check
✅ Raise PermissionDeniedError if denied

3. Permission System > 3.5 Team-Based Access (Indie)

Progress: 5/5 (100%)

Completed:

✅ Add Meta.rls_field option:
✅ Update get_permitted_filters():
✅ If rls_field = "owner" (default): WHERE owner = :user_id
✅ If rls_field = "team": WHERE team IN :user_teams
✅ Support custom fields

3. Permission System > 3.6 Explicit Sharing (`DocumentShare`)

Progress: 6/6 (100%)

Completed:

✅ Create built-in DocumentShare DocType:
✅ Update get_permitted_filters() to include shares via get_rls_filters_with_shares():
✅ Add share management API:
✅ POST /api/v1/share — Create share
✅ DELETE /api/v1/share/{id} — Remove share
✅ GET /api/v1/{doctype}/{id}/shares — List shares for a doc

3. Permission System > 3.7 Indie Mode: Permission Conveniences

Progress: 14/14 (100%)

Completed:

✅ Create src/framework_m/core/decorators.py (if not exists)
✅ Implement @requires_permission(action: PermissionAction):
✅ Internally builds PolicyEvaluateRequest from self.user, self.doctype_name
✅ Calls permission.evaluate(request)
✅ Raises PermissionDeniedError if not authorized
✅ Works on Controller methods
✅ Add to BaseController:
✅ Add to BaseController:
✅ Verify: All auto-generated CRUD routes (Section 4) automatically call permission.evaluate():
✅ POST /api/v1/{doctype} → checks CREATE permission (verified via test)
✅ GET /api/v1/{doctype}/{id} → checks READ permission (verified via test)
✅ PUT /api/v1/{doctype}/{id} → checks WRITE permission (verified via test)
✅ DELETE /api/v1/{doctype}/{id} → checks DELETE permission (verified via test)
✅ No manual code required for indie devs using auto-CRUD (verified via test)

4. Auto-CRUD Router > 4.1 Meta Router Implementation

Progress: 6/6 (100%)

Completed:

✅ Create src/framework_m/adapters/web/meta_router.py
✅ Implement create_crud_routes(doctype_class: type[BaseDocType]):
✅ Check Opt-in: Verify DocType.Meta.api_resource is True. Skip if False.
✅ Generate 5 routes per DocType
✅ Inject repository and permission service
✅ Parse filters from JSON string via _parse_filters()

4. Auto-CRUD Router > 4.2 List Endpoint

Progress: 5/5 (100%)

Completed:

✅ Create GET /api/v1/{doctype}:
✅ Accept query parameters: limit, offset, filters, order_by
✅ Parse filters from JSON string
✅ Call repository list_entities() with RLS filters via apply_rls_filters()
✅ Return paginated response with metadata:

4. Auto-CRUD Router > 4.3 Create Endpoint

Progress: 6/6 (100%)

Completed:

✅ Create POST /api/v1/{doctype}:
✅ Validate request body against Pydantic model
✅ Check CREATE permission via _check_permission()
✅ Set owner to current user (data["owner"] = user_id)
✅ Call repository save()
✅ Return 201 with created document

4. Auto-CRUD Router > 4.4 Read Endpoint

Progress: 6/6 (100%)

Completed:

✅ Create GET /api/v1/{doctype}/{id}:
✅ Call repository get(id)
✅ Check READ permission
✅ Return 404 if not found (via NotFoundException)
✅ Return 403 if permission denied
✅ Return document

4. Auto-CRUD Router > 4.5 Update Endpoint

Progress: 7/7 (100%)

Completed:

✅ Create PUT /api/v1/{doctype}/{id}:
✅ Load existing document via repo.get()
✅ Check WRITE permission
✅ Check if submitted (deny if immutable) via _check_submitted()
✅ Merge changes via entity.model_copy(update=data)
✅ Call repository save()
✅ Return updated document

4. Auto-CRUD Router > 4.6 Delete Endpoint

Progress: 6/6 (100%)

Completed:

✅ Create DELETE /api/v1/{doctype}/{id}:
✅ Load document
✅ Check DELETE permission
✅ Check if submitted (deny if immutable) via _check_submitted()
✅ Call repository delete()
✅ Return 204 No Content

4. Auto-CRUD Router > 4.7 Router Registration

Progress: 5/5 (100%)

Completed:

✅ Implement create_meta_router():
✅ Get all DocTypes from MetaRegistry
✅ For each DocType with api_resource=True, call create_crud_routes()
✅ Register all routes with Litestar
✅ Return Router instance

5. RPC System > 5.1 Whitelisted Controller Methods

Progress: 13/13 (100%)

Completed:

✅ Create @whitelist decorator:
✅ Mark controller methods as publicly callable
✅ Store metadata on method (WHITELIST_ATTR)
✅ is_whitelisted() helper function
✅ get_whitelist_options() helper function
✅ Create POST /api/v1/rpc/{doctype}/{method}:
✅ Load DocType controller
✅ Check if method has @whitelist decorator
✅ Validate method exists
✅ Parse request body as method arguments
✅ Instantiate controller with document (if doc_id provided)
✅ Call method
✅ Return result

5. RPC System > 5.2 Arbitrary RPC Functions

Progress: 16/16 (100%)

Completed:

✅ Create @rpc decorator for standalone functions:
✅ Register function in RPC registry (RpcRegistry singleton)
✅ Store function path (e.g., my_app.api.send_email)
✅ is_rpc_function() helper
✅ get_rpc_options() helper
✅ Create POST /api/v1/rpc/fn/{dotted.path}:
✅ Parse dotted path (e.g., my_app.api.send_email)
✅ Look up function in registry
✅ Validate function has @rpc decorator
✅ Parse request body as function arguments
✅ Call function
✅ Return result
✅ Add permission check for RPC:
✅ Support @rpc(permission="custom_permission")
✅ Check permission before calling function
✅ Support @rpc(allow_guest=True) for public endpoints

6. Metadata API

Progress: 11/11 (100%)

Completed:

✅ Create GET /api/meta/{doctype}:
✅ Get DocType class from registry
✅ Generate JSON Schema from Pydantic model:
✅ Extract layout from Meta.layout
✅ Extract permissions from Meta.permissions
✅ Return:
✅ Add field metadata:
✅ Field labels (from Field(title=...)) - included in JSON Schema
✅ Help text (from Field(description=...)) - included in JSON Schema
✅ Validation rules (from Pydantic validators) - included in JSON Schema
✅ Field types and options - included in JSON Schema

7. OpenAPI Documentation

Progress: 10/10 (100%)

Completed:

✅ Configure Litestar OpenAPI:
✅ Set title, version, description
✅ Add authentication scheme (header-based: x-user-id, x-roles)
✅ Enable Swagger UI at /schema/swagger
✅ Enable ReDoc at /schema/redoc
✅ Enhance auto-generated docs:
✅ Add descriptions to CRUD endpoints (via docstrings)
✅ Add examples for request/response (via Pydantic models)
✅ Document RPC endpoints (auto-generated from route handlers)
✅ Add permission requirements to docs (via security schemes)

8. Error Handling

Progress: 7/7 (100%)

Completed:

✅ Create exception handlers:
✅ ValidationError → 400 Bad Request (validation_error_handler)
✅ PermissionDeniedError → 403 Forbidden (permission_denied_handler)
✅ DocTypeNotFoundError → 404 Not Found (not_found_handler)
✅ DuplicateNameError → 409 Conflict (duplicate_name_handler)
✅ Generic exceptions → 500 Internal Server Error (framework_error_handler)
✅ Return consistent error format:

9. Request/Response DTOs

Progress: 7/7 (100%)

Completed:

✅ Use Litestar DTOs for validation:
✅ Auto-generate from Pydantic models (via Litestar)
✅ Add DTO for list response (pagination): PaginatedResponse[T]
✅ Add DTO for error responses: ErrorResponse
✅ Implement response filtering:
✅ Exclude sensitive fields based on permissions (via field selection)
✅ Support field selection via query param: ?fields=name,title

10. Testing > 10.1 Unit Tests

Progress: 8/8 (100%)

Completed:

✅ Test permission system:
✅ Test role-based access (test_rbac_permission.py)
✅ Test RLS filter generation (test_rls.py)
✅ Test object-level permissions (test_object_level_permissions.py)
✅ Test RPC system:
✅ Test @whitelist decorator (test_whitelist_decorator.py)
✅ Test @rpc decorator (test_rpc_decorator.py)
✅ Test dotted path resolution (test_rpc_routes.py)

10. Testing > 10.2 Integration Tests

Progress: 15/15 (100%)

Completed:

✅ Test CRUD endpoints (test_api_endpoints.py):
✅ Test create with valid data
✅ Test create with invalid data (validation)
✅ Test create without permission (403)
✅ Test list with RLS (only see own docs)
✅ Test update immutable doc (covered by existing test_meta_router.py)
✅ Test delete with permission (covered by existing permission tests)
✅ Test RPC endpoints (test_api_endpoints.py, test_rpc_routes.py):
✅ Test whitelisted controller method
✅ Test arbitrary RPC function
✅ Test RPC with permissions
✅ Test with real HTTP client (test_real_http_client.py):
✅ Uses httpx.AsyncClient against real uvicorn server
✅ Tests: list endpoint, create endpoint, health check
✅ Manual testing scripts available in examples/ folder

Validation Checklist

Progress: 6/6 (100%)

Completed:

✅ All CRUD endpoints work with permissions (verified via test_meta_router.py, test_api_endpoints.py)
✅ RLS filters are applied correctly (verified via test_rls.py, test_object_level_permissions.py)
✅ RPC system works for controller methods and functions (verified via test_rpc_routes.py)
✅ OpenAPI docs are generated correctly (SwaggerUI + ReDoc enabled)
✅ Error handling returns proper status codes (verified via test_app.py)
✅ Integration tests pass with real HTTP requests (verified via test_real_http_client.py)

Edge Cases to Handle > Child Table Permissions

Progress: 4/4 (100%)

Completed:

✅ Define: Child tables inherit parent's RLS by default (Meta.is_child_table = True)
✅ Child rows are not independently permission-checked (api_resource = False)
✅ Loading parent loads all its children (no separate RLS query)
✅ Document: If independent child access needed, make it a separate DocType (tests in test_child_table_permissions.py)

Edge Cases to Handle > Link Field Data Leakage

Progress: 3/3 (100%)

Completed:

✅ When serializing a doc with Link fields, do NOT auto-include linked doc data (Link fields store UUID, not embedded objects)
✅ If linked doc data is needed, make a separate API call (permission checked) - documented pattern
✅ Optional: ?expand=customer param design documented (checks permission before expanding)

Edge Cases to Handle > Bulk Operations & RLS

Progress: 3/3 (100%)

Completed:

✅ GenericRepository.delete_many_for_user(filters) applies RLS
✅ GenericRepository.update_many_for_user(filters, data) applies RLS
✅ User can only bulk-modify docs they have access to (tests in test_bulk_operations_rls.py)

Phase 04: Background Jobs & Events

Phase: 04 Objective: Implement asynchronous job processing, event-driven architecture, and webhook system with pluggable backends (Redis, NATS, In-Memory). Status: 98% Complete

0. Backend Selection (Cross-Platform) > 0.2 Dev Mode (No External Deps)

Progress: 3/3 (100%)

Completed:

✅ Implement InMemoryJobQueue for local dev (asyncio.Queue).
✅ Implement InMemoryEventBus for local dev.
✅ Auto-detect: If NATS_URL not set, use in-memory.

1. Taskiq + NATS Integration (Primary) > 1.1 Taskiq Setup

Progress: 8/8 (100%)

Completed:

✅ Add dependencies: taskiq, taskiq-nats.
✅ Create src/framework_m/adapters/jobs/taskiq_adapter.py
✅ Configure Taskiq with NATS JetStream:
✅ Set NATS connection URL from environment (NATS_URL)
✅ Configure job timeout (default: 300s)
✅ Configure max retries (default: 3)
✅ Use PullBasedJetStreamBroker for reliable delivery
✅ Create broker instance:

1. Taskiq + NATS Integration (Primary) > 1.2 Job Queue Protocol Implementation

Progress: 7/7 (100%)

Completed:

✅ Implement TaskiqJobQueueAdapter:
✅ async def enqueue(job_name: str, deduplication_id: str | None = None, **kwargs) -> str:
✅ Get registered task
✅ Call .kiq() to enqueue
✅ Map deduplication_id to Nats-Msg-Id header for broker-level deduplication
✅ Return job ID

1. Taskiq + NATS Integration (Primary) > 1.3 Job Registration

Progress: 4/4 (100%)

Completed:

✅ Create job decorator @job:
✅ Implement job registry:
✅ Store all @broker.task decorated functions
✅ Auto-discover on startup

2. Worker Process > 2.1 Worker Implementation

Progress: 7/7 (100%)

Completed:

✅ Create src/framework_m/cli/worker.py
✅ Implement worker command:
✅ Worker startup:
✅ Initialize database connection
✅ Discover and register all jobs
✅ Start Taskiq worker
✅ Listen for jobs from NATS JetStream

2. Worker Process > 2.2 Job Context

Progress: 10/10 (100%)

Completed:

✅ Create job context for dependency injection:
✅ Provide database session
✅ Provide repository factory
✅ Provide event bus
✅ Pass to job functions
✅ Add job metadata:
✅ Job ID
✅ Enqueued time
✅ Started time
✅ User context (Must re-hydrate from job arguments if applicable)

3. Scheduler > 3.1 Cron Jobs

Progress: 3/3 (100%)

Completed:

✅ Define Pydantic ScheduleDefinition model for compile-time cron validation
✅ Implement InProcessTickerAdapter running stateless async cron check loops
✅ Implement NoopTickerAdapter for external orchestrator environments (e.g. Temporal)

3. Scheduler > 3.2 Dynamic Scheduler

Progress: 0/0 (100%)

Completed:

✅ Deferred to application level/plugins to keep core framework lightweight.

4. Event Bus > 4.1 NATS Event Bus Implementation

Progress: 9/9 (100%)

Completed:

✅ Create src/framework_m/adapters/events/nats_event_bus.py
✅ Implement NatsEventBusAdapter:
✅ async def publish(topic: str, event: BaseModel):
✅ Serialize event to JSON
✅ Publish to NATS JetStream subject
✅ async def subscribe(topic: str, handler: Callable):
✅ Subscribe to NATS JetStream subject
✅ Deserialize event
✅ Call handler function

4. Event Bus > 4.2 Event Types

Progress: 7/7 (100%)

Completed:

✅ Create base event class:
✅ Define standard events:
✅ DocCreated
✅ DocUpdated
✅ DocDeleted
✅ DocSubmitted
✅ DocCancelled

4. Event Bus > 4.3 Event Publishing from Lifecycle Hooks

Progress: 6/6 (100%)

Completed:

✅ Update GenericRepository:
✅ Inject event bus
✅ Publish DocCreated after insert
✅ Publish DocUpdated after update
✅ Publish DocDeleted after delete
✅ Add event publishing to controller hooks:

5. Webhook System > 5.1 Webhook DocType

Progress: 9/9 (100%)

Completed:

✅ Create Webhook DocType:
✅ name: str
✅ event: str - Event to listen to (e.g., "doc.created")
✅ doctype_filter: str | None - Filter by DocType
✅ url: str - Webhook endpoint URL
✅ method: str - HTTP method (POST, PUT)
✅ headers: dict - Custom headers
✅ enabled: bool
✅ secret: str - For signature verification

5. Webhook System > 5.2 Webhook Listener

Progress: 6/6 (100%)

Completed:

✅ Create src/framework_m/adapters/webhooks/listener.py
✅ Implement webhook listener:
✅ Subscribe to all events
✅ Load active webhooks from database
✅ Filter events by webhook configuration
✅ Trigger HTTP call to webhook URL

5. Webhook System > 5.3 Webhook Delivery

Progress: 9/9 (100%)

Completed:

✅ Implement webhook delivery:
✅ Use httpx for async HTTP calls
✅ Add signature header (HMAC-SHA256 with secret)
✅ Set timeout (default: 30s)
✅ Handle errors and retries
✅ Add retry logic:
✅ Retry on failure (exponential backoff)
✅ Max retries: 3
✅ Log failed deliveries

5. Webhook System > 5.4 Webhook Logs

Progress: 8/8 (100%)

Completed:

✅ Create WebhookLog DocType:
✅ webhook: str - Reference to Webhook
✅ event: str - Event that triggered
✅ status: str - Success/Failed
✅ response_code: int
✅ response_body: str
✅ error: str | None
✅ timestamp: datetime

6. Job Monitoring > 6.1 Job Status Tracking

Progress: 13/13 (100%)

Completed:

✅ Create JobLog DocType:
✅ job_id: str
✅ job_name: str
✅ status: str - Queued/Running/Success/Failed
✅ enqueued_at: datetime
✅ started_at: datetime | None
✅ completed_at: datetime | None
✅ error: str | None
✅ result: dict | None
✅ Update job execution to log status:
✅ Create log entry on enqueue
✅ Update on start
✅ Update on completion/failure

6. Job Monitoring > 6.2 Job Management API

Progress: 5/5 (100%)

Completed:

✅ Create endpoints:
✅ GET /api/v1/jobs - List jobs
✅ GET /api/v1/jobs/{job_id} - Get job status
✅ POST /api/v1/jobs/{job_id}/cancel - Cancel job
✅ POST /api/v1/jobs/{job_id}/retry - Retry failed job

6. Job Monitoring > 6.3 Job Monitor (Admin UI)

Progress: 3/6 (50%)

Completed:

✅ Option A: Native Desk UI (IMPLEMENTED):
✅ Job Log List View (auto-generated via api_resource=True).
✅ Shows Status, Error, Duration (duration_seconds property).
✅ Mount taskiq-dashboard / Worker Health (Superseded by native HealthServer HTTP probes and Desk JobLog UI).
✅ Sysadmin worker health debugging (Implemented via HealthServer /health & /ready probes + CLI --health-port).

4. Real-time Events (WebSockets) > 4.1 WebSocket Protocol (The Port)

Progress: 4/4 (100%)

Completed:

✅ Create src/framework_m/core/interfaces/socket.py
✅ Define SocketProtocol:
✅ async def broadcast(topic: str, message: dict)
✅ async def send_to_user(user_id: str, message: dict)

4. Real-time Events (WebSockets) > 4.2 NATS Backplane (The Adapter)

Progress: 7/7 (100%)

Completed:

✅ Create tests/adapters/socket/test_nats_socket.py
✅ Write tests using testcontainers-nats to verify Pub/Sub.
✅ Create src/framework_m/adapters/socket/nats_socket.py
✅ Implement NatsSocketAdapter:
✅ On Boot: Subscribe to framework.events.*.
✅ On Event: await nc.publish(subject, json.dumps(msg).encode()).
✅ On Receive (Sub): Forward to local connected Websocket clients (Connection Manager).

4. Real-time Events (WebSockets) > 4.3 WebSocket Endpoint (Litestar)

Progress: 8/8 (100%)

Completed:

✅ Create src/framework_m/adapters/web/socket.py
✅ Implement ConnectionManager:
✅ Store active connections Dict[UserId, List[WebSocket]].
✅ Add route WS /api/v1/stream:
✅ Authenticate user (Query param token).
✅ Upgrade connection.
✅ Register with ConnectionManager.
✅ Listen for client disconnect.

7. Testing > 7.1 Unit Tests

Progress: 6/6 (100%)

Completed:

✅ Test job registration:
✅ Test @job decorator
✅ Test job registry
✅ Test event bus:
✅ Test publish/subscribe
✅ Test event serialization

7. Testing > 7.2 Integration Tests

Progress: 13/13 (100%)

Completed:

✅ Test job execution:
✅ Enqueue job
✅ Verify job runs
✅ Check job result
✅ Test scheduled jobs:
✅ Register cron job
✅ Verify execution at scheduled time (use time mocking)
✅ Test webhooks:
✅ Create webhook
✅ Trigger event
✅ Verify HTTP call made
✅ Check webhook log
✅ Use testcontainers for NATS

8. CLI Commands

Progress: 5/5 (100%)

Completed:

✅ Add job management commands:
✅ m worker - Start worker process
✅ m job list - List registered jobs
✅ m job run <name> - Run job immediately
✅ m job status <id> - Check job status

Validation Checklist

Progress: 6/6 (100%)

Completed:

✅ Jobs can be enqueued and executed
✅ Scheduled jobs run at correct times
✅ Events are published and received
✅ Webhooks are triggered on events
✅ Job logs are created and updated
✅ Worker process starts and processes jobs

Phase 05: CLI & Developer Tools

Phase: 05 Objective: Build a comprehensive CLI tool (m) to replace Frappe's "bench", with commands for development, deployment, and management. Status: 97% Complete

1. CLI Framework Setup

Progress: 5/5 (100%)

Completed:

✅ Create src/framework_m/cli/main.py
✅ Use cyclopts for CLI framework (native async, better type support)
✅ Setup main CLI app:
✅ Add version command:
✅ Add help documentation for all commands

1. CLI Framework Setup > 1.1 Pluggable CLI Architecture (Entry Points)

Progress: 11/11 (100%)

Completed:

✅ Define Entry Point Group: framework_m.cli_commands
✅ Implement Plugin Loader in plugin_loader.py:
✅ Use importlib.metadata.entry_points(group="framework_m.cli_commands")
✅ Iterate over registered entry points
✅ If entry point is a cyclopts.App: app.command(plugin_app, name=ep.name)
✅ If entry point is a function: app.command(plugin_func, name=ep.name)
✅ Goal: 3rd party apps (and framework-m-studio) can add commands to m without modifying core.
✅ Customization Philosophy:
✅ Standard commands (start, test) are strict relays.
✅ For customization (e.g., start-mqtt, super-test): Users MUST register a custom command via entry points.
✅ Result: m remains a clean, standard runner.

2. Development Server (Uvicorn Relay) > 2.1 Start Command

Progress: 5/5 (100%)

Completed:

✅ Implement m prod:
✅ Implementation: Wrapper around uvicorn.
✅ Argument Forwarding: Uses cyclopts Parameter for options.
✅ Usage: m prod --workers 4 --log-level debug
✅ Value Add: Sets PYTHONPATH, auto-detects app, transparently passes args.

2. Development Server (Uvicorn Relay) > 2.2 Worker Command (Taskiq Relay)

Progress: 4/4 (100%)

Completed:

✅ Implement m worker:
✅ Implementation: Native Taskiq integration (better than wrapper).
✅ Options: --concurrency, --verbose.
✅ Usage: m worker --concurrency 8

2. Development Server (Uvicorn Relay) > 2.3 Studio Command (Litestar Relay)

Progress: 4/4 (100%)

Completed:

✅ Implement m studio:
✅ Implementation: Wrapper around uvicorn for Studio app.
✅ Options: --host, --port, --reload, --log-level.
✅ Usage: m studio --port 9000 --reload

3. Database Commands (Alembic Relay) > 3.1 Migration Commands

Progress: 5/5 (100%)

Completed:

✅ Implement m migrate:
✅ Implementation: Wrapper around MigrationManager (uses Alembic).
✅ Commands: migrate, create, rollback, status, history, init.
✅ Usage: m migrate create "add users" --autogenerate
✅ Value Add: Auto-detects schema changes, supports custom DB URL, env vars.

4. Scaffolding (Cruft/Cookiecutter Relay) > 4.1 New App (Starter Template)

Progress: 4/4 (100%)

Completed:

✅ Implement m new app:
✅ Implementation: Wrapper around cruft (cookiecutter).
✅ Options: --template, --checkout, --no-input.
✅ Usage: m new app myapp --checkout main

4. Scaffolding (Cruft/Cookiecutter Relay) > 4.2 New DocType (Jinja Relay)

Progress: 26/26 (100%)

Completed:

✅ Implement m new doctype <name>:
✅ Implementation: Template rendering (embedded templates).
✅ Output: doctype.py, controller.py, test_*.py, __init__.py.
✅ App Detection: --app > pyproject.toml > interactive prompt.
✅ CLI Parameter (Unattended/Automation):
✅ --app <app_name>: Explicit app target.
✅ If --app provided, use it directly (no prompt).
✅ Automation Friendly: Scripts can pass --app to avoid prompts.
✅ Auto-Detection from CWD:
✅ If --app not provided, attempt to detect from current directory.
✅ Check if CWD is inside an app's source tree.
✅ Look for pyproject.toml in CWD or parent directories.
✅ Read [project] name to determine app.
✅ Interactive Prompt (Default Fallback):
✅ If detection fails AND stdin is interactive (TTY):
✅ List installed apps from framework_m.apps entry points.
✅ Prompt user to select: "Which app? [my_app, other_app]".
✅ If stdin is NOT interactive (pipe/automation):
✅ Fail with clear error: "Cannot detect app. Use --app <name>.".
✅ Exit code 1 (enables CI/automation to catch failures).
✅ Implementation: check require_app() in new.py
✅ Scaffold Output Location:
✅ Files created in: {cwd}/doctypes/{doctype_name}/
✅ Structure: __init__.py, doctype.py, controller.py, test_*.py
✅ Templates:
✅ Embedded templates in new.py (no external Jinja2 files needed).

5. Testing & Quality (Standard Tool Relays) > 5.1 Test Relay

Progress: 5/5 (100%)

Completed:

✅ Implement m test:
✅ Command: pytest
✅ Options: --verbose, --coverage, -k.
✅ Usage: m test -k "user" --verbose
✅ Value Add: Configures PYTHONPATH.

5. Testing & Quality (Standard Tool Relays) > 5.2 Lint/Format Relay

Progress: 6/6 (100%)

Completed:

✅ Implement m lint:
✅ Command: ruff check . --fix
✅ Implement m format:
✅ Command: ruff format .
✅ Implement m typecheck:
✅ Command: mypy .

6. Configuration Management > 6.1 Config File

Progress: 4/4 (100%)

Completed:

✅ Create framework_config.toml structure:
✅ Supports [framework], [apps] sections
✅ Load/save with TOML format
✅ Auto-detect in CWD or parent directories

6. Configuration Management > 6.2 Config Commands

Progress: 2/2 (100%)

Completed:

✅ Implement m config:show: Display current config.
✅ Implement m config:set <key> <value>: Update config.

7. Utility Relays > 7.1 Console (IPython Relay)

Progress: 4/4 (100%)

Completed:

✅ Implement m console:
✅ Implementation: Wrapper around ipython or python -m asyncio.
✅ Value Add: Pre-imports framework, async support.
✅ Options: --no-ipython for plain Python.

7. Utility Relays > 7.2 System Info

Progress: 3/3 (100%)

Completed:

✅ Implement m info:
✅ Show versions (Framework, Python).
✅ --verbose for platform and service info.

7. Utility Relays > 7.3 Routes

Progress: 2/2 (100%)

Completed:

✅ Implement m routes:
✅ Points to OpenAPI/Swagger docs.

8. Build Commands > 8.1 Frontend Build

Progress: 3/3 (100%)

Completed:

✅ Implement m build (placeholder):
✅ Relay to frontend build system (see Phase 09).
✅ Detects package.json and runs npm run build.

8. Build Commands > 8.2 Docker Build

Progress: 4/4 (100%)

Completed:

✅ Implement m build:docker:
✅ Command: Wrapper around docker build.
✅ Value Add: Reads image name/tag from framework_config.toml.
✅ Options: --tag, --file, --no-cache, --push.

9. Pluggable Extensions

Progress: 1/4 (25%)

Completed:

✅ Plugin loader implemented (load_plugins())
✅ m studio (Implemented in framework_m_studio)
✅ m docs generate (Implemented in framework_m_studio)
✅ m codegen (Implemented in framework_m_studio)

10. CLI Entry Point

Progress: 2/2 (100%)

Completed:

✅ Configure pyproject.toml scripts: m = "framework_m.cli.main:app"
✅ Test installation (m --help) - 22 commands registered.

Phase 06: Built-in DocTypes & Core Features

Phase: 06 Objective: Implement essential built-in DocTypes (User, Role, Permission) and core features like file attachments, audit logging, and printing. Status: 99% Complete

1. Identity & Access Management (PIM) > 1.1 Identity Provider Protocol

Progress: 4/4 (100%)

Completed:

✅ Create IdentityProtocol (Interface):
✅ get_user(id: str) -> User
✅ authenticate(credentials) -> Token
✅ get_attributes(user) -> dict[str, Any] (ABAC replacement for get_roles)

1. Identity & Access Management (PIM) > 1.2 User DocTypes (Pluggable)

Progress: 8/8 (100%)

Completed:

✅ Mode A: Local (Indie):
✅ LocalUser (SQL Table): Stores email, password_hash, full_name.
✅ LocalIdentityAdapter: argon2 password hashing, JWT generation.
✅ Mode B: Federated (Enterprise):
✅ FederatedIdentityAdapter (Proxy):
✅ No SQL Table required.
✅ Hydrates from Auth Header (X-User-ID, X-Email) via hydrate_from_headers().
✅ UserPreferences DocType stores settings locally.

1. Identity & Access Management (PIM) > 1.3 User Controller (Adapter)

Progress: 4/4 (100%)

Completed:

✅ Implement UserManager:
✅ Delegates to configured IdentityProvider via IdentityProtocol.
✅ Abstraction layer: user = await user_manager.get(id).
✅ Additional create() method for Indie mode with password hashing.

1. Identity & Access Management (PIM) > 1.4 Auth API (Indie Mode)

Progress: 8/8 (100%)

Completed:

✅ Create GET /api/v1/auth/me:
✅ Returns current user context (or 401).
✅ Create POST /api/v1/auth/login (Indie Only):
✅ Accepts username, password.
✅ Validates against LocalUser via UserManager.authenticate().
✅ Returns JWT token.
✅ Create POST /api/v1/auth/logout:
✅ Returns success message (JWT auth is client-side logout).

1. Identity & Access Management (PIM) > 1.5 Authentication Strategies (Multi-Method)

Progress: 24/24 (100%)

Completed:

✅ Create AuthenticationProtocol (Interface):
✅ supports(headers: Mapping) -> bool — Can this strategy handle this request?
✅ authenticate(headers: Mapping) -> UserContext | None — Extract user from request.
✅ Built-in Strategies:
✅ SessionCookieAuth — Browser sessions via cookies.
✅ BearerTokenAuth — JWT/OAuth2 access tokens.
✅ ApiKeyAuth — X-API-Key header for scripts/integrations.
✅ HeaderAuth — Federated mode gateway header hydration.
✅ BasicAuth — HTTP Basic for CLI tools.
✅ Strategy Chain (AuthChain):
✅ Try each strategy in priority order.
✅ First successful match wins.
✅ Configurable order via framework_config.toml (create_auth_chain_from_config).
✅ API Key DocType (Indie Mode):
✅ ApiKey DocType:
✅ key_hash: str — Hashed API key (never store plaintext, excluded from serialization).
✅ user_id: str — Owner of the key.
✅ name: str — Human-readable label.
✅ scopes: list[str] — Optional permission scopes.
✅ expires_at: datetime | None — Expiration.
✅ last_used_at: datetime | None — Usage tracking.
✅ POST /api/v1/auth/api-keys — Create new key.
✅ DELETE /api/v1/auth/api-keys/{id} — Revoke key.
✅ GET /api/v1/auth/api-keys — List user's keys.

Progress: 26/27 (96%)

Completed:

✅ Option A: Built-in OAuth2 (Indie):
✅ Supported Providers (config structure ready):
✅ Google (well-known URLs configured)
✅ GitHub (well-known URLs configured)
✅ Microsoft (well-known URLs configured)
✅ Generic OIDC (any compliant provider via get_oidc_well_known)
✅ Configuration via framework_config.toml:
✅ Endpoints:
✅ GET /api/v1/auth/oauth/{provider}/start — Redirect to provider.
✅ GET /api/v1/auth/oauth/{provider}/callback — Handle callback. (Placeholder)
✅ Option B: Federated (Enterprise):
✅ Delegate to Keycloak / Authelia / Auth0 via HeaderAuth strategy.
✅ Framework receives headers only (X-User-ID, X-Email).
✅ Zero PII stored locally.
✅ SocialAccount DocType:
✅ provider: str — OAuth provider name.
✅ provider_user_id: str — Unique ID from provider.
✅ user_id: str — Links to local user.
✅ email: str | None — For lookup (optional).
✅ display_name: str — For UI display.
✅ One User can have multiple SocialAccounts.
✅ Passwordless Option (Requires Section 10: Email):
✅ POST /api/v1/auth/magic-link — Send email with login link.
✅ GET /api/v1/auth/magic-link/{token} — Verify and create session.
✅ MagicLinkCredentials added to identity protocol
✅ Token generation with HMAC-SHA256 signing
✅ Use authlib for OAuth2 flows (Implemented in framework_m_standard/adapters/auth/oauth.py).

1. Identity & Access Management (PIM) > 1.7 Session Management

Progress: 20/20 (100%)

Completed:

✅ Session Storage:
✅ Redis (Default): RedisSessionAdapter with JSON storage and TTL.
✅ Database (Fallback): DatabaseSessionAdapter for indie apps.
✅ Configurable via framework_config.toml:
✅ Session DocType (database backend):
✅ session_id: str — Unique session identifier.
✅ user_id: str — Links to user.
✅ expires_at: datetime
✅ ip_address: str | None
✅ user_agent: str | None
✅ SessionProtocol Interface:
✅ create() — Create new session
✅ get() — Retrieve by ID
✅ delete() — Remove session
✅ delete_all_for_user() — Logout all
✅ list_for_user() — List active sessions
✅ Session API:
✅ GET /api/v1/auth/sessions — List active sessions.
✅ DELETE /api/v1/auth/sessions/{id} — Revoke session.
✅ DELETE /api/v1/auth/sessions — Logout all.

1. Identity & Access Management (PIM) > 1.8 PII Handling (Indie Guidance)

Progress: 16/16 (100%)

Completed:

✅ Auth Mode Presets (AuthMode enum in core/pii.py):
✅ Minimal PII Pattern:
✅ LocalUser stores only:
✅ email (for login/notification)
✅ display_name (for UI)
✅ password_hash (only if mode = "local")
✅ Do NOT store by default (PII_SENSITIVE_FIELDS):
✅ Full legal name
✅ Phone number
✅ Address
✅ Date of birth
✅ Helper: is_sensitive_pii(field_name) to detect sensitive fields.
✅ Data Deletion (DeletionMode enum):
✅ DELETE /api/v1/auth/me — Delete user account (GDPR right to erasure).
✅ GET /api/v1/auth/me/data — Export user data (GDPR data portability).
✅ Configurable: Hard delete vs anonymize.

2. Tenancy & Attributes (Multi-Tenant Core) > 2.1 Tenant Protocol

Progress: 5/5 (100%)

Completed:

✅ Create TenantProtocol (Interface in core/interfaces/tenant.py):
✅ get_current_tenant() -> str
✅ get_tenant_attributes(tenant_id) -> dict
✅ TenantContext model for request context
✅ Configuration helpers: is_multi_tenant(), get_default_tenant_id()

2. Tenancy & Attributes (Multi-Tenant Core) > 2.2 Tenancy Adapters

Progress: 12/12 (100%)

Completed:

✅ Mode A: Single Tenant (Indie):
✅ ImplicitTenantAdapter (adapters/tenant.py):
✅ tenant_id="default" (configurable).
✅ attributes={"plan": "unlimited", "features": "*"}.
✅ get_context() returns TenantContext with is_default=True.
✅ Factory: create_tenant_adapter_from_headers() auto-selects.
✅ Mode B: Multi-Tenant (Enterprise):
✅ HeaderTenantAdapter (adapters/tenant.py):
✅ Extracts X-Tenant-ID from Gateway headers.
✅ Extracts X-Tenant-Attributes (JSON) for feature flags, plan.
✅ get_context() parses headers into TenantContext.
✅ Benefit: Feature toggling per tenant without DB lookup.

2. Tenancy & Attributes (Multi-Tenant Core) > 2.3 Attributes (The New Roles)

Progress: 7/7 (100%)

Completed:

✅ Goal: Replace "Roles" with refined "Attributes" (ABAC).
✅ Implementation (in UserContext):
✅ user.attributes (dict): Stores {"department": "sales", "level": 5, "roles": ["admin"]}.
✅ user.get_attribute(key, default) — Get attribute value.
✅ user.has_attribute(key, value) — Check attribute equality.
✅ Legacy Support: user.has_role("x") checks both roles list AND attributes.roles.
✅ is_system_user uses has_role("System") for consistency.

3. Permission Management (Pluggable) > 3.1 Permission Protocol

Progress: 7/7 (100%)

Completed:

✅ Create PermissionProtocol (core/interfaces/permission.py):
✅ PolicyEvaluateRequest - Stateless authorization request
✅ PolicyEvaluateResult - Result with authorized, decision_source, reason
✅ async def evaluate(request) -> PolicyEvaluateResult
✅ async def get_permitted_filters(...) -> dict - RLS support
✅ PermissionAction enum (read, write, create, delete, submit, cancel, amend)
✅ DecisionSource enum (rbac, abac, rebac, combo, custom)

3. Permission Management (Pluggable) > 3.2 Permission Adapters

Progress: 18/18 (100%)

Completed:

✅ Mode A: Standard (Indie) - RbacPermissionAdapter:
✅ Reads permissions from DocType.Meta.permissions.
✅ Checks CustomPermission rules from database.
✅ Role matching against principal_attributes["roles"].
✅ Admin role bypass (configurable).
✅ RLS via get_permitted_filters() using Meta.rls_field.
✅ Mode B: Remote Policy (Enterprise) - RemotePolicyAdapter:
✅ AuthorizationRequest model (Cedar-compatible format):
✅ Principal: user.id
✅ Action: doctype:action (e.g., Invoice:create)
✅ Resource: doctype:name (e.g., Invoice:INV-001)
✅ TenantID: user.tenant_id
✅ Context: doc.as_dict()
✅ PrincipalAttributes: user.attributes (Diet Claims)
✅ AuthorizationResponse model with allowed, reason, policy_id
✅ Calls configurable policy endpoint (Cedar/OPA)
✅ Fallback behavior on error (configurable)
✅ Configuration via [permissions.remote_policy] in framework_config.toml

4. File Management > 4.1 File DocType

Progress: 10/10 (100%)

Completed:

✅ Create src/framework_m/core/doctypes/file.py
✅ Define File DocType:
✅ name: str - File ID (inherited from BaseDocType)
✅ file_name: str - Original filename
✅ file_url: str - Storage URL
✅ file_size: int - Size in bytes
✅ content_type: str - MIME type
✅ attached_to_doctype: str | None
✅ attached_to_name: str | None
✅ is_private: bool - Private/public file (default: True)

4. File Management > 4.2 File Upload API

Progress: 9/9 (100%)

Completed:

✅ Create POST /api/v1/file/upload (adapters/web/file_routes.py):
✅ Accept multipart/form-data via UploadFile
✅ Validate file size (max 10MB default, configurable)
✅ Validate file type (configurable whitelist)
✅ Generate unique storage path (YYYY/MM/DD/random_filename)
✅ Create File DocType record
✅ Return file URL and metadata
✅ Configuration via [files] in framework_config.toml
✅ Response models: FileUploadResponse, FileDeleteResponse

4. File Management > 4.3 File Download API

Progress: 8/8 (100%)

Completed:

✅ Create GET /api/v1/file/{file_id}:
✅ Check permissions for private files (RLS on owner)
✅ Prepare streaming response from storage
✅ Set correct Content-Type header
✅ Set Content-Disposition header (inline/attachment)
✅ Support ?inline=true for browser display
✅ Create GET /api/v1/file/{file_id}/info:
✅ Return metadata without downloading content

4. File Management > 4.4 Storage Adapters

Progress: 13/13 (100%)

Completed:

✅ Implement LocalStorageAdapter (adapters/storage/local.py):
✅ Save files to local directory with atomic writes
✅ Organize by date (YYYY/MM/DD) via path parameter
✅ Generate unique filenames (handled by file_routes)
✅ Path traversal protection
✅ Async operations via aiofiles
✅ Implement S3StorageAdapter (adapters/storage/s3.py):
✅ Use aioboto3 for async S3 operations
✅ Presigned URLs for direct client uploads (PUT method)
✅ Multipart upload for large files (>25MB)
✅ Configure bucket and region from framework_config.toml
✅ Support S3-compatible services (MinIO, DigitalOcean Spaces)
✅ InMemoryStorageAdapter for testing (adapters/storage/memory.py)

5. Audit Logging (Pluggable) > 5.1 Audit Log Protocol

Progress: 5/5 (100%)

Completed:

✅ Create AuditLogProtocol (core/interfaces/audit.py):
✅ AuditEntry model (user, action, doctype, document_id, changes, metadata)
✅ async def log(...) - Record audit entry
✅ async def query(...) - Query with filters/pagination
✅ InMemoryAuditAdapter for testing

5. Audit Logging (Pluggable) > 5.2 Audit Adapters

Progress: 6/7 (86%)

Completed:

✅ Mode A: Database (Indie):
✅ DatabaseAuditAdapter: Writes to ActivityLog records.
✅ Uses ActivityLog DocType from 5.3
✅ Mode B: External (Enterprise):
✅ FileAuditAdapter: Writes to audit.log (JSONL) for Splunk/Filebeat.
✅ Prevents main DB bloat.
✅ ElasticAuditAdapter: Writes directly to Elasticsearch (Implemented in framework_m_standard/adapters/audit/elastic.py).

5. Audit Logging (Pluggable) > 5.3 Activity Log DocType (Indie Only)

Progress: 10/10 (100%)

Completed:

✅ Create src/framework_m/core/doctypes/activity_log.py
✅ Define ActivityLog DocType:
✅ user_id: str
✅ action: str (create, read, update, delete)
✅ doctype: str
✅ document_id: str
✅ timestamp: datetime
✅ changes: dict | None
✅ metadata: dict | None
✅ Immutable (no write/delete permissions)

5. Audit Logging (Pluggable) > 5.4 Activity Feed API

Progress: 4/4 (100%)

Completed:

✅ Create GET /api/v1/activity:
✅ List recent activities
✅ Filter by user, doctype, document, action
✅ Paginate results (limit, offset)

6. Error Logging > 6.1 Error Log DocType

Progress: 12/12 (100%)

Completed:

✅ Create src/framework_m/core/doctypes/error_log.py
✅ Define ErrorLog DocType:
✅ title: str - Short error description
✅ error_type: str - Exception class name
✅ error_message: str - Full error message
✅ traceback: str | None - Full stack trace
✅ request_url: str | None
✅ user_id: str | None
✅ request_id: str | None - For log correlation
✅ timestamp: datetime
✅ context: dict | None - Additional context
✅ Admin-only permissions (immutable)

6. Error Logging > 6.2 Error Logging Integration

Progress: 6/6 (100%)

Completed:

✅ Add global exception handler (adapters/web/error_handler.py):
✅ Catch all unhandled exceptions
✅ Create ErrorLog entry with full context
✅ Return user-friendly ErrorResponse (no sensitive data)
✅ Log to console in dev mode (configurable)
✅ create_error_handler() factory for Litestar

7. Printing & PDF Generation > 7.1 Print Format DocType

Progress: 10/10 (100%)

Completed:

✅ Create src/framework_m/core/doctypes/print_format.py
✅ Define PrintFormat DocType:
✅ name: str - Display name
✅ doctype: str - Target DocType
✅ template: str - Jinja2 template path
✅ is_default: bool
✅ css: str | None - Custom CSS
✅ header_html: str | None, footer_html: str | None
✅ page_size: str (A4, Letter, etc.)
✅ orientation: str (portrait, landscape)

7. Printing & PDF Generation > 7.2 Jinja Print Adapter

Progress: 8/8 (100%)

Completed:

✅ Create src/framework_m/adapters/print/jinja_adapter.py
✅ Implement JinjaPrintAdapter:
✅ async def render_html(doc, template) -> str
✅ Load Jinja2 template
✅ Render with document context
✅ Return HTML string
✅ async def render_html_string(template_string, doc) -> str
✅ Custom filters: currency, date, datetime

7. Printing & PDF Generation > 7.3 Gotenberg PDF Adapter

Progress: 8/8 (100%)

Completed:

✅ Create src/framework_m/adapters/print/gotenberg_adapter.py
✅ Implement GotenbergPrintAdapter:
✅ async def html_to_pdf(html, page_size, orientation, ...) -> bytes
✅ Send HTML to Gotenberg service
✅ Return PDF bytes
✅ async def is_available() -> bool - Health check
✅ GotenbergConfig with URL, timeout, page size, margins
✅ Page sizes: A4, Letter, Legal, A3, A5

7. Printing & PDF Generation > 7.4 Print API

Progress: 9/9 (100%)

Completed:

✅ Create GET /api/v1/print/{doctype}/{id}:
✅ Query parameters:
✅ format - pdf/html (default: pdf)
✅ print_format - Custom format name
✅ Load document (TODO: integrate with repository - phase 10)
✅ Check read permission (TODO: integrate with permission - phase 10)
✅ Render using JinjaPrintAdapter → GotenbergPrintAdapter
✅ Return PDF or HTML with appropriate headers
✅ Fallback to HTML if Gotenberg unavailable

8. System Settings > 8.1 System Settings DocType

Progress: 11/11 (100%)

Completed:

✅ Create src/framework_m/core/doctypes/system_settings.py
✅ Define SystemSettings DocType (singleton):
✅ name: str - Always "System Settings" (frozen)
✅ app_name: str - Application display name
✅ timezone: str - Default timezone
✅ date_format: str - Date format string
✅ time_format: str - Time format string
✅ language: str - Default language code
✅ enable_signup: bool - Allow registration
✅ session_expiry: int - Session timeout in minutes
✅ maintenance_mode: bool - Admin-only access

8. System Settings > 8.2 Settings API

Progress: 5/5 (100%)

Completed:

✅ Create GET /api/v1/settings:
✅ Return system settings via SettingsResponse
✅ Cache aggressively (5 min Cache-Control header)
✅ invalidate_settings_cache() helper
✅ set_settings_cache() helper

9. Notification System > 9.1 Notification DocType

Progress: 13/13 (100%)

Completed:

✅ Create src/framework_m/core/doctypes/notification.py
✅ Define Notification DocType:
✅ user_id: str - Recipient
✅ subject: str
✅ message: str
✅ notification_type: str (info, success, warning, error, etc.)
✅ read: bool
✅ doctype: str | None - Related DocType
✅ document_id: str | None
✅ timestamp: datetime
✅ from_user: str | None
✅ metadata: dict | None
✅ RLS on user_id field

9. Notification System > 9.2 Notification API

Progress: 9/9 (100%)

Completed:

✅ Create GET /api/v1/notifications:
✅ List user's notifications
✅ Filter by read/unread
✅ Create PATCH /api/v1/notifications/{id}/read - Mark as read
✅ Create DELETE /api/v1/notifications/{id} - Delete notification
✅ Create WebSocket endpoint for real-time notifications:
✅ WS /api/v1/notifications/stream - Real-time notification stream
✅ push_notification(user_id, notification) - Push to connected clients
✅ create_notification_websocket_router() factory

10. Email Integration > 10.1 Email Queue DocType

Progress: 20/20 (100%)

Completed:

✅ Create src/framework_m/core/doctypes/email_queue.py
✅ Define EmailQueue DocType:
✅ to: list[str] - Recipients
✅ cc/bcc: list[str] | None
✅ subject: str
✅ body: str - HTML body
✅ text_body: str | None - Plain text alt
✅ status: str - Queued/Sending/Sent/Failed/Cancelled
✅ priority: str - low/normal/high
✅ error: str | None
✅ retry_count/max_retries: int
✅ queued_at/sent_at: datetime
✅ reference_doctype/reference_id - for tracking
✅ Port-Adapter Pattern:
✅ EmailQueueProtocol (interface/port)
✅ DirectEmailQueueAdapter (default - dispatches via job queue)
✅ InMemoryEmailQueueAdapter (testing)
✅ queue_email() helper function
✅ configure_email_queue() for swapping adapters
✅ Ready for NotificationServiceAdapter integration

10. Email Integration > 10.2 Email Sender

Progress: 13/13 (100%)

Completed:

✅ Create email sender protocol and adapters:
✅ EmailSenderProtocol (interface/port)
✅ SMTPEmailSender - SMTP delivery adapter
✅ LogEmailSender - Console logging (development)
✅ SMTPConfig - Configuration dataclass
✅ Create background job EmailProcessor:
✅ process_queue() - Process pending emails
✅ process_single(queue_id) - Process single email
✅ Update EmailQueue status
✅ Retry logic (placeholder)
✅ Add helper function:
✅ queue_email() - Queue for sending
✅ send_queued_email() - Process single email

11. Testing > 11.1 Unit Tests

Progress: 5/5 (100%)

Completed:

✅ Test User DocType validation (core/doctypes/test_user.py, core/services/test_user_manager.py)
✅ Test Role assignment (adapters/auth/test_rbac_permission.py)
✅ Test Permission checking (8 permission test files)
✅ Test File upload/download (core/test_file.py, adapters/web/test_file_routes.py)
✅ Test Print rendering (core/test_print_format.py, adapters/print/test_*.py)

11. Testing > 11.2 Integration Tests

Progress: 5/5 (100%)

Completed:

✅ Test user creation and login (integration/test_api_endpoints.py)
✅ Test permission enforcement (core/test_object_level_permissions.py)
✅ Test file storage (local and S3) (adapters/storage/test_local.py, adapters/storage/test_s3.py)
✅ Test PDF generation with Gotenberg (adapters/print/test_gotenberg_adapter.py)
✅ Test email sending (adapters/email/test_sender_processor.py)

Validation Checklist

Progress: 6/6 (100%)

Completed:

✅ User and Role management works
✅ Permissions are enforced correctly
✅ Files can be uploaded and downloaded
✅ Audit logs are created
✅ PDFs can be generated
✅ Emails can be sent

Phase 07: Studio (Code Generation UI)

Phase: 07 Objective: Build a visual DocType builder (framework-m-studio) that generates Python code, running as a Litestar app. Status: 93% Complete

1. Packaging Strategy (Runtime vs DevTools) > 1.1 framework-m (Runtime)

Progress: 4/4 (100%)

Completed:

✅ Goal: Production-grade kernel.
✅ Content: API Server, ORM, DocType Engine, Basic CLI (start, migrate, worker).
✅ Dependencies: litestar, sqlalchemy, pydantic, cyclopts.
✅ Excluded: Code generators, Studio UI, Heavy formatting/linting libs.

1. Packaging Strategy (Runtime vs DevTools) > 1.2 framework-m-studio (DevTools)

Progress: 4/4 (100%)

Completed:

✅ Goal: Developer Productivity & SaaS Component.
✅ Content: Studio UI, Code Generators, Visual Editors.
✅ Dependencies: framework-m, libcst, jinja2.
✅ Integration: Plugs into m CLI via entry points (m codegen).

2. Studio Backend (Litestar App) > 1. Project Structure (Monorepo)

Progress: 6/6 (100%)

Completed:

✅ Create apps/studio/ in the workspace
✅ Initialize apps/studio/pyproject.toml:
✅ Add dependency: framework-m = { workspace = true }
✅ Add dependency: libcst (for code modification)
✅ Add dependency: jinja2 (for code generation)
✅ Add code generation endpoints

2. Studio Backend (Litestar App) > 1.2 File System API

Progress: 14/14 (100%)

Completed:

✅ Create GET /studio/api/doctypes:
✅ Scan project for *.py files containing DocType classes
✅ Return list of DocTypes with metadata
✅ Create GET /studio/api/doctype/{name}:
✅ Read DocType file
✅ Parse with LibCST
✅ Return structured JSON
✅ Create POST /studio/api/doctype/{name}:
✅ Accept DocType schema JSON
✅ Generate Python code
✅ Write to file system
✅ Create DELETE /studio/api/doctype/{name}:
✅ Delete DocType file
✅ Delete Controller file (if exists)

2. LibCST Code Transformer > 2.1 Parser

Progress: 7/7 (100%)

Completed:

✅ Create src/framework_m_studio/codegen/parser.py
✅ Implement parse_doctype(file_path: str) -> dict:
✅ Use LibCST to parse Python file
✅ Extract class definition
✅ Extract fields with types and defaults
✅ Extract Config metadata
✅ Return structured dict

2. LibCST Code Transformer > 2.2 Generators (The Strategy)

Progress: 8/8 (100%)

Completed:

✅ Creation (Scaffolding):
✅ Use jinja2 templates (shared with CLI m new doctype).
✅ Goal: Generate clean, standard Python code from scratch.
✅ Implement generate_doctype_source(schema: dict) -> str.
✅ Mutation (Transformer):
✅ Use LibCST to parse and modify existing files.
✅ Goal: Add/Edit fields while preserving comments and custom methods.
✅ Implement update_doctype_source(source: str, schema: dict) -> str.

2. LibCST Code Transformer > 2.3 Test Generator

Progress: 6/6 (100%)

Completed:

✅ Create src/framework_m_studio/codegen/test_generator.py
✅ Implement generate_test(schema: dict) -> str:
✅ Generate test_{doctype}.py
✅ Import pytest and DocType
✅ Generate basic CRUD test (Create, Read, Update, Delete)
✅ Generate validation failure test (if required fields exist)

2. LibCST Code Transformer > 2.4 Transformer

Progress: 12/12 (100%)

Completed:

✅ Create src/framework_m_studio/codegen/transformer.py
✅ Implement update_doctype(file_path: str, schema: dict):
✅ Parse existing file with LibCST
✅ Locate DocType class
✅ Update/add/remove fields
✅ Preserve comments and custom methods
✅ Write back to file
✅ Handle edge cases:
✅ Field renamed (detect and update)
✅ Field type changed
✅ Field deleted (remove from class)
✅ Custom methods preserved

3. Studio Frontend > 3.1 React App Setup (Refine Stack)

Progress: 8/8 (100%)

Completed:

✅ Create apps/studio/studio_ui/ directory
✅ Initialize Refine app:
✅ Install dependencies per ADR-0005:
✅ Create Framework M Data Provider (src/providers/dataProvider.ts):
✅ Configure API base URLs (per ADR-0005):
✅ Read from window.__FRAMEWORK_CONFIG__ or env vars
✅ Support same-origin, subdomain, and CDN scenarios
✅ Default to /studio/api for Studio endpoints

3. Studio Frontend > 3.2 DocType List View (Refine Resource)

Progress: 5/5 (100%)

Completed:

✅ Create pages/doctypes/list.tsx:
✅ Use useList hook from @refinedev/core to fetch DocTypes
✅ Display with @tanstack/react-table + Tailwind styling
✅ Add search/filter with useTable globalFilter
✅ Add "New DocType" button → navigates to /doctypes/create

3. Studio Frontend > 3.3 DocType Editor (Refine Form)

Progress: 5/5 (100%)

Completed:

✅ Create pages/doctypes/edit.tsx:
✅ Use useOne/useUpdate/useCreate hooks for data
✅ Left panel: Field list with @dnd-kit/sortable
✅ Right panel: Field properties form (name, type, required, default, description)
✅ Top bar: DocType name input + Save button

3. Studio Frontend > 3.4 Field Editor (Uniforms Schema-Driven)

Progress: 5/5 (100%)

Completed:

✅ Create components/FieldEditor.tsx:
✅ Render with Vendored Uniforms using JSON Schema for field properties
✅ Custom Tailwind widgets: TextWidget, SelectWidget, CheckboxWidget, TextareaWidget
✅ Dynamic "Validators" collapsible for min/max length, pattern, etc.
✅ Live preview of generated Python type annotation

3. Studio Frontend > 3.5 Drag & Drop

Progress: 4/4 (100%)

Completed:

✅ Implement field reordering:
✅ Use @dnd-kit/core + @dnd-kit/sortable
✅ Allow dragging fields to reorder
✅ Update schema on drop (via arrayMove)

3. Studio Frontend > 3.6 Code Preview (Monaco Editor)

Progress: 6/6 (100%)

Completed:

✅ Create components/CodePreview.tsx:
✅ Use @monaco-editor/react with language="python"
✅ Read-only mode (readOnly: true)
✅ Theme: vs-dark
✅ Auto-update on schema change (memoized generation)
✅ Copy button for generated code

3. Studio Frontend > 3.7 Save Flow

Progress: 5/5 (100%)

Completed:

✅ Implement save functionality:
✅ Validate schema (name required)
✅ Call backend API via useCreate/useUpdate
✅ Navigate back to list on success
✅ Loading state on save button

3. Studio Frontend > 3.8 Visual Data Modeling (Mind Map)

Progress: 5/5 (100%)

Completed:

✅ Implement ERD/Mind Map View:
✅ Central Node: Current DocType (highlighted)
✅ Connected Nodes: Related DocTypes via Link fields
✅ Visual Action: Drag to connect nodes → Creates Link Field
✅ Using @xyflow/react (React Flow v12)

3. Studio Frontend > 3.9 Layout Designer (Grid System)

Progress: 5/5 (100%)

Completed:

✅ Implement Visual Layout Drag & Drop:
✅ Define Sections with title and Columns (1/2/3 cols selector)
✅ Drag fields from palette into grid cells
✅ Real-time form preview toggle
✅ Add/delete sections

3. Studio Frontend > 3.10 Module Explorer

Progress: 5/5 (100%)

Completed:

✅ Implement Module Scanner:
✅ Group DocTypes by directory/app module (tree structure)
✅ Tree view navigation in Sidebar
✅ Expand/collapse all, search filter
✅ Click to navigate to DocType editor

4. Studio Cloud Mode (Ephemeral & Git-Backed) > 4.2 GitOps Integration (FluxCD/ArgoCD)

Progress: 2/2 (100%)

Completed:

✅ Implement "Pull Latest" Button (POST /workspace/{id}/pull).
✅ Show "Updates Available" indicator (POST /workspace/{id}/check-updates).

4. Studio Cloud Mode (Ephemeral & Git-Backed) > 4.3 Git Adapter (Data Agnostic)

Progress: 11/11 (100%)

Completed:

✅ Create src/framework_m_studio/git/protocol.py - GitAdapterProtocol (Port)
✅ Create src/framework_m_studio/git/adapter.py - GitAdapter (Adapter)
✅ Implement GitAdapter:
✅ Wraps standard git binary via asyncio subprocess.
✅ clone(repo_url, auth): Clone to temp dir.
✅ commit(message): Stage and commit changes.
✅ push(branch): Push to remote.
✅ pull(): Pull latest changes.
✅ create_branch(name): Create new branch.
✅ get_status(): Get workspace status.
✅ Unit tests in tests/test_git_adapter.py (12 tests)

4. Studio Cloud Mode (Ephemeral & Git-Backed) > 4.4 GitHub Provider

Progress: 9/9 (100%)

Completed:

✅ Create src/framework_m_studio/git/github_provider.py
✅ Implement GitHubProvider:
✅ Use GitHub API for operations (REST API with urllib)
✅ Support personal access tokens (Bearer auth)
✅ Handle authentication (GitHubAuthError)
✅ Create pull requests
✅ List pull requests
✅ Validate tokens
✅ Unit tests in tests/test_github_provider.py (9 tests)

4. Studio Cloud Mode (Ephemeral & Git-Backed) > 4.5 Generic Git Provider

Progress: 2/2 (100%)

Completed:

✅ Support HTTPS and SSH (via git CLI)
✅ Work with any Git server (generic implementation)

4. Studio Cloud Mode (Ephemeral & Git-Backed) > 4.6 Workspace Management

Progress: 7/7 (100%)

Completed:

✅ Create src/framework_m_studio/workspace.py
✅ Implement workspace lifecycle:
✅ Clone repo to temp directory
✅ Track workspace sessions
✅ Clean up old workspaces (TTL-based)
✅ Handle concurrent edits (asyncio lock)
✅ Unit tests in tests/test_workspace.py (12 tests)

4. Studio Cloud Mode (Ephemeral & Git-Backed) > 4.7 Cloud Studio API

Progress: 12/12 (100%)

Completed:

✅ Create POST /studio/api/workspace/connect:
✅ Accept repo URL and token
✅ Clone repository
✅ Return workspace ID
✅ Create POST /studio/api/workspace/{id}/commit:
✅ Commit changes
✅ Push to branch
✅ Return commit SHA
✅ Create GET /studio/api/workspace/{id}/status: Git status
✅ Create POST /studio/api/workspace/{id}/pull: Pull latest
✅ Create DELETE /studio/api/workspace/{id}: Cleanup workspace
✅ Create GET /studio/api/workspace/sessions: List all sessions

5. Studio CLI Command > 5.1 SPA Packaging & Serving (Path Prefix Architecture)

Progress: 20/21 (95%)

Completed:

✅ Build Process:
✅ pnpm run build in studio_ui outputs to src/framework_m_studio/static/ (vite.config.ts).
✅ pyproject.toml includes framework_m_studio/static/**/*.
✅ Serving Strategy (Litestar) (in app.py):
✅ UI Routes: Serve /studio/ui/* → React SPA (HTML pages)
✅ API Routes: Serve /studio/api/* → JSON endpoints
✅ Assets: Serve /studio/ui/assets/* as static files (Cached)
✅ SPA Catch-all: /studio/ui/{path:path} → returns index.html for client-side routing
✅ Redirect: /studio → /studio/ui/ for convenience
✅ Content-Type Fix: All responses use explicit media_type to prevent download issues
✅ Implementation: serve_spa() and _get_spa_response() functions in app.py
✅ Implement m studio (in framework-m-studio package):
✅ Register CLI command via entry point:
✅ Start uvicorn server on port 9000 (default)
✅ Serve Studio UI
✅ Options:
✅ --port - Custom port
✅ --host - Custom host
✅ --reload - Development mode
✅ --cloud - Enable cloud mode
✅ Open browser automatically (webbrowser.open_new_tab in studio CLI).

6. DevTools CLI (Extensions) > 6.1 Documentation Generator (`m docs generate`)

Progress: 6/6 (100%)

Completed:

✅ Dependencies: Uses stdlib urllib, json, subprocess (no external deps required).
✅ Command: m docs generate:
✅ API Reference: Generates markdown from DocTypes in docs_generator.py.
✅ OpenAPI Export: Exports openapi.json from URL (--openapi-url).
✅ Site Build: Runs mkdocs build if available (--build-site).
✅ Unit tests in tests/test_docs_generator.py (10 tests)

6. DevTools CLI (Extensions) > 6.2 Client SDK Generator (`m codegen client`)

Progress: 8/8 (100%)

Completed:

✅ Dependencies: Uses stdlib only (no external deps required).
✅ Command: m codegen client:
✅ Flow:
✅ Arguments:
✅ TypeScript: Generates interfaces + fetch client in sdk_generator.py.
✅ Python: Generates Pydantic models in sdk_generator.py.
✅ Unit tests in tests/test_codegen.py (6 tests)
✅ CLI tests updated in tests/test_cli.py

7. Field Type Library > 7.1 Supported Field Types

Progress: 13/13 (100%)

Completed:

✅ Implement field type definitions:
✅ Text (str)
✅ Number (int, float)
✅ Checkbox (bool)
✅ Date (date)
✅ DateTime (datetime)
✅ Select (enum)
✅ Link (foreign key)
✅ Table (child table)
✅ Level 1: Grid/Table view
✅ Level 2+: Drill-down pattern (Button -> Drawer/Dialog) for infinite nesting
✅ JSON (dict)
✅ File (file upload)

7. Field Type Library > 7.2 Custom Field Type Discovery

Progress: 5/5 (100%)

Completed:

✅ Dynamic Loading: Studio reads available types from FieldRegistry at runtime.
✅ API Endpoint: GET /studio/api/field-types
✅ Returns: Built-in types + Types registered by installed apps.
✅ Each type includes: name, pydantic_type, sqlalchemy_type, ui_component (optional).
✅ UI Integration: Field Type Selector in DocType Editor dynamically populates from this API.

7. Field Type Library > 7.3 Custom UI Components (Client-Side)

Progress: 6/6 (100%)

Completed:

✅ Goal: Allow apps to register custom React components for their field types.
✅ Registration (App-Side):
✅ Discovery (Studio-Side):
✅ Shadow Build (Phase 09) loads all app frontend/ plugins.
✅ Studio UI checks window.__STUDIO_FIELD_COMPONENTS__ map.
✅ Falls back to default <input> if no custom component.

7. Field Type Library > 7.5 Live Preview & Sandbox Mode

Progress: 0/11 (0%)

✅ Feature branch publishing and branch selector (workspace_manager.py + Studio UI).
✅ Preview Tab & Sandbox: Interactive mock data generation using @faker-js/faker, AutoForm, AutoTable, in-memory schema evaluation (SandboxPreview.tsx).

7. Field Type Library > 7.6 Sandbox: Interactive CRUD Simulation

Progress: 11/11 (100%)

Completed:

✅ Goal: Let users test the full UX before committing the DocType.
✅ Features:
✅ New: Create a new mock document. Test mandatory field validation.
✅ Update: Edit an existing mock document. See form behavior.
✅ Delete: Delete mock document. See confirmation dialogs.
✅ List (Few): Show 5 rows. Test basic list rendering.
✅ List (Many): Paginate 100+ mock rows. Test performance.
✅ Filter: Apply filters and see results change.
✅ **Validation

Overview
Phase 01: Core Kernel & Interfaces
Phase 02: DocType Engine & Database
Phase 03: API Layer & Authorization
Phase 04: Background Jobs & Events
Phase 05: CLI & Developer Tools
Phase 06: Built-in DocTypes & Core Features
Phase 07: Studio (Code Generation UI)

Overview​

Phase 01: Core Kernel & Interfaces​

1. Project Initialization​

2. Define Port Interfaces > 2.1 Repository Protocol​

2. Define Port Interfaces > 2.2 Event Bus Protocol​

2. Define Port Interfaces > 2.3 Auth Context Protocol​

2. Define Port Interfaces > 2.4 Storage Protocol​

2. Define Port Interfaces > 2.5 Job Queue Protocol​

2. Define Port Interfaces > 2.6 Permission Protocol​

2. Define Port Interfaces > 2.7 Print Protocol​

2. Define Port Interfaces > 2.8 Cache Protocol​

2. Define Port Interfaces > 2.9 Notification Protocol​

2. Define Port Interfaces > 2.10 Search Protocol​

2. Define Port Interfaces > 2.11 I18n Protocol​

3. Define Domain Layer > 3.1 Base DocType​

3. Define Domain Layer > 3.2 Base Controller​

3. Define Domain Layer > 3.3 Mixins​

4. Meta Registry​

5. Port Implementation (Adapters) > 5.1 Dependency Injection​

5. Port Implementation (Adapters) > 5.2 Provider Types​

5. Port Implementation (Adapters) > 5.3 Configuration Provider​

5. Port Implementation (Adapters) > 5.4 Wiring​

5. Port Implementation (Adapters) > 5.5 Entrypoint Scanning for Overrides​

5. Port Implementation (Adapters) > 5.6 Testing Support​

6. Testing Setup​

7. Documentation​

Validation Checklist​

Phase 02: DocType Engine & Database​

1. Schema Mapper (Pydantic → SQLAlchemy) > 1.1 Field Registry & Type Mapping​

1. Schema Mapper (Pydantic → SQLAlchemy) > 1.2 Schema Mapper Implementation​

1. Schema Mapper (Pydantic → SQLAlchemy) > 1.3 Table Registry​

2. Database Connection Setup > 2.1 Connection Factory & Virtual DocTypes​

2. Database Connection Setup > 2.2 Session Factory (SQL Support)​

2. Database Connection Setup > 2.3 Non-SQL Virtual DocTypes (Custom Repositories)​

3. Generic Repository Implementation​

3. Generic Repository Implementation > 3.1 CRUD Operations​

3. Generic Repository Implementation > 3.2 Lifecycle Hook Integration​

4. Migration System > 4.1 Alembic Integration​

4. Migration System > 4.2 Auto-Migration Detection​

4. Migration System > 4.3 CLI Commands​

5. Repository Factory​

6. Engine, Session Factory & Unit of Work > 6.1 Engine & Session Factory Setup​

6. Engine, Session Factory & Unit of Work > 6.2 Unit of Work (UnitOfWork)​

6. Engine, Session Factory & Unit of Work > 6.3 Multi-Source Coordination (Outbox Pattern)​

6. Engine, Session Factory & Unit of Work > 6.4 Startup Sequence (Schema Sync)​

7. Testing > 7.1 Unit Tests​

7. Testing > 7.2 Integration Tests​

8. Error Handling​

9. Performance Optimizations​

Validation Checklist​

Phase 03: API Layer & Authorization​

1. Litestar Application Setup​

2. Authentication Middleware​

3. Permission System > 3.1 Permission Protocol Implementation​

3. Permission System > 3.2 Permission Configuration​

3. Permission System > 3.3 System Context (Elevated Operations)​

3. Permission System > 3.4 Row-Level Security (RLS)​

3. Permission System > 3.5 Team-Based Access (Indie)​

3. Permission System > 3.6 Explicit Sharing (DocumentShare)​

3. Permission System > 3.7 Indie Mode: Permission Conveniences​

4. Auto-CRUD Router > 4.1 Meta Router Implementation​

4. Auto-CRUD Router > 4.2 List Endpoint​

4. Auto-CRUD Router > 4.3 Create Endpoint​

4. Auto-CRUD Router > 4.4 Read Endpoint​

4. Auto-CRUD Router > 4.5 Update Endpoint​

4. Auto-CRUD Router > 4.6 Delete Endpoint​

4. Auto-CRUD Router > 4.7 Router Registration​

5. RPC System > 5.1 Whitelisted Controller Methods​

5. RPC System > 5.2 Arbitrary RPC Functions​

6. Metadata API​

7. OpenAPI Documentation​

8. Error Handling​

9. Request/Response DTOs​

10. Testing > 10.1 Unit Tests​

10. Testing > 10.2 Integration Tests​

Validation Checklist​

Edge Cases to Handle > Child Table Permissions​

Edge Cases to Handle > Link Field Data Leakage​

Edge Cases to Handle > Bulk Operations & RLS​

Phase 04: Background Jobs & Events​

Overview

Phase 01: Core Kernel & Interfaces

1. Project Initialization

2. Define Port Interfaces > 2.1 Repository Protocol

2. Define Port Interfaces > 2.2 Event Bus Protocol

2. Define Port Interfaces > 2.3 Auth Context Protocol

2. Define Port Interfaces > 2.4 Storage Protocol

2. Define Port Interfaces > 2.5 Job Queue Protocol

2. Define Port Interfaces > 2.6 Permission Protocol

2. Define Port Interfaces > 2.7 Print Protocol

2. Define Port Interfaces > 2.8 Cache Protocol

2. Define Port Interfaces > 2.9 Notification Protocol

2. Define Port Interfaces > 2.10 Search Protocol

2. Define Port Interfaces > 2.11 I18n Protocol

3. Define Domain Layer > 3.1 Base DocType

3. Define Domain Layer > 3.2 Base Controller

3. Define Domain Layer > 3.3 Mixins

4. Meta Registry

5. Port Implementation (Adapters) > 5.1 Dependency Injection

5. Port Implementation (Adapters) > 5.2 Provider Types

5. Port Implementation (Adapters) > 5.3 Configuration Provider

5. Port Implementation (Adapters) > 5.4 Wiring

5. Port Implementation (Adapters) > 5.5 Entrypoint Scanning for Overrides

5. Port Implementation (Adapters) > 5.6 Testing Support

6. Testing Setup

7. Documentation

Validation Checklist

Phase 02: DocType Engine & Database

1. Schema Mapper (Pydantic → SQLAlchemy) > 1.1 Field Registry & Type Mapping

1. Schema Mapper (Pydantic → SQLAlchemy) > 1.2 Schema Mapper Implementation

1. Schema Mapper (Pydantic → SQLAlchemy) > 1.3 Table Registry

2. Database Connection Setup > 2.1 Connection Factory & Virtual DocTypes

2. Database Connection Setup > 2.2 Session Factory (SQL Support)

2. Database Connection Setup > 2.3 Non-SQL Virtual DocTypes (Custom Repositories)

3. Generic Repository Implementation

3. Generic Repository Implementation > 3.1 CRUD Operations

3. Generic Repository Implementation > 3.2 Lifecycle Hook Integration

4. Migration System > 4.1 Alembic Integration

4. Migration System > 4.2 Auto-Migration Detection

4. Migration System > 4.3 CLI Commands

5. Repository Factory

6. Engine, Session Factory & Unit of Work > 6.1 Engine & Session Factory Setup

6. Engine, Session Factory & Unit of Work > 6.2 Unit of Work (`UnitOfWork`)

6. Engine, Session Factory & Unit of Work > 6.3 Multi-Source Coordination (Outbox Pattern)

6. Engine, Session Factory & Unit of Work > 6.4 Startup Sequence (Schema Sync)

7. Testing > 7.1 Unit Tests

7. Testing > 7.2 Integration Tests

8. Error Handling

9. Performance Optimizations

Validation Checklist

Phase 03: API Layer & Authorization

1. Litestar Application Setup

2. Authentication Middleware

3. Permission System > 3.1 Permission Protocol Implementation

3. Permission System > 3.2 Permission Configuration

3. Permission System > 3.3 System Context (Elevated Operations)

3. Permission System > 3.4 Row-Level Security (RLS)

3. Permission System > 3.5 Team-Based Access (Indie)

3. Permission System > 3.6 Explicit Sharing (`DocumentShare`)

3. Permission System > 3.7 Indie Mode: Permission Conveniences

4. Auto-CRUD Router > 4.1 Meta Router Implementation

4. Auto-CRUD Router > 4.2 List Endpoint

4. Auto-CRUD Router > 4.3 Create Endpoint

4. Auto-CRUD Router > 4.4 Read Endpoint

4. Auto-CRUD Router > 4.5 Update Endpoint

4. Auto-CRUD Router > 4.6 Delete Endpoint

4. Auto-CRUD Router > 4.7 Router Registration

5. RPC System > 5.1 Whitelisted Controller Methods

5. RPC System > 5.2 Arbitrary RPC Functions

6. Metadata API

7. OpenAPI Documentation

8. Error Handling

9. Request/Response DTOs

10. Testing > 10.1 Unit Tests

10. Testing > 10.2 Integration Tests

Validation Checklist

Edge Cases to Handle > Child Table Permissions

Edge Cases to Handle > Link Field Data Leakage

Edge Cases to Handle > Bulk Operations & RLS

Phase 04: Background Jobs & Events