Framework M Features

Auto-generated feature list from phase checklists.

Overview

Phase	Title	Completion
01	Phase 01: Core Kernel & Interfaces	██████████ 100%
02	Phase 02: DocType Engine & Database	██████████ 100%
03	Phase 03: API Layer & Authorization	██████████ 100%
04	Phase 04: Background Jobs & Events	█████████░ 98%
05	Phase 05: CLI & Developer Tools	█████████░ 97%
06	Phase 06: Built-in DocTypes & Core Features	█████████░ 99%
07	Phase 07: Studio (Code Generation UI)	█████████░ 93%
08	Phase 08: Workflows & Advanced Features	██████████ 100%
09a	Phase 09A: Frontend	█████████░ 97%
09b	Phase 09B: Documentation	██████████ 100%
10	Phase 10: ERP Enterprise Features	██████████ 100%
10	Phase 10: Production Readiness & Deployment	░░░░░░░░░░ 0%
11	Phase 11: Package Split & MX Pattern	██████████ 100%
12	Phase 12: LLM-Ready Documentation & Automation	███░░░░░░░ 39%
13	Phase 13: UI/UX Polish & Design System Enhancement	░░░░░░░░░░ 0%
14	Phase 14: Studio Enhancements	░░░░░░░░░░ 0%

Phase 01: Core Kernel & Interfaces

Phase: 01 Objective: Set up the project foundation with clean hexagonal architecture. Define all protocol interfaces (ports) without any implementation. Status: 100% Complete

1. Project Initialization

Progress: 17/17 (100%)

Completed:

✅ Create project structure using uv:
✅ Setup pyproject.toml with dependencies
✅ Add litestar[standard]>=2.0.0
✅ Add sqlalchemy[asyncio]>=2.0.0 (ensure agnostic, test with sqlite)
✅ Add pydantic>=2.0.0
✅ Add pydantic-settings>=2.0.0
✅ Add dependency-injector>=4.41.0 (powerful DI container)
✅ Add asyncpg>=0.29.0 (PostgreSQL async driver)
✅ Add redis>=5.0.0
✅ Remove arq>=0.26.0
✅ Add taskiq>=0.11.0, taskiq-nats>=0.4.0, nats-py>=2.0.0
✅ Add dev dependencies: pytest, pytest-asyncio, mypy, ruff
✅ Configure development tools
✅ Setup mypy with strict mode in pyproject.toml
✅ Configure ruff for linting and formatting
✅ Add .gitignore for Python projects
✅ Create directory structure

2. Define Port Interfaces > 2.1 Repository Protocol

Progress: 16/16 (100%)

Completed:

✅ Create tests/core/interfaces/test_repository.py
✅ Define test for RepositoryProtocol interface compliance
✅ Create src/framework_m/core/interfaces/repository.py
✅ Define supporting models:
✅ FilterSpec - Typed filter specification (field, operator, value)
✅ OrderSpec - Sorting specification (field, direction)
✅ PaginatedResult[T] - Result with items, total, limit, offset, has_more
✅ Define RepositoryProtocol[T] (Generic):
✅ async def get(self, id: UUID) -> T | None
✅ async def save(self, entity: T, version: int | None = None) -> T (OCC support)
✅ async def delete(self, id: UUID) -> None
✅ async def exists(self, id: UUID) -> bool
✅ async def count(self, filters: list[FilterSpec] | None = None) -> int
✅ async def list(self, filters: list[FilterSpec] | None, order_by: list[OrderSpec] | None, limit: int, offset: int) -> PaginatedResult[T]
✅ async def bulk_save(self, entities: list[T]) -> list[T]
✅ Add type hints with Generic[T] and Protocol

2. Define Port Interfaces > 2.2 Event Bus Protocol

Progress: 10/10 (100%)

Completed:

✅ Create src/framework_m/core/interfaces/event_bus.py
✅ Define Event base model with id, timestamp, source, type, data
✅ Define EventBusProtocol with methods:
✅ async def connect(self) -> None (for NATS/Kafka)
✅ async def disconnect(self) -> None
✅ def is_connected(self) -> bool
✅ async def publish(self, topic: str, event: Event) -> None
✅ async def subscribe(self, topic: str, handler: Callable[[Event], Awaitable[None]]) -> str (returns subscription_id)
✅ async def subscribe_pattern(self, pattern: str, handler: Callable) -> str (e.g., doc.*)
✅ async def unsubscribe(self, subscription_id: str) -> None

2. Define Port Interfaces > 2.3 Auth Context Protocol

Progress: 8/8 (100%)

Completed:

✅ Create src/framework_m/core/interfaces/auth_context.py
✅ Define UserContext Pydantic model with fields:
✅ id: str
✅ email: str
✅ roles: list[str]
✅ tenants: list[str]
✅ Define AuthContextProtocol with methods:
✅ async def get_current_user() -> UserContext

2. Define Port Interfaces > 2.4 Storage Protocol

Progress: 11/11 (100%)

Completed:

✅ Create src/framework_m/core/interfaces/storage.py
✅ Define FileMetadata model with path, size, content_type, modified, etag
✅ Define StorageProtocol with methods:
✅ async def save_file(self, path: str, content: bytes, content_type: str | None = None) -> str
✅ async def get_file(self, path: str) -> bytes
✅ async def delete_file(self, path: str) -> None
✅ async def list_files(self, prefix: str) -> list[str]
✅ async def get_metadata(self, path: str) -> FileMetadata | None
✅ async def get_url(self, path: str, expires: int = 3600) -> str (presigned URL for S3)
✅ async def copy(self, src: str, dest: str) -> str
✅ async def move(self, src: str, dest: str) -> str

2. Define Port Interfaces > 2.5 Job Queue Protocol

Progress: 9/9 (100%)

Completed:

✅ Create src/framework_m/core/interfaces/job_queue.py
✅ Define JobStatus enum: PENDING, RUNNING, SUCCESS, FAILED, CANCELLED
✅ Define JobInfo model with id, name, status, enqueued_at, started_at, result, error
✅ Define JobQueueProtocol with methods:
✅ async def enqueue(self, job_name: str, **kwargs) -> str (returns job_id)
✅ async def schedule(self, job_name: str, cron: str, **kwargs) -> str
✅ async def cancel(self, job_id: str) -> bool
✅ async def get_status(self, job_id: str) -> JobInfo | None
✅ async def retry(self, job_id: str) -> str (returns new job_id)

2. Define Port Interfaces > 2.6 Permission Protocol

Progress: 5/5 (100%)

Completed:

✅ Create src/framework_m/core/interfaces/permission.py
✅ Define PermissionProtocol with methods:
✅ async def has_permission(user: UserContext, doctype: str, action: str, doc_id: str | None) -> bool
✅ async def get_permitted_filters(user: UserContext, doctype: str) -> dict
✅ Add action types: "read", "write", "create", "delete", "submit"

2. Define Port Interfaces > 2.7 Print Protocol

Progress: 3/3 (100%)

Completed:

✅ Create src/framework_m/core/interfaces/print.py
✅ Define PrintProtocol with methods:
✅ async def render(doc: BaseModel, template: str, format: str = "pdf") -> bytes

2. Define Port Interfaces > 2.8 Cache Protocol

Progress: 10/10 (100%)

Completed:

✅ Create src/framework_m/core/interfaces/cache.py
✅ Define CacheProtocol:
✅ async def get(self, key: str) -> Any | None
✅ async def set(self, key: str, value: Any, ttl: int | None = None) -> None
✅ async def delete(self, key: str) -> None
✅ async def exists(self, key: str) -> bool
✅ async def get_many(self, keys: list[str]) -> dict[str, Any]
✅ async def set_many(self, items: dict[str, Any], ttl: int | None = None) -> None
✅ async def delete_pattern(self, pattern: str) -> int (returns count deleted)
✅ async def ttl(self, key: str) -> int | None (remaining TTL)

2. Define Port Interfaces > 2.9 Notification Protocol

Progress: 4/4 (100%)

Completed:

✅ Create src/framework_m/core/interfaces/notification.py
✅ Define NotificationProtocol:
✅ async def send_email(to: str, subject: str, body: str)
✅ async def send_sms(to: str, body: str)

2. Define Port Interfaces > 2.10 Search Protocol

Progress: 7/7 (100%)

Completed:

✅ Create src/framework_m/core/interfaces/search.py
✅ Define SearchResult model with items, total, facets, highlights
✅ Define SearchProtocol:
✅ async def index(self, doctype: str, doc_id: str, doc: dict) -> None
✅ async def delete_index(self, doctype: str, doc_id: str) -> None
✅ async def search(self, doctype: str, query: str, filters: dict | None = None, limit: int = 20, offset: int = 0) -> SearchResult
✅ async def reindex(self, doctype: str) -> int (returns count indexed)

2. Define Port Interfaces > 2.11 I18n Protocol

Progress: 4/4 (100%)

Completed:

✅ Create src/framework_m/core/interfaces/i18n.py
✅ Define I18nProtocol:
✅ async def translate(text: str, locale: str) -> str
✅ async def get_locale() -> str

3. Define Domain Layer > 3.1 Base DocType

Progress: 12/12 (100%)

Completed:

✅ Create src/framework_m/core/domain/base_doctype.py
✅ Define BaseDocType class inheriting from pydantic.BaseModel
✅ Add standard fields:
✅ name: Optional[str] (primary key, auto-generated if None)
✅ creation: datetime
✅ modified: datetime
✅ modified_by: Optional[str]
✅ owner: Optional[str]
✅ Add Meta nested class for metadata:
✅ layout: dict = {} (via get_layout())
✅ permissions: dict = {} (via get_permissions())
✅ Add class method get_doctype_name() -> str

3. Define Domain Layer > 3.2 Base Controller

Progress: 12/12 (100%)

Completed:

✅ Create src/framework_m/core/domain/base_controller.py
✅ Define BaseController[T] generic class
✅ Add lifecycle hook methods:
✅ async def validate(self, context: Any = None) -> None
✅ async def before_insert(self, context: Any = None) -> None
✅ async def after_insert(self, context: Any = None) -> None
✅ async def before_save(self, context: Any = None) -> None
✅ async def after_save(self, context: Any = None) -> None
✅ async def before_delete(self, context: Any = None) -> None
✅ async def after_delete(self, context: Any = None) -> None
✅ async def on_submit(self, context: Any = None) -> None
✅ async def on_cancel(self, context: Any = None) -> None

3. Define Domain Layer > 3.3 Mixins

Progress: 10/10 (100%)

Completed:

✅ Create src/framework_m/core/domain/mixins.py
✅ Define DocStatus enum:
✅ DRAFT = 0
✅ SUBMITTED = 1
✅ CANCELLED = 2
✅ Define SubmittableMixin class with:
✅ docstatus: DocStatus = DocStatus.DRAFT
✅ def is_submitted() -> bool
✅ def is_cancelled() -> bool
✅ def can_edit() -> bool (returns False if submitted)

4. Meta Registry

Progress: 13/13 (100%)

Completed:

✅ Create src/framework_m/core/registry.py
✅ Implement MetaRegistry as singleton
✅ Add storage dictionaries:
✅ _doctypes: Dict[str, Type[BaseDocType]]
✅ _controllers: Dict[str, Type[BaseController]]
✅ Implement methods:
✅ register_doctype(doctype_class, controller_class=None)
✅ get_doctype(name: str) -> Type[BaseDocType]
✅ get_controller(name: str) -> Type[BaseController] | None
✅ list_doctypes() -> list[str]
✅ discover_doctypes(package_name: str) (scans for BaseDocType subclasses)
✅ Load Order: Follows installed_apps list.
✅ Conflict Policy: Raise DuplicateDocTypeError if same name registered twice.

5. Port Implementation (Adapters) > 5.1 Dependency Injection

Progress: 5/5 (100%)

Completed:

✅ Create tests/core/test_container.py
✅ Write test for Container initialization and service resolution
✅ Create src/framework_m/core/container.py
✅ Implement Container class (using dependency_injector):
✅ Define Container class:

5. Port Implementation (Adapters) > 5.2 Provider Types

Progress: 6/6 (100%)

Completed:

✅ Understand provider types:
✅ Singleton - Single instance shared across app
✅ Factory - New instance each time
✅ Resource - For resources with lifecycle (db connections)
✅ Callable - For functions
✅ Dependency - For protocol injection

5. Port Implementation (Adapters) > 5.3 Configuration Provider

Progress: 1/1 (100%)

Completed:

✅ Setup configuration loading:

5. Port Implementation (Adapters) > 5.4 Wiring

Progress: 2/2 (100%)

Completed:

✅ Configure wiring for automatic injection:
✅ Use @inject decorator in functions:

5. Port Implementation (Adapters) > 5.5 Entrypoint Scanning for Overrides

Progress: 5/5 (100%)

Completed:

✅ Create override mechanism:
✅ Scan framework_m.overrides entrypoints
✅ Allow apps to override providers
✅ Example:
✅ Implement override loader:

5. Port Implementation (Adapters) > 5.6 Testing Support

Progress: 2/2 (100%)

Completed:

✅ Use override for testing:
✅ Reset overrides after tests:

6. Testing Setup

Progress: 8/8 (100%)

Completed:

✅ Create tests/ directory structure
✅ Setup conftest.py with fixtures:
✅ pytest_asyncio configuration
✅ Mock implementations of all protocols
✅ Write initial tests:
✅ Test MetaRegistry registration and retrieval
✅ Test BaseDocType field validation
✅ Test BaseController hook method existence

7. Documentation

Progress: 7/7 (100%)

Completed:

✅ Create README.md with:
✅ Project overview
✅ Installation instructions
✅ Basic usage example
✅ Create docs/ folder with:
✅ architecture.md - Hexagonal architecture explanation
✅ ports.md - List of all protocol interfaces (Repository, EventBus, Cache, Search, etc.)

Validation Checklist

Progress: 6/6 (100%)

Completed:

✅ All protocol interfaces are defined with proper type hints
✅ mypy --strict passes with no errors
✅ No infrastructure dependencies in core/ (no imports of litestar, sqlalchemy, redis)
✅ BaseDocType and BaseController are properly generic
✅ MetaRegistry can register and retrieve DocTypes
✅ All tests pass with pytest

Phase 02: DocType Engine & Database

Phase: 02 Objective: Implement the metadata engine that dynamically creates database tables from Pydantic models and provides generic CRUD operations. Status: 100% Complete

1. Schema Mapper (Pydantic → SQLAlchemy) > 1.1 Field Registry & Type Mapping

Progress: 15/15 (100%)

Completed:

✅ Create src/framework_m/adapters/db/field_registry.py
✅ Implement FieldRegistry class (Singleton):
✅ Start with standard types: str, int, float, bool, datetime, date, decimal, uuid, json
✅ Allow registration of custom types: register_type(pydantic_type, sqlalchemy_type)
✅ Define type mapping using Registry:
✅ str → String
✅ int → Integer
✅ float → Float
✅ bool → Boolean
✅ datetime → DateTime
✅ date → Date
✅ Decimal → Numeric
✅ UUID → UUID
✅ list[str] → JSON (portable, works on all databases)
✅ dict → JSON

1. Schema Mapper (Pydantic → SQLAlchemy) > 1.2 Schema Mapper Implementation

Progress: 19/19 (100%)

Completed:

✅ Create tests/adapters/db/test_schema_mapper.py
✅ Test mapping simple fields (str, int)
✅ Test mapping relationships
✅ Create SchemaMapper class
✅ Use FieldRegistry to look up SQLAlchemy types
✅ Implement create_table(model: Type[BaseDocType]) -> Table:
✅ Extract table name from model class name (lowercase)
✅ Iterate over model.model_fields
✅ Map each field to SQLAlchemy column
✅ Handle Optional types (nullable=True)
✅ Set id field as primary key (UUID, auto-generated)
✅ Set name field as unique index (for human-readable lookup)
✅ OCC: Add _version (Integer, default=0) if Meta.concurrency="optimistic"
✅ Return SQLAlchemy Table object
✅ Handle special field types:
✅ Relationships (ForeignKey) - detect by field name pattern *_id
✅ Enums - map to SQLAlchemy String type (database agnostic)
✅ Nested Pydantic models (list[BaseModel]) - store as JSON
✅ Child DocTypes (list[DocType]) - map to Relational Table with Foreign Key

1. Schema Mapper (Pydantic → SQLAlchemy) > 1.3 Table Registry

Progress: 5/5 (100%)

Completed:

✅ Create TableRegistry class to store created tables
✅ Add methods:
✅ register_table(doctype_name: str, table: Table)
✅ get_table(doctype_name: str) -> Table
✅ table_exists(doctype_name: str) -> bool

2. Database Connection Setup > 2.1 Connection Factory & Virtual DocTypes

Progress: 5/5 (100%)

Completed:

✅ Create src/framework_m/adapters/db/connection.py
✅ Implement ConnectionFactory (supports Multiple Bindings):
✅ Load db_binds from config (e.g., legacy, timescale)
✅ Create generic async engine (supports Postgres, SQLite for testing)
✅ Maintain map of engine_name -> AsyncEngine

2. Database Connection Setup > 2.2 Session Factory (SQL Support)

Progress: 3/3 (100%)

Completed:

✅ Add session factory:
✅ async def get_session(bind: str = "default") -> AsyncSession
✅ Support VirtualDocType (SQL Bind) by accepting bind_key from DocType Meta

2. Database Connection Setup > 2.3 Non-SQL Virtual DocTypes (Custom Repositories)

Progress: 7/7 (100%)

Completed:

✅ Create concept of RepositoryOverride:
✅ Allow DocType to define repository_class in Meta
✅ RepositoryFactory instantiates this instead of GenericRepository
✅ Create tests/adapters/db/test_repository_factory.py:
✅ Implement a Mock FileRepository that reads from JSON file
✅ Register it for a VirtualDoc
✅ Verify VirtualDoc.get() calls FileRepository.get()

3. Generic Repository Implementation

Progress: 9/9 (100%)

Completed:

✅ Create tests/adapters/db/test_generic_repository.py
✅ Define tests for CRUD operations (mock AsyncSession)
✅ Create src/framework_m/adapters/db/generic_repository.py
✅ Implement GenericRepository[T] class
✅ Constructor dependencies (session is NOT stored here):
✅ model: Type[T]
✅ table: Table
✅ controller_class: Type[BaseController] | None
✅ event_bus: EventBusProtocol | None (InMemoryEventBus for dev)

3. Generic Repository Implementation > 3.1 CRUD Operations

Progress: 49/49 (100%)

Completed:

✅ Implement async def get(session: AsyncSession, id: UUID) -> Optional[T]:
✅ Build SELECT query
✅ Filter deleted_at IS NULL (unless specific flag overrides)
✅ Execute with provided session (not stored session)
✅ Convert row to Pydantic model
✅ Return None if not found
✅ Implement async def get_by_name(session: AsyncSession, name: str) -> Optional[T]:
✅ Helper for looking up by human-readable name
✅ Implement async def save(session: AsyncSession, entity: T, version: int | None = None) -> T:
✅ Check if entity exists (by id)
✅ If new:
✅ Generate id (UUIDv7)
✅ Note: created_by / owner must be set by Controller/Service before calling save
✅ Call controller validate()
✅ Call controller before_create()
✅ Call controller before_save()
✅ Execute INSERT with provided session
✅ Call controller after_save()
✅ Call controller after_create()
✅ Emit Event: event_bus.publish(f"{doctype}.create", payload)
✅ If existing:
✅ Note: modified_by must be set by Controller/Service before calling save
✅ Call controller validate()
✅ Call controller before_save()
✅ OCC: If optimistic:
✅ UPDATE table SET ..., _version=_version+1 WHERE id=:id AND _version=:old_ver
✅ If rowcount == 0, raise VersionConflictError
✅ Else: Execute standard UPDATE
✅ Call controller after_save()
✅ Emit Event: event_bus.publish(f"{doctype}.update", payload)
✅ Return saved entity (caller calls uow.commit() when ready)
✅ Implement async def delete(session: AsyncSession, id: UUID, hard: bool = False) -> None:
✅ Load entity
✅ Call controller before_delete()
✅ If hard:
✅ Execute DELETE
✅ Else (Soft Delete):
✅ Update deleted_at = now()
✅ Call controller after_delete()
✅ Emit Event: event_bus.publish(f"{doctype}.delete", payload)
✅ Return (caller calls uow.commit() when ready)
✅ Implement async def list(session: AsyncSession, filters, limit, offset, order_by) -> Sequence[T]:
✅ Build SELECT query with filters
✅ Default filter: deleted_at IS NULL
✅ Apply pagination (limit, offset)
✅ Apply sorting (order_by)
✅ Execute query
✅ Convert rows to Pydantic models
✅ Return list

3. Generic Repository Implementation > 3.2 Lifecycle Hook Integration

Progress: 5/5 (100%)

Completed:

✅ Create helper method _call_hook(hook_name: str):
✅ Check if controller exists
✅ Check if hook method exists on controller
✅ Call hook method if present
✅ Handle exceptions and rollback on error

4. Migration System > 4.1 Alembic Integration

Progress: 8/8 (100%)

Completed:

✅ Initialize Alembic in project:
✅ Configure alembic.ini:
✅ Set database URL from environment
✅ Configure migration file location
✅ Create alembic/env.py:
✅ Import MetaRegistry
✅ Import all registered DocTypes
✅ Set target_metadata from SchemaMapper tables

4. Migration System > 4.2 Auto-Migration Detection

Progress: 11/11 (100%)

Completed:

✅ Create src/framework_m/adapters/db/migration.py
✅ Implement detect_schema_changes():
✅ Compare registered DocTypes with database schema
✅ Detect new tables
✅ Detect new columns
✅ Detect type changes
✅ Return list of changes
✅ Implement auto_migrate():
✅ Call detect_schema_changes()
✅ Generate Alembic migration if changes detected
✅ Apply migration automatically (dev mode only)

4. Migration System > 4.3 CLI Commands

Progress: 5/5 (100%)

Completed:

✅ Add migration commands to CLI:
✅ m migrate - run pending migrations
✅ m migrate:create <name> - create new migration
✅ m migrate:rollback - rollback last migration
✅ m migrate:status - show migration status

5. Repository Factory

Progress: 9/9 (100%)

Completed:

✅ Create src/framework_m/adapters/db/repository_factory.py
✅ Implement RepositoryFactory class:
✅ create_generic_repository(doctype_name: str) -> GenericRepository
✅ Look up DocType from MetaRegistry
✅ Look up Table from TableRegistry
✅ Look up Controller from MetaRegistry
✅ Create and return GenericRepository instance
✅ Support event_bus parameter for domain events
✅ Support custom repository overrides for Virtual DocTypes

6. Engine, Session Factory & Unit of Work > 6.1 Engine & Session Factory Setup

Progress: 8/8 (100%)

Completed:

✅ Create src/framework_m/adapters/db/connection.py (ConnectionFactory)
✅ Implement create_engine(url: str) -> AsyncEngine
✅ Pool configuration (pool_size, max_overflow, timeout, recycle, pre_ping)
✅ Environment variable expansion (${VAR} syntax)
✅ Implement SessionFactory (returns AsyncSession context managers)
✅ Support multiple binds (for Virtual DocTypes with SQL binds)
✅ Configuration from ConnectionFactory
✅ Auto commit/rollback in context manager

6. Engine, Session Factory & Unit of Work > 6.2 Unit of Work (`UnitOfWork`)

Progress: 14/14 (100%)

Completed:

✅ Create tests/core/test_unit_of_work.py
✅ Test: UoW provides session
✅ Test: commit() persists changes
✅ Test: Exception causes rollback (no explicit rollback call needed)
✅ Test: Session is closed after __aexit__
✅ Create src/framework_m/core/unit_of_work.py
✅ Implement UnitOfWork context manager:
✅ __init__(session_factory: Callable[[], AsyncSession])
✅ session property with safety check
✅ async __aenter__() creates session
✅ async commit() calls session.commit()
✅ async rollback() for explicit rollback
✅ async __aexit__() with auto-rollback on exception
✅ Register UnitOfWorkFactory in DI container

6. Engine, Session Factory & Unit of Work > 6.3 Multi-Source Coordination (Outbox Pattern)

Progress: 17/17 (100%)

Completed:

✅ Create src/framework_m/core/domain/outbox.py
✅ Define OutboxEntry model:
✅ id: UUID
✅ target: str (e.g., "mongodb.audit_log", "api.payment_gateway")
✅ payload: dict
✅ status: str (pending, processed, failed)
✅ created_at: datetime
✅ processed_at: datetime | None
✅ error_message: str | None
✅ retry_count: int
✅ Create OutboxRepository (SQL-backed) in adapters/db/outbox_repository.py
✅ add(session, entry) - add entry in same transaction
✅ get_pending(session, limit) - get pending entries
✅ mark_processed(session, id) - mark as processed
✅ mark_failed(session, id, error) - mark as failed
✅ Document: Services write to Outbox in the same SQL transaction
✅ (Phase 04) Background worker processes Outbox entries

6. Engine, Session Factory & Unit of Work > 6.4 Startup Sequence (Schema Sync)

Progress: 7/7 (100%)

Completed:

✅ Create src/framework_m/adapters/db/__init__.py
✅ Implement startup sequence (called once at app boot, NOT per-request):
✅ Initialize database engine (NOT session)
✅ Discover all DocTypes via MetaRegistry
✅ Create/sync tables via SchemaMapper
✅ Register tables in TableRegistry
✅ Run auto-migration (if enabled)

7. Testing > 7.1 Unit Tests

Progress: 9/9 (100%)

Completed:

✅ Test SchemaMapper:
✅ Test type mapping for all supported types (test_schema_mapper.py)
✅ Test primary key creation (test_table_has_id_column_as_primary_key)
✅ Test nullable fields (TestSchemaMapperNullableFields)
✅ Test enum mapping (TestSchemaMapperEnums)
✅ Test GenericRepository:
✅ Test CRUD operations with mock data (test_generic_repository.py)
✅ Test lifecycle hook calls (TestControllerHooks)
✅ Test transaction rollback on error (TestTransactionRollback)

7. Testing > 7.2 Integration Tests

Progress: 13/13 (100%)

Completed:

✅ Setup testcontainers for PostgreSQL (skips if Docker unavailable)
✅ Create test DocType (IntegrationTestDoc / PostgresTestDoc)
✅ Test full flow (SQLite & Postgres - Postgres skipped without Docker):
✅ Register DocType (test_register_doctype)
✅ Create table (test_table_created)
✅ Insert document (test_insert_document)
✅ Query document (test_query_document)
✅ Update document (test_update_document)
✅ Delete document (test_delete_document)
✅ Test migration:
✅ Add field to DocType
✅ Run auto-migration
✅ Verify column added to table

8. Error Handling

Progress: 10/10 (100%)

Completed:

✅ Create custom exceptions (core/exceptions.py):
✅ DocTypeNotFoundError
✅ ValidationError
✅ PermissionDeniedError
✅ DuplicateNameError
✅ RepositoryError, EntityNotFoundError, DatabaseError, IntegrityError
✅ Add error handling in repository (generic_repository.py):
✅ Catch SQLAlchemy errors (IntegrityError, OperationalError, SQLAlchemyError)
✅ Convert to domain exceptions (DuplicateNameError, DatabaseError, etc.)
✅ Log errors with context (using logger.error with extra dict)

9. Performance Optimizations

Progress: 13/13 (100%)

Completed:

✅ Add query result caching (phase-04):
✅ Cache get() results by ID
✅ Invalidate cache on save/delete
✅ Use Redis for distributed cache
✅ Add bulk operations (generic_repository.py):
✅ async def bulk_save(entities) - separates new/existing
✅ async def _bulk_insert(entities) - true SQLAlchemy bulk insert
✅ Uses insert().values([...]) for performance
✅ Add query optimization (add indexes after the API layer exists):
✅ Add indexes for common queries (schema_mapper: owner, creation, Meta.indexes)
✅ Add parent index for child tables (efficient joins)
✅ Use select_in_loading for relationships (load_children_for_parents method)
✅ Add query logging in dev mode (logger.debug for GET/LIST)

Validation Checklist

Progress: 6/6 (100%)

Completed:

✅ Can create tables from Pydantic models dynamically
✅ CRUD operations work with lifecycle hooks
✅ Migrations are generated and applied correctly
✅ All integration tests pass with real PostgreSQL (skips if Docker unavailable)
✅ No direct SQLAlchemy imports in domain layer
✅ Repository implements RepositoryProtocol correctly

Phase 03: API Layer & Authorization

Phase: 03 Objective: Expose DocTypes via HTTP with auto-generated CRUD endpoints, implement permission system with row-level security, and add RPC support. Status: 100% Complete

1. Litestar Application Setup

Progress: 14/14 (100%)

Completed:

✅ Create src/framework_m/adapters/web/app.py
✅ Initialize Litestar app:
✅ Configure CORS for development
✅ Add exception handlers
✅ Configure OpenAPI documentation
✅ Integrate with dependency-injector Container (wire modules)
✅ Middleware Discovery:
✅ Scan MiddlewareRegistry for registered App Middlewares.
✅ Allow Apps to inject ASGI Middleware (e.g. RateLimit, cors, Compression).
✅ APM & Tracing: Support OpenTelemetry, DataDog, Sentry via standard ASGI middleware.
✅ Support ordering (priority).
✅ Add application lifecycle:
✅ on_startup: Initialize Container, database, discover DocTypes
✅ on_shutdown: Close database connections, unwire Container

2. Authentication Middleware

Progress: 9/9 (100%)

Completed:

✅ Create src/framework_m/adapters/web/middleware.py
✅ Implement AuthMiddleware:
✅ Extract x-user-id header
✅ Extract x-roles header (comma-separated)
✅ Extract x-tenants header (optional, comma-separated)
✅ Create UserContext object
✅ Store in request.state.user
✅ Return 401 if headers missing (configurable)
✅ Add middleware to Litestar app (available via create_auth_middleware())

3. Permission System > 3.1 Permission Protocol Implementation

Progress: 7/7 (100%)

Completed:

✅ Create src/framework_m/adapters/auth/rbac_permission.py
✅ Implement RbacPermissionAdapter:
✅ Implement evaluate(request: PolicyEvaluateRequest) -> PolicyEvaluateResult
✅ Read permissions from DocType.Meta.permissions
✅ Match request.principal_attributes["roles"] against allowed roles
✅ CustomPermission DocType created (integration pending)
✅ Return PolicyEvaluateResult(authorized=True/False, decision_source=DecisionSource.RBAC)

3. Permission System > 3.2 Permission Configuration

Progress: 7/7 (100%)

Completed:

✅ Define permission structure in DocType:
✅ Implement has_permission():
✅ Check requires_auth — if False and no user, allow read
✅ Check if user has required role
✅ Check database overrides via PermissionLookupService (CustomPermission DocType)
✅ For object-level: check ownership or custom rules
✅ Return boolean

3. Permission System > 3.3 System Context (Elevated Operations)

Progress: 4/4 (100%)

Completed:

✅ Create src/framework_m/core/system_context.py
✅ Implement SystemContext:
✅ Usage in background jobs:
✅ Audit: All system operations logged with principal="system"

3. Permission System > 3.4 Row-Level Security (RLS)

Progress: 13/13 (100%)

Completed:

✅ Implement get_permitted_filters():
✅ For standard users: add WHERE owner = :user_id
✅ For admins: no filter (see all)
✅ For custom rules: via Meta.rls_field (completed by Section 3.5 Team-Based Access)
✅ Return dict of SQLAlchemy filters
✅ Update GenericRepository.list():
✅ list_for_user() method with RLS filtering
✅ apply_rls_filters() helper available
✅ Merge with user-provided filters
✅ Apply to SQL query
✅ Update GenericRepository.get():
✅ get_for_user() method with permission check
✅ Raise PermissionDeniedError if denied

3. Permission System > 3.5 Team-Based Access (Indie)

Progress: 5/5 (100%)

Completed:

✅ Add Meta.rls_field option:
✅ Update get_permitted_filters():
✅ If rls_field = "owner" (default): WHERE owner = :user_id
✅ If rls_field = "team": WHERE team IN :user_teams
✅ Support custom fields

3. Permission System > 3.6 Explicit Sharing (`DocumentShare`)

Progress: 6/6 (100%)

Completed:

✅ Create built-in DocumentShare DocType:
✅ Update get_permitted_filters() to include shares via get_rls_filters_with_shares():
✅ Add share management API:
✅ POST /api/v1/share — Create share
✅ DELETE /api/v1/share/{id} — Remove share
✅ GET /api/v1/{doctype}/{id}/shares — List shares for a doc

3. Permission System > 3.7 Indie Mode: Permission Conveniences

Progress: 14/14 (100%)

Completed:

✅ Create src/framework_m/core/decorators.py (if not exists)
✅ Implement @requires_permission(action: PermissionAction):
✅ Internally builds PolicyEvaluateRequest from self.user, self.doctype_name
✅ Calls permission.evaluate(request)
✅ Raises PermissionDeniedError if not authorized
✅ Works on Controller methods
✅ Add to BaseController:
✅ Add to BaseController:
✅ Verify: All auto-generated CRUD routes (Section 4) automatically call permission.evaluate():
✅ POST /api/v1/{doctype} → checks CREATE permission (verified via test)
✅ GET /api/v1/{doctype}/{id} → checks READ permission (verified via test)
✅ PUT /api/v1/{doctype}/{id} → checks WRITE permission (verified via test)
✅ DELETE /api/v1/{doctype}/{id} → checks DELETE permission (verified via test)
✅ No manual code required for indie devs using auto-CRUD (verified via test)

4. Auto-CRUD Router > 4.1 Meta Router Implementation

Progress: 6/6 (100%)

Completed:

✅ Create src/framework_m/adapters/web/meta_router.py
✅ Implement create_crud_routes(doctype_class: type[BaseDocType]):
✅ Check Opt-in: Verify DocType.Meta.api_resource is True. Skip if False.
✅ Generate 5 routes per DocType
✅ Inject repository and permission service
✅ Parse filters from JSON string via _parse_filters()

4. Auto-CRUD Router > 4.2 List Endpoint

Progress: 5/5 (100%)

Completed:

✅ Create GET /api/v1/{doctype}:
✅ Accept query parameters: limit, offset, filters, order_by
✅ Parse filters from JSON string
✅ Call repository list_entities() with RLS filters via apply_rls_filters()
✅ Return paginated response with metadata:

4. Auto-CRUD Router > 4.3 Create Endpoint

Progress: 6/6 (100%)

Completed:

✅ Create POST /api/v1/{doctype}:
✅ Validate request body against Pydantic model
✅ Check CREATE permission via _check_permission()
✅ Set owner to current user (data["owner"] = user_id)
✅ Call repository save()
✅ Return 201 with created document

4. Auto-CRUD Router > 4.4 Read Endpoint

Progress: 6/6 (100%)

Completed:

✅ Create GET /api/v1/{doctype}/{id}:
✅ Call repository get(id)
✅ Check READ permission
✅ Return 404 if not found (via NotFoundException)
✅ Return 403 if permission denied
✅ Return document

4. Auto-CRUD Router > 4.5 Update Endpoint

Progress: 7/7 (100%)

Completed:

✅ Create PUT /api/v1/{doctype}/{id}:
✅ Load existing document via repo.get()
✅ Check WRITE permission
✅ Check if submitted (deny if immutable) via _check_submitted()
✅ Merge changes via entity.model_copy(update=data)
✅ Call repository save()
✅ Return updated document

4. Auto-CRUD Router > 4.6 Delete Endpoint

Progress: 6/6 (100%)

Completed:

✅ Create DELETE /api/v1/{doctype}/{id}:
✅ Load document
✅ Check DELETE permission
✅ Check if submitted (deny if immutable) via _check_submitted()
✅ Call repository delete()
✅ Return 204 No Content

4. Auto-CRUD Router > 4.7 Router Registration

Progress: 5/5 (100%)

Completed:

✅ Implement create_meta_router():
✅ Get all DocTypes from MetaRegistry
✅ For each DocType with api_resource=True, call create_crud_routes()
✅ Register all routes with Litestar
✅ Return Router instance

5. RPC System > 5.1 Whitelisted Controller Methods

Progress: 13/13 (100%)

Completed:

✅ Create @whitelist decorator:
✅ Mark controller methods as publicly callable
✅ Store metadata on method (WHITELIST_ATTR)
✅ is_whitelisted() helper function
✅ get_whitelist_options() helper function
✅ Create POST /api/v1/rpc/{doctype}/{method}:
✅ Load DocType controller
✅ Check if method has @whitelist decorator
✅ Validate method exists
✅ Parse request body as method arguments
✅ Instantiate controller with document (if doc_id provided)
✅ Call method
✅ Return result

5. RPC System > 5.2 Arbitrary RPC Functions

Progress: 16/16 (100%)

Completed:

✅ Create @rpc decorator for standalone functions:
✅ Register function in RPC registry (RpcRegistry singleton)
✅ Store function path (e.g., my_app.api.send_email)
✅ is_rpc_function() helper
✅ get_rpc_options() helper
✅ Create POST /api/v1/rpc/fn/{dotted.path}:
✅ Parse dotted path (e.g., my_app.api.send_email)
✅ Look up function in registry
✅ Validate function has @rpc decorator
✅ Parse request body as function arguments
✅ Call function
✅ Return result
✅ Add permission check for RPC:
✅ Support @rpc(permission="custom_permission")
✅ Check permission before calling function
✅ Support @rpc(allow_guest=True) for public endpoints

6. Metadata API

Progress: 11/11 (100%)

Completed:

✅ Create GET /api/meta/{doctype}:
✅ Get DocType class from registry
✅ Generate JSON Schema from Pydantic model:
✅ Extract layout from Meta.layout
✅ Extract permissions from Meta.permissions
✅ Return:
✅ Add field metadata:
✅ Field labels (from Field(title=...)) - included in JSON Schema
✅ Help text (from Field(description=...)) - included in JSON Schema
✅ Validation rules (from Pydantic validators) - included in JSON Schema
✅ Field types and options - included in JSON Schema

7. OpenAPI Documentation

Progress: 10/10 (100%)

Completed:

✅ Configure Litestar OpenAPI:
✅ Set title, version, description
✅ Add authentication scheme (header-based: x-user-id, x-roles)
✅ Enable Swagger UI at /schema/swagger
✅ Enable ReDoc at /schema/redoc
✅ Enhance auto-generated docs:
✅ Add descriptions to CRUD endpoints (via docstrings)
✅ Add examples for request/response (via Pydantic models)
✅ Document RPC endpoints (auto-generated from route handlers)
✅ Add permission requirements to docs (via security schemes)

8. Error Handling

Progress: 7/7 (100%)

Completed:

✅ Create exception handlers:
✅ ValidationError → 400 Bad Request (validation_error_handler)
✅ PermissionDeniedError → 403 Forbidden (permission_denied_handler)
✅ DocTypeNotFoundError → 404 Not Found (not_found_handler)
✅ DuplicateNameError → 409 Conflict (duplicate_name_handler)
✅ Generic exceptions → 500 Internal Server Error (framework_error_handler)
✅ Return consistent error format:

9. Request/Response DTOs

Progress: 7/7 (100%)

Completed:

✅ Use Litestar DTOs for validation:
✅ Auto-generate from Pydantic models (via Litestar)
✅ Add DTO for list response (pagination): PaginatedResponse[T]
✅ Add DTO for error responses: ErrorResponse
✅ Implement response filtering:
✅ Exclude sensitive fields based on permissions (via field selection)
✅ Support field selection via query param: ?fields=name,title

10. Testing > 10.1 Unit Tests

Progress: 8/8 (100%)

Completed:

✅ Test permission system:
✅ Test role-based access (test_rbac_permission.py)
✅ Test RLS filter generation (test_rls.py)
✅ Test object-level permissions (test_object_level_permissions.py)
✅ Test RPC system:
✅ Test @whitelist decorator (test_whitelist_decorator.py)
✅ Test @rpc decorator (test_rpc_decorator.py)
✅ Test dotted path resolution (test_rpc_routes.py)

10. Testing > 10.2 Integration Tests

Progress: 15/15 (100%)

Completed:

✅ Test CRUD endpoints (test_api_endpoints.py):
✅ Test create with valid data
✅ Test create with invalid data (validation)
✅ Test create without permission (403)
✅ Test list with RLS (only see own docs)
✅ Test update immutable doc (covered by existing test_meta_router.py)
✅ Test delete with permission (covered by existing permission tests)
✅ Test RPC endpoints (test_api_endpoints.py, test_rpc_routes.py):
✅ Test whitelisted controller method
✅ Test arbitrary RPC function
✅ Test RPC with permissions
✅ Test with real HTTP client (test_real_http_client.py):
✅ Uses httpx.AsyncClient against real uvicorn server
✅ Tests: list endpoint, create endpoint, health check
✅ Manual testing scripts available in examples/ folder

Validation Checklist

Progress: 6/6 (100%)

Completed:

✅ All CRUD endpoints work with permissions (verified via test_meta_router.py, test_api_endpoints.py)
✅ RLS filters are applied correctly (verified via test_rls.py, test_object_level_permissions.py)
✅ RPC system works for controller methods and functions (verified via test_rpc_routes.py)
✅ OpenAPI docs are generated correctly (SwaggerUI + ReDoc enabled)
✅ Error handling returns proper status codes (verified via test_app.py)
✅ Integration tests pass with real HTTP requests (verified via test_real_http_client.py)

Edge Cases to Handle > Child Table Permissions

Progress: 4/4 (100%)

Completed:

✅ Define: Child tables inherit parent's RLS by default (Meta.is_child_table = True)
✅ Child rows are not independently permission-checked (api_resource = False)
✅ Loading parent loads all its children (no separate RLS query)
✅ Document: If independent child access needed, make it a separate DocType (tests in test_child_table_permissions.py)

Edge Cases to Handle > Link Field Data Leakage

Progress: 3/3 (100%)

Completed:

✅ When serializing a doc with Link fields, do NOT auto-include linked doc data (Link fields store UUID, not embedded objects)
✅ If linked doc data is needed, make a separate API call (permission checked) - documented pattern
✅ Optional: ?expand=customer param design documented (checks permission before expanding)

Edge Cases to Handle > Bulk Operations & RLS

Progress: 3/3 (100%)

Completed:

✅ GenericRepository.delete_many_for_user(filters) applies RLS
✅ GenericRepository.update_many_for_user(filters, data) applies RLS
✅ User can only bulk-modify docs they have access to (tests in test_bulk_operations_rls.py)

Phase 04: Background Jobs & Events

Phase: 04 Objective: Implement asynchronous job processing, event-driven architecture, and webhook system with pluggable backends (Redis, NATS, In-Memory). Status: 98% Complete

0. Backend Selection (Cross-Platform) > 0.2 Dev Mode (No External Deps)

Progress: 3/3 (100%)

Completed:

✅ Implement InMemoryJobQueue for local dev (asyncio.Queue).
✅ Implement InMemoryEventBus for local dev.
✅ Auto-detect: If NATS_URL not set, use in-memory.

1. Taskiq + NATS Integration (Primary) > 1.1 Taskiq Setup

Progress: 8/8 (100%)

Completed:

✅ Add dependencies: taskiq, taskiq-nats.
✅ Create src/framework_m/adapters/jobs/taskiq_adapter.py
✅ Configure Taskiq with NATS JetStream:
✅ Set NATS connection URL from environment (NATS_URL)
✅ Configure job timeout (default: 300s)
✅ Configure max retries (default: 3)
✅ Use PullBasedJetStreamBroker for reliable delivery
✅ Create broker instance:

1. Taskiq + NATS Integration (Primary) > 1.2 Job Queue Protocol Implementation

Progress: 7/7 (100%)

Completed:

✅ Implement TaskiqJobQueueAdapter:
✅ async def enqueue(job_name: str, **kwargs) -> str:
✅ Get registered task
✅ Call .kiq() to enqueue
✅ Return job ID
✅ async def schedule(job_name: str, cron: str, **kwargs):
✅ Use TaskiqScheduler with cron triggers

1. Taskiq + NATS Integration (Primary) > 1.3 Job Registration

Progress: 4/4 (100%)

Completed:

✅ Create job decorator @job:
✅ Implement job registry:
✅ Store all @broker.task decorated functions
✅ Auto-discover on startup

2. Worker Process > 2.1 Worker Implementation

Progress: 7/7 (100%)

Completed:

✅ Create src/framework_m/cli/worker.py
✅ Implement worker command:
✅ Worker startup:
✅ Initialize database connection
✅ Discover and register all jobs
✅ Start Taskiq worker
✅ Listen for jobs from NATS JetStream

2. Worker Process > 2.2 Job Context

Progress: 10/10 (100%)

Completed:

✅ Create job context for dependency injection:
✅ Provide database session
✅ Provide repository factory
✅ Provide event bus
✅ Pass to job functions
✅ Add job metadata:
✅ Job ID
✅ Enqueued time
✅ Started time
✅ User context (Must re-hydrate from job arguments if applicable)

3. Scheduler > 3.1 Cron Jobs

Progress: 8/8 (100%)

Completed:

✅ Support Taskiq's native cron syntax:
✅ Create ScheduledJob DocType:
✅ name: str - Job identifier
✅ function: str - Dotted path to function
✅ cron_expression: str - Cron syntax
✅ enabled: bool - Active/inactive
✅ last_run: datetime | None
✅ next_run: datetime | None

3. Scheduler > 3.2 Dynamic Scheduler

Progress: 3/4 (75%)

Completed:

✅ Implement dynamic job scheduling:
✅ Load ScheduledJob DocTypes from database
✅ Register with Taskiq at startup

Pending:

⏳ Support adding/removing jobs without restart (via signal) NOT NEEDED: Worker restart is acceptable for schedule changes; k8s rolling updates handle this gracefully. If needed in future, can use NATS pub/sub to signal workers in Phase 06+ (Ops/Deployment).

4. Event Bus > 4.1 NATS Event Bus Implementation

Progress: 9/9 (100%)

Completed:

✅ Create src/framework_m/adapters/events/nats_event_bus.py
✅ Implement NatsEventBusAdapter:
✅ async def publish(topic: str, event: BaseModel):
✅ Serialize event to JSON
✅ Publish to NATS JetStream subject
✅ async def subscribe(topic: str, handler: Callable):
✅ Subscribe to NATS JetStream subject
✅ Deserialize event
✅ Call handler function

4. Event Bus > 4.2 Event Types

Progress: 7/7 (100%)

Completed:

✅ Create base event class:
✅ Define standard events:
✅ DocCreated
✅ DocUpdated
✅ DocDeleted
✅ DocSubmitted
✅ DocCancelled

4. Event Bus > 4.3 Event Publishing from Lifecycle Hooks

Progress: 6/6 (100%)

Completed:

✅ Update GenericRepository:
✅ Inject event bus
✅ Publish DocCreated after insert
✅ Publish DocUpdated after update
✅ Publish DocDeleted after delete
✅ Add event publishing to controller hooks:

5. Webhook System > 5.1 Webhook DocType

Progress: 9/9 (100%)

Completed:

✅ Create Webhook DocType:
✅ name: str
✅ event: str - Event to listen to (e.g., "doc.created")
✅ doctype_filter: str | None - Filter by DocType
✅ url: str - Webhook endpoint URL
✅ method: str - HTTP method (POST, PUT)
✅ headers: dict - Custom headers
✅ enabled: bool
✅ secret: str - For signature verification

5. Webhook System > 5.2 Webhook Listener

Progress: 6/6 (100%)

Completed:

✅ Create src/framework_m/adapters/webhooks/listener.py
✅ Implement webhook listener:
✅ Subscribe to all events
✅ Load active webhooks from database
✅ Filter events by webhook configuration
✅ Trigger HTTP call to webhook URL

5. Webhook System > 5.3 Webhook Delivery

Progress: 9/9 (100%)

Completed:

✅ Implement webhook delivery:
✅ Use httpx for async HTTP calls
✅ Add signature header (HMAC-SHA256 with secret)
✅ Set timeout (default: 30s)
✅ Handle errors and retries
✅ Add retry logic:
✅ Retry on failure (exponential backoff)
✅ Max retries: 3
✅ Log failed deliveries

5. Webhook System > 5.4 Webhook Logs

Progress: 8/8 (100%)

Completed:

✅ Create WebhookLog DocType:
✅ webhook: str - Reference to Webhook
✅ event: str - Event that triggered
✅ status: str - Success/Failed
✅ response_code: int
✅ response_body: str
✅ error: str | None
✅ timestamp: datetime

6. Job Monitoring > 6.1 Job Status Tracking

Progress: 13/13 (100%)

Completed:

✅ Create JobLog DocType:
✅ job_id: str
✅ job_name: str
✅ status: str - Queued/Running/Success/Failed
✅ enqueued_at: datetime
✅ started_at: datetime | None
✅ completed_at: datetime | None
✅ error: str | None
✅ result: dict | None
✅ Update job execution to log status:
✅ Create log entry on enqueue
✅ Update on start
✅ Update on completion/failure

6. Job Monitoring > 6.2 Job Management API

Progress: 5/5 (100%)

Completed:

✅ Create endpoints:
✅ GET /api/v1/jobs - List jobs
✅ GET /api/v1/jobs/{job_id} - Get job status
✅ POST /api/v1/jobs/{job_id}/cancel - Cancel job
✅ POST /api/v1/jobs/{job_id}/retry - Retry failed job

6. Job Monitoring > 6.3 Job Monitor (Admin UI)

Progress: 3/6 (50%)

Completed:

✅ Option A: Native Desk UI (IMPLEMENTED):
✅ Job Log List View (auto-generated via api_resource=True).
✅ Shows Status, Error, Duration (duration_seconds property).

Pending:

⏳ ~~Option B: Taskiq Dashboard (External)~~ - NOT NEEDED for MVP: Native JobLog UI is sufficient for debugging. External dashboards can be mounted in Phase 06+ (Ops/Deployment) if production monitoring requires deeper worker introspection.
⏳ Mount taskiq-dashboard (if available) or similar tool.
⏳ Useful for sysadmins to debug worker health.

4. Real-time Events (WebSockets) > 4.1 WebSocket Protocol (The Port)

Progress: 4/4 (100%)

Completed:

✅ Create src/framework_m/core/interfaces/socket.py
✅ Define SocketProtocol:
✅ async def broadcast(topic: str, message: dict)
✅ async def send_to_user(user_id: str, message: dict)

4. Real-time Events (WebSockets) > 4.2 NATS Backplane (The Adapter)

Progress: 7/7 (100%)

Completed:

✅ Create tests/adapters/socket/test_nats_socket.py
✅ Write tests using testcontainers-nats to verify Pub/Sub.
✅ Create src/framework_m/adapters/socket/nats_socket.py
✅ Implement NatsSocketAdapter:
✅ On Boot: Subscribe to framework.events.*.
✅ On Event: await nc.publish(subject, json.dumps(msg).encode()).
✅ On Receive (Sub): Forward to local connected Websocket clients (Connection Manager).

4. Real-time Events (WebSockets) > 4.3 WebSocket Endpoint (Litestar)

Progress: 8/8 (100%)

Completed:

✅ Create src/framework_m/adapters/web/socket.py
✅ Implement ConnectionManager:
✅ Store active connections Dict[UserId, List[WebSocket]].
✅ Add route WS /api/v1/stream:
✅ Authenticate user (Query param token).
✅ Upgrade connection.
✅ Register with ConnectionManager.
✅ Listen for client disconnect.

7. Testing > 7.1 Unit Tests

Progress: 6/6 (100%)

Completed:

✅ Test job registration:
✅ Test @job decorator
✅ Test job registry
✅ Test event bus:
✅ Test publish/subscribe
✅ Test event serialization

7. Testing > 7.2 Integration Tests

Progress: 13/13 (100%)

Completed:

✅ Test job execution:
✅ Enqueue job
✅ Verify job runs
✅ Check job result
✅ Test scheduled jobs:
✅ Register cron job
✅ Verify execution at scheduled time (use time mocking)
✅ Test webhooks:
✅ Create webhook
✅ Trigger event
✅ Verify HTTP call made
✅ Check webhook log
✅ Use testcontainers for NATS

8. CLI Commands

Progress: 5/5 (100%)

Completed:

✅ Add job management commands:
✅ m worker - Start worker process
✅ m job:list - List registered jobs
✅ m job:run <name> - Run job immediately
✅ m job:status <id> - Check job status

Validation Checklist

Progress: 6/6 (100%)

Completed:

✅ Jobs can be enqueued and executed
✅ Scheduled jobs run at correct times
✅ Events are published and received
✅ Webhooks are triggered on events
✅ Job logs are created and updated
✅ Worker process starts and processes jobs

Phase 05: CLI & Developer Tools

Phase: 05 Objective: Build a comprehensive CLI tool (m) to replace Frappe's "bench", with commands for development, deployment, and management. Status: 97% Complete

1. CLI Framework Setup

Progress: 5/5 (100%)

Completed:

✅ Create src/framework_m/cli/main.py
✅ Use cyclopts for CLI framework (native async, better type support)
✅ Setup main CLI app:
✅ Add version command:
✅ Add help documentation for all commands

1. CLI Framework Setup > 1.1 Pluggable CLI Architecture (Entry Points)

Progress: 11/11 (100%)

Completed:

✅ Define Entry Point Group: framework_m.cli_commands
✅ Implement Plugin Loader in plugin_loader.py:
✅ Use importlib.metadata.entry_points(group="framework_m.cli_commands")
✅ Iterate over registered entry points
✅ If entry point is a cyclopts.App: app.command(plugin_app, name=ep.name)
✅ If entry point is a function: app.command(plugin_func, name=ep.name)
✅ Goal: 3rd party apps (and framework-m-studio) can add commands to m without modifying core.
✅ Customization Philosophy:
✅ Standard commands (start, test) are strict relays.
✅ For customization (e.g., start-mqtt, super-test): Users MUST register a custom command via entry points.
✅ Result: m remains a clean, standard runner.

2. Development Server (Uvicorn Relay) > 2.1 Start Command

Progress: 5/5 (100%)

Completed:

✅ Implement m start:
✅ Implementation: Wrapper around uvicorn.
✅ Argument Forwarding: Uses cyclopts Parameter for options.
✅ Usage: m start --workers 4 --log-level debug
✅ Value Add: Sets PYTHONPATH, auto-detects app, transparently passes args.

2. Development Server (Uvicorn Relay) > 2.2 Worker Command (Taskiq Relay)

Progress: 4/4 (100%)

Completed:

✅ Implement m worker:
✅ Implementation: Native Taskiq integration (better than wrapper).
✅ Options: --concurrency, --verbose.
✅ Usage: m worker --concurrency 8

2. Development Server (Uvicorn Relay) > 2.3 Studio Command (Litestar Relay)

Progress: 4/4 (100%)

Completed:

✅ Implement m studio:
✅ Implementation: Wrapper around uvicorn for Studio app.
✅ Options: --host, --port, --reload, --log-level.
✅ Usage: m studio --port 9000 --reload

3. Database Commands (Alembic Relay) > 3.1 Migration Commands

Progress: 5/5 (100%)

Completed:

✅ Implement m migrate:
✅ Implementation: Wrapper around MigrationManager (uses Alembic).
✅ Commands: migrate, create, rollback, status, history, init.
✅ Usage: m migrate create "add users" --autogenerate
✅ Value Add: Auto-detects schema changes, supports custom DB URL, env vars.

4. Scaffolding (Cruft/Cookiecutter Relay) > 4.1 New App (Starter Template)

Progress: 4/4 (100%)

Completed:

✅ Implement m new:app:
✅ Implementation: Wrapper around cruft (cookiecutter).
✅ Options: --template, --checkout, --no-input.
✅ Usage: m new:app myapp --checkout main

4. Scaffolding (Cruft/Cookiecutter Relay) > 4.2 New DocType (Jinja Relay)

Progress: 26/26 (100%)

Completed:

✅ Implement m new:doctype <name>:
✅ Implementation: Template rendering (embedded templates).
✅ Output: doctype.py, controller.py, test_*.py, __init__.py.
✅ App Detection: --app > pyproject.toml > interactive prompt.
✅ CLI Parameter (Unattended/Automation):
✅ --app <app_name>: Explicit app target.
✅ If --app provided, use it directly (no prompt).
✅ Automation Friendly: Scripts can pass --app to avoid prompts.
✅ Auto-Detection from CWD:
✅ If --app not provided, attempt to detect from current directory.
✅ Check if CWD is inside an app's source tree.
✅ Look for pyproject.toml in CWD or parent directories.
✅ Read [project] name to determine app.
✅ Interactive Prompt (Default Fallback):
✅ If detection fails AND stdin is interactive (TTY):
✅ List installed apps from framework_m.apps entry points.
✅ Prompt user to select: "Which app? [my_app, other_app]".
✅ If stdin is NOT interactive (pipe/automation):
✅ Fail with clear error: "Cannot detect app. Use --app <name>.".
✅ Exit code 1 (enables CI/automation to catch failures).
✅ Implementation: check require_app() in new.py
✅ Scaffold Output Location:
✅ Files created in: {cwd}/doctypes/{doctype_name}/
✅ Structure: __init__.py, doctype.py, controller.py, test_*.py
✅ Templates:
✅ Embedded templates in new.py (no external Jinja2 files needed).

5. Testing & Quality (Standard Tool Relays) > 5.1 Test Relay

Progress: 5/5 (100%)

Completed:

✅ Implement m test:
✅ Command: pytest
✅ Options: --verbose, --coverage, -k.
✅ Usage: m test -k "user" --verbose
✅ Value Add: Configures PYTHONPATH.

5. Testing & Quality (Standard Tool Relays) > 5.2 Lint/Format Relay

Progress: 6/6 (100%)

Completed:

✅ Implement m lint:
✅ Command: ruff check . --fix
✅ Implement m format:
✅ Command: ruff format .
✅ Implement m typecheck:
✅ Command: mypy .

6. Configuration Management > 6.1 Config File

Progress: 4/4 (100%)

Completed:

✅ Create framework_config.toml structure:
✅ Supports [framework], [apps] sections
✅ Load/save with TOML format
✅ Auto-detect in CWD or parent directories

6. Configuration Management > 6.2 Config Commands

Progress: 2/2 (100%)

Completed:

✅ Implement m config:show: Display current config.
✅ Implement m config:set <key> <value>: Update config.

7. Utility Relays > 7.1 Console (IPython Relay)

Progress: 4/4 (100%)

Completed:

✅ Implement m console:
✅ Implementation: Wrapper around ipython or python -m asyncio.
✅ Value Add: Pre-imports framework, async support.
✅ Options: --no-ipython for plain Python.

7. Utility Relays > 7.2 System Info

Progress: 3/3 (100%)

Completed:

✅ Implement m info:
✅ Show versions (Framework, Python).
✅ --verbose for platform and service info.

7. Utility Relays > 7.3 Routes

Progress: 2/2 (100%)

Completed:

✅ Implement m routes:
✅ Points to OpenAPI/Swagger docs.

8. Build Commands > 8.1 Frontend Build

Progress: 3/3 (100%)

Completed:

✅ Implement m build (placeholder):
✅ Relay to frontend build system (see Phase 09).
✅ Detects package.json and runs npm run build.

8. Build Commands > 8.2 Docker Build

Progress: 4/4 (100%)

Completed:

✅ Implement m build:docker:
✅ Command: Wrapper around docker build.
✅ Value Add: Reads image name/tag from framework_config.toml.
✅ Options: --tag, --file, --no-cache, --push.

9. Pluggable Extensions

Progress: 1/4 (25%)

Completed:

✅ Plugin loader implemented (load_plugins())

Pending:

⏳ m studio (external)
⏳ m docs:generate (external)
⏳ m codegen (external)

10. CLI Entry Point

Progress: 2/2 (100%)

Completed:

✅ Configure pyproject.toml scripts: m = "framework_m.cli.main:app"
✅ Test installation (m --help) - 22 commands registered.

Phase 06: Built-in DocTypes & Core Features

Phase: 06 Objective: Implement essential built-in DocTypes (User, Role, Permission) and core features like file attachments, audit logging, and printing. Status: 99% Complete

1. Identity & Access Management (PIM) > 1.1 Identity Provider Protocol

Progress: 4/4 (100%)

Completed:

✅ Create IdentityProtocol (Interface):
✅ get_user(id: str) -> User
✅ authenticate(credentials) -> Token
✅ get_attributes(user) -> dict[str, Any] (ABAC replacement for get_roles)

1. Identity & Access Management (PIM) > 1.2 User DocTypes (Pluggable)

Progress: 8/8 (100%)

Completed:

✅ Mode A: Local (Indie):
✅ LocalUser (SQL Table): Stores email, password_hash, full_name.
✅ LocalIdentityAdapter: argon2 password hashing, JWT generation.
✅ Mode B: Federated (Enterprise):
✅ FederatedIdentityAdapter (Proxy):
✅ No SQL Table required.
✅ Hydrates from Auth Header (X-User-ID, X-Email) via hydrate_from_headers().
✅ UserPreferences DocType stores settings locally.

1. Identity & Access Management (PIM) > 1.3 User Controller (Adapter)

Progress: 4/4 (100%)

Completed:

✅ Implement UserManager:
✅ Delegates to configured IdentityProvider via IdentityProtocol.
✅ Abstraction layer: user = await user_manager.get(id).
✅ Additional create() method for Indie mode with password hashing.

1. Identity & Access Management (PIM) > 1.4 Auth API (Indie Mode)

Progress: 8/8 (100%)

Completed:

✅ Create GET /api/v1/auth/me:
✅ Returns current user context (or 401).
✅ Create POST /api/v1/auth/login (Indie Only):
✅ Accepts username, password.
✅ Validates against LocalUser via UserManager.authenticate().
✅ Returns JWT token.
✅ Create POST /api/v1/auth/logout:
✅ Returns success message (JWT auth is client-side logout).

1. Identity & Access Management (PIM) > 1.5 Authentication Strategies (Multi-Method)

Progress: 24/24 (100%)

Completed:

✅ Create AuthenticationProtocol (Interface):
✅ supports(headers: Mapping) -> bool — Can this strategy handle this request?
✅ authenticate(headers: Mapping) -> UserContext | None — Extract user from request.
✅ Built-in Strategies:
✅ SessionCookieAuth — Browser sessions via cookies.
✅ BearerTokenAuth — JWT/OAuth2 access tokens.
✅ ApiKeyAuth — X-API-Key header for scripts/integrations.
✅ HeaderAuth — Federated mode gateway header hydration.
✅ BasicAuth — HTTP Basic for CLI tools.
✅ Strategy Chain (AuthChain):
✅ Try each strategy in priority order.
✅ First successful match wins.
✅ Configurable order via framework_config.toml (create_auth_chain_from_config).
✅ API Key DocType (Indie Mode):
✅ ApiKey DocType:
✅ key_hash: str — Hashed API key (never store plaintext, excluded from serialization).
✅ user_id: str — Owner of the key.
✅ name: str — Human-readable label.
✅ scopes: list[str] — Optional permission scopes.
✅ expires_at: datetime | None — Expiration.
✅ last_used_at: datetime | None — Usage tracking.
✅ POST /api/v1/auth/api-keys — Create new key.
✅ DELETE /api/v1/auth/api-keys/{id} — Revoke key.
✅ GET /api/v1/auth/api-keys — List user's keys.

Progress: 26/27 (96%)

Completed:

✅ Option A: Built-in OAuth2 (Indie):
✅ Supported Providers (config structure ready):
✅ Google (well-known URLs configured)
✅ GitHub (well-known URLs configured)
✅ Microsoft (well-known URLs configured)
✅ Generic OIDC (any compliant provider via get_oidc_well_known)
✅ Configuration via framework_config.toml:
✅ Endpoints:
✅ GET /api/v1/auth/oauth/{provider}/start — Redirect to provider.
✅ GET /api/v1/auth/oauth/{provider}/callback — Handle callback. (Placeholder)
✅ Option B: Federated (Enterprise):
✅ Delegate to Keycloak / Authelia / Auth0 via HeaderAuth strategy.
✅ Framework receives headers only (X-User-ID, X-Email).
✅ Zero PII stored locally.
✅ SocialAccount DocType:
✅ provider: str — OAuth provider name.
✅ provider_user_id: str — Unique ID from provider.
✅ user_id: str — Links to local user.
✅ email: str | None — For lookup (optional).
✅ display_name: str — For UI display.
✅ One User can have multiple SocialAccounts.
✅ Passwordless Option (Requires Section 10: Email):
✅ POST /api/v1/auth/magic-link — Send email with login link.
✅ GET /api/v1/auth/magic-link/{token} — Verify and create session.
✅ MagicLinkCredentials added to identity protocol
✅ Token generation with HMAC-SHA256 signing

Pending:

⏳ Use authlib for OAuth2 flows. (Full implementation deferred)

1. Identity & Access Management (PIM) > 1.7 Session Management

Progress: 20/20 (100%)

Completed:

✅ Session Storage:
✅ Redis (Default): RedisSessionAdapter with JSON storage and TTL.
✅ Database (Fallback): DatabaseSessionAdapter for indie apps.
✅ Configurable via framework_config.toml:
✅ Session DocType (database backend):
✅ session_id: str — Unique session identifier.
✅ user_id: str — Links to user.
✅ expires_at: datetime
✅ ip_address: str | None
✅ user_agent: str | None
✅ SessionProtocol Interface:
✅ create() — Create new session
✅ get() — Retrieve by ID
✅ delete() — Remove session
✅ delete_all_for_user() — Logout all
✅ list_for_user() — List active sessions
✅ Session API:
✅ GET /api/v1/auth/sessions — List active sessions.
✅ DELETE /api/v1/auth/sessions/{id} — Revoke session.
✅ DELETE /api/v1/auth/sessions — Logout all.

1. Identity & Access Management (PIM) > 1.8 PII Handling (Indie Guidance)

Progress: 16/16 (100%)

Completed:

✅ Auth Mode Presets (AuthMode enum in core/pii.py):
✅ Minimal PII Pattern:
✅ LocalUser stores only:
✅ email (for login/notification)
✅ display_name (for UI)
✅ password_hash (only if mode = "local")
✅ Do NOT store by default (PII_SENSITIVE_FIELDS):
✅ Full legal name
✅ Phone number
✅ Address
✅ Date of birth
✅ Helper: is_sensitive_pii(field_name) to detect sensitive fields.
✅ Data Deletion (DeletionMode enum):
✅ DELETE /api/v1/auth/me — Delete user account (GDPR right to erasure).
✅ GET /api/v1/auth/me/data — Export user data (GDPR data portability).
✅ Configurable: Hard delete vs anonymize.

2. Tenancy & Attributes (Multi-Tenant Core) > 2.1 Tenant Protocol

Progress: 5/5 (100%)

Completed:

✅ Create TenantProtocol (Interface in core/interfaces/tenant.py):
✅ get_current_tenant() -> str
✅ get_tenant_attributes(tenant_id) -> dict
✅ TenantContext model for request context
✅ Configuration helpers: is_multi_tenant(), get_default_tenant_id()

2. Tenancy & Attributes (Multi-Tenant Core) > 2.2 Tenancy Adapters

Progress: 12/12 (100%)

Completed:

✅ Mode A: Single Tenant (Indie):
✅ ImplicitTenantAdapter (adapters/tenant.py):
✅ tenant_id="default" (configurable).
✅ attributes={"plan": "unlimited", "features": "*"}.
✅ get_context() returns TenantContext with is_default=True.
✅ Factory: create_tenant_adapter_from_headers() auto-selects.
✅ Mode B: Multi-Tenant (Enterprise):
✅ HeaderTenantAdapter (adapters/tenant.py):
✅ Extracts X-Tenant-ID from Gateway headers.
✅ Extracts X-Tenant-Attributes (JSON) for feature flags, plan.
✅ get_context() parses headers into TenantContext.
✅ Benefit: Feature toggling per tenant without DB lookup.

2. Tenancy & Attributes (Multi-Tenant Core) > 2.3 Attributes (The New Roles)

Progress: 7/7 (100%)

Completed:

✅ Goal: Replace "Roles" with refined "Attributes" (ABAC).
✅ Implementation (in UserContext):
✅ user.attributes (dict): Stores {"department": "sales", "level": 5, "roles": ["admin"]}.
✅ user.get_attribute(key, default) — Get attribute value.
✅ user.has_attribute(key, value) — Check attribute equality.
✅ Legacy Support: user.has_role("x") checks both roles list AND attributes.roles.
✅ is_system_user uses has_role("System") for consistency.

3. Permission Management (Pluggable) > 3.1 Permission Protocol

Progress: 7/7 (100%)

Completed:

✅ Create PermissionProtocol (core/interfaces/permission.py):
✅ PolicyEvaluateRequest - Stateless authorization request
✅ PolicyEvaluateResult - Result with authorized, decision_source, reason
✅ async def evaluate(request) -> PolicyEvaluateResult
✅ async def get_permitted_filters(...) -> dict - RLS support
✅ PermissionAction enum (read, write, create, delete, submit, cancel, amend)
✅ DecisionSource enum (rbac, abac, rebac, combo, custom)

3. Permission Management (Pluggable) > 3.2 Permission Adapters

Progress: 18/18 (100%)

Completed:

✅ Mode A: Standard (Indie) - RbacPermissionAdapter:
✅ Reads permissions from DocType.Meta.permissions.
✅ Checks CustomPermission rules from database.
✅ Role matching against principal_attributes["roles"].
✅ Admin role bypass (configurable).
✅ RLS via get_permitted_filters() using Meta.rls_field.
✅ Mode B: Citadel Policy (Enterprise) - CitadelPolicyAdapter:
✅ AuthorizationRequest model (Cedar-compatible format):
✅ Principal: user.id
✅ Action: doctype:action (e.g., Invoice:create)
✅ Resource: doctype:name (e.g., Invoice:INV-001)
✅ TenantID: user.tenant_id
✅ Context: doc.as_dict()
✅ PrincipalAttributes: user.attributes (Diet Claims)
✅ AuthorizationResponse model with allowed, reason, policy_id
✅ Calls configurable policy endpoint (Cedar/OPA)
✅ Fallback behavior on error (configurable)
✅ Configuration via [permissions.citadel] in framework_config.toml

4. File Management > 4.1 File DocType

Progress: 10/10 (100%)

Completed:

✅ Create src/framework_m/core/doctypes/file.py
✅ Define File DocType:
✅ name: str - File ID (inherited from BaseDocType)
✅ file_name: str - Original filename
✅ file_url: str - Storage URL
✅ file_size: int - Size in bytes
✅ content_type: str - MIME type
✅ attached_to_doctype: str | None
✅ attached_to_name: str | None
✅ is_private: bool - Private/public file (default: True)

4. File Management > 4.2 File Upload API

Progress: 9/9 (100%)

Completed:

✅ Create POST /api/v1/file/upload (adapters/web/file_routes.py):
✅ Accept multipart/form-data via UploadFile
✅ Validate file size (max 10MB default, configurable)
✅ Validate file type (configurable whitelist)
✅ Generate unique storage path (YYYY/MM/DD/random_filename)
✅ Create File DocType record
✅ Return file URL and metadata
✅ Configuration via [files] in framework_config.toml
✅ Response models: FileUploadResponse, FileDeleteResponse

4. File Management > 4.3 File Download API

Progress: 8/8 (100%)

Completed:

✅ Create GET /api/v1/file/{file_id}:
✅ Check permissions for private files (RLS on owner)
✅ Prepare streaming response from storage
✅ Set correct Content-Type header
✅ Set Content-Disposition header (inline/attachment)
✅ Support ?inline=true for browser display
✅ Create GET /api/v1/file/{file_id}/info:
✅ Return metadata without downloading content

4. File Management > 4.4 Storage Adapters

Progress: 13/13 (100%)

Completed:

✅ Implement LocalStorageAdapter (adapters/storage/local.py):
✅ Save files to local directory with atomic writes
✅ Organize by date (YYYY/MM/DD) via path parameter
✅ Generate unique filenames (handled by file_routes)
✅ Path traversal protection
✅ Async operations via aiofiles
✅ Implement S3StorageAdapter (adapters/storage/s3.py):
✅ Use aioboto3 for async S3 operations
✅ Presigned URLs for direct client uploads (PUT method)
✅ Multipart upload for large files (>25MB)
✅ Configure bucket and region from framework_config.toml
✅ Support S3-compatible services (MinIO, DigitalOcean Spaces)
✅ InMemoryStorageAdapter for testing (adapters/storage/memory.py)

5. Audit Logging (Pluggable) > 5.1 Audit Log Protocol

Progress: 5/5 (100%)

Completed:

✅ Create AuditLogProtocol (core/interfaces/audit.py):
✅ AuditEntry model (user, action, doctype, document_id, changes, metadata)
✅ async def log(...) - Record audit entry
✅ async def query(...) - Query with filters/pagination
✅ InMemoryAuditAdapter for testing

5. Audit Logging (Pluggable) > 5.2 Audit Adapters

Progress: 6/7 (86%)

Completed:

✅ Mode A: Database (Indie):
✅ DatabaseAuditAdapter: Writes to ActivityLog records.
✅ Uses ActivityLog DocType from 5.3
✅ Mode B: External (Enterprise):
✅ FileAuditAdapter: Writes to audit.log (JSONL) for Splunk/Filebeat.
✅ Prevents main DB bloat.

Pending:

⏳ ElasticAuditAdapter: Writes directly to Elasticsearch (Phase 10).

5. Audit Logging (Pluggable) > 5.3 Activity Log DocType (Indie Only)

Progress: 10/10 (100%)

Completed:

✅ Create src/framework_m/core/doctypes/activity_log.py
✅ Define ActivityLog DocType:
✅ user_id: str
✅ action: str (create, read, update, delete)
✅ doctype: str
✅ document_id: str
✅ timestamp: datetime
✅ changes: dict | None
✅ metadata: dict | None
✅ Immutable (no write/delete permissions)

5. Audit Logging (Pluggable) > 5.4 Activity Feed API

Progress: 4/4 (100%)

Completed:

✅ Create GET /api/v1/activity:
✅ List recent activities
✅ Filter by user, doctype, document, action
✅ Paginate results (limit, offset)

6. Error Logging > 6.1 Error Log DocType

Progress: 12/12 (100%)

Completed:

✅ Create src/framework_m/core/doctypes/error_log.py
✅ Define ErrorLog DocType:
✅ title: str - Short error description
✅ error_type: str - Exception class name
✅ error_message: str - Full error message
✅ traceback: str | None - Full stack trace
✅ request_url: str | None
✅ user_id: str | None
✅ request_id: str | None - For log correlation
✅ timestamp: datetime
✅ context: dict | None - Additional context
✅ Admin-only permissions (immutable)

6. Error Logging > 6.2 Error Logging Integration

Progress: 6/6 (100%)

Completed:

✅ Add global exception handler (adapters/web/error_handler.py):
✅ Catch all unhandled exceptions
✅ Create ErrorLog entry with full context
✅ Return user-friendly ErrorResponse (no sensitive data)
✅ Log to console in dev mode (configurable)
✅ create_error_handler() factory for Litestar

7. Printing & PDF Generation > 7.1 Print Format DocType

Progress: 10/10 (100%)

Completed:

✅ Create src/framework_m/core/doctypes/print_format.py
✅ Define PrintFormat DocType:
✅ name: str - Display name
✅ doctype: str - Target DocType
✅ template: str - Jinja2 template path
✅ is_default: bool
✅ css: str | None - Custom CSS
✅ header_html: str | None, footer_html: str | None
✅ page_size: str (A4, Letter, etc.)
✅ orientation: str (portrait, landscape)

7. Printing & PDF Generation > 7.2 Jinja Print Adapter

Progress: 8/8 (100%)

Completed:

✅ Create src/framework_m/adapters/print/jinja_adapter.py
✅ Implement JinjaPrintAdapter:
✅ async def render_html(doc, template) -> str
✅ Load Jinja2 template
✅ Render with document context
✅ Return HTML string
✅ async def render_html_string(template_string, doc) -> str
✅ Custom filters: currency, date, datetime

7. Printing & PDF Generation > 7.3 Gotenberg PDF Adapter

Progress: 8/8 (100%)

Completed:

✅ Create src/framework_m/adapters/print/gotenberg_adapter.py
✅ Implement GotenbergPrintAdapter:
✅ async def html_to_pdf(html, page_size, orientation, ...) -> bytes
✅ Send HTML to Gotenberg service
✅ Return PDF bytes
✅ async def is_available() -> bool - Health check
✅ GotenbergConfig with URL, timeout, page size, margins
✅ Page sizes: A4, Letter, Legal, A3, A5

7. Printing & PDF Generation > 7.4 Print API

Progress: 9/9 (100%)

Completed:

✅ Create GET /api/v1/print/{doctype}/{id}:
✅ Query parameters:
✅ format - pdf/html (default: pdf)
✅ print_format - Custom format name
✅ Load document (TODO: integrate with repository - phase 10)
✅ Check read permission (TODO: integrate with permission - phase 10)
✅ Render using JinjaPrintAdapter → GotenbergPrintAdapter
✅ Return PDF or HTML with appropriate headers
✅ Fallback to HTML if Gotenberg unavailable

8. System Settings > 8.1 System Settings DocType

Progress: 11/11 (100%)

Completed:

✅ Create src/framework_m/core/doctypes/system_settings.py
✅ Define SystemSettings DocType (singleton):
✅ name: str - Always "System Settings" (frozen)
✅ app_name: str - Application display name
✅ timezone: str - Default timezone
✅ date_format: str - Date format string
✅ time_format: str - Time format string
✅ language: str - Default language code
✅ enable_signup: bool - Allow registration
✅ session_expiry: int - Session timeout in minutes
✅ maintenance_mode: bool - Admin-only access

8. System Settings > 8.2 Settings API

Progress: 5/5 (100%)

Completed:

✅ Create GET /api/v1/settings:
✅ Return system settings via SettingsResponse
✅ Cache aggressively (5 min Cache-Control header)
✅ invalidate_settings_cache() helper
✅ set_settings_cache() helper

9. Notification System > 9.1 Notification DocType

Progress: 13/13 (100%)

Completed:

✅ Create src/framework_m/core/doctypes/notification.py
✅ Define Notification DocType:
✅ user_id: str - Recipient
✅ subject: str
✅ message: str
✅ notification_type: str (info, success, warning, error, etc.)
✅ read: bool
✅ doctype: str | None - Related DocType
✅ document_id: str | None
✅ timestamp: datetime
✅ from_user: str | None
✅ metadata: dict | None
✅ RLS on user_id field

9. Notification System > 9.2 Notification API

Progress: 9/9 (100%)

Completed:

✅ Create GET /api/v1/notifications:
✅ List user's notifications
✅ Filter by read/unread
✅ Create PATCH /api/v1/notifications/{id}/read - Mark as read
✅ Create DELETE /api/v1/notifications/{id} - Delete notification
✅ Create WebSocket endpoint for real-time notifications:
✅ WS /api/v1/notifications/stream - Real-time notification stream
✅ push_notification(user_id, notification) - Push to connected clients
✅ create_notification_websocket_router() factory

10. Email Integration > 10.1 Email Queue DocType

Progress: 20/20 (100%)

Completed:

✅ Create src/framework_m/core/doctypes/email_queue.py
✅ Define EmailQueue DocType:
✅ to: list[str] - Recipients
✅ cc/bcc: list[str] | None
✅ subject: str
✅ body: str - HTML body
✅ text_body: str | None - Plain text alt
✅ status: str - Queued/Sending/Sent/Failed/Cancelled
✅ priority: str - low/normal/high
✅ error: str | None
✅ retry_count/max_retries: int
✅ queued_at/sent_at: datetime
✅ reference_doctype/reference_id - for tracking
✅ Port-Adapter Pattern:
✅ EmailQueueProtocol (interface/port)
✅ DatabaseEmailQueueAdapter (default - uses EmailQueue DocType)
✅ InMemoryEmailQueueAdapter (testing)
✅ queue_email() helper function
✅ configure_email_queue() for swapping adapters
✅ Ready for NotificationServiceAdapter integration

10. Email Integration > 10.2 Email Sender

Progress: 13/13 (100%)

Completed:

✅ Create email sender protocol and adapters:
✅ EmailSenderProtocol (interface/port)
✅ SMTPEmailSender - SMTP delivery adapter
✅ LogEmailSender - Console logging (development)
✅ SMTPConfig - Configuration dataclass
✅ Create background job EmailProcessor:
✅ process_queue() - Process pending emails
✅ process_single(queue_id) - Process single email
✅ Update EmailQueue status
✅ Retry logic (placeholder)
✅ Add helper function:
✅ queue_email() - Queue for sending
✅ send_queued_email() - Process single email

11. Testing > 11.1 Unit Tests

Progress: 5/5 (100%)

Completed:

✅ Test User DocType validation (core/doctypes/test_user.py, core/services/test_user_manager.py)
✅ Test Role assignment (adapters/auth/test_rbac_permission.py)
✅ Test Permission checking (8 permission test files)
✅ Test File upload/download (core/test_file.py, adapters/web/test_file_routes.py)
✅ Test Print rendering (core/test_print_format.py, adapters/print/test_*.py)

11. Testing > 11.2 Integration Tests

Progress: 5/5 (100%)

Completed:

✅ Test user creation and login (integration/test_api_endpoints.py)
✅ Test permission enforcement (core/test_object_level_permissions.py)
✅ Test file storage (local and S3) (adapters/storage/test_local.py, adapters/storage/test_s3.py)
✅ Test PDF generation with Gotenberg (adapters/print/test_gotenberg_adapter.py)
✅ Test email sending (adapters/email/test_sender_processor.py)

Validation Checklist

Progress: 6/6 (100%)

Completed:

✅ User and Role management works
✅ Permissions are enforced correctly
✅ Files can be uploaded and downloaded
✅ Audit logs are created
✅ PDFs can be generated
✅ Emails can be sent

Phase 07: Studio (Code Generation UI)

Phase: 07 Objective: Build a visual DocType builder (framework-m-studio) that generates Python code, running as a Litestar app. Status: 93% Complete

1. Packaging Strategy (Runtime vs DevTools) > 1.1 framework-m (Runtime)

Progress: 4/4 (100%)

Completed:

✅ Goal: Production-grade kernel.
✅ Content: API Server, ORM, DocType Engine, Basic CLI (start, migrate, worker).
✅ Dependencies: litestar, sqlalchemy, pydantic, cyclopts.
✅ Excluded: Code generators, Studio UI, Heavy formatting/linting libs.

1. Packaging Strategy (Runtime vs DevTools) > 1.2 framework-m-studio (DevTools)

Progress: 4/4 (100%)

Completed:

✅ Goal: Developer Productivity & SaaS Component.
✅ Content: Studio UI, Code Generators, Visual Editors.
✅ Dependencies: framework-m, libcst, jinja2.
✅ Integration: Plugs into m CLI via entry points (m codegen).

2. Studio Backend (Litestar App) > 1. Project Structure (Monorepo)

Progress: 6/6 (100%)

Completed:

✅ Create apps/studio/ in the workspace
✅ Initialize apps/studio/pyproject.toml:
✅ Add dependency: framework-m = { workspace = true }
✅ Add dependency: libcst (for code modification)
✅ Add dependency: jinja2 (for code generation)
✅ Add code generation endpoints

2. Studio Backend (Litestar App) > 1.2 File System API

Progress: 14/14 (100%)

Completed:

✅ Create GET /studio/api/doctypes:
✅ Scan project for *.py files containing DocType classes
✅ Return list of DocTypes with metadata
✅ Create GET /studio/api/doctype/{name}:
✅ Read DocType file
✅ Parse with LibCST
✅ Return structured JSON
✅ Create POST /studio/api/doctype/{name}:
✅ Accept DocType schema JSON
✅ Generate Python code
✅ Write to file system
✅ Create DELETE /studio/api/doctype/{name}:
✅ Delete DocType file
✅ Delete Controller file (if exists)

2. LibCST Code Transformer > 2.1 Parser

Progress: 7/7 (100%)

Completed:

✅ Create src/framework_m_studio/codegen/parser.py
✅ Implement parse_doctype(file_path: str) -> dict:
✅ Use LibCST to parse Python file
✅ Extract class definition
✅ Extract fields with types and defaults
✅ Extract Config metadata
✅ Return structured dict

2. LibCST Code Transformer > 2.2 Generators (The Strategy)

Progress: 8/8 (100%)

Completed:

✅ Creation (Scaffolding):
✅ Use jinja2 templates (shared with CLI m new:doctype).
✅ Goal: Generate clean, standard Python code from scratch.
✅ Implement generate_doctype_source(schema: dict) -> str.
✅ Mutation (Transformer):
✅ Use LibCST to parse and modify existing files.
✅ Goal: Add/Edit fields while preserving comments and custom methods.
✅ Implement update_doctype_source(source: str, schema: dict) -> str.

2. LibCST Code Transformer > 2.3 Test Generator

Progress: 6/6 (100%)

Completed:

✅ Create src/framework_m_studio/codegen/test_generator.py
✅ Implement generate_test(schema: dict) -> str:
✅ Generate test_{doctype}.py
✅ Import pytest and DocType
✅ Generate basic CRUD test (Create, Read, Update, Delete)
✅ Generate validation failure test (if required fields exist)

2. LibCST Code Transformer > 2.4 Transformer

Progress: 12/12 (100%)

Completed:

✅ Create src/framework_m_studio/codegen/transformer.py
✅ Implement update_doctype(file_path: str, schema: dict):
✅ Parse existing file with LibCST
✅ Locate DocType class
✅ Update/add/remove fields
✅ Preserve comments and custom methods
✅ Write back to file
✅ Handle edge cases:
✅ Field renamed (detect and update)
✅ Field type changed
✅ Field deleted (remove from class)
✅ Custom methods preserved

3. Studio Frontend > 3.1 React App Setup (Refine Stack)

Progress: 8/8 (100%)

Completed:

✅ Create apps/studio/studio_ui/ directory
✅ Initialize Refine app:
✅ Install dependencies per ADR-0005:
✅ Create Framework M Data Provider (src/providers/dataProvider.ts):
✅ Configure API base URLs (per ADR-0005):
✅ Read from window.__FRAMEWORK_CONFIG__ or env vars
✅ Support same-origin, subdomain, and CDN scenarios
✅ Default to /studio/api for Studio endpoints

3. Studio Frontend > 3.2 DocType List View (Refine Resource)

Progress: 5/5 (100%)

Completed:

✅ Create pages/doctypes/list.tsx:
✅ Use useList hook from @refinedev/core to fetch DocTypes
✅ Display with @tanstack/react-table + Tailwind styling
✅ Add search/filter with useTable globalFilter
✅ Add "New DocType" button → navigates to /doctypes/create

3. Studio Frontend > 3.3 DocType Editor (Refine Form)

Progress: 5/5 (100%)

Completed:

✅ Create pages/doctypes/edit.tsx:
✅ Use useOne/useUpdate/useCreate hooks for data
✅ Left panel: Field list with @dnd-kit/sortable
✅ Right panel: Field properties form (name, type, required, default, description)
✅ Top bar: DocType name input + Save button

3. Studio Frontend > 3.4 Field Editor (RJSF Schema-Driven)

Progress: 5/5 (100%)

Completed:

✅ Create components/FieldEditor.tsx:
✅ Render with RJSF using JSON Schema for field properties
✅ Custom Tailwind widgets: TextWidget, SelectWidget, CheckboxWidget, TextareaWidget
✅ Dynamic "Validators" collapsible for min/max length, pattern, etc.
✅ Live preview of generated Python type annotation

3. Studio Frontend > 3.5 Drag & Drop

Progress: 4/4 (100%)

Completed:

✅ Implement field reordering:
✅ Use @dnd-kit/core + @dnd-kit/sortable
✅ Allow dragging fields to reorder
✅ Update schema on drop (via arrayMove)

3. Studio Frontend > 3.6 Code Preview (Monaco Editor)

Progress: 6/6 (100%)

Completed:

✅ Create components/CodePreview.tsx:
✅ Use @monaco-editor/react with language="python"
✅ Read-only mode (readOnly: true)
✅ Theme: vs-dark
✅ Auto-update on schema change (memoized generation)
✅ Copy button for generated code

3. Studio Frontend > 3.7 Save Flow

Progress: 5/5 (100%)

Completed:

✅ Implement save functionality:
✅ Validate schema (name required)
✅ Call backend API via useCreate/useUpdate
✅ Navigate back to list on success
✅ Loading state on save button

3. Studio Frontend > 3.8 Visual Data Modeling (Mind Map)

Progress: 5/5 (100%)

Completed:

✅ Implement ERD/Mind Map View:
✅ Central Node: Current DocType (highlighted)
✅ Connected Nodes: Related DocTypes via Link fields
✅ Visual Action: Drag to connect nodes → Creates Link Field
✅ Using @xyflow/react (React Flow v12)

3. Studio Frontend > 3.9 Layout Designer (Grid System)

Progress: 5/5 (100%)

Completed:

✅ Implement Visual Layout Drag & Drop:
✅ Define Sections with title and Columns (1/2/3 cols selector)
✅ Drag fields from palette into grid cells
✅ Real-time form preview toggle
✅ Add/delete sections

3. Studio Frontend > 3.10 Module Explorer

Progress: 5/5 (100%)

Completed:

✅ Implement Module Scanner:
✅ Group DocTypes by directory/app module (tree structure)
✅ Tree view navigation in Sidebar
✅ Expand/collapse all, search filter
✅ Click to navigate to DocType editor

4. Studio Cloud Mode (Ephemeral & Git-Backed) > 4.3 Git Adapter (Data Agnostic)

Progress: 11/11 (100%)

Completed:

✅ Create src/framework_m_studio/git/protocol.py - GitAdapterProtocol (Port)
✅ Create src/framework_m_studio/git/adapter.py - GitAdapter (Adapter)
✅ Implement GitAdapter:
✅ Wraps standard git binary via asyncio subprocess.
✅ clone(repo_url, auth): Clone to temp dir.
✅ commit(message): Stage and commit changes.
✅ push(branch): Push to remote.
✅ pull(): Pull latest changes.
✅ create_branch(name): Create new branch.
✅ get_status(): Get workspace status.
✅ Unit tests in tests/test_git_adapter.py (12 tests)

4. Studio Cloud Mode (Ephemeral & Git-Backed) > 4.4 GitHub Provider

Progress: 9/9 (100%)

Completed:

✅ Create src/framework_m_studio/git/github_provider.py
✅ Implement GitHubProvider:
✅ Use GitHub API for operations (REST API with urllib)
✅ Support personal access tokens (Bearer auth)
✅ Handle authentication (GitHubAuthError)
✅ Create pull requests
✅ List pull requests
✅ Validate tokens
✅ Unit tests in tests/test_github_provider.py (9 tests)

4. Studio Cloud Mode (Ephemeral & Git-Backed) > 4.5 Generic Git Provider

Progress: 2/2 (100%)

Completed:

✅ Support HTTPS and SSH (via git CLI)
✅ Work with any Git server (generic implementation)

4. Studio Cloud Mode (Ephemeral & Git-Backed) > 4.6 Workspace Management

Progress: 7/7 (100%)

Completed:

✅ Create src/framework_m_studio/workspace.py
✅ Implement workspace lifecycle:
✅ Clone repo to temp directory
✅ Track workspace sessions
✅ Clean up old workspaces (TTL-based)
✅ Handle concurrent edits (asyncio lock)
✅ Unit tests in tests/test_workspace.py (12 tests)

4. Studio Cloud Mode (Ephemeral & Git-Backed) > 4.7 Cloud Studio API

Progress: 12/12 (100%)

Completed:

✅ Create POST /studio/api/workspace/connect:
✅ Accept repo URL and token
✅ Clone repository
✅ Return workspace ID
✅ Create POST /studio/api/workspace/{id}/commit:
✅ Commit changes
✅ Push to branch
✅ Return commit SHA
✅ Create GET /studio/api/workspace/{id}/status: Git status
✅ Create POST /studio/api/workspace/{id}/pull: Pull latest
✅ Create DELETE /studio/api/workspace/{id}: Cleanup workspace
✅ Create GET /studio/api/workspace/sessions: List all sessions

5. Studio CLI Command > 5.1 SPA Packaging & Serving (Path Prefix Architecture)

Progress: 20/21 (95%)

Completed:

✅ Build Process:
✅ pnpm run build in studio_ui outputs to src/framework_m_studio/static/ (vite.config.ts).
✅ pyproject.toml includes framework_m_studio/static/**/*.
✅ Serving Strategy (Litestar) (in app.py):
✅ UI Routes: Serve /studio/ui/* → React SPA (HTML pages)
✅ API Routes: Serve /studio/api/* → JSON endpoints
✅ Assets: Serve /studio/ui/assets/* as static files (Cached)
✅ SPA Catch-all: /studio/ui/{path:path} → returns index.html for client-side routing
✅ Redirect: /studio → /studio/ui/ for convenience
✅ Content-Type Fix: All responses use explicit media_type to prevent download issues
✅ Implementation: serve_spa() and _get_spa_response() functions in app.py
✅ Implement m studio (in framework-m-studio package):
✅ Register CLI command via entry point:
✅ Start uvicorn server on port 9000 (default)
✅ Serve Studio UI
✅ Options:
✅ --port - Custom port
✅ --host - Custom host
✅ --reload - Development mode
✅ --cloud - Enable cloud mode

Pending:

⏳ Open browser automatically (TODO)

6. DevTools CLI (Extensions) > 6.1 Documentation Generator (`m docs:generate`)

Progress: 6/6 (100%)

Completed:

✅ Dependencies: Uses stdlib urllib, json, subprocess (no external deps required).
✅ Command: m docs:generate:
✅ API Reference: Generates markdown from DocTypes in docs_generator.py.
✅ OpenAPI Export: Exports openapi.json from URL (--openapi-url).
✅ Site Build: Runs mkdocs build if available (--build-site).
✅ Unit tests in tests/test_docs_generator.py (10 tests)

6. DevTools CLI (Extensions) > 6.2 Client SDK Generator (`m codegen client`)

Progress: 8/8 (100%)

Completed:

✅ Dependencies: Uses stdlib only (no external deps required).
✅ Command: m codegen client:
✅ Flow:
✅ Arguments:
✅ TypeScript: Generates interfaces + fetch client in sdk_generator.py.
✅ Python: Generates Pydantic models in sdk_generator.py.
✅ Unit tests in tests/test_codegen.py (6 tests)
✅ CLI tests updated in tests/test_cli.py

7. Field Type Library > 7.1 Supported Field Types

Progress: 13/13 (100%)

Completed:

✅ Implement field type definitions:
✅ Text (str)
✅ Number (int, float)
✅ Checkbox (bool)
✅ Date (date)
✅ DateTime (datetime)
✅ Select (enum)
✅ Link (foreign key)
✅ Table (child table)
✅ Level 1: Grid/Table view
✅ Level 2+: Drill-down pattern (Button -> Drawer/Dialog) for infinite nesting
✅ JSON (dict)
✅ File (file upload)

7. Field Type Library > 7.2 Custom Field Type Discovery

Progress: 5/5 (100%)

Completed:

✅ Dynamic Loading: Studio reads available types from FieldRegistry at runtime.
✅ API Endpoint: GET /studio/api/field-types
✅ Returns: Built-in types + Types registered by installed apps.
✅ Each type includes: name, pydantic_type, sqlalchemy_type, ui_component (optional).
✅ UI Integration: Field Type Selector in DocType Editor dynamically populates from this API.

7. Field Type Library > 7.3 Custom UI Components (Client-Side)

Progress: 6/6 (100%)

Completed:

✅ Goal: Allow apps to register custom React components for their field types.
✅ Registration (App-Side):
✅ Discovery (Studio-Side):
✅ Shadow Build (Phase 09) loads all app frontend/ plugins.
✅ Studio UI checks window.__STUDIO_FIELD_COMPONENTS__ map.
✅ Falls back to default <input> if no custom component.

7. Field Type Library > 7.5 Live Preview & Sandbox Mode

Progress: 0/11 (0%)

Pending:

⏳ Recommendation: Default to feature branches for teams. Add branch selector to Studio UI.
⏳ Goal: Preview the DocType being edited with mock data AND simulate CRUD.
⏳ Implementation:
⏳ Preview Tab: Toggle between "Schema Editor" and "Sandbox".
⏳ Mock Data Generation:
⏳ Use @faker-js/faker or simple generators.
⏳ Generate sample rows based on field types.
⏳ Form Preview: Render AutoForm with generated mock doc.
⏳ List Preview: Render AutoTable with 5-10 mock rows.
⏳ Schema Source: Uses the in-memory schema being edited, NOT the saved file.
⏳ Allows instant feedback without saving.

7. Field Type Library > 7.6 Sandbox: Interactive CRUD Simulation

Progress: 11/11 (100%)

Completed:

✅ Goal: Let users test the full UX before committing the DocType.
✅ Features:
✅ New: Create a new mock document. Test mandatory field validation.
✅ Update: Edit an existing mock document. See form behavior.
✅ Delete: Delete mock document. See confirmation dialogs.
✅ List (Few): Show 5 rows. Test basic list rendering.
✅ List (Many): Paginate 100+ mock rows. Test performance.
✅ Filter: Apply filters and see results change.
✅ Validation: Submit form with missing required fields. See error messages.
✅ Data Store: In-memory array (resets on page reload).
✅ Benefit: No database required. Test UX before writing to disk.

7. Field Type Library > 7.7 Local Hot Reload (Optional Enhancement)

Progress: 0/8 (0%)

Pending:

⏳ Goal: For users who want to test with a real local database instead of mock data.
⏳ Prerequisites: Local Postgres/SQLite + m start running.
⏳ Flow:
⏳ Studio saves DocType to file.
⏳ File watcher detects change → triggers schema sync.
⏳ Table created/updated in local DB.
⏳ User can now test with real CRUD operations.
⏳ Benefit: Bridge between mock sandbox and production—test with actual data locally.

7. Field Type Library > 7.2 Field Metadata

Progress: 7/7 (100%)

Completed:

✅ Define field options:
✅ Label
✅ Description
✅ Required
✅ Default value
✅ Validation rules
✅ Display options (hidden, read-only)

8. Controller Scaffolding

Progress: 8/8 (100%)

Completed:

✅ Add controller generation:
✅ Generate controller file template
✅ Add common hook methods
✅ Add validation examples
✅ Controller editor in UI:
✅ List hook methods
✅ Add custom methods
✅ Code editor for method bodies

9. Testing > 9.1 Unit Tests

Progress: 8/8 (100%)

Completed:

✅ Test LibCST parser:
✅ Parse valid DocType
✅ Extract fields correctly
✅ Handle edge cases
✅ Test code generator:
✅ Generate valid Python code
✅ Format correctly
✅ Handle all field types

9. Testing > 9.2 Integration Tests

Progress: 11/11 (100%)

Completed:

✅ Test full flow:
✅ Create DocType via UI
✅ Save to file system
✅ Reload and verify
✅ Update DocType
✅ Verify changes preserved
✅ Test Git integration:
✅ Clone repository
✅ Make changes
✅ Commit and push
✅ Verify on GitHub

10. Documentation

Progress: 6/6 (100%)

Completed:

✅ Create Studio user guide:
✅ How to start Studio
✅ How to create DocType
✅ How to add fields
✅ How to configure permissions
✅ How to use Git mode

Validation Checklist

Progress: 6/6 (100%)

Completed:

✅ Studio UI can create and edit DocTypes
✅ Generated code is valid Python
✅ Changes are saved to file system
✅ LibCST preserves custom code
✅ Git mode can commit and push
✅ Studio works in both local and cloud modes

Phase 08: Workflows & Advanced Features

Phase: 08 Objective: Implement pluggable workflow engine, DocType overrides, app-defined ports, and advanced extensibility features. Status: 100% Complete

1. Workflow System > 1.1 Workflow Protocol

Progress: 6/6 (100%)

Completed:

✅ Create src/framework_m/core/interfaces/workflow.py
✅ Define WorkflowProtocol:
✅ async def start_workflow(doctype: str, doc_id: str, workflow_name: str)
✅ async def get_workflow_state(doc_id: str) -> str
✅ async def transition(doc_id: str, action: str, user: UserContext)
✅ async def get_available_actions(doc_id: str, user: UserContext) -> list[str]

1. Workflow System > 1.2 Internal Workflow Adapter

Progress: 28/28 (100%)

Completed:

✅ Create src/framework_m/adapters/workflow/internal_workflow.py
✅ Create WorkflowState DocType:
✅ name: str
✅ workflow: str - Workflow name
✅ doctype: str
✅ document_name: str
✅ current_state: str
✅ updated_at: datetime
✅ Create WorkflowTransition DocType:
✅ name: str
✅ workflow: str
✅ from_state: str
✅ to_state: str
✅ action: str - Action name
✅ allowed_roles: list[str]
✅ condition: str | None - Python expression
✅ Create Workflow DocType:
✅ name: str - Workflow name
✅ doctype: str - Target DocType
✅ initial_state: str
✅ states: list[dict] - State definitions
✅ transitions: list[WorkflowTransition]
✅ Implement InternalWorkflowAdapter:
✅ Load workflow definition
✅ Validate transitions
✅ Check permissions
✅ Update state
✅ Trigger hooks / Emit Event (Side effects via Event Bus only)

1. Workflow System > 1.3 Temporal Workflow Adapter (Optional)

Progress: 6/6 (100%)

Completed:

✅ Create src/framework_m/adapters/workflow/temporal_adapter.py
✅ Implement TemporalWorkflowAdapter:
✅ Connect to Temporal server
✅ Start workflows
✅ Query workflow state
✅ Signal workflows for transitions

2. DocType Overrides > 2.1 Override Registry

Progress: 4/4 (100%)

Completed:

✅ Update MetaRegistry:
✅ Add register_override(base_doctype: str, override_class: Type[BaseDocType])
✅ When loading DocType, check for overrides
✅ Use override class if registered

2. DocType Overrides > 2.2 Schema Extension

Progress: 5/5 (100%)

Completed:

✅ Implement schema merging:
✅ Combine fields from base and override
✅ Override can add new fields
✅ Override can modify field properties
✅ Base fields cannot be removed

2. DocType Overrides > 2.3 Table Alteration

Progress: 5/5 (100%)

Completed:

✅ Update SchemaMapper:
✅ Detect schema changes from overrides
✅ Generate ALTER TABLE migrations
✅ Add new columns
✅ Modify column types (with caution)

3. App-Defined Ports > 3.1 Custom Protocol Registration

Progress: 4/4 (100%)

Completed:

✅ Update Container to support app-defined protocols:
✅ Allow apps to define custom protocols
✅ Register via entrypoints
✅ Other apps can override using container.override()

4. Child Tables (Nested DocTypes) > 4.1 Child Table Support

Progress: 3/3 (100%)

Completed:

✅ Add is_child flag to DocType Config
✅ Child tables don't have independent routes
✅ Stored as JSON or separate table with parent reference

4. Child Tables (Nested DocTypes) > 4.2 Implementation

Progress: 8/8 (100%)

Completed:

✅ Update SchemaMapper:
✅ Create separate table for child
✅ Add parent and parenttype columns
✅ Add idx for ordering
✅ Update GenericRepository:
✅ Save child records when saving parent
✅ Delete old children and insert new ones
✅ Load children when loading parent

5. Virtual Fields > 5.1 Computed Fields

Progress: 4/4 (100%)

Completed:

✅ Add @computed_field decorator:
✅ Virtual fields not stored in database
✅ Computed on load
✅ Included in API responses

6. Link Fields (Foreign Keys) > 6.1 Link Field Type

Progress: 1/1 (100%)

Completed:

✅ Add Link field type using json_schema_extra:

6. Link Fields (Foreign Keys) > 6.2 Implementation

Progress: 8/8 (100%)

Completed:

✅ Update SchemaMapper:
✅ Detect link fields via json_schema_extra
✅ Create foreign key constraint to target table
✅ Reference target table's id column
✅ Add database enforcement:
✅ Foreign key constraints enforce referential integrity
✅ Works with SQLite and PostgreSQL (database-agnostic)
✅ SQLite foreign keys enabled via PRAGMA

6. Link Fields (Foreign Keys) > 6.3 Link Fetching

Progress: 6/6 (100%)

Completed:

✅ Add fetch_from option:
✅ Auto-populate field from linked document
✅ Update GenericRepository to fetch values before save
✅ Fetch values automatically on insert and update
✅ Handle null links gracefully
✅ Support multiple fetch_from fields

7. Naming Series (Human-Readable Names) > 7.1 Auto-Naming Configuration

Progress: 7/7 (100%)

Completed:

✅ Add naming configuration to DocType:
✅ Implement naming patterns:
✅ .YYYY. - Year (2026)
✅ .MM. - Month (01-12)
✅ .DD. - Day (01-31)
✅ .#### - Sequential number with padding (0001, 0002...)
✅ {field} - Field value from entity

7. Naming Series (Human-Readable Names) > 7.2 Implementation (Optimistic Approach)

Progress: 2/2 (100%)

Completed:

✅ Name generation happens AFTER insert (id already assigned):
✅ For high-volume DocTypes, use PostgreSQL sequences:

7. Naming Series (Human-Readable Names) > 7.3 Counter Storage

Progress: 4/4 (100%)

Completed:

✅ Create NamingCounter DocType:
✅ prefix: str - e.g., "INV-2024-"
✅ current: int - Current counter value
✅ Note: No row-level locking needed. Use optimistic update with retry.

8. Validation Rules > 8.1 Field Validators

Progress: 1/1 (100%)

Completed:

✅ Add Pydantic validators:

8. Validation Rules > 8.2 Document Validators

Progress: 1/1 (100%)

Completed:

✅ Use controller validate() hook:

9. Testing > 9.1 Unit Tests

Progress: 5/5 (100%)

Completed:

✅ Test workflow transitions (15 tests in test_workflow.py)
✅ Test DocType overrides (16 tests in test_meta_registry_overrides.py)
✅ Test child table operations (38 tests in test_child_tables.py)
✅ Test link field validation (16 tests in test_link_fields.py)
✅ Test naming series (29 tests in test_naming_series.py + validation tests)

9. Testing > 9.2 Integration Tests

Progress: 4/4 (100%)

Completed:

✅ Test full workflow lifecycle (6 tests in test_workflow_lifecycle.py)
✅ Test override schema migration (10 tests in test_override_migration.py)
✅ Test child table CRUD (12 tests in test_child_table_integration.py)
✅ Test app-defined ports (9 tests in test_app_ports.py)

Validation Checklist

Progress: 6/6 (100%)

Completed:

✅ Workflows can be defined and executed (15 unit + 6 integration tests)
✅ DocTypes can be extended via overrides (16 unit + 10 integration tests)
✅ Child tables work correctly (38 unit + 12 integration tests)
✅ Link fields create proper foreign keys (16 tests in test_link_fields.py)
✅ Naming series generates unique names (29 tests in test_naming_series.py)
✅ App-defined ports can be registered (9 tests in test_app_ports.py)

Phase 09A: Frontend

Phase: 09a Objective: Build a generic admin UI (the "Desk") that renders forms and lists from metadata. Status: 97% Complete

1. Frontend Architecture > 1.2 Project Setup

Progress: 1/6 (17%)

Completed:

✅ Option A: Use Refine CLI (Recommended)

Pending:

⏳ Option B: Manual Setup (Full Control)
⏳ Install form libraries (RJSF):
⏳ Install UI layer (shadcn/ui):
⏳ Install dashboard/charts (Tremor):
⏳ Install Studio-specific tools:

1. Frontend Architecture > 1.3 Refine Data Provider

Progress: 4/4 (100%)

Completed:

✅ Create frameworkMDataProvider.ts:
✅ Implements Refine's DataProvider interface
✅ Maps to Framework M REST API (/api/v1/{resource})
✅ Handles pagination, sorting, filtering

1. Frontend Architecture > 1.4 Base URL Configuration

Progress: 2/2 (100%)

Completed:

✅ Support configurable base URLs for different deployment scenarios:
✅ Inject configuration via window.__FRAMEWORK_CONFIG__:

1. Frontend Architecture > 1.5 App Entry Point

Progress: 1/1 (100%)

Completed:

✅ Create Refine app entry (src/App.tsx):

2. Metadata-Driven UI > 2.1 Metadata Fetcher

Progress: 3/3 (100%)

Completed:

✅ Create useDocTypeMeta hook:
✅ Fetch from GET /api/meta/{doctype}
✅ Cache metadata aggressively using React Query

2. Metadata-Driven UI > 2.2 Auto Form Generator (RJSF Integration)

Progress: 14/14 (100%)

Completed:

✅ Create AutoForm component using RJSF:
✅ Accept JSON Schema from GET /api/meta/{doctype}
✅ Generate form fields dynamically
✅ Map schema types to shadcn/ui components:
✅ string → <Input />
✅ number → <Input type="number" />
✅ boolean → <Checkbox />
✅ enum → <Select />
✅ date → <DatePicker />
✅ Link → <Combobox /> with async search (LinkWidget)
✅ Use RJSF with custom widget registry for shadcn/ui
✅ Architecture: Export AutoForm as standalone component
✅ Allow developers to use it inside custom pages ("Ejection Strategy")
✅ Do not couple strictly to a "Meta-Driven Page" route

2. Metadata-Driven UI > 2.3 Auto Table Generator (TanStack Table)

Progress: 4/4 (100%)

Completed:

✅ Create AutoTable component using @refinedev/react-table:
✅ Columns generated from metadata
✅ Sorting, filtering, pagination built-in
✅ Row selection for bulk actions

2. Metadata-Driven UI > 2.4 Workflow UI (State Buttons)

Progress: 6/6 (100%)

Completed:

✅ State Indicator:
✅ Show current state badge (e.g., "Draft", "Approved") in Form Header
✅ Action Buttons:
✅ Fetch available transitions: GET /api/workflow/{doctype}/{id}/actions
✅ Render buttons: "Submit", "Approve", "Reject"
✅ Handle transition API call on click

2. Metadata-Driven UI > 2.5 Plugin System (The "Shadow Build")

Progress: 7/7 (100%)

Completed:

✅ Goal: "Easy as Frappe" but with Vite/React tooling
✅ Strategy:
✅ m start detects frontend/index.ts in all installed apps
✅ Generates a temporary entry.tsx that imports all app plugins
✅ Runs Vite Dev Server on this generated entry
✅ Result: Hot Module Replacement (HMR) for custom apps inside the main Desk
✅ No Monkey Patching: Plugins register via app.registerPlugin()

2. Metadata-Driven UI > 2.6 Progressive Customization (The "Zero Cliff")

Progress: 11/11 (100%)

Completed:

✅ Level 1: Configuration (Low Code)
✅ Edit Labels/Visibility via Client Script or JSON overrides
✅ No build step required
✅ Level 2: Injection (Mid Code)
✅ Custom Fields: app.registerField('rating', StarRating)
✅ Slot Injection: app.registerSlot('form-header', PaymentButton)
✅ Uses Shadow Build (HMR)
✅ Level 3: Ejection (Pro Code)
✅ Page Override: Replace FormView entirely with custom React component
✅ Whitelabel: Replace Layout / Theme via Plugin
✅ Uses Shadow Build (HMR)

3. Core UI Pages > 3.1 List View

Progress: 7/7 (100%)

Completed:

✅ Create ListView component:
✅ Route: /app/{doctype}/list
✅ Fetch metadata
✅ Render AutoTable
✅ Add "New" button
✅ Add search and filters
✅ Add bulk actions

3. Core UI Pages > 3.2 Form View

Progress: 10/10 (100%)

Completed:

✅ Create FormView component:
✅ Route - Create: /app/{doctype}/new
✅ Route - Edit: /app/{doctype}/detail/{id}
✅ Benefit: No reserved keywords for IDs. An ID can be "new", "create", or "dashboard" without collision.
✅ Fetch metadata
✅ Fetch document (if editing)
✅ Render AutoForm
✅ Render Workflow Actions (if applicable)
✅ Save/Submit/Cancel buttons
✅ Show validation errors

3. Core UI Pages > 3.3 Dashboard

Progress: 5/5 (100%)

Completed:

✅ Create Dashboard component:
✅ Route: /app/dashboard
✅ Show recent documents
✅ Show notifications
✅ Show quick links

Progress: 4/4 (100%)

Completed:

✅ Inject window.__FRAMEWORK_CONFIG__ (or fetch /api/v1/config):
✅ authStrategy: "cookie" (Indie/BFF) or "oidc" (Client-Side)
✅ loginUrl: URL to redirect to if 401 (for Cookie mode)
✅ oidcConfig: { authority, clientId, redirectUri } (for OIDC mode)

Progress: 1/1 (100%)

Completed:

✅ Create authProvider.ts implementing Refine's AuthProvider interface:

Progress: 8/8 (100%)

Completed:

✅ Strategy A: Cookie (Indie / Gateway BFF):
✅ "It Just Works". Relies on browser's HttpOnly cookie
✅ credentials: "include" in all fetch calls
✅ No token management in frontend
✅ Strategy B: OIDC (Client-Side PKCE):
✅ Uses oidc-client-ts library
✅ Store tokens in memory (not localStorage for security)
✅ Inject Authorization: Bearer <token> header via custom fetch

Progress: 3/3 (100%)

Completed:

✅ Create LoginPage component:
✅ Route: /login
✅ Use Refine's useLogin hook:

Progress: 5/5 (100%)

Completed:

✅ Create Sidebar component:
✅ List all DocTypes (grouped by module)
✅ Use Refine's useMenu() hook for resource navigation
✅ Collapsible sections
✅ Search DocTypes

Progress: 6/6 (100%)

Completed:

✅ Create Navbar component:
✅ App logo/name
✅ Global search
✅ Use Refine's useGetIdentity() for user menu
✅ Use Refine's useLogout() for logout button
✅ Notifications icon

5. Advanced UI Features > 5.1 Custom Views

Progress: 5/5 (100%)

Completed:

✅ Support custom view types:
✅ Kanban view (for status-based DocTypes)
✅ Calendar view (for date-based DocTypes)
✅ Gantt view (for project management)
✅ Register custom views in DocType Config:

5. Advanced UI Features > 5.2 Inline Editing

Progress: 4/4 (100%)

Completed:

✅ Add inline edit for list view:
✅ Click cell to edit
✅ Save on blur
✅ Show validation errors

5. Advanced UI Features > 5.3 Bulk Operations

Progress: 3/3 (100%)

Completed:

✅ Add checkbox column to table
✅ Select multiple rows
✅ Bulk actions: Delete, Export, Update field

6. Real-time Updates > 6.1 Refine Live Provider Implementation

Progress: 1/1 (100%)

Completed:

✅ Create liveProvider.ts implementing Refine's LiveProvider interface:

6. Real-time Updates > 6.2 Register Live Provider with Refine

Progress: 1/1 (100%)

Completed:

✅ Add liveProvider to Refine app:

6. Real-time Updates > 6.3 Automatic UI Updates

Progress: 5/5 (100%)

Completed:

✅ With liveMode: "auto", Refine automatically:
✅ Refetches list data on created, updated, deleted events
✅ Shows notification on document changes
✅ Invalidates React Query cache
✅ Manual control with useLiveMode() hook:

6. Real-time Updates > 6.4 Desk Workspace Configuration (Code-First)

Progress: 11/11 (100%)

Completed:

✅ Define Workspace Schema (Pydantic):
✅ name: str — unique identifier
✅ label: str — display name in sidebar
✅ icon: str — sidebar icon
✅ items: list[Shortcut | Link | Chart] — workspace contents
✅ roles: list[str] — who can see this workspace
✅ Implement desktop.py loader:
✅ Scan apps for desktop.py
✅ Load workspace definitions
✅ Merge into unified Desk navigation
✅ Respect role-based visibility

6. Real-time Updates > 6.5 Global Command Palette

Progress: 4/4 (100%)

Completed:

✅ Implement Ctrl+K Global Search:
✅ Search DocTypes (Navigation)
✅ Search Documents (Title)
✅ Run Commands (e.g., "New User")

6. Real-time Updates > 6.6 Public API Specs

Progress: 3/3 (100%)

Completed:

✅ Ensure strict OpenAPI 3.1 generation:
✅ m docs:generate --json — Litestar provides /schema/openapi.json
✅ Verify compatibility with low-code tools (Retool, Refine)

7. Frontend Build & Serving > 7.1 Build Command (`m build`)

Progress: 17/17 (100%)

Completed:

✅ Implement build logic:
✅ Detect Mode from framework_config.toml:
✅ frontend_mode = "indie" (default)
✅ frontend_mode = "plugin" (Shadow Build)
✅ frontend_mode = "eject" (skip, user handles build)
✅ Indie Mode:
✅ No custom frontend code.
✅ Build default Desk: cd framework_m/frontend && pnpm build
✅ Output: framework_m/static/dist/
✅ Plugin Mode (Shadow Build):
✅ Detect frontend/index.ts in all installed apps.
✅ Generate temp entry.tsx importing all app plugins.
✅ Run vite build on generated entry.
✅ Output: dist/ (unified build with all app assets)
✅ Eject Mode:
✅ Skip. User has their own frontend repo and build process.
✅ Framework M just serves API.

7. Frontend Build & Serving > 7.2 Static File Serving (Production)

Progress: 5/5 (100%)

Completed:

✅ Configure Litestar StaticFilesConfig:
✅ Serve from dist/ or static/dist/
✅ Route: /static/* or /assets/*
✅ Option: Configure CDN URL in framework_config.toml:
✅ If CDN configured, inject CDN prefix into HTML asset URLs.

Phase 09B: Documentation

Phase: 09b Objective: Create comprehensive documentation covering fundamentals, migration guides, i18n, multi-tenancy usage, and API references. Status: 100% Complete

1. Documentation (Basic Setup) > 1.1 Basic User Documentation ✅

Progress: 9/9 (100%)

Completed:

✅ Create docs/user/ directory
✅ Write essential guides:
✅ Getting started (docs/user/getting-started.md)
✅ Creating DocTypes (docs/user/creating-doctypes.md)
✅ Using the Desk (docs/user/using-the-desk.md)
✅ Auto-generated docs (scripts/generate_docs.py):
✅ docs/user/_generated/field-types.md
✅ docs/user/_generated/api-endpoints.md
✅ Truthfulness tests (tests/docs/test_user_docs.py - 17 tests passing)

1. Documentation (Basic Setup) > 1.2 Basic Developer Documentation ✅

Progress: 10/10 (100%)

Completed:

✅ Create docs/developer/ directory
✅ Write essential guides:
✅ Architecture overview (docs/developer/architecture.md)
✅ Creating custom apps (docs/developer/creating-apps.md)
✅ Defining DocTypes (docs/developer/defining-doctypes.md)
✅ Auto-generated docs (scripts/generate_docs.py):
✅ docs/developer/_generated/protocols.md (20 protocols)
✅ docs/developer/_generated/hooks.md (10 hooks)
✅ docs/developer/_generated/cli-reference.md
✅ Truthfulness tests (tests/docs/test_developer_docs.py - 16 tests passing)

1. Documentation (Basic Setup) > 1.3 Framework Fundamentals ✅

Progress: 6/6 (100%)

Completed:

✅ Minimal core principle (irreplaceable: async Python, DI, protocols, entrypoints)
✅ DI as universal composer
✅ Bootstrap flow
✅ Layer composition
✅ Adding custom adapters
✅ Testing with DI

1. Documentation (Basic Setup) > 1.4 Migration Guide (Frappe → Framework M) ✅

Progress: 3/3 (100%)

Completed:

✅ Cover backend patterns
✅ Cover frontend patterns
✅ Practical examples

1. Documentation (Basic Setup) > 1.5 API Documentation ✅

Progress: 4/4 (100%)

Completed:

✅ Litestar auto-generates OpenAPI spec (/schema)
✅ Swagger UI (/schema/swagger)
✅ ReDoc (/schema/redoc)
✅ Truthfulness tests (test_developer_docs.py::TestAPIDocumentation - 3 tests)

1. Documentation (Basic Setup) > 1.6 Example Apps ✅

Progress: 4/4 (100%)

Completed:

✅ Create minimal example apps:
✅ Simple TODO app (docs/examples/todo/)
✅ Basic CRM (docs/examples/crm/)
✅ Example index (docs/examples/README.md)

2. Internationalization (i18n) > 2.1 Backend i18n ✅

Progress: 27/27 (100%)

Completed:

✅ Create I18nProtocol interface (core/interfaces/i18n.py):
✅ translate(text, locale, context, default) -> str
✅ get_locale() -> str
✅ set_locale(locale) -> None
✅ get_available_locales() -> list[str]
✅ Create InMemoryI18nAdapter (adapters/i18n/__init__.py):
✅ Dict-based translation storage
✅ Add translations dynamically
✅ String interpolation with context
✅ Create Translation DocType:
✅ source_text: str (original)
✅ translated_text: str
✅ locale: str (e.g. "hi", "ta", "en")
✅ context: str | None (e.g. "button", "label")
✅ Unique constraint: (source_text, locale, context)
✅ Create DefaultI18nAdapter:
✅ Load translations from DB on startup
✅ Cache in memory (Redis optional)
✅ Fallback chain: request locale → default locale → source text
✅ Locale resolution order:
✅ 1. Accept-Language header
✅ 2. User preference (user.locale)
✅ 3. Tenant default (tenant.attributes.default_locale)
✅ 4. System default (settings.DEFAULT_LOCALE)
✅ DocType field labels:
✅ Field.label supports i18n key
✅ Meta API returns translated labels based on request locale

2. Internationalization (i18n) > 2.2 Frontend i18n

Progress: 11/11 (100%)

Completed:

✅ Install react-i18next:
✅ Configure with backend translations endpoint
✅ Lazy load locale bundles
✅ Translation workflow:
✅ Extract strings from components (i18next-parser)
✅ Upload to Translation DocType
✅ Download merged translations as JSON
✅ Components:
✅ useTranslation() hook in all components
✅ <Trans> for interpolated strings
✅ Locale switcher in navbar

2. Internationalization (i18n) > 2.3 Per-Tenant Locales

Progress: 3/3 (100%)

Completed:

✅ Tenant can override default locale
✅ Tenant can provide custom translations (override system)
✅ Tenant locale from TenantContext.default_locale

3. Multi-Tenancy Usage & Documentation > 3.1 Tenancy Modes Documentation ✅

Progress: 12/12 (100%)

Completed:

✅ Document Mode A (Single Tenant / Indie):
✅ ImplicitTenantAdapter usage
✅ Default tenant ID configuration
✅ When to use (standalone deployments)
✅ Document Mode B (Multi-Tenant / Enterprise):
✅ HeaderTenantAdapter usage
✅ Gateway configuration (X-Tenant-ID, X-Tenant-Attributes)
✅ Citadel IAM integration for tenant extraction
✅ Document tenant isolation patterns:
✅ Logical isolation (tenant_id column, RLS)
✅ Database-per-tenant (db_binds per tenant)
✅ Schema-per-tenant (PostgreSQL schemas)

3. Multi-Tenancy Usage & Documentation > 3.2 Using TenantContext in Code ✅

Progress: 8/8 (100%)

Completed:

✅ Document injection pattern:
✅ Dependency injection via request.state.tenant
✅ Feature flags from tenant attributes
✅ Example endpoints using TenantContext
✅ Document tenant-aware queries:
✅ Automatic RLS filtering with GenericRepository
✅ Manual tenant_id filtering with FilterSpec
✅ Cross-tenant queries (admin only)

3. Multi-Tenancy Usage & Documentation > 3.3 Frontend Multi-Tenancy ✅

Progress: 8/8 (100%)

Completed:

✅ Tenant context in frontend:
✅ Extract from JWT or API response
✅ Store in React context (TenantProvider, useTenant hook)
✅ Pass tenant_id in API requests (if not in headers)
✅ Tenant-specific UI:
✅ Tenant branding (logo, colors) - useTenantBranding hook
✅ Feature flags from tenant.attributes - FeatureGate component
✅ Tenant selector (admin UI) - TenantSelector component

3. Multi-Tenancy Usage & Documentation > 3.4 Switching Adapters ✅

Progress: 5/5 (100%)

Completed:

✅ Document adapter selection:
✅ Configuration via framework_config.toml (mode = "single" | "multi")
✅ Factory function create_tenant_adapter_from_headers()
✅ Entrypoint registration for custom adapters
✅ Testing with MockTenantAdapter

4. Theming ✅

Progress: 5/5 (100%)

Completed:

✅ Add theme support:
✅ Light/dark mode with ThemeContext and useTheme hook
✅ Custom color schemes via CSS variables (:root and .dark)
✅ Store user preference in localStorage with system detection fallback
✅ ThemeToggle component integrated in Navbar

5. Testing > 5.1 Frontend Tests ✅

Progress: 20/20 (100%)

Completed:

✅ Unit tests for components:
✅ Test AutoForm rendering (12 tests)
✅ Test AutoTable rendering (11 tests)
✅ Test ThemeToggle functionality (6 tests)
✅ Test form validation
✅ Integration tests:
✅ Test full CRUD flow (3 tests)
✅ Test theme persistence (7 tests)
✅ Test navigation
✅ Test real-time updates
✅ E2E tests with Playwright:
✅ Test user login workflows (7 tests)
✅ Test create record workflows (9 tests)
✅ Test cross-browser compatibility (10 tests)
✅ Test infrastructure:
✅ Vitest setup with happy-dom
✅ Testing Library integration
✅ Playwright configuration
✅ CI/CD test scripts
✅ Coverage reporting

Validation Checklist

Progress: 6/6 (100%)

Completed:

✅ Frontend can render any DocType
✅ Forms are generated from metadata
✅ Tables support sorting and filtering
✅ Real-time updates work
✅ Documentation is comprehensive
✅ Example apps demonstrate features

Phase 10: ERP Enterprise Features

Phase: 10 Objective: Zero-code integration with 3rd party tools (Slack, Zapier). Status: 100% Complete

1. Analytics & Read Models (CQRS) > 1.1 Read Model Protocol

Progress: 5/5 (100%)

Completed:

✅ Create src/framework_m/core/interfaces/read_model.py:
✅ Define ReadModelProtocol:
✅ async def project(event: Event) -> None
✅ async def query(filters: dict) -> list[dict]
✅ async def rebuild() -> None

1. Analytics & Read Models (CQRS) > 1.2 Enterprise Projectors

Progress: 4/4 (100%)

Completed:

✅ Implement Projector (Event Handler):
✅ Subscribe to doc.created, doc.updated
✅ Transform transactional data into "Read Optimized" format
✅ Upsert into Read Store (Postgres Read Replica, ClickHouse, Elastic)

1. Analytics & Read Models (CQRS) > 1.3 Report Engine (Scalable)

Progress: 5/5 (100%)

Completed:

✅ Create Report DocType:
✅ report_type:
✅ Code Report (SQL file in repo or Python Class - Indie Mode)
✅ Analytics Report (Queries Read Model / OLAP - Enterprise Mode)
✅ data_source: SQL / Elastic / ClickHouse adapter

2. Webhook System (Integration) > 2.1 Webhook DocType

Progress: 8/8 (100%)

Completed:

✅ Create src/framework_m/public/webhook.py
✅ Define Webhook DocType:
✅ doctype: str - Trigger DocType
✅ event: str - Event name (after_save, on_submit)
✅ url: str - Target Endpoint
✅ headers: dict - Auth/Custom Headers
✅ condition: str | None - JMESPath or SimpleEval expression (No eval())
✅ payload_transform: str | None - Jinja2 template for JSON payload

2. Webhook System (Integration) > 2.2 Webhook Dispatcher

Progress: 4/4 (100%)

Completed:

✅ Implement WebhookListener:
✅ Subscribe to doc.* events
✅ Queue background job fire_webhook
✅ Resilience: Exponential Backoff (handled by Job Queue)

2. Webhook System (Integration) > 2.3 Webhook Visualization (Native UI)

Progress: 6/6 (100%)

Completed:

✅ Webhook Log DocType (defined in Phase 04):
✅ Stores Request/Response/Error.
✅ Status: Success (Green), Failed (Red).
✅ Desk Integration:
✅ Add "Recent Logs" table to Webhook Form View.
✅ Action: "Retry Now" button (re-queues the job).

3. Data Import / Export (Bulk)

Progress: 1/1 (100%)

Completed:

✅ Defined in Phase 05 CLI (See m import --bypass-controllers)

Phase 10: Production Readiness & Deployment

Phase: 10 Objective: Optimize for production, add monitoring, implement security best practices, and create deployment configurations. Status: 0% Complete

1. Performance Optimization > 1.1 Database Optimization

Progress: 0/12 (0%)

Pending:

⏳ Add database indexes:
⏳ Index on id (primary key, UUID)
⏳ Index on name (unique, human-readable identifier)
⏳ Index on owner (for RLS)
⏳ Index on modified (for sorting)
⏳ Index on foreign keys
⏳ Composite indexes for common queries
⏳ Implement query optimization:
⏳ Use select_in_loading for relationships
⏳ Avoid N+1 queries
⏳ Add query logging in dev mode
⏳ Analyze slow queries

1. Performance Optimization > 1.2 Caching Strategy

Progress: 0/10 (0%)

Pending:

⏳ Implement multi-level caching:
⏳ L1: In-memory cache (per-request)
⏳ L2: Redis cache (shared)
⏳ Cache metadata (DocType schemas)
⏳ Cache permission checks
⏳ Cache frequently accessed documents
⏳ Add cache invalidation:
⏳ Invalidate on document update
⏳ Invalidate on permission change
⏳ Use cache tags for bulk invalidation

1. Performance Optimization > 1.3 Connection Pooling

Progress: 0/7 (0%)

Pending:

⏳ Configure SQLAlchemy pool:
⏳ Pool size: 20
⏳ Max overflow: 10
⏳ Pool timeout: 30s
⏳ Pool recycle: 3600s
⏳ Configure Redis pool:
⏳ Max connections: 50

1. Performance Optimization > 1.4 Async Optimization

Progress: 0/3 (0%)

Pending:

⏳ Use asyncio.gather() for parallel operations
⏳ Avoid blocking calls in async functions
⏳ Use connection pools efficiently

2. Security Hardening > 2.1 Input Validation

Progress: 0/4 (0%)

Pending:

⏳ Validate all inputs with Pydantic
⏳ Sanitize user-provided data
⏳ Prevent SQL injection (use parameterized queries)
⏳ Prevent XSS (API-only, no HTML rendering)

2. Security Hardening > 2.2 Authentication Security

Progress: 0/8 (0%)

Pending:

⏳ Support JWT token validation:
⏳ Verify signature
⏳ Check expiration
⏳ Validate issuer
⏳ Add rate limiting:
⏳ Limit login attempts
⏳ Limit API requests per user
⏳ Use Redis for rate limit storage

2. Security Hardening > 2.3 Authorization Security

Progress: 0/4 (0%)

Pending:

⏳ Enforce permissions on all endpoints
⏳ Apply RLS to all queries
⏳ Validate object-level permissions
⏳ Log permission denials

2. Security Hardening > 2.4 Secrets Management

Progress: 0/6 (0%)

Pending:

⏳ Never commit secrets to Git
⏳ Use environment variables
⏳ Support secret managers:
⏳ AWS Secrets Manager
⏳ Google Secret Manager
⏳ HashiCorp Vault

2. Security Hardening > 2.5 HTTPS & CORS

Progress: 0/4 (0%)

Pending:

⏳ Enforce HTTPS in production
⏳ Configure CORS properly:
⏳ Whitelist allowed origins
⏳ Restrict methods and headers

3. Monitoring & Observability > 3.1 Logging

Progress: 0/10 (0%)

Pending:

⏳ Implement structured logging:
⏳ Use JSON format
⏳ Include request ID
⏳ Include user context
⏳ Log levels: DEBUG, INFO, WARNING, ERROR
⏳ Log important events:
⏳ API requests (with timing)
⏳ Database queries (in dev)
⏳ Permission checks
⏳ Errors and exceptions

3. Monitoring & Observability > 3.2 Metrics

Progress: 0/7 (0%)

Pending:

⏳ Add Prometheus metrics:
⏳ Request count by endpoint
⏳ Request duration histogram
⏳ Database query duration
⏳ Cache hit/miss rate
⏳ Job queue length
⏳ Create /metrics endpoint

3. Monitoring & Observability > 3.3 Tracing

Progress: 0/5 (0%)

Pending:

⏳ Implement OpenTelemetry:
⏳ Trace HTTP requests
⏳ Trace database queries
⏳ Trace background jobs
⏳ Export to Jaeger or Zipkin

3. Monitoring & Observability > 3.4 Health Checks

Progress: 0/7 (0%)

Pending:

⏳ Create /health endpoint:
⏳ Check database connection
⏳ Check Redis connection
⏳ Return 200 if healthy, 503 if not
⏳ Create /ready endpoint:
⏳ Check if app is ready to serve traffic
⏳ Check migrations are up to date

4. Error Handling & Resilience > 4.1 Graceful Degradation

Progress: 0/3 (0%)

Pending:

⏳ Handle database failures gracefully
⏳ Handle Redis failures (fallback to no cache)
⏳ Retry transient errors

4. Error Handling & Resilience > 4.2 Circuit Breaker

Progress: 0/4 (0%)

Pending:

⏳ Implement circuit breaker for external services:
⏳ Webhook calls
⏳ Email sending
⏳ Print service (Gotenberg)

4. Error Handling & Resilience > 4.3 Timeouts

Progress: 0/4 (0%)

Pending:

⏳ Set timeouts for all operations:
⏳ HTTP requests: 30s
⏳ Database queries: 10s
⏳ Background jobs: 300s

5. Deployment Configurations > 5.1 Docker

Progress: 0/4 (0%)

Pending:

⏳ Create production Dockerfile:
⏳ Multi-stage build for smaller images
⏳ Use non-root user
⏳ Add health check

5. Deployment Configurations > 5.2 Docker Compose

Progress: 0/8 (0%)

Pending:

⏳ Create docker-compose.yml:
⏳ API service
⏳ Worker service
⏳ PostgreSQL
⏳ Redis
⏳ Gotenberg (for PDFs)
⏳ Add volumes for persistence
⏳ Add networks for isolation

5. Deployment Configurations > 5.3 Kubernetes

Progress: 0/11 (0%)

Pending:

⏳ Create Kubernetes manifests:
⏳ Deployment for API
⏳ Deployment for Worker
⏳ StatefulSet for PostgreSQL
⏳ StatefulSet for Redis
⏳ Service for API
⏳ Ingress for routing
⏳ Add resource limits:
⏳ CPU: 500m - 2000m
⏳ Memory: 512Mi - 2Gi
⏳ Add liveness and readiness probes

5. Deployment Configurations > 5.4 Cloud Platforms

Progress: 0/7 (0%)

Pending:

⏳ Create deployment guides for:
⏳ Google Cloud Run
⏳ AWS ECS/Fargate
⏳ Azure Container Apps
⏳ Heroku
⏳ Railway
⏳ Render

6. CI/CD Pipeline > 6.1 GitLab CI

Progress: 0/1 (0%)

Pending:

⏳ Configure .gitlab-ci.yml (already exists in repo):

6. CI/CD Pipeline > 6.2 Pre-commit Hooks

Progress: 0/4 (0%)

Pending:

⏳ Setup pre-commit:
⏳ Run ruff formatter
⏳ Run mypy
⏳ Run tests

6. CI/CD Pipeline > 6.3 Release Versioning & Publishing

Progress: 0/22 (0%)

Pending:

⏳ Adopt Semantic Versioning (SemVer):
⏳ MAJOR.MINOR.PATCH format
⏳ Document breaking change policy
⏳ Use 0.x.y for pre-1.0 releases
⏳ CHANGELOG management:
⏳ Create CHANGELOG.md following Keep a Changelog format
⏳ Sections: Added, Changed, Deprecated, Removed, Fixed, Security
⏳ Option: Use git-cliff for automation
⏳ Link CHANGELOG to documentation site
⏳ Conventional Commits (optional but recommended):
⏳ feat: → MINOR bump
⏳ fix: → PATCH bump
⏳ feat!: or BREAKING CHANGE: → MAJOR bump
⏳ Setup commitlint for enforcement
⏳ Release workflow (add to .gitlab-ci.yml):
⏳ Version sources:
⏳ Single source of truth in pyproject.toml
⏳ Use dynamic versioning from git tags OR __version__ in package
⏳ Expose version via m --version CLI
⏳ Pre-release versions:
⏳ Support alpha/beta/rc: 1.0.0-alpha.1, 1.0.0-beta.2, 1.0.0-rc.1
⏳ Publish pre-releases to TestPyPI first

6. CI/CD Pipeline > 6.4 Git Branching Strategy

Progress: 0/22 (0%)

Pending:

⏳ Trunk-Based Development (recommended for M):
⏳ Branch naming conventions:
⏳ main — Always deployable, latest development
⏳ feature/<name> — New features (merge to main via PR)
⏳ fix/<issue-id> — Bug fixes (merge to main via PR)
⏳ release/v<major>.<minor> — Created only when LTS support needed
⏳ Release branches (only for LTS):
⏳ LTS (Long-Term Support) Policy:
⏳ Support last 2 major versions (not 3 like Frappe — M is simpler)
⏳ Security fixes: Backport to all supported versions
⏳ Bug fixes: Backport to latest minor of each supported major
⏳ Features: Only on main, no backports
⏳ Backporting workflow:
⏳ Fix on main first (always forward-fix)
⏳ Cherry-pick to release/vX branch
⏳ Tag patch release: vX.Y.Z
⏳ Tool: Use git cherry-pick -x to track source commit
⏳ Why simpler than Frappe?
⏳ Hexagonal architecture = Adapters can be versioned independently
⏳ No DB schema in core = Migrations are app-specific, not framework-specific
⏳ Protocol stability = Breaking changes are explicit, not hidden
⏳ Apps can pin framework version = No forced upgrade cascades

7. Database Migration Strategy > 7.2 Safe Changes (Auto-Migration in Dev Mode)

Progress: 0/4 (0%)

Pending:

⏳ Add table: New DocType → new table
⏳ Add nullable column: New optional field → ALTER TABLE ADD COLUMN ... NULL
⏳ Add index: Meta.indexes → CREATE INDEX
⏳ Add foreign key: Link field → ALTER TABLE ADD CONSTRAINT

7. Database Migration Strategy > 7.3 Review-Required Changes

Progress: 0/7 (0%)

Pending:

⏳ Add non-nullable column without default:
⏳ Migration will fail if table has existing rows
⏳ Solution: Add with default → backfill → remove default
⏳ Change column type (compatible):
⏳ str → Text (widening): Usually safe
⏳ int → bigint (widening): Usually safe
⏳ Review for edge cases

7. Database Migration Strategy > 7.6 Zero-Downtime Migrations

Progress: 0/4 (0%)

Pending:

⏳ Never drop columns in the same deploy as code changes
⏳ Always add columns as nullable first
⏳ Always backfill data before making non-nullable
⏳ Always deploy code changes before dropping columns

8. Backup & Recovery > 8.1 Database Backups

Progress: 0/4 (0%)

Pending:

⏳ Automated daily backups
⏳ Store backups in S3 or equivalent
⏳ Retention policy (30 days)
⏳ Test restore process

8. Backup & Recovery > 8.2 Disaster Recovery

Progress: 0/4 (0%)

Pending:

⏳ Document recovery procedures
⏳ Test recovery regularly
⏳ RTO (Recovery Time Objective): 4 hours
⏳ RPO (Recovery Point Objective): 24 hours

9. Scaling Strategy > 9.1 Horizontal Scaling

Progress: 0/3 (0%)

Pending:

⏳ API servers: Stateless, scale horizontally
⏳ Workers: Scale based on queue length
⏳ Database: Use read replicas for read-heavy workloads

9. Scaling Strategy > 9.2 Load Balancing

Progress: 0/3 (0%)

Pending:

⏳ Use load balancer (Nginx, ALB)
⏳ Health check endpoints
⏳ Session affinity not required (stateless)

10. Documentation > 10.1 Deployment Guide

Progress: 0/3 (0%)

Pending:

⏳ Write step-by-step deployment guide
⏳ Include environment variables reference
⏳ Include troubleshooting section

10. Documentation > 10.2 Operations Runbook

Progress: 0/5 (0%)

Pending:

⏳ Document common operations:
⏳ Scaling up/down
⏳ Running migrations
⏳ Backup and restore
⏳ Monitoring and alerts

11. Testing > 11.1 Load Testing

Progress: 0/9 (0%)

Pending:

⏳ Use Locust or k6 for load testing
⏳ Test scenarios:
⏳ 1000 concurrent users
⏳ 10,000 requests/minute
⏳ Sustained load for 1 hour
⏳ Measure:
⏳ Response times (p50, p95, p99)
⏳ Error rate
⏳ Resource usage

11. Testing > 11.2 Security Testing

Progress: 0/4 (0%)

Pending:

⏳ Run security scans:
⏳ OWASP ZAP
⏳ Dependency vulnerability scan
⏳ Container image scan

Validation Checklist

Progress: 0/10 (0%)

Pending:

⏳ All tests pass (unit, integration, E2E)
⏳ Load testing shows acceptable performance
⏳ Security scans show no critical issues
⏳ Monitoring and alerts are configured
⏳ Backups are automated and tested
⏳ Documentation is complete
⏳ Deployment pipeline works
⏳ Health checks return 200
⏳ Logs are structured and useful
⏳ Secrets are not in code

Phase 11: Package Split & MX Pattern

Phase: 11 Objective: Enable enterprises to build custom "MX" variants (e.g., MongoDB-primary, Cassandra-primary) without forking, via pure package composition. Status: 100% Complete

1. Package Split Strategy > 1.1 Split into Core + Standard

Progress: 17/17 (100%)

Completed:

✅ Create framework-m-core package:
✅ All Protocols (Repository, Permission, EventBus, Cache, etc.)
✅ DI Container setup (dependency-injector)
✅ BaseDocTypeProtocol (not concrete class)
✅ Controller/Hook system
✅ CLI framework (cyclopts base)
✅ Zero adapters, zero concrete implementations
✅ Create framework-m-standard package:
✅ Depends on framework-m-core
✅ SQLAlchemy adapters (SchemaMapper, GenericRepository, UnitOfWork)
✅ RBAC PermissionAdapter
✅ Redis EventBusAdapter
✅ Arq JobQueueAdapter
✅ Default Bootstrap sequence
✅ Create framework-m meta-package:
✅ Depends on both framework-m-core + framework-m-standard
✅ "Batteries included" install for Indie devs

1. Package Split Strategy > 1.2 Decouple Development Tooling

Progress: 7/7 (100%)

Completed:

✅ Move scaffolding and codegen from framework-m to framework-m-studio:
✅ Move new:app command and templates
✅ Move new:doctype command and templates
✅ Move new:site (placeholder) to Studio
✅ Move all Jinja2-based scaffolding code
✅ Goal: framework-m (Core/Standard) should have zero dependencies on Jinja2 or LibCST.
✅ Result: framework-m is optimized for production containers, while framework-m-studio remains the developer's workbench.

2. Protocols to Extract to Core > 2.1 Ensure All Protocols are Pure

Progress: 5/5 (100%)

Completed:

✅ RepositoryProtocol — no SQLAlchemy types
✅ UnitOfWorkProtocol — generic session interface
✅ SchemaMapperProtocol — Pydantic → Storage mapping
✅ BootstrapProtocol — Startup step interface
✅ Verify: No imports from sqlalchemy in framework-m-core

2. Protocols to Extract to Core > 2.2 BaseDocType as Protocol

Progress: 3/3 (100%)

Completed:

✅ Create BaseDocTypeProtocol:
✅ framework-m-standard provides BaseDocType(BaseDocTypeProtocol) with owner, _version, etc.
✅ MX packages can provide their own BaseDocType

3. Bootstrap via Entry Points > 3.1 Define Bootstrap Plugin System

Progress: 4/4 (100%)

Completed:

✅ Create entry point group: framework_m.bootstrap
✅ Define BootstrapStep protocol:
✅ Discover and sort steps from entry points
✅ Allow MX packages to override steps by name

3. Bootstrap via Entry Points > 3.2 Default Bootstrap Steps (framework-m-standard)

Progress: 4/4 (100%)

Completed:

✅ init_engine (order=10) — Create SQLAlchemy engine
✅ init_registries (order=20) — MetaRegistry, FieldRegistry
✅ sync_schema (order=30) — Run SchemaMapper on all DocTypes
✅ init_adapters (order=40) — Bind adapters to DI container

4. Adapter Auto-Discovery > 4.1 Entry Point Groups for Adapters

Progress: 5/5 (100%)

Completed:

✅ framework_m.adapters.repository — GenericRepository implementation
✅ framework_m.adapters.schema_mapper — SchemaMapper implementation
✅ framework_m.adapters.unit_of_work — UnitOfWork implementation
✅ framework_m.adapters.permission — PermissionAdapter
✅ framework_m.adapters.event_bus — EventBusAdapter

4. Adapter Auto-Discovery > 4.2 DI Auto-Binding

Progress: 3/3 (100%)

Completed:

✅ On startup, scan entry points
✅ Bind discovered adapters to Protocols in DI container
✅ Priority: Explicit config > Entry point > Default

5. Example: framework-mx-mongo > 5.1 Package Structure

Progress: 5/5 (100%)

Completed:

✅ Create framework-mx-mongo package with proper structure
✅ Implement MongoSchemaMapper (Pydantic → MongoDB collections)
✅ Implement MongoGenericRepository (CRUD via Motor)
✅ Implement MongoSessionFactory (MongoDB sessions & transactions)
✅ Implement MongoInit bootstrap step

5. Example: framework-mx-mongo > 5.2 Entry Point Registration

Progress: 4/4 (100%)

Completed:

✅ Register repository adapter entry point
✅ Register schema_mapper adapter entry point
✅ Register unit_of_work adapter entry point
✅ Register bootstrap entry point

5. Example: framework-mx-mongo > 5.3 Testing

Progress: 4/4 (100%)

Completed:

✅ Test protocol compliance for all adapters
✅ Test entry point registration and discovery
✅ Test MX pattern override scenarios
✅ Verify 20/20 tests passing

6. Documentation

Progress: 3/3 (100%)

Completed:

✅ Architecture guide: "MX Pattern ADR" (docs/adr/0006-mx-pattern.md)
✅ Tutorial: "MongoDB-Primary Framework MX" (docs/tutorials/mongodb-mx-tutorial.md)
✅ Reference: All protocol definitions (docs/reference/protocol-reference.md)

7. Testing

Progress: 5/5 (100%)

Completed:

✅ Test: framework-m-core installs without SQLAlchemy
✅ Test: framework-m-standard adds SQLAlchemy adapters
✅ Test: MX package can override adapters
✅ Test: Coverage configuration includes all workspace packages
✅ Test: Bootstrap order respects priority

Phase 12: LLM-Ready Documentation & Automation

Phase: 12 Objective: Create comprehensive, machine-readable documentation with automated screenshot generation and LLM-assisted content workflows. Status: 39% Complete

1. Documentation Architecture > 1.1 Source-of-Truth Hierarchy

Progress: 9/9 (100%)

Completed:

✅ Define documentation sources:
✅ ADRs (docs/adr/) → Design decisions, rationale
✅ Checklists (checklists/) → Feature completion status
✅ DocType Source (*.py) → API reference (auto-generated)
✅ User Docs (docs/user/) → How-to guides
✅ Developer Docs (docs/developer/) → Architecture, patterns
✅ Create documentation discovery logic (Dynamic manifest):
✅ Implemented in m docs:generate and m docs:export
✅ Auto-scans docs/, checklists/, and src/doctypes/

1. Documentation Architecture > 2.2 Website Setup (Docusaurus)

Progress: 10/10 (100%)

Completed:

✅ Initialize Docusaurus in website/ directory:
✅ Configure website/docusaurus.config.ts:
✅ Set docs.path to ../docs (reuse existing markdown)
✅ Set docs.routeBasePath to /docs
✅ Site metadata (title, tagline, URL)
✅ Setup website/ folder structure:
✅ Reuse existing docs/ for content:
✅ docs/user/ -> Documentation content
✅ docs/developer/ -> Documentation content
✅ docs/adr/ -> Documentation content

2. Auto-Generated Reference Documentation > 2.1 DocType API Reference

Progress: 10/10 (100%)

Completed:

✅ Implement m docs:generate CLI command:
✅ Parse all DocTypes from MetaRegistry
✅ Extract JSON Schema from Pydantic models
✅ Scope: Includes Core, Standard, and any installed 3rd-party Apps.
✅ Generate markdown for each DocType:
✅ Field table (name, type, required, default, description)
✅ Permission matrix (roles × actions)
✅ Controller hooks (before_create, validate, etc.)
✅ Link to source file
✅ Output format matching standard template:

2. Auto-Generated Reference Documentation > 2.2 Protocol Reference

Progress: 5/5 (100%)

Completed:

✅ Auto-generate Protocol documentation:
✅ List all Protocols in framework_m_core
✅ Extract method signatures and docstrings
✅ List available adapters for each Protocol
✅ Generate comparison table (Indie vs Enterprise adapters)

2. Auto-Generated Reference Documentation > 2.3 OpenAPI Documentation

Progress: 4/4 (100%)

Completed:

✅ Export OpenAPI spec: m docs:openapi --output docs/api/openapi.json
✅ Generate human-readable API docs from OpenAPI
✅ Include request/response examples
✅ Group by DocType and module

2. Auto-Generated Reference Documentation > 3.1 ADR/RFC Structure Standardization

Progress: 7/7 (100%)

Completed:

✅ Create ADR/RFC template (docs/adr/template.md):
✅ Implement incremental numbering logic:
✅ Start with 4 digits (0001-0999).
✅ Automatically expand to more digits if needed (10000+).
✅ CI check to ensure no duplicate or skipped numbers in PRs.
✅ Add CLI: m docs:adr create "Title" → Creates numbered ADR from template.
✅ Add CLI: m docs:rfc create "Title" → Creates numbered RFC from template.

2. Auto-Generated Reference Documentation > 3.2 Automated Chronological Indexing

Progress: 5/5 (100%)

Completed:

✅ Implement m docs:adr index:
✅ Scans docs/adr/ and docs/rfcs/.
✅ Generates index.md for both with a chronological list.
✅ Anchored "At-a-glance" view for quick reference.
✅ Support lazy-loading behavior in Docusaurus (similar to blog).

4. Checklist-to-Feature Documentation > 4.1 Feature Extraction

Progress: 0/4 (0%)

Pending:

⏳ Parse checklists for completed items ([x])
⏳ Group by phase and category
⏳ Generate "Features" page with completion status
⏳ Link features to relevant documentation sections

4. Checklist-to-Feature Documentation > 4.2 Release Notes Generation

Progress: 0/3 (0%)

Pending:

⏳ Track checklist changes between versions (git diff)
⏳ Auto-generate release notes from newly completed items
⏳ Format: CHANGELOG.md or GitHub Release notes

5. Automated Screenshot Pipeline > 5.1 Screenshot Manifest

Progress: 0/1 (0%)

Pending:

⏳ Create docs/screenshots.yaml:

5. Automated Screenshot Pipeline > 5.2 Playwright Screenshot Automation

Progress: 0/10 (0%)

Pending:

⏳ Create scripts/generate-screenshots.ts:
⏳ Read screenshots.yaml
⏳ Start dev server (m start)
⏳ For each screenshot:
⏳ Navigate to URL
⏳ Wait for selector
⏳ Capture screenshot
⏳ Save to output path
⏳ Generate diff report (new vs old screenshots)
⏳ Install dependencies:

5. Automated Screenshot Pipeline > 5.3 Screenshot CI Workflow

Progress: 0/4 (0%)

Pending:

⏳ Add to .gitlab-ci.yml:
⏳ Auto-commit screenshots (optional):
⏳ Use git push in CI with deploy token
⏳ Or: Create MR with updated screenshots for review

5. Automated Screenshot Pipeline > 5.4 Screenshot Staleness Detection

Progress: 0/4 (0%)

Pending:

⏳ Add timestamp to screenshot manifest
⏳ Compare file modification time vs manifest timestamp
⏳ Warn in CI if screenshots are stale (>30 days)
⏳ Option to regenerate on schedule (weekly)

6. Machine-Readable Documentation (RAG/LLM) > 6.1 Structured DocType Export

Progress: 0/1 (0%)

Pending:

⏳ Implement m docs:export --format yaml:

6. Machine-Readable Documentation (RAG/LLM) > 6.2 RAG Corpus Generation

Progress: 3/3 (100%)

Completed:

✅ Implement m docs:export --format jsonl --output docs/machine/corpus.jsonl:
✅ Chunk content appropriately for embeddings (500-1000 tokens)
✅ Include metadata for filtering (type, module, version)

6. Machine-Readable Documentation (RAG/LLM) > 6.3 Agent Knowledge Base

Progress: 0/2 (0%)

Pending:

⏳ Create .agent/knowledge/ directory structure:
⏳ Auto-update on each build/deploy

6. Machine-Readable Documentation (RAG/LLM) > 6.4 Inline Code Documentation

Progress: 0/3 (0%)

Pending:

⏳ Ensure all Protocols have comprehensive docstrings:
⏳ Add Examples sections to all public APIs
⏳ Use Google-style docstrings for consistency

6. Machine-Readable Documentation (RAG/LLM) > 6.5 LLM Docs Distribution & Sync (Federated Knowledge)

Progress: 0/9 (0%)

Pending:

⏳ Publication Strategy:
⏳ Include machine-readable assets in documentation build artifacts.
⏳ Path: docs/static/llm/v{version}/corpus.jsonl.
⏳ Configuration (framework_config.toml):
⏳ Sync Command (m docs:sync):
⏳ Fetches upstream corpus from upstream_url.
⏳ Generates local project corpus via m docs:export.
⏳ Merges upstream + local into .agent/knowledge/combined.jsonl.
⏳ Updates .agent/context/project.md with summary of available knowledge.

7. LLM-Assisted Documentation Workflow > 7.1 Documentation Gap Detection

Progress: 0/6 (0%)

Pending:

⏳ Create m docs:audit command:
⏳ Scan all DocTypes for missing descriptions
⏳ Check all Protocols for missing docstrings
⏳ Identify ADRs without implementation links
⏳ Report undocumented controller hooks
⏳ Output: docs/audit-report.md

7. LLM-Assisted Documentation Workflow > 7.2 LLM Documentation Generator

Progress: 0/6 (0%)

Pending:

⏳ Create m docs:draft command:
⏳ Input: Source file or DocType name
⏳ Process: Send code + context to LLM
⏳ Output: Draft documentation in markdown
⏳ Human reviews and edits before commit
⏳ Prompt template:

7. LLM-Assisted Documentation Workflow > 7.3 Diff-Based Documentation Updates

Progress: 0/5 (0%)

Pending:

⏳ Create m docs:update command:
⏳ Input: Git diff of code changes
⏳ Process: Identify affected documentation sections
⏳ Output: Suggested documentation updates
⏳ Human reviews and applies changes

8. Documentation Deployment > 8.1 Build Pipeline

Progress: 0/4 (0%)

Pending:

⏳ Add docs build to .gitlab-ci.yml:
⏳ Deploy to GitLab Pages:
⏳ Docs available at https://\<namespace\>.gitlab.io/\<project\>/
⏳ Custom domain: Configure in Settings → Pages

8. Documentation Deployment > 8.2 Versioned Documentation

Progress: 0/21 (0%)

Pending:

⏳ Use Docusaurus for documentation (update Section 1.2):
⏳ Enable Docusaurus versioning:
⏳ Version strategy (aligned with git branching):
⏳ Next → Current main branch (unstable, latest features)
⏳ 1.0.x, 2.0.x → Tagged releases (stable)
⏳ Archive versions when LTS support ends
⏳ Configure docusaurus.config.js:
⏳ Sync with release workflow:
⏳ On git tag vX.Y.0 (minor/major): Create new docs version
⏳ On git tag vX.Y.Z (patch): Update existing version docs
⏳ Archive docs when release/vX branch is EOL
⏳ Version banner for old docs:
⏳ Docusaurus built-in: "This is documentation for version X.0."
⏳ Per-version API reference:
⏳ Generate OpenAPI spec per version
⏳ Store in docs/static/api/v{version}/openapi.json
⏳ Embed Swagger UI or Redoc in versioned docs
⏳ Per-version LLM Corpus:
⏳ Generate corpus.jsonl and doctypes.yaml for each version.
⏳ Store in docs/static/llm/v{version}/.
⏳ Allows agents to sync docs matching their installed framework version.

8. Documentation Deployment > 8.3 Search Integration

Progress: 4/4 (100%)

Completed:

✅ Add search integration (Searchable ADRs, RFCs, and Docs):
✅ Use docusaurus-plugin-search-local or similar
✅ Ensure docs/adr/ and docs/rfcs/ are indexed
✅ Include code examples in search

9. Testing Documentation > 9.1 Documentation Tests

Progress: 0/7 (0%)

Pending:

⏳ Test code examples compile/run:
⏳ Extract code blocks from markdown
⏳ Run against test environment
⏳ Fail CI if examples are broken
⏳ Test internal links:
⏳ Check all markdown links resolve
⏳ Check all file references exist

9. Testing Documentation > 9.2 Screenshot Regression Tests

Progress: 0/3 (0%)

Pending:

⏳ Compare new screenshots to baseline
⏳ Alert on unexpected visual changes
⏳ Require manual approval for UI changes

10. Documentation Contribution Guide > 10.1 For Contributors

Progress: 0/5 (0%)

Pending:

⏳ Create docs/CONTRIBUTING.md:
⏳ How to write documentation
⏳ Style guide (tone, formatting)
⏳ How to add new screenshots
⏳ How to update ADRs

10. Documentation Contribution Guide > 10.2 For AI Agents

Progress: 0/4 (0%)

Pending:

⏳ Create .agent/knowledge/documentation-patterns.md:
⏳ How Framework M documentation is structured
⏳ Where to find different types of information
⏳ How to suggest documentation improvements

11. LLM-Assisted Development Support > 11.1 Why TDD Enables LLM Coding

Progress: 5/5 (100%)

Completed:

✅ Document the TDD-LLM workflow:
✅ Tests are executable specifications — LLM knows what "correct" means
✅ Test failures are feedback — LLM can self-correct
✅ Type hints + mypy are compile-time checks — catch errors before runtime
✅ Protocol interfaces are contracts (Documented in developer guides)

11. LLM-Assisted Development Support > 11.2 Agent-Ready Repository Structure

Progress: 2/2 (100%)

Completed:

✅ Create .agent/ directory for AI coding assistants:
✅ .agent/workflows/ populated with key tasks

11. LLM-Assisted Development Support > 11.3 Test-First Development for LLM

Progress: 0/2 (0%)

Pending:

⏳ Provide test templates for common patterns:
⏳ Document test commands in .agent/workflows/run-tests.md:

11. LLM-Assisted Development Support > 11.4 LLM-Friendly Error Messages

Progress: 0/5 (0%)

Pending:

⏳ Ensure error messages are actionable:
⏳ Include expected vs actual values
⏳ Reference relevant documentation
⏳ Suggest fixes when possible
⏳ Custom exceptions with context:

11. LLM-Assisted Development Support > 11.5 Agentic Coding Workflow

Progress: 2/2 (100%)

Completed:

✅ Document the agentic workflow for Framework M (See Architecture and Quality guides)
✅ Create .agent/workflows/ with development best practices

11. LLM-Assisted Development Support > 11.7 RAG Corpus for Code Completion

Progress: 0/7 (0%)

Pending:

⏳ Include code examples in RAG corpus:
⏳ Annotated examples of each pattern
⏳ Common mistakes and their fixes
⏳ Before/after refactoring examples
⏳ Export tests as documentation:
⏳ Tests show "how to use" each component
⏳ m docs:export --include-tests adds test patterns to corpus

Validation Checklist

Progress: 9/12 (75%)

Completed:

✅ m docs:generate produces complete API reference
✅ m docs:export --format jsonl exports RAG corpus
✅ All DocTypes have descriptions
✅ All Protocols have docstrings (in code)
✅ Documentation site builds without errors
✅ Search works across all documentation
✅ .agent/ directory structure in place
✅ Agentic workflow documented (in intro/developer docs)
✅ LLM can run tests and self-correct (verified manually)

Pending:

⏳ Screenshot pipeline runs in CI
⏳ Code examples in docs are tested
⏳ Test templates provided for LLM coding

Phase 13: UI/UX Polish & Design System Enhancement

Phase: 13 Objective: Polish the existing custom design system to create a modern, accessible UI that attracts both users and developers. Status: 0% Complete

1. Audit Current Design System > 1.1 Review Existing Implementation

Progress: 0/9 (0%)

Pending:

⏳ Audit frontend/src/App.css design tokens:
⏳ Color palette (currently sky blue #0ea5e9)
⏳ Typography (Inter font family)
⏳ Spacing scale (xs to xl)
⏳ Border radius (sm, md, lg)
⏳ Shadows
⏳ Review dark mode implementation in ThemeContext
⏳ Test current mobile responsiveness
⏳ Run accessibility audit (Lighthouse, axe-core)

1. Audit Current Design System > 1.2 Identify Gaps

Progress: 0/4 (0%)

Pending:

⏳ Document missing components (tooltips, dropdowns, etc.)
⏳ Identify inconsistent spacing/colors
⏳ List accessibility issues (contrast, keyboard nav, ARIA)
⏳ Note missing animations/transitions

2. Refine Color Palette > 2.1 Update to Deep Blue (Trust Theme)

Progress: 0/2 (0%)

Pending:

⏳ Replace current sky blue (#0ea5e9) with deep blue palette:
⏳ Update semantic colors:

2. Refine Color Palette > 2.2 Verify WCAG AA Contrast

Progress: 0/5 (0%)

Pending:

⏳ Test all text/background combinations
⏳ Ensure minimum 4.5:1 contrast for normal text
⏳ Ensure minimum 3:1 contrast for UI components
⏳ Test in both light and dark modes
⏳ Use tools: WebAIM Contrast Checker, Lighthouse

2. Refine Color Palette > 2.3 Update Dark Mode Colors

Progress: 0/3 (0%)

Pending:

⏳ Refine dark mode palette for better contrast
⏳ Test readability in low-light environments
⏳ Ensure primary blue works in dark mode (#38bdf8)

Progress: 0/4 (0%)

Pending:

⏳ Review current font sizes (14px base)
⏳ Add missing heading sizes if needed
⏳ Ensure consistent line-heights (1.3 for headings, 1.5 for body)
⏳ Test readability at different viewport sizes

Progress: 0/4 (0%)

Pending:

⏳ Audit all components for consistent spacing
⏳ Use spacing scale consistently (xs, sm, md, lg, xl)
⏳ Fix any hard-coded pixel values
⏳ Document spacing patterns (e.g., card padding, button spacing)

4. Component Polish > 4.1 Buttons

Progress: 0/5 (0%)

Pending:

⏳ Add hover/active/focus states
⏳ Add loading state (spinner)
⏳ Add disabled state styling
⏳ Ensure consistent sizing (height, padding)
⏳ Add keyboard focus indicators (visible outline)

4. Component Polish > 4.2 Form Inputs

Progress: 0/5 (0%)

Pending:

⏳ Polish input focus states (already has blue ring)
⏳ Add error state styling (red border + message)
⏳ Add success state styling (green border)
⏳ Improve placeholder text contrast
⏳ Add floating labels (optional enhancement)

4. Component Polish > 4.3 Tables

Progress: 0/5 (0%)

Pending:

⏳ Add row hover effects (already exists)
⏳ Add sortable column indicators
⏳ Add loading skeleton for data tables
⏳ Improve mobile table responsiveness (card view)
⏳ Add empty state illustration

4. Component Polish > 4.4 Cards

Progress: 0/4 (0%)

Pending:

⏳ Add subtle shadow on hover
⏳ Ensure consistent border radius
⏳ Add loading skeleton variant
⏳ Test card layouts on mobile

Progress: 0/5 (0%)

Pending:

⏳ Polish active state (already uses primary color)
⏳ Add smooth collapse animation
⏳ Improve search input styling
⏳ Add keyboard navigation support
⏳ Test mobile drawer behavior

Progress: 0/4 (0%)

Pending:

⏳ Polish search bar styling
⏳ Add user menu dropdown
⏳ Add notification badge
⏳ Ensure consistent height (56px)

5. Animations & Micro-interactions > 5.1 Add CSS Transitions

Progress: 0/5 (0%)

Pending:

⏳ Button hover effects (0.15s ease)
⏳ Input focus transitions (0.15s ease)
⏳ Sidebar item hover (0.15s ease)
⏳ Card hover effects (subtle lift)
⏳ Modal/drawer slide-in animations

5. Animations & Micro-interactions > 5.2 Loading States

Progress: 0/6 (0%)

Pending:

⏳ Add skeleton loaders for:
⏳ List view (table rows)
⏳ Form view (form fields)
⏳ Dashboard (cards)
⏳ Add spinner component for buttons
⏳ Add page transition fade

5. Animations & Micro-interactions > 5.3 Optimistic UI Updates

Progress: 0/4 (0%)

Pending:

⏳ Instant feedback on create (fade in new row)
⏳ Instant feedback on update (highlight changed row)
⏳ Instant feedback on delete (fade out row)
⏳ Rollback on error with toast notification

6. Mobile Optimization > 6.1 Responsive Layouts

Progress: 0/8 (0%)

Pending:

⏳ Test all screens on mobile viewports:
⏳ 320px (iPhone SE)
⏳ 375px (iPhone 12/13)
⏳ 414px (iPhone 12 Pro Max)
⏳ 768px (iPad)
⏳ Ensure sidebar collapses to drawer on mobile
⏳ Test table responsiveness (switch to card view)
⏳ Ensure touch targets are min 44px

6. Mobile Optimization > 6.2 Mobile-Specific Enhancements

Progress: 0/4 (0%)

Pending:

⏳ Add swipe gestures for sidebar drawer
⏳ Optimize form layouts for mobile (single column)
⏳ Add bottom navigation (optional)
⏳ Test keyboard behavior on mobile browsers

7. Accessibility (WCAG 2.1 AA) > 7.1 Color Contrast

Progress: 0/4 (0%)

Pending:

⏳ Run automated contrast checks (Lighthouse)
⏳ Fix any failing combinations
⏳ Ensure all interactive elements meet 3:1 minimum
⏳ Test with color blindness simulators

Progress: 0/8 (0%)

Pending:

⏳ Test tab order for all pages
⏳ Add visible focus indicators (already has some)
⏳ Implement keyboard shortcuts:
⏳ Ctrl+K for command palette (already exists)
⏳ Esc to close modals/drawers
⏳ Arrow keys for table navigation
⏳ Enter to activate buttons
⏳ Add skip navigation links

Progress: 0/7 (0%)

Pending:

⏳ Add ARIA labels to all interactive elements
⏳ Add ARIA live regions for dynamic content (toasts, notifications)
⏳ Test with screen readers:
⏳ NVDA (Windows)
⏳ JAWS (Windows)
⏳ VoiceOver (macOS/iOS)
⏳ Add meaningful alt text for images/icons

7. Accessibility (WCAG 2.1 AA) > 7.4 Form Accessibility

Progress: 0/4 (0%)

Pending:

⏳ Ensure all inputs have associated labels
⏳ Add ARIA descriptions for error messages
⏳ Add required field indicators (* or "required" text)
⏳ Add field help text with aria-describedby

8. Missing Components > 8.1 Add Essential Components

Progress: 0/8 (0%)

Pending:

⏳ Tooltip - For icon buttons and help text
⏳ Dropdown Menu - For user menu, actions
⏳ Toast Notifications - For success/error messages
⏳ Modal Dialog - For confirmations, forms
⏳ Badge - For counts, status indicators
⏳ Alert - For warnings, errors, info messages
⏳ Tabs - For multi-section forms/views
⏳ Accordion - For collapsible sections

8. Missing Components > 8.2 Implementation Approach

Progress: 0/4 (0%)

Pending:

⏳ Use vanilla CSS + React state (no library)
⏳ Follow existing design token patterns
⏳ Ensure dark mode support
⏳ Add to component library documentation

9. Documentation > 9.1 Design System Documentation

Progress: 0/7 (0%)

Pending:

⏳ Create docs/design-system.md:
⏳ Color palette with hex codes
⏳ Typography scale
⏳ Spacing system
⏳ Component patterns
⏳ Animation principles
⏳ Accessibility guidelines

9. Documentation > 9.2 Component Documentation

Progress: 0/5 (0%)

Pending:

⏳ Document each component with:
⏳ Usage examples
⏳ Props/variants
⏳ Accessibility notes
⏳ Code snippets

9. Documentation > 9.3 Storybook (Optional)

Progress: 0/4 (0%)

Pending:

⏳ Set up Storybook for component showcase
⏳ Create stories for all components
⏳ Add dark mode toggle in Storybook
⏳ Deploy Storybook for team reference

10. Performance Optimization > 10.1 CSS Optimization

Progress: 0/4 (0%)

Pending:

⏳ Remove unused CSS rules
⏳ Minimize CSS file size
⏳ Use CSS containment where applicable
⏳ Optimize font loading (already using Google Fonts)

10. Performance Optimization > 10.2 Bundle Size

Progress: 0/4 (0%)

Pending:

⏳ Analyze bundle size with Vite build
⏳ Code-split large components
⏳ Lazy load routes
⏳ Optimize images (use WebP, lazy loading)

10. Performance Optimization > 10.3 Runtime Performance

Progress: 0/4 (0%)

Pending:

⏳ Test First Contentful Paint (FCP) - target < 1.5s
⏳ Test Time to Interactive (TTI) - target < 3s
⏳ Test Largest Contentful Paint (LCP) - target < 2.5s
⏳ Optimize re-renders (React.memo, useMemo)

11. Testing > 11.1 Visual Regression Testing

Progress: 0/7 (0%)

Pending:

⏳ Set up Chromatic or Percy (optional)
⏳ Create visual snapshots for key pages
⏳ Test across browsers:
⏳ Chrome
⏳ Firefox
⏳ Safari
⏳ Edge

11. Testing > 11.2 Accessibility Testing

Progress: 0/4 (0%)

Pending:

⏳ Run Lighthouse accessibility audit (target 100)
⏳ Run axe-core automated tests
⏳ Manual keyboard-only testing
⏳ Manual screen reader testing

11. Testing > 11.3 Cross-Browser Testing

Progress: 0/4 (0%)

Pending:

⏳ Test on Windows (Chrome, Firefox, Edge)
⏳ Test on macOS (Chrome, Firefox, Safari)
⏳ Test on iOS (Safari, Chrome)
⏳ Test on Android (Chrome)

12. Example Screens Enhancement > 12.1 Polish Reference Screens

Progress: 0/5 (0%)

Pending:

⏳ Dashboard - Add chart widgets, metric cards
⏳ List View - Polish filters, search, pagination
⏳ Form View - Polish validation, error states
⏳ Detail View - Add tabs, related lists
⏳ Settings Page - Add sections, save states

12. Example Screens Enhancement > 12.2 Create Demo Content

Progress: 0/6 (0%)

Pending:

⏳ Add sample data for screenshots
⏳ Create video walkthrough showing:
⏳ Dark mode toggle
⏳ Mobile responsiveness
⏳ Smooth animations
⏳ Accessibility features

Validation Checklist

Progress: 0/8 (0%)

Pending:

⏳ Deep blue color palette is consistently applied
⏳ Dark mode works perfectly across all screens
⏳ Mobile responsiveness is 100% (tested on real devices)
⏳ WCAG 2.1 AA compliance verified (Lighthouse score 100)
⏳ All animations are smooth (60fps)
⏳ No UI library dependencies (pure CSS + React)
⏳ Design system is documented
⏳ Developers say: "Wow, this looks professional!"

Phase 14: Studio Enhancements

Phase: 14 Objective: Add advanced features to Studio for improved developer experience, including live preview, sandbox mode, and hot reload. Status: 0% Complete

1. Live Preview & Sandbox Mode > 1.2 Branch Selector

Progress: 0/4 (0%)

Pending:

⏳ Add branch selector to Studio UI
⏳ Allow switching between branches
⏳ Show current branch in header
⏳ Warn before switching with unsaved changes

1. Live Preview & Sandbox Mode > 1.3 Preview Tab Implementation

Progress: 0/3 (0%)

Pending:

⏳ Add "Preview" tab to DocType editor
⏳ Toggle between "Schema Editor" and "Sandbox"
⏳ Preserve editor state when switching tabs

1. Live Preview & Sandbox Mode > 1.4 Mock Data Generation

Progress: 0/10 (0%)

Pending:

⏳ Install @faker-js/faker for mock data
⏳ Create mock data generator service:
⏳ Generate data based on field types:
⏳ String → faker.lorem.words()
⏳ Number → faker.number.int()
⏳ Date → faker.date.recent()
⏳ Email → faker.internet.email()
⏳ Link → Random ID from related DocType
⏳ Respect field constraints (min, max, required)
⏳ Generate 5-100 mock records

1. Live Preview & Sandbox Mode > 1.5 Form Preview

Progress: 0/4 (0%)

Pending:

⏳ Render AutoForm component with mock document
⏳ Show validation errors for required fields
⏳ Allow editing mock data
⏳ Show save/cancel buttons (non-functional)

1. Live Preview & Sandbox Mode > 1.6 List Preview

Progress: 0/4 (0%)

Pending:

⏳ Render AutoTable component with mock rows
⏳ Show 5-10 rows by default
⏳ Add pagination controls
⏳ Show filters and search (non-functional)

1. Live Preview & Sandbox Mode > 1.7 In-Memory Schema Source

Progress: 0/4 (0%)

Pending:

⏳ Use in-memory schema (not saved file)
⏳ Allow instant preview without saving
⏳ Update preview on field changes
⏳ Debounce preview updates (500ms)

2. Interactive Sandbox (CRUD Simulation) > 2.1 In-Memory Data Store

Progress: 0/4 (0%)

Pending:

⏳ Create in-memory store for mock documents
⏳ Store per DocType (isolated)
⏳ Reset on page reload
⏳ Support up to 1000 mock records

2. Interactive Sandbox (CRUD Simulation) > 2.2 CRUD Operations

Progress: 0/16 (0%)

Pending:

⏳ Create: Add new mock document
⏳ Validate required fields
⏳ Show validation errors
⏳ Add to in-memory store
⏳ Show success message
⏳ Read: View mock document details
⏳ Render form in read-only mode
⏳ Show all field values
⏳ Update: Edit existing mock document
⏳ Validate changes
⏳ Update in-memory store
⏳ Show success message
⏳ Delete: Remove mock document
⏳ Show confirmation dialog
⏳ Remove from in-memory store
⏳ Show success message

2. Interactive Sandbox (CRUD Simulation) > 2.3 List Operations

Progress: 0/7 (0%)

Pending:

⏳ List (Few): Show 5 rows
⏳ Test basic list rendering
⏳ Test row actions (edit, delete)
⏳ List (Many): Generate 100+ rows
⏳ Test pagination
⏳ Test performance
⏳ Test scroll behavior

2. Interactive Sandbox (CRUD Simulation) > 2.4 Filter & Search

Progress: 0/4 (0%)

Pending:

⏳ Implement client-side filtering
⏳ Implement client-side search
⏳ Show filtered results count
⏳ Test filter combinations

2. Interactive Sandbox (CRUD Simulation) > 2.5 Validation Testing

Progress: 0/4 (0%)

Pending:

⏳ Submit form with missing required fields
⏳ Show field-level error messages
⏳ Show form-level error summary
⏳ Test custom validation rules

3. Local Hot Reload (Advanced) > 3.1 Prerequisites

Progress: 0/3 (0%)

Pending:

⏳ Local Postgres/SQLite running
⏳ m start running on localhost
⏳ Database connection configured

3. Local Hot Reload (Advanced) > 3.2 File Watcher Setup

Progress: 0/4 (0%)

Pending:

⏳ Install chokidar for file watching
⏳ Watch DocType files for changes
⏳ Debounce file change events (1s)
⏳ Ignore temporary files (.swp, .tmp)

3. Local Hot Reload (Advanced) > 3.3 Schema Sync Trigger

Progress: 0/4 (0%)

Pending:

⏳ Detect DocType file changes
⏳ Parse updated DocType
⏳ Generate schema diff
⏳ Send sync request to backend

3. Local Hot Reload (Advanced) > 3.4 Backend Schema Sync Endpoint

Progress: 0/6 (0%)

Pending:

⏳ Create POST /studio/api/sync-schema
⏳ Accept DocType schema
⏳ Generate SQLAlchemy table
⏳ Run ALTER TABLE if table exists
⏳ Create table if new
⏳ Return sync status

3. Local Hot Reload (Advanced) > 3.5 Real CRUD Operations

Progress: 0/4 (0%)

Pending:

⏳ Switch from mock data to real API calls
⏳ Use Framework M API (/api/v1/{doctype})
⏳ Show real validation errors
⏳ Persist data to local database

3. Local Hot Reload (Advanced) > 3.6 Hot Reload UI

Progress: 0/4 (0%)

Pending:

⏳ Add "Hot Reload" toggle in Studio settings
⏳ Show connection status indicator
⏳ Show sync progress
⏳ Show sync errors

4. Advanced Preview Features > 4.1 Responsive Preview

Progress: 0/6 (0%)

Pending:

⏳ Add device size selector:
⏳ Desktop (1920px)
⏳ Tablet (768px)
⏳ Mobile (375px)
⏳ Resize preview iframe
⏳ Test responsive layouts

4. Advanced Preview Features > 4.2 Dark Mode Preview

Progress: 0/3 (0%)

Pending:

⏳ Add dark mode toggle in preview
⏳ Test components in both modes
⏳ Ensure contrast compliance

4. Advanced Preview Features > 4.3 Accessibility Preview

Progress: 0/4 (0%)

Pending:

⏳ Add accessibility checker
⏳ Show WCAG violations
⏳ Highlight problematic elements
⏳ Suggest fixes

4. Advanced Preview Features > 4.4 Performance Preview

Progress: 0/4 (0%)

Pending:

⏳ Show render time
⏳ Show component count
⏳ Show bundle size estimate
⏳ Warn on performance issues

5. Collaboration Features > 5.1 Real-Time Collaboration (Optional)

Progress: 0/4 (0%)

Pending:

⏳ Add WebSocket support
⏳ Show other users editing same DocType
⏳ Show cursor positions
⏳ Implement operational transformation (OT)

5. Collaboration Features > 5.2 Change Tracking

Progress: 0/4 (0%)

Pending:

⏳ Show unsaved changes indicator
⏳ Show diff view (before/after)
⏳ Allow reverting changes
⏳ Show change history

5. Collaboration Features > 5.3 Comments & Annotations

Progress: 0/4 (0%)

Pending:

⏳ Add comment threads on fields
⏳ Show unresolved comments count
⏳ Allow @mentions
⏳ Integrate with Git PR comments

6. Schema Change Detection (Developer Guardrails)

Progress: 0/13 (0%)

Pending:

⏳ Implement SchemaAnalyzer class:
⏳ Compare Pydantic models with current DB schema
⏳ Classify changes: SAFE, WARNING, DANGEROUS
⏳ Generate actionable messages with docs links
⏳ CLI integration (m migrate:create):
⏳ Show analysis before generating migration
⏳ Block on DANGEROUS changes (cannot auto-generate)
⏳ Warn on WARNING changes (require --force)
⏳ Link to migration pattern documentation
⏳ Studio UI integration:
⏳ Show warnings when saving DocType
⏳ Recommend safer alternatives (nullable, defaults)
⏳ "Save Anyway" escape hatch

7. DocType Discovery & Scanning

Progress: 0/4 (0%)

Pending:

⏳ Optimize DocType scanning performance
⏳ Implement AST-based parsing
⏳ Handle large codebases (1000+ DocTypes)
⏳ Add caching strategy for development mode

9. Desk View Configuration

Progress: 0/11 (0%)

Pending:

⏳ Workspace Configurator:
⏳ UI to creating/managing workspaces (Sidebar items)
⏳ Assign DocTypes to workspaces
⏳ Define workspace shortcuts and charts
⏳ Configure sidebar ordering and visibility
⏳ Role-based workspace access permissions
⏳ View Builders:
⏳ Kanban View configurator (columns, cards)
⏳ Calendar View configurator (start/end fields)
⏳ Gantt View configurator (dependencies)
⏳ Map View configurator (geo fields)

10. Portal Builder (Puck Integration)

Progress: 0/14 (0%)

Pending:

⏳ Puck Integration:
⏳ Create PuckEditor wrapper component in Studio
⏳ Implement StorageAdapter to save pages as WebPage DocType
⏳ Implement ViewAdapter to render published pages
⏳ Component Bridge (The "Design System"):
⏳ Map Framework M UI components to Puck Config
⏳ Create Hero component (Title, Subtitle, Image, CTA)
⏳ Create CardGrid component (Dynamic List)
⏳ Create Form component (Connect to DocType)
⏳ Create Login component
⏳ Data Binding (Advanced):
⏳ Create FrameworkMDataProvider for Puck "Root"
⏳ Fetch data from /api/v1/content/:slug
⏳ Pass data into Puck zones/components

11. Testing > 11.1 Unit Tests

Progress: 0/4 (0%)

Pending:

⏳ Test mock data generator
⏳ Test in-memory CRUD operations
⏳ Test file watcher
⏳ Test schema sync logic

11. Testing > 11.2 Integration Tests

Progress: 0/4 (0%)

Pending:

⏳ Test full sandbox workflow
⏳ Test hot reload with real database
⏳ Test preview rendering
⏳ Test collaboration features

12. Validation Checklist

Progress: 0/6 (0%)

Pending:

⏳ Sandbox preview works for all field types
⏳ Mock data generation is realistic
⏳ CRUD operations work in sandbox
⏳ Hot reload syncs schema correctly
⏳ Preview is responsive and accessible
⏳ File watcher is reliable

Overview
Phase 01: Core Kernel & Interfaces
Phase 02: DocType Engine & Database
Phase 03: API Layer & Authorization
Phase 04: Background Jobs & Events
Phase 05: CLI & Developer Tools
Phase 06: Built-in DocTypes & Core Features
Phase 07: Studio (Code Generation UI)
Phase 08: Workflows & Advanced Features
Phase 09A: Frontend
Phase 09B: Documentation
Phase 10: ERP Enterprise Features
Phase 10: Production Readiness & Deployment
Phase 11: Package Split & MX Pattern
Phase 12: LLM-Ready Documentation & Automation
Phase 13: UI/UX Polish & Design System Enhancement
Phase 14: Studio Enhancements

Overview​

Phase 01: Core Kernel & Interfaces​

1. Project Initialization​

2. Define Port Interfaces > 2.1 Repository Protocol​

2. Define Port Interfaces > 2.2 Event Bus Protocol​

2. Define Port Interfaces > 2.3 Auth Context Protocol​

2. Define Port Interfaces > 2.4 Storage Protocol​

2. Define Port Interfaces > 2.5 Job Queue Protocol​

2. Define Port Interfaces > 2.6 Permission Protocol​

2. Define Port Interfaces > 2.7 Print Protocol​

2. Define Port Interfaces > 2.8 Cache Protocol​

2. Define Port Interfaces > 2.9 Notification Protocol​

2. Define Port Interfaces > 2.10 Search Protocol​

2. Define Port Interfaces > 2.11 I18n Protocol​

3. Define Domain Layer > 3.1 Base DocType​

3. Define Domain Layer > 3.2 Base Controller​

3. Define Domain Layer > 3.3 Mixins​

4. Meta Registry​

5. Port Implementation (Adapters) > 5.1 Dependency Injection​

5. Port Implementation (Adapters) > 5.2 Provider Types​

5. Port Implementation (Adapters) > 5.3 Configuration Provider​

5. Port Implementation (Adapters) > 5.4 Wiring​

5. Port Implementation (Adapters) > 5.5 Entrypoint Scanning for Overrides​

5. Port Implementation (Adapters) > 5.6 Testing Support​

6. Testing Setup​

7. Documentation​

Validation Checklist​

Phase 02: DocType Engine & Database​

1. Schema Mapper (Pydantic → SQLAlchemy) > 1.1 Field Registry & Type Mapping​

1. Schema Mapper (Pydantic → SQLAlchemy) > 1.2 Schema Mapper Implementation​

1. Schema Mapper (Pydantic → SQLAlchemy) > 1.3 Table Registry​

2. Database Connection Setup > 2.1 Connection Factory & Virtual DocTypes​

2. Database Connection Setup > 2.2 Session Factory (SQL Support)​

2. Database Connection Setup > 2.3 Non-SQL Virtual DocTypes (Custom Repositories)​

3. Generic Repository Implementation​

3. Generic Repository Implementation > 3.1 CRUD Operations​

3. Generic Repository Implementation > 3.2 Lifecycle Hook Integration​

4. Migration System > 4.1 Alembic Integration​

4. Migration System > 4.2 Auto-Migration Detection​

4. Migration System > 4.3 CLI Commands​

5. Repository Factory​

6. Engine, Session Factory & Unit of Work > 6.1 Engine & Session Factory Setup​

6. Engine, Session Factory & Unit of Work > 6.2 Unit of Work (UnitOfWork)​

6. Engine, Session Factory & Unit of Work > 6.3 Multi-Source Coordination (Outbox Pattern)​

6. Engine, Session Factory & Unit of Work > 6.4 Startup Sequence (Schema Sync)​

7. Testing > 7.1 Unit Tests​

7. Testing > 7.2 Integration Tests​

8. Error Handling​

9. Performance Optimizations​

Validation Checklist​

Phase 03: API Layer & Authorization​

1. Litestar Application Setup​

2. Authentication Middleware​

3. Permission System > 3.1 Permission Protocol Implementation​

3. Permission System > 3.2 Permission Configuration​

3. Permission System > 3.3 System Context (Elevated Operations)​

3. Permission System > 3.4 Row-Level Security (RLS)​

3. Permission System > 3.5 Team-Based Access (Indie)​

3. Permission System > 3.6 Explicit Sharing (DocumentShare)​

3. Permission System > 3.7 Indie Mode: Permission Conveniences​

4. Auto-CRUD Router > 4.1 Meta Router Implementation​

4. Auto-CRUD Router > 4.2 List Endpoint​

4. Auto-CRUD Router > 4.3 Create Endpoint​

4. Auto-CRUD Router > 4.4 Read Endpoint​

4. Auto-CRUD Router > 4.5 Update Endpoint​

4. Auto-CRUD Router > 4.6 Delete Endpoint​

4. Auto-CRUD Router > 4.7 Router Registration​

5. RPC System > 5.1 Whitelisted Controller Methods​

5. RPC System > 5.2 Arbitrary RPC Functions​

6. Metadata API​

7. OpenAPI Documentation​

8. Error Handling​

9. Request/Response DTOs​

10. Testing > 10.1 Unit Tests​

10. Testing > 10.2 Integration Tests​

Validation Checklist​

Edge Cases to Handle > Child Table Permissions​

Edge Cases to Handle > Link Field Data Leakage​

Edge Cases to Handle > Bulk Operations & RLS​

Phase 04: Background Jobs & Events​

Overview

Phase 01: Core Kernel & Interfaces

1. Project Initialization

2. Define Port Interfaces > 2.1 Repository Protocol

2. Define Port Interfaces > 2.2 Event Bus Protocol

2. Define Port Interfaces > 2.3 Auth Context Protocol

2. Define Port Interfaces > 2.4 Storage Protocol

2. Define Port Interfaces > 2.5 Job Queue Protocol

2. Define Port Interfaces > 2.6 Permission Protocol

2. Define Port Interfaces > 2.7 Print Protocol

2. Define Port Interfaces > 2.8 Cache Protocol

2. Define Port Interfaces > 2.9 Notification Protocol

2. Define Port Interfaces > 2.10 Search Protocol

2. Define Port Interfaces > 2.11 I18n Protocol

3. Define Domain Layer > 3.1 Base DocType

3. Define Domain Layer > 3.2 Base Controller

3. Define Domain Layer > 3.3 Mixins

4. Meta Registry

5. Port Implementation (Adapters) > 5.1 Dependency Injection

5. Port Implementation (Adapters) > 5.2 Provider Types

5. Port Implementation (Adapters) > 5.3 Configuration Provider

5. Port Implementation (Adapters) > 5.4 Wiring

5. Port Implementation (Adapters) > 5.5 Entrypoint Scanning for Overrides

5. Port Implementation (Adapters) > 5.6 Testing Support

6. Testing Setup

7. Documentation

Validation Checklist

Phase 02: DocType Engine & Database

1. Schema Mapper (Pydantic → SQLAlchemy) > 1.1 Field Registry & Type Mapping

1. Schema Mapper (Pydantic → SQLAlchemy) > 1.2 Schema Mapper Implementation

1. Schema Mapper (Pydantic → SQLAlchemy) > 1.3 Table Registry

2. Database Connection Setup > 2.1 Connection Factory & Virtual DocTypes

2. Database Connection Setup > 2.2 Session Factory (SQL Support)

2. Database Connection Setup > 2.3 Non-SQL Virtual DocTypes (Custom Repositories)

3. Generic Repository Implementation

3. Generic Repository Implementation > 3.1 CRUD Operations

3. Generic Repository Implementation > 3.2 Lifecycle Hook Integration

4. Migration System > 4.1 Alembic Integration

4. Migration System > 4.2 Auto-Migration Detection

4. Migration System > 4.3 CLI Commands

5. Repository Factory

6. Engine, Session Factory & Unit of Work > 6.1 Engine & Session Factory Setup

6. Engine, Session Factory & Unit of Work > 6.2 Unit of Work (`UnitOfWork`)

6. Engine, Session Factory & Unit of Work > 6.3 Multi-Source Coordination (Outbox Pattern)

6. Engine, Session Factory & Unit of Work > 6.4 Startup Sequence (Schema Sync)

7. Testing > 7.1 Unit Tests

7. Testing > 7.2 Integration Tests

8. Error Handling

9. Performance Optimizations

Validation Checklist

Phase 03: API Layer & Authorization

1. Litestar Application Setup

2. Authentication Middleware

3. Permission System > 3.1 Permission Protocol Implementation

3. Permission System > 3.2 Permission Configuration

3. Permission System > 3.3 System Context (Elevated Operations)

3. Permission System > 3.4 Row-Level Security (RLS)

3. Permission System > 3.5 Team-Based Access (Indie)

3. Permission System > 3.6 Explicit Sharing (`DocumentShare`)

3. Permission System > 3.7 Indie Mode: Permission Conveniences

4. Auto-CRUD Router > 4.1 Meta Router Implementation

4. Auto-CRUD Router > 4.2 List Endpoint

4. Auto-CRUD Router > 4.3 Create Endpoint

4. Auto-CRUD Router > 4.4 Read Endpoint

4. Auto-CRUD Router > 4.5 Update Endpoint

4. Auto-CRUD Router > 4.6 Delete Endpoint

4. Auto-CRUD Router > 4.7 Router Registration

5. RPC System > 5.1 Whitelisted Controller Methods

5. RPC System > 5.2 Arbitrary RPC Functions

6. Metadata API

7. OpenAPI Documentation

8. Error Handling

9. Request/Response DTOs

10. Testing > 10.1 Unit Tests

10. Testing > 10.2 Integration Tests

Validation Checklist

Edge Cases to Handle > Child Table Permissions

Edge Cases to Handle > Link Field Data Leakage

Edge Cases to Handle > Bulk Operations & RLS

Phase 04: Background Jobs & Events