Document: docs/rfcs/rfc-0006-authentication.md

# RFC-0006: Authentication — Simple Login & Social OAuth for Framework M

- **Status**: Accepted
- **Author(s)**: @anshpansuriya14
- **Created**: 2026-02-11
- **Updated**: 2026-02-12
- **TSC Decision**: Approved

## Summary

This RFC documents how authentication works (and should work) in Framework M, covering **simple email/password login** and **Login with Google/GitHub (OAuth2)**. It maps the current state of backend and frontend implementations, identifies critical gaps that prevent a working end-to-end login flow, analyzes alternative approaches, and proposes a concrete implementation plan.

## Motivation

Framework M's architecture mandates **stateless, JWT-based auth** with pluggable identity providers. The codebase has significant **scaffolding code** across four layers:

- **Core Interfaces**: `AuthenticationProtocol`, `IdentityProtocol`, `OAuth2Protocol`, `AuthContextProtocol`
- **Backend Adapters**: `LocalIdentityAdapter` (argon2 + JWT), `FederatedIdentityAdapter`, 6 auth strategies
- **Auth Routes**: `/api/v1/auth/login`, `/logout`, `/me`, `/oauth/{provider}/start|callback`
- **Frontend**: Cookie-based `authProvider`, OIDC `oidcAuthProvider`, `LoginPage` component

> [!CAUTION]
> **Nothing currently works end-to-end.** All of the above is scaffolding/mock code:
>
> - Backend login returns a hardcoded dev-mode mock token (UserManager is never wired)
> - Auth middleware runs with `require_auth=False`, so all requests pass through unauthenticated
> - OAuth routes are not even registered in the running application
> - Frontend authProvider expects cookies but backend never sets any
> - `oidc-client-ts` is not installed
>
> This RFC documents the scaffolding that exists, identifies what needs to be implemented to make login actually work, and proposes the implementation plan.

---

## Current State Analysis

### How Login Currently "Works" (All Mock)

#### Simple Login (Email + Password)

The `create_app()` factory in `app.py` registers `auth_routes_router` but **never calls `configure_auth_routes(user_manager)`**. This means `_user_manager` is always `None`, so the login endpoint always hits the dev-mode mock path:

```mermaid
sequenceDiagram
    participant Browser
    participant Frontend as Frontend (Vite + Refine)
    participant Backend as Backend (Litestar)

    Browser->>Frontend: User enters email + password
    Frontend->>Backend: POST /api/v1/auth/login {email, password}
    Note over Backend: _user_manager is None (never configured)
    Note over Backend: FRAMEWORK_M_DEV_MODE defaults to "true"
    Backend-->>Frontend: {access_token: "dev-token-user-at-example", user: {id: "dev-user-1", mock data}}
    Note over Frontend: authProvider expects Set-Cookie but gets JSON body
    Frontend->>Frontend: Login "succeeds" but no cookie is set
    Frontend->>Backend: GET /api/v1/auth/me (no cookie sent)
    Note over Backend: request.state.user is None (require_auth=False)
    Backend-->>Frontend: {authenticated: false, id: "guest", name: "Guest User"}
    Note over Frontend: authProvider.check() sees unauthenticated → redirects to /login
```

> [!CAUTION]
> **Nothing works**: The backend returns a mock JWT in the response body, but the frontend `authProvider` uses `credentials: "include"` expecting **HttpOnly cookies**. The backend never calls `Set-Cookie`. The middleware runs with `require_auth=False`, so `request.state.user` is always `None`. The `/auth/me` endpoint returns a guest user, and `authProvider.check()` redirects back to `/login`.

#### Social Login (Google OAuth2)

> [!CAUTION]
> **Not registered.** The `oauth_routes_router` is **not added to `create_app()`'s route handlers list** — it exists as code in `oauth_routes.py` but is completely unreachable. Even if it were registered, the callback handler is a placeholder that returns `{"note": "Full implementation requires authlib integration"}`.

The OAuth routes exist as scaffolding:

- `GET /api/v1/auth/oauth/{provider}/start` — generates a state token and redirects to the provider
- `GET /api/v1/auth/oauth/{provider}/callback` — placeholder, no actual token exchange
- Well-known configs defined for Google, GitHub, Microsoft
- No `SocialAccount` DocType exists to link provider accounts to local users

---

### Existing Code Map

#### Backend Layer

| File                                                                                | Purpose                                                                                |  Code Exists  |              Actually Works              |
| -------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- | :-----------: | :--------------------------------------: |
| [authentication.py](file:///Users/ansh/Desktop/CastleCraft/m/libs/framework-m-core/src/framework_m_core/interfaces/authentication.py)                    | `AuthenticationProtocol` — chain-of-responsibility interface                                 |      ✅       |             N/A (interface)              |
| [auth_context.py](file:///Users/ansh/Desktop/CastleCraft/m/libs/framework-m-core/src/framework_m_core/interfaces/auth_context.py)                        | `AuthContextProtocol` + `UserContext` model                                                  |      ✅       |             N/A (interface)              |
| [identity.py](file:///Users/ansh/Desktop/CastleCraft/m/libs/framework-m-core/src/framework_m_core/interfaces/identity.py)                                | `IdentityProtocol`, `Token`, `Credentials` models                                            |      ✅       |             N/A (interface)              |
| [oauth.py](file:///Users/ansh/Desktop/CastleCraft/m/libs/framework-m-core/src/framework_m_core/interfaces/oauth.py)                                      | `OAuth2Protocol`, `OAuth2Token`, `OAuth2UserInfo`                                            |      ✅       |             N/A (interface)              |
| [local_identity.py](file:///Users/ansh/Desktop/CastleCraft/m/libs/framework-m-standard/src/framework_m_standard/adapters/auth/local_identity.py)         | `LocalIdentityAdapter` — argon2 hashing, JWT generation                                      |      ✅       |          ❌ Never instantiated           |
| [federated_identity.py](file:///Users/ansh/Desktop/CastleCraft/m/libs/framework-m-standard/src/framework_m_standard/adapters/auth/federated_identity.py) | `FederatedIdentityAdapter` — hydrate from gateway headers                                    |      ✅       |          ❌ Never instantiated           |
| [strategies.py](file:///Users/ansh/Desktop/CastleCraft/m/libs/framework-m-standard/src/framework_m_standard/adapters/auth/strategies.py)                 | `BearerTokenAuth`, `ApiKeyAuth`, `BasicAuth`, `HeaderAuth`, `SessionCookieAuth`, `AuthChain` |      ✅       |           ❌ Never configured            |
| [auth_routes.py](file:///Users/ansh/Desktop/CastleCraft/m/libs/framework-m-standard/src/framework_m_standard/adapters/web/auth_routes.py)                | `/login`, `/logout`, `/me` endpoints                                                         | ✅ Registered |          ❌ Always returns mock          |
| [oauth_routes.py](file:///Users/ansh/Desktop/CastleCraft/m/libs/framework-m-standard/src/framework_m_standard/adapters/web/oauth_routes.py)              | `/oauth/{provider}/start`, `/callback`                                                       |      ✅       |         ❌ Not registered in app         |
| [auth_middleware.py](file:///Users/ansh/Desktop/CastleCraft/m/libs/framework-m-standard/src/framework_m_standard/adapters/web/auth_middleware.py)        | ASGI middleware — reads `x-user-id` headers                                                  | ✅ Registered |         ❌ `require_auth=False`          |
| [app.py](file:///Users/ansh/Desktop/CastleCraft/m/libs/framework-m-standard/src/framework_m_standard/adapters/web/app.py)                                | `create_app()` — wires everything together                                                   |      ✅       | ❌ Never calls `configure_auth_routes()` |

#### Frontend Layer

| File                                                                                | Purpose                                   | Code Exists |              Actually Works              |
| ---------------------------------------------------------------------------------------------------------- | ----------------------------------------- | :---------: | :--------------------------------------: |
| [authProvider.ts](file:///Users/ansh/Desktop/CastleCraft/m/frontend/src/providers/authProvider.ts)         | Cookie-based auth provider for Refine     |     ✅      | ❌ Expects cookies, backend returns JSON |
| [oidcAuthProvider.ts](file:///Users/ansh/Desktop/CastleCraft/m/frontend/src/providers/oidcAuthProvider.ts) | OIDC PKCE flow with `oidc-client-ts`      |     ✅      |    ❌ `oidc-client-ts` not installed     |
| [authConfig.ts](file:///Users/ansh/Desktop/CastleCraft/m/frontend/src/providers/authConfig.ts)             | Auth strategy config (`cookie` vs `oidc`) |     ✅      |         ❌ Config never injected         |
| [LoginPage.tsx](file:///Users/ansh/Desktop/CastleCraft/m/frontend/src/pages/LoginPage.tsx)                 | Email/password form using `useLogin`      |     ✅      |     ❌ Submits but auth check fails      |
| [App.tsx](file:///Users/ansh/Desktop/CastleCraft/m/frontend/src/App.tsx)                                   | Main app with auth routing                |     ✅      |       ⚠ Routes work, auth doesn't       |

---

## Detailed Design

### Strategy A: BFF Cookie Mode (Recommended for Indie/Self-hosted)

This is the **recommended default** for self-hosted deployments. The backend acts as a Backend-for-Frontend (BFF), setting HttpOnly cookies.

#### Simple Login Flow

```mermaid
sequenceDiagram
    participant Browser
    participant Frontend
    participant Backend
    participant DB

    Browser->>Frontend: Submit email + password
    Frontend->>Backend: POST /api/v1/auth/login {email, password}
    Backend->>DB: LocalUser lookup + argon2 verify
    Backend->>Backend: Create session (UUID → Redis/DB)
    Backend-->>Frontend: 200 OK + Set-Cookie: session_id=UUID; HttpOnly; Secure; SameSite=Lax
    Frontend->>Backend: GET /api/v1/auth/me (cookie auto-sent)
    Backend->>Backend: SessionCookieAuth validates session_id
    Backend-->>Frontend: {id, email, name, roles}
    Frontend->>Browser: Redirect to /
```

**Key Changes Required:**

1. **`auth_routes.py` `/login`**: After `_user_manager.authenticate()`, create a session entry and return `Set-Cookie` header instead of raw JWT
2. **`auth_routes.py` `/logout`**: Delete session from store, return `Set-Cookie` with `Max-Age=0`
3. **`auth_middleware.py`**: Add `SessionCookieAuth` to the auth chain (already implemented in `strategies.py`)
4. **Session Store**: New `SessionStoreProtocol` with `RedisSessionAdapter` or `DatabaseSessionAdapter`

#### Google OAuth Flow

```mermaid
sequenceDiagram
    participant Browser
    participant Frontend
    participant Backend
    participant Google
    participant DB

    Browser->>Frontend: Click "Login with Google"
    Frontend->>Backend: GET /api/v1/auth/oauth/google/start
    Backend->>Backend: Generate state + PKCE verifier, store in session
    Backend-->>Browser: 302 → accounts.google.com/o/oauth2/auth
    Browser->>Google: User consents
    Google-->>Browser: 302 → /api/v1/auth/oauth/google/callback?code=X&state=Y
    Browser->>Backend: GET /callback?code=X&state=Y
    Backend->>Backend: Validate state (CSRF check)
    Backend->>Google: POST token endpoint (exchange code)
    Google-->>Backend: {access_token, id_token}
    Backend->>Google: GET /userinfo
    Google-->>Backend: {sub, email, name, picture}
    Backend->>DB: Find or create user (upsert SocialAccount)
    Backend->>Backend: Create session
    Backend-->>Browser: 302 → / + Set-Cookie: session_id=UUID
```

**Key Changes Required:**

1. **`oauth_routes.py` `/callback`**: Implement actual code exchange using `httpx`
2. **`SocialAccount` DocType**: Link provider user IDs to local users
3. **User upsert logic**: Create user on first OAuth login, link on subsequent
4. **Frontend `LoginPage.tsx`**: Add "Login with Google" / "Login with GitHub" buttons

### Strategy B: Client-Side OIDC (For Enterprise/External IdP)

For enterprise deployments where an external Identity Provider (Keycloak, Auth0, Okta) handles all authentication.

```mermaid
sequenceDiagram
    participant Browser
    participant Frontend
    participant IdP as Keycloak / Auth0
    participant Backend

    Browser->>Frontend: Click "Login"
    Frontend->>IdP: PKCE Authorization Request
    IdP-->>Browser: Login page
    Browser->>IdP: Credentials
    IdP-->>Frontend: Authorization code
    Frontend->>IdP: Exchange code (with PKCE verifier)
    IdP-->>Frontend: {access_token (JWT), id_token}
    Frontend->>Frontend: Store tokens in memory
    Frontend->>Backend: API calls with Authorization: Bearer <token>
    Backend->>Backend: BearerTokenAuth validates JWT (public key)
    Backend-->>Frontend: Response
```

**Existing Support**: The `oidcAuthProvider.ts` and `BearerTokenAuth` strategy already handle this. Requires `pnpm add oidc-client-ts` and `window.__FRAMEWORK_CONFIG__` configuration.

---

## Alternatives

| Alternative                                                                       | Pros                                                                                | Cons                                                                                | Verdict                    |
| --------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- | -------------------------- |
| **A. BFF Cookie Mode** (HttpOnly cookie, server-side session)                     | Most secure for web apps. No XSS token theft. Simple frontend. CSRF protection via `SameSite=Lax`. | Requires session store (Redis/DB). Slightly more backend complexity.                    | **✅ Recommended default** |
| **B. Client-Side OIDC** (PKCE + in-memory tokens)                                 | Works with any OIDC provider. No backend session state. Enterprise-ready.                          | Requires `oidc-client-ts`. Tokens in JS memory (lost on refresh). Complex silent-renew. | ✅ Good for enterprise     |
| **C. JWT in localStorage**                                                        | Simplest to implement. Survives page refresh.                                                      | XSS vulnerability — any script can steal tokens. Not recommended by OWASP.              | ❌ Rejected (security)     |
| **D. JWT in HttpOnly cookie** (no session store)                                  | Stateless. No session store needed. Cookie-based security.                                         | Can't revoke tokens. Large cookies (JWT ≥ 500B). Token rotation complexity.             | ⚠ Acceptable tradeoff     |
| **E. Refresh token rotation** (access JWT in memory + refresh in HttpOnly cookie) | Best security. Short-lived access tokens. Revocable refresh.                                       | Most complex. Requires refresh endpoint. Token rotation logic.                          | ⚠ Future enhancement      |
| **F. Passkeys / WebAuthn**                                                        | Phishing-resistant. No passwords. Modern UX.                                                       | Browser support varies. Complex server implementation. Users unfamiliar.                | ⚠ Future phase            |

### Recommended Approach

**Strategy A (BFF Cookie Mode) as default** with Strategy B available via config toggle.

Rationale:

- Framework M targets **indie/self-hosted** deployments primarily — cookie mode is simplest and most secure
- The existing `authProvider.ts` already expects cookies (`credentials: "include"`)
- The `SessionCookieAuth` strategy already exists in `strategies.py`
- Enterprise users can switch to Strategy B via `window.__FRAMEWORK_CONFIG__.auth.authStrategy = "oidc"`

---

## Drawbacks

- **Session store dependency**: BFF mode requires Redis or a sessions DB table. This adds infrastructure, but Redis is already required for cache/events. For dev/indie without Redis, `DatabaseSessionAdapter` provides a zero-dependency fallback.
- **CORS complexity**: Cookie-based auth requires correct `SameSite`, `Secure`, and CORS configuration. Misconfiguration leads to silent failures.
- **OAuth provider registration**: Each OIDC provider requires manual app registration and client secret management. Mitigated by the generic OIDC approach — users only need to provide well-known URLs.

---

## Implementation Plan

> [!NOTE]
> The codebase already has extensive scaffolding. The work is primarily **wiring existing code** and filling specific gaps. Existing code:
>
> - `session_store.py`: `DatabaseSessionAdapter` + `RedisSessionAdapter` (438 lines, fully implemented)
> - `strategies.py`: `SessionCookieAuth` + `AuthChain` + 4 other strategies (555 lines)
> - `local_identity.py`: `LocalIdentityAdapter` with argon2 + JWT (237 lines)
> - `user_manager.py`: `UserManager` service (188 lines)
> - `auth_routes.py`: Login/logout/me endpoints (252 lines, needs session cookie wiring)
> - `oauth_routes.py`: OAuth start/callback with generic OIDC support (279 lines, callback needs implementation)
> - `SocialAccount` DocType: Provider→user linking (89 lines)
> - `LocalUser` DocType: Email/password user (116 lines)

### Phase 1: Wire Login + Session Cookies (Make Login Work)

Connect the existing scaffolding so email/password login produces a working session.

- [ ] **Wire `create_app()`**: Instantiate `LocalIdentityAdapter` → `UserManager` → call `configure_auth_routes(user_manager)`
- [ ] **Session store selection**: Use `RedisSessionAdapter` if `REDIS_URL` is set, else `DatabaseSessionAdapter`
- [ ] **Modify `/login`**: After `_user_manager.authenticate()`, create session via session store + return `Set-Cookie: session_id=UUID; HttpOnly; Secure; SameSite=Lax`
- [ ] **Modify `/logout`**: Delete session + set `Set-Cookie` with `Max-Age=0`
- [ ] **Wire `AuthChain`**: Replace header-based `AuthMiddleware` with `SessionCookieAuth` → `BearerTokenAuth` → `HeaderAuth` chain
- [ ] **Enable auth**: Set `require_auth=True` with excluded paths: `/health`, `/schema`, `/api/v1/auth/login`, `/api/v1/auth/logout`, `/desk`
- [ ] **Admin seeding**: Add `m create-admin` CLI command using `UserManager.create()` + `hash_password()`
- [ ] **Ensure tables**: Include `LocalUser` + `SocialAccount` in schema sync (currently only `api_resource=True` DocTypes get tables)

### Phase 2: Generic OIDC + Auto-Linking

- [ ] **Register OAuth routes**: Add `oauth_routes_router` to `create_app()` route handlers
- [ ] **Implement `/callback`**: Exchange code for tokens using `httpx`, validate state
- [ ] **Implement `auto_link_user()` service**:
  1. Match `SocialAccount` by `(provider, provider_user_id)` → login
  2. If missing + `email_verified=True` → match `LocalUser` by email → link + login
  3. If no match → create `LocalUser` (random password) + `SocialAccount` → login
- [ ] **CSRF state storage**: Store OAuth state in session store with short TTL
- [ ] **Provider config**: Load OIDC provider configs from `framework_config.toml`

### Phase 3: Frontend Integration

- [ ] Update `authProvider.ts` to use cookie-based flow (remove JWT body parsing)
- [ ] Add social login buttons to `LoginPage.tsx` (redirect to `/api/v1/auth/oauth/{provider}/start`)
- [ ] Handle OAuth callback redirect (cookie already set by backend, just redirect to `/`)
- [ ] Remove unused `oidcAuthProvider.ts` complexity (Strategy B deferred)

### Phase 4: Tests + Docs

- [ ] Unit tests for session store adapters (Redis + DB)
- [ ] Integration tests for login → session → /me flow
- [ ] Integration tests for OAuth start → callback → session flow
- [ ] Documentation updates

---

## Architecture Decisions

### 1. Session Storage: Redis (with DB fallback)

- **Decision**: Use Redis for session storage. Redis is already a dependency for cache/events.
- **Implementation**: `RedisSessionAdapter` (already implemented in `session_store.py`) when `REDIS_URL` is set. `DatabaseSessionAdapter` (also implemented) as fallback for development/indie deployments without Redis.
- **Data**: Store minimal session info (`user_id`, `created_at`, `ip_address`, `user_agent`, `csrf_token`).
- **TTL**: 14 days (rolling), configurable via `SessionConfig`.
- **Existing Code**: `framework_m_standard/adapters/auth/session_store.py` — both adapters are fully implemented.

### 2. OAuth Strategy: Generic OIDC

- **Decision**: Use a single **Generic OIDC-compliant** implementation for ALL providers (Google, GitHub, Microsoft, and any custom OIDC provider like Keycloak, Auth0, Okta).
- **Configuration** (in `framework_config.toml`):

  ```toml
  [auth.oauth]
  enabled = true

  [auth.oauth.providers.google]
  client_id = "..."
  client_secret = "..."
  # Well-known URLs auto-resolved for google/github/microsoft

  [auth.oauth.providers.my-keycloak]
  client_id = "..."
  client_secret = "..."
  authorization_url = "https://keycloak.example.com/auth/realms/master/protocol/openid-connect/auth"
  token_url = "https://keycloak.example.com/auth/realms/master/protocol/openid-connect/token"
  userinfo_url = "https://keycloak.example.com/auth/realms/master/protocol/openid-connect/userinfo"
  scope = "openid email profile"
  ```

- **Well-Known Discovery**: For custom providers, optionally support `.well-known/openid-configuration` auto-discovery.
- **Existing Code**: `oauth_routes.py` already has `WELL_KNOWN_PROVIDERS`, `get_oidc_well_known()`, and `is_generic_oidc_provider()`. The `/start` endpoint works; the `/callback` needs token exchange implementation.

### 3. Account Linking: Auto-Link via Verified Email

- **Decision**: Automatically link OAuth accounts to local users if **emails match** AND **email is verified** by the provider.
- **Security**: Only trust `email_verified=true` OIDC claim. If provider does not return `email_verified` or it is `false`, create a new `LocalUser` instead of linking.
- **Flow**:
  1. Login with OIDC provider
  2. Check for existing `SocialAccount` by `(provider, provider_user_id)` → login immediately
  3. If not found, check `email_verified` claim from provider
  4. If verified, check for `LocalUser` with same email → create `SocialAccount` link → login
  5. If no email match, create new `LocalUser` (with random password) + `SocialAccount` → login
  6. If `email_verified` is `false` or missing, always create a new `LocalUser` (never auto-link)
- **Existing Code**: `SocialAccount` DocType is defined at `framework_m_core/doctypes/social_account.py`.

### 4. Roles & Permissions: Best Practices

- **Decision**: Keep roles simple — a `roles` field (JSON list of strings) on `LocalUser`. No separate Role DocType for now.
- **Storage**: `["Admin", "Manager", "User"]` — simple JSON array.
- **Initial Admin**: Created via CLI command `m create-admin`. This is the easiest and most secure approach for bootstrapping.
- **Best Practices (noted for future reference)**:
  - **RBAC (Role-Based Access Control)**: Standard approach. Roles grant permissions. Users get roles. Framework M uses this via the `x-roles` header already.
  - **ABAC (Attribute-Based Access Control)**: `LocalIdentityAdapter.get_attributes()` already supports this. Can be extended for fine-grained policies.
  - **Permission resolution**: In the future, a `Permission` model can map `(role, doctype, action)` → allow/deny. For now, the `Meta.requires_auth` DocType flag + `apply_rls` is sufficient.
  - **Principle of Least Privilege**: New users default to `["User"]` role. Admin must be explicitly granted via CLI.
  - **CLI-first admin management**: `m create-admin --email admin@example.com` is the simplest, most secure bootstrapping method. No need for admin UI in Phase 1.
  - **Future enhancements**: Role management UI, permission matrices, team-based access can be layered on top of this simple foundation.

---

## References

- [ARCHITECTURE.md](file:///Users/ansh/Desktop/CastleCraft/m/ARCHITECTURE.md) — §3.3 Authorization Strategy, §5.4 Metadata-Driven UI
- [migration-from-frappe.md](file:///Users/ansh/Desktop/CastleCraft/m/docs/notes/migration-from-frappe.md) — §2 Getting Current User, §9 Permissions
- [auth_routes.py](file:///Users/ansh/Desktop/CastleCraft/m/libs/framework-m-standard/src/framework_m_standard/adapters/web/auth_routes.py) — Current login endpoint
- [oauth_routes.py](file:///Users/ansh/Desktop/CastleCraft/m/libs/framework-m-standard/src/framework_m_standard/adapters/web/oauth_routes.py) — OAuth routes (placeholder callback)
- [strategies.py](file:///Users/ansh/Desktop/CastleCraft/m/libs/framework-m-standard/src/framework_m_standard/adapters/auth/strategies.py) — All 6 auth strategies including `SessionCookieAuth`
- [authProvider.ts](file:///Users/ansh/Desktop/CastleCraft/m/frontend/src/providers/authProvider.ts) — Frontend cookie auth provider
- [OWASP Session Management Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Session_Management_Cheat_Sheet.html)
- [OAuth 2.0 for Browser-Based Apps (RFC draft)](https://datatracker.ietf.org/doc/html/draft-ietf-oauth-browser-based-apps)


Document: docs/rfcs/rfc-0008-compliance-edit-mode.md

# RFC-0008: Compliance Edit Mode for Submitted Documents

- **Status**: Approved
- **Author**: @revant.one
- **Created**: 2026-02-24
- **Updated**: 2026-02-24
- **TSC Decision**: Approved

## Summary

This RFC proposes an "Explicit Edit Mode" for Submitted Documents to complement the `allow_on_submit` feature introduced in [RFC-0004](./rfc-0004-submitted-document-mutability.md). While `allow_on_submit` is suitable for minor metadata updates, heavily regulated environments require a formal, auditable state machine for modifying documents after submission. This RFC defines a `ComplianceSubmittable` pattern.

## Motivation

**The Problem:**
In Framework M, RFC-0004 allows developers to mark specific fields as `allow_on_submit=True`. This permits transparent, low-friction updates to metadata (like "Remarks" or "Vehicle Number") without unsubmitting the document.

However, compliance teams often reject "silent" updates to submitted financial or legal documents, even for metadata. They demand that:
1. The intent to edit is explicitly declared before making changes.
2. A required justification/reason is provided.
3. The document is temporarily locked from downstream processing while being edited.
4. An explicit "diff" and the reason are permanently logged.

**Our Approach:**
Instead of making RFC-0004's simple `allow_on_submit` overly complex for everyone, we introduce a progressive feature. We define an opt-in `ComplianceSubmittableMixin` (or an extension of the existing submission state machine) that enforces an explicit "Unlock -> Edit -> Relock" workflow when high auditability is required.

## Detailed Design

### 1. The State Machine

We introduce new conceptual states to the standard `DocStatus` (Draft -> Submitted -> Cancelled).

- `SUBMITTED`: Normal submitted state.
- `EDITING`: The document has been unlocked for authorized changes and cannot currently be processed by downstream systems (e.g., cannot generate a payment entry against it while it is being edited).

### 2. The `ComplianceSubmittableMixin`

We introduce a mixin that enforces the edit-mode workflow.

```python
class ComplianceSubmittableMixin(SubmittableMixin):
    async def unlock_for_edit(self, reason: str, user: UserContext):
        # 1. Verify user permissions
        # 2. Change state to EDITING
        # 3. Log the "Unlock" event with the reason
        pass

    async def save_and_relock(self, user: UserContext):
        # 1. Diff against original SUBMITTED state
        # 2. Ensure only `allow_on_submit` fields were changed
        # 3. Change state back to SUBMITTED
        # 4. Generate DocumentEditLog entry containing the diff
        pass
```

### 3. Controller / Repository Enforcement

The `GenericRepository.save()` logic defined in RFC-0004 must be extended for models implementing this mixin:

1. If model has `ComplianceSubmittableMixin` AND is currently `SUBMITTED`:
   - Reject standard `repo.save(doc)` operations with a `DocumentLockedError`.
2. If model has `ComplianceSubmittableMixin` AND is currently `EDITING`:
   - Allow `repo.save(doc)` but strictly enforce the diff-check (only `allow_on_submit` fields can change).

### 4. Example Usage API

```python
async def update_regulated_document(doc_id: str, new_remarks: str, reason: str, user: UserContext):
    repo = unit_of_work.repository(RegulatedInvoice)
    doc = await repo.get(doc_id)

    # Standard save fails on compliance-enabled docs
    # await repo.save(doc) -> Raises DocumentLockedError

    # 1. Explicitly unlock
    await doc.unlock_for_edit(reason=reason, user=user)
    await repo.save(doc) # Saves state as EDITING

    # 2. Make changes
    doc.remarks = new_remarks

    # 3. Relock and finalize
    await doc.save_and_relock(user=user)
    await repo.save(doc)
```

## Drawbacks

- **Complexity**: Adds a secondary state machine and specialized database tables for tracking the edit logs.
- **UI Overhead**: Frontends must implement specific "Unlock" dialogs to capture the reason before allowing edits.

## Alternatives

### 1. Force `allow_on_submit` for everyone (Status Quo RFC-0004)
- **Pros**: Simple, uniform.
- **Cons**: Fails strict compliance audits where intentionality must be proven.

### 2. Versioning / Snapshots
- Create a full copy of the document (v1, v2) upon edit.
- **Pros**: Complete historical record.
- **Cons**: Massive data duplication for changing a single "Remarks" field. Breaks foreign key relationships (which version is the "canonical" one linked to the payment?).

## Unresolved Questions

- **Downstream Locking**: How strictly do we enforce the `EDITING` lock? Should a document in `EDITING` state be totally invisible to aggregate queries? (Recommendation: Keep it visible to queries, but prevent state-mutating actions like "Make Payment" until relocked).

## Implementation Plan

- [ ] Define `ComplianceSubmittableMixin`.
- [ ] Implement `unlock_for_edit` and `save_and_relock` methods.
- [ ] Define `DocumentEditLog` table schema.
- [ ] Update `GenericRepository.save` to check for the editing state constraint.
- [ ] Write integration tests for the unlock -> edit -> relock flow.


Document: docs/rfcs/rfc-0002-doctype-discovery.md

# RFC-0002: DocType Discovery & Scanning

- **Status**: Implemented
- **Author**: @revant.one
- **Created**: 2026-01-14
- **Updated**: 2026-02-22
- **Package**: `framework-m-core`, `framework-m-studio`

## Summary

Define how Studio discovers and scans DocTypes from the filesystem. Must be deterministic, fast (1000s of DocTypes), and cache-free.

## Requirements

| Requirement | Rationale |
|-------------|-----------|
| **Deterministic** | Same filesystem state → same output order. Critical for diffs, testing, CI. |
| **Fast** | `<1s` for 1000 DocTypes. Developers expect instant feedback. |
| **No caching** | Fresh scan every time. Avoid stale data, cache invalidation bugs. |
| **Recursive** | Scan all nested directories under workspace root. |

## Discovery Algorithm

### Step 1: Find Python Files (Fast Filesystem Scan)

```python
# Use pathlib with glob - fast, stdlib, no external deps
from pathlib import Path

def find_python_files(root: Path) -> list[Path]:
    """Find all .py files, sorted for determinism."""
    files = list(root.rglob("*.py"))

    # Exclude common non-doctype directories
    exclude_patterns = {
        "__pycache__", ".git", ".venv", "venv",
        "node_modules", ".pytest_cache", "dist", "build"
    }

    files = [
        f for f in files
        if not any(p in f.parts for p in exclude_patterns)
    ]

    # Sort for determinism
    return sorted(files)
```

### Step 2: Fast DocType Detection (AST, not Import)

**Critical**: Do NOT import modules to detect DocTypes. Use AST scanning.

```python
import ast
from dataclasses import dataclass

@dataclass
class DocTypeInfo:
    name: str
    file_path: Path
    line_number: int
    bases: list[str]

def scan_file_for_doctypes(file_path: Path) -> list[DocTypeInfo]:
    """Fast AST scan - no imports, no side effects."""
    try:
        source = file_path.read_text(encoding="utf-8")
        tree = ast.parse(source, filename=str(file_path))
    except (SyntaxError, UnicodeDecodeError):
        return []  # Skip invalid files

    doctypes = []

    for node in ast.walk(tree):
        if isinstance(node, ast.ClassDef):
            # Check if class inherits from DocType/BaseDocType
            base_names = [get_base_name(b) for b in node.bases]

            if is_doctype_class(base_names):
                doctypes.append(DocTypeInfo(
                    name=node.name,
                    file_path=file_path,
                    line_number=node.lineno,
                    bases=base_names,
                ))

    return doctypes

def get_base_name(base: ast.expr) -> str:
    """Extract base class name from AST node."""
    if isinstance(base, ast.Name):
        return base.id
    elif isinstance(base, ast.Attribute):
        return base.attr
    elif isinstance(base, ast.Subscript):
        # Handle Generic[T] style
        return get_base_name(base.value)
    return ""

def is_doctype_class(bases: list[str]) -> bool:
    """Check if any base indicates a DocType."""
    doctype_bases = {"DocType", "BaseDocType", "BaseChildDocType"}
    return bool(set(bases) & doctype_bases)
```

### Step 3: Parallel Scanning (Optional, for Large Codebases)

```python
import asyncio
from concurrent.futures import ProcessPoolExecutor

async def scan_doctypes_parallel(
    root: Path,
    max_workers: int = 4
) -> list[DocTypeInfo]:
    """Parallel scan for large codebases."""
    files = find_python_files(root)

    # For small counts, sequential is faster (no process overhead)
    if len(files) < 100:
        return scan_doctypes_sequential(files)

    # Parallel for large codebases
    loop = asyncio.get_event_loop()
    with ProcessPoolExecutor(max_workers=max_workers) as executor:
        results = await loop.run_in_executor(
            executor,
            lambda: [scan_file_for_doctypes(f) for f in files]
        )

    # Flatten and sort for determinism
    doctypes = [dt for file_results in results for dt in file_results]
    return sorted(doctypes, key=lambda d: (str(d.file_path), d.name))
```

## Determinism Guarantees

| Aspect | Strategy |
|--------|----------|
| File order | `sorted(files)` - alphabetical by full path |
| DocType order | `sorted(doctypes, key=(file_path, name))` |
| Output format | Dataclass with defined field order |
| Filesystem race | Accept: if file changes mid-scan, result reflects point-in-time |

## Performance Targets

| Metric | Target | Strategy |
|--------|--------|----------|
| 100 DocTypes | `<100ms` | Sequential scan |
| 1000 DocTypes | `<500ms` | Sequential with fast glob |
| 5000 DocTypes | `<1s` | Parallel with ProcessPoolExecutor |

## Why No Caching?

| Problem with Caching | Consequence |
|---------------------|-------------|
| Cache invalidation | Developer edits file, sees stale DocType |
| Filesystem watching | Complex, platform-specific, race conditions |
| Cache persistence | Where to store? Disk? Memory? Per-workspace? |
| Determinism | Cache hit vs miss could change output order |

**Solution**: Just be fast enough that caching isn't needed.

## API Surface

```python
# Public API in framework-m-studio

class DocTypeScanner:
    """Scan workspace for DocType definitions."""

    def __init__(self, workspace_root: Path):
        self.root = workspace_root

    def scan(self) -> list[DocTypeInfo]:
        """
        Synchronous scan of all DocTypes.

        Returns:
            Sorted list of DocTypeInfo (deterministic order).
        """
        files = find_python_files(self.root)
        doctypes = []
        for f in files:
            doctypes.extend(scan_file_for_doctypes(f))
        return sorted(doctypes, key=lambda d: (str(d.file_path), d.name))

    async def scan_async(self) -> list[DocTypeInfo]:
        """
        Async parallel scan for large workspaces.
        """
        return await scan_doctypes_parallel(self.root)

    def scan_file(self, file_path: Path) -> list[DocTypeInfo]:
        """
        Scan single file. For incremental use cases.
        """
        return scan_file_for_doctypes(file_path)
```

## CLI Integration

```bash
# List all DocTypes (uses scanner)
$ m studio:list-doctypes

Name                 File                                    Line
─────────────────────────────────────────────────────────────────
Customer             apps/crm/doctypes/customer.py           12
Invoice              apps/accounting/doctypes/invoice.py     8
InvoiceItem          apps/accounting/doctypes/invoice.py     45
...

Found 1,247 DocTypes in 0.42s

# JSON output for scripting
$ m studio:list-doctypes --json
[
  {"name": "Customer", "file": "apps/crm/doctypes/customer.py", "line": 12},
  ...
]
```

## Studio API Endpoint

```python
# GET /studio/api/doctypes
@get("/studio/api/doctypes")
async def list_doctypes(workspace: WorkspaceContext) -> list[DocTypeInfo]:
    """List all DocTypes in workspace (fresh scan, no cache)."""
    scanner = DocTypeScanner(workspace.root)
    return await scanner.scan_async()
```

## Edge Cases

| Case | Handling |
|------|----------|
| Syntax error in .py file | Skip file, log warning |
| Non-UTF8 file | Skip file |
| Circular imports | N/A - we use AST, not import |
| Symlinks | Follow by default (pathlib behavior) |
| Permission denied | Skip file, log warning |
| Empty workspace | Return empty list |

## What Scanner Does NOT Do

- ❌ Import Python modules
- ❌ Execute any code
- ❌ Validate DocType correctness
- ❌ Cache results
- ❌ Watch for changes

Validation and detailed parsing happen in a separate phase (LibCST parser).

## Testing

```python
def test_scanner_determinism():
    """Same filesystem → same output."""
    scanner = DocTypeScanner(test_workspace)

    result1 = scanner.scan()
    result2 = scanner.scan()

    assert result1 == result2

def test_scanner_performance():
    """1000 files in `<500ms`."""
    scanner = DocTypeScanner(large_workspace)

    start = time.perf_counter()
    result = scanner.scan()
    elapsed = time.perf_counter() - start

    assert elapsed < 0.5
    assert len(result) > 1000
```

## References

- Phase 07: Studio
- [Python AST module](https://docs.python.org/3/library/ast.html)
- [pathlib.rglob](https://docs.python.org/3/library/pathlib.html#pathlib.Path.rglob)


Document: docs/rfcs/rfc-0005-after-naming-hook.md

# RFC-0005: After Naming Hook

- **Status**: Implemented
- **Author**: @revant.one
- **Created**: 2026-02-13
- **Updated**: 2026-02-21
- **TSC Decision**: Accepted

## Summary

This RFC proposes adding a specific lifecycle hook `after_naming` (or `on_name_assigned`) to the `BaseController`. This hook is triggered *after* the document has received its final, human-readable name, but before the transaction is fully committed (or immediately after, depending on the implementation flexibility). This is crucial for the "Optimistic Naming" strategy where the document is first inserted with a UUID and then named in a subsequent step.

## Motivation

In Phase 08, we adopted an **Optimistic Naming Strategy** to avoid row-level locking on a central `Series` table.
1.  **Insert**: Document inserted with UUID `id` (Primary Key).
2.  **Naming Service**: Calculates the next name (e.g., "INV-2024-001").
3.  **Update**: Updates the document row with `name="INV-2024-001"`.

**The Problem:**
Standard hooks like `after_insert` run *before* step 3 in this flow (or immediately after step 1).
- If a developer puts code in `after_insert` to send an email ("Order Created"), the email might say "Order #None Created" or "Order #[UUID] Created".
- We need a hook that guarantees `doc.name` is set and final.

**Why `after_naming` and not reuse `after_save`?**
- `after_save` runs on *every* update. Naming happens only once.
- We want a specific event for "Identity Assigned".

## Detailed Design

### 1. The Hook Definition

Added to `BaseController`:

```python
class BaseController(Generic[T]):
    async def after_naming(self, context: Any = None) -> None:
        """
        Called after the human-readable 'name' has been generated and assigned
        to the document.
        Guarantees that self.doc.name is set.
        """
        pass
```

### 2. Execution Flow in `GenericRepository`

```python
async def save(self, entity):
    # ... existing insert logic ...

    # 1. Insert with UUID
    await session.execute(insert_stmt)

    # ...

    # 2. If Naming Strategy is enabled
    if self.has_naming_strategy:
        new_name = await self.naming_service.generate_name(entity)
        entity.name = new_name
        await self.update_name(entity.id, new_name)

        # 3. Trigger Hook
        if controller:
             await controller.after_naming()
```

### 3. Usage Example

```python
class InvoiceController(BaseController[Invoice]):
    async def after_naming(self):
        # Safe to use self.doc.name here
        await self.email_service.send(
            subject=f"Invoice {self.doc.name} Created",
            body="..."
        )
```

## Drawbacks

- **Another Hook**: Increases the surface area of the API.
- **Timing Complexity**: Developers might be confused whether to use `after_insert` or `after_naming`. (Guidance: `after_insert` for DB ID existence, `after_naming` for Human ID existence).

## Alternatives

### 1. `after_name` (Property Setter)
- Frappe uses `autoname` as a hook *before* saving to set the name.
- Since we are doing *Optimistic* (post-insert) naming, `autoname` implies "calculate it now", whereas `after_naming` implies "it has been calculated and saved".
- `after_name` sounds like a property setter callback. `after_naming` sounds like a lifecycle event.

### 2. Use `after_save` with a check
```python
async def after_save(self):
    if self.doc.name and not self._previous_doc.name:
         # logic
```
- **Cons**: Verbose and error-prone. Requires access to previous state which might not be easily available in all contexts.

## Unresolved Questions

- **Transaction Context**: Should `after_naming` run in the same transaction as the name update? (Yes, generally, so if the hook fails, the name assignment rolls back).

## Implementation Plan

- [ ] Add `after_naming` to `BaseController` protocol.
- [ ] Update `GenericRepository` to call this hook after the naming service returns.
- [ ] Document the lifecycle order clearly: `before_insert` -> `insert` -> `after_insert` -> `naming` -> `after_naming` -> `commit`.


Document: docs/rfcs/rfc-0001-frontend-init.md

# RFC-0001: Frontend Initialization and Development Experience

- **Status**: Proposed
- **Date**: 2026-01-29
- **Author**: @anshpansuriya14
- **Related**: [ADR-0005: Use Refine for Frontend](../adr/0005-refine-for-frontend.md)

## Summary

Define how third-party developers use Framework M with the Desk frontend without cloning the monorepo. Provide a "batteries included" experience similar to Frappe's `bench start`.

## Problem Statement

**Current State:**

- Frontend (`/frontend`) exists only in the monorepo
- Developers must clone the repo to get the Desk UI
- No single command starts both backend and frontend

**Desired State:**

```bash
pip install framework-m
m new:project crm
cd crm
m prod                     # Works immediately with bundled Desk
m prod --with-frontend     # For frontend customization
```

---

## Proposed Solution

### Option A: Bundled Static + npm Package (Recommended ✓)

```
┌─────────────────────────────────────────────────────────────────┐
│                     Developer Experience                         │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  pip install framework-m                                         │
│                                                                  │
│  ┌──────────────────────────┐    ┌──────────────────────────┐   │
│  │     m prod               │    │ m prod --with-frontend   │   │
│  │                          │    │                          │   │
│  │  Backend + Bundled Desk  │    │  Backend + Vite Dev      │   │
│  │  (Static files in pkg)   │    │  (Uses @framework-m/desk)│   │
│  │                          │    │                          │   │
│  │  For: API users,         │    │  For: Frontend devs,     │   │
│  │       Quick start        │    │       Customizers        │   │
│  └──────────────────────────┘    └──────────────────────────┘   │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘
```

**Components:**

| Component         | Location                          | Purpose                              | Registry       |
| ----------------- | --------------------------------- | ------------------------------------ | -------------- |
| Bundled Desk      | `framework_m/static/`             | Pre-built React app in PyPI package  | PyPI           |
| Frontend Template | `framework_m/templates/frontend/` | Scaffold source for customization    | PyPI (bundled) |
| npm Package       | `@framework-m/desk`               | Reusable Refine providers/components | GitLab npm     |

**GitLab npm Registry Benefits:**

- Private by default (uses CI_JOB_TOKEN)
- No extra infrastructure or auth setup
- Same monorepo hosts both packages
- Can switch to public npmjs.com later via CLI

### Option B: npm Package Only

- No bundled static, always require `--with-frontend`
- Simpler PyPI package, but worse DX for non-frontend developers

### Option C: Docker-Only Distribution

- Ship Desk as a Docker container
- Good for production, bad for development

---

## Recommended: Option A Implementation

### Phase 1: `m prod` with Bundled Desk

**Pre-build Desk UI during release:**

```yaml
# .gitlab-ci.yml
build:frontend:
  script:
    - cd frontend && pnpm install && pnpm build
    - cp -r dist libs/framework-m/src/framework_m/static/
```

**Backend serves static:**

```python
# libs/framework-m-standard/src/adapters/web/app.py
from importlib.resources import files

STATIC_DIR = files("framework_m") / "static"

app.mount("/", StaticFilesMiddleware(directory=STATIC_DIR))
```

**CLI command:**

```python
# libs/framework-m/src/framework_m/cli/serve.py
def start_command(
    port: int = 8000,
    reload: bool = False,
    with_frontend: bool = False,
):
    """Start Framework M application."""
    if with_frontend:
        _start_with_frontend(port, reload)
    else:
        _start_backend_only(port, reload)
```

---

### Phase 2: `m prod --with-frontend`

**Auto-scaffold frontend if not exists:**

```python
def _start_with_frontend(port: int, reload: bool):
    frontend_dir = Path.cwd() / "frontend"

    # Auto-scaffold if not exists
    if not frontend_dir.exists():
        _scaffold_frontend(frontend_dir)

    # Start both processes
    backend = _start_backend(port, reload)
    frontend = _start_frontend_dev(frontend_dir)

    # Wait for both
    try:
        backend.wait()
        frontend.wait()
    except KeyboardInterrupt:
        backend.terminate()
        frontend.terminate()
```

**Scaffold from template:**

```python
def _scaffold_frontend(target: Path):
    """Copy frontend template from package."""
    from importlib.resources import files, as_file

    template = files("framework_m") / "templates/frontend"
    with as_file(template) as template_dir:
        shutil.copytree(template_dir, target)

    print(f"✓ Scaffolded frontend to {target}")
    print("  Installing dependencies...")

    subprocess.run(["pnpm", "install"], cwd=target, check=True)
```

---

### Phase 3: npm Package `@framework-m/desk`

Publish reusable components to GitLab npm registry:

```typescript
// @framework-m/desk/src/index.ts
export { frameworkMDataProvider } from "./providers/data";
export { authProvider } from "./providers/auth";
export { liveProvider } from "./providers/live";
export { DocTypeList, DocTypeForm, DocTypeShow } from "./components";
export { useDocType, useMetadata, usePermissions } from "./hooks";
```

**Template consumes npm package:**

```json
// templates/frontend/package.json
{
  "dependencies": {
    "@framework-m/desk": "^0.1.0",
    "@refinedev/core": "^5.0.0",
    "react": "^19.0.0"
  }
}
```

**GitLab npm Registry Setup:**

```bash
# .npmrc in scaffolded frontend/
@framework-m:registry=https://gitlab.com/api/v4/projects/${CI_PROJECT_ID}/packages/npm/

# CI publishes with CI_JOB_TOKEN
# Users can optionally configure personal token for local dev
# Or switch to public npmjs.com later
```

---

## Command Reference

| Command                  | Description                         |
| ------------------------ | ----------------------------------- |
| `m prod`                 | Start backend + serve bundled Desk  |
| `m prod --port 8000`     | Custom port                         |
| `m prod --reload`        | Dev mode with auto-reload           |
| `m prod --with-frontend` | Start backend + frontend dev server |
| `m init:frontend`        | Scaffold frontend only (no start)   |
| `m build`                | Build frontend for production       |

---

## Implementation Checklist

### Phase 1: Bundled Desk (Required for v1)

- [ ] CI job to build frontend during release
- [ ] Include `static/` in `pyproject.toml` package data
- [ ] Backend serves static files at `/`
- [ ] `m prod` command in framework-m CLI

### Phase 2: Frontend Scaffold (Required for customizers)

- [ ] Create `templates/frontend/` in package
- [ ] `m prod --with-frontend` command
- [ ] `m init:frontend` command
- [ ] Process manager for backend + frontend

### Phase 3: npm Package (Required for ecosystem)

- [ ] Extract Refine providers to `@framework-m/desk`
- [ ] Publish to npm public registry
- [ ] Template consumes npm package
- [ ] Documentation for frontend customization

---

## Alternatives Considered

| Option                                | Pros                                                  | Cons                                        |
| ------------------------------------- | ----------------------------------------------------- | ------------------------------------------- |
| **A: Bundled + npm Package** (Chosen) | Best DX, reusable components, GitLab private registry | PyPI package ~10MB, users need GitLab token |
| **B: npm Package Only**               | Smaller PyPI package                                  | Poor DX for non-frontend devs               |
| **C: Docker Only**                    | Clean separation                                      | Bad for development                         |
| **D: Clone Monorepo**                 | Current state                                         | Terrible DX                                 |

---

## Migration Path

**For existing users:**

```bash
# Before (current)
git clone https://gitlab.com/castlecraft/m.git
cd m/frontend && pnpm dev

# After (proposed)
pip install framework-m
m new:project myapp && cd myapp
m prod                     # Just works
m prod --with-frontend     # If customizing
```

---

## Files to Create/Modify

| File                                                   | Action  | Description                           |
| ------------------------------------------------------ | ------- | ------------------------------------- |
| `libs/framework-m/src/framework_m/cli/serve.py`        | **NEW** | `m prod` command                      |
| `libs/framework-m/src/framework_m/cli/init.py`         | **NEW** | `m init:frontend` command             |
| `libs/framework-m/src/framework_m/static/`             | **NEW** | Pre-built Desk (from CI)              |
| `libs/framework-m/src/framework_m/templates/frontend/` | **NEW** | Frontend scaffold template            |
| `libs/framework-m-desk/`                               | **NEW** | npm package for Refine providers      |
| `libs/framework-m/pyproject.toml`                      | MODIFY  | Include static + templates            |
| `.gitlab-ci.yml`                                       | MODIFY  | Add frontend + npm package build jobs |

---

## Questions for Review

1. **Package Size**: Is ~10MB increase in PyPI package acceptable for bundled Desk?
2. **npm Scope**: Should we use `@framework-m/desk` or `@castlecraft/framework-m-desk`?
3. **Template Updates**: How do users get template updates after scaffolding?

---

## References

- [ADR-0005: Use Refine for Frontend](../adr/0005-refine-for-frontend.md)
- [Frappe Bench Architecture](https://frappeframework.com/docs/user/en/bench)
- [Refine CLI Documentation](https://refine.dev/docs/packages/cli/)


Document: docs/rfcs/rfc-0007-sidebar-architecture.md

# RFC-0007: ERP Sidebar Architecture

> **Related**: [ADR-0008: Frontend Plugin Architecture](../adr/0008-frontend-plugin-architecture.md) - Defines plugin menu contract and PluginRegistry integration

## Current State Analysis

### Existing Implementation (`frontend/src/layout/Sidebar.tsx`)

| Feature               | Status | Notes                             |
| --------------------- | ------ | --------------------------------- |
| Groups by module      | ✅     | Parses `module.DocType` from name |
| Collapsible sections  | ✅     | Toggle per group                  |
| Search/filter         | ✅     | Filters by label                  |
| Plugin slots          | ✅     | `sidebar-top`, `sidebar-bottom`   |
| Collapse toggle       | ✅     | 60px vs 240px width               |
| Active item highlight | ✅     | Border + background               |

### Current Limitations

1. **Flat grouping** - Only groups by first segment (e.g., "core.User" → "Core")
2. **No nested hierarchy** - Can't have multi-level menus (Module → Category → DocType)
3. **No icons** - Text only, no visual differentiation
4. **No pinned/favorites** - Can't pin frequently used items
5. **No recent items** - No quick access to recently viewed
6. **Static structure** - Menu comes only from DocTypes, no custom links

---

## Proposed Architecture

### Sidebar Navigation Hierarchy

The sidebar displays a hierarchical menu structure with user-specific enhancements:

```
┌─────────────────────────────────────┐
│ 🏠 Home                             │ ← Static entry
├─────────────────────────────────────┤
│ ⭐ Favorites                        │ ← User-specific
│   └── Sales Invoice                 │
│   └── Customer                      │
├─────────────────────────────────────┤
│ 🕐 Recent                           │ ← Last 5-10 visited
│   └── INV-00123                     │
│   └── CUST-00045                    │
├─────────────────────────────────────┤
│ 📦 Sales                            │ ← Module (from plugins)
│   ├── Transactions                  │ ← Category
│   │   ├── Sales Invoice             │
│   │   ├── Sales Order               │
│   │   └── Quotation                 │
│   └── Masters                       │
│       ├── Customer                  │
│       └── Price List                │
├─────────────────────────────────────┤
│ 🏭 Inventory                        │
│   ├── Stock Entry                   │
│   └── Item                          │
├─────────────────────────────────────┤
│ ⚙ Settings                         │ ← Always at bottom
│   └── User                          │
│   └── Company                       │
└─────────────────────────────────────┘
```

> **Note**: Module and category structure comes from `PluginRegistry.getMenu()` (see ADR-0008). Sidebar adds Favorites and Recent sections on top.

### Menu Source Priority

1. **Backend metadata** - DocTypes with `show_in_menu: true`
2. **Plugin contributions** - See [ADR-0008 Plugin Menu Contract](../adr/0008-frontend-plugin-architecture.md#plugin-manifest-contract)
3. **User preferences** - Favorites, recent, collapsed state (sidebar-specific)
4. **Static entries** - Home, Settings (always present)

> **Note**: Menu items from plugins are registered via `PluginRegistry.getMenu()` as defined in ADR-0008. The sidebar consumes this merged menu tree and adds user-specific features (favorites, recent, search).

---

## Sidebar-Specific Features

These features extend the base menu system defined in ADR-0008:

### Favorites

- User can pin frequently used items
- Stored in localStorage or user preferences API
- Appears as top-level "⭐ Favorites" section
- Right-click or star icon to pin/unpin

### Recent Items

- Track last 10 viewed DocTypes/records
- Stored in localStorage
- Appears as "🕐 Recent" section below Favorites
- Auto-updates on navigation

### Search & Filter

- Fuzzy search across all menu items
- Shows matched path (Sales > Invoice > ...)
- Highlights matches
- Filters entire tree, preserving hierarchy

### Responsive Behavior

- **Mobile**: Slide-out drawer (overlay)
- **Tablet**: Collapsible sidebar
- **Desktop**: Always visible, collapsible to 60px icons-only

---

## Data Model Extensions

The sidebar extends the base `MenuItem` interface from ADR-0008:

```typescript
// Base MenuItem from ADR-0008 plugin contract
import { MenuItem } from "@framework-m/plugin-sdk";

// Sidebar-specific extensions
interface SidebarMenuItem extends MenuItem {
  // Runtime state (not persisted)
  isExpanded?: boolean;
  isFavorite?: boolean;
  lastVisited?: Date;

  // UI-specific
  depth?: number; // Nesting level for indentation
  parentId?: string; // For breadcrumb/path display
}

// User preferences
interface SidebarPreferences {
  favorites: string[]; // MenuItem IDs
  recent: Array<{
    id: string;
    timestamp: Date;
  }>;
  collapsed: string[]; // Collapsed section IDs
  customOrder?: Record<string, number>; // Custom sort order
}
```

---

## Integration with PluginRegistry

The sidebar integrates with the plugin system defined in ADR-0008:

```typescript
// Sidebar.tsx
import { usePluginMenu } from '@framework-m/plugin-sdk';

function Sidebar() {
  // Get merged menu from all registered plugins
  const pluginMenu = usePluginMenu();

  // Load user preferences
  const [preferences, setPreferences] = useSidebarPreferences();

  // Enhance with sidebar-specific features
  const menuTree = useMemo(() => {
    return enhanceMenu(pluginMenu, preferences);
  }, [pluginMenu, preferences]);

  return (
    <nav>
      <FavoritesSection items={menuTree.favorites} />
      <RecentSection items={menuTree.recent} />
      <MenuTree items={menuTree.modules} />
    </nav>
  );
}

function enhanceMenu(
  pluginMenu: MenuItem[],
  preferences: SidebarPreferences
): EnhancedMenuTree {
  // Add favorites section
  // Add recent section
  // Apply custom ordering
  // Set expanded states
  // Calculate depths for indentation
}
```

---

## Scalability Patterns

### Pattern 1: Lazy Loading Large Modules

```typescript
// Load children on expand (for modules with 100+ items)
async function loadMenuChildren(moduleId: string) {
  const response = await fetch(`/api/meta/menu/${moduleId}`);
  return response.json();
}
```

### Pattern 2: Virtual Scrolling

For 500+ menu items, use virtual scrolling:

```typescript
import { FixedSizeList } from 'react-window';

<FixedSizeList height={400} itemCount={items.length} itemSize={36}>
  {({ index, style }) => <MenuItem item={items[index]} style={style} />}
</FixedSizeList>
```

---

## Implementation Checklist

### Phase 1: Core Improvements ✅ COMPLETE

- [x] **Add icons support** ✅
  - ✅ `Icon` component implemented (`frontend/src/components/Icon.tsx`)
  - ✅ `getModuleIcon()` function in `frontend/src/types/menu.ts`
  - ✅ Icons used throughout `Sidebar.tsx`

- [x] **Multi-level nesting** ✅
  - ✅ `MenuItem` interface supports `children` array
  - ✅ Backend API returns nested structure
  - ✅ Frontend groups by module (flat, not recursive yet)

- [x] **Improve grouping logic** ✅
  - ✅ Backend: Parses `doctype_name.split(".")` for module extraction
  - ✅ Module icons mapped in both backend and frontend
  - ✅ DocTypes grouped by module in sidebar

### Phase 2: User Experience ✅ COMPLETE

- [x] **Favorites section** ✅
  - ✅ `useFavorites` hook (`frontend/src/hooks/useFavorites.ts`)
  - ✅ localStorage persistence with `addFavorite`, `removeFavorite`, `toggleFavorite`
  - ✅ Star icon on each menu item for toggling
  - ✅ Collapsible "⭐ Favorites" section at top of sidebar
  - ✅ Drag-to-reorder within favorites (via `useSortable`)

- [x] **Recent items** ✅
  - ✅ `useRecent` hook (`frontend/src/hooks/useRecent.ts`)
  - ✅ localStorage persistence with `addRecentItem`, max 10 items
  - ✅ Tracks both DocType views and specific records
  - ✅ Collapsible "🕐 Recent" section in sidebar
  - ✅ Wired to FormView and ListView with automatic tracking

- [x] **Keyboard navigation** ✅
  - ✅ Arrow key navigation implemented in `Sidebar.tsx`
  - ✅ Focus state tracked with `focusedIndex`
  - ✅ Enter key to navigate to focused item
  - ✅ Escape key to clear focus
  - ✅ Visual focus indicator (2px outline) on all menu items
  - ✅ Focus wrapping at list boundaries

### Phase 3: Scalability ✅ MOSTLY COMPLETE

- [x] **Backend menu API** ✅
  - ✅ Endpoint: `GET /api/meta/menu` (`menu_routes.py`)
  - ✅ Returns structured menu tree with modules and children
  - ✅ Module icons, ordering, and metadata
  - ✅ Filters by `api_resource=True`
  - ⚠ **TODO**: Wire up router in `app.py` route handlers (currently NOT registered)

- [x] **Plugin menu integration** ❌ NOT STARTED
  - ❌ No `@framework-m/plugin-sdk` package exists yet (see ADR-0008)
  - ❌ No `PluginRegistry.getMenu()` implementation
  - ❌ No plugin menu merging logic
  - ✅ Hook placeholder exists: `useMenuData` can be extended
  - 📝 **Status**: Blocked by ADR-0008 implementation

- [ ] **Badge counts** ⏸ DEFERRED
  - ⏸ Not started (as planned)
  - 📝 Future enhancement

### Phase 4: Polish ✅ COMPLETE

- [x] **Drag-to-reorder** ✅
  - ✅ `useSortable` hook (`frontend/src/hooks/useSortable.ts`)
  - ✅ localStorage persistence of custom order
  - ✅ Drag handlers with visual feedback
  - ✅ Applied to favorites list
  - ⏸ Module groups reordering deferred (optional enhancement)

- [x] **Search improvements** ✅
  - ✅ `fuzzySearch.ts` utility with scoring and highlighting
  - ✅ `fuzzyMatch()` and `getHighlightedSegments()` functions
  - ✅ Filters entire menu tree
  - ✅ Visual highlighting of matched text with background color
  - ✅ Breadcrumb display showing module path for search results

- [x] **Responsive behavior** ✅
  - ✅ Collapse toggle button implemented (60px ↔ 240px)
  - ✅ Icons-only mode when collapsed
  - ✅ Mobile slide-out drawer with overlay
  - ✅ Backdrop overlay that closes on click
  - ✅ Touch gestures for swipe-to-close
  - ✅ Mobile menu button in Navbar
  - ✅ Media queries for responsive behavior

---

## Implementation Status Summary

### ✅ Fully Complete (80%)

1. Icons system with module mapping
2. Favorites with localStorage + drag-to-reorder
3. Recent items tracking
4. Fuzzy search algorithm
5. Backend menu API structure
6. Type definitions and data models
7. Core sidebar layout and grouping
8. **Keyboard navigation** - All features complete (Arrow keys, Enter, Escape, visual focus, wrapping)

### ⚠ Partially Complete (12%)

1. **Search highlighting** - Scoring works, needs visual highlights in UI
2. **Responsive** - Desktop works, needs mobile drawer
3. **Backend integration** - API exists but not wired to app router

### ❌ Not Started (5%)

1. **Plugin menu integration** - Blocked by ADR-0008 (plugin SDK doesn't exist yet)
2. **Badge counts** - Intentionally deferred

---

## Critical Issues to Fix

### 🔴 HIGH PRIORITY

1. **Menu API not registered in app** 🔴
   - **File**: `libs/framework-m-standard/src/framework_m_standard/adapters/web/app.py`
   - **Issue**: `menu_routes_router` created but never added to `route_handlers`
   - **Fix**: Add import and register router

   ```python
   from framework_m_standard.adapters.web.menu_routes import menu_routes_router

   route_handlers: list[Any] = [
       # ... existing handlers
       menu_routes_router,  # ADD THIS
   ]
   ```

2. **Recent items not tracked on navigation** 🟡
   - **Files**: Need to hook into navigation events
   - **Issue**: `useRecent` hook exists but `addRecentItem()` never called
   - **Fix**: Add effect in `FormView.tsx` and `ListView.tsx`:

   ```tsx
   const { addRecentItem } = useRecent();

   useEffect(() => {
     if (doctype) {
       addRecentItem({
         name: doctype,
         label: doctype,
         route: location.pathname,
       });
     }
   }, [doctype, location.pathname]);
   ```

### 🟡 MEDIUM PRIORITY

3. **Search highlighting not visible** 🟡
   - **Issue**: `fuzzyMatch()` returns highlights but UI doesn't show them
   - **Fix**: Use `getHighlightedSegments()` in menu item rendering

4. **Mobile responsive drawer** 🟡
   - **Issue**: Sidebar just shrinks on mobile, no overlay drawer
   - **Fix**: Add media queries + overlay backdrop

5. **Plugin integration blocked** 🔵
   - **Blocked by**: ADR-0008 implementation (plugin SDK doesn't exist)
   - **Status**: Cannot proceed until `@framework-m/plugin-sdk` package is created

---

## Files Inventory

### ✅ Implemented and Working

| File                                 | Purpose                 | Status                          |
| ------------------------------------ | ----------------------- | ------------------------------- |
| `frontend/src/layout/Sidebar.tsx`    | Main sidebar component  | ✅ Core complete                |
| `frontend/src/hooks/useFavorites.ts` | Favorites management    | ✅ Fully functional             |
| `frontend/src/hooks/useRecent.ts`    | Recent items tracking   | ✅ Hook ready, needs wiring     |
| `frontend/src/hooks/useSortable.ts`  | Drag-to-reorder         | ✅ Fully functional             |
| `frontend/src/hooks/useMenuData.ts`  | Backend menu API client | ✅ Ready (API not wired)        |
| `frontend/src/utils/fuzzySearch.ts`  | Search algorithm        | ✅ Fully functional             |
| `frontend/src/types/menu.ts`         | TypeScript types        | ✅ Complete                     |
| `frontend/src/components/Icon.tsx`   | Icon component          | ✅ Fully functional             |
| `libs/.../menu_routes.py`            | Backend menu API        | ✅ Implemented (not registered) |

### ⚠ Needs Attention

| File                              | Issue                               | Priority  |
| --------------------------------- | ----------------------------------- | --------- |
| `frontend/src/layout/Sidebar.tsx` | Add Enter key handler, focus styles | 🟡 Medium |
| `frontend/src/layout/Sidebar.tsx` | Mobile drawer + overlay             | 🟡 Medium |
| `frontend/src/layout/Sidebar.tsx` | Render search highlights            | 🟡 Medium |
| `frontend/src/pages/FormView.tsx` | Call `addRecentItem()` on load      | 🟡 Medium |
| `frontend/src/pages/ListView.tsx` | Call `addRecentItem()` on load      | 🟡 Medium |
| `.../adapters/web/app.py`         | Register `menu_routes_router`       | 🔴 High   |

### ❌ Missing (Per RFC Spec)

| File                                              | Purpose          | Status                                    |
| ------------------------------------------------- | ---------------- | ----------------------------------------- |
| ~~`frontend/src/hooks/useSidebarPreferences.ts`~~ | User preferences | ⚠ Not needed (localStorage in each hook) |
| `@framework-m/plugin-sdk`                         | Plugin system    | ❌ Blocked by ADR-0008                    |

---

## Files to Modify

| File                                           | Changes                              |
| ---------------------------------------------- | ------------------------------------ |
| `frontend/src/layout/Sidebar.tsx`              | Main implementation                  |
| `frontend/src/hooks/useMenu.ts`                | Integrates with PluginRegistry (new) |
| `frontend/src/hooks/useFavorites.ts`           | Favorites management (new)           |
| `frontend/src/hooks/useRecent.ts`              | Recent items tracking (new)          |
| `frontend/src/hooks/useSidebarPreferences.ts`  | User preferences (new)               |
| `frontend/src/types/sidebar.ts`                | TypeScript interfaces (new)          |
| `libs/framework-m-standard/.../routes/meta.py` | Backend menu API (new endpoint)      |

> **Note**: Plugin menu registration handled by `@framework-m/plugin-sdk` (see ADR-0008)

---

## Architecture Alignment

This RFC implements the sidebar consumer of the menu system defined in ADR-0008:

```
┌─────────────────────────────────────────────────────────────┐
│ ADR-0008: Plugin Architecture                               │
│ - PluginRegistry.getMenu() → Merged menu tree              │
│ - MenuItem interface (base contract)                        │
│ - Plugin contributions via plugin.config.ts                 │
└────────────────────┬────────────────────────────────────────┘
                     │ provides menu data
                     ▼
┌─────────────────────────────────────────────────────────────┐
│ This RFC: Sidebar Implementation                           │
│ - Consumes PluginRegistry.getMenu()                         │
│ - Adds favorites, recent, search (UI features)             │
│ - Manages user preferences (collapsed, custom order)       │
│ - Renders responsive navigation                            │
└─────────────────────────────────────────────────────────────┘
```

**Separation of Concerns**:

- **ADR-0008**: Defines _what_ menus exist (data model, plugin contract)
- **This RFC**: Defines _how_ menus are displayed (UI, UX, interactions)

---

## Quick Start: Minimal Changes

For immediate improvement without breaking changes:

```typescript
// 1. Add icons to groupMenuItems()
const MODULE_ICONS: Record<string, string> = {
  Core: '⚙',
  Sales: '💰',
  Inventory: '📦',
  HR: '👥',
  Other: '📄',
};

// 2. Use icon in group header
<span>{MODULE_ICONS[groupName] || '📁'} {groupName}</span>
```

This gives visual differentiation immediately while planning the full refactor.


Document: docs/examples/crm/README.md

# CRM Example

A basic CRM (Customer Relationship Management) application with Framework M.

## DocTypes

### Contact

```python
# apps/crm/src/doctypes/contact/doctype.py
from __future__ import annotations

from typing import ClassVar

from framework_m import DocType, Field

class Contact(DocType):
    """Customer or lead contact."""

    __doctype_name__: ClassVar[str] = "Contact"

    # Name
    first_name: str = Field(description="First name")
    last_name: str = Field(description="Last name")

    # Contact info
    email: str = Field(default="", description="Email address")
    phone: str = Field(default="", description="Phone number")

    # Organization
    company: str = Field(default="", description="Company name")
    job_title: str = Field(default="", description="Job title")

    # Status
    type: str = Field(default="Lead", description="Contact type: Lead, Customer, Partner")
    status: str = Field(default="Active", description="Status")

    class Meta:
        naming_rule: ClassVar[str] = "autoincrement"
        api_resource: ClassVar[bool] = True
```

### Deal

```python
# apps/crm/src/doctypes/deal/doctype.py
from __future__ import annotations

from datetime import date
from decimal import Decimal
from typing import ClassVar
from uuid import UUID

from framework_m import DocType, Field

class Deal(DocType):
    """Sales opportunity/deal."""

    __doctype_name__: ClassVar[str] = "Deal"

    # Basic info
    title: str = Field(description="Deal title")
    contact_id: UUID = Field(description="Primary contact")

    # Value
    amount: Decimal = Field(default=Decimal("0"), description="Deal value")
    currency: str = Field(default="USD", description="Currency")

    # Pipeline
    stage: str = Field(
        default="Qualification",
        description="Pipeline stage",
    )
    probability: int = Field(default=10, ge=0, le=100, description="Win probability %")

    # Dates
    expected_close: date | None = Field(default=None, description="Expected close date")
    closed_date: date | None = Field(default=None, description="Actual close date")

    class Meta:
        naming_rule: ClassVar[str] = "autoincrement"
        api_resource: ClassVar[bool] = True
        is_submittable: ClassVar[bool] = True
```

## Controllers

### ContactController

```python
# apps/crm/src/doctypes/contact/controller.py
from __future__ import annotations

from framework_m import Controller

from .doctype import Contact

class ContactController(Controller[Contact]):
    """Contact business logic."""

    async def validate(self, context=None):
        if not self.doc.first_name.strip():
            raise ValueError("First name is required")

    async def before_save(self, context=None):
        # Normalize email
        if self.doc.email:
            self.doc.email = self.doc.email.lower().strip()
```

### DealController

```python
# apps/crm/src/doctypes/deal/controller.py
from __future__ import annotations

from datetime import date
from decimal import Decimal

from framework_m import Controller

from .doctype import Deal

class DealController(Controller[Deal]):
    """Deal business logic."""

    # Pipeline stages with probabilities
    STAGE_PROBABILITIES = {
        "Qualification": 10,
        "Proposal": 30,
        "Negotiation": 60,
        "Closed Won": 100,
        "Closed Lost": 0,
    }

    async def validate(self, context=None):
        if self.doc.amount < 0:
            raise ValueError("Amount cannot be negative")

    async def before_save(self, context=None):
        # Auto-set probability based on stage
        if self.doc.stage in self.STAGE_PROBABILITIES:
            self.doc.probability = self.STAGE_PROBABILITIES[self.doc.stage]

        # Set closed date when won/lost
        if self.doc.stage in ("Closed Won", "Closed Lost"):
            if not self.doc.closed_date:
                self.doc.closed_date = date.today()

    async def on_submit(self, context=None):
        """Called when deal is submitted (won)."""
        if self.doc.stage != "Closed Won":
            raise ValueError("Can only submit deals that are Closed Won")
```

## API Usage

```bash
# Create a contact
curl -X POST http://localhost:8000/api/v1/resource/Contact \
  -H "Content-Type: application/json" \
  -d '{
    "first_name": "John",
    "last_name": "Doe",
    "email": "john@example.com",
    "company": "Acme Inc",
    "type": "Lead"
  }'

# Create a deal linked to contact
curl -X POST http://localhost:8000/api/v1/resource/Deal \
  -H "Content-Type: application/json" \
  -d '{
    "title": "Acme Enterprise License",
    "contact_id": "contact-uuid-here",
    "amount": 50000,
    "stage": "Proposal",
    "expected_close": "2024-03-01"
  }'

# Update deal stage
curl -X PUT http://localhost:8000/api/v1/resource/Deal/1 \
  -H "Content-Type: application/json" \
  -d '{"stage": "Negotiation"}'

# List deals by stage
curl "http://localhost:8000/api/v1/resource/Deal?filters=[{\"field\":\"stage\",\"operator\":\"eq\",\"value\":\"Proposal\"}]"
```

## Quick Start

```bash
# Create the app
uv run m new:app crm

# Create DocTypes
cd crm
uv run m new:doctype Contact
uv run m new:doctype Deal

# Edit doctype.py and controller.py as shown above

# Run migrations
uv run m migrate init
uv run m migrate create "Add CRM doctypes" --autogenerate
uv run m migrate

# Start Studio
uv run m studio
```


Document: docs/examples/todo/README.md

# TODO App Example

A simple TODO application built with Framework M.

## DocType: Task

```python
# apps/todo/src/doctypes/task/doctype.py
from __future__ import annotations

from datetime import date
from typing import ClassVar

from framework_m import DocType, Field

class Task(DocType):
    """A simple task/todo item."""

    __doctype_name__: ClassVar[str] = "Task"

    # Core fields
    title: str = Field(description="Task title")
    description: str = Field(default="", description="Task description")

    # Status
    status: str = Field(
        default="Open",
        description="Current status",
    )
    priority: str = Field(
        default="Medium",
        description="Priority level",
    )

    # Dates
    due_date: date | None = Field(default=None, description="Due date")
    completed_date: date | None = Field(default=None, description="Completed date")

    class Meta:
        naming_rule: ClassVar[str] = "autoincrement"
        api_resource: ClassVar[bool] = True
```

## Controller: TaskController

```python
# apps/todo/src/doctypes/task/controller.py
from __future__ import annotations

from datetime import date

from framework_m import Controller

from .doctype import Task

class TaskController(Controller[Task]):
    """Business logic for Task."""

    async def validate(self, context=None):
        """Validate task before save."""
        if not self.doc.title.strip():
            raise ValueError("Title is required")

    async def before_save(self, context=None):
        """Auto-set completed date when status changes to Done."""
        if self.doc.status == "Done" and not self.doc.completed_date:
            self.doc.completed_date = date.today()
```

## API Usage

```bash
# List all tasks
curl http://localhost:8000/api/v1/resource/Task

# Create a task
curl -X POST http://localhost:8000/api/v1/resource/Task \
  -H "Content-Type: application/json" \
  -d '{
    "title": "Buy groceries",
    "priority": "High",
    "due_date": "2024-01-15"
  }'

# Update task status
curl -X PUT http://localhost:8000/api/v1/resource/Task/1 \
  -H "Content-Type: application/json" \
  -d '{"status": "Done"}'

# Filter by status
curl "http://localhost:8000/api/v1/resource/Task?filters=[{\"field\":\"status\",\"operator\":\"eq\",\"value\":\"Open\"}]"
```

## Quick Start

```bash
# Create the app
uv run m new:app todo

# Create the Task doctype
cd todo
uv run m new:doctype Task

# Edit doctype.py and controller.py as shown above

# Run migration
uv run m migrate init
uv run m migrate create "Add Task doctype" --autogenerate
uv run m migrate

# Start Studio
uv run m studio
```


Document: docs/developer/protocols/repositoryprotocol.md

# RepositoryProtocol

Protocol defining the contract for data repositories.

    This is the primary port for data persistence in the hexagonal architecture.
    All repository adapters must implement this interface.

    Type parameter T represents the entity type being managed.

    Features:
    - Generic CRUD operations (get, save, delete)
    - Existence and count queries
    - Paginated listing with filtering and sorting
    - Bulk operations for performance
    - Optional Optimistic Concurrency Control (OCC) via version parameter

    Example implementation:
        class SQLAlchemyRepository(RepositoryProtocol[User]):
            async def get(self, id: UUID) -> User | None:
                ...

**Source**: [repository.py](file:///builds/castlecraft/framework-m/libs/framework-m-core/src/framework_m_core/interfaces/repository.py)

## Methods

### `get`

```python
async def get(self, id: UUID) -> T | None
```

Retrieve an entity by its unique identifier.

        Args:
            id: The unique identifier of the entity

        Returns:
            The entity if found, None otherwise

### `save`

```python
async def save(self, entity: T, version: int | None = None) -> T
```

Persist an entity (insert or update).

        If the entity has an id, it will be updated. Otherwise, inserted.
        For Optimistic Concurrency Control, pass the expected version.

        Args:
            entity: The entity to save
            version: Expected version for OCC (raises VersionConflictError if mismatch)

        Returns:
            The saved entity with updated fields (id, modified, version, etc.)

        Raises:
            VersionConflictError: If version doesn't match (OCC)

### `delete`

```python
async def delete(self, id: UUID) -> None
```

Delete an entity by its unique identifier.

        Args:
            id: The unique identifier of the entity to delete

        Raises:
            EntityNotFoundError: If entity doesn't exist

### `exists`

```python
async def exists(self, id: UUID) -> bool
```

Check if an entity exists by its identifier.

        More efficient than get() when you only need existence check.

        Args:
            id: The unique identifier to check

        Returns:
            True if entity exists, False otherwise

### `count`

```python
async def count(self, filters: list[FilterSpec] | None = None) -> int
```

Count entities matching the given filters.

        Args:
            filters: Optional list of filter conditions

        Returns:
            Count of matching entities

### `list`

```python
async def list(self,
    filters: list[FilterSpec] | None = None,
    order_by: list[OrderSpec] | None = None,
    limit: int = 20,
    offset: int = 0,
) -> PaginatedResult[T]
```

List entities with filtering, sorting, and pagination.

        Args:
            filters: Optional list of filter conditions
            order_by: Optional list of sort specifications
            limit: Maximum number of items to return (default: 20)
            offset: Number of items to skip (default: 0)

        Returns:
            PaginatedResult containing items and pagination metadata

### `bulk_save`

```python
async def bulk_save(self, entities: Sequence[T]) -> Sequence[T]
```

Save multiple entities in a single operation.

        More efficient than calling save() multiple times.
        All entities are saved within a single transaction.

        Args:
            entities: List of entities to save

        Returns:
            List of saved entities with updated fields


Document: docs/developer/protocols/storageprotocol.md

# StorageProtocol

Protocol defining the contract for file storage.

    This is the primary port for file operations in the hexagonal architecture.
    Implementations handle different storage backends.

    Implementations include:
    - LocalStorageAdapter: Local filesystem storage
    - S3StorageAdapter: Amazon S3 / compatible object storage
    - GCSStorageAdapter: Google Cloud Storage

    Example usage:
        storage: StorageProtocol = container.get(StorageProtocol)

        # Save a file
        path = await storage.save_file(
            "uploads/doc.pdf",
            content,
            content_type="application/pdf"
        )

        # Get presigned URL for download
        url = await storage.get_url(path, expires=3600)

**Source**: [storage.py](file:///builds/castlecraft/framework-m/libs/framework-m-core/src/framework_m_core/interfaces/storage.py)

## Methods

### `save_file`

```python
async def save_file(self,
    path: str,
    content: bytes,
    content_type: str | None = None,
) -> str
```

Save file content to storage.

        Args:
            path: Destination path/key for the file
            content: File content as bytes
            content_type: Optional MIME type

        Returns:
            The actual path where the file was saved

### `get_file`

```python
async def get_file(self, path: str) -> bytes
```

Retrieve file content from storage.

        Args:
            path: Path/key of the file

        Returns:
            File content as bytes

        Raises:
            FileNotFoundError: If file doesn't exist

### `delete_file`

```python
async def delete_file(self, path: str) -> None
```

Delete a file from storage.

        Args:
            path: Path/key of the file to delete

        Raises:
            FileNotFoundError: If file doesn't exist

### `exists`

```python
async def exists(self, path: str) -> bool
```

Check if a file exists in storage.

        Args:
            path: Path/key to check

        Returns:
            True if file exists, False otherwise

### `list_files`

```python
async def list_files(self, prefix: str) -> list[str]
```

List files matching a prefix.

        Args:
            prefix: Path prefix to filter by (e.g., "uploads/user-001/")

        Returns:
            List of file paths matching the prefix

### `get_metadata`

```python
async def get_metadata(self, path: str) -> FileMetadata | None
```

Get metadata for a file without fetching content.

        More efficient than get_file when you only need metadata.

        Args:
            path: Path/key of the file

        Returns:
            FileMetadata if file exists, None otherwise

### `get_url`

```python
async def get_url(self, path: str, expires: int = 3600) -> str
```

Generate a presigned URL for file access.

        For S3/cloud storage, generates a time-limited URL.
        For local storage, may return a direct path or API endpoint.

        Args:
            path: Path/key of the file
            expires: URL expiration time in seconds (default: 1 hour)

        Returns:
            URL for accessing the file

### `copy`

```python
async def copy(self, src: str, dest: str) -> str
```

Copy a file within storage.

        Args:
            src: Source file path
            dest: Destination file path

        Returns:
            Path of the copied file

### `move`

```python
async def move(self, src: str, dest: str) -> str
```

Move a file within storage.

        Equivalent to copy + delete.

        Args:
            src: Source file path
            dest: Destination file path

        Returns:
            Path of the moved file


Document: docs/developer/protocols/jobqueueprotocol.md

# JobQueueProtocol

Protocol defining the contract for job queue implementations.

    This is the primary port for background task processing in the
    hexagonal architecture.

    Implementations include:
    - ArqJobQueue: Redis-backed queue using Arq
    - CeleryJobQueue: Celery distributed task queue

    Example usage:
        queue: JobQueueProtocol = container.get(JobQueueProtocol)

        # Enqueue a job
        job_id = await queue.enqueue("send_email", to="user@example.com")

        # Check status
        job = await queue.get_status(job_id)
        if job.is_complete:
            print(f"Job finished: {job.result}")

        # Schedule recurring job
        await queue.schedule("cleanup_temp", cron="0 0 * * *")

**Source**: [job_queue.py](file:///builds/castlecraft/framework-m/libs/framework-m-core/src/framework_m_core/interfaces/job_queue.py)

## Methods

### `enqueue`

```python
async def enqueue(self, job_name: str, **kwargs: Any) -> str
```

Add a job to the queue for immediate processing.

        Args:
            job_name: Name of the registered job function
            **kwargs: Arguments to pass to the job function

        Returns:
            Unique job ID for tracking

### `schedule`

```python
async def schedule(self, job_name: str, cron: str, **kwargs: Any) -> str
```

Schedule a recurring job using cron syntax.

        Args:
            job_name: Name of the registered job function
            cron: Cron expression (e.g., "0 0 * * *" for daily at midnight)
            **kwargs: Arguments to pass to the job function

        Returns:
            Schedule ID for tracking/cancellation

### `cancel`

```python
async def cancel(self, job_id: str) -> bool
```

Cancel a pending or running job.

        Args:
            job_id: ID of the job to cancel

        Returns:
            True if job was cancelled, False if already completed

### `get_status`

```python
async def get_status(self, job_id: str) -> JobInfo | None
```

Get current status and info for a job.

        Args:
            job_id: ID of the job to check

        Returns:
            JobInfo if job exists, None otherwise

### `retry`

```python
async def retry(self, job_id: str) -> str
```

Retry a failed job.

        Creates a new job with the same parameters.

        Args:
            job_id: ID of the failed job to retry

        Returns:
            New job ID for the retry

        Raises:
            ValueError: If original job not found or wasn't failed


Document: docs/developer/protocols/emailqueueprotocol.md

# EmailQueueProtocol

Protocol for email queue operations.

    This is the PORT in the ports-and-adapters pattern.
    Implementations (adapters) handle the actual queuing mechanism.

    Adapters:
    - DatabaseEmailQueueAdapter: Stores in EmailQueue DocType
    - NotificationServiceAdapter: Calls external notification service
    - InMemoryEmailQueueAdapter: For testing

    Example:
        # Get adapter from DI container
        queue: EmailQueueProtocol = container.email_queue()

        # Queue an email
        result = await queue.queue_email(EmailRequest(
            to=["user@example.com"],
            subject="Welcome!",
            body="
Hello
",
        ))

**Source**: [email_queue.py](file:///builds/castlecraft/framework-m/libs/framework-m-core/src/framework_m_core/interfaces/email_queue.py)

## Methods

### `queue_email`

```python
async def queue_email(self, request: EmailRequest) -> EmailQueueResult
```

Queue an email for sending.

        Args:
            request: Email request with recipients, subject, body, etc.

        Returns:
            EmailQueueResult with queue_id and status

        Raises:
            EmailQueueError: If queuing fails

### `get_status`

```python
async def get_status(self, queue_id: str) -> str | None
```

Get the status of a queued email.

        Args:
            queue_id: The queue ID from EmailQueueResult

        Returns:
            Status string (Queued, Sending, Sent, Failed) or None if not found

### `cancel`

```python
async def cancel(self, queue_id: str) -> bool
```

Cancel a queued email (if not yet sent).

        Args:
            queue_id: The queue ID from EmailQueueResult

        Returns:
            True if cancelled, False if already sent or not found


Document: docs/developer/protocols/permissionprotocol.md

# PermissionProtocol

Protocol defining the contract for permission checking.

    This is the primary port for authorization in the hexagonal architecture.

    Example usage:
        perm: PermissionProtocol = container.get(PermissionProtocol)

        result = await perm.evaluate(PolicyEvaluateRequest(
            principal="user-001",
            action=PermissionAction.WRITE,
            resource="Invoice",
            resource_id="inv-123",
            principal_attributes={"roles": ["Manager"], "department": "Sales"},
        ))

        if result.authorized:
            # Allow access
            pass

**Source**: [permission.py](file:///builds/castlecraft/framework-m/libs/framework-m-core/src/framework_m_core/interfaces/permission.py)

## Methods

### `evaluate`

```python
async def evaluate(self,
    request: PolicyEvaluateRequest,
) -> PolicyEvaluateResult
```

Evaluate a policy request.

        Args:
            request: Stateless request containing all context

        Returns:
            PolicyEvaluateResult with authorized flag and decision source

### `get_permitted_filters`

```python
async def get_permitted_filters(self,
    principal: str,
    principal_attributes: dict[str, Any],
    resource: str,
    tenant_id: str | None = None,
) -> dict[str, Any]
```

Get filters for Row-Level Security.

        Returns SQL filters to apply to list queries for this principal.
        For ReBAC, may query the graph engine for allowed IDs.

        Args:
            principal: User ID
            principal_attributes: User attributes (roles, groups, etc.)
            resource: DocType name
            tenant_id: Optional tenant context

        Returns:
            Filter dict to apply (e.g., {"owner": principal} or {"id": ["doc1", "doc2"]})

## Adapters

- `RbacPermissionAdapter`
- `OpaPermissionAdapter`
- `SpiceDbPermissionAdapter`
- `ComboPermissionAdapter`


Document: docs/developer/protocols/reportengineprotocol.md

# ReportEngineProtocol

Protocol defining the contract for report engine implementations.

    Report engines execute queries against various data sources
    and return structured results for display or export.

    Key concepts:
    - **Execute**: Run a query and return results
    - **Validate**: Check query syntax before execution
    - **Zero-cliff**: Same protocol for SQL, Elastic, ClickHouse

**Source**: [report_engine.py](file:///builds/castlecraft/framework-m/libs/framework-m-core/src/framework_m_core/interfaces/report_engine.py)

## Methods

### `execute`

```python
async def execute(self,
    query: str,
    parameters: dict[str, Any] | None = None,
) -> ReportResult
```

Execute a report query and return results.

        Args:
            query: The query string (SQL, Elastic DSL JSON, etc.)
            parameters: Optional query parameters for parameterized queries

        Returns:
            ReportResult containing rows and metadata

        Raises:
            QueryExecutionError: If query execution fails
            ValidationError: If query is invalid

### `validate`

```python
async def validate(self, query: str) -> bool
```

Validate query syntax without executing.

        Args:
            query: The query string to validate

        Returns:
            True if query is syntactically valid, False otherwise

## Adapters

- `SQLReportEngine`
- `ElasticReportEngine`
- `ClickHouseReportEngine`


Document: docs/developer/protocols/tenantprotocol.md

# TenantProtocol

Protocol for tenant resolution and attribute retrieval.

    Implementations must provide:
    - get_current_tenant(): Resolve tenant from request context
    - get_tenant_attributes(): Fetch tenant-specific attributes

    Two standard implementations:
    - ImplicitTenantAdapter: Single-tenant mode (Indie)
    - HeaderTenantAdapter: Multi-tenant from gateway headers (Enterprise)

    Example:
        class MyTenantAdapter:
            def get_current_tenant(self) -> str:
                return "my-tenant"

            def get_tenant_attributes(self, tenant_id: str) -> dict[str, Any]:
                return {"plan": "pro"}

**Source**: [tenant.py](file:///builds/castlecraft/framework-m/libs/framework-m-core/src/framework_m_core/interfaces/tenant.py)

## Methods

### `get_current_tenant`

```python
def get_current_tenant(self) -> str
```

Get the current tenant ID from request context.

        Returns:
            Tenant identifier string

### `get_tenant_attributes`

```python
def get_tenant_attributes(self, tenant_id: str) -> dict[str, Any]
```

Get attributes for a specific tenant.

        Args:
            tenant_id: Tenant identifier

        Returns:
            Dict of tenant attributes (plan, features, flags, etc.)


Document: docs/developer/protocols/searchprotocol.md

# SearchProtocol

Protocol defining the contract for full-text search.

    This is the primary port for search operations in the hexagonal
    architecture. Implementations handle different search engines.

    Implementations include:
    - MeilisearchAdapter: Meilisearch search engine
    - ElasticsearchAdapter: Elasticsearch/OpenSearch
    - WhooshAdapter: Pure Python search (for development)

    Example usage:
        search: SearchProtocol = container.get(SearchProtocol)

        # Index a document
        await search.index("Todo", "todo-001", {"title": "Buy milk"})

        # Search
        result = await search.search("Todo", "milk", limit=10)
        for item in result.items:
            print(item["title"])

**Source**: [search.py](file:///builds/castlecraft/framework-m/libs/framework-m-core/src/framework_m_core/interfaces/search.py)

## Methods

### `index`

```python
async def index(self,
    doctype: str,
    doc_id: str,
    doc: dict[str, Any],
) -> None
```

Index a document for searching.

        Args:
            doctype: DocType name (used as index name)
            doc_id: Document identifier
            doc: Document data to index

### `delete_index`

```python
async def delete_index(self,
    doctype: str,
    doc_id: str,
) -> None
```

Remove a document from the search index.

        Args:
            doctype: DocType name
            doc_id: Document identifier

### `search`

```python
async def search(self,
    doctype: str,
    query: str,
    *,
    filters: dict[str, Any] | None = None,
    limit: int = 20,
    offset: int = 0,
) -> SearchResult
```

Search for documents.

        Args:
            doctype: DocType name to search in
            query: Search query string
            filters: Optional field filters
            limit: Maximum results to return
            offset: Number of results to skip

        Returns:
            SearchResult with matching documents

### `reindex`

```python
async def reindex(self, doctype: str) -> int
```

Rebuild the entire index for a DocType.

        Fetches all documents from the repository and re-indexes them.

        Args:
            doctype: DocType name to reindex

        Returns:
            Number of documents indexed