Document: checklists/phase-14-studio-enhancements.md

# Phase 14: Studio Enhancements

**Objective**: Add advanced features to Studio for improved developer experience, including live preview, sandbox mode, and hot reload.

**Prerequisites**: Phase 07 (Studio core) must be complete.

---

## 1. Live Preview & Sandbox Mode

### 1.1 Sandbox Architecture

> [!IMPORTANT]
> **Sandbox = Local Development Environment**
> The Sandbox runs on the **same machine** where Studio is running. It is NOT a separate cloud environment.
> This enables citizen developers to design, test, and iterate locally before committing to Git.

**Citizen Developer Workflow:**

```
┌─────────────────────────────────────────────────────────────────────┐
│  LOCAL MACHINE (Developer or Citizen)                               │
│                                                                     │
│   1. Studio UI          2. Sandbox              3. Git              │
│   ┌──────────────┐     ┌──────────────┐       ┌──────────────┐      │
│   │ Design       │ ──► │ Test with    │ ──►   │ Commit &     │      │
│   │ DocType      │     │ mock data    │       │ Push         │      │
│   └──────────────┘     └──────────────┘       └──────────────┘      │
│                                                      │              │
└──────────────────────────────────────────────────────│──────────────┘
                                                       ▼
                                              CI/CD → Production
```

> [!WARNING]
> **Collaborative Environments: Conflict Risk**
> When multiple citizens work on the **same repo/branch**, local sandboxes can cause Git conflicts.
> The sandbox is best suited for **solo work** or **feature branches**.

**Alternatives for Collaborative Teams:**

| Approach                  | How It Works                                         | Best For          |
| ------------------------- | ---------------------------------------------------- | ----------------- |
| **Feature Branches**      | Each citizen works on their own branch → PR to merge | Small teams (2-5) |
| **Gitpod/Codespaces**     | Each citizen gets isolated cloud dev environment     | Larger teams      |
| **Assigned Modules**      | Citizens own specific modules/apps, no overlap       | Enterprise        |
| **Trunk-Based + Fast CI** | All push to main, CI deploys in `<2 min`             | Mature teams      |
| **Studio Cloud Mode**     | Hosted Studio with workspace isolation               | SaaS offering     |

### 1.2 Branch Selector

- [ ] Add branch selector to Studio UI
- [ ] Allow switching between branches
- [ ] Show current branch in header
- [ ] Warn before switching with unsaved changes

### 1.3 Preview Tab Implementation

- [ ] Add "Preview" tab to DocType editor
- [ ] Toggle between "Schema Editor" and "Sandbox"
- [ ] Preserve editor state when switching tabs

### 1.4 Mock Data Generation

- [ ] Install `@faker-js/faker` for mock data
- [ ] Create mock data generator service:
  - [ ] Generate data based on field types:
    - [ ] String → faker.lorem.words()
    - [ ] Number → faker.number.int()
    - [ ] Date → faker.date.recent()
    - [ ] Email → faker.internet.email()
    - [ ] Link → Random ID from related DocType
  - [ ] Respect field constraints (min, max, required)
  - [ ] Generate 5-100 mock records

### 1.5 Form Preview

- [ ] Render `AutoForm` component with mock document
- [ ] Show validation errors for required fields
- [ ] Allow editing mock data
- [ ] Show save/cancel buttons (non-functional)

### 1.6 List Preview

- [ ] Render `AutoTable` component with mock rows
- [ ] Show 5-10 rows by default
- [ ] Add pagination controls
- [ ] Show filters and search (non-functional)

### 1.7 In-Memory Schema Source

- [ ] Use in-memory schema (not saved file)
- [ ] Allow instant preview without saving
- [ ] Update preview on field changes
- [ ] Debounce preview updates (500ms)

---

## 2. Interactive Sandbox (CRUD Simulation)

> [!NOTE]
> **Why Mock Data?**
> Sandbox uses mock data so citizens can test the full UX (forms, lists, validation)
> without needing a database setup. Once satisfied, they commit the code and deploy
> to an environment with real data.

### 2.1 In-Memory Data Store

- [ ] Create in-memory store for mock documents
- [ ] Store per DocType (isolated)
- [ ] Reset on page reload
- [ ] Support up to 1000 mock records

### 2.2 CRUD Operations

- [ ] **Create**: Add new mock document
  - [ ] Validate required fields
  - [ ] Show validation errors
  - [ ] Add to in-memory store
  - [ ] Show success message
- [ ] **Read**: View mock document details
  - [ ] Render form in read-only mode
  - [ ] Show all field values
- [ ] **Update**: Edit existing mock document
  - [ ] Validate changes
  - [ ] Update in-memory store
  - [ ] Show success message
- [ ] **Delete**: Remove mock document
  - [ ] Show confirmation dialog
  - [ ] Remove from in-memory store
  - [ ] Show success message

### 2.3 List Operations

- [ ] **List (Few)**: Show 5 rows
  - [ ] Test basic list rendering
  - [ ] Test row actions (edit, delete)
- [ ] **List (Many)**: Generate 100+ rows
  - [ ] Test pagination
  - [ ] Test performance
  - [ ] Test scroll behavior

### 2.4 Filter & Search

- [ ] Implement client-side filtering
- [ ] Implement client-side search
- [ ] Show filtered results count
- [ ] Test filter combinations

### 2.5 Validation Testing

- [ ] Submit form with missing required fields
- [ ] Show field-level error messages
- [ ] Show form-level error summary
- [ ] Test custom validation rules

---

## 3. Local Hot Reload (Advanced)

> [!NOTE]
> **Optional Enhancement**
> For users who want to test with a **real local database** instead of mock data.

### 3.1 Prerequisites

- [ ] Local Postgres/SQLite running
- [ ] `m prod` running on localhost
- [ ] Database connection configured

### 3.2 File Watcher Setup

- [ ] Install `chokidar` for file watching
- [ ] Watch DocType files for changes
- [ ] Debounce file change events (1s)
- [ ] Ignore temporary files (.swp, .tmp)

### 3.3 Schema Sync Trigger

- [ ] Detect DocType file changes
- [ ] Parse updated DocType
- [ ] Generate schema diff
- [ ] Send sync request to backend

### 3.4 Backend Schema Sync Endpoint

- [ ] Create `POST /studio/api/sync-schema`
- [ ] Accept DocType schema
- [ ] Generate SQLAlchemy table
- [ ] Run ALTER TABLE if table exists
- [ ] Create table if new
- [ ] Return sync status

### 3.5 Real CRUD Operations

- [ ] Switch from mock data to real API calls
- [ ] Use Framework M API (`/api/v1/{doctype}`)
- [ ] Show real validation errors
- [ ] Persist data to local database

### 3.6 Hot Reload UI

- [ ] Add "Hot Reload" toggle in Studio settings
- [ ] Show connection status indicator
- [ ] Show sync progress
- [ ] Show sync errors

---

## 4. Advanced Preview Features

### 4.1 Responsive Preview

- [ ] Add device size selector:
  - [ ] Desktop (1920px)
  - [ ] Tablet (768px)
  - [ ] Mobile (375px)
- [ ] Resize preview iframe
- [ ] Test responsive layouts

### 4.2 Dark Mode Preview

- [ ] Add dark mode toggle in preview
- [ ] Test components in both modes
- [ ] Ensure contrast compliance

### 4.3 Accessibility Preview

- [ ] Add accessibility checker
- [ ] Show WCAG violations
- [ ] Highlight problematic elements
- [ ] Suggest fixes

### 4.4 Performance Preview

- [ ] Show render time
- [ ] Show component count
- [ ] Show bundle size estimate
- [ ] Warn on performance issues

---

## 5. Collaboration Features

### 5.1 Real-Time Collaboration (Optional)

- [ ] Add WebSocket support
- [ ] Show other users editing same DocType
- [ ] Show cursor positions
- [ ] Implement operational transformation (OT)

### 5.2 Change Tracking

- [ ] Show unsaved changes indicator
- [ ] Show diff view (before/after)
- [ ] Allow reverting changes
- [ ] Show change history

### 5.3 Comments & Annotations

- [ ] Add comment threads on fields
- [ ] Show unresolved comments count
- [ ] Allow @mentions
- [ ] Integrate with Git PR comments

---

## 6. Schema Change Detection (Developer Guardrails)

> [!NOTE]
> **RFC**: [Schema Change Detection & Developer Guardrails](/docs/rfcs/rfc-0003-schema-change-detection)
>
> This feature lives in Studio (devtools), NOT in `framework-m` runtime.
> Production containers stay minimal.

- [ ] Implement `SchemaAnalyzer` class:
  - [ ] Compare Pydantic models with current DB schema
  - [ ] Classify changes: SAFE, WARNING, DANGEROUS
  - [ ] Generate actionable messages with docs links

- [ ] CLI integration (`m migrate:create`):
  - [ ] Show analysis before generating migration
  - [ ] Block on DANGEROUS changes (cannot auto-generate)
  - [ ] Warn on WARNING changes (require `--force`)
  - [ ] Link to migration pattern documentation

- [ ] Studio UI integration:
  - [ ] Show warnings when saving DocType
  - [ ] Recommend safer alternatives (nullable, defaults)
  - [ ] "Save Anyway" escape hatch

---

## 7. DocType Discovery & Scanning

> [!NOTE]
> **RFC**: [DocType Discovery & Scanning](/docs/rfcs/rfc-0002-doctype-discovery)
>
> Deterministic, fast AST-based scanning. No caching. Handles 1000s of DocTypes in `<1s`.

- [ ] Optimize DocType scanning performance
- [ ] Implement AST-based parsing
- [ ] Handle large codebases (1000+ DocTypes)
- [ ] Add caching strategy for development mode

---

## 9. Desk View Configuration

- [ ] **Workspace Configurator**:
  - [ ] UI to creating/managing workspaces (Sidebar items)
  - [ ] Assign DocTypes to workspaces
  - [ ] Define workspace shortcuts and charts
  - [ ] Configure sidebar ordering and visibility
  - [ ] Role-based workspace access permissions

- [ ] **View Builders**:
  - [ ] Kanban View configurator (columns, cards)
  - [ ] Calendar View configurator (start/end fields)
  - [ ] Gantt View configurator (dependencies)
  - [ ] Map View configurator (geo fields)

---

## 10. Portal Builder (Puck Integration)

> [!NOTE]
> **Puck Integration**
> Instead of building a custom visual builder, we will integrate [Puck](https://puckeditor.com/) directly into Studio.
> Puck is a React-native, open-source visual editor perfect for this use case.

- [ ] **Puck Integration**:
  - [ ] Create `PuckEditor` wrapper component in Studio
  - [ ] Implement `StorageAdapter` to save pages as `WebPage` DocType
  - [ ] Implement `ViewAdapter` to render published pages

- [ ] **Component Bridge (The "Design System")**:
  - [ ] Map Framework M UI components to Puck Config
  - [ ] Create `Hero` component (Title, Subtitle, Image, CTA)
  - [ ] Create `CardGrid` component (Dynamic List)
  - [ ] Create `Form` component (Connect to DocType)
  - [ ] Create `Login` component

- [ ] **Data Binding (Advanced)**:
  - [ ] Create `FrameworkMDataProvider` for Puck "Root"
  - [ ] Fetch data from `/api/v1/content/:slug`
  - [ ] Pass data into Puck zones/components

---

## 11. Testing

### 11.1 Unit Tests

- [ ] Test mock data generator
- [ ] Test in-memory CRUD operations
- [ ] Test file watcher
- [ ] Test schema sync logic

### 11.2 Integration Tests

- [ ] Test full sandbox workflow
- [ ] Test hot reload with real database
- [ ] Test preview rendering
- [ ] Test collaboration features

---

## 12. Validation Checklist

Before considering Phase 14 complete, verify:

- [ ] Sandbox preview works for all field types
- [ ] Mock data generation is realistic
- [ ] CRUD operations work in sandbox
- [ ] Hot reload syncs schema correctly
- [ ] Preview is responsive and accessible
- [ ] File watcher is reliable

---

## Success Metrics

### Before (Phase 07)

- Static code generation
- No preview capability
- Manual testing required

### After (Phase 14)

- **Live preview** with mock data
- **Interactive sandbox** for UX testing
- **Hot reload** for rapid iteration
- **Collaboration features** for teams
- **Citizen developers** can build without coding

---

## Notes

- **Mock data is sufficient** for most use cases
- **Hot reload is optional** - adds complexity
- **Collaboration is advanced** - defer if not needed
- **Focus on preview quality** - make it feel real


Document: checklists/phase-02-doctype-engine.md

# Phase 02: DocType Engine & Database

**Objective**: Implement the metadata engine that dynamically creates database tables from Pydantic models and provides generic CRUD operations.

> [!IMPORTANT]
> **Database Agnostic**: All database implementations MUST work with both SQLite and PostgreSQL. Avoid database-specific features (e.g., PostgreSQL's `ARRAY`). Use portable SQL types that work across databases.

---

## 1. Schema Mapper (Pydantic → SQLAlchemy)

### 1.1 Field Registry & Type Mapping

- [x] Create `src/framework_m/adapters/db/field_registry.py`
- [x] Implement `FieldRegistry` class (Singleton):
  - [x] Start with standard types: str, int, float, bool, datetime, date, decimal, uuid, json
  - [x] Allow registration of custom types: `register_type(pydantic_type, sqlalchemy_type)`
- [x] Define type mapping using Registry:
  - [x] `str` → `String`
  - [x] `int` → `Integer`
  - [x] `float` → `Float`
  - [x] `bool` → `Boolean`
  - [x] `datetime` → `DateTime`
  - [x] `date` → `Date`
  - [x] `Decimal` → `Numeric`
  - [x] `UUID` → `UUID`
  - [x] `list[str]` → `JSON` (portable, works on all databases)
  - [x] `dict` → `JSON`

### 1.2 Schema Mapper Implementation
**TDD Focus**: Write `tests/adapters/db/test_schema_mapper.py` first!

> [!IMPORTANT]
> **Primary Key Design Decision**: Use `id` (UUID) as primary key, `name` as unique index.
> - **Rationale**: Allows document renames without cascading foreign key updates
> - **Frappe Issue**: Using `name` as PK causes expensive updates on rename
> - **Standard Practice**: All major databases use auto-generated `id` as PK
> - **Flexibility**: Framework users can rename documents cheaply

- [x] Create `tests/adapters/db/test_schema_mapper.py`
  - [x] Test mapping simple fields (str, int)
  - [x] Test mapping relationships
- [x] Create `SchemaMapper` class
- [x] Use `FieldRegistry` to look up SQLAlchemy types
- [x] Implement `create_table(model: Type[BaseDocType]) -> Table`:
  - [x] Extract table name from model class name (lowercase)
  - [x] Iterate over `model.model_fields`
  - [x] Map each field to SQLAlchemy column
  - [x] Handle `Optional` types (nullable=True)
  - [x] Set `id` field as primary key (UUID, auto-generated)
  - [x] Set `name` field as unique index (for human-readable lookup)
  - [x] **OCC**: Add `_version` (Integer, default=0) if `Meta.concurrency="optimistic"`
  - [x] Return SQLAlchemy `Table` object

- [x] Handle special field types:
  - [x] Relationships (ForeignKey) - detect by field name pattern `*_id`
  - [x] Enums - map to SQLAlchemy String type (database agnostic)
  - [x] Nested Pydantic models (`list[BaseModel]`) - store as JSON
  - [x] Child DocTypes (`list[DocType]`) - map to Relational Table with Foreign Key

### 1.3 Table Registry

- [x] Create `TableRegistry` class to store created tables
- [x] Add methods:
  - [x] `register_table(doctype_name: str, table: Table)`
  - [x] `get_table(doctype_name: str) -> Table`
  - [x] `table_exists(doctype_name: str) -> bool`

---

## 2. Database Connection Setup

### 2.1 Connection Factory & Virtual DocTypes

- [x] Create `src/framework_m/adapters/db/connection.py`
- [x] Implement `ConnectionFactory` (supports Multiple Bindings):
  - [x] Load `db_binds` from config (e.g., `legacy`, `timescale`)
  - [x] Create generic async engine (supports Postgres, SQLite for testing)
  - [x] Maintain map of `engine_name -> AsyncEngine`

### 2.2 Session Factory (SQL Support)

- [x] Add session factory:
  - [x] `async def get_session(bind: str = "default") -> AsyncSession`
  - [x] Support `VirtualDocType` (SQL Bind) by accepting `bind_key` from DocType Meta

### 2.3 Non-SQL Virtual DocTypes (Custom Repositories)

- [x] Create concept of `RepositoryOverride`:
  - [x] Allow DocType to define `repository_class` in Meta
  - [x] `RepositoryFactory` instantiates this instead of `GenericRepository`
- [x] Create `tests/adapters/db/test_repository_factory.py`:
  - [x] Implement a Mock `FileRepository` that reads from JSON file
  - [x] Register it for a `VirtualDoc`
  - [x] Verify `VirtualDoc.get()` calls `FileRepository.get()`

---

## 3. Generic Repository Implementation

- [x] Create `tests/adapters/db/test_generic_repository.py`
- [x] Define tests for CRUD operations (mock `AsyncSession`)
- [x] Create `src/framework_m/adapters/db/generic_repository.py`
- [x] Implement `GenericRepository[T]` class
- [x] Constructor dependencies (session is NOT stored here):
  - [x] `model: Type[T]`
  - [x] `table: Table`
  - [x] `controller_class: Type[BaseController] | None`
  - [x] `event_bus: EventBusProtocol | None` *(InMemoryEventBus for dev)*

> [!NOTE]
> All CRUD methods accept `session: AsyncSession` as the FIRST argument.
> The caller (Service/Controller) owns the session via `UnitOfWork`.

### 3.1 CRUD Operations

- [x] Implement `async def get(session: AsyncSession, id: UUID) -> Optional[T]`:
  - [x] Build SELECT query
  - [x] Filter `deleted_at IS NULL` (unless specific flag overrides)
  - [x] Execute with provided session (not stored session)
  - [x] Convert row to Pydantic model
  - [x] Return None if not found

- [x] Implement `async def get_by_name(session: AsyncSession, name: str) -> Optional[T]`:
  - [x] Helper for looking up by human-readable name

- [x] Implement `async def save(session: AsyncSession, entity: T, version: int | None = None) -> T`:
  - [x] Check if entity exists (by `id`)
  - [x] If new:
    - [x] Generate `id` (UUIDv7)
    - [x] **Note**: `created_by` / `owner` must be set by Controller/Service before calling save
    - [x] Call controller `validate()`
    - [x] Call controller `before_create()`
    - [x] Call controller `before_save()`
    - [x] Execute INSERT with provided session
    - [x] Call controller `after_save()`
    - [x] Call controller `after_create()`
    - [x] **Emit Event**: `event_bus.publish(f"{doctype}.create", payload)`
  - [x] If existing:
    - [x] **Note**: `modified_by` must be set by Controller/Service before calling save
    - [x] Call controller `validate()`
    - [x] Call controller `before_save()`
    - [x] **OCC**: If `optimistic`:
      - [x] `UPDATE table SET ..., _version=_version+1 WHERE id=:id AND _version=:old_ver`
      - [x] If rowcount == 0, raise `VersionConflictError`
    - [x] Else: Execute standard UPDATE
    - [x] Call controller `after_save()`
  - [x] **Emit Event**: `event_bus.publish(f"{doctype}.update", payload)`
  - [x] Return saved entity (caller calls `uow.commit()` when ready)

- [x] Implement `async def delete(session: AsyncSession, id: UUID, hard: bool = False) -> None`:
  - [x] Load entity
  - [x] Call controller `before_delete()`
  - [x] If `hard`:
    - [x] Execute DELETE
  - [x] Else (Soft Delete):
    - [x] Update `deleted_at = now()`
  - [x] Call controller `after_delete()`
  - [x] **Emit Event**: `event_bus.publish(f"{doctype}.delete", payload)`
  - [x] Return (caller calls `uow.commit()` when ready)

- [x] Implement `async def list(session: AsyncSession, filters, limit, offset, order_by) -> Sequence[T]`:
  - [x] Build SELECT query with filters
  - [x] Default filter: `deleted_at IS NULL`
  - [x] Apply pagination (limit, offset)
  - [x] Apply sorting (order_by)
  - [x] Execute query
  - [x] Convert rows to Pydantic models
  - [x] Return list

### 3.2 Lifecycle Hook Integration

- [x] Create helper method `_call_hook(hook_name: str)`:
  - [x] Check if controller exists
  - [x] Check if hook method exists on controller
  - [x] Call hook method if present
  - [x] Handle exceptions and rollback on error

---

## 4. Migration System

### 4.1 Alembic Integration

- [x] Initialize Alembic in project:
  ```bash
  alembic init alembic
  ```
  *Implemented via `MigrationManager.init()` which creates alembic directory structure*

- [x] Configure `alembic.ini`:
  - [x] Set database URL from environment
  - [x] Configure migration file location

- [x] Create `alembic/env.py`:
  - [x] Import `MetaRegistry`
  - [x] Import all registered DocTypes
  - [x] Set `target_metadata` from SchemaMapper tables
  *Auto-generated with async support via `MigrationManager.init()`*

### 4.2 Auto-Migration Detection

- [x] Create `src/framework_m/adapters/db/migration.py`
- [x] Implement `detect_schema_changes()`:
  - [x] Compare registered DocTypes with database schema
  - [x] Detect new tables
  - [x] Detect new columns
  - [x] Detect type changes
  - [x] Return list of changes

- [x] Implement `auto_migrate()`:
  - [x] Call `detect_schema_changes()`
  - [x] Generate Alembic migration if changes detected
  - [x] Apply migration automatically (dev mode only)

### 4.3 CLI Commands

- [x] Add migration commands to CLI:
  - [x] `m migrate` - run pending migrations
  - [x] `m migrate:create <name>` - create new migration
  - [x] `m migrate:rollback` - rollback last migration
  - [x] `m migrate:status` - show migration status
  *Also added: `m migrate:history`, `m migrate:init`*

---

## 5. Repository Factory

- [x] Create `src/framework_m/adapters/db/repository_factory.py`
- [x] Implement `RepositoryFactory` class:
  - [x] `create_generic_repository(doctype_name: str) -> GenericRepository`
  - [x] Look up DocType from MetaRegistry
  - [x] Look up Table from TableRegistry
  - [x] Look up Controller from MetaRegistry
  - [x] Create and return GenericRepository instance
  - [x] Support `event_bus` parameter for domain events
  - [x] Support custom repository overrides for Virtual DocTypes

---

## 6. Engine, Session Factory & Unit of Work

> [!IMPORTANT]
> **Anti-Pattern to Avoid**: Do NOT tie transaction lifecycle to HTTP request lifecycle.
> Repositories receive sessions; they do NOT create or own them.

### 6.1 Engine & Session Factory Setup

- [x] Create `src/framework_m/adapters/db/connection.py` *(ConnectionFactory)*
- [x] Implement `create_engine(url: str) -> AsyncEngine`
  - [x] Pool configuration (pool_size, max_overflow, timeout, recycle, pre_ping)
  - [x] Environment variable expansion (`${VAR}` syntax)
- [x] Implement `SessionFactory` (returns `AsyncSession` context managers)
  - [x] Support multiple binds (for Virtual DocTypes with SQL binds)
  - [x] Configuration from ConnectionFactory
  - [x] Auto commit/rollback in context manager

### 6.2 Unit of Work (`UnitOfWork`)

- [x] Create `tests/core/test_unit_of_work.py`
  - [x] Test: UoW provides session
  - [x] Test: `commit()` persists changes
  - [x] Test: Exception causes rollback (no explicit rollback call needed)
  - [x] Test: Session is closed after `__aexit__`
- [x] Create `src/framework_m/core/unit_of_work.py`
- [x] Implement `UnitOfWork` context manager:
  - [x] `__init__(session_factory: Callable[[], AsyncSession])`
  - [x] `session` property with safety check
  - [x] `async __aenter__()` creates session
  - [x] `async commit()` calls session.commit()
  - [x] `async rollback()` for explicit rollback
  - [x] `async __aexit__()` with auto-rollback on exception
- [x] Register `UnitOfWorkFactory` in DI container

### 6.3 Multi-Source Coordination (Outbox Pattern)

- [x] Create `src/framework_m/core/domain/outbox.py`
- [x] Define `OutboxEntry` model:
  - [x] `id: UUID`
  - [x] `target: str` (e.g., "mongodb.audit_log", "api.payment_gateway")
  - [x] `payload: dict`
  - [x] `status: str` (pending, processed, failed)
  - [x] `created_at: datetime`
  - [x] `processed_at: datetime | None`
  - [x] `error_message: str | None`
  - [x] `retry_count: int`
- [x] Create `OutboxRepository` (SQL-backed) in `adapters/db/outbox_repository.py`
  - [x] `add(session, entry)` - add entry in same transaction
  - [x] `get_pending(session, limit)` - get pending entries
  - [x] `mark_processed(session, id)` - mark as processed
  - [x] `mark_failed(session, id, error)` - mark as failed
- [x] Document: Services write to Outbox in the *same* SQL transaction
- [x] (Phase 04) Background worker processes Outbox entries

### 6.4 Startup Sequence (Schema Sync)

- [x] Create `src/framework_m/adapters/db/__init__.py`
- [x] Implement startup sequence (called once at app boot, NOT per-request):
  - [x] Initialize database engine (NOT session)
  - [x] Discover all DocTypes via MetaRegistry
  - [x] Create/sync tables via SchemaMapper
  - [x] Register tables in TableRegistry
  - [x] Run auto-migration (if enabled)

---

## 7. Testing

### 7.1 Unit Tests

- [x] Test `SchemaMapper`:
  - [x] Test type mapping for all supported types (`test_schema_mapper.py`)
  - [x] Test primary key creation (`test_table_has_id_column_as_primary_key`)
  - [x] Test nullable fields (`TestSchemaMapperNullableFields`)
  - [x] Test enum mapping (`TestSchemaMapperEnums`)

- [x] Test `GenericRepository`:
  - [x] Test CRUD operations with mock data (`test_generic_repository.py`)
  - [x] Test lifecycle hook calls (`TestControllerHooks`)
  - [x] Test transaction rollback on error (`TestTransactionRollback`)

### 7.2 Integration Tests

- [x] Setup testcontainers for PostgreSQL (skips if Docker unavailable)
- [x] Create test DocType (`IntegrationTestDoc` / `PostgresTestDoc`)

- [x] Test full flow (SQLite & Postgres - Postgres skipped without Docker):
  - [x] Register DocType (`test_register_doctype`)
  - [x] Create table (`test_table_created`)
  - [x] Insert document (`test_insert_document`)
  - [x] Query document (`test_query_document`)
  - [x] Update document (`test_update_document`)
  - [x] Delete document (`test_delete_document`)

- [x] Test migration:
  - [x] Add field to DocType
  - [x] Run auto-migration
  - [x] Verify column added to table

---

## 8. Error Handling

- [x] Create custom exceptions (`core/exceptions.py`):
  - [x] `DocTypeNotFoundError`
  - [x] `ValidationError`
  - [x] `PermissionDeniedError`
  - [x] `DuplicateNameError`
  - [x] `RepositoryError`, `EntityNotFoundError`, `DatabaseError`, `IntegrityError`

- [x] Add error handling in repository (`generic_repository.py`):
  - [x] Catch SQLAlchemy errors (`IntegrityError`, `OperationalError`, `SQLAlchemyError`)
  - [x] Convert to domain exceptions (`DuplicateNameError`, `DatabaseError`, etc.)
  - [x] Log errors with context (using `logger.error` with `extra` dict)

---

## 9. Performance Optimizations

- [x] Add query result caching (phase-04):
  - [x] Cache `get()` results by ID
  - [x] Invalidate cache on save/delete
  - [x] Use Redis for distributed cache

- [x] Add bulk operations (`generic_repository.py`):
  - [x] `async def bulk_save(entities)` - separates new/existing
  - [x] `async def _bulk_insert(entities)` - true SQLAlchemy bulk insert
  - [x] Uses `insert().values([...])` for performance

- [x] Add query optimization (add indexes after the API layer exists):
  - [x] Add indexes for common queries (schema_mapper: owner, creation, Meta.indexes)
  - [x] Add parent index for child tables (efficient joins)
  - [x] Use `select_in_loading` for relationships (`load_children_for_parents` method)
  - [x] Add query logging in dev mode (`logger.debug` for GET/LIST)

---

## Validation Checklist

Before moving to Phase 03, verify:

- [x] Can create tables from Pydantic models dynamically
- [x] CRUD operations work with lifecycle hooks
- [x] Migrations are generated and applied correctly
- [x] All integration tests pass with real PostgreSQL *(skips if Docker unavailable)*
- [x] No direct SQLAlchemy imports in domain layer
- [x] Repository implements `RepositoryProtocol` correctly

---

## Anti-Patterns to Avoid

❌ **Don't**: Use raw SQL queries in business logic
✅ **Do**: Use repository methods and let SQLAlchemy handle queries

❌ **Don't**: Hardcode supported field types
✅ **Do**: Use `FieldRegistry` to allow plugins to add types (e.g. GeoLocation)

❌ **Don't**: Store metadata in database like Frappe
✅ **Do**: Generate tables from code-first Pydantic models

❌ **Don't**: Use synchronous database calls
✅ **Do**: Use async/await throughout

❌ **Don't**: Hardcode table names or column names
✅ **Do**: Derive from Pydantic model metadata


Document: checklists/phase-07-studio.md

# Phase 07: Studio (Code Generation UI)

**Objective**: Build a visual DocType builder (`framework-m-studio`) that generates Python code, running as a Litestar app.

---

## 1. Packaging Strategy (Runtime vs DevTools)

### 1.1 framework-m (Runtime)

- [x] **Goal**: Production-grade kernel.
- [x] **Content**: API Server, ORM, DocType Engine, Basic CLI (`start`, `migrate`, `worker`).
- [x] **Dependencies**: `litestar`, `sqlalchemy`, `pydantic`, `cyclopts`.
- [x] **Excluded**: Code generators, Studio UI, Heavy formatting/linting libs.

### 1.2 framework-m-studio (DevTools)

- [x] **Goal**: Developer Productivity & SaaS Component.
- [x] **Content**: Studio UI, Code Generators, Visual Editors.
- [x] **Dependencies**: `framework-m`, `libcst`, `jinja2`.
- [x] **Integration**: Plugs into `m` CLI via entry points (`m codegen`).

## 2. Studio Backend (Litestar App)

### 1. Project Structure (Monorepo)

- [x] Create `apps/studio/` in the workspace
- [x] Initialize `apps/studio/pyproject.toml`:
  - [x] Add dependency: `framework-m = { workspace = true }`
  - [x] Add dependency: `libcst` (for code modification)
  - [x] Add dependency: `jinja2` (for code generation)
  - [x] Add code generation endpoints

### 1.2 File System API

- [x] Create `GET /studio/api/doctypes`:
  - [x] Scan project for `*.py` files containing DocType classes
  - [x] Return list of DocTypes with metadata

- [x] Create `GET /studio/api/doctype/{name}`:
  - [x] Read DocType file
  - [x] Parse with LibCST
  - [x] Return structured JSON

- [x] Create `POST /studio/api/doctype/{name}`:
  - [x] Accept DocType schema JSON
  - [x] Generate Python code
  - [x] Write to file system

- [x] Create `DELETE /studio/api/doctype/{name}`:
  - [x] Delete DocType file
  - [x] Delete Controller file (if exists)

---

## 2. LibCST Code Transformer

### 2.1 Parser

- [x] Create `src/framework_m_studio/codegen/parser.py`
- [x] Implement `parse_doctype(file_path: str) -> dict`:
  - [x] Use LibCST to parse Python file
  - [x] Extract class definition
  - [x] Extract fields with types and defaults
  - [x] Extract Config metadata
  - [x] Return structured dict

### 2.2 Generators (The Strategy)

> "Jinja for Creation, LibCST for Mutation"

- [x] **Creation (Scaffolding)**:
  - [x] Use `jinja2` templates (shared with CLI `m new:doctype`).
  - [x] Goal: Generate clean, standard Python code from scratch.
  - [x] Implement `generate_doctype_source(schema: dict) -> str`.

- [x] **Mutation (Transformer)**:
  - [x] Use `LibCST` to parse and modify existing files.
  - [x] Goal: Add/Edit fields while preserving comments and custom methods.
  - [x] Implement `update_doctype_source(source: str, schema: dict) -> str`.

### 2.3 Test Generator

- [x] Create `src/framework_m_studio/codegen/test_generator.py`
- [x] Implement `generate_test(schema: dict) -> str`:
  - [x] Generate `test_{doctype}.py`
  - [x] Import `pytest` and DocType
  - [x] Generate basic CRUD test (Create, Read, Update, Delete)
  - [x] Generate validation failure test (if required fields exist)

### 2.4 Transformer

- [x] Create `src/framework_m_studio/codegen/transformer.py`
- [x] Implement `update_doctype(file_path: str, schema: dict)`:
  - [x] Parse existing file with LibCST
  - [x] Locate DocType class
  - [x] Update/add/remove fields
  - [x] Preserve comments and custom methods
  - [x] Write back to file

- [x] Handle edge cases:
  - [x] Field renamed (detect and update)
  - [x] Field type changed
  - [x] Field deleted (remove from class)
  - [x] Custom methods preserved

---

## 3. Studio Frontend

> **Stack**: Per ADR-0005 (Use Refine for Frontend) — Refine + shadcn/ui + RJSF + Tailwind

### 3.1 React App Setup (Refine Stack)

- [x] Create `apps/studio/studio_ui/` directory
- [x] Initialize Refine app:

  ```bash
  pnpm create refine-app@latest studio_ui
  # Selected: refine-vite, custom-json-rest, no UI, no examples, pnpm
  ```

- [x] Install dependencies per ADR-0005:
      | Layer | Package | Purpose |
      |-------|---------|---------|
      | Data/State | `@refinedev/core` | CRUD, caching, auth |
      | Forms | `@rjsf/core` + `@rjsf/validator-ajv8` | Schema-driven forms |
      | Tables | `@tanstack/react-table` | Data tables |
      | UI Components | `shadcn/ui` | Accessible components |
      | Styling | `tailwindcss` | Utility-first CSS |
      | Code Editor | `@monaco-editor/react` | Python code preview |
      | Drag & Drop | `@dnd-kit/core` + `@dnd-kit/sortable` | Field reordering |

- [x] Create Framework M Data Provider (`src/providers/dataProvider.ts`):

  ```typescript
  // Maps Refine's DataProvider to Framework M API
  const dataProvider = frameworkMDataProvider(config.apiBaseUrl);
  ```

- [x] Configure API base URLs (per ADR-0005):
  - [x] Read from `window.__FRAMEWORK_CONFIG__` or env vars
  - [x] Support same-origin, subdomain, and CDN scenarios
  - [x] Default to `/studio/api` for Studio endpoints

### 3.2 DocType List View (Refine Resource)

- [x] Create `pages/doctypes/list.tsx`:
  - [x] Use `useList` hook from `@refinedev/core` to fetch DocTypes
  - [x] Display with `@tanstack/react-table` + Tailwind styling
  - [x] Add search/filter with `useTable` globalFilter
  - [x] Add "New DocType" button → navigates to `/doctypes/create`

### 3.3 DocType Editor (Refine Form)

- [x] Create `pages/doctypes/edit.tsx`:
  - [x] Use `useOne`/`useUpdate`/`useCreate` hooks for data
  - [x] Left panel: Field list with `@dnd-kit/sortable`
  - [x] Right panel: Field properties form (name, type, required, default, description)
  - [x] Top bar: DocType name input + Save button

### 3.4 Field Editor (RJSF Schema-Driven)

- [x] Create `components/FieldEditor.tsx`:
  - [x] Render with RJSF using JSON Schema for field properties
  - [x] Custom Tailwind widgets: `TextWidget`, `SelectWidget`, `CheckboxWidget`, `TextareaWidget`
  - [x] Dynamic "Validators" collapsible for min/max length, pattern, etc.
  - [x] Live preview of generated Python type annotation

### 3.5 Drag & Drop

- [x] Implement field reordering:
  - [x] Use `@dnd-kit/core` + `@dnd-kit/sortable`
  - [x] Allow dragging fields to reorder
  - [x] Update schema on drop (via `arrayMove`)

### 3.6 Code Preview (Monaco Editor)

- [x] Create `components/CodePreview.tsx`:
  - [x] Use `@monaco-editor/react` with `language="python"`
  - [x] Read-only mode (`readOnly: true`)
  - [x] Theme: `vs-dark`
  - [x] Auto-update on schema change (memoized generation)
  - [x] Copy button for generated code

### 3.7 Save Flow

- [x] Implement save functionality:
  - [x] Validate schema (name required)
  - [x] Call backend API via `useCreate`/`useUpdate`
  - [x] Navigate back to list on success
  - [x] Loading state on save button

### 3.8 Visual Data Modeling (Mind Map)

- [x] Implement ERD/Mind Map View:
  - [x] Central Node: Current DocType (highlighted)
  - [x] Connected Nodes: Related DocTypes via Link fields
  - [x] Visual Action: Drag to connect nodes → Creates Link Field
  - [x] Using `@xyflow/react` (React Flow v12)

### 3.9 Layout Designer (Grid System)

- [x] Implement Visual Layout Drag & Drop:
  - [x] Define Sections with title and Columns (1/2/3 cols selector)
  - [x] Drag fields from palette into grid cells
  - [x] Real-time form preview toggle
  - [x] Add/delete sections

### 3.10 Module Explorer

- [x] Implement Module Scanner:
  - [x] Group DocTypes by directory/app module (tree structure)
  - [x] Tree view navigation in Sidebar
  - [x] Expand/collapse all, search filter
  - [x] Click to navigate to DocType editor

---

## 4. Studio Cloud Mode (Ephemeral & Git-Backed)

"Stateless Studio": No RWX storage required. Repo is cloned to ephemeral path.

> [!NOTE]
> **Workspace Terminology**
>
> - **Studio Workspace** = The Git monorepo where code lives (this section)
> - **Desk Workspace** = End-user navigation groupings in sidebar (Phase 09, Section 6.2)
>
> A Studio Workspace (monorepo) can contain multiple Desk Workspace definitions in code.

### 4.1 Architecture

- **Storage**: `/tmp/workspace_{session_id}` (Ephemeral/Pod-local). ✅
- **Persistence**: Git Provider (GitHub/GitLab). ✅
- **Concurrency**: Optimistic Locking (Last push wins, or Rebase flow). ✅
- **Deployment**: **StatefulSet** or **Deployment (Replicas=1)**.
  - _Reasoning_: Workspaces are local to the Pod. Scaling requires Sticky Sessions or RWX.
  - _Recommendation_: Keep Studio as a Singleton. Scale the Application, not the Editor.

### 4.2 GitOps Integration (FluxCD/ArgoCD)

- **Configuration**: Inject `GIT_REPO`, `GIT_TOKEN` via Environment Variables. ✅
- **Read State**: Clone target repo on startup (or user connect). ✅
- **Write State**:
  - Create Branch + Open PR (Review Process). ✅
- **Update State**:
  - [x] Implement "Pull Latest" Button (`POST /workspace/{id}/pull`).
  - [x] Show "Updates Available" indicator (`POST /workspace/{id}/check-updates`).

### 4.3 Git Adapter (Data Agnostic)

- [x] Create `src/framework_m_studio/git/protocol.py` - GitAdapterProtocol (Port)
- [x] Create `src/framework_m_studio/git/adapter.py` - GitAdapter (Adapter)
- [x] Implement `GitAdapter`:
  - [x] Wraps standard `git` binary via asyncio subprocess.
  - [x] `clone(repo_url, auth)`: Clone to temp dir.
  - [x] `commit(message)`: Stage and commit changes.
  - [x] `push(branch)`: Push to remote.
  - [x] `pull()`: Pull latest changes.
  - [x] `create_branch(name)`: Create new branch.
  - [x] `get_status()`: Get workspace status.
- [x] Unit tests in `tests/test_git_adapter.py` (12 tests)

### 4.4 GitHub Provider

- [x] Create `src/framework_m_studio/git/github_provider.py`
- [x] Implement `GitHubProvider`:
  - [x] Use GitHub API for operations (REST API with urllib)
  - [x] Support personal access tokens (Bearer auth)
  - [x] Handle authentication (GitHubAuthError)
  - [x] Create pull requests
  - [x] List pull requests
  - [x] Validate tokens
- [x] Unit tests in `tests/test_github_provider.py` (9 tests)

### 4.5 Generic Git Provider

> Note: Implemented as GitAdapter using git CLI - no need for separate provider.

- [x] Support HTTPS and SSH (via git CLI)
- [x] Work with any Git server (generic implementation)

### 4.6 Workspace Management

- [x] Create `src/framework_m_studio/workspace.py`
- [x] Implement workspace lifecycle:
  - [x] Clone repo to temp directory
  - [x] Track workspace sessions
  - [x] Clean up old workspaces (TTL-based)
  - [x] Handle concurrent edits (asyncio lock)
- [x] Unit tests in `tests/test_workspace.py` (12 tests)

### 4.7 Cloud Studio API

- [x] Create `POST /studio/api/workspace/connect`:
  - [x] Accept repo URL and token
  - [x] Clone repository
  - [x] Return workspace ID

- [x] Create `POST /studio/api/workspace/{id}/commit`:
  - [x] Commit changes
  - [x] Push to branch
  - [x] Return commit SHA

- [x] Create `GET /studio/api/workspace/{id}/status`: Git status
- [x] Create `POST /studio/api/workspace/{id}/pull`: Pull latest
- [x] Create `DELETE /studio/api/workspace/{id}`: Cleanup workspace
- [x] Create `GET /studio/api/workspace/sessions`: List all sessions

---

## 5. Studio CLI Command

### 5.1 SPA Packaging & Serving (Path Prefix Architecture)

- [x] **Build Process**:
  - [x] `pnpm run build` in `studio_ui` outputs to `src/framework_m_studio/static/` (vite.config.ts).
  - [x] `pyproject.toml` includes `framework_m_studio/static/**/*`.

- [x] **Serving Strategy (Litestar)** (in `app.py`):
  - [x] **UI Routes**: Serve `/studio/ui/*` → React SPA (HTML pages)
  - [x] **API Routes**: Serve `/studio/api/*` → JSON endpoints
  - [x] **Assets**: Serve `/studio/ui/assets/*` as static files (Cached)
  - [x] **SPA Catch-all**: `/studio/ui/{path:path}` → returns `index.html` for client-side routing
  - [x] **Redirect**: `/studio` → `/studio/ui/` for convenience
  - [x] **Content-Type Fix**: All responses use explicit `media_type` to prevent download issues
  - [x] **Implementation**: `serve_spa()` and `_get_spa_response()` functions in `app.py`

- [x] Implement `m studio` (in `framework-m-studio` package):
  - [x] Register CLI command via entry point:
    ```toml
    [project.entry-points."framework_m.cli_commands"]
    studio = "framework_m_studio.cli:studio_app"
    ```
  - [x] Start uvicorn server on port 9000 (default)
  - [x] Serve Studio UI
  - [ ] Open browser automatically (TODO)
  - [x] Options:
    - [x] `--port` - Custom port
    - [x] `--host` - Custom host
    - [x] `--reload` - Development mode
    - [x] `--cloud` - Enable cloud mode

---

## 6. DevTools CLI (Extensions)

> "cli needs clear separation... m docs|codegen needs to be part of studio"

These commands are registered via `entry_points` in `framework-m-studio`. They are NOT available in the base `framework-m` package.

### 6.1 Documentation Generator (`m docs:generate`)

- [x] Dependencies: Uses stdlib `urllib`, `json`, `subprocess` (no external deps required).
- [x] Command: `m docs:generate`:
  - [x] **API Reference**: Generates markdown from DocTypes in `docs_generator.py`.
  - [x] **OpenAPI Export**: Exports `openapi.json` from URL (`--openapi-url`).
  - [x] **Site Build**: Runs `mkdocs build` if available (`--build-site`).
- [x] Unit tests in `tests/test_docs_generator.py` (10 tests)

### 6.2 Client SDK Generator (`m codegen client`)

- [x] Dependencies: Uses stdlib only (no external deps required).
- [x] Command: `m codegen client`:
  - [x] **Flow**:
    1.  Fetches OpenAPI JSON from URL.
    2.  Generates code using stdlib.
  - [x] **Arguments**:
    - `--lang [ts|py]`: Target language.
    - `--out [dir]`: Output directory.
    - `--openapi-url`: URL to fetch schema from.
  - [x] **TypeScript**: Generates interfaces + fetch client in `sdk_generator.py`.
  - [x] **Python**: Generates Pydantic models in `sdk_generator.py`.
- [x] Unit tests in `tests/test_codegen.py` (6 tests)
- [x] CLI tests updated in `tests/test_cli.py`

---

## 7. Field Type Library

### 7.1 Supported Field Types

- [x] Implement field type definitions:
  - [x] Text (str)
  - [x] Number (int, float)
  - [x] Checkbox (bool)
  - [x] Date (date)
  - [x] DateTime (datetime)
  - [x] Select (enum)
  - [x] Link (foreign key)
  - [x] Table (child table)
    - [x] Level 1: Grid/Table view
    - [x] Level 2+: Drill-down pattern (Button -> Drawer/Dialog) for infinite nesting
  - [x] JSON (dict)
  - [x] File (file upload)

### 7.2 Custom Field Type Discovery

- [x] **Dynamic Loading**: Studio reads available types from `FieldRegistry` at runtime.
  - [x] API Endpoint: `GET /studio/api/field-types`
  - [x] Returns: Built-in types + Types registered by installed apps.
  - [x] Each type includes: `name`, `pydantic_type`, `sqlalchemy_type`, `ui_component` (optional).
- [x] **UI Integration**: Field Type Selector in DocType Editor dynamically populates from this API.

### 7.3 Custom UI Components (Client-Side)

- [x] **Goal**: Allow apps to register custom React components for their field types.
- [x] **Registration (App-Side)**:

  ```typescript
  // my_app/frontend/index.ts
  import { registerFieldComponent } from "@framework-m/studio";
  import StarRating from "./components/StarRating";

  registerFieldComponent("Rating", StarRating);
  ```

- [x] **Discovery (Studio-Side)**:
  - [x] **Shadow Build** (Phase 09) loads all app `frontend/` plugins.
  - [x] Studio UI checks `window.__STUDIO_FIELD_COMPONENTS__` map.
  - [x] Falls back to default `<input>` if no custom component.

### 7.5 Live Preview & Sandbox Mode

> [!IMPORTANT]
> **Sandbox = Local Development Environment**
> The Sandbox runs on the **same machine** where Studio is running. It is NOT a separate cloud environment.
> This enables citizen developers to design, test, and iterate locally before committing to Git.

**Citizen Developer Workflow:**

```
┌─────────────────────────────────────────────────────────────────────┐
│  LOCAL MACHINE (Developer or Citizen)                               │
│                                                                     │
│   1. Studio UI          2. Sandbox              3. Git              │
│   ┌──────────────┐     ┌──────────────┐       ┌──────────────┐      │
│   │ Design       │ ──► │ Test with    │ ──►   │ Commit &     │      │
│   │ DocType      │     │ mock data    │       │ Push         │      │
│   └──────────────┘     └──────────────┘       └──────────────┘      │
│                                                      │              │
└──────────────────────────────────────────────────────│──────────────┘
                                                       ▼
                                              CI/CD → Production
```

> [!WARNING]
> **Collaborative Environments: Conflict Risk**
> When multiple citizens work on the **same repo/branch**, local sandboxes can cause Git conflicts.
> The sandbox is best suited for **solo work** or **feature branches**.

**Alternatives for Collaborative Teams:**

| Approach                  | How It Works                                         | Best For          |
| ------------------------- | ---------------------------------------------------- | ----------------- |
| **Feature Branches**      | Each citizen works on their own branch → PR to merge | Small teams (2-5) |
| **Gitpod/Codespaces**     | Each citizen gets isolated cloud dev environment     | Larger teams      |
| **Assigned Modules**      | Citizens own specific modules/apps, no overlap       | Enterprise        |
| **Trunk-Based + Fast CI** | All push to main, CI deploys in `<2 min`             | Mature teams      |
| **Studio Cloud Mode**     | Hosted Studio with workspace isolation (Section 4)   | SaaS offering     |

- [ ] **Recommendation**: Default to feature branches for teams. Add branch selector to Studio UI.

- [ ] **Goal**: Preview the DocType being edited with mock data AND simulate CRUD.
- [ ] **Implementation**:
  - [ ] **Preview Tab**: Toggle between "Schema Editor" and "Sandbox".
  - [ ] **Mock Data Generation**:
    - [ ] Use `@faker-js/faker` or simple generators.
    - [ ] Generate sample rows based on field types.
  - [ ] **Form Preview**: Render `AutoForm` with generated mock doc.
  - [ ] **List Preview**: Render `AutoTable` with 5-10 mock rows.
- [ ] **Schema Source**: Uses the _in-memory_ schema being edited, NOT the saved file.
  - [ ] Allows instant feedback without saving.

### 7.6 Sandbox: Interactive CRUD Simulation

> [!NOTE]
> **Why Mock Data?**
> Sandbox uses mock data so citizens can test the full UX (forms, lists, validation)
> without needing a database setup. Once satisfied, they commit the code and deploy
> to an environment with real data.

- [x] **Goal**: Let users test the full UX before committing the DocType.
- [x] **Features**:
  - [x] **New**: Create a new mock document. Test mandatory field validation.
  - [x] **Update**: Edit an existing mock document. See form behavior.
  - [x] **Delete**: Delete mock document. See confirmation dialogs.
  - [x] **List (Few)**: Show 5 rows. Test basic list rendering.
  - [x] **List (Many)**: Paginate 100+ mock rows. Test performance.
  - [x] **Filter**: Apply filters and see results change.
  - [x] **Validation**: Submit form with missing required fields. See error messages.
- [x] **Data Store**: In-memory array (resets on page reload).
- [x] **Benefit**: No database required. Test UX before writing to disk.

### 7.7 Local Hot Reload (Optional Enhancement)

- [ ] **Goal**: For users who want to test with a **real local database** instead of mock data.
- [ ] **Prerequisites**: Local Postgres/SQLite + `m prod` running.
- [ ] **Flow**:
  - [ ] Studio saves DocType to file.
  - [ ] File watcher detects change → triggers schema sync.
  - [ ] Table created/updated in local DB.
  - [ ] User can now test with real CRUD operations.
- [ ] **Benefit**: Bridge between mock sandbox and production—test with actual data locally.

### 7.2 Field Metadata

- [x] Define field options:
  - [x] Label
  - [x] Description
  - [x] Required
  - [x] Default value
  - [x] Validation rules
  - [x] Display options (hidden, read-only)

---

## 8. Controller Scaffolding

- [x] Add controller generation:
  - [x] Generate controller file template
  - [x] Add common hook methods
  - [x] Add validation examples

- [x] Controller editor in UI:
  - [x] List hook methods
  - [x] Add custom methods
  - [x] Code editor for method bodies

---

## 9. Testing

### 9.1 Unit Tests

- [x] Test LibCST parser:
  - [x] Parse valid DocType
  - [x] Extract fields correctly
  - [x] Handle edge cases

- [x] Test code generator:
  - [x] Generate valid Python code
  - [x] Format correctly
  - [x] Handle all field types

### 9.2 Integration Tests

- [x] Test full flow:
  - [x] Create DocType via UI
  - [x] Save to file system
  - [x] Reload and verify
  - [x] Update DocType
  - [x] Verify changes preserved

- [x] Test Git integration:
  - [x] Clone repository
  - [x] Make changes
  - [x] Commit and push
  - [x] Verify on GitHub

---

## 10. Documentation

- [x] Create Studio user guide:
  - [x] How to start Studio
  - [x] How to create DocType
  - [x] How to add fields
  - [x] How to configure permissions
  - [x] How to use Git mode

---

## Validation Checklist

Before moving to Phase 08, verify:

- [x] Studio UI can create and edit DocTypes
- [x] Generated code is valid Python
- [x] Changes are saved to file system
- [x] LibCST preserves custom code
- [x] Git mode can commit and push
- [x] Studio works in both local and cloud modes

---

## Anti-Patterns to Avoid

❌ **Don't**: Store DocType metadata in database like Frappe
✅ **Do**: Generate Python source code

❌ **Don't**: Use AST (destroys formatting)
✅ **Do**: Use LibCST (preserves formatting)

❌ **Don't**: Overwrite custom code
✅ **Do**: Preserve comments and custom methods

❌ **Don't**: Require complex setup for Studio
✅ **Do**: Make it work in browser with Git


Document: checklists/phase-01-core-kernel.md

# Phase 01: Core Kernel & Interfaces

**Objective**: Set up the project foundation with clean hexagonal architecture. Define all protocol interfaces (ports) without any implementation.
**Principle**: Test-Driven Development (TDD). Write tests first. Goal: 95% Coverage.

---

## 1. Project Initialization

- [x] Create project structure using `uv`:
  ```bash
  uv init framework-m
  cd framework-m
  ```

- [x] Setup `pyproject.toml` with dependencies
  - [x] Add `litestar[standard]>=2.0.0`
  - [x] Add `sqlalchemy[asyncio]>=2.0.0` (ensure agnostic, test with sqlite)
  - [x] Add `pydantic>=2.0.0`
  - [x] Add `pydantic-settings>=2.0.0`
  - [x] Add `dependency-injector>=4.41.0` (powerful DI container)
  - [x] Add `asyncpg>=0.29.0` (PostgreSQL async driver)
  - [x] Add `redis>=5.0.0`
  - [x] Remove `arq>=0.26.0`
  - [x] Add `taskiq>=0.11.0`, `taskiq-nats>=0.4.0`, `nats-py>=2.0.0`
  - [x] Add dev dependencies: `pytest`, `pytest-asyncio`, `mypy`, `ruff`

- [x] Configure development tools
  - [x] Setup `mypy` with strict mode in `pyproject.toml`
  - [x] Configure `ruff` for linting and formatting
  - [x] Add `.gitignore` for Python projects

- [x] Create directory structure
  ```
  src/framework_m/
  ├── core/
  │   ├── interfaces/     # Ports (protocols)
  │   └── domain/         # Domain models
  ├── adapters/           # Infrastructure (empty for now)
  ├── cli/                # CLI commands (empty for now)
  └── public/             # Built-in DocTypes (empty for now)
  ```

---

## 2. Define Port Interfaces

**Key Principle**: Ports are ONLY interfaces. No implementations. Use Python `Protocol` for structural typing.

### 2.1 Repository Protocol

- [x] Create `tests/core/interfaces/test_repository.py`
- [x] Define test for `RepositoryProtocol` interface compliance
- [x] Create `src/framework_m/core/interfaces/repository.py`
- [x] Define supporting models:
  - [x] `FilterSpec` - Typed filter specification (field, operator, value)
  - [x] `OrderSpec` - Sorting specification (field, direction)
  - [x] `PaginatedResult[T]` - Result with `items`, `total`, `limit`, `offset`, `has_more`
- [x] Define `RepositoryProtocol[T]` (Generic):
  - [x] `async def get(self, id: UUID) -> T | None`
  - [x] `async def save(self, entity: T, version: int | None = None) -> T` (OCC support)
  - [x] `async def delete(self, id: UUID) -> None`
  - [x] `async def exists(self, id: UUID) -> bool`
  - [x] `async def count(self, filters: list[FilterSpec] | None = None) -> int`
  - [x] `async def list(self, filters: list[FilterSpec] | None, order_by: list[OrderSpec] | None, limit: int, offset: int) -> PaginatedResult[T]`
  - [x] `async def bulk_save(self, entities: list[T]) -> list[T]`
- [x] Add type hints with `Generic[T]` and `Protocol`

### 2.2 Event Bus Protocol

- [x] Create `src/framework_m/core/interfaces/event_bus.py`
- [x] Define `Event` base model with `id`, `timestamp`, `source`, `type`, `data`
- [x] Define `EventBusProtocol` with methods:
  - [x] `async def connect(self) -> None` (for NATS/Kafka)
  - [x] `async def disconnect(self) -> None`
  - [x] `def is_connected(self) -> bool`
  - [x] `async def publish(self, topic: str, event: Event) -> None`
  - [x] `async def subscribe(self, topic: str, handler: Callable[[Event], Awaitable[None]]) -> str` (returns subscription_id)
  - [x] `async def subscribe_pattern(self, pattern: str, handler: Callable) -> str` (e.g., `doc.*`)
  - [x] `async def unsubscribe(self, subscription_id: str) -> None`

### 2.3 Auth Context Protocol

- [x] Create `src/framework_m/core/interfaces/auth_context.py`
- [x] Define `UserContext` Pydantic model with fields:
  - [x] `id: str`
  - [x] `email: str`
  - [x] `roles: list[str]`
  - [x] `tenants: list[str]`
- [x] Define `AuthContextProtocol` with methods:
  - [x] `async def get_current_user() -> UserContext`

### 2.4 Storage Protocol

- [x] Create `src/framework_m/core/interfaces/storage.py`
- [x] Define `FileMetadata` model with `path`, `size`, `content_type`, `modified`, `etag`
- [x] Define `StorageProtocol` with methods:
  - [x] `async def save_file(self, path: str, content: bytes, content_type: str | None = None) -> str`
  - [x] `async def get_file(self, path: str) -> bytes`
  - [x] `async def delete_file(self, path: str) -> None`
  - [x] `async def list_files(self, prefix: str) -> list[str]`
  - [x] `async def get_metadata(self, path: str) -> FileMetadata | None`
  - [x] `async def get_url(self, path: str, expires: int = 3600) -> str` (presigned URL for S3)
  - [x] `async def copy(self, src: str, dest: str) -> str`
  - [x] `async def move(self, src: str, dest: str) -> str`

### 2.5 Job Queue Protocol

- [x] Create `src/framework_m/core/interfaces/job_queue.py`
- [x] Define `JobStatus` enum: `PENDING`, `RUNNING`, `SUCCESS`, `FAILED`, `CANCELLED`
- [x] Define `JobInfo` model with `id`, `name`, `status`, `enqueued_at`, `started_at`, `result`, `error`
- [x] Define `JobQueueProtocol` with methods:
  - [x] `async def enqueue(self, job_name: str, **kwargs) -> str` (returns job_id)
  - [x] `async def schedule(self, job_name: str, cron: str, **kwargs) -> str`
  - [x] `async def cancel(self, job_id: str) -> bool`
  - [x] `async def get_status(self, job_id: str) -> JobInfo | None`
  - [x] `async def retry(self, job_id: str) -> str` (returns new job_id)

### 2.6 Permission Protocol

- [x] Create `src/framework_m/core/interfaces/permission.py`
- [x] Define `PermissionProtocol` with methods:
  - [x] `async def has_permission(user: UserContext, doctype: str, action: str, doc_id: str | None) -> bool`
  - [x] `async def get_permitted_filters(user: UserContext, doctype: str) -> dict`
- [x] Add action types: `"read"`, `"write"`, `"create"`, `"delete"`, `"submit"`

### 2.7 Print Protocol

- [x] Create `src/framework_m/core/interfaces/print.py`
- [x] Define `PrintProtocol` with methods:
  - [x] `async def render(doc: BaseModel, template: str, format: str = "pdf") -> bytes`

### 2.8 Cache Protocol

- [x] Create `src/framework_m/core/interfaces/cache.py`
- [x] Define `CacheProtocol`:
  - [x] `async def get(self, key: str) -> Any | None`
  - [x] `async def set(self, key: str, value: Any, ttl: int | None = None) -> None`
  - [x] `async def delete(self, key: str) -> None`
  - [x] `async def exists(self, key: str) -> bool`
  - [x] `async def get_many(self, keys: list[str]) -> dict[str, Any]`
  - [x] `async def set_many(self, items: dict[str, Any], ttl: int | None = None) -> None`
  - [x] `async def delete_pattern(self, pattern: str) -> int` (returns count deleted)
  - [x] `async def ttl(self, key: str) -> int | None` (remaining TTL)

### 2.9 Notification Protocol

- [x] Create `src/framework_m/core/interfaces/notification.py`
- [x] Define `NotificationProtocol`:
  - [x] `async def send_email(to: str, subject: str, body: str)`
  - [x] `async def send_sms(to: str, body: str)`

### 2.10 Search Protocol

- [x] Create `src/framework_m/core/interfaces/search.py`
- [x] Define `SearchResult` model with `items`, `total`, `facets`, `highlights`
- [x] Define `SearchProtocol`:
  - [x] `async def index(self, doctype: str, doc_id: str, doc: dict) -> None`
  - [x] `async def delete_index(self, doctype: str, doc_id: str) -> None`
  - [x] `async def search(self, doctype: str, query: str, filters: dict | None = None, limit: int = 20, offset: int = 0) -> SearchResult`
  - [x] `async def reindex(self, doctype: str) -> int` (returns count indexed)

### 2.11 I18n Protocol

- [x] Create `src/framework_m/core/interfaces/i18n.py`
- [x] Define `I18nProtocol`:
  - [x] `async def translate(text: str, locale: str) -> str`
  - [x] `async def get_locale() -> str`

---

## 3. Define Domain Layer

### 3.1 Base DocType

- [x] Create `src/framework_m/core/domain/base_doctype.py`
- [x] Define `BaseDocType` class inheriting from `pydantic.BaseModel`
- [x] Add standard fields:
  - [x] `name: Optional[str]` (primary key, auto-generated if None)
  - [x] `creation: datetime`
  - [x] `modified: datetime`
  - [x] `modified_by: Optional[str]`
  - [x] `owner: Optional[str]`
- [x] Add `Meta` nested class for metadata:
  - [x] `layout: dict = {}` (via `get_layout()`)
  - [x] `permissions: dict = {}` (via `get_permissions()`)
- [x] Add class method `get_doctype_name() -> str`

### 3.2 Base Controller

- [x] Create `src/framework_m/core/domain/base_controller.py`
- [x] Define `BaseController[T]` generic class
- [x] Add lifecycle hook methods:
  - [x] `async def validate(self, context: Any = None) -> None`
  - [x] `async def before_insert(self, context: Any = None) -> None`
  - [x] `async def after_insert(self, context: Any = None) -> None`
  - [x] `async def before_save(self, context: Any = None) -> None`
  - [x] `async def after_save(self, context: Any = None) -> None`
  - [x] `async def before_delete(self, context: Any = None) -> None`
  - [x] `async def after_delete(self, context: Any = None) -> None`
  - [x] `async def on_submit(self, context: Any = None) -> None`
  - [x] `async def on_cancel(self, context: Any = None) -> None`

### 3.3 Mixins

- [x] Create `src/framework_m/core/domain/mixins.py`
- [x] Define `DocStatus` enum:
  - [x] `DRAFT = 0`
  - [x] `SUBMITTED = 1`
  - [x] `CANCELLED = 2`
- [x] Define `SubmittableMixin` class with:
  - [x] `docstatus: DocStatus = DocStatus.DRAFT`
  - [x] `def is_submitted() -> bool`
  - [x] `def is_cancelled() -> bool`
  - [x] `def can_edit() -> bool` (returns False if submitted)

---

## 4. Meta Registry

- [x] Create `src/framework_m/core/registry.py`
- [x] Implement `MetaRegistry` as singleton
- [x] Add storage dictionaries:
  - [x] `_doctypes: Dict[str, Type[BaseDocType]]`
  - [x] `_controllers: Dict[str, Type[BaseController]]`
- [x] Implement methods:
  - [x] `register_doctype(doctype_class, controller_class=None)`
  - [x] `get_doctype(name: str) -> Type[BaseDocType]`
  - [x] `get_controller(name: str) -> Type[BaseController] | None`
  - [x] `list_doctypes() -> list[str]`
  - [x] `discover_doctypes(package_name: str)` (scans for BaseDocType subclasses)
  - [x] **Load Order**: Follows `installed_apps` list.
  - [x] **Conflict Policy**: Raise `DuplicateDocTypeError` if same name registered twice.

---

## 5. Port Implementation (Adapters)

### 5.1 Dependency Injection

- [x] Create `tests/core/test_container.py`
- [x] Write test for Container initialization and service resolution
- [x] Create `src/framework_m/core/container.py`
- [x] Implement `Container` class (using `dependency_injector`):
  ```python
  from dependency_injector import containers, providers
  ```

- [x] Define `Container` class:
  ```python
  class Container(containers.DeclarativeContainer):
      # Configuration
      config = providers.Configuration()

      # Core services (will be populated in later phases)
      # database = providers.Singleton(...)
      # repository_factory = providers.Factory(...)
  ```

### 5.2 Provider Types

- [x] Understand provider types:
  - [x] `Singleton` - Single instance shared across app
  - [x] `Factory` - New instance each time
  - [x] `Resource` - For resources with lifecycle (db connections)
  - [x] `Callable` - For functions
  - [x] `Dependency` - For protocol injection

### 5.3 Configuration Provider

- [x] Setup configuration loading:
  ```python
  config = providers.Configuration()
  config.from_pydantic(Settings())  # Pydantic settings
  config.from_env("APP", as_=str)   # Environment variables
  ```

### 5.4 Wiring

- [x] Configure wiring for automatic injection:
  ```python
  class Container(containers.DeclarativeContainer):
      wiring_config = containers.WiringConfiguration(
          modules=[
              "framework_m.adapters.web",
              "framework_m.adapters.db",
          ]
      )
  ```

- [x] Use `@inject` decorator in functions:
  ```python
  from dependency_injector.wiring import inject, Provide

  @inject
  async def my_function(
      repo: RepositoryProtocol = Provide[Container.repository]
  ):
      # repo is automatically injected
      pass
  ```

### 5.5 Entrypoint Scanning for Overrides

- [x] Create override mechanism:
  - [x] Scan `framework_m.overrides` entrypoints
  - [x] Allow apps to override providers
  - [x] Example:
    ```python
    # In app's pyproject.toml
    [project.entry-points."framework_m.overrides"]
    repository = "my_app.custom:CustomRepository"
    ```

- [x] Implement override loader:
  ```python
  def load_overrides(container: Container):
      for entry_point in entry_points(group="framework_m.overrides"):
          provider_name = entry_point.name
          override_class = entry_point.load()
          # Override the provider
          getattr(container, provider_name).override(override_class)
  ```

### 5.6 Testing Support

- [x] Use `override` for testing:
  ```python
  # In tests
  container = Container()
  container.repository.override(MockRepository())
  ```

- [x] Reset overrides after tests:
  ```python
  container.reset_singletons()
  container.unwire()
  ```

---

## 6. Testing Setup

- [x] Create `tests/` directory structure
  ```
  tests/
  ├── unit/
  ├── integration/
  └── conftest.py
  ```

- [x] Setup `conftest.py` with fixtures:
  - [x] `pytest_asyncio` configuration
  - [x] Mock implementations of all protocols

- [x] Write initial tests:
  - [x] Test `MetaRegistry` registration and retrieval
  - [x] Test `BaseDocType` field validation
  - [x] Test `BaseController` hook method existence

---

## 7. Documentation

- [x] Create `README.md` with:
  - [x] Project overview
  - [x] Installation instructions
  - [x] Basic usage example

- [x] Create `docs/` folder with:
  - [x] `architecture.md` - Hexagonal architecture explanation
  - [x] `ports.md` - List of all protocol interfaces (Repository, EventBus, Cache, Search, etc.)

---

## Validation Checklist

Before moving to Phase 02, verify:

- [x] All protocol interfaces are defined with proper type hints
- [x] `mypy --strict` passes with no errors
- [x] No infrastructure dependencies in `core/` (no imports of litestar, sqlalchemy, redis)
- [x] `BaseDocType` and `BaseController` are properly generic
- [x] `MetaRegistry` can register and retrieve DocTypes
- [x] All tests pass with `pytest`

---

## Anti-Patterns to Avoid (Learning from Frappe)

❌ **Don't**: Use global state like `frappe.db` or `frappe.session`
✅ **Do**: Use dependency injection to pass context

❌ **Don't**: Import infrastructure libraries in domain layer
✅ **Do**: Define protocols and inject implementations

❌ **Don't**: Use monkey patching for extensibility
✅ **Do**: Use DI and entrypoints for overrides

❌ **Don't**: Mix business logic with database queries
✅ **Do**: Keep controllers focused on business rules, repositories handle data

❌ **Don't**: Write code without tests
✅ **Do**: Follow TDD. 95% Coverage goal.


Document: frontend/docs/i18n-workflow.md

# Translation Workflow

This document describes the i18n workflow for Framework M frontend.

## Architecture

- **Backend**: Translation DocType stores all translations in the database
- **Frontend**: i18next with HttpBackend fetches translations from API
- **Fallback**: Local JSON files used if backend is unavailable

## Components

### 1. useTranslation Hook

```tsx
import { useTranslation } from "./hooks/useTranslation";

function MyComponent() {
  const { t, language, changeLanguage } = useTranslation();

  return (
    <div>
      <h1>{t("Welcome")}</h1>
      <p>{t("Hello, {{name}}!", { name: "John" })}</p>
      <button onClick={() => changeLanguage("hi")}>हिन्दी</button>
    </div>
  );
}
```

### 2. Trans Component

For complex interpolations with React components:

```tsx
import { Trans } from "react-i18next";

function MyComponent() {
  return (
    <Trans i18nKey="welcomeMessage">
      Welcome to <strong>Framework M</strong>, {{ username }}!
    </Trans>
  );
}
```

### 3. LocaleSwitcher

Available in Navbar for language selection:

- English (en)
- हिन्दी (hi)
- தமிழ் (ta)
- Español (es)
- Français (fr)

## Workflow

### 1. Extract Strings

Extract translation keys from all components:

```bash
pnpm extract-translations
```

This runs `i18next-parser` which:

- Scans all `.ts` and `.tsx` files
- Extracts `t()` and `<Trans>` calls
- Generates JSON files in `locales/[locale]/common.json`

### 2. Upload to Backend

Upload extracted translations to the Translation DocType:

```bash
pnpm upload-translations
```

This script:

- Reads all locale JSON files
- POSTs each translation to `/api/v1/Translation`
- Skips duplicates (409 Conflict)

### 3. Manage Translations

Use the Translation DocType in Framework M to:

- Add new translations
- Edit existing translations
- Add context-specific translations (e.g., "button.Save" vs "menu.Save")

### 4. Download Merged Translations

Download backend translations merged with local files:

```bash
pnpm download-translations
```

This script:

- Fetches all translations from backend
- Merges with local JSON files (local takes precedence)
- Saves merged translations back to `locales/`

## Translation Keys

### Context Prefixes

Use context prefixes for disambiguation:

```tsx
// Button context
t("button.Save"); // "Save" button
t("button.Cancel"); // "Cancel" button

// Menu context
t("menu.Save"); // "Save" menu item
t("menu.Cancel"); // "Cancel" menu item

// Validation context
t("validation.required"); // "This field is required"
```

### Interpolation

Use `{{variable}}` for dynamic values:

```tsx
t("Hello, {{name}}!", { name: user.name });
t("{{count}} items", { count: items.length });
```

### Pluralization

Use plural forms:

```tsx
t("item", { count: 1 }); // "1 item"
t("item", { count: 5 }); // "5 items"
```

## Backend API

### Fetch Translations

```http
GET /api/v1/Translation?filters=locale:eq:hi&limit=1000
```

Response:

```json
{
  "items": [
    {
      "source_text": "Welcome",
      "translated_text": "स्वागत है",
      "locale": "hi",
      "context": null
    }
  ],
  "total": 1,
  "limit": 1000,
  "offset": 0
}
```

### Create Translation

```http
POST /api/v1/Translation
Content-Type: application/json
x-user-id: admin

{
  "source_text": "Welcome",
  "translated_text": "स्वागत है",
  "locale": "hi",
  "context": "greeting"
}
```

## Configuration

### i18next Config (`src/config/i18n.ts`)

```typescript
i18n
  .use(HttpBackend)
  .use(LanguageDetector)
  .use(initReactI18next)
  .init({
    fallbackLng: "en",
    backend: {
      loadPath: "http://localhost:8000/api/v1/Translation?locale={{lng}}",
      parse: data => {
        // Transform backend response to i18next format
      },
    },
  });
```

### i18next-parser Config (`i18next-parser.config.js`)

```javascript
export default {
  locales: ["en", "hi", "ta", "es", "fr"],
  output: "locales/$LOCALE/$NAMESPACE.json",
  input: ["src/**/*.{ts,tsx}"],
};
```

## Best Practices

1. **Always use keys**: Never hardcode user-facing strings

   ```tsx
   // ❌ Bad
   <h1>Welcome</h1>

   // ✅ Good
   <h1>{t("Welcome")}</h1>
   ```

2. **Use context for disambiguation**:

   ```tsx
   t("button.Save"); // Save button
   t("menu.Save"); // Save menu item
   ```

3. **Extract regularly**: Run `pnpm extract-translations` after adding new strings

4. **Test all locales**: Switch languages to verify translations work

5. **Lazy loading**: i18next loads translations on-demand (configured with HttpBackend)

## Troubleshooting

### Translations not loading

1. Check backend is running: `http://localhost:8000/api/v1/Translation`
2. Check browser console for CORS errors
3. Verify Translation DocType has `api_resource = True`

### Missing translations

1. Run `pnpm extract-translations` to extract new keys
2. Add translations to Translation DocType via UI or API
3. Reload page to fetch latest translations

### Locale not switching

1. Check LocaleSwitcher is in Navbar
2. Verify `localStorage.getItem('i18nextLng')` changes
3. Check i18next debug logs in console (dev mode)


Document: frontend/docs/dark-mode-implementation.md

# Dark Mode Implementation

## Overview

Complete dark mode implementation across all pages and components using CSS custom properties (CSS variables) for theme switching.

## Architecture

### Theme Context

- **Location**: `src/contexts/ThemeContext.tsx`
- **State Management**: React Context with localStorage persistence
- **Key**: `framework-m-theme` (stores "light" or "dark")
- **Features**:
  - System preference detection via `matchMedia("prefers-color-scheme: dark")`
  - Automatic theme application on mount
  - Theme toggle function
  - Class-based theme switching (`.dark` class on `documentElement`)

### CSS Variables

- **Location**: `src/App.css`
- **Structure**: Two sets of variables (`:root` for light, `.dark` for dark mode)

#### Core Color Variables

**Light Mode** (`:root`):

```css
--color-bg: #ffffff --color-bg-secondary: #f9fafb --color-bg-tertiary: #f3f4f6
  --color-border: #e5e7eb --color-text: #0f172a --color-text-secondary: #64748b
  --color-text-muted: #94a3b8 --color-primary: #1890ff --color-success: #52c41a
  --color-warning: #faad14 --color-error: #ff4d4f --color-error-bg: #fff2f0
  --color-error-border: #ffccc7;
```

**Dark Mode** (`.dark`):

```css
--color-bg: #0f172a --color-bg-secondary: #1e293b --color-bg-tertiary: #334155
  --color-border: #334155 --color-text: #f1f5f9 --color-text-secondary: #cbd5e1
  --color-text-muted: #94a3b8 --color-primary: #38bdf8 --color-success: #22c55e
  --color-warning: #f59e0b --color-error: #ef4444
  --color-error-bg: rgba(239, 68, 68, 0.1)
  --color-error-border: rgba(239, 68, 68, 0.3);
```

#### Sidebar-Specific Variables

**Light Mode**:

```css
--color-sidebar-bg: #001529 --color-sidebar-text: #ffffff
  --color-sidebar-text-muted: rgba(255, 255, 255, 0.65)
  --color-sidebar-search-bg: rgba(255, 255, 255, 0.1)
  --color-sidebar-item-active-bg: rgba(24, 144, 255, 0.3);
```

**Dark Mode**:

```css
--color-sidebar-bg: #020617 --color-sidebar-text: #e2e8f0
  --color-sidebar-text-muted: #94a3b8
  --color-sidebar-search-bg: rgba(255, 255, 255, 0.05)
  --color-sidebar-item-active-bg: rgba(56, 189, 248, 0.2);
```

#### Shadow Variables

**Light Mode**:

```css
--shadow-sm: 0 1px 2px 0 rgb(0 0 0 / 0.05) --shadow-md: 0 4px 6px -1px
  rgb(0 0 0 / 0.1);
```

**Dark Mode**:

```css
--shadow-sm: 0 1px 2px 0 rgb(0 0 0 / 0.3) --shadow-md: 0 4px 6px -1px
  rgb(0 0 0 / 0.4);
```

## Theme Toggle UI

### Location

- Moved from standalone navbar button to user menu dropdown
- Positioned above the logout button

### Implementation

```tsx
// In Navbar.tsx user menu dropdown
<button
  onClick={toggleTheme}
  style={{
    display: "flex",
    alignItems: "center",
    gap: "0.5rem",
    // ... styles using CSS variables
  }}
>
  {theme === "dark" ? "☀" : "🌙"}
  {theme === "dark" ? "Light" : "Dark"} Mode
</button>
```

## Updated Files

### Pages

1. **Dashboard.tsx**
   - Main container: `backgroundColor: "var(--color-bg-secondary)"`
   - Section cards: `backgroundColor: "var(--color-bg)"`, `border: "1px solid var(--color-border)"`
   - Text colors: `color: "var(--color-text)"`, `color: "var(--color-text-secondary)"`
   - Notification backgrounds: Dynamic based on type, using CSS variables
   - Hover states: `backgroundColor: "var(--color-bg-tertiary)"`

2. **ListView.tsx**
   - Loading/error states: Background and text colors
   - Main container: Full-height background
   - Search input: Background, border, and text colors
   - Action buttons: Primary, error, and secondary button colors
   - Bulk action indicators: Text colors

3. **FormView.tsx**
   - Loading/error states: Background and text colors
   - Form container: Background color
   - Validation error box: Error background and border
   - Action buttons: Primary and secondary colors
   - Border separators: Border color

### Layout Components

4. **Navbar.tsx**
   - Header background: `var(--color-bg)`
   - Search input: Background and text colors
   - User menu: Background, border, and text colors
   - Theme toggle: Moved to user dropdown
   - All icon colors: `var(--color-text-secondary)`
   - Hover effects: `var(--color-bg-secondary)`

5. **Sidebar.tsx**
   - Container: `var(--color-sidebar-bg)`, `var(--color-sidebar-text)`
   - Header border: `var(--color-border)`
   - Search input: `var(--color-sidebar-search-bg)`
   - Group headers: `var(--color-sidebar-text-muted)`
   - Menu items: Active state with `var(--color-sidebar-item-active-bg)`
   - Border indicators: `var(--color-primary)`

### UI Components

6. **LocaleSwitcher.tsx**
   - Select dropdown: Background, border, and text colors
   - Label color: `var(--color-text)`

7. **CommandPalette.tsx**
   - Modal container: Background and border
   - Backdrop: Darkened overlay
   - Search input: Background and text colors
   - Result items: Background on hover
   - Type badges: Tertiary background
   - All text: Appropriate text color variables

8. **BulkActionsBar.tsx**
   - Bar background: `var(--color-bg-secondary)`
   - Border: `var(--color-border)`
   - Delete button: `var(--color-error)`
   - Text colors: `var(--color-text)`

9. **EditableCell.tsx**
   - Input borders: Error vs primary colors
   - Input backgrounds: `var(--color-bg)`
   - Error tooltip: Error background and border
   - Text colors: `var(--color-text)`

## Usage

### For Users

1. Click on user avatar in navbar
2. Click "🌙 Dark Mode" to switch to dark mode
3. Click "☀ Light Mode" to switch back to light mode
4. Theme preference is automatically saved and persisted

### For Developers

#### Using CSS Variables in Components

```tsx
// Always use CSS variables instead of hardcoded colors
style={{
  backgroundColor: "var(--color-bg)",
  color: "var(--color-text)",
  border: "1px solid var(--color-border)",
}}
```

#### Adding New Colors

1. Add to both `:root` and `.dark` selectors in `App.css`
2. Follow naming convention: `--color-{purpose}-{variant}`
3. Ensure sufficient contrast in both modes

#### Testing Dark Mode

1. Toggle theme in UI
2. Check all pages: Dashboard, ListView, FormView
3. Verify text readability
4. Check border and shadow visibility
5. Test interactive states (hover, active, focus)

## Benefits

1. **Consistency**: Single source of truth for colors
2. **Maintainability**: Update colors globally from CSS
3. **Performance**: Native CSS feature, no runtime JS cost
4. **Accessibility**: Respects system preference
5. **User Choice**: Manual override with persistence

## Future Enhancements

- [ ] Additional theme options (e.g., high contrast)
- [ ] Per-component theme customization
- [ ] Automatic theme scheduling (day/night)
- [ ] Theme preview before applying
- [ ] Export/import custom themes