Document: docs/adr/0011-modular-decentralized-migrations.md

# ADR 0011: Modular and Decentralized Migration System

## Status
Accepted

## Context
Framework M aims to support "Modular Monoliths" and high-scale distributed applications. Developers coming from Frappe expect a "Declarative/Sync" experience (where the database matches the code), while enterprise production environments require "Imperative/Migration" control (where patches are ordered and verified).

Previously, migrations were centralized in a single `alembic/versions` directory, which made it difficult to manage migrations for third-party or modular apps.

## Decision
We decided to implement a **Dual-Track, Decentralized Migration System**.

### 1. Dual-Track Strategy
- **Sync Track (`m migrate sync`)**: A declarative auto-sync mechanism that compares current DocTypes with the database schema and applies changes directly. Supports `before_sync` and `after_sync` hooks in each app.
- **Migrate Track (`m migrate run`)**: An imperative version-based system using Alembic. Each app maintains its own `alembic/versions` directory.
- **Unified Command (`m migrate all`)**: A safe default that runs all pending migrations (run track) followed by the declarative sync.

### 2. Decentralized Version Locations
Alembic is configured to scan multiple `version_locations` by merging paths from all `INSTALLED_APPS`. Each app's `migrations/versions` (or `alembic/versions`) folder is automatically included.

### 3. Smart App Discovery
The system uses `importlib.metadata` entry points (group `framework_m.apps`) to discover modular apps. This eliminates the need for manual registration in a central configuration file.

## Consequences
- **Pros**:
    - "It just works" experience for developers via `m migrate sync`.
    - Production safety via `m migrate run` for data patches.
    - True modularity: Apps are self-contained with their own migrations.
    - Zero-downtime deployment patterns are natively supported.
- **Cons**:
    - Increased complexity in the `env.py` and `MigrationManager` orchestration.
    - Developers must understand the difference between a "sync" and a "run".

## References
- [Developer Guide: Migrations](file:///home/revant/Projects/framework/m/docs/developer/migrations.md)
- [RFC-0002: DocType Discovery](file:///home/revant/Projects/framework/m/docs/rfcs/rfc-0002-doctype-discovery.md)
- [RFC-0003: Schema Change Detection](file:///home/revant/Projects/framework/m/docs/rfcs/rfc-0003-schema-change-detection.md)


Document: docs/adr/0001-taskiq-nats-for-jobs.md

# ADR-0001: Use NATS JetStream with Taskiq for Background Jobs

- **Status**: Accepted
- **Date**: 2025-12-29
- **Deciders**: @anshpansuriya14
- **Supersedes**: N/A
- **Superseded by**: N/A

## Context

Framework M needs a reliable background job system for:
- Sending emails/notifications asynchronously
- Processing webhooks
- Running scheduled tasks (cron jobs)
- Handling long-running operations outside HTTP request lifecycle

**Forces at play:**
- **Cross-platform**: Must work on Windows, macOS, Linux (unlike Celery which has Windows issues)
- **Persistence**: Jobs must survive restarts (at-least-once delivery)
- **Observability**: Need job status tracking and retry logic
- **Python Ecosystem**: Integration with async/await patterns
- **Dev Experience**: Simple decorator-based job definition

## Decision

> We will use **Taskiq** with **NATS JetStream** as the job queue backend.

```python
from taskiq_nats import PullBasedJetStreamBroker

broker = PullBasedJetStreamBroker(
    servers=["nats://localhost:4222"],
    stream="framework_m_jobs",
)

@broker.task(name="send_email", retry=3)
async def send_email(to: str, subject: str, body: str):
    # Job logic
    pass
```

## Consequences

### Positive

- **Cross-platform**: NATS runs on Windows, macOS, Linux natively
- **Persistent**: JetStream provides durable message storage and replay
- **Modern Architecture**: Native async/await, works well with Litestar
- **Single Infrastructure**: NATS handles both jobs AND events (Section 4.1)
- **Taskiq Philosophy**: Clean decorator-based API similar to Celery but modern

### Negative

- **Learning Curve**: NATS is less known than Redis/Celery
- **Operational Complexity**: Requires running NATS server
- **Ecosystem Size**: Smaller than Celery ecosystem

### Neutral

- Uses gRPC-style pull-based consumers (efficient for high throughput)
- Supports scheduled jobs via `TaskiqScheduler`

## Alternatives Considered

| Option | Pros | Cons |
|--------|------|------|
| **Chosen: Taskiq + NATS** | Cross-platform, persistent, modern async | Smaller ecosystem |
| ARQ + Redis | Python native, simple | No Windows support, less features |
| Celery + Redis | Most popular, mature | Windows issues, sync-first design |
| Dramatiq + Redis | Modern API | Limited scheduling, Redis dependency |
| Temporal | Enterprise-grade workflows | Heavy, separate system |

## References

- [Taskiq Documentation](https://taskiq-python.github.io/)
- [taskiq-nats](https://github.com/taskiq-python/taskiq-nats)
- [NATS JetStream](https://docs.nats.io/nats-concepts/jetstream)
- Related: [ADR-0002](./0002-nats-for-event-bus.md)


Document: docs/adr/0002-nats-for-event-bus.md

# ADR-0002: Use NATS JetStream for Event Bus

- **Status**: Accepted
- **Date**: 2025-12-29
- **Deciders**: @anshpansuriya14
- **Supersedes**: N/A
- **Superseded by**: N/A

## Context

Framework M needs an event-driven architecture for:
- Domain events (`doc.created`, `doc.updated`, `doc.deleted`)
- Webhook triggers
- Audit logging
- Real-time notifications
- Decoupled service communication

**Forces at play:**
- **Reliability**: Events must not be lost
- **Replay Capability**: New subscribers should access historical events
- **Pattern Matching**: Subscribe to `doc.*` or `todo.created`
- **Consistency**: Same infrastructure for jobs and events
- **Performance**: High throughput, low latency

## Decision

> We will use **NATS JetStream** via `nats-py` for the event bus.

```python
from nats.aio.client import Client

async def publish_event(nc: Client, topic: str, event: BaseEvent):
    await nc.publish(topic, event.model_dump_json().encode())

async def subscribe_events(nc: Client, pattern: str, handler):
    await nc.subscribe(pattern, cb=handler)
```

## Consequences

### Positive

- **Shared Infrastructure**: Same NATS cluster for jobs (ADR-0001) and events
- **JetStream Durability**: Messages persisted, consumers can replay from any point
- **Wildcard Subscriptions**: `doc.*`, `todo.>` patterns for flexible routing
- **At-Least-Once**: Acknowledgment-based delivery prevents message loss
- **Cross-Language**: NATS has clients for Go, Rust, JS (future microservices)

### Negative

- **Complexity**: JetStream streams require configuration (retention, limits)
- **Ordering**: Partitioned ordering (by key) not global order
- **Learning Curve**: Different model than Redis Pub/Sub

### Neutral

- Messages use CloudEvents format for interoperability
- Consumer groups supported via queue subscriptions

## Alternatives Considered

| Option | Pros | Cons |
|--------|------|------|
| **Chosen: NATS JetStream** | Durable, replay, shared with jobs | Learning curve |
| Redis Pub/Sub | Simple, fast | No persistence, no replay |
| Redis Streams | Persistent, consumer groups | No pattern matching, Redis-only |
| Kafka | Enterprise-grade, proven | Heavy, operational complexity |
| RabbitMQ | Mature, flexible routing | Separate system from jobs |

## References

- [NATS JetStream Docs](https://docs.nats.io/nats-concepts/jetstream)
- [nats-py](https://github.com/nats-io/nats.py)
- [CloudEvents Specification](https://cloudevents.io/)
- Related: [ADR-0001](./0001-taskiq-nats-for-jobs.md), [ADR-0003](./0003-nats-for-websocket-backplane.md)


Document: docs/adr/0010-multi-package-ui-composition.md

# ADR-0010: Multi-Package UI Composition Architecture

- **Status**: Accepted
- **Date**: 2026-02-19
- **Deciders**: @anshpansuriya14
- **Supersedes**: N/A
- **Superseded by**: N/A

## Context

**Current State:**

- Framework M bundles Desk UI in `framework-m/src/framework_m/static/`
- Apps use `framework-m/templates/frontend` to scaffold custom UIs
- `m build` supports plugin mode for apps in `apps/` directory only
- No mechanism for installed Python packages to contribute frontend code

**Desired Architecture:**

```
business-m (base package)
  ├── Python backend (reusable DocTypes, services)
  ├── Frontend base components (shared UI library)
  └── Entry point registration

├── wms-app (depends on business-m + framework-m-desk)
│   ├── WMS-specific DocTypes
│   ├── WMS-specific UI components
│   └── Extends business-m's UI

├── hr-app (depends on business-m + framework-m-desk)
│   ├── HR-specific DocTypes
│   ├── HR-specific UI components
│   └── Extends business-m's UI

├── crm-app (depends on business-m + framework-m-desk)
│   └── Similar structure

└── custom-app (depends on all above + framework-m-desk)
    ├── Composes/overrides existing apps
    ├── Adds new features
    └── Final integrated UI
```

## Current Architecture Analysis

### Backend (Working ✅)

**DocType Discovery via Entry Points:**

```python
# In app.py (libs/framework-m-standard/src/framework_m_standard/adapters/web/app.py:500-543)
from importlib.metadata import entry_points

eps = entry_points(group="framework_m.apps")
for ep in eps:
    ep.load()
    module_path = ep.module.split(":")[0]
    doctypes_package = f"{module_path}.doctypes"
    count = registry.discover_doctypes(doctypes_package)
```

**Entry Point Definition:**

```toml
# In pyproject.toml
[project.entry-points."framework_m.apps"]
my_app = "my_app:app"
```

### Frontend (Partially Working ⚠)

**Plugin Registry (frontend/src/plugins/registry.ts):**

```typescript
// Basic plugin system exists
registerPlugin({
  name: "custom_app",
  version: "1.0.0",
  fields: [...],
  slots: [...],
  pages: [...],
  resources: [...],
});
```

**Build System (apps/studio/src/framework_m_studio/cli/build.py):**

```python
# Plugin discovery ONLY in apps/ directory
apps_dir = Path.cwd() / "apps"
for app_dir in apps_dir.iterdir():
    plugin_entry = app_dir / "frontend" / "index.ts"
    if plugin_entry.exists():
        plugins.append((app_dir.name, plugin_entry))
```

**Gap:** No discovery of frontend code from installed Python packages!

## Problem Statement

### What's Missing:

1. **Frontend Entry Point System**: No Python entry point for frontend plugin discovery
2. **Multi-Package Frontend**: Can't install `business-m` via pip and have its UI automatically included
3. **UI Composition**: No way for multiple installed packages to contribute UI
4. **Build Discovery**: `m build` only scans `apps/` directory, not installed packages

### Why This Matters:

**Development Scenario:**

```bash
# Local development with editable installs
pip install -e ./libs/business-m
pip install -e ./libs/wms-app
pip install -e ./libs/hr-app
pip install -e ./libs/crm-app

# Run dev server - ALL UIs should be integrated
m dev

# Build for testing - ALL UIs should be bundled
m build
```

**Production Scenario (after CI/release):**

```bash
# Install released packages from PyPI
pip install business-m==1.0.0
pip install wms-app==1.0.0 hr-app==1.0.0 crm-app==1.0.0

# Build production - ALL UIs should be bundled
m build
```

**Currently:** Only works if apps are in `apps/` directory. Installed packages don't contribute UI.

**Desired:** Works like backend DocType discovery - scan entry points, compose all UIs (from both editable and installed packages).

## Decision

> Adopt **Option B: Single Bundle Composition** for multi-package frontend integration.

### Developer Experience (DevEx)

**Developers build UI per plugin and per base app independently:**

````bash
# CRM Package Developer (building standard crm-app for PyPI)
cd libs/crm-app
git clone https://github.com/framework-m/crm-app
cd crm-app/src/crm_app/frontend
# Develop standard CRM features
# - Lead management
# - Deal pipeline
# - Activity tracking
pip install -e .
m dev  # Develop with HMR
# Later: Publish to PyPI as crm-app==1.0.0

# Third-Party Developer (customer customizing CRM for their company)
# Option 1: Fork and customize
git clone https://github.com/acme-corp/custom-crm  # Fork of crm-app
cd custom-crm/src/custom_crm/frontend
# Customize for Acme Corp needs:
# - Edit pages/LeadForm.tsx - add company-specific fields
# - Edit pages/LeadList.tsx - custom kanban stages
# - Edit components/DealPipeline.tsx - custom approval workflow
pip install -e .
m dev  # Test customizations with HMR

# Option 2: Extension package (no fork)
mkdir -p libs/acme-crm-extensions/src/acme_crm_extensions/frontend
cd libs/acme-crm-extensions/src/acme_crm_extensions/frontend
# Create extension that OVERRIDES standard crm-app:
# - pages/LeadForm.tsx - wraps standard form with extra fields
# - No need to fork entire CRM codebase
pip install -e ./libs/acme-crm-extensions
pip install crm-app  # Standard CRM also installed
m dev  # Extension overrides run last, win over standard CRM

# Business-M Developer (building base package)
cd libs/business-m
# Develop frontend components, pages, DocTypes
# Work in isolation with their own package.json
# Can even run standalone dev server for testing components

# WMS Developer (building standard wms-app)
cd libs/wms-app
# Develop WMS-specific features
# Import types/components from business-m via TypeScript paths
# Work independently without affecting other developers

- ✅ Each package has its own `frontend/` directory
- ✅ Independent development with local `m dev` or standalone Vite
- ✅ TypeScript autocomplete works across packages via path aliases
- ✅ Git repos can be separate (business-m, wms-app, crm-app in different repos)
- ✅ Plugin registration is declarative (no build config needed)
- ✅ **Customize any app by editing its package** (e.g., fork crm-app, add custom UI)
- ✅ Override pattern: later registrations win (custom-app > crm-app > business-m)
- ✅ Slot injection: add features to other apps without forking them

### DevOps Experience (DevOpsEx)

**DevOps deploys main app + needed plugins with single build:**

```bash
# Production Dockerfile
FROM python:3.12

# Install backend packages (includes TypeScript source)
# Use customized CRM version instead of standard
RUN pip install framework-m business-m wms-app hr-app custom-crm-app

# Compose ALL frontend plugins into single optimized bundle
RUN m build --output /app/framework_m/static

# Result: Single bundle at /app/framework_m/static/
# - Contains Desk + business-m + wms + hr + custom-crm
# - CRM customizations included automatically
# - Tree-shaken, minified, code-split
# - No duplicate dependencies

# Start backend (serves bundled UI + API)
CMD ["uvicorn", "framework_m_standard.adapters.web.app:create_app", "--host", "0.0.0.0"]
````

**Key DevOpsEx Benefits:**

- ✅ Simple deployment: `pip install` + `m build`
- ✅ One command composes all plugins
- ✅ Single artifact to deploy (no coordination of multiple bundles)
- ✅ Automatic plugin discovery (no manual configuration)
- ✅ Smaller total size (tree-shaking across all packages)
- ✅ **Customize by swapping packages** (deploy `custom-crm-app` instead of `crm-app`)
- ✅ Version control: pin customized app versions independently

### Why Option B Works Best

**Architecture Alignment:**

- Current plugin registry designed for build-time composition ✅
- Backend serves ONE static directory (no multi-static routing) ✅
- Build system already generates temp entry files ✅
- No complex module federation needed ✅

**Performance:**

- Tree-shaking works across ALL packages
- No duplicate React/dependencies
- Code splitting by route (not by package)
- Smaller bundle size than separate builds

### Phase 1: Frontend Entry Point System

**Define new entry point group:**

```python
# Entry point group for frontend plugins
FRONTEND_PLUGINS_GROUP = "framework_m.frontend"
```

**Package registration:**

```toml
# business-m/pyproject.toml
[project.entry-points."framework_m.frontend"]
business_m = "business_m.frontend:plugin"
```

**Plugin definition:**

```typescript
// business-m/frontend/plugin.ts
export const plugin = {
  name: "business_m",
  version: "1.0.0",

  // Entry point for the frontend code
  entry: "./src/index.ts", // or pre-built bundle

  // Metadata for build system
  package: "business-m",
  source: "installed", // vs "local"
};
```

### Phase 2: Enhanced Build Discovery

**Update `_build_plugin_mode()` in build.py:**

```python
def _discover_all_plugins() -> list[tuple[str, Path, str]]:
    """Discover frontend plugins from:
    1. Installed Python packages (via entry points) - works for both:
       - Editable installs (pip install -e ./libs/business-m) for development
       - PyPI installs (pip install business-m) for production
    2. Local apps/ directory (legacy/convenience)

    Returns:
        List of (name, path, source) tuples
    """
    plugins = []

    # 1. Discover from installed packages (editable or PyPI)
    from importlib.metadata import entry_points

    eps = entry_points(group="framework_m.frontend")
    for ep in eps:
        try:
            plugin_info = ep.load()
            plugin_name = ep.name

            # Get the package's frontend directory
            import importlib.resources
            package_module = ep.module.split(":")[0]

            # Option A: Pre-bundled in package
            try:
                frontend_dist = importlib.resources.files(package_module) / "static" / "frontend"
                if frontend_dist.is_dir():
                    plugins.append((plugin_name, Path(str(frontend_dist)), "installed-bundled"))
                    continue
            except:
                pass

            # Option B: Source code in package (for development)
            try:
                frontend_src = importlib.resources.files(package_module) / "frontend"
                if frontend_src.is_dir():
                    entry_point = frontend_src / "index.ts"
                    if entry_point.is_file():
                        plugins.append((plugin_name, Path(str(entry_point)), "installed-source"))
                        continue
            except:
                pass

        except Exception as e:
            print(f"Warning: Failed to load frontend plugin '{ep.name}': {e}")

    # 2. Discover from local apps/ directory
    apps_dir = Path.cwd() / "apps"
    if apps_dir.exists():
        for app_dir in apps_dir.iterdir():
            if app_dir.is_dir():
                plugin_entry = app_dir / "frontend" / "index.ts"
                if plugin_entry.exists():
                    plugins.append((app_dir.name, plugin_entry, "local"))

    return plugins
```

### Phase 3: Build Composition Implementation

**Strategy: Single Bundle Composition**

All discovered plugins (from entry points + apps/) are composed into ONE optimized bundle at build time.

**Current Implementation Analysis:**

```python
# File: apps/studio/src/framework_m_studio/cli/build.py
# Lines: 216-286

def _build_plugin_mode(mode: str, output: Path) -> None:
    """Build plugin mode - Shadow Build with app plugins."""

    # LIMITATION: Only scans apps/ directory
    apps_dir = Path.cwd() / "apps"
    plugins: list[tuple[str, Path]] = []

    for app_dir in apps_dir.iterdir():
        if app_dir.is_dir():
            plugin_entry = app_dir / "frontend" / "index.ts"
            if plugin_entry.exists():
                plugins.append((app_dir.name, plugin_entry))

    # Generate temporary entry file
    temp_dir = Path.cwd() / ".m" / "build"
    entry_file = temp_dir / "entry.tsx"
    _generate_plugin_entry(entry_file, plugins)

    # Build with custom entry via env var
    env = {**os.environ, "VITE_ENTRY": str(entry_file)}
    subprocess.run([pkg_manager, "run", "build"], cwd=frontend_dir, env=env)
```

```python
# File: apps/studio/src/framework_m_studio/cli/build.py
# Lines: 289-313

def _generate_plugin_entry(entry_file: Path, plugins: list[tuple[str, Path]]) -> None:
    """Generate temporary entry.tsx that imports all app plugins."""

    imports = []
    registrations = []

    for app_name, plugin_path in plugins:
        relative_path = plugin_path.relative_to(Path.cwd())
        import_name = app_name.replace("-", "_").replace(".", "_")

        # Example output:
        # import { register as register_wms } from "../../apps/wms/frontend/index.ts";
        imports.append(
            f'import {{ register as register_{import_name} }} from "../../{relative_path}";'
        )
        registrations.append(f"  register_{import_name}();")

    # Generates:
    # export function registerAllPlugins() {
    #   register_wms();
    #   register_hr();
    # }
    # registerAllPlugins(); // Auto-register on import
```

**What Works ✅:**

- Temp entry generation
- Custom entry via `VITE_ENTRY` env var
- Auto-registration on import
- Single bundle output

**What Needs to Change 🔧:**

1. **Plugin Discovery** - Add entry point support:

   ```python
   def _discover_all_plugins() -> list[tuple[str, Path, str]]:
       """Discover plugins from BOTH sources:
       1. Entry points (installed packages via pip)
       2. Local apps/ directory (existing behavior)

       Returns: List of (name, path, source) tuples
       """
       plugins = []

       # NEW: Discover from installed packages
       from importlib.metadata import entry_points
       import importlib.resources

       eps = entry_points(group="framework_m.frontend")
       for ep in eps:
           package_module = ep.module.split(":")[0]
           frontend_src = importlib.resources.files(package_module) / "frontend"
           if frontend_src.is_dir():
               entry_file = frontend_src / "index.ts"
               if entry_file.is_file():
                   plugins.append((ep.name, Path(str(entry_file)), "installed-source"))

       # EXISTING: Scan apps/ directory
       apps_dir = Path.cwd() / "apps"
       if apps_dir.exists():
           for app_dir in apps_dir.iterdir():
               plugin_entry = app_dir / "frontend" / "index.ts"
               if plugin_entry.exists():
                   plugins.append((app_dir.name, plugin_entry, "local"))

       return plugins
   ```

2. **Entry Generation** - Handle package imports:

   ```python
   def _generate_plugin_entry(plugins: list[tuple[str, Path, str]]) -> str:
       """Generate entry with local AND package imports."""

       imports = []
       registrations = []

       for name, path, source in plugins:
           import_name = name.replace("-", "_").replace(".", "_")

           if source == "local":
               # EXISTING: Relative path for apps/
               relative_path = path.relative_to(Path.cwd())
               imports.append(
                   f'import {{ register as register_{import_name} }} '
                   f'from "../../{relative_path}";'
               )

           elif source == "installed-source":
               # NEW: Package alias import
               # Will be resolved via Vite config aliases
               imports.append(
                   f'import {{ register as register_{import_name} }} '
                   f'from "@{name}/frontend/index.ts";'
               )

           registrations.append(f"  register_{import_name}();")

       return content
   ```

3. **Vite Config** - Generate config with path aliases:

   ```python
   def _generate_vite_config(
       plugins: list[tuple[str, Path, str]],
       base_config: Path
   ) -> Path:
       """Generate vite.config.ts with package aliases."""

       # Read base vite.config.ts from frontend/
       base_content = base_config.read_text()

       # Generate aliases for installed packages
       aliases = {}
       for name, path, source in plugins:
           if source == "installed-source":
               # Map @business-m to filesystem path
               package_frontend_dir = path.parent
               aliases[f"@{name}"] = str(package_frontend_dir)

       # Inject aliases into config
       aliases_js = json.dumps(aliases, indent=6)

       config = f"""
       import {{ defineConfig }} from 'vite'
       // ... other imports ...

       export default defineConfig({{
         // ... other config ...
         resolve: {{
           alias: {aliases_js}
         }}
       }})
       """

       temp_config = Path(".m/build/vite.config.ts")
       temp_config.write_text(config)
       return temp_config
   ```

4. **Build Execution** - Use generated config:

   ```python
   def _build_plugin_mode(mode: str, output: Path) -> None:
       """Enhanced build with multi-package support."""

       plugins = _discover_all_plugins()  # NEW

       if not plugins:
           _build_indie_mode(mode, output)
           return

       print(f"Found {len(plugins)} plugins:")
       for name, path, source in plugins:
           print(f"  - {name}: {path} ({source})")

       temp_dir = Path(".m/build")
       temp_dir.mkdir(parents=True, exist_ok=True)

       # Generate entry
       entry_file = temp_dir / "entry.tsx"
       _generate_plugin_entry(entry_file, plugins)

       # Generate Vite config with aliases
       base_config = Path("frontend/vite.config.ts")
       vite_config = _generate_vite_config(plugins, base_config)  # NEW

       # Build with custom config AND entry
       subprocess.run([
           "pnpm", "exec", "vite", "build",
           "--config", str(vite_config),  # NEW
           "--outDir", str(temp_dir / "dist")
       ], cwd="frontend", env={"VITE_ENTRY": str(entry_file)})

       # Copy bundle to output
       shutil.copytree(temp_dir / "dist", output, dirs_exist_ok=True)
   ```

**Testing Strategy:**

```bash
# 1. Test with local apps (existing behavior)
m build  # Should work as before

# 2. Test with editable install
pip install -e ./libs/business-m
m build  # Should discover business-m plugin

# 3. Test with multiple packages
pip install -e ./libs/business-m ./libs/wms-app
m build  # Should compose both into single bundle

# 4. Verify bundle
ls -la libs/framework-m/src/framework_m/static/
# Should contain: index.html, assets/index-[hash].js, etc.
```

### Phase 4: Frontend Package Structure

**Standard structure for packages with frontend:**

```

business-m/
├── pyproject.toml # Python package config
├── src/
│ └── business_m/
│ ├── **init**.py
│ ├── doctypes/ # Backend DocTypes
│ │ ├── customer.py
│ │ └── invoice.py
│ └── frontend/ # Frontend code
│ ├── index.ts # Plugin entry point
│ ├── components/ # Shared components
│ │ ├── CustomerCard.tsx
│ │ └── InvoiceForm.tsx
│ ├── pages/ # Custom pages
│ └── package.json # Frontend dependencies

```

**pyproject.toml:**

```toml
[project]
name = "business-m"
version = "1.0.0"

[project.entry-points."framework_m.apps"]
business_m = "business_m:app"  # Backend entry point

[project.entry-points."framework_m.frontend"]
business_m = "business_m.frontend:plugin"  # Frontend entry point
```

**frontend/index.ts:**

```typescript
import { registerPlugin } from "@framework-m/core";
import { CustomerCard } from "./components/CustomerCard";
import { InvoiceForm } from "./components/InvoiceForm";
import { CustomerListPage } from "./pages/CustomerList";

export const plugin = {
  name: "business_m",
  version: "1.0.0",
};

// Register plugin on import
registerPlugin({
  name: "business_m",
  version: "1.0.0",

  fields: [{ name: "customer", component: CustomerCard, fieldTypes: ["Link"] }],

  pages: [
    {
      pageType: "list",
      doctype: "Customer",
      component: CustomerListPage,
    },
  ],

  // Slots for extensibility
  slots: [
    {
      slot: "customer-header",
      component: CustomerCard,
      doctypes: ["Customer"],
    },
  ],
});
```

### Phase 5: Composition & Override Pattern

**Customization in Option B works through plugin composition at build time.**

Each app can:

- Override pages/components from other apps
- Inject into slots defined by other apps
- Extend forms with custom fields
- Replace entire UI sections

**Example 1: WMS App extends Business-M**

```typescript
// libs/wms-app/src/wms_app/frontend/index.ts
import { registerPlugin } from "@framework-m/core";
import { WarehouseMap } from "./components/WarehouseMap";
import { StockLevelWidget } from "./components/StockLevel";

registerPlugin({
  name: "wms_app",
  version: "1.0.0",

  dependencies: ["business_m"], // Requires business-m plugin

  fields: [
    { name: "warehouse_map", component: WarehouseMap, fieldTypes: ["Data"] },
  ],

  slots: [
    // Inject into business-m's customer card
    {
      slot: "customer-header",
      component: StockLevelWidget,
      doctypes: ["Customer"],
      priority: 10, // Higher priority = renders first
    },
  ],
});
```

**Example 2: CRM App with Custom UI**

```typescript
// libs/crm-app/src/crm_app/frontend/index.ts
import { registerPlugin } from "@framework-m/core";
import { CustomLeadForm } from "./pages/LeadForm";
import { CustomLeadList } from "./pages/LeadList";
import { DealPipeline } from "./components/DealPipeline";
import { ActivityTimeline } from "./components/ActivityTimeline";

registerPlugin({
  name: "crm_app",
  version: "1.0.0",

  dependencies: ["business_m"],

  // CRM-specific pages (NEW doctypes)
  pages: [
    {
      pageType: "form",
      doctype: "Lead",
      component: CustomLeadForm, // Custom CRM form layout
    },
    {
      pageType: "list",
      doctype: "Lead",
      component: CustomLeadList, // Custom CRM list with kanban view
    },
  ],

  // Override business-m's Customer page with CRM version
  pages: [
    {
      pageType: "form",
      doctype: "Customer",
      component: CustomLeadForm, // CRM-specific customer form
      override: true, // ✅ Replaces business-m's form
    },
  ],

  // CRM-specific components
  fields: [
    { name: "deal_pipeline", component: DealPipeline, fieldTypes: ["Data"] },
    {
      name: "activity_timeline",
      component: ActivityTimeline,
      fieldTypes: ["Table"],
    },
  ],

  // Inject CRM features into other apps
  slots: [
    {
      slot: "customer-sidebar",
      component: ActivityTimeline,
      doctypes: ["Customer"],
      priority: 5,
    },
  ],
});
```

**Example 3: Custom App overrides everything**

```typescript
// custom-deployment/frontend/index.ts (deployed as apps/custom/frontend/index.ts)
import { registerPlugin } from "@framework-m/core";
import { CustomDashboard } from "./pages/Dashboard";
import { CustomHeader } from "./components/Header";

registerPlugin({
  name: "custom_deployment",
  version: "1.0.0",

  dependencies: ["business_m", "wms_app", "hr_app", "crm_app"],

  // Override CRM's lead list with company-specific version
  pages: [
    {
      pageType: "list",
      doctype: "Lead",
      component: CustomDashboard, // Company-specific dashboard
      override: true, // ✅ Replaces CRM's lead list
    },
  ],

  // Replace global header
  slots: [
    {
      slot: "app-header",
      component: CustomHeader,
      priority: 100, // Highest priority = renders first
    },
  ],
});
```

**How Customization Works:**

1. **Registration Order:**

   ```typescript
   // At build time, plugins register in dependency order:
   registerPlugin({ name: "business_m", ... });     // 1st
   registerPlugin({ name: "crm_app", ... });        // 2nd
   registerPlugin({ name: "custom_deployment", ... }); // 3rd (can override)
   ```

2. **Override Resolution:**

   ```typescript
   // In frontend/src/plugins/registry.ts
   export function registerPlugin(plugin: Plugin): void {
     if (plugin.pages) {
       for (const page of plugin.pages) {
         const key = `${page.pageType}:${page.doctype}`;

         if (page.override) {
           // Replace existing page
           registry.pages.set(key, page);
         } else if (!registry.pages.has(key)) {
           // Only add if not already registered
           registry.pages.set(key, page);
         }
       }
     }
   }
   ```

3. **Slot Injection:**
   ```typescript
   // Multiple components can inject into the same slot
   // Rendered in priority order (highest first)
   const headerSlots = registry.slots.get("customer-header");
   // [
   //   { component: CustomHeader, priority: 100 },
   //   { component: StockLevelWidget, priority: 10 },
   //   { component: ActivityTimeline, priority: 5 },
   // ]
   ```

**Key Benefit: Customize WITHOUT Forking**

```bash
# Developer customizes CRM app locally
cd libs/crm-app/src/crm_app/frontend
# Edit CustomLeadForm.tsx, CustomLeadList.tsx
# Test: m dev  (HMR works!)

# DevOps deploys with customized CRM
pip install business-m wms-app hr-app crm-app  # ✅ Custom CRM version
m build  # Single bundle with CRM customizations

# Result: CRM UI is customized, everything else is standard
```

**Customization Workflow:**

```mermaid
graph TD
    A[business-m: Base Customer form] -->|extends| B[crm-app: CRM Customer form]
    B -->|override| C[custom-deployment: Company-specific form]
    C -->|builds into| D[Single Bundle]
    D -->|serves| E[/desk/]
```

**Summary:**

✅ **Option B supports full UI customization:**

- Override pages per app (e.g., custom CRM Lead form)
- Inject components via slots (e.g., add widgets to Customer page)
- Replace global UI elements (e.g., custom header)
- Extend forms with custom fields
- All customizations compose at build time into single bundle
- No runtime overhead, fully tree-shaken

## Third-Party Customization: Two Approaches

**Scenario:** Acme Corp (customer) wants to customize CRM for their specific sales process.

### Approach 1: Fork (Heavy Customization)

```bash
# Acme Corp forks standard crm-app
git clone https://github.com/framework-m/crm-app acme-crm
cd acme-crm

# Rename to acme-crm
# pyproject.toml
[project]
name = "acme-crm"

[project.entry-points."framework_m.frontend"]
acme_crm = "acme_crm.frontend:plugin"

# Customize components
cd src/acme_crm/frontend
# Edit pages/LeadForm.tsx - Acme-specific fields/layout
# Edit pages/LeadList.tsx - Acme workflow stages
# Test: pip install -e . && m dev

# Deploy: pip install acme-crm (instead of crm-app)
```

**Pros:** Full control, can change anything
**Cons:** Must maintain fork, merge updates manually

### Approach 2: Extension Package (Light Customization)

```bash
# Acme creates extension package (no fork!)
mkdir -p libs/acme-crm-ext/src/acme_crm_ext/frontend
```

```toml
# acme-crm-ext/pyproject.toml
[project]
name = "acme-crm-extensions"
dependencies = ["crm-app"]  # ⬅ Uses standard CRM

[project.entry-points."framework_m.frontend"]
acme_crm_ext = "acme_crm_ext.frontend:plugin"
```

```typescript
// acme-crm-ext/src/acme_crm_ext/frontend/index.ts
import { registerPlugin } from "@framework-m/core";
import { AcmeLeadForm } from "./pages/AcmeLeadForm";

registerPlugin({
  name: "acme_crm_ext",
  dependencies: ["crm_app"], // Requires standard CRM

  pages: [
    {
      pageType: "form",
      doctype: "Lead",
      component: AcmeLeadForm, // Override ONLY Lead form
      override: true,
    },
  ],
});
```

```typescript
// acme-crm-ext/src/acme_crm_ext/frontend/pages/AcmeLeadForm.tsx
import { LeadForm as StandardForm } from "crm-app/frontend/pages/LeadForm";

export function AcmeLeadForm(props) {
  return (
    <div>
      <StandardForm {...props} />  {/* Reuse standard form */}
      <AcmeExtraFields {...props} />  {/* Add Acme fields */}
    </div>
  );
}
```

```bash
# Deploy both packages
pip install crm-app acme-crm-extensions
m build  # Extension overrides run last, win over standard
```

**Pros:** Lightweight, auto-updates with standard CRM
**Cons:** Limited to override/extend points (slots)

### Comparison

| Aspect           | Fork                | Extension                 |
| ---------------- | ------------------- | ------------------------- |
| **Setup**        | Clone & rename      | New package               |
| **Dependencies** | None                | Requires standard crm-app |
| **Updates**      | Manual merge        | Automatic (inherits)      |
| **Control**      | Full                | Partial (slots/overrides) |
| **Size**         | Full CRM copy       | Only customizations       |
| **Best For**     | Heavy customization | Light tweaks              |

### Build Order

```typescript
// With acme-crm-extensions installed:
registerPlugin({ name: "business_m" }); // 1st
registerPlugin({ name: "crm_app" }); // 2nd - standard CRM
registerPlugin({ name: "acme_crm_ext" }); // 3rd - overrides win!

// Result: Acme's Lead form replaces standard, everything else is standard CRM
```

## Implementation Checklist

### Phase 0: Customization Support ✅ Already Works

The plugin registry already supports customization patterns:

- [x] Override mechanism (`override: true` in page registration)
- [x] Slot injection with priority ordering
- [x] Last-registration-wins for pages
- [x] Component composition via imports

**No changes needed** - customization works out of the box once entry point discovery is added.

### Phase 1: Entry Point System ✅ Complete

- [x] Define `framework_m.frontend` entry point group
- [x] Create `FrontendPluginInfo` TypedDict schema
- [x] Add `discover_frontend_plugins()` function
- [x] Document entry point format

### Phase 2: Build Discovery ✅ Complete

- [x] Update `build.py::_discover_all_plugins()`
- [x] Add `importlib.resources` support for installed packages
- [x] Handle both source and pre-bundled plugins
- [x] Add tests for multi-source discovery

### Phase 3: Build Composition ✅ Complete

- [x] Update `_generate_plugin_entry()` for multi-package imports
- [x] Add temp directory management for installed sources
- [x] Update Vite config to support plugin paths
- [x] Test build with mix of local + installed plugins

### Phase 4: Documentation ✅ Complete

- [x] Document package frontend structure
- [x] Add business-m example package
- [x] Document plugin composition patterns
- [x] Add migration guide for existing apps

### Phase 5: Dev Mode Support ✅ Complete

- [x] Update `m dev` to discover plugins from editable installs
- [x] Support HMR for both local apps/ and libs/ package frontends
- [x] Generate Vite config with path aliases in dev mode
- [x] Ensure Vite can resolve imports from editable package sources

### Phase 6: Testing ✅ Complete

- [x] Unit tests for plugin discovery (7 integration tests)
- [x] Integration tests for multi-package builds (4 end-to-end tests)
- [x] Test override/composition patterns (frontend TypeScript - N/A for Python layer)
- [x] Test dependency resolution (plugin ordering and error handling tests)

**Test Coverage:**

- 45 total tests in `test_build.py` (up from 34)
- 11 new tests for Phase 6:
  - 7 tests in `TestPluginDiscoveryIntegration`: error handling, edge cases, deduplication
  - 4 tests in `TestBuildPluginModeIntegration`: fallback, file generation, plugin ordering
- All tests passing ✅

## Migration Path

### For Existing Apps (apps/ directory):

**No changes required!** Continue working as before.

### For New Business-M Style Packages (Development):

1. Create package structure in `libs/business-m/`
2. Add frontend directory and entry point in pyproject.toml
3. Install in editable mode: `pip install -e ./libs/business-m`
4. Develop with live reload: `m dev` (auto-discovers)
5. Build for testing: `m build` (includes all installed plugins)

### For Production Deployment:

1. CI builds and publishes packages to PyPI
2. Production installs released versions: `pip install business-m==1.0.0`
3. Build production bundle: `m build` (discovers from installed packages)
4. Deploy static assets

### For Gradual Migration:

1. Start with apps/ directory (current)
2. Extract reusable components to libs/business-m package
3. Install business-m in editable mode, keep apps/ for app-specific code
4. Gradually move more to libs/ packages as needed
5. Publish to PyPI when ready for production use

## Benefits

1. **Backend Parity**: Frontend discovery works like backend DocType discovery
2. **Pip Install UI**: `pip install business-m` includes both backend and frontend
3. **True Modularity**: Each package is independently installable
4. **Composition**: Apps can extend/override base packages
5. **Gradual Adoption**: Works with existing apps/ directory
6. **No Lock-in**: Can still use eject mode for custom builds
7. **Full Customization**: Override any UI without forking entire codebase
8. **Slot-based Extension**: Inject features into other apps' UI

## Customization Example: Custom CRM Deployment

**Scenario:** Company wants standard business-m, wms-app, hr-app, but customized CRM UI.

**Step 1: Fork CRM Package**

```bash
# Fork crm-app to custom-crm
git clone https://github.com/company/crm-app custom-crm
cd custom-crm

# Rename package
# pyproject.toml
[project]
name = "custom-crm"

[project.entry-points."framework_m.apps"]
custom_crm = "custom_crm:app"

[project.entry-points."framework_m.frontend"]
custom_crm = "custom_crm.frontend:plugin"
```

**Step 2: Customize Frontend**

```typescript
// custom-crm/src/custom_crm/frontend/index.ts
import { registerPlugin } from "@framework-m/core";
import { CustomLeadForm } from "./pages/LeadForm"; // ⬅ Customized
import { CustomLeadList } from "./pages/LeadList"; // ⬅ Customized
import { CustomDealPipeline } from "./components/DealPipeline"; // ⬅ Customized

registerPlugin({
  name: "custom_crm",
  version: "1.0.0",

  dependencies: ["business_m"],

  pages: [
    {
      pageType: "form",
      doctype: "Lead",
      component: CustomLeadForm, // Uses company-specific layout
    },
    {
      pageType: "list",
      doctype: "Lead",
      component: CustomLeadList, // Company-specific kanban board
    },
  ],

  fields: [
    {
      name: "deal_pipeline",
      component: CustomDealPipeline, // Custom visualization
      fieldTypes: ["Data"],
    },
  ],

  // Override business-m's Customer page
  pages: [
    {
      pageType: "form",
      doctype: "Customer",
      component: CustomLeadForm,
      override: true, // ✅ Replace standard form
    },
  ],
});
```

**Step 3: Build Custom Components**

```bash
# Develop custom UI
cd custom-crm/src/custom_crm/frontend

# pages/LeadForm.tsx - Company-specific layout
# pages/LeadList.tsx - Custom kanban view with stages
# components/DealPipeline.tsx - Custom chart

# Test locally
pip install -e .
m dev  # HMR works for custom components
```

**Step 4: Deploy**

```bash
# Production Dockerfile
FROM python:3.12

# Install standard packages
RUN pip install framework-m business-m wms-app hr-app

# Install CUSTOMIZED CRM (instead of standard crm-app)
RUN pip install custom-crm==1.2.3

# Build composes ALL plugins into single bundle
# custom-crm's UI automatically included
RUN m build --output /app/framework_m/static

CMD ["uvicorn", "framework_m_standard.adapters.web.app:create_app"]
```

**Result:**

```
Single Bundle contains:
├── Desk UI (from framework-m)
├── Business-M UI (standard)
├── WMS UI (standard)
├── HR UI (standard)
└── Custom CRM UI (customized)  ⬅ Replaces standard CRM
    ├── Custom Lead Form
    ├── Custom Lead List (kanban)
    ├── Custom Deal Pipeline
    └── Override: Custom Customer Form
```

**Key Points:**

- ✅ Only CRM is customized, other apps remain standard
- ✅ Single `pip install custom-crm` deploys custom UI
- ✅ No manual bundle coordination
- ✅ Tree-shaking removes unused CRM components
- ✅ Can version custom-crm independently
- ✅ Easy to switch back: `pip install crm-app` instead

## Benefits

1. **Backend Parity**: Frontend discovery works like backend DocType discovery
2. **Pip Install UI**: `pip install business-m` includes both backend and frontend
3. **True Modularity**: Each package is independently installable
4. **Composition**: Apps can extend/override base packages
5. **Gradual Adoption**: Works with existing apps/ directory
6. **No Lock-in**: Can still use eject mode for custom builds

## Open Questions

1. **NPM Dependencies**: How do installed packages manage their npm dependencies?
   - Option A: Package frontend/ includes package.json, merge all during build
   - Option B: Packages pre-bundle their frontend (recommended for PyPI distribution)
   - Option C: Monorepo with workspace dependencies (recommended for development)
   - **Note**: Development (editable installs) likely uses Option C, production (PyPI) uses Option B

2. **Version Conflicts**: What if business-m@1.0 and wms-app expect different React versions?
   - Solution: Peer dependencies, strict version constraints
   - Development: Single workspace manages all deps
   - Production: Pre-bundled packages avoid runtime conflicts

3. **Build Performance**: Does scanning all installed packages slow down builds?
   - Solution: Cache plugin metadata, only rebuild changed plugins

4. **TypeScript**: How do apps get types from installed packages?
   - Solution: Packages export .d.ts files, configured in tsconfig paths

## Current UI Bundling Architecture

### Development Mode (HMR)

**How `m dev` works:**

1. **Separate Processes with Honcho:**

   ```bash
   m dev
   # Starts:
   # - Backend: uvicorn with --reload on :8888
   # - Frontend: Vite dev server with HMR on :5173
   ```

2. **Vite Dev Server:**
   - Runs `pnpm dev` in `frontend/` directory
   - Serves source files directly (no bundling)
   - Hot Module Replacement (HMR) for instant updates
   - Proxies API calls to backend: `/api` → `http://localhost:8888/api`

3. **Backend in Dev Mode:**
   - `M_DEV_MODE=true` environment variable set
   - Backend does NOT serve static files
   - Only serves API routes
   - No UI bundling required

4. **Vite Configuration:**
   ```typescript
   // frontend/vite.config.ts
   export default defineConfig({
     server: {
       port: 5173,
       proxy: {
         "/api": { target: "http://localhost:8888" }, // API requests
         "/api/v1/stream": { target: "ws://localhost:8888", ws: true }, // WebSocket
       },
     },
   });
   ```

**Current Limitation:** Only discovers plugins from `apps/` directory.

**With Multi-Package Support (Planned):**

- Discover plugins from editable installs: `pip install -e ./libs/business-m`
- Generate temp Vite entry that imports all discovered plugins
- HMR works across all package frontends (using symlinks or path aliases)

### Production Mode (Static Bundle)

**How `m build` works:**

1. **Build Command:**

   ```bash
   m build
   # Default output: libs/framework-m/src/framework_m/static/
   ```

2. **Build Process:**

   ```python
   # In build.py
   def _build_indie_mode(mode: str, output: Path):
       # 1. Find frontend directory
       frontend_dir = Path("frontend")

       # 2. Install dependencies
       subprocess.run(["pnpm", "install"], cwd=frontend_dir)

       # 3. Build with Vite
       subprocess.run(["pnpm", "run", "build"], cwd=frontend_dir)

       # 4. Copy dist/ to output
       shutil.copytree(frontend_dir / "dist", output)
   ```

3. **Vite Production Build:**
   - Bundles all source files into optimized assets
   - Tree-shaking, minification, code splitting
   - Outputs to `frontend/dist/`:
     ```
     dist/
     ├── index.html
     ├── assets/
     │   ├── index-[hash].js   # Main bundle
     │   ├── index-[hash].css  # Styles
     │   └── vendor-[hash].js  # Third-party libs
     └── favicon.ico
     ```

4. **Packaging into Python Wheel:**

   ```toml
   # libs/framework-m/pyproject.toml
   [tool.hatch.build.targets.wheel]
   packages = ["src/framework_m"]

   # Exclude source, then force-include built assets
   exclude = ["src/framework_m/static"]

   [tool.hatch.build.targets.wheel.force-include]
   "src/framework_m/static" = "framework_m/static"
   ```

5. **Result:**
   - `pip install framework-m` includes bundled UI in `framework_m/static/`
   - Backend serves static files: `app.mount("/desk", StaticFilesConfig(...))`
   - Single `.whl` file contains both Python + UI

**Current Limitation:** Only bundles framework-m's Desk UI. Apps in `apps/` are not included.

**With Multi-Package Support (Planned):**

```bash
# Development
pip install -e ./libs/business-m
pip install -e ./libs/wms-app
m build  # Discovers ALL plugins, builds ONE bundle

# Production (after CI publishes to PyPI)
pip install business-m wms-app  # Both include their own pre-built UIs
m build  # Discovers ALL installed plugins, builds ONE bundle
```

### Multi-Package Bundling Strategy (Planned)

**Development Workflow:**

```bash
# 1. Install packages in editable mode
pip install -e ./libs/business-m
pip install -e ./libs/wms-app
pip install -e ./libs/hr-app

# 2. Dev mode (HMR across all packages)
m dev
# - Discovers plugins via entry points
# - Generates temp entry: .m/dev/entry.tsx
# - Vite dev server with path aliases to package sources
# - HMR works across business-m, wms-app, hr-app

# 3. Build for testing
m build
# - Discovers all installed plugins (from editable installs)
# - Generates temp entry: .m/build/entry.tsx
# - Single Vite build bundles ALL plugins
# - Output: .m/build/dist/ (not packaged into wheel)
```

**Production Workflow:**

```bash
# Option A: Each package pre-builds its own UI
# - business-m publishes with pre-built static/frontend/
# - wms-app publishes with pre-built static/frontend/
# - Packages are larger but self-contained

# Option B: Composition at deployment (recommended)
# - Packages only include source: src/business_m/frontend/*.ts
# - CI/CD runs `m build` to compose all UIs
# - Single bundle deployed to framework-m/static/

# In production deployment:
docker build:
  - pip install business-m wms-app hr-app
  - m build --output /app/static
  - Static files served by backend or CDN
```

### Implementation Considerations

1. **NPM Dependencies:**
   - **Dev:** Monorepo workspace manages all deps
     ```json
     // pnpm-workspace.yaml
     packages:
       - 'libs/*/frontend'
       - 'frontend'
     ```
   - **Prod:** Pre-bundle packages OR use peer dependencies

2. **TypeScript Module Resolution:**

   ```typescript
   // tsconfig.json for apps
   {
     "compilerOptions": {
       "paths": {
         "@business-m/*": ["../../libs/business-m/src/business_m/frontend/*"],
         "@wms-app/*": ["../../libs/wms-app/src/wms_app/frontend/*"]
       }
     }
   }
   ```

3. **Vite Build Config:**
   ```typescript
   // Generated dynamically by m build
   export default defineConfig({
     resolve: {
       alias: {
         "@business-m": "/path/to/libs/business-m/src/business_m/frontend",
         "@wms-app": "/path/to/libs/wms-app/src/wms_app/frontend",
       },
     },
   });
   ```

## Summary: How UI is Bundled

### Development (`m dev`)

- **No bundling** - Vite dev server serves source files directly
- **HMR enabled** - Instant hot reload for all changes
- **Multi-process** - Backend (8888) + Frontend (5173) run separately via honcho
- **API proxy** - Vite proxies `/api` requests to backend
- **Plugin discovery** - Will scan entry points to find all package frontends
- **Result**: Fast development with instant feedback

### Build (`m build`)

- **Single bundle** - Vite bundles ALL discovered plugins into one artifact
- **Plugin discovery** - Scans entry points for both editable + PyPI installs
- **Optimized** - Tree-shaking, minification, code splitting
- **Output**: `framework_m/static/` (bundled into Python wheel)
- **Result**: One `.whl` file contains Python backend + UI frontend

### Production Deployment

**Option A: Pre-bundled Packages**

```bash
# Each package publishes with its own built UI
pip install business-m  # Includes business_m/static/frontend/
pip install wms-app     # Includes wms_app/static/frontend/
# Backend serves all static directories
```

**Option B: Composition at Deploy (Recommended)**

```bash
# Packages only include source code
pip install business-m wms-app hr-app
m build --output /app/static  # Composes ALL into single bundle
# Backend serves single static directory
```

## Next Steps

1. Create proof-of-concept in `libs/business-m/` (editable install for dev)
2. Implement Phase 1 (entry point discovery from installed packages)
3. Update build.py for multi-package support (both editable and PyPI installs)
4. Test with `libs/business-m` + `libs/wms-app` example
5. Document patterns and best practices for development workflow
6. Add CI/CD examples for publishing to PyPI


Document: docs/adr/0007-bundled-desk-distribution.md

# ADR-0007: Bundled Desk Distribution via PyPI

- **Status**: Accepted
- **Date**: 2026-01-29
- **Updated**: 2026-02-13 (Multi-Package Composition)
- **Deciders**: @anshpansuriya14
- **Supersedes**: N/A
- **Superseded by**: N/A

## Context

Framework M requires a distribution strategy for frontend UI that provides an excellent developer experience without adding package management complexity. This includes both the core Desk UI and domain-specific UIs from business apps (business-m, wms-app, crm-app, etc.).

**Forces at play:**

- **Developer Experience**: Third-party developers should not need to clone the monorepo to use Framework M
- **Single Command Start**: Similar to Frappe's `bench start`, we need `m prod` to work immediately
- **Package Management**: Multiple package registries (PyPI + npm) create deployment complexity
- **Frontend Customization**: Advanced users need ability to customize the Desk UI
- **Maintenance Burden**: Each package registry requires CI/CD, versioning, and support
- **Bundle Size**: Pre-built frontends increase PyPI package size

**Current State:**

- Frontend exists only in `/frontend` directory of monorepo
- Developers must clone entire repository to access Desk UI
- No integrated start command for backend + frontend

**Desired Developer Experience:**

```bash
pip install framework-m
m new:project crm
cd crm
m prod                     # Works immediately with bundled Desk
m prod --with-frontend     # For frontend customization (scaffolds React app)
```

**Alternatives Evaluated:**

| Option                        | Distribution       | Pros                       | Cons                          |
| ----------------------------- | ------------------ | -------------------------- | ----------------------------- |
| **Bundled Static + Scaffold** | PyPI only          | Single registry, simple DX | ~10MB package size            |
| **npm Package**               | PyPI + npm         | Smaller Python package     | Two registries, complex CI/CD |
| **Docker Only**               | Docker Hub         | Clean separation           | Poor development DX           |
| **No Bundling**               | Git clone required | No bundle size             | Terrible DX                   |

## Decision

> We will **bundle pre-built frontend static files** in PyPI packages (framework-m for Desk, business-m/wms-app/etc. for domain UIs), provide **optional frontend scaffolding** for customization, and use **Python entry points** (`framework_m.frontend`) for multi-package UI composition. For development, publish **`@framework-m/desk` npm package** to GitLab registry for reusable Refine providers and components.

**Architecture:**

```
framework-m (PyPI Package - Base UI)
│
├── framework_m/
│   ├── static/                    # Pre-built Desk (from CI)
│   │   ├── index.html
│   │   ├── assets/
│   │   └── ...

business-m (PyPI Package - Business UI)
│
├── business_m/
│   ├── frontend/                  # TypeScript source (for dev)
│   │   ├── index.ts              # Plugin entry
│   │   ├── components/
│   │   └── pages/
│   └── static/                    # Pre-built business UI (optional)
│       └── frontend/

wms-app, crm-app, hr-app (PyPI Packages - Domain UIs)
│
├── Similar structure to business-m
│   ├── frontend/                  # TypeScript source
│   └── static/frontend/          # Pre-built (optional)

Deployment App (Final Composition)
│
├── m build                        # Composes ALL UIs via entry points
│   └── discovers: framework-m + business-m + wms-app + crm-app
├── Output: combined static bundle
│   ├── index.html
│   ├── assets/
│   │   ├── main-[hash].js        # Framework + Desk
│   │   ├── business-[hash].js    # Business-m UI
│   │   ├── wms-[hash].js         # WMS UI
│   │   └── crm-[hash].js         # CRM UI

framework-m (PyPI Package - Base)
│
├── framework_m/
│   ├── static/                    # Pre-built Desk (from CI)
│   │   ├── index.html
│   │   ├── assets/
│   │   └── ...
│   │
│   ├── templates/
│   │   └── frontend/              # Scaffold source
│   │       ├── package.json       # Uses @framework-m/desk from GitLab npm
│   │       ├── .npmrc             # GitLab registry config
│   │       ├── src/
│   │       │   ├── App.tsx
│   │       │   └── components/
│   │       └── vite.config.ts
│   │
│   └── cli/
│       ├── serve.py               # m prod command
│       └── init.py                # m init:frontend command

@framework-m/desk (GitLab npm Package)
│
├── src/
│   ├── providers/
│   │   ├── data.ts                # frameworkMDataProvider
│   │   ├── auth.ts                # authProvider
│   │   ├── live.ts                # liveProvider
│   │   ├── constants.ts
│   │   └── types.ts
│   ├── components/
│   │   ├── DocTypeList.tsx
│   │   ├── DocTypeForm.tsx
│   │   └── DocTypeShow.tsx
│   └── hooks/
│       ├── useDocType.ts
│       └── useMetadata.ts
└── dist/                          # Built library
```

**Distribution Strategy:**

| Component             | Location                          | Purpose                              | Registry       |
| --------------------- | --------------------------------- | ------------------------------------ | -------------- |
| **Bundled Desk**      | `framework_m/static/`             | Pre-built React app in PyPI package  | PyPI           |
| **Domain UIs**        | `business_m/frontend/`, etc.      | TypeScript source in domain packages | PyPI           |
| **Entry Points**      | `pyproject.toml` (all packages)   | Frontend plugin discovery            | PyPI           |
| **Frontend Template** | `framework_m/templates/frontend/` | Scaffold source for customization    | PyPI (bundled) |
| **Shared Components** | `@framework-m/desk`               | Reusable Refine providers/components | GitLab npm     |

**Why GitLab npm Registry:**

- ✅ **Private by Default**: Uses CI_JOB_TOKEN, no public exposure until ready
- ✅ **Simple CI/CD**: Already in GitLab, uses existing auth
- ✅ **Optional Public**: Can point to npmjs.com later via CLI config
- ✅ **Monorepo Friendly**: Same repo hosts Python and npm packages
- ✅ **No Extra Infra**: GitLab provides npm registry for free

**Template npm Configuration:**

```bash
# .npmrc in scaffolded frontend/
@framework-m:registry=https://gitlab.com/api/v4/projects/${CI_PROJECT_ID}/packages/npm/
# For local dev, users can optionally configure GitLab token or use public npm later
```

## Consequences

### Positive

- **Excellent DX**: `pip install framework-m && m prod` works immediately
- **Best of Both Worlds**: Bundled static for quick start + npm package for customization
- **GitLab Integration**: Private npm registry, uses CI_JOB_TOKEN, no extra auth setup
- **Atomic Updates**: Bug fixes in Desk UI ship with backend updates
- **Reusable Components**: `@framework-m/desk` prevents code duplication in customized projects
- **Monorepo Clean**: All packages (Python + npm) in same repo, shared CI/CD
- **Flexible Publishing**: Can switch to public npmjs.com later via CLI config

### Negative

- **Larger PyPI Package**: ~10MB vs ~2MB (bundled static files)
  - _Mitigation_: Acceptable for modern bandwidth, one-time download cost
- **npm Registry Dependency**: Users need GitLab token for scaffolded frontend development
  - _Mitigation_: Template includes `.npmrc` setup instructions, can fallback to public npm later
- **Two Package Systems**: PyPI for backend, GitLab npm for frontend components
  - _Mitigation_: Simpler than vendoring, better for ecosystem growth

### Neutral

- **Build Time**: CI must build React app + npm package during release
  - Added to `.gitlab/ci/build.yml`, runs in parallel with other jobs
- **Template Updates**: Scaffolded projects pull `@framework-m/desk` updates via `pnpm update`
  - Better than vendored approach, users get component updates easily

## Implementation Plan

### Phase 1: Bundled Desk (Required for v1.0)

**CI Build Pipeline** (`.gitlab-ci.yml`):

```yaml
build:frontend:
  stage: build
  script:
    - cd frontend
    - pnpm install --frozen-lockfile
    - pnpm build
    - mkdir -p ../libs/framework-m/src/framework_m/static
    - cp -r dist/* ../libs/framework-m/src/framework_m/static/
  artifacts:
    paths:
      - libs/framework-m/src/framework_m/static/
    expire_in: 1 day
  only:
    - tags # Only build for releases
```

**Package Data** (`libs/framework-m/pyproject.toml`):

```toml
[tool.setuptools.package-data]
framework_m = [
    "static/**/*",
    "templates/**/*",
]
```

**Static File Serving** (`libs/framework-m-standard/src/adapters/web/app.py`):

```python
from importlib.resources import files
from litestar.static_files import create_static_files_router

STATIC_DIR = files("framework_m") / "static"

app.register(
    create_static_files_router(
        path="/",
        directories=[STATIC_DIR],
        html_mode=True,  # Serve index.html for SPA routing
    )
)
```

**CLI Command** (`libs/framework-m/src/framework_m/cli/serve.py`):

```python
def prod_command(
    port: int = 8000,
    reload: bool = False,
    with_frontend: bool = False,
):
    """Start Framework M application.

    Examples:
      m prod                    # Backend + bundled Desk
      m prod --with-frontend    # Backend + Vite dev server
      m prod --reload           # Auto-reload on code changes
    """
    if with_frontend:
        _start_with_frontend(port, reload)
    else:
        _start_backend_only(port, reload)

def _start_backend_only(port: int, reload: bool):
    """Start backend serving bundled static Desk."""
    uvicorn.run(
        "framework_m.app:app",
        host="0.0.0.0",
        port=port,
        reload=reload,
    )
```

### Phase 2: Frontend Scaffold (Required for Customizers)

**Auto-Scaffold Command** (`libs/framework-m/src/framework_m/cli/init.py`):

```python
def init_frontend_command(target: Path | None = None):
    """Scaffold frontend for customization.

    Examples:
        m init:frontend           # Creates ./frontend
        m init:frontend my-ui     # Creates ./my-ui
    """
    from importlib.resources import files, as_file

    target = target or Path.cwd() / "frontend"

    if target.exists():
        raise ValueError(f"Directory {target} already exists")

    # Copy template from package
    template = files("framework_m") / "templates/frontend"
    with as_file(template) as template_dir:
        shutil.copytree(template_dir, target)

    print(f"✓ Scaffolded frontend to {target}")
    print("  Installing dependencies...")

    subprocess.run(["pnpm", "install"], cwd=target, check=True)
    print("✓ Ready! Run: m prod --with-frontend")
```

**Start with Frontend Dev Server**:

```python
def _start_with_frontend(port: int, reload: bool):
    """Start backend + frontend dev server."""
    frontend_dir = Path.cwd() / "frontend"

    # Auto-scaffold if not exists
    if not frontend_dir.exists():
        print("No frontend/ found. Scaffolding...")
        init_frontend_command(frontend_dir)

    # Start both processes
    backend = subprocess.Popen([
        "uvicorn",
        "framework_m.app:app",
        f"--port={port}",
        "--reload" if reload else "",
    ])

    frontend = subprocess.Popen(
        ["pnpm", "dev"],
        cwd=frontend_dir,
    )

    try:
        backend.wait()
        frontend.wait()
    except KeyboardInterrupt:
        backend.terminate()
        frontend.terminate()
```

### Phase 3: Template Design (Vendor Shared Code)

**Template Structure**:

```
templates/frontend/
├── package.json               # Minimal deps, no @framework-m scope
├── vite.config.ts
├── tsconfig.json
├── src/
│   ├── App.tsx
│   ├── providers/
│   │   ├── data.ts           # Full implementation (vendored)
│   │   ├── auth.ts
│   │   └── live.ts
│   ├── components/
│   │   ├── DocTypeList.tsx
│   │   ├── DocTypeForm.tsx
│   │   └── DocTypeShow.tsx
│   └── hooks/
│       ├── useDocType.ts
│       └── useMetadata.ts
└── public/
```

**No npm Package, Direct Vendoring**:

```typescript
// src/providers/data.ts (copied into every scaffolded project)
import type { DataProvider } from "@refinedev/core";

export const frameworkMDataProvider = (apiUrl: string): DataProvider => ({
  getList: async ({ resource, pagination, filters, sorters, meta }) => {
    const url = new URL(`${apiUrl}/${resource}`);

    // Pagination
    if (pagination) {
      url.searchParams.set("page", String(pagination.current));
      url.searchParams.set("pageSize", String(pagination.pageSize));
    }

    // Filters
    if (filters) {
      url.searchParams.set("filters", JSON.stringify(filters));
    }

    const response = await fetch(url.toString());
    const data = await response.json();

    return {
      data: data.items,
      total: data.total,
    };
  },

  getOne: async ({ resource, id }) => {
    const response = await fetch(`${apiUrl}/${resource}/${id}`);
    return { data: await response.json() };
  },

  create: async ({ resource, variables }) => {
    const response = await fetch(`${apiUrl}/${resource}`, {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify(variables),
    });
    return { data: await response.json() };
  },

  update: async ({ resource, id, variables }) => {
    const response = await fetch(`${apiUrl}/${resource}/${id}`, {
      method: "PUT",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify(variables),
    });
    return { data: await response.json() };
  },

  deleteOne: async ({ resource, id }) => {
    await fetch(`${apiUrl}/${resource}/${id}`, { method: "DELETE" });
    return { data: {} };
  },

  // Full implementation lives in template, no npm package needed
});
```

**Package.json (No Custom Packages)**:

```json
{
  "name": "framework-m-frontend",
  "version": "0.1.0",
  "dependencies": {
    "@refinedev/core": "^5.0.0",
    "@refinedev/react-router-v6": "^5.0.0",
    "react": "^19.0.0",
    "react-dom": "^19.0.0",
    "react-router-dom": "^6.0.0"
  },
  "devDependencies": {
    "@vitejs/plugin-react": "^4.0.0",
    "typescript": "^5.0.0",
    "vite": "^5.0.0"
  }
}
```

## Command Reference

| Command                  | Description                                       |
| ------------------------ | ------------------------------------------------- |
| `m prod`                 | Start backend + serve bundled Desk (static files) |
| `m prod --port 8000`     | Custom port                                       |
| `m prod --reload`        | Dev mode with auto-reload                         |
| `m prod --with-frontend` | Start backend + Vite dev server (auto-scaffolds)  |
| `m init:frontend`        | Scaffold frontend only (no start)                 |

## Phase 4: Multi-Package UI Composition (2026-02-13 Addition)

**Entry Point System for Frontend Plugins:**

```toml
# business-m/pyproject.toml
[project.entry-points."framework_m.frontend"]
business_m = "business_m.frontend:plugin"

# wms-app/pyproject.toml
[project.entry-points."framework_m.frontend"]
wms_app = "wms_app.frontend:plugin"
```

**Frontend Plugin Structure:**

```typescript
// business-m/src/business_m/frontend/index.ts
import { registerPlugin } from "@framework-m/core";

export const plugin = {
  name: "business_m",
  version: "1.0.0",
  entry: "./src/index.ts",
};

registerPlugin({
  name: "business_m",
  pages: [
    { pageType: "list", doctype: "Customer", component: CustomerList },
  ],
  components: [...],
});
```

**Build Discovery (m build):**

```python
# apps/studio/src/framework_m_studio/cli/build.py

def _discover_all_plugins() -> list[tuple[str, Path, str]]:
    """Discover frontend plugins from:
    1. Installed Python packages (via entry points) - PyPI or editable
    2. Local apps/ directory (legacy/convenience)
    """
    plugins = []

    # NEW: Discover from installed packages
    from importlib.metadata import entry_points
    import importlib.resources

    eps = entry_points(group="framework_m.frontend")
    for ep in eps:
        package_module = ep.module.split(":")[0]
        frontend_src = importlib.resources.files(package_module) / "frontend"
        if frontend_src.is_dir():
            entry_file = frontend_src / "index.ts"
            if entry_file.is_file():
                plugins.append((ep.name, Path(str(entry_file)), "installed"))

    # EXISTING: Scan apps/ directory
    apps_dir = Path.cwd() / "apps"
    if apps_dir.exists():
        for app_dir in apps_dir.iterdir():
            plugin_entry = app_dir / "frontend" / "index.ts"
            if plugin_entry.exists():
                plugins.append((app_dir.name, plugin_entry, "local"))

    return plugins
```

**Developer Workflow:**

```bash
# Development (editable installs)
pip install -e ./libs/business-m
pip install -e ./libs/wms-app
m dev   # Discovers all plugins via entry points, HMR works

# Production (PyPI installs)
pip install framework-m business-m wms-app crm-app
m build # Composes ALL UIs into single bundle
```

**Deployment Scenarios:**

```bash
# Scenario 1: Deploy with standard packages
pip install framework-m business-m wms-app
m build  # Output: framework + business + wms UIs

# Scenario 2: Deploy with custom CRM
pip install framework-m business-m wms-app custom-crm-app
m build  # Output: framework + business + wms + custom-crm UIs

# Scenario 3: Minimal deployment
pip install framework-m
m build  # Output: just Desk UI
```

## Implementation Checklist

**Phase 1-3 (Original):**

- [ ] Create `build:frontend` CI job in `.gitlab/ci/build.yml`
- [ ] Include `static/` and `templates/` in `pyproject.toml` package data
- [ ] Implement static file serving in `libs/framework-m-standard/src/adapters/web/app.py`
- [ ] Create `m prod` command in `libs/framework-m/src/framework_m/cli/serve.py`
- [ ] Create `m init:frontend` command in `libs/framework-m/src/framework_m/cli/init.py`
- [ ] Design frontend template with vendored providers in `templates/frontend/`
- [ ] Update documentation for frontend customization workflow
- [ ] Test package size (ensure < 15MB total)

**Phase 4 (Multi-Package - Added 2026-02-13):**

- [ ] Define `framework_m.frontend` entry point group
- [ ] Update `build.py::_discover_all_plugins()` to scan entry points
- [ ] Add `importlib.resources` support for installed package frontends
- [ ] Generate Vite config with path aliases for discovered packages
- [ ] Update `m dev` to discover plugins from editable installs
- [ ] Test with business-m + wms-app editable installs
- [ ] Document multi-package frontend distribution pattern
- [ ] Add CI examples for building multi-package frontends

## Alternatives Considered

### Option A: npm Package Distribution

**Rejected** due to:

- **Complexity**: Requires npm registry setup, separate CI/CD pipeline
- **Version Drift**: Frontend and backend versions can get out of sync
- **Extra Registry**: Developers need both PyPI and npm configured
- **CI Maintenance**: Must publish to both registries on each release

### Option B: Docker-Only Distribution

**Rejected** due to:

- **Poor Development DX**: Requires Docker for local development
- **Slow Iteration**: Rebuilding containers on each change
- **Complex Debugging**: Harder to debug inside containers

### Option C: No Bundling (Git Clone Required)

**Rejected** due to:

- **Terrible DX**: Forces third-party developers to clone monorepo
- **Confusing Structure**: Monorepo has many unrelated directories
- **High Barrier**: Discourages adoption for simple use cases

## Migration Path

**For existing users (current monorepo developers):**

No change required. Monorepo development continues as-is.

**For new third-party developers:**

```bash
# After this ADR implementation
pip install framework-m
m new:project myapp
cd myapp
m prod                     # Just works with bundled Desk

# If customizing frontend
m prod --with-frontend     # Auto-scaffolds and starts Vite
```

## Files to Create/Modify

| File                                                   | Action     | Description                                        |
| ------------------------------------------------------ | ---------- | -------------------------------------------------- |
| `.gitlab/ci/build.yml`                                 | **MODIFY** | Add `build:frontend` job                           |
| `libs/framework-m/pyproject.toml`                      | **MODIFY** | Include `static/` and `templates/` in package data |
| `libs/framework-m/src/framework_m/static/`             | **NEW**    | Pre-built Desk (from CI)                           |
| `libs/framework-m/src/framework_m/templates/frontend/` | **NEW**    | Frontend scaffold template                         |
| `libs/framework-m/src/framework_m/cli/serve.py`        | **NEW**    | `m prod` command                                   |
| `libs/framework-m/src/framework_m/cli/init.py`         | **MODIFY** | Add `m init:frontend` command                      |
| `libs/framework-m-standard/src/adapters/web/app.py`    | **MODIFY** | Serve static files at `/`                          |

## Questions for Review

1. **Package Size**: Is ~10MB PyPI package acceptable for bundled Desk? (PyPI limit is 100MB)
2. **Template Updates**: How should users get template updates after scaffolding? (Manual merge vs. re-scaffold)
3. **CDN Strategy**: Should template use CDN for large deps in dev mode to reduce scaffold size?

## References

- RFC-0001: Frontend Initialization and Development Experience (../rfcs/rfc-0001-frontend-init.md)
- ADR-0005: Use Refine for Frontend (./0005-refine-for-frontend.md)
- GitLab CI Publish Pipeline: `.gitlab/ci/publish.yml`
- [Frappe Bench Architecture](https://frappeframework.com/docs/user/en/bench)
- [Python Packaging: Including Data Files](https://packaging.python.org/en/latest/guides/using-manifest-in/)


Document: docs/adr/0004-redis-for-cache.md

# ADR-0004: Use Redis for Caching

- **Status**: Accepted
- **Date**: 2025-12-29
- **Deciders**: @anshpansuriya14
- **Supersedes**: N/A
- **Superseded by**: N/A

## Context

Framework M needs a caching layer for:
- Session data (if stateful auth is used)
- Query result caching
- Rate limiting counters
- Temporary data storage
- Distributed locks

**Forces at play:**
- **Performance**: Sub-millisecond reads
- **TTL Support**: Automatic expiration
- **Cross-Platform**: Must work on Windows (development)
- **Data Structures**: Need more than key-value (counters, sets, sorted sets)
- **Industry Standard**: Developers expect Redis

## Decision

> We will use **Redis** via `redis-py` for caching.

For Windows development, we recommend **Memurai** (commercial) or **Garnet** (Microsoft open-source).

```python
from redis.asyncio import Redis

cache = Redis.from_url("redis://localhost:6379")

# CacheProtocol implementation
await cache.set("user:123:profile", json.dumps(profile), ex=3600)
profile = json.loads(await cache.get("user:123:profile"))
```

## Consequences

### Positive

- **Industry Standard**: Most developers know Redis
- **Rich Data Structures**: Strings, hashes, lists, sets, sorted sets, streams
- **TTL Native**: Built-in expiration, no cleanup jobs needed
- **Ecosystem**: Great tooling, monitoring, managed services (ElastiCache, Upstash)
- **Atomic Operations**: INCR, DECR, SETNX for counters and locks

### Negative

- **Separate System**: Not same as NATS (jobs/events)
- **Windows Caveat**: Native Redis doesn't support Windows; alternatives needed
- **Memory Bound**: Data must fit in RAM

### Neutral

- Uses `CacheProtocol` abstraction (can swap for Memcached, etc.)
- Connection pooling via `redis-py`

## Windows Alternatives

| Option | License | Notes |
|--------|---------|-------|
| **Memurai** | Commercial (free tier) | Full Redis compatibility |
| **Garnet** | MIT (Microsoft) | Redis-compatible, .NET-based |
| **Docker Redis** | Open Source | Works via WSL2/Docker Desktop |

## Alternatives Considered

| Option | Pros | Cons |
|--------|------|------|
| **Chosen: Redis** | Standard, rich features, ecosystem | Windows complexity |
| Memcached | Simple, fast | No data structures, no TTL on read |
| NATS KV | Single infrastructure | Limited features, new |
| DragonflyDB | Redis-compatible, faster | Less mature |
| Local Memory | No dependencies | Not distributed, lost on restart |

## References

- [redis-py](https://github.com/redis/redis-py)
- [Memurai (Windows)](https://www.memurai.com/)
- [Garnet (Microsoft)](https://github.com/microsoft/garnet)
- [CacheProtocol](file://src/framework_m/core/interfaces/cache.py)


Document: docs/processes/cla.md

---
id: cla
title: Contributor License Agreement
sidebar_label: CLA
---

# Contributor License Agreement (CLA)

Thank you for your interest in contributing to Framework M!

Before we can accept your contributions (code, documentation, or other materials), we need you to sign a Contributor License Agreement (CLA) to protect the project and its users. The CLA ensures that you retain the copyright to your contribution, but grant us the necessary rights to use, distribute, and build upon it under the project's open-source license.

## Types of Agreements

There are two forms of the CLA:

- **Individual Contributor License Agreement (ICLA)**: For individuals contributing on their own behalf.
- **Corporate Contributor License Agreement (CCLA)**: For individuals contributing on behalf of their employer or organization. This requires authorization from someone who has the authority to bind the organization.

## How to Sign the CLA

## How to Sign the CLA

To legally bind your contributions to the Apache 2.0 license, we maintain a strict traceability mapping between your legal agreement (the Issue) and the programmatic proof (the Merge Request).

1. **Review the Agreement**: Ensure you agree to the terms of the Apache 2.0 license and the standard CLA terms for open-source contributions.
2. **Open a CLA Issue (The Legal Trace)**: Go to the project's GitLab issue tracker and click **New Issue**.
3. **Select a Template**:
   - Choose the **ICLA** template if you are contributing on your own behalf.
   - Choose the **CCLA** template if you are contributing on behalf of your organization.
4. **Agree and Submit**: Fill out the requested details (your GitLab handle, name, and if CCLA, the GitLab handles of the covered employees) and check the "I AGREE" box. Submit the issue.
5. **Create the Merge Request (The Programmatic Proof)**: Create a new branch, manually add your agreed username to `contributors.json`, and open a Merge Request.
6. **Map the Trade**: In your Merge Request description, explicitly link your CLA Issue (e.g., `Closes #123`).
7. **Approval**: A project maintainer will verify your issue agreement matches the username in your `contributors.json` MR, and then merge it. Once merged, future code contributions will automatically pass the CLA verification pipeline.

By submitting a contribution to this project, you agree to license your work under the same terms as the project (Apache License 2.0) and you represent that you have the right to do so.


Document: docs/processes/adr-template.md

# ADR-0000: [Short Title]

- **Status**: Proposed | Accepted | Deprecated | Superseded
- **Date**: YYYY-MM-DD
- **Deciders**: @github-username, @another-username
- **Supersedes**: ADR-XXXX (if applicable)
- **Superseded by**: ADR-XXXX (if applicable)

## Context

What is the issue that we're seeing that is motivating this decision or change?

Describe the forces at play:
- Technical constraints
- Business requirements
- Team capabilities
- Timeline pressures

## Decision

What is the change that we're proposing and/or doing?

State the decision concisely:

> We will use [X] for [Y].

## Consequences

What becomes easier or more difficult because of this decision?

### Positive

- ...
- ...

### Negative

- ...
- ...

### Neutral

- ...

## Alternatives Considered

| Option | Pros | Cons |
|--------|------|------|
| Chosen: X | ... | ... |
| Alternative: Y | ... | ... |
| Alternative: Z | ... | ... |

## References

- Related ADRs: ADR-XXXX (./XXXX-title.md)
- Related RFCs: RFC-XXXX (../rfcs/XXXX-title.md)
- External resources: [Link](https://example.com)


Document: docs/processes/ADR_PROCESS.md

# Architecture Decision Records (ADR)

> **ADR** = Architecture Decision Record

ADRs document significant architectural decisions, their context, and consequences. Unlike RFCs (which propose changes), ADRs record decisions that have been made.

## When to Write an ADR

**Write an ADR when you:**
- Choose between competing technologies
- Define a pattern that others should follow
- Make a decision that constrains future options
- Establish a convention that's not obvious

**Examples:**
- "Use Litestar instead of FastAPI"
- "Implement Repository pattern with explicit session passing"
- "Choose NATS over Redis for job queue"
- "Adopt CloudEvents format for domain events"

## ADR Format

We use the **MADR (Markdown Any Decision Records)** format:

```
docs/adr/
├── 0001-use-litestar-as-web-framework.md
├── 0002-explicit-unit-of-work.md
├── 0003-pydantic-for-doctype-schemas.md
└── ...
```

## ADR Template

See [adr-template.md](adr-template.md).

## ADR Lifecycle

```
┌──────────┐     ┌────────────┐     ┌────────────┐
│ Proposed │ ──► │  Accepted  │ ──► │ Deprecated │
└──────────┘     └────────────┘     └────────────┘
                       │                   │
                       ▼                   ▼
                 ┌────────────┐     ┌─────────────┐
                 │ Superseded │ ◄── │ Superseded  │
                 │   by XXXX  │     │   by XXXX   │
                 └────────────┘     └─────────────┘
```

| Status | Meaning |
|--------|---------|
| **Proposed** | Under discussion |
| **Accepted** | Approved and in effect |
| **Deprecated** | No longer recommended but still documented |
| **Superseded** | Replaced by a newer ADR |

## ADR vs RFC

| Aspect | RFC | ADR |
|--------|-----|-----|
| Purpose | **Propose** a change | **Record** a decision |
| Timing | Before implementation | During/after decision |
| Approval | Requires TSC vote | Author discretion (review welcome) |
| Length | Detailed, includes alternatives | Concise, focused on rationale |

## Guidelines

### Do

- Write ADRs close to when decisions are made
- Include the "why" — context matters for future readers
- Reference related ADRs and RFCs
- Update status when decisions change

### Don't

- Write ADRs for trivial decisions
- Remove or delete old ADRs (mark as superseded instead)
- Include implementation details (that goes in code/docs)

## Existing ADRs

The project may have ADRs in various locations. Consolidate to:
```
docs/adr/XXXX-title.md
```

## Creating an ADR

```bash
# Copy template
cp docs/processes/adr-template.md docs/adr/0000-your-decision.md

# Assign next number
# Edit content
# Open PR or commit directly (for non-controversial decisions)
```


Document: docs/processes/rfc-template.md

# RFC-0000: [Title]

- **Status**: Draft | Review | Accepted | Rejected | Merged | Implemented | Superseded
- **Author(s)**: @github-username
- **Created**: YYYY-MM-DD
- **Updated**: YYYY-MM-DD
- **TSC Decision**: Pending | Accepted (YYYY-MM-DD) | Rejected (YYYY-MM-DD)
- **Supersedes**: RFC-XXXX (if applicable)
- **Superseded by**: RFC-XXXX (if applicable)

## Summary

One paragraph explaining the proposal.

## Motivation

Why are we doing this? What problem does it solve? Who benefits?

## Detailed Design

Technical details of the proposal. Include:

- API changes (before/after examples)
- New types, protocols, or interfaces
- Database schema changes
- Configuration changes
- CLI changes

### Example

```python
# Show how the feature would be used
```

## Drawbacks

- Why should we *not* do this?
- What are the risks?
- What's the maintenance burden?

## Alternatives

What other designs were considered? Why were they rejected?

| Alternative | Pros | Cons | Why Not |
|-------------|------|------|---------|
| Option A | ... | ... | ... |
| Option B | ... | ... | ... |

## Migration

How do existing users migrate?

- Is this a breaking change?
- Deprecation timeline
- Migration guide or tooling

## Unresolved Questions

- What needs further discussion?
- What will be decided during implementation?

## Implementation Plan

- [ ] Phase 1: ...
- [ ] Phase 2: ...
- [ ] Documentation updates
- [ ] Tests

## References

- Related issues: #123, #456
- Prior art: [Link to similar feature in other projects]
- Discussion threads: [Link]


Document: docs/processes/RFC_PROCESS.md

# RFC Process

> **RFC** = Request for Comments

RFCs are the mechanism for proposing significant changes to Framework M. They ensure that major decisions are documented, discussed, and deliberated before implementation.

## When to Write an RFC

**Required for:**
- New Protocols (Ports) in the core
- Breaking changes to existing APIs
- New adapters that ship with core
- Significant architectural changes
- New CLI commands or major flags
- Changes to the metadata/DocType system

**Not required for:**
- Bug fixes
- Performance improvements (without API change)
- Documentation updates
- Refactoring (without behavioral change)
- New adapters in separate packages

## RFC Lifecycle

```
┌─────────┐     ┌─────────┐     ┌──────────┐     ┌──────────┐     ┌────────────┐
│  Draft  │ ──► │ Review  │ ──► │ Accepted │ ──► │ Merged   │ ──► │ Implemented│
└─────────┘     └─────────┘     └──────────┘     └──────────┘     └────────────┘
                     │                                                   │
                     ▼                                                   ▼
               ┌──────────┐                                       ┌───────────┐
               │ Rejected │                                       │ Superseded│
               └──────────┘                                       └───────────┘
```

| Status | Meaning |
|--------|---------|
| **Draft** | Under active authoring |
| **Review** | Open for community feedback (minimum 14 days) |
| **Accepted** | TSC approved, ready for implementation |
| **Rejected** | TSC declined with documented rationale |
| **Merged** | Implementation PR merged |
| **Implemented** | Fully released |
| **Superseded** | Replaced by a newer RFC |

## Process

### 1. Create RFC

```bash
# Fork and create a new branch
git checkout -b rfc/your-proposal-name

# Copy template
cp docs/processes/rfc-template.md docs/rfcs/0000-your-proposal-name.md

# Edit and commit
```

### 2. Open Pull Request

- Title: `RFC: Your Proposal Title`
- Target: `main` branch
- Label: `rfc`

### 3. Discussion Period

- Minimum **14 days** for review
- RFC author addresses feedback
- Anyone may comment; be constructive

### 4. TSC Decision

- TSC reviews during regular meeting
- **Lazy consensus**: Approved if no objections after 7 days
- **Contested**: 2/3 TSC vote required
- Decision recorded in RFC header

### 5. Implementation

- Accepted RFCs are tracked in the project issue board.
- Implementation PRs should reference the RFC.
- **Important**: The PR that implements the feature **must** also include a commit that changes the RFC's `Status` to `Implemented` in its frontmatter. This ensures the documentation stays in sync with the codebase.

---

## RFC Template

See [rfc-template.md](rfc-template.md).

## RFC Numbering

- To avoid race conditions, RFCs are assigned numbers sequentially by the maintainers **when the PR is merged** (or approved).
- When creating a new RFC draft, use `0000` as a placeholder number in the filename (e.g., `0000-your-proposal-name.md` or `rfc-0000-your-proposal-name.md`) and within the document title.
- The core team will rename the file and update the title to the next available sequential number (`0001`, `0002`, etc.) before merging.
- Rejected RFCs may be assigned a number upon rejection for the public record.

## Where RFCs Live

```
docs/rfcs/
├── 0001-protocol-versioning.md
├── 0002-virtual-doctype-api.md
└── ...
```

---

## Tips for Authors

1. **Start with the problem**, not the solution
2. **Be specific** — avoid vague proposals
3. **Consider alternatives** — show you've thought about other approaches
4. **Keep it focused** — one major change per RFC
5. **Anticipate questions** — address likely objections upfront


Document: docs/rfcs/rfc-0003-schema-change-detection.md

# RFC-0003: Schema Change Detection & Developer Guardrails

- **Status**: Implemented
- **Author**: @revant.one
- **Created**: 2026-01-14
- **Updated**: 2026-02-22
- **Package**: `framework-m-standard`, `framework-m-studio`

## Summary

Detect and warn developers about potentially dangerous schema changes before they cause production issues. Provides guardrails with escape hatches (`--force`).

## Motivation

Developers can inadvertently make DocType changes that:

- Break existing data (incompatible type changes)
- Cause downtime (non-nullable columns without defaults)
- Lose data (dropping columns)

Framework should catch these early and guide developers toward safe patterns.

## Design Principles

1. **Studio-only**: These checks live in `framework-m-studio`, not `framework-m` runtime
2. **Runtime stays minimal**: Production container doesn't include analysis/warning code
3. **Errors with escape hatch**: Dangerous changes error by default, `--force` to override
4. **Docs-driven**: Warnings link to migration pattern documentation

## Package Placement

```
framework-m         (Runtime - Production Container)
├── Minimal core: API, ORM, DocType engine
├── No schema analysis, no heavy dev tooling
└── m prod, m migrate (apply only, no analysis)

framework-m-studio  (DevTools - Development Only)
├── Schema change analyzer
├── Migration warnings/errors
├── m migrate:create (with analysis)
├── Studio UI warnings
└── CI integration helpers
```

## Change Classification

| Risk Level    | Auto-Apply?     | CLI Behavior                          | Example                                       |
| ------------- | --------------- | ------------------------------------- | --------------------------------------------- |
| **SAFE**      | ✅ Yes          | Proceeds silently                     | Add nullable column, add table                |
| **WARNING**   | ⚠ With --force | Shows warning, blocks without --force | Add non-nullable (no default), type widening  |
| **DANGEROUS** | ❌ No           | Hard error, cannot auto-generate      | Drop column, incompatible type change, rename |

## CLI Experience (cyclopts)

```python
# apps/studio/src/framework_m_studio/cli/migrate.py
from cyclopts import App
from framework_m_studio.analysis import SchemaAnalyzer, ChangeRisk

migrate_app = App(name="migrate")

@migrate_app.command
async def create(
    name: str,
    *,
    force: bool = False,
    skip_analysis: bool = False,
):
    """Create a new migration with schema change analysis."""

    if not skip_analysis:
        analyzer = SchemaAnalyzer()
        changes = await analyzer.analyze_pending_changes()

        has_warnings = any(c.risk == ChangeRisk.WARNING for c in changes)
        has_dangerous = any(c.risk == ChangeRisk.DANGEROUS for c in changes)

        # Display changes
        for change in changes:
            display_change(change)

        if has_dangerous:
            console.print("[red]❌ DANGEROUS changes detected.[/red]")
            console.print("Cannot auto-generate migration. Write manually.")
            console.print(f"See: {change.docs_url}")
            raise SystemExit(1)

        if has_warnings and not force:
            console.print("[yellow]⚠ Warnings detected.[/yellow]")
            console.print("Use --force to proceed anyway.")
            raise SystemExit(1)

    # Generate migration
    await generate_migration(name)
```

### Example Output

```
$ m migrate:create add_phone_field

📊 Schema Change Analysis (framework-m-studio)

  ✅ SAFE: Adding 'notes' (nullable Text) to Customer

  ⚠  WARNING: Adding 'phone' (non-nullable String) to Customer
     │ Existing rows will cause migration to fail.
     │ Recommendation: Add with default or make nullable.
     │ Docs: https://framework-m.dev/migrations/zero-downtime
     │
     └─ Use --force to proceed anyway.

  ❌ DANGEROUS: Changing 'amount' from String → Decimal in Invoice
     │ Incompatible type change requires manual data migration.
     │ Pattern: Two-Phase Migration
     │ Docs: https://framework-m.dev/migrations/type-changes
     │
     └─ Cannot auto-generate. Write migration manually.

Migration NOT created.
```

## Studio UI Integration

When saving a DocType in Studio:

```
┌─────────────────────────────────────────────────────────────────┐
│  Save DocType: Customer                                         │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  ⚠  Schema Change Warning                                      │
│  ─────────────────────────                                      │
│  Adding 'phone' as non-nullable without default.                │
│  This will fail if Customer table has existing rows.            │
│                                                                 │
│  Recommendations:                                               │
│  • Add a default value: phone: str = ""                         │
│  • Make nullable: phone: str | None = None                      │
│                                                                 │
│  [Learn More]  [Save Anyway]  [Cancel]                          │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘
```

## Schema Analyzer Implementation

```python
# apps/studio/src/framework_m_studio/analysis/schema_analyzer.py
from enum import Enum
from pydantic import BaseModel

class ChangeRisk(Enum):
    SAFE = "safe"
    WARNING = "warning"
    DANGEROUS = "dangerous"

class SchemaChange(BaseModel):
    doctype: str
    field: str | None
    change_type: str  # "add_field", "drop_field", "modify_type", "add_table"
    risk: ChangeRisk
    message: str
    recommendation: str | None
    docs_url: str | None

class SchemaAnalyzer:
    """Analyzes DocType changes for migration risk."""

    DOCS_BASE = "https://framework-m.dev/migrations"

    async def analyze_pending_changes(self) -> list[SchemaChange]:
        """Compare registered DocTypes with database schema."""
        # Load current DB schema
        # Load current Pydantic models
        # Diff and classify
        ...

    def _classify_new_field(self, field_info) -> ChangeRisk:
        if field_info.is_required and field_info.default is None:
            return ChangeRisk.WARNING
        return ChangeRisk.SAFE

    def _classify_type_change(self, old_type, new_type) -> ChangeRisk:
        if self._is_compatible_widening(old_type, new_type):
            return ChangeRisk.WARNING
        return ChangeRisk.DANGEROUS

    def _classify_drop_field(self) -> ChangeRisk:
        return ChangeRisk.DANGEROUS
```

## CI Integration

```yaml
# .gitlab-ci.yml
schema-check:
  stage: test
  image: framework-m-studio:latest
  script:
    - m migrate:create --dry-run --fail-on-warning
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
```

## Documentation Links

Each change type links to specific pattern documentation:

| Change                       | Linked Pattern           |
| ---------------------------- | ------------------------ |
| Non-nullable without default | Zero-Downtime Migrations |
| Compatible type widening     | Review-Required Changes  |
| Incompatible type change     | Two-Phase Migration      |
| Drop column                  | Expand-Contract Pattern  |
| Rename column                | Two-Phase Migration      |

## Open Questions

1. **Should we detect changes at file save in Studio?**
   - Pro: Immediate feedback
   - Con: Noisy if developer is mid-edit

2. **Track "acknowledged" warnings?**
   - Once --force is used, remember for that change?
   - Or always warn on every run?

3. **Integration with CI merge checks?**
   - GitLab/GitHub status check integration?

## Implementation Phases

- [ ] **Phase 1**: Core analyzer (compare Pydantic ↔ DB schema)
- [ ] **Phase 2**: CLI integration with cyclopts
- [ ] **Phase 3**: Studio UI warnings
- [ ] **Phase 4**: CI helpers and status checks

## References

- Phase 10: Migration Strategy
- Phase 07: Studio
- [dependency-injector wiring](https://python-dependency-injector.ets-labs.org/)


Document: docs/rfcs/rfc-0004-submitted-document-mutability.md

# RFC-0004: Submitted Document Mutability

- **Status**: Rejected
- **Author**: @revant.one
- **Created**: 2026-02-13
- **Updated**: 2026-02-24
- **TSC Decision**: Rejected (Anti-Pattern)

## Summary

This RFC originally proposed a "Field-Level Mutability" model for Submitted Docs using an `allow_on_submit` flag. Based on compliance and auditability reviews, this proposal is formally **rejected**. Submitted documents in Framework M are strictly immutable.

## Motivation for Rejection

In business applications, documents are often parsed into:
1.  **Ledger Data**: Quantities, Rates, Items, Posting Dates. (Strictly immutable)
2.  **Metadata**: Transporter details, internal remarks, dispatch status. (Historically considered mutable)

**The Flaw in `allow_on_submit`:**
While frameworks like Frappe support an "Allow on Submit" feature to let users change metadata without amending the document, this is fundamentally an anti-pattern. A submitted document represents an immutable, legally binding, or finalized state. Allowing post-submission mutations (even for metadata) compromises the database row as a single source of truth.

## The Framework M Standard: Strict Immutability

Framework M enforces strict immutability for all documents where `docstatus == SUBMITTED`. The only permitted change is a state transition to `CANCELLED`.

If a user needs to update information related to a submitted document, they must use one of the following compliant patterns:

### 1. The Amend Pattern (For Core Corrections)
If the document itself is incorrect (e.g., wrong address, wrong item), the user must:
1. **Cancel** the original document (`docstatus` goes to 2).
2. **Amend** it (create a new draft document linked to the cancelled one).
3. Update the data and **Submit** the new version.

### 2. The Sibling Entity Pattern (For Volatile Metadata)
If the document requires ongoing operational updates (e.g., updating a Transporter Name or Dispatch Status on a Delivery Note), these fields **should not exist** on the submitted document schema.

Instead, create a separate "volatile" DocType:
```python
class DeliveryDispatchInfo(DocType):
    delivery_note: Annotated[str, Link("DeliveryNote")]
    vehicle_no: str
    dispatch_status: str
```
This allows the operational data to change independently without breaking the immutability of the financial/ledger document.

### 3. The Timeline / Event Pattern
If the user simply wants to add a "Remark" or note to a submitted document, this should be implemented as an append-only Comment via the `TimelineMixin`. This provides an auditable, timestamped log without attempting to mutate the underlying document row.

## Implementation Standard

- The `BaseController` must forcibly reject any `save` operation on a `SUBMITTED` document where fields (other than `docstatus`) have changed by raising `ImmutableDocumentError`.
- No properties like `allow_on_submit` will be supported in the schema.