Phase 07: Studio (Code Generation UI)

Objective: Build a visual DocType builder (framework-m-studio) that generates Python code, running as a Litestar app.

1. Packaging Strategy (Runtime vs DevTools)

1.1 framework-m (Runtime)

Goal: Production-grade kernel.
Content: API Server, ORM, DocType Engine, Basic CLI (start, migrate, worker).
Dependencies: litestar, sqlalchemy, pydantic, cyclopts.
Excluded: Code generators, Studio UI, Heavy formatting/linting libs.

1.2 framework-m-studio (DevTools)

Goal: Developer Productivity & SaaS Component.
Content: Studio UI, Code Generators, Visual Editors.
Dependencies: framework-m, libcst, jinja2.
Integration: Plugs into m CLI via entry points (m codegen).

2. Studio Backend (Litestar App)

1. Project Structure (Monorepo)

Create apps/studio/ in the workspace
Initialize apps/studio/pyproject.toml:
- Add dependency: framework-m = { workspace = true }
- Add dependency: libcst (for code modification)
- Add dependency: jinja2 (for code generation)
- Add code generation endpoints

1.2 File System API

2. LibCST Code Transformer

2.1 Parser

Create src/framework_m_studio/codegen/parser.py
Implement parse_doctype(file_path: str) -> dict:
- Use LibCST to parse Python file
- Extract class definition
- Extract fields with types and defaults
- Extract Config metadata
- Return structured dict

2.2 Generators (The Strategy)

"Jinja for Creation, LibCST for Mutation"

Creation (Scaffolding):
- Use jinja2 templates (shared with CLI m new:doctype).
- Goal: Generate clean, standard Python code from scratch.
- Implement generate_doctype_source(schema: dict) -> str.
Mutation (Transformer):
- Use LibCST to parse and modify existing files.
- Goal: Add/Edit fields while preserving comments and custom methods.
- Implement update_doctype_source(source: str, schema: dict) -> str.

2.3 Test Generator

Create src/framework_m_studio/codegen/test_generator.py
Implement generate_test(schema: dict) -> str:
- Generate test_{doctype}.py
- Import pytest and DocType
- Generate basic CRUD test (Create, Read, Update, Delete)
- Generate validation failure test (if required fields exist)

2.4 Transformer

3. Studio Frontend

Stack: Per ADR-0005 (Use Refine for Frontend) — Refine + shadcn/ui + RJSF + Tailwind

3.1 React App Setup (Refine Stack)

Create apps/studio/studio_ui/ directory

Initialize Refine app:

pnpm create refine-app@latest studio_ui
# Selected: refine-vite, custom-json-rest, no UI, no examples, pnpm

Install dependencies per ADR-0005:

Layer	Package	Purpose
Data/State	`@refinedev/core`	CRUD, caching, auth
Forms	`@rjsf/core` + `@rjsf/validator-ajv8`	Schema-driven forms
Tables	`@tanstack/react-table`	Data tables
UI Components	`shadcn/ui`	Accessible components
Styling	`tailwindcss`	Utility-first CSS
Code Editor	`@monaco-editor/react`	Python code preview
Drag & Drop	`@dnd-kit/core` + `@dnd-kit/sortable`	Field reordering

Create Framework M Data Provider (src/providers/dataProvider.ts):

// Maps Refine's DataProvider to Framework M API
const dataProvider = frameworkMDataProvider(config.apiBaseUrl);

Configure API base URLs (per ADR-0005):
- Read from window.__FRAMEWORK_CONFIG__ or env vars
- Support same-origin, subdomain, and CDN scenarios
- Default to /studio/api for Studio endpoints

3.2 DocType List View (Refine Resource)

Create pages/doctypes/list.tsx:
- Use useList hook from @refinedev/core to fetch DocTypes
- Display with @tanstack/react-table + Tailwind styling
- Add search/filter with useTable globalFilter
- Add "New DocType" button → navigates to /doctypes/create

3.3 DocType Editor (Refine Form)

Create pages/doctypes/edit.tsx:
- Use useOne/useUpdate/useCreate hooks for data
- Left panel: Field list with @dnd-kit/sortable
- Right panel: Field properties form (name, type, required, default, description)
- Top bar: DocType name input + Save button

3.4 Field Editor (RJSF Schema-Driven)

Create components/FieldEditor.tsx:
- Render with RJSF using JSON Schema for field properties
- Custom Tailwind widgets: TextWidget, SelectWidget, CheckboxWidget, TextareaWidget
- Dynamic "Validators" collapsible for min/max length, pattern, etc.
- Live preview of generated Python type annotation

3.5 Drag & Drop

Implement field reordering:
- Use @dnd-kit/core + @dnd-kit/sortable
- Allow dragging fields to reorder
- Update schema on drop (via arrayMove)

3.6 Code Preview (Monaco Editor)

Create components/CodePreview.tsx:
- Use @monaco-editor/react with language="python"
- Read-only mode (readOnly: true)
- Theme: vs-dark
- Auto-update on schema change (memoized generation)
- Copy button for generated code

3.7 Save Flow

Implement save functionality:
- Validate schema (name required)
- Call backend API via useCreate/useUpdate
- Navigate back to list on success
- Loading state on save button

3.8 Visual Data Modeling (Mind Map)

Implement ERD/Mind Map View:
- Central Node: Current DocType (highlighted)
- Connected Nodes: Related DocTypes via Link fields
- Visual Action: Drag to connect nodes → Creates Link Field
- Using @xyflow/react (React Flow v12)

3.9 Layout Designer (Grid System)

Implement Visual Layout Drag & Drop:
- Define Sections with title and Columns (1/2/3 cols selector)
- Drag fields from palette into grid cells
- Real-time form preview toggle
- Add/delete sections

3.10 Module Explorer

Implement Module Scanner:
- Group DocTypes by directory/app module (tree structure)
- Tree view navigation in Sidebar
- Expand/collapse all, search filter
- Click to navigate to DocType editor

4. Studio Cloud Mode (Ephemeral & Git-Backed)

"Stateless Studio": No RWX storage required. Repo is cloned to ephemeral path.

[!NOTE] Workspace Terminology

Studio Workspace = The Git monorepo where code lives (this section)

Desk Workspace = End-user navigation groupings in sidebar (Phase 09, Section 6.2)

A Studio Workspace (monorepo) can contain multiple Desk Workspace definitions in code.

4.1 Architecture

Storage: /tmp/workspace_{session_id} (Ephemeral/Pod-local). ✅
Persistence: Git Provider (GitHub/GitLab). ✅
Concurrency: Optimistic Locking (Last push wins, or Rebase flow). ✅
Deployment: StatefulSet or Deployment (Replicas=1).
- Reasoning: Workspaces are local to the Pod. Scaling requires Sticky Sessions or RWX.
- Recommendation: Keep Studio as a Singleton. Scale the Application, not the Editor.

4.2 GitOps Integration (FluxCD/ArgoCD)

Configuration: Inject GIT_REPO, GIT_TOKEN via Environment Variables. ✅
Read State: Clone target repo on startup (or user connect). ✅
Write State:
- Create Branch + Open PR (Review Process). ✅
Update State:
- Implement "Pull Latest" Button (POST /workspace/{id}/pull).
- Show "Updates Available" indicator (POST /workspace/{id}/check-updates).

4.3 Git Adapter (Data Agnostic)

4.4 GitHub Provider

Create src/framework_m_studio/git/github_provider.py
Implement GitHubProvider:
- Use GitHub API for operations (REST API with urllib)
- Support personal access tokens (Bearer auth)
- Handle authentication (GitHubAuthError)
- Create pull requests
- List pull requests
- Validate tokens
Unit tests in tests/test_github_provider.py (9 tests)

4.5 Generic Git Provider

Note: Implemented as GitAdapter using git CLI - no need for separate provider.

Support HTTPS and SSH (via git CLI)
Work with any Git server (generic implementation)

4.6 Workspace Management

Create src/framework_m_studio/workspace.py
Implement workspace lifecycle:
- Clone repo to temp directory
- Track workspace sessions
- Clean up old workspaces (TTL-based)
- Handle concurrent edits (asyncio lock)
Unit tests in tests/test_workspace.py (12 tests)

4.7 Cloud Studio API

5. Studio CLI Command

5.1 SPA Packaging & Serving (Path Prefix Architecture)

Build Process:
- pnpm run build in studio_ui outputs to src/framework_m_studio/static/ (vite.config.ts).
- pyproject.toml includes framework_m_studio/static/**/*.
Serving Strategy (Litestar) (in app.py):
- UI Routes: Serve /studio/ui/* → React SPA (HTML pages)
- API Routes: Serve /studio/api/* → JSON endpoints
- Assets: Serve /studio/ui/assets/* as static files (Cached)
- SPA Catch-all: /studio/ui/{path:path} → returns index.html for client-side routing
- Redirect: /studio → /studio/ui/ for convenience
- Content-Type Fix: All responses use explicit media_type to prevent download issues
- Implementation: serve_spa() and _get_spa_response() functions in app.py
Implement m studio (in framework-m-studio package):
- Register CLI command via entry point:
```
[project.entry-points."framework_m.cli_commands"]
studio = "framework_m_studio.cli:studio_app"
```
- Start uvicorn server on port 9000 (default)
- Serve Studio UI
- Open browser automatically (TODO)
- Options:
  - --port - Custom port
  - --host - Custom host
  - --reload - Development mode
  - --cloud - Enable cloud mode

6. DevTools CLI (Extensions)

"cli needs clear separation... m docs|codegen needs to be part of studio"

These commands are registered via entry_points in framework-m-studio. They are NOT available in the base framework-m package.

6.1 Documentation Generator (`m docs:generate`)

Dependencies: Uses stdlib urllib, json, subprocess (no external deps required).
Command: m docs:generate:
- API Reference: Generates markdown from DocTypes in docs_generator.py.
- OpenAPI Export: Exports openapi.json from URL (--openapi-url).
- Site Build: Runs mkdocs build if available (--build-site).
Unit tests in tests/test_docs_generator.py (10 tests)

6.2 Client SDK Generator (`m codegen client`)

Dependencies: Uses stdlib only (no external deps required).
Command: m codegen client:
- Flow:
  1. Fetches OpenAPI JSON from URL.
  2. Generates code using stdlib.
- Arguments:
  - --lang [ts|py]: Target language.
  - --out [dir]: Output directory.
  - --openapi-url: URL to fetch schema from.
- TypeScript: Generates interfaces + fetch client in sdk_generator.py.
- Python: Generates Pydantic models in sdk_generator.py.
Unit tests in tests/test_codegen.py (6 tests)
CLI tests updated in tests/test_cli.py

7. Field Type Library

7.1 Supported Field Types

7.2 Custom Field Type Discovery

Dynamic Loading: Studio reads available types from FieldRegistry at runtime.
- API Endpoint: GET /studio/api/field-types
- Returns: Built-in types + Types registered by installed apps.
- Each type includes: name, pydantic_type, sqlalchemy_type, ui_component (optional).
UI Integration: Field Type Selector in DocType Editor dynamically populates from this API.

7.3 Custom UI Components (Client-Side)

Goal: Allow apps to register custom React components for their field types.

Registration (App-Side):

// my_app/frontend/index.ts
import { registerFieldComponent } from '@framework-m/studio';
import StarRating from './components/StarRating';

registerFieldComponent('Rating', StarRating);

Discovery (Studio-Side):
- Shadow Build (Phase 09) loads all app frontend/ plugins.
- Studio UI checks window.__STUDIO_FIELD_COMPONENTS__ map.
- Falls back to default <input> if no custom component.

7.5 Live Preview & Sandbox Mode

[!IMPORTANT] Sandbox = Local Development Environment The Sandbox runs on the same machine where Studio is running. It is NOT a separate cloud environment. This enables citizen developers to design, test, and iterate locally before committing to Git.

Citizen Developer Workflow:

┌─────────────────────────────────────────────────────────────────────┐
│  LOCAL MACHINE (Developer or Citizen)                               │
│                                                                     │
│   1. Studio UI          2. Sandbox              3. Git              │
│   ┌──────────────┐     ┌──────────────┐       ┌──────────────┐      │
│   │ Design       │ ──► │ Test with    │ ──►   │ Commit &     │      │
│   │ DocType      │     │ mock data    │       │ Push         │      │
│   └──────────────┘     └──────────────┘       └──────────────┘      │
│                                                      │              │
└──────────────────────────────────────────────────────│──────────────┘
                                                       ▼
                                              CI/CD → Production

[!WARNING] Collaborative Environments: Conflict Risk When multiple citizens work on the same repo/branch, local sandboxes can cause Git conflicts. The sandbox is best suited for solo work or feature branches.

Alternatives for Collaborative Teams:

Approach	How It Works	Best For
Feature Branches	Each citizen works on their own branch → PR to merge	Small teams (2-5)
Gitpod/Codespaces	Each citizen gets isolated cloud dev environment	Larger teams
Assigned Modules	Citizens own specific modules/apps, no overlap	Enterprise
Trunk-Based + Fast CI	All push to main, CI deploys in `<2 min`	Mature teams
Studio Cloud Mode	Hosted Studio with workspace isolation (Section 4)	SaaS offering

7.6 Sandbox: Interactive CRUD Simulation

[!NOTE] Why Mock Data? Sandbox uses mock data so citizens can test the full UX (forms, lists, validation) without needing a database setup. Once satisfied, they commit the code and deploy to an environment with real data.

7.7 Local Hot Reload (Optional Enhancement)

Goal: For users who want to test with a real local database instead of mock data.
Prerequisites: Local Postgres/SQLite + m start running.
Flow:
- Studio saves DocType to file.
- File watcher detects change → triggers schema sync.
- Table created/updated in local DB.
- User can now test with real CRUD operations.
Benefit: Bridge between mock sandbox and production—test with actual data locally.

7.2 Field Metadata

8. Controller Scaffolding

9. Testing

9.1 Unit Tests

9.2 Integration Tests

10. Documentation

Validation Checklist

Before moving to Phase 08, verify:

Studio UI can create and edit DocTypes
Generated code is valid Python
Changes are saved to file system
LibCST preserves custom code
Git mode can commit and push
Studio works in both local and cloud modes

Anti-Patterns to Avoid

❌ Don't: Store DocType metadata in database like Frappe ✅ Do: Generate Python source code

❌ Don't: Use AST (destroys formatting) ✅ Do: Use LibCST (preserves formatting)

❌ Don't: Overwrite custom code ✅ Do: Preserve comments and custom methods

❌ Don't: Require complex setup for Studio ✅ Do: Make it work in browser with Git

1. Packaging Strategy (Runtime vs DevTools)​

1.1 framework-m (Runtime)​

1.2 framework-m-studio (DevTools)​

2. Studio Backend (Litestar App)​

1. Project Structure (Monorepo)​

1.2 File System API​

2. LibCST Code Transformer​

2.1 Parser​

2.2 Generators (The Strategy)​

2.3 Test Generator​

2.4 Transformer​

3. Studio Frontend​

3.1 React App Setup (Refine Stack)​

3.2 DocType List View (Refine Resource)​

3.3 DocType Editor (Refine Form)​

3.4 Field Editor (RJSF Schema-Driven)​

3.5 Drag & Drop​

3.6 Code Preview (Monaco Editor)​

3.7 Save Flow​

3.8 Visual Data Modeling (Mind Map)​

3.9 Layout Designer (Grid System)​

3.10 Module Explorer​

4. Studio Cloud Mode (Ephemeral & Git-Backed)​

4.1 Architecture​

4.2 GitOps Integration (FluxCD/ArgoCD)​

4.3 Git Adapter (Data Agnostic)​

4.4 GitHub Provider​

4.5 Generic Git Provider​

4.6 Workspace Management​

4.7 Cloud Studio API​

5. Studio CLI Command​

5.1 SPA Packaging & Serving (Path Prefix Architecture)​

6. DevTools CLI (Extensions)​

6.1 Documentation Generator (m docs:generate)​

6.2 Client SDK Generator (m codegen client)​

7. Field Type Library​

7.1 Supported Field Types​

7.2 Custom Field Type Discovery​

7.3 Custom UI Components (Client-Side)​

7.5 Live Preview & Sandbox Mode​

7.6 Sandbox: Interactive CRUD Simulation​

7.7 Local Hot Reload (Optional Enhancement)​

7.2 Field Metadata​

8. Controller Scaffolding​

9. Testing​

9.1 Unit Tests​

9.2 Integration Tests​

10. Documentation​

Validation Checklist​

Anti-Patterns to Avoid​