Skip to main content

Decoupled File Storage Adapters

· 3 min read
Ansh Pansuriya
Ansh Pansuriya
Main Developer of Framework M

Managing user-uploaded files, media assets, and generated PDF invoices in modern cloud applications requires flexibility. Hardcoding file uploads to local disk directories makes horizontal scaling impossible, while hardcoding direct AWS S3 calls couples your codebase tightly to a single cloud provider.

Framework M solves asset management by routing all file upload streams through an abstract, pluggable Storage Adapter architecture.

1. Storage Architecture Overview​

In Framework M, file attachments associated with DocTypes are managed via the standard File DocType metadata entity, while the underlying binary streams are handled by configurable StorageProtocol implementations:

Storage Adapter Stream Sequence

2. Standard Storage Adapters & Configuration​

Framework M provides three standard, production-ready storage adapters out of the box:

  • LocalStorageAdapter (local): Standard local filesystem storage with path sanitization, atomic writes, and public/private asset partitioning. Ideal for single-server and on-premise deployments.
  • InMemoryStorageAdapter (memory): In-memory binary buffer storage designed for fast unit testing and ephemeral processing pipelines.
  • S3StorageAdapter (s3): Enterprise cloud storage supporting AWS S3, Cloudflare R2, MinIO, and DigitalOcean Spaces with presigned download URLs and streaming multipart uploads.

Configuration in framework_config.toml​

Switching storage adapters requires zero code changes. Configuration is set in framework_config.toml:

# framework_config.toml

[files]
storage_backend = "s3"
s3_bucket = "company-framework-m-assets"
s3_region = "us-east-1"
s3_endpoint = "https://s3.us-east-1.amazonaws.com"

Configuration via Environment Variables​

Production container environments can supply configuration directly via environment variables:

export FRAMEWORK_FILES_STORAGE_BACKEND="s3"
export S3_BUCKET="company-framework-m-assets"
export S3_REGION="us-east-1"
export AWS_ACCESS_KEY_ID="AKIAIOSFODNN7EXAMPLE"
export AWS_SECRET_ACCESS_KEY="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"

3. Streaming Chunked Uploads & Security​

To protect servers against memory exhaustion during large file uploads, Framework M processes files as async streams using chunked buffering:

from framework_m_standard.adapters.storage import (
    create_storage_adapter,
    get_file_config,
)

config = get_file_config()
storage = create_storage_adapter(config)

# Save file stream directly to storage backend
file_info = await storage.save(
    filename="document.pdf", content_stream=upload_file_stream, is_private=True
)

print(f"Stored file path: {file_info.file_url}")

Security & Access Control:​

  • Private Asset Enclosure: Files marked is_private=True cannot be accessed via direct public URLs. Requests pass through Framework M authorization middleware to verify user read permissions before serving the asset or generating temporary presigned URLs.
  • Path Traversal Shield: All incoming filenames undergo strict sanitization to eliminate path traversal attacks (../).

Guides & Documentation​

Explore file storage architecture and configuration in our docs: