Skip to main content

How-To: Integrate with Dify Knowledge Base

This guide outlines how to sync compiled machine assets (.jsonl) to your Dify knowledge base segment-by-segment to prevent random text chunking.

1. Get Dify API Credentials​

  1. Log into your Dify Dashboard.
  2. Navigate to Knowledge -> click Create Dataset (or select an existing one).
  3. Navigate to your Dataset's API Access tab (located in the top menu bar of the dataset view).
  4. Generate and copy the API Token / API Key.
  5. Copy the Dataset ID from the API Access documentation page or extract it from the dashboard URL: https://cloud.dify.ai/datasets/{dataset_id}/documents

2. Syncing Datasets Using dify-sync.py​

Dify's default file uploader is designed for long text and splits files randomly by token counts. This splits Python code classes or OpenAPI JSON objects in half.

To prevent this, use the local synchronization script bin/dify-sync.py:

# Set your environment variables
export DIFY_API_KEY="your-dify-dataset-api-key"
export DIFY_DATASET_ID="your-dataset-uuid"

# Sync API contracts (openapi.jsonl)
python3 bin/dify-sync.py \
  --file docs/machine/openapi.jsonl \
  --api-key $DIFY_API_KEY \
  --dataset-id $DIFY_DATASET_ID

# Sync core AST structures (core.jsonl)
python3 bin/dify-sync.py \
  --file docs/machine/core.jsonl \
  --api-key $DIFY_API_KEY \
  --dataset-id $DIFY_DATASET_ID

# Sync DocTypes metadata (doctypes.jsonl)
python3 bin/dify-sync.py \
  --file docs/machine/doctypes.jsonl \
  --api-key $DIFY_API_KEY \
  --dataset-id $DIFY_DATASET_ID

# Sync the full text RAG corpus (corpus.jsonl)
python3 bin/dify-sync.py \
  --file docs/machine/corpus.jsonl \
  --api-key $DIFY_API_KEY \
  --dataset-id $DIFY_DATASET_ID

# Sync UI primitives & component props (ui.jsonl)
python3 bin/dify-sync.py \
  --file docs/machine/ui.jsonl \
  --api-key $DIFY_API_KEY \
  --dataset-id $DIFY_DATASET_ID

3. Configuring the Dify Agent Prompt​

Once the dataset is imported into Dify, associate the knowledge base with your Dify Coding Agent. Update the agent's System Prompt / Instructions to teach it how to interpret the JSONL structures:

You are a coding assistant specializing in Framework M development. 
You have access to the Framework M Knowledge Base containing machine-readable AST structures and OpenAPI specifications.

### Knowledge Base Format Guidelines
1. **Class Definitions (`type: class`)**:
   - Understand the method signatures and generic type structures.
   - Strictly follow argument types and return annotations.
2. **API Routes (`method` and `path`)**:
   - Each endpoint block is self-contained.
   - Use the exact parameters and schemas when generating client calls.
3. **Golden Fixtures (`type: fixture`)**:
   - Refer to these fixtures as the "source of truth" coding style.
   - Do not hallucinate imports or inject arguments outside standard patterns.