Fault-Tolerant Functional Programming with Automatic Checkpointing
Never lose your computational progress again.
Quick Start • Reference • Tutorials • Changelog
Long-running computations in R are vulnerable to interruptions:
# Processing 10,000 API calls...
result <- purrr::map(urls, fetch_data)
# ❌ Crashes at item 9,847 after 3 hours
# ❌ All progress lost
# ❌ Must restart from scratchCommon failure scenarios:
- R session crashes or runs out of memory
- Network timeouts during API calls
- System restarts or power failures
- Accidental interruption (Ctrl+C)
SafeMapper provides drop-in replacements for purrr and furrr functions with automatic checkpoint-based recovery:
# Same code, but fault-tolerant
result <- s_map(urls, fetch_data)
# ⚡ Crashes at item 9,847...
# Just re-run the same code:
result <- s_map(urls, fetch_data)
# ✅ "Resuming from checkpoint: 9800/10000 items completed"
# ✅ Continues from where it left off
# ✅ No configuration needed# From r-universe (recommended)
install.packages("SafeMapper", repos = "https://zaoqu-liu.r-universe.dev")
# From GitHub
devtools::install_github("Zaoqu-Liu/SafeMapper")library(SafeMapper)
# Replace purrr::map() with s_map() - that's it!
results <- s_map(1:1000, function(x) {
Sys.sleep(0.1) # Simulate slow operation
x^2
})
# If interrupted, just re-run - automatic recovery!📖 See full tutorial: Quick Start Guide
| Feature | Description | Learn More |
|---|---|---|
| Zero Configuration | Works out of the box - no setup required | Quick Start |
| Automatic Recovery | Detects previous runs and resumes automatically | Core Concepts |
| Drop-in Replacement | Same API as purrr and furrr |
Map Functions |
| Transparent Checkpointing | Progress saved at configurable intervals | Session Management |
| Parallel Support | Full furrr compatibility for parallel processing |
Parallel Processing |
| Robust Error Handling | Built-in retry and error capture | Error Handling |
Sequential Processing (purrr replacements)
| SafeMapper | purrr | Returns | Docs |
|---|---|---|---|
s_map() |
map() |
list | 📖 |
s_map_chr() |
map_chr() |
character | 📖 |
s_map_dbl() |
map_dbl() |
numeric | 📖 |
s_map_int() |
map_int() |
integer | 📖 |
s_map_lgl() |
map_lgl() |
logical | 📖 |
s_map_dfr() |
map_dfr() |
data.frame (row-bind) | 📖 |
s_map_dfc() |
map_dfc() |
data.frame (col-bind) | 📖 |
s_map2() |
map2() |
list (two inputs) | 📖 |
s_pmap() |
pmap() |
list (multiple inputs) | 📖 |
s_imap() |
imap() |
list (with index) | 📖 |
s_walk() |
walk() |
side effects | 📖 |
Parallel Processing (furrr replacements)
| SafeMapper | furrr | Docs |
|---|---|---|
s_future_map() |
future_map() |
📖 |
s_future_map2() |
future_map2() |
📖 |
s_future_pmap() |
future_pmap() |
📖 |
s_future_walk() |
future_walk() |
📖 |
s_future_imap() |
future_imap() |
📖 |
All variants (_chr, _dbl, _int, _lgl, _dfr, _dfc) are supported.
| SafeMapper | purrr | Description | Docs |
|---|---|---|---|
s_safely() |
safely() |
Capture errors | 📖 |
s_possibly() |
possibly() |
Return default on error | 📖 |
s_quietly() |
quietly() |
Capture messages/warnings | 📖 |
📚 Full API Reference: Reference Index
# Optional: customize settings
s_configure(
batch_size = 100, # Items per checkpoint (default: 100)
retry_attempts = 3 # Retry failed batches (default: 3)
)
# Clean old checkpoint files
s_clean_sessions(older_than_days = 7)📖 Learn more: Session Management Guide
┌─────────────────────────────────────────────────────────────┐
│ First Execution │
├─────────────────────────────────────────────────────────────┤
│ Input Data ──► Fingerprint ──► Process Batches │
│ [1:1000] "abc123..." [1-100] ✓ checkpoint │
│ [101-200] ✓ checkpoint │
│ [201-300] ✗ CRASH! │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Re-execution │
├─────────────────────────────────────────────────────────────┤
│ Input Data ──► Fingerprint ──► Find Checkpoint │
│ [1:1000] "abc123..." "200 items completed" │
│ │
│ Resume from 201 ──► Complete! │
└─────────────────────────────────────────────────────────────┘
- Fingerprinting: Each task is identified by a hash of input data characteristics
- Checkpointing: Results are saved to disk at batch intervals
- Recovery: On re-run, matching fingerprints trigger automatic restoration
- Cleanup: Checkpoints are removed after successful completion
📖 Deep dive: Core Concepts & Architecture
| Use Case | Description | Example |
|---|---|---|
| API Data Collection | Web scraping, REST API calls with rate limits | View Example |
| File Processing | ETL pipelines, batch transformations | View Example |
| Machine Learning | Cross-validation, hyperparameter tuning | View Example |
| Web Scraping | Extracting data from thousands of pages | View Example |
| Bioinformatics | Processing large genomic datasets | View Example |
| Database Migration | Moving data between systems | View Example |
📖 All examples: Real-World Examples
| Tutorial | Description | Time |
|---|---|---|
| 🚀 Quick Start | Get up and running | 5 min |
| 🧠 Core Concepts | Understand the architecture | 15 min |
| Tutorial | Description | Level |
|---|---|---|
| 🗺️ Map Functions | Complete guide to all s_map variants | Intermediate |
| ⚡ Parallel Processing | Speed up with s_future_map | Intermediate |
| 🛡️ Error Handling | s_safely, s_possibly, s_quietly | Intermediate |
| 📋 Session Management | Configure and manage checkpoints | Intermediate |
| Tutorial | Description | Level |
|---|---|---|
| 🎯 Real-World Examples | Complete production examples | Advanced |
| 🏆 Best Practices | Production patterns & anti-patterns | Advanced |
Contributions are welcome! Please feel free to submit a Pull Request or open an Issue.
Zaoqu Liu
- 📧 Email: liuzaoqu@163.com
- 🐙 GitHub: @Zaoqu-Liu
- 🔬 ORCID: 0000-0002-0452-742X
MIT © 2026 Zaoqu Liu