Engineering Deep Dive & Security
Under the Hood: Serverless GPU Architecture & Zero-Retention Privacy in Docling Web
How we built an instant browser interface for IBM Docling with sub-3 second latency, $0 idle compute standby costs, and strict in-memory file shredding.
1. The Architectural Challenge
Hosting modern vision-language models like Granite-Docling and TableFormer on traditional cloud VMs (e.g. AWS EC2 g4dn.xlarge) creates a painful trade-off:
- A dedicated GPU instance costs $380–$500/month 24/7, even when no users are converting documents.
- If traffic spikes on Hacker News or Reddit, a single VM gets overwhelmed, queue times explode, and users abandon the site.
- Cold-starting deep neural nets with 4GB+ weights typically takes 40–90 seconds on standard Docker containers.
2. The Scale-to-Zero Dual-Track Solution
SheetParser implements an elastic, dual-track dynamic routing architecture powered by Modal.com and Cloudflare Pages Edge Functions:
┌────────────────────────────────────────────────────────┐
│ Cloudflare Global Edge Proxy (/api/convert) │
└────────────────────────────────────────────────────────┘
│
├──► Track A: Digital Text Fast CPU ($0.001 / run)
└──► Track B: TableFormer & Scanned GPU (NVIDIA T4 on Modal)
- ✓Pre-warmed Container Snapshotting: PyTorch and model weights are baked into snapshot memory, reducing cold start times to ~0.3 seconds.
- ✓True Scale-to-Zero: When no requests are active, GPU instances shut down completely ($0 idle cost). When 100 users upload simultaneously, Modal spins up parallel worker containers instantly.
3. Zero-Data-Retention Security & Cryptographic Shredding
Researchers and financial analysts deal with proprietary preprints, unpublished patents, and confidential 10-K filings. Free online converters that log files to S3 buckets are unacceptable.
Our Security Guarantee:
- 🔒 In-memory execution: PDFs are buffered in volatile RAM, never written to persistent disk.
- 🔒 Immediate shredding: The moment the response payload is returned to your browser, memory buffers are explicitly wiped.
- 🔒 Zero AI training: Your documents are never logged, stored, or used to train language or vision models.
Experience Secure, Instant Document Intelligence
Convert your sensitive documents with confidence on SheetParser.
Launch Secure Web Converter