Trove: Multi-Backend Object Storage for Go

Trove is a multi-backend object storage engine for Go that delivers a unified API across storage backends. It answers the fundamental question: "How do I store and retrieve objects with maximum flexibility and zero vendor lock-in?"

Trove supports multiple storage backends through dedicated drivers:

• Local Filesystem — Atomic file operations with sidecar .meta.json metadata
• In-Memory — Ephemeral storage for testing and caching
• Amazon S3 — S3-compatible storage (AWS, MinIO, R2, DigitalOcean Spaces)
• Google Cloud Storage — GCS with resumable uploads and compose-based multipart
• Azure Blob — Block blob storage with SAS tokens and multipart staging
• SFTP — Remote file storage over SSH with directory-as-bucket layout

Key Features

• Unified Storage API — Same Put, Get, Delete, Head, List, Copy operations across all backends
• Composable Middleware — Direction-aware pipeline with compression (zstd), encryption (AES-256-GCM), deduplication (BLAKE3), virus scanning (ClamAV), and watermarking
• Streaming Engine — Chunked transfers with configurable sizes, backpressure handling, pause/resume, and managed stream pools
• Content-Addressable Storage — Store by content hash with BLAKE3, SHA-256, or XXHash, plus reference counting and garbage collection
• Virtual Filesystem — io/fs.FS-compatible interface for hierarchical access over flat object storage
• Multi-Backend Routing — Glob patterns and custom functions to route objects to the right backend
• Capability Interfaces — Opt-in driver capabilities for multipart, presigned URLs, server-side copy, versioning, and notifications
• Forge Integration — First-class Forge extension with REST API, ORM models, migrations, and ecosystem hooks

Quick Example

// Create and open the driver, then pass it to Trove
drv := localdriver.New()
drv.Open(ctx, "file:///tmp/storage")

t, _ := trove.Open(drv,
    trove.WithDefaultBucket("uploads"),
    trove.WithMiddleware(
        compress.New(),
        encrypt.New(encrypt.WithKeyProvider(kp)),
    ),
)
defer t.Close(ctx)

// Store and retrieve objects
t.Put(ctx, "uploads", "photo.jpg", reader)
obj, _ := t.Get(ctx, "uploads", "photo.jpg")
defer obj.Close()

Next Steps