Choose Local S3 Storage Wisely

I’ve spent years architecting cloud infrastructure, and one pattern I see repeatedly is the need for local S3 storage during development. MinIO has been the go-to solution for most teams, but it’s far from the only option—and increasingly, it’s not always the best one.

After evaluating alternatives across multiple projects, I’ve developed strong opinions about when to use what. The right choice depends on your team’s needs, infrastructure constraints, and how closely you need to mirror production behavior.

Evaluate MinIO Alternatives for Local S3

MinIO is powerful, but it comes with baggage. The binary is large (80+ MB), memory consumption can spike to 500MB+ under load, and the API surface area is massive if you only need basic S3 operations. I’ve seen development laptops grind to a halt when running full MinIO alongside other services.

More importantly, many teams don’t need distributed storage, erasure coding, or multi-tenancy in local environments. If you’re just testing uploads, downloads, and presigned URLs, you’re carrying unnecessary complexity.

Deploy Lightweight Local S3 Solutions

Configure LocalStack S3 for Local Development

LocalStack’s S3 implementation strikes the best balance for most of my projects. It’s actively maintained, has excellent AWS service parity, and integrates seamlessly with existing AWS SDKs.

# docker-compose.yml
version: '3.8'
services:
  localstack:
    image: localstack/localstack:latest
    ports:
      - "4566:4566"
    environment:
      - SERVICES=s3
      - DEBUG=1
      - DATA_DIR=/tmp/localstack/data
    volumes:
      - "./localstack-data:/tmp/localstack"
      - "/var/run/docker.sock:/var/run/docker.sock"

The killer feature? LocalStack persists data to disk by default, so restarting containers doesn’t lose your test fixtures. I configure it once and forget about it.

# Python SDK configuration
import boto3

s3_client = boto3.client(
    's3',
    endpoint_url='http://localhost:4566',
    aws_access_key_id='test',
    aws_secret_access_key='test',
    region_name='us-east-1'
)

# Create bucket and upload
s3_client.create_bucket(Bucket='dev-bucket')
s3_client.put_object(
    Bucket='dev-bucket',
    Key='config/app.json',
    Body='{"feature": "enabled"}'
)

LocalStack’s free tier supports S3, Lambda, DynamoDB, and SQS—more than enough for integration testing. The Pro version adds IAM enforcement and CloudFormation, which I’ve found essential for infrastructure-as-code validation.

Optimize CI/CD with s3mock

For CI/CD pipelines where startup time matters, I use Adobe’s s3mock. It’s a Spring Boot application that boots in under 5 seconds and uses minimal memory (150MB typical).

# Minimal s3mock setup
services:
  s3mock:
    image: adobe/s3mock:latest
    ports:
      - "9090:9090"
    environment:
      - initialBuckets=test-bucket,another-bucket
      - debug=true

The initialBuckets parameter is brilliant—buckets exist at startup, eliminating race conditions in parallel test suites. I’ve replaced complex initialization scripts with a single environment variable.

# Using AWS CLI with s3mock
aws configure set aws_access_key_id test
aws configure set aws_secret_access_key test

aws --endpoint-url=http://localhost:9090 s3 ls
aws --endpoint-url=http://localhost:9090 s3 cp ./file.txt s3://test-bucket/

s3mock shines in GitHub Actions and GitLab CI. The fast startup time means integration tests run 30-40% faster compared to MinIO. For teams running hundreds of pipeline jobs daily, that compounds quickly.

Run Cloudflare R2 Locally Without Docker

This is my current favorite for projects already on Cloudflare. R2’s Wrangler CLI has a local development mode that emulates R2 behavior without requiring Docker.

# Install Wrangler
npm install -g wrangler

# Start local R2
wrangler r2 bucket create dev-bucket --local
wrangler dev --local --persist

The --persist flag maintains state across restarts using SQLite. I love this approach—no containers, no resource overhead, just local files. It’s perfect for frontend developers who don’t want to manage Docker.

// Cloudflare Worker with local R2
export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    const bucket = env.MY_BUCKET;
    
    // Upload with automatic MD5
    await bucket.put('data.json', JSON.stringify({ foo: 'bar' }), {
      httpMetadata: { contentType: 'application/json' }
    });
    
    // Retrieve
    const object = await bucket.get('data.json');
    return new Response(await object?.text());
  }
};

The local R2 implementation doesn’t support every edge case (multipart uploads have quirks), but for standard read/write/delete operations, it’s indistinguishable from production.

Match Production S3 Behavior with MinIO

MinIO remains the right choice for specific scenarios:

1. Multi-tenant testing: If your application has complex bucket policies or cross-account access patterns, MinIO’s IAM implementation is more complete.

2. Large file handling: For testing 10GB+ files with multipart uploads, MinIO’s performance characteristics match production S3 more closely.

3. Versioning and lifecycle policies: When you need to validate object versioning or lifecycle rule behavior before deploying.

# Production-like MinIO setup
services:
  minio:
    image: minio/minio:latest
    command: server /data --console-address ":9001"
    ports:
      - "9000:9000"
      - "9001:9001"
    environment:
      MINIO_ROOT_USER: minioadmin
      MINIO_ROOT_PASSWORD: minioadmin
    volumes:
      - minio-data:/data
    healthcheck:
      test: ["CMD", "mc", "ready", "local"]
      interval: 5s
      timeout: 5s
      retries: 5

volumes:
  minio-data:

I run this configuration for contract testing when I need to ensure API compatibility before cutting releases.

Select the Right Local S3 Tool

Here’s the decision tree I use:

Use LocalStack if:

• You need multiple AWS services (S3 + SQS + Lambda)
• Your team already uses AWS SDKs
• You want data persistence across restarts

Use s3mock if:

• CI/CD pipeline speed is critical
• You only need S3 (no other services)
• Memory constraints matter (shared CI runners)

Use Cloudflare R2 local if:

• You’re building on Cloudflare Workers
• You want zero Docker dependencies
• Simple CRUD operations are sufficient

Use MinIO if:

• You need advanced IAM or policy testing
• Multipart upload behavior must match production
• You’re testing distributed storage scenarios

Manage Local S3 Configurations Consistently

I maintain environment-specific configuration using a single pattern across all alternatives:

// Go configuration helper
package storage

import (
    "os"
    "github.com/aws/aws-sdk-go/aws"
    "github.com/aws/aws-sdk-go/aws/credentials"
    "github.com/aws/aws-sdk-go/aws/session"
)

func NewS3Client() (*session.Session, error) {
    endpoint := os.Getenv("S3_ENDPOINT")
    region := os.Getenv("AWS_REGION")
    
    config := &aws.Config{
        Region: aws.String(region),
        Credentials: credentials.NewEnvCredentials(),
    }
    
    // Only set endpoint for non-production
    if endpoint != "" {
        config.Endpoint = aws.String(endpoint)
        config.S3ForcePathStyle = aws.Bool(true)
    }
    
    return session.NewSession(config)
}

Environment files make switching between alternatives trivial:

# .env.localstack
S3_ENDPOINT=http://localhost:4566
AWS_REGION=us-east-1
AWS_ACCESS_KEY_ID=test
AWS_SECRET_ACCESS_KEY=test

# .env.s3mock
S3_ENDPOINT=http://localhost:9090
AWS_REGION=us-east-1

# .env.production (no endpoint)
AWS_REGION=us-west-2

The application code never changes—only the environment configuration.

Benchmark Local S3 Storage Performance

I ran a benchmark suite across 10,000 operations (uploads, downloads, deletes) to compare real-world performance:

Tests run on M1 MacBook Pro, 100KB average file size

MinIO wins on raw throughput, but s3mock’s lower memory footprint makes it ideal for CI environments where you’re running multiple services simultaneously. R2 local’s near-instant startup time is unbeatable for quick feedback loops.

Avoid Common Local S3 Pitfalls

LocalStack quirks: IAM policy evaluation can behave differently than real AWS, especially around conditional operators. Always validate critical security policies in a real AWS account.

s3mock limitations: Presigned URL expiration isn’t enforced—URLs work indefinitely. Don’t rely on expiration logic in tests.

R2 local: Object metadata is simplified. Custom metadata keys work, but some AWS-specific headers are ignored.

MinIO: The management console is resource-intensive. Disable it in CI with --console-address "" to save memory.

Emerging Local S3 Storage Tools

The landscape is evolving. Supabase recently released local S3 emulation in their CLI, and I’m evaluating it for projects using their ecosystem. Early results are promising—native integration with Postgres storage and auth.

Garage, a lightweight distributed object storage system written in Rust, is another project on my radar. It’s designed for self-hosting and uses less memory than MinIO while maintaining S3 compatibility.

Implement Your Local S3 Strategy

For greenfield projects, I start with LocalStack for the flexibility. Once the architecture stabilizes and I understand which AWS services matter, I’ll switch to s3mock if S3 is the only dependency—or stick with LocalStack if multi-service integration is critical.

For Cloudflare-based applications, R2 local is a no-brainer. The development experience is seamless, and the production deployment is a single wrangler deploy.

The important thing isn’t which tool you pick—it’s having a consistent local development experience that mirrors production behavior closely enough to catch bugs before they reach users. Choose based on your constraints, not what’s most popular.

Whatever you choose, make sure your entire team uses the same setup. Inconsistent local environments are a constant source of “works on my machine” bugs, and no amount of tooling can fix that organizational problem.