Realistic Development with Ruby on Rails ActiveStorage and Minio (S3)

ActiveStorage is an integral part of Ruby on Rails applications. It offers an easy way to store and retrieve objects across a variety of backends with a straightforward and consistent interface. In this post we're going to explore how we can supercharge ActiveStorage in development with a realistic object storage service.

Newly generated Ruby on Rails (yes, we continue to love Ruby on Rails in 2024!) applications come configured with a development environment that uses a Disk storage service. Basically a directory within the project that manages files (uploads / downloads) during development and test. While this is convenient, it usually strays quite far from the application's production configuration. In production, services like AWS S3, Digital Ocean Spaces or Azure Blob storage are more common as backing object stores.

Thus, development organizations are left with the perpetual evaluation of cost vs benefit in the context of dev/prod parity. However, thanks to the open source project MinIO a local S3-like object store can be run right alongside the rest of the application in a standard development environment.

Read on to learn how easy it is to bring a representative object storage interface to ActiveRecord and Ruby on Rails development (and test!) environment.

Why MinIO?

MinIO offers a S3-compatible interface, which is embraced by a number of major industry players. That means, the gem you'll be using to interface with the storage and the code paths executed will closely resemble that of a production environment. This comfort is even more apparent when running with more complex ActiveStorage configurations, like proxies and authentication. These are historically more difficult (or impossible) to test with the local Disk driver alone.

Having the option to run a real object store is also just fun! It creates an approachable way for developers to understand what the platform engineers and site reliability engineers are working with on a daily basis. And although the goal is not to have developers become experts in object storage, venturing a little into the cloud runtime land isn't a bad thing.

MinIO is a well supported project with checkboxes for all platforms. This makes it a great choice for big or small development teams with low to high complexity development environments. And best of all, running it as part of development closes one more pesky dev/prod parity gap that can avoid production bugs or incidents.

The Basic Idea

The default assumption is that all development is being done within devcontainers with the ability to leverage docker compose. A Ruby on Rails development environment is likely already set up this way thanks to needing multiple services like a database, key/value store and maybe an E2E testing service. To make use of MinIO, we'll simply add an additional service to our compose file, configure a default user/password (which seeds the access key / secret key) and then configure Ruby on Rails to use it as our ActiveStorage backend.

Towards the bottom of this post there is an optional step to include seed data from an existing environment.

Two Methods

This post will cover two methods to integrate MinIO into a docker compose configuration. The main problem comes down to dealing with the networking and host name resolution when hitting MinIO from within a container vs the host browser. I prefer neither of them as they both require doing unnatural things that also lead to snags down the road. So, pick your poison!

Both methods require updating the config/environments/development.rb file to replace the default :local service with the new :minio service for config.active_storage.service.

If the project isn't already using a S3 compatible service in production, then the aws-sdk-s3 gem must be added to the Gemfile. Full details are available via the Rails Guides: Active Storage Overview.

Finally, both configurations make use of the bitnami/minio image vs the minio/minio image as it offers initial configurations for access credentials and a default bucket.

Method 1

This method makes a change in the docker compose configuration and only requires a couple of new lines in the docker compose file. The trick is to change the application's default network to use bridged networking. At the end of the day the MinIO service will be available from http://localhost:9000 from the containers as well as the host.

Here is a code sample of the docker-compose.yml file with the bridged networking:

# method 1 - bridged networking
volumes:
  postgres-data:
  redis-data:
  minio-data:

networks:
  default:
    driver: bridge

services:
  app:
    container_name: harled_app
    build:
      context: .
      dockerfile: docker/app/Dockerfile.dev
    image: harled:dev
    environment:
      RAILS_ENV: development
      REDIS_URL: 'redis://redis:6379/1'
    working_dir: /app
    depends_on:
      - db
      - minio
    links:
      - db
      - minio
    stdin_open: true
    tty: true
    volumes:
      - .:/app:cached
    tmpfs:
      - /tmp
  db:
    image: postgres:17-alpine
    environment:
      POSTGRES_HOST_AUTH_METHOD: trust
    container_name: harled_db
    volumes:
      - postgres-data:/var/lib/postgresql/data
  redis:
    image: redis:latest
    container_name: harled_redis
    volumes:
      - redis-data:/var/lib/redis/data
  minio:
    image: bitnami/minio:latest
    container_name: harled_minio
    ports:
      - '9000:9000'
      - '9001:9001'
    volumes:
      - 'minio-data:/data'
    environment:
      - MINIO_ROOT_USER=user
      - MINIO_ROOT_PASSWORD=password
      - MINIO_DEFAULT_BUCKETS=bucket

And here is the corresponding storage.yml file:

# storage.yml
test:
  service: Disk
  root: /rails/tmp/storage

local:
  service: Disk
  root: /rails/storage

minio:
  service: S3
  access_key_id: user
  secret_access_key: password
  region: nyc3
  bucket: bucket
  endpoint: http://localhost:9000
  force_path_style: true

Method 2

The second method uses the default docker compose networking and relies on tweaks to the /etc/hosts file. With this method MinIO is always accessed via http://minio:9000 from within the containers as well as the host browser.

Here is an example of a working /etc/hosts file when running on a Mac:

# /etc/hosts
##
# Host Database
#
# localhost is used to configure the loopback interface
# when the system is booting.  Do not change this entry.
##
127.0.0.1	localhost minio
255.255.255.255	broadcasthost
::1             localhost minio

# Added by Docker Desktop
# To allow the same kube context to work on the host and the container:
127.0.0.1 kubernetes.docker.internal
# End of section

Here is the corresponding docker-compose.yml file:

# method 2 - /etc/hosts resolution
volumes:
  postgres-data:
  redis-data:
  minio-data:

services:
  app:
    container_name: harled_app
    build:
      context: .
      dockerfile: docker/app/Dockerfile.dev
    image: harled:dev
    environment:
      RAILS_ENV: development
      REDIS_URL: 'redis://redis:6379/1'
    working_dir: /app
    depends_on:
      - db
      - minio
    links:
      - db
      - minio
    stdin_open: true
    tty: true
    volumes:
      - .:/app:cached
    tmpfs:
      - /tmp
  db:
    image: postgres:17-alpine
    environment:
      POSTGRES_HOST_AUTH_METHOD: trust
    container_name: harled_db
    volumes:
      - postgres-data:/var/lib/postgresql/data
  redis:
    image: redis:latest
    container_name: harled_redis
    volumes:
      - redis-data:/var/lib/redis/data
  minio:
    image: bitnami/minio:latest
    container_name: harled_minio
    ports:
      - '9000:9000'
      - '9001:9001'
    volumes:
      - 'minio-data:/data'
    environment:
      - MINIO_ROOT_USER=user
      - MINIO_ROOT_PASSWORD=password
      - MINIO_DEFAULT_BUCKETS=bucket
      - MINIO_SERVER_URL=http://0.0.0.0:9000

And finally the storage.yml file:

# storage.yml
test:
  service: Disk
  root: /rails/tmp/storage

local:
  service: Disk
  root: /rails/storage

minio:
  service: S3
  access_key_id: user
  secret_access_key: password
  region: nyc3
  bucket: bucket
  endpoint: http://minio:9000
  force_path_style: true

Object Storage Replication

In the vein of dev/prod parity, there is nothing better than developing on data that actually looks like production data. In problem determination scenarios it is even more helpful to have access to production-like or production data to solve a complex deployment / development issue. This section will provide a brief example of one method to bring existing object storage into the local MinIO and then modify the database so that the files are served correctly. This procedure works equally with Method 1 and Method 2 above if running within a container.

This data population strategy assumes that the source database has already been restored locally, that is, there are local database records of all of the remote objects. This could be something as simple as a pg_dump and pg_restore.

This process leverages rlcone, however there are a number of other options that should work equally as well.

Step 1: Configuration Files

The first step is to create a rclone configuration file that includes a target for the source object storage service as well as the local MinIO service. Here is an example based on a source in DigitalOcean Spaces.

 # /app/rclone.conf
[spaces]
type = s3
provider = DigitalOcean
env_auth = false
access_key_id = access_key_id
secret_access_key = secret_access_key+7qHqmmgX89p4FXn6WQ
region =
endpoint = xxx.digitaloceanspaces.com
location_constraint =
acl =
server_side_encryption =
storage_class =

[minio]
type = s3
provider = Minio
env_auth = false
access_key_id = user
secret_access_key = password
region = us-east-1
endpoint = http://minio:9000
location_constraint =
server_side_encryption =

Step 2: Copy Objects

The second step is to sync the bucket(s). Depending on the size of data, this process can be limited or scoped based on unique needs. In this example the entire bucket will be copied over:

/app # rclone --config rclone.conf sync spaces:bucket minio:bucket

Step 3: Update Database

The third and final step requires modifying the database so that the existing ActiveStorage::Blob records reference the local minio service. The objects won't load if this step is skipped as Rails will be looking in the wrong place for the objects. Here is an example of the code that can be run in a rails console:

ActiveStorage::Blob.where(service_name: "spaces").update_all(service_name: "minio")

Is Minio Right for your Ruby on Rails Application?

Is minio a more realistic object storage backend for ActiveRecord right for your application? As always, it depends. It depends on your goals, on the complexity of your ActiveRecord usage and configuration and your tolerance for surprises in production. Although it is fairly straightforward, it is yet another moving part in the development environment, with configurations to debug, images to update and just a little slower performance which all eat into developer experience. At Harled, we have projects that use it all the time, that have it as an optional configuration based on the work being done and some that don't use it at all.

At any rate, integrating minio into your development environment is a great exercise and fairly low cost as an experiment. Like most things in the modern age, if it doesn't suit the project's needs then throw it out!