How to export Gluster volume (via Gluster-mount) as an S3 compliant object store

Gluster is a highly scalable, distributed filesystem and well known in the storage world for its good bunch of data management services and stability. Gluster with so much potential eventually can serve as a backend for a distributed object-store. So I was thinking to club the available resources around and build an object storage service on top of Gluster and effectively utilize the power of distributed storage and its services. The idea is to use Gluster for data management services and Minio as an S3 endpoint. In my previous post, I explained how to export Gluster volume via Gluster-block and in this blog, I simplify it a bit and remove the intermediate layer ie Gluster block, and host Minio server directly from Gluster mount.

I have divided this write up into 2 parts

1. Build and mount Gluster volume
2. Launch Gluster volume as object store using Minio server

So let’s start:

Part 1: Build Gluster volume

Bringing up Gluster volume is very simple and can be set up on a single node. (of course, a multinode setup is preferred to get most of Gluster but just for an experiment purpose, I am referring a single node setup. If you are interested in multinode/replicated/features setup, please refer Gluster quick start guide). In the below steps, I have explained how to bring up 4 bricks, simple distributed volume on a single node.

1. # dnf -y install glusterfs-server

2. # systemctl start glusterd.service

3. # systemctl status glusterd.service

4. # gluster volume create testvol 127.0.0.2:/bricks/brick{1..4} force

Note: As we are running Gluster volume on a single node setup, I am referring loopback IP address and testvol is the volume name.

5. # gluster volume start testvol

6. # gluster vol status

7. # gluster vol info

8. # mount -t glusterfs 127.0.0.1:testvol /mnt/gluster-mnt/

9. # df -Th

Part 2: Launch Minio server using Gluster block storage

To start with, I just love Minio for its simplicity. It opened up a new perception of storage. Minio server binary just can take a “directory/mount” point as an argument and export the same as an object-store. How cool is that !! Here are the steps to deal with Minio.

1. # cd .. && mkdir minio

2. # wget https://dl.min.io/server/minio/release/linux-amd64/minio

3. # chmod +x minio && cp minio /usr/local/bin/.

4. # minio server /mnt/gluster-mnt/

Note: Keep the Minio server process running and close when you want to close the object server.

5. Open “http://192.168.56.101:9000" in your browser

6. Login with the default credentials shown after running the Minio server, create a bucket, and upload a file

So if you want to close the object server, just go to the terminal and do ctrl + c.

We can enable different Gluster data management services like replication (provides highly available data), sharding (enables us to store very big files > brick size), EC coded volumes, etc. depending on our use case.

Thanks for reading. See you soon