MinIO S3 (and Filestash)

S3 is a storage API first implemented at Amazon AWS, but has since been copied and re-implemented as open source MinIO. MinIO is a service you can install to centrally store files in your network via the S3 API. With a web frontend like Filestash, you can host a file manager webapp for easy file sharing.

Tip

Running MinIO on a single Raspberry Pi is perfect for personal use. However, for heavier production use, you will need to plan to use beefier hardware with redundant storage.

Configure MinIO

Run this on your Raspberry Pi

pi make minio config

(stdout)

MINIO_TRAEFIK_HOST: Enter the minio domain name (eg. s3.example.com)

: s3.pi.example.com

MINIO_CONSOLE_TRAEFIK_HOST: Enter the minio console domain name (eg. console.s3.example.com)

: s3-console.pi.example.com

MINIO_SITE_REGION: Enter the self-described region of the server (eg. default)

: default

MINIO_ROOT_USER: Enter the minio root username (eg. root)

: root

? Do you want to enable sentry authorization in front of this app (effectively making the entire site private)?
> No
  Yes, with Mutual TLS (mTLS)

Enable admin console (optional)

The admin console is optional and it is blocked by default. If you want to access the console, you must unblock it, by providing your client IP address:

Run this on your Raspberry Pi

## Unblock the console for your specific IP address:
pi make minio reconfigure var=CONSOLE_SOURCERANGE=X.X.X.X/32

Tip

To allow any IP address access to the minio console use 0.0.0.0/0:

Run this on your Raspberry Pi

## To unblock the console for any IP address:
pi make minio reconfigure var=CONSOLE_SOURCERANGE=0.0.0.0/0

Once installed, the console will be accessible at https://s3-console.pi.example.com. You will need to enter the username (root) and the password can be retrieved from the .env_{CONTEXT}_{INSTANCE} file:

Run this on your Raspberry Pi

## To retrieve the minio console root password:
pi make minio dotenv_get var=MINIO_ROOT_PASSWORD

(stdout)

XxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXx

Install MinIO

Run this on your Raspberry Pi

pi make minio install wait

Add two new routes on the sentry (droplet)

Create a route for S3 and the console:

Run this on the Raspberry Pi

sentry make traefik config

(stdout)

? Traefik:
> Config
  Install (make install)
  Admin
  Exit (ESC)

? Traefik Configuration:
^ Entrypoints (including dashboard)
  TLS certificates and authorities
  Middleware (including sentry auth)
> Advanced Routing (Layer 7 / Layer 4 / WireGuard)
  Error page template
  Logging level
  Access logs

? Traefik routes
> Configure layer 7 TLS proxy
  Configure layer 4 TCP/UDP proxy
  Configure wireguard VPN

? Layer 7 TLS Proxy:
  List layer 7 ingress routes
> Add new layer 7 ingress route
  Remove layer 7 ingress routes
  Disable layer 7 TLS Proxy

Enter the public domain (SNI) for the route:

: s3.pi.example.com

Enter the destination IP address to forward to:

: 10.13.16.2

Enter the destination TCP port to forward to:

: 443

> Do you want to enable Proxy Protocol for this route? Yes

## Layer 7 TLS Proxy is ENABLED.
## Configured Layer 7 Routes:
Entrypoint         Destination_address  Destination_port  Proxy_protocol
----------         -------------------  ----------------  --------------
s3.pi.example.com  10.13.16.2           443               2

? Layer 7 TLS Proxy:
  List layer 7 ingress routes
> Add new layer 7 ingress route
  Remove layer 7 ingress routes
  Disable layer 7 TLS Proxy

Enter the public domain (SNI) for the route:

: s3-console.pi.example.com

Enter the destination IP address to forward to:

: 10.13.16.2

Enter the destination TCP port to forward to:

: 443

> Do you want to enable Proxy Protocol for this route? Yes

## Layer 7 TLS Proxy is ENABLED.
## Configured Layer 7 Routes:
Entrypoint                 Destination_address  Destination_port  Proxy_protocol
----------                 -------------------  ----------------  --------------
s3.pi.example.com          10.13.16.2           443               2
s3-console.pi.example.com  10.13.16.2           443               2

Press ESC three times to go back to the main menu, and re-install Traefik:

(stdout)

? Traefik:
  Config
> Install (make install)
  Admin
  Exit (ESC)

After installation, press ESC to quit the config tool.

Create an S3 bucket and credentials

Run this on your Raspberry Pi

pi make minio bucket

Create the bucket name: demo and then leave the policy, group, and username blank to use the same value:

(stdout)

Enter a new bucket name (test): demo
Enter a new policy name (demo):
Enter a new group name (demo):
Enter a new user name (demo):

This will create the bucket and the output the endpoint and access credentials, which is all of the information necessary to provide access:

(stdout)

Bucket: demo
Endpoint: s3.pi.example.com
Access Key: demo
Secret Key: XxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXx

Configure Filestash instance

It is likely you will want several instances of filestash for different buckets, so you you should configure them separately as named instances:

Run this on your Raspberry Pi

## Create an instance named test:

pi make filestash config instance=test

(stdout)

FILESTASH_TRAEFIK_HOST: Enter the filestash domain name (eg. filestash.example.com)

: filestash-test.pi.example.com

? Do you want to enable sentry authorization in front of this app (effectively making the entire site private)?
  No
  Yes, with HTTP Basic Authentication
> Yes, with Oauth2
  Yes, with Mutual TLS (mTLS)

Warning

sentry authorization is a requirement as the S3 credentials are stored in the filestash client (i.e., web browser), therefore you should not allow anyone you do not trust with this information to access the page.

(stdout)

? Which authorization group do you want to permit access to this app?
> admin

Tip

This will allow only the admin authorization group to access this instance.

Remember, you can create extra authorization groups in the Traefik config, that way you can have separate user access per instance.

Install the Filestash instance

Run this on your Raspberry Pi

## Install the instance named test:

pi make filestash install instance=test

Info

The build process for filestash may take awhile, as it is built from source code. This is because the upstream images were out of date for ARM64.

Add a new route on the sentry (droplet)

Run this on the Raspberry Pi

sentry make traefik config

(stdout)

? Traefik:
> Config
  Install (make install)
  Admin
  Exit (ESC)

? Traefik Configuration:
^ Entrypoints (including dashboard)
  TLS certificates and authorities
  Middleware (including sentry auth)
> Advanced Routing (Layer 7 / Layer 4 / WireGuard)
  Error page template
  Logging level
  Access logs

? Traefik routes
> Configure layer 7 TLS proxy
  Configure layer 4 TCP/UDP proxy
  Configure wireguard VPN

? Layer 7 TLS Proxy:
  List layer 7 ingress routes
> Add new layer 7 ingress route
  Remove layer 7 ingress routes
  Disable layer 7 TLS Proxy

Enter the public domain (SNI) for the route:

: filestash-test.pi.example.com

Enter the destination IP address to forward to:

: 10.13.16.2

Enter the destination TCP port to forward to:

: 443

> Do you want to enable Proxy Protocol for this route? Yes

## Layer 7 TLS Proxy is ENABLED.
## Configured Layer 7 Routes:
Entrypoint                     Destination_address  Destination_port  Proxy_protocol
----------                     -------------------  ----------------  --------------
filestash-test.pi.example.com  10.13.16.2           443               2

Press ESC three times to go back to the main menu, and re-install Traefik:

(stdout)

? Traefik:
  Config
> Install (make install)
  Admin
  Exit (ESC)

After installation, press ESC to quit the config tool.

Open Filestash

Open the URL in your browser: https://filestash-test.pi.example.com.

The initial page will ask you to create an admin password.

Administer Filestash

The admin page is only accessible from https://filestash-test.pi.example.com/admin/, and it is here you must configure the storage backend.

Click Backend in the menu.
Remove every backend except for S3.
Do not select any authentication middleware.
Click the icon in the upper left to go to main page, or simply go to https://filestash-test.pi.example.com/.

On the main page, enter the credentials for the S3 connection:

Click Advanced
Enter the Access key ID
Enter the Secret Access Key
Enter the Endpoint
Leave the other options blank.

Click Connect.

Once logged in you should see a single folder with the same name as the bucket (test):

Click the Share icon in the top right of the folder container.

Choose the appropriate permission for the person you are sharing with:
- Editor has full access, download, upload, view, and delete.
- Viewer is read only.
- Uploader is write only (they won’t be able to see any files, not even the ones they upload.)
Under Advanced settings, you may optionally set an expiration for the link, or customize the link URL.
Do not set a password, as you can rely upon the sentry authorization instead.
Copy the link, and share this link with your friends!
Try the link yourself in an Incognito browser window so that you test it with no existing cookies. It should force you to login through Forgejo first, and then once on the filestash page, test the permissions you set are working.

Tip

Because you enabled sentry authorization for this route, your friends will also need to create Forgejo accounts, and you will need to add them to the sentry authorization group in the Traefik config.

Tip

Create an admin link for yourself, so that you don’t need to enter the S3 credentials again, and you can create new share links too:

Create a new share link like before.
Choose Editor privilege.
Click Advanced.
Click Can Reshare.

Using filestash

You can upload multiple files at a time by draging and dropping them directly from your computer’s file manager into the browser window (FYI the dropzone is only near the top of the page).
Many media types are supported with rich viewers, including images, audio, and video.