Synology HyperBackup S3 backups integration

[capusta]

Description

Synology HyperBackup (allegedly) supports S3 backups to "the cloud". It would be very helpful to have Sia S3 integration being handled at the API level. Would it be possible to do ?

Other than tcpdump provided in the screenshot - the synology client implementation remains closed source, unfortunately.

Sia is running in a docker container on the same host.

Version

v1.0.7 beta3

What operating system are you running (e.g. Ubuntu 22.04, macOS, Windows 11)?

Custom (Synology OS => DSM)

Anything else?

• https://www.synology.com/en-us/releaseNote/HyperBackup
• https://community.synology.com/enu/forum/1/post/130151

[ChrisSchinnerl]

It seems like you are missing a bucket and directory? Make sure you create a bucket via the Sia UI first and then try to use that. If the tool requires a directory also make sure to create one or try specifying the root explicitly.

[capusta]

The buckets in Sia have certainly been created. The issue is that Synology S3 service first tries to populate ALL the buckets in the dropdown menu (shown in red, from the screenshot above). And it is having the issue of not finding any. renterd shows no logs of the event. 400 error still persists. using the renterd.yml and disabling authentication via disableAuth: true also does not seem to do anything.

Typically during Amazon S3 configuration the bucket names are populated on the dropdown menu - so the user is allowed to choose one particular bucket.

 # curl -u :$MYPASS localhost:9980/api/bus/buckets                              
[                                                                                
        {                                                                        
                "createdAt": "2024-01-21T17:05:12.033935422Z",                   
                "name": "default",                                               
                "policy": {                                                      
                        "publicReadAccess": false                                
                }                                                                
        },                                                                       
        {                                                                        
                "createdAt": "2024-02-09T01:26:03.285140261Z",                   
                "name": "test",                                                  
                "policy": {                                                      
                        "publicReadAccess": true                                 
                }                                                                
        },                                                                       
        {                                                                        
                "createdAt": "2024-02-09T04:22:37.615842882Z",                   
                "name": "test-two",                                                   
                "policy": {                                                      
                        "publicReadAccess": false                                
                }                                                                
        },                                                    

[ChrisSchinnerl]

You could try to increase the log level to debug to see if renterd starts printing some useful information. Have you been able to access renterd via its S3 API through a different tool like rclone to rule out it's a Synology HyperBackup specific issue?

[jwaspi]

I am having the same issue. My setup is similar except for renterd (v1.0.7 latest) running on separate host. rclone works successfully so I would have to imagine this issue is specific to Synology HyperBackup. I will try and gather additional logs but I at least wanted to send a quick comment to share my findings.

Update: Ran renterd in debug mode and did not find anything relevant to S3.

[jwaspi]

It seems Synology Hyper Backup will only talk to S3 over an encrypted connection. So this is what I did as a workaround.

Synology > Control Panel > Login Portal > Advanced > Reverse Proxy > Made the below entry

A local DNS entry was made for the hostname specified in the reverse proxy and pointed to the Synology's IP address. In my case I did this in OPNsense.

I then created the S3 destination in Hyper Backup, the only way I could get it to work was via Path-Style.

Hope this helps.

[ChrisSchinnerl]

Thanks for doing that research! Interesting that it doesn't like "unsafe" connections but doesn't really tell you.
You might be able to get it to work without path style if you set RENTERD_S3_HOST_BUCKET_ENABLED or the corresponding yaml config option in renterd.

Closing this since it appears to be fixed with additional configuration.

[capusta]

confirmed working ... thanks reverse proxy :)
Enabling renterd_s3_host_bucket also seems to break the implementation ... it does not see any buckets upon the initial connection.