[performance] Benchmark for micro-instances

[hatonthecat]

Hi, I have been reading some load-testing on virtual machines work really well with some cloud providers, and heard of an benchmark tool called Vegeta: https://github.com/tsenart/vegeta

It's not for scalability or multi-node development, but rather to benchmark the maximum number of user accounts on an instance. In the use-case I am testing, it is on a 256MB RAM VM with a shared CPU using SQLite. The instance is limiting media file sizes to 20-30KB instead of 10MB. This is to test which combination of character limit + max media file size can maximize the number of users/HTTP connections without running into I/O bottlenecks.

[tsmethurst]

Hiya, I'm not really sure what the issue is here or what you're getting at, since this isn't a bug report or feature request or anything. Were you going to share your findings?

EDIT: Just to clarify, it's cool that you're testing this and I think your findings would be interesting for us :P

[NyaaaWhatsUpDoc]

yes we would be very interested to see the methodology and results if you do fancy sharing them :)

[hatonthecat]

Hiya, I'm not really sure what the issue is here or what you're getting at, since this isn't a bug report or feature request or anything. Were you going to share your findings?

EDIT: Just to clarify, it's cool that you're testing this and I think your findings would be interesting for us :P

Hi,

Yes, the Issue that was opened was a Custom option, as I wanted to select Feature request, but it is more a developer feature than a user feature (Though with admins, it is sort of the same):

I suppose it would be more of a "Developer feature" request, if Github had the option.

Specifically, I read the Vegeta benchmark showed excellent performance for fly.io: https://benhoyt.com/writings/flyio/#load-testing

Originally, I began this idea with Pleroma, as someone was able to run it on 256MB of RAM with a Postgresql: https://sal.dev/fediverse/running-pleroma-on-fly-io/

As I tried to replicate the server setup with my own fly.io account, I noticed that GtS uses just SQLite and under 100MB RAM, and that it might be lighter for storage and throughput (relying on Semaphore or other front-end)
https://space.matthewphillips.info/posts/flyio-gotosocial/

The free fly.io account allows up to three machines (256MB each), and three 1GB volumes. It seems that would be an excellent way to create routine SQLite backups alongside media storage for the instance, without requiring S3 or other outbound backup (but that could be an option too). That said, some cloud servers may charge for intranet traffic, thus the outbound limit may be used. Even if that is the case, 100GB per month seems like it would sufficient to allow a highly active mini-instance with 10s of users as long as the images are ~20KB each.

I plan to test this in both a synthetic and real-world setting, hopefully with enough users to determine the maximum users that could run. :) Ultimately the goal isn't to limit it to 20KB media files, but to determine a replicable estimate of loads at increments of 20KB, 50KB, or 100KB so that the maximum number of accounts can be recommended based on intended server parameters. I am documenting my progress here: https://github.com/hatonthecat/fly.iotests Thanks!