Doh! Forgot earlier in the night, so here you are… technically Saturday.

CPU:


The lemmy devs have made some major strides in improving performance recently, as you can see by the overall reduced CPU load.

Memory:

I need to figure out why swap is continuing to be used, when there is cache/buffer available to be used. But as you can see, the upgrade to 8GB of RAM is being put to good use.

Network:

The two large spikes here are from some backups being uploaded to object storage. Apart from that, traffic levels are fine.

Storage:


A HUGE win here this week, turns out a huge portion of the database is data we don’t need, and can be safely deleted pretty much any time. The large drop in storage on the 9th was from me manually deleting all but the most recent ~100k rows in the guilty table. Devs are aware of this issue, and are actively working on making DB storage more efficient. While a better fix is being worked on, I have a cronjob running every hour to delete all but the most recent 200k rows.

Cloudflare caching:

Cloudflare still saving us substantial egress traffic from the VPS, though no 14MB “icons” being grabbed thousands of times this week 😀

Summary:

All things considered, we’re in a much better place today than a week ago. Storage is much less of a concern, and all other server resources are doing well… though I need to investigate swap usage.

Longer term it still looks as though storage will become the trigger for further upgrades. However storage growth will be much more slow and under our control. The recent upward trend is predominantly from locally cached images from object storage, which can be deleted at any time as required.

As usual, feel free to ask questions.

  • Lodion 🇦🇺@aussie.zoneOPM
    link
    fedilink
    arrow-up
    1
    ·
    1 year ago

    S3fs, but it’s only images as required, not large ongoing transfers. Have uploaded multi GB backups with s3fs and monitoring showed gigabit speeds.

    Latency is a major factor. The server is sub 1ms to the Wasabi endpoint. Higher latency will negatively impact transfers that involves lots of files. Small numbers of large files should be fine.