How to lose data: 1. A problem process eats disk space 2. Your email alert threshold is at 10% free 3. Your paging (wake me up) threshold is at 5% free 4. The ext4 reserved blocks are the default 5%. Woke up to FS at 5.01% free, a pile of <10% alert emails, and lost data.
-
What do you use for paging? Not sure I want more than alertmanager emails, but I'm curious what's being used nowadays.
I use Pushover, which is nice and simple, one time app purchase fee, integrated with Alertmanager. Recommended for personal use.
