Paged Again
4 am, the sound I least want to hear, my phone paging me. It was the monitoring software telling me there was a problem with the mail gateway. Straight out of REM sleep so deep I couldn’t see for a little while I move to my computer to check on it.
Apparently the gateway hung and as should happen it was powercycled by the monitoring software when it could not connect nor respond to ping. It came back and responded, but all connections to it just hung indefinitely. It accepted them fine, but they all just hung. Even the remote console was sluggish, making troubleshooting more difficult.
Turned out to be corrupted data that hosed NFS, a manual fsck (the auto fsck on reboot didn’t do the trick apparently) boot back into multiuser and all seems to be well. If you could not connect for a brief period, this was why. Back to sleep.
