It was hot this weekend – over 35°C (92°F) outside – and the home servers were suffering somewhat.
Western Washington is pretty friendly for servers – it’s usually cool – but we do get occasional spikes.
Normally I’ll add an extra fan to the rack – but this time I was away for the weekend. The firewall/IDS server hard drive collapsed and died. The alert monitors sent a final notification that the internal temperature reached 120°C. The final straw was a power outage for over an hour that triggered the UPS and graceful shutdown. On power restoration everything came back as expected – but the firewall sat with an unbootable file system.
It only took twenty minutes to replace the hardware – and it running happily at 42°C right now.