Its Winter here in the Atlanta area, and the temperatures have been dropping down close to freezing. Likewise, the temperatures in my basement Homelab have been dipping below 66 degrees Fahrenheit, and apparently this makes my Supermicro Servers a bit unhappy.
A bit of background. When I set out to build my lab I decided to transfer my Supermicro X8DTI boards out of their stock rack mount enclosures (like what you see above) and into Standard EATX Towers. Running my boards in these towers allowed me to use large 120mm and 140mm fans for cooling. Sound wise this is a huge improvement over the stock 80mm fans used in the default enclosure as these larger fans can spin much slower than stock and still keep the system cool.
On several occasions I have been working in my office and have heard the fans in my lab servers revving up and back down again. At first I thought that maybe a fan was failing, and that one of my systems was overheating. Or that one of my systems had sucked in a bit too much basement dirt and dust. However neither were the case.
Specifically what was happening was this… The systems were running cool, so the fans would spin down to a low rpm and the system would then throw a low rpm threshold alert and spin the fan back up.When this occurs the system switches into some sort of “Critical Cooling Mode” and spins all the fans up to 100% for a few seconds. Rinse and repeat a few dozen times and you hear what almost sounds like an intoxicated neighbor playing with his new weedeater.
Using the IPMIitool command from my Linux desktop first logged into IPMI controller on my systems and checked to make sure that the fans were actually working properly. SDR is short for Sensor Data Repository
# ipmitool -H 10.1.0.104 -U admin -P <password> sdr list
Fan2 | 2176 RPM | ok
Fan3 | 340 RPM | ok
Fan4 | no reading | ns
Fan5 | 544 RPM | ok
Fan6 | 340 RPM | ok
…truncated…
Fan6 above is spinning mighty slow. Slow enough to drop below the Lower Non-Critical Value threshold. Rather than increase the speed of the fan as I have seen others do, I decided to lower the thresholds for this fan with the command below. Since Fan5 is also a bigun’ I decided to preemptively adjust its thresholds as well.
ipmitool -H 10.1.0.104 -U admin -P <password> sensor thres Fan4 lower 100 200 300
ipmitool -H 10.1.0.104 -U admin -P <password> sensor thres Fan5 lower 100 200 300