Database Failure (Apr 2012)

Early in the morning on 2012-04-10, the machine hosting ECE's PostgreSQL and MySQL databases suffered a hardware failure. As a result, many of our services became unavailable.

We are working to replicate the databases to a spare machine. No data loss is expected, but the failover procedure will take some time.

The failover process for the PostgreSQL server was completed at 08:45 PDT. The services that depend on PostgreSQL (namely Request Tracker, GADS, Meeting Room Booking System, and some MediaWiki-based websites) should be fully operational now.

The failover process for MySQL is still in progress. The websites that rely on MySQL (the ECE department's website in particular) remain down.