Lightning strikes four times and Google loses cloud data

They say lightning never strikes twice, but they're wrong as Google reports that it's lost some data after the power grid serving one of its European data centers suffered four consecutive lightning strikes last Thursday.

Google Compute Engine (GCE) disks in the europe-west1-b zone data center, located in Belgium, suffered I/O errors in the strikes and 0.000001 percent of disks suffered permanent data loss. GCE allows customers to run virtual machines and store data in the cloud.

It's not known what data was lost or which customers have been affected, but writing on its Cloud Platform status page yesterday Google points out that, "The issue only affected Standard Persistent Disks that existed when the incident began at 09:19 PDT. There was no effect on Standard Persistent Disks created after 09:19. SSD Persistent Disks, disk snapshots, and Local SSDs were not affected by the incident". We understand that customers have been able to successfully recover from Persistent Disk Snapshots and from their own backups.

Google has apologized to affected customers and says that it's working to improve the reliability of its GCE storage layer. However it does say that, "GCE instances and Persistent Disks within a zone exist in a single Google datacenter and are therefore unavoidably vulnerable to datacenter-scale disasters. Customers who need maximum availability should be prepared to switch their operations to another GCE zone. For maximum durability we recommend GCE snapshots and Google Cloud Storage as resilient, geographically replicated repositories for your data".

Fans of Douglas Adams' Hitchhiker's Guide series will know that 'Belgium' is the most unspeakably rude word in the galaxy, customers of GCE now understand why.

Photo Credit: Igor Zh /Shutterstock