Today, at 2:23pm CEST, a deployment went wrong, resulting in a brief downtime for around half of the requests. For our zero-downtime deployments to work, we automatically spin up new virtual servers before taking the old ones out. In this particular case, however, the old code was incompatible with a database change introduced by the new version. This resulted in a number of failed requests for a period of 2 minutes, namely the ones that were still being served by the old servers.
At 2:25pm CEST, the new virtual servers took over and the situation returned back to normal.
We’re sorry for the inconvenience and are already working out a strategy to prevent this from happening again.
All systems are go.