Yesterday CircleCI had probably the worst outage I've ever seen from them (I still thoroughly recommend them, at the very least you can answer the question "Do we have to do anything to fix this?" with "No, someone else's problem").

From the looks of things I think this happened:

1. CloudFlare noticed a DNS issue at Dyn DNS. (CloudFlare post)
2. Dyn DNS had a mistake up updating critical records. (Dyn DNS post)
3. It looks like this caused a failure in GitHub, in particular delivering web hooks.
4. This in turn backed up event delivery, when it was fixed it landed a torrent of build requests onto CircleCI simultaneously.
5. CircleCI promptly had a meltdown, recovered and limped on. (CircleCI status post)
6. This also seems to have triggered a DB failure, which took a day to fix. (CircleCI status post)

Note that this is all conjecture on my part :)

I'm not sure what the moral of the story here is, with all these services you're trading off functionality, development time and reliability. Sometimes that trade off can bite you when you inadvertently become reliant on the reliability of a random DNS provider :)

Update 2015-07-15

CircleCI have posted a post mortem.