We had a busy week: Shellshock and XSA-108

Published on October 1, 2014 by Frederik Denkens

One of the things customers expect from us, is that we manage the security risks of their cloud-environments. To show our job is never finished, last week we got treated to 2 big security threats in the cloud. This article gives some detail on the events and how we handled them.

Shellshock

Last week Wednesday 24/9 started with a series of Bash securityissues better known as ‘Shellshock’. Some call it even worse than the Heartbleed SSL security issue from some months ago. It is effectively a major security risk for many web-applications, potentially leading to data-theft, compromised systems, etc.

Thanks to our excellent Ops team and their automated configuration management, we had all of our customers’ environments patched within 20 hours. Just as we thought our job was finished, a second and even third patch was released shortly after. We managed to patch those even faster.

It might be interesting to note that Bash is widely used. Thus the impact of the security problem is quite significant. Various control systems, core internet infrastructure, devices, etc use Bash somewhere.  Even your own Linux or Mac system could be vulnerable. You can test this with this simple test:

env x='() { :;}; echo vulnerable’ bash -c “echo this is a test”

If you get the result:

vulnerable this is a test

Bad news, your version of Bash can be hacked. You can find more tests and info here.

All hosting/cloud/IaaS providers could be potentially affected. Most issued a statement on how the issue impacted them. The main provider we work with, Amazon Web Services, posted that they were not affected. Are your provider and instances safe?

AWS and XSA-108

It wasn’t over. Thursday 25/9 we got a notification from Amazon Web Services letting us know an ‘urgent’ scheduled maintenance of many of our customers’ instances was necessary. It was due to a critical upgrade of the underlying virtualization layer XEN. Most reboots were scheduled mostly in off-hours periods and the past weekend.

The same day, we communicated this planned downtime to affected customers. As part of our managed services,  we followed up on the scheduled reboots to make sure the environments came back up normally.

Just today the public was made aware of what the underlying problem was. XSE-108 explains that a buggy or malicious virtual server (maybe your neighbour on the same server) could ‘break-out’ and read data from other virtual servers.

Not only major providers like RackSpace and SoftLayer were affected. Basically any cloud/IaaS/hosting provider using XEN better make sure they have their act together.

Conclusion

We had a busy week. It is rather exceptional that 2 security events with such impact occur in such a short time-frame. But thanks to our great team providing the managed services and the world-class providers we use to build hosting solutions, our customers slept as peacefully as ever 🙂

Want to sleep peacefully too? Get in touch and let us build and manage your cloud platform.

Comments on this article

Top of page

Close

Cloud services & management for: