A series of three hot patches issued by Amazon Web Services (AWS) to address the Log4Shell vulnerability in Apache Log4j at the end of 2021 have turned out to themselves contain serious security issues that leave standalone AWS servers, Kubernetes clusters, Elastic Container Service (ECS) clusters and Fargate vulnerable to attack.
The quartet of vulnerabilities are being tracked as CVE-2021-3100, CVE-2021-3101, CVE-2022-0070 and CVE-2022-0071 and were discovered by researchers at Palo Alto Networks’ Unit 42, which has been working closely with AWS since December to fix them, and is now able to publicly disclose their existence.
“Given the urgency surrounding Log4Shell, it’s possible that this hot patch was deployed at scale, inadvertently putting all sorts of container environments at risk,” said Unit 42 researcher Yuval Avrahai.
“Multi-tenant container environments and clusters running untrusted images are especially at risk. Palo Alto Networks encourages users to upgrade to the fixed hot patch version as soon as possible.”
However, Avrahai added, IT teams who have (for some reason) not yet patched their AWS environments against it should still prioritise the original patches.
“While the presented issues can lead to severe attacks against container environments, Log4Shell has rightfully earned its spot as one of the worst vulnerabilities of all time and is still being actively exploited,” he said.
“We would like to thank AWS for their partnership and coordination in remediating this vulnerability efficiently. As Log4Shell exploitation peaked, AWS’s hot patch helped the community stop countless attacks. With these vulnerabilities fixed, it is now possible to use the hot patch to address Log4Shell while also keeping container environments secure.”
Unit 42 said that after installing the patch service to a server or cluster, every container in that environment was able to exploit it to take over the underlying host. For example, if installed to a Kubernetes cluster, every container in that cluster would have been able to escape until the original patch was rolled back or upgraded. Unprivileged processes could also have exploited the patch to escalate privileges and obtain root code execution.
It added that container escape was possible regardless of whether or not the user is running any Java applications, or whether their underlying host is running the hardened AWS Linux distribution for containers, Bottlerocket. Containers running with user namespaces or as a non-root user are also affected.
Unit 42 and AWS discovered that the issue arose because the hot patches were continuously searching for Java processes and patching them against Log4Shell on the fly, with any process running a binary named “java” considered a candidate, whether inside or outside a container.
Within containers, the hot patches invoked the container’s “java” binary twice, once to retrieve it and then again to inject the hot patch. But they did so without properly containerising the binaries. This meant new container processes could then run without the limitations that would normally apply to them, so a malicious container could include a malicious “java” binary to trick the hot patch into invoking it with elevated privileges. Outside of containers, the service patched host processes similarly, with the same basic result.