Tackling the recent Kong ingress controller security incident with ARMO’s behavioral CADR
Imagine this situation: you recently updated one of your infrastructure software components. A few weeks...
Mar 25, 2025
On March 24, 2025, Wiz Research disclosed a series of critical vulnerabilities in Ingress NGINX Controller for Kubernetes, collectively dubbed #IngressNightmare:
These unauthenticated Remote Code Execution (RCE) vulnerabilities have been assigned a CVSS base score of 9.8. According to Wiz Research, exploitation allows attackers to gain unauthorized access to all secrets across all namespaces in affected Kubernetes clusters, potentially leading to complete cluster takeover.
Wiz’s analysis indicates approximately 43% of cloud environments are vulnerable, with over 6,500 clusters (including Fortune 500 companies) publicly exposing vulnerable Kubernetes ingress controllers’ admission controllers to the internet. Our security team has conducted an independent assessment of these vulnerabilities, and we believe this represents a significant risk to organizations using Kubernetes.
Ingress NGINX Controller is one of the most popular ingress controllers for Kubernetes, with over 18,000 stars on GitHub. As a core Kubernetes project, it’s explicitly highlighted in the Kubernetes documentation as an example Ingress controller that fulfills the prerequisite for using Ingress in Kubernetes.
Its primary function is to accept incoming traffic and route it to the relevant Kubernetes Services, which then forward the traffic to the appropriate Pods based on defined rules. Specifically, Ingress NGINX Controller is built on top of the popular NGINX reverse proxy.
According to Wiz’s research, over 41% of internet-facing clusters run Ingress-NGINX, making this vulnerability particularly concerning due to its widespread deployment.
At the heart of these vulnerabilities is the admission controller component of Ingress NGINX. This controller validates incoming ingress objects before they are deployed. By default, these admission controllers are accessible over the network without authentication, making them highly appealing attack vectors.
When the Ingress-NGINX admission controller processes an incoming ingress object, it:
The vulnerabilities exist in this validation process.
The most severe vulnerability in the chain exists in how the admission controller validates NGINX configurations. When processing an incoming ingress object, the controller constructs a temporary NGINX configuration and then runs the NGINX binary with the -t flag to validate it.
// testTemplate checks if the NGINX configuration inside the byte array is valid // running the command "nginx -t" using a temporal file. func (n *NGINXController) testTemplate(cfg []byte) error { ... tmpfile, err := os.CreateTemp(filepath.Join(os.TempDir(), "nginx"), tempNginxPattern) ... err = os.WriteFile(tmpfile.Name(), cfg, file.ReadWriteByUser) ... out, err := n.command.Test(tmpfile.Name()) func (nc NginxCommand) Test(cfg string) ([]byte, error) { //nolint:gosec // Ignore G204 error return exec.Command(nc.Binary, "-c", cfg, "-t").CombinedOutput() }
This validation process was not properly sandboxed. As a result, an attacker could craft a malicious ingress object that, when processed, would generate an NGINX configuration containing code that would be executed during the validation phase.
To address this vulnerability, the Kubernetes team completely removed the NGINX configuration validation step during admission control:
/* Deactivated to mitigate CVE-2025-1974
// TODO: Implement sandboxing so this test can be done safely
err = n.testTemplate(content)
if err != nil {
n.metricCollector.IncCheckErrorCount(ing.ObjectMeta.Namespace, ing.Name)
return err
}
*/
The other three CVEs relate to template injection vulnerabilities in the NGINX configuration template. These vulnerabilities allowed attackers to inject malicious content into the NGINX configuration through various annotations.
The authreq parser is responsible for handling authentication-related annotations. When the temporary configuration is created, $externalAuth.URL
(which corresponds to the URL from the auth-url annotation) is incorporated without proper sanitization:
proxy_http_version 1.1; proxy_set_header Connection ""; set $target {{ changeHostPort $externalAuth.URL $authUpstreamName }}; {{ else }} proxy_http_version {{ $location.Proxy.ProxyHTTPVersion }}; set $target {{ $externalAuth.URL }}; {{ end }}
This lack of proper sanitization allows an attacker to inject arbitrary NGINX configuration directives. The patch added proper quoting:
- set $target {{ $externalAuth.URL }}; + set $target {{ $externalAuth.URL | quote }};
The auth-tls-match-cn annotation requires:
While these requirements seem restrictive, Wiz researchers discovered techniques to bypass both requirements and inject arbitrary NGINX configurations. The fix implemented proper quoting:
- if ( $ssl_client_s_dn !~ {{ $server.CertificateAuth.MatchCN }} ) { + if ( $ssl_client_s_dn !~ {{ $server.CertificateAuth.MatchCN | quote }} ) {
In the mirror annotation parser, the code processes the UID from the ingress object and inserts it into $location.Mirror.Source
in the temporary NGINX configuration. Since this injection is in the UID parameter (which is not a Kubernetes annotation), the input does not get sanitized by the annotations’ regex rules.
The patch added proper quoting:
- mirror {{ $location.Mirror.Source }}; + mirror {{ $location.Mirror.Source | quote }};
Wiz researchers demonstrated how these injection vulnerabilities could be leveraged to achieve Remote Code Execution. The key insight was discovering that the ssl_engine
directive, part of the OpenSSL module, can load shared libraries—an undocumented behavior. Unlike other directives like load_module
, the ssl_engine
directive can be used at any point within the configuration file.
The attack chain Wiz documented works as follows:
This creative attack chain demonstrates the severity of these vulnerabilities when chained together.
These vulnerabilities have a severe impact due to:
The most effective mitigation is to update to patched versions:
Consider these alternative mitigations recommended by Wiz and the Kubernetes security team:
controller.admissionWebhooks.enabled=false
ingress-nginx-admission
and remove the --validating-webhook
argument from the ingress-nginx-controller container’s Deployment or DaemonSetRemember to re-enable the Validating Admission Controller after you upgrade, as it provides important safeguards for your Ingress configurations.
The Wiz Research team followed responsible disclosure practices:
Based on our independent review of the disclosed vulnerabilities, we assess that IngressNightmare represents a critical security risk for organizations using Kubernetes with Ingress NGINX Controller. The attack surface—an unauthenticated admission controller with access to cluster secrets—is particularly concerning.
We recommend organizations take the following steps:
The IngressNightmare vulnerabilities highlight two important lessons for Kubernetes security:
Organizations should review their Kubernetes security posture holistically, particularly focusing on components that process external input or have elevated privileges.
The IngressNightmare vulnerabilities discovered by Wiz Research emphasize the importance of securing every component in complex systems like Kubernetes. Even internal components not directly exposed to the internet can become attack vectors if they process untrusted input.
We commend the Kubernetes security team and the Ingress-NGINX maintainers for their prompt response and comprehensive fixes to these critical vulnerabilities. Their work highlights the strength of the open-source security model and responsible disclosure process.
Organizations using Kubernetes should prioritize patching these vulnerabilities and implementing security best practices to protect their containerized environments.
Imagine this situation: you recently updated one of your infrastructure software components. A few weeks...
In 2024, several significant vulnerabilities were identified within the Kubernetes and broader cloud-native ecosystem. In...
Have you ever heard of CVEs? Maybe not by their acronym, but Common Vulnerabilities and...