Introduction

When you expose your site to the world via a kubernetes ingress-controller (like nginx-ingress or traefik) that is then fronted by a CDN or reverse proxy like Cloudfront it can be easy to forget that it's still possible to access your site directly without going via the CDN unless additional controls are added. This can be especially problematic if you use your CDN as a central point to control your edge security such as using a Web Application Firewall (WAF). Let's explore some of the different methods to enhance the security of a Kubernetes Ingress when it is publicly exposed. While I've focused on the nginx-ingress/Cloudfront combination, the same principles apply to other ingress controllers and CDNs.

All the following approaches work by configuring your Cloudfront distribution to pass additional headers to the ingress controller. The ingress controller then uses these headers to determine whether to allow or deny access to the resource.

Per-ingress using location-snippets

If the nginx-ingress controller has the allow-snippet-annotations1 option set to true (the default behavior), we can leverage snippet annotations to secure the ingress resource.

It's important to keep in mind that allowing Ingress resources to customise their nginx behaviour with snippets comes with security and operational overhead. Misconfigured snippets, in addition to leading to unexpected behaviour, can block the ability of other Ingress resources from updating the controller configuration. Therefore, it's recommended to use this approach only if you can trust your teams to configure the snippets securely and correctly.

On the Ingress resource to be secured, set the following annotation (where random_value is a long random string):

nginx.ingress.kubernetes.io/configuration-snippet: |
  if ($http_cf_token != 'random_value') {
    return 403;
  }

Configure the custom origin in the CloudFront distribution, to include a custom header called cf-token set to the value of random_value.

Per-ingress with HTTP Basic Authentication

If allow-snippet-annotations is set to false, an alternative method to secure the ingress resource is by employing HTTP Basic authentication annotations. This configuration is incompatible with an application that uses HTTP Basic authentication for its own purposes, but is otherwise a good option.

On the Ingress resource to be secured, set the following annotations2:

nginx.ingress.kubernetes.io/auth-type: basic
nginx.ingress.kubernetes.io/auth-secret: basic-auth
nginx.ingress.kubernetes.io/auth-realm: "Authentication Required"

You'll need to create a kubernetes Secret named basic-auth containing the username and password to be used for authentication. The secret should be created in the same namespace as the ingress resource.

$ echo -n 'secured_ingress:' >> auth
$ openssl <secure_password> -apr1 >> auth
$ kubectl create secret generic basic-auth --from-file=auth

For the Cloudfront side you will need the base64 encoded username:password pair. You can generate this using the following command:

$ echo "secure_ingress:<secure_password>" | base64

This value can then be used to set the authorization header on your Cloudfront origin. Note that the value must be prefixed with Basic and a space.

Securing everything behind the ingress controller

If you want to protect all resources exposed by a specific ingress controller (rather than a specific Ingress resource) you can use the first method but configure the ingress-controller to add the snippet to every location block it manages. 3

The ingress-controller can be configured by directly editing the ConfigMap, or it can be set as a Helm chart value:

ConfigMap:

location-snippet: |
  if ($http_cf_token != 'random_value') {
    return 403;
  }

Helm chart:

controller:
  config:
   location-snippet: |
     if ($http_cf_token != ‘random_value’) {
       return 403;
     }

Configure the CloudFront distribution in the same way as the other methods setting the header cf-token to our secure random string.

By implementing this approach, all resources behind the ingress controller will require the correct cf_token value to be accessed.

Conclusion

Securing a Kubernetes Ingress when exposed behind a CloudFront distribution is crucial to protect your applications and data, and to provide a guaranteed single path of entry. Depending on the configuration of the nginx-ingress controller, you can use snippet-annotations or basic authentication annotations to enforce security measures. When using CloudFront, custom origin headers allow you to pass necessary headers, such as tokens or authentication information, further enhancing the security of your ingress controller.