Container Images

General Discussion
1

Hi Ryan,

Following from Security Scanners , I have completed an audit across base container images used within the podplane, netsy.dev and nstance.dev projects. Most of these were OK, typically using scratch, gcr.io/distroless/static:nonroot or appropriate images except for a few cases.

Case 1: traefik/bootstrap-tls

One of the podplane helm components bootstrap-tls afaik is used to generate self-signed kubernetes.io/tls secrets for ingress domain at install/upgrade time before cert-manager is ready to issue real ones.

The component is based on dynamic Go source code, managed by Helm and uses the docker.io/library/golang:alpine image. I was thinking this could be made into a gcr.io/distroless/static:nonroot by pre-compiling the code with rules and shipping as a static binary for an image. Currently it uses the go run command whenever there is a change to Helm chart values.

This reduces a few risks:

  • kubectl update/patch on the ConfigMap provides direct code injection in the pod
  • Full Go toolchain gives an attacker more than static binary or image would
  • Fewer Build-time security gates don’t apply to runtime-compiled code
  • Helm release behaviour can drift without a chart version bump

It’d be good to have your input here as there are a few ways to approach this. In my mind, this would be significant enough to refactor to avoid some large vulnerabilities within a strongly held permission workload, that is writing the ConfigMap for tls-certitificate management.

Case 2: platform-certs/secret-sync

Currently, this service serves a very simple purpose which is solely to keep a secrets-store.csi.k8s.io volume mount alive so kubelet keeps syncing secrets into it.

The image is currently docker.io/library/golang:alpine which is 356 MB and is only used to call /bin/sh -c "trap : TERM INT; sleep infinity & wait" which waits for the pod to be killed (somewhat expensively).

There is an image registry.k8s.io/pause:3.10 which does exactly what the intention of the command is, and doesn’t over-expose it’s surface relative to it’s purpose. It’s ideal in the way where it is provided without a shell and ~300KB in size.

Finally

Side note, for gcr.io/distroless/static:nonroot images it’s recommended to state static-debian12: or static-debian13: instead of the static:* explicitly as this can impact applications dependent on the current filesystem structure with future changes across major Debian versions. Not really an issue for now.

For reference: Please add note into README.md about status of "non-debian" versions · Issue #2052 · GoogleContainerTools/distroless · GitHub .

Additionally, I can provide details on my analysis if you like. The most highly permissioned image used was debian:13-slim on vmconfig/test-install which is appropriate. Happy to take on the fixes for these cases too.

Security Scanners
2

Great work here. Notes on each case:

  1. The goal of Podplane is to host apps & agents. So coding agents should be able to securely write and run Go code in-cluster. That’s why the golang:alpine image is included in the components manifest as an “extra image”. The reason traefik/bootstrap-tls using this rather than pre-compiles is simply to avoid adding an additional image to the dependency list. Provided golang:alpine is an option, it’s actually easier to keep traefik/bootstrap-tls more secure because the golang image is more frequently updated, so we’d need to be constantly re-building bootstrap-tls each time we get a new base image. So the two disadvantages are: 1. larger total dependency filesize and count, 2. more complexity in build pipeline to achieve regular patching/updating. So I don’t see a need to change right now.
  2. Excellent call out. Fixed in Use pause image for certificate secret sync · podplane/components@f590bd6 · GitHub

I hear you on debain12/13 with the static, I would prefer err on the side of lets use the latest for now, and if it becomes a problem at any point can look at pinning.

The vmconfig/test-install isn’t deployed in-cluster it’s for vmconfig/scripts/test-install at main · podplane/vmconfig · GitHub

Great work!