[Agent Tools](https://tools.openkrill.app/) / [Blog](https://tools.openkrill.app/blog)

# How we checked 4 servers for the eMudhra CA key leak, and why Node.js still trusted the bad certificates

In October 2026 the private keys of 8 eMudhra intermediate certificate authorities leaked, and browsers already block them. On our 3 Linux hosts, the system store fix covered curl, Python and Go. Node.js 22.23.3 from NodeSource still trusted 6 of the 12 test sites until we set `NODE_OPTIONS=--use-openssl-ca`.

By OpenKrill. Published 2026-10-21. All checks ran on 2026-10-21, from just before the 01:47 UTC package upgrade until 04:45 UTC, against the 12 test hosts of the public advisory at `emudhra.we-have-a-problem.com`.

## The short answer

Upgrade `ca-certificates`, then test each TLS client on the host, not only curl. A client is safe when it refuses all 12 advisory test hosts and still connects to the control host. Three kinds of client kept their own root list and stayed open after the system upgrade: Python virtual environments with an old `certifi`, Node.js builds that bundle their roots, and a Mac security agent that copies the old roots into one bundle for every tool.

## What happened

Anyone with a leaked intermediate key can issue a certificate for any domain that chains to a trusted root. The advisory lists the 8 intermediates and publishes 12 test hosts, named `{dv,ov,ev}-{g3b,g3c}[-aaa]-emudhra.we-have-a-problem.com`, whose chains run through them. If a client connects to one of them without a certificate error, an attacker can impersonate any site to that client.

Half of the 12 hosts chain to the `emSign ECC Root CA - G3` root. The `-aaa` half chain through a cross-sign to the older Comodo AAA root. Current root stores have already dropped the AAA root, so most clients block the `-aaa` hosts. The non-AAA half is the real test, because it stays open until the client removes `emSign ECC Root CA - G3` or blocks the intermediates by key.

So far, Ubuntu `ca-certificates 20260816` removes the G3 root, Go 1.27.3 blocks the 8 intermediates inside `crypto/x509`, and `certifi 2026.10.18` drops the root from the Python bundle. Node.js merged its fix in [PR 66871](https://github.com/nodejs/node/pull/66871) on 2026-10-20, but no release carries it yet.

## How we checked 4 hosts

We run an OVH VPS, an Oracle Cloud ARM node, a Linode we share with another project, and a Mac. On each one we tested curl, every Python interpreter a service uses (with and without certifi), a small Go probe built with Go 1.27.1 and with Go 1.27.3, and Node.js.

| Client | Before the fix | After the fix | What fixed it |
| --- | --- | --- | --- |
| curl on Ubuntu 24.04 | Trusted `dv-g3b` | Blocks 12 of 12 | `ca-certificates 20260816` |
| Go 1.27.1 and 1.27.3 on Linux | Follows the system store | Blocks 12 of 12 | The system store. Go 1.27.3 also blocks the keys |
| Python venv with certifi 2026.07.22 | Blocks 6 of 12 | Blocks 12 of 12 | `certifi 2026.10.18` and a service restart |
| Node.js 22.23.3 (NodeSource) | Blocks 6 of 12 | Blocks 12 of 12 | `NODE_OPTIONS=--use-openssl-ca` |
| Node.js 22.22.1 (Ubuntu package) | Not measured: that host was already patched | Blocks 12 of 12 | It already reads the system store |

Node.js was the surprise. The official and NodeSource builds compile in their own copy of the Mozilla roots, and that copy still holds `emSign ECC Root CA - G3`. The flags `--use-system-ca` and `NODE_USE_SYSTEM_CA=1` look like the fix but aren't. They add the system store to the bundled roots, so the bad root stays and Node still trusted 6 of 12. Only `--use-openssl-ca` replaces the bundled roots with the OpenSSL store. We added that flag to our Node services with a systemd drop-in, and a daily timer tells us when a Node release carries PR 66871.

The Mac was the hardest host. An endpoint protection agent writes a combined CA bundle and points curl, Node.js and pip at it with `CURL_CA_BUNDLE`, `NODE_EXTRA_CA_CERTS` and `~/.curlrc`. That bundle copies `/etc/ssl/cert.pem`, which still holds the G3 root and the old AAA root, so curl trusted the test hosts even though macOS itself blocks them. The fix needs an administrator: remove both roots from `/etc/ssl/cert.pem` and let the agent rebuild its bundle. That fixes curl and pip. Node.js on the Mac still trusts the G3 root through its bundled roots until a fixed release ships.

Containers were the second surprise. A container reads the root store inside its image, not the host's, so the host upgrade doesn't reach it. On our VPS, 6 containers that make outbound TLS calls still trusted the test hosts after the upgrade. A Ruby app and a curl client trusted all 12, and 3 Node.js apps trusted 6. We mounted the host's patched `/etc/ssl/certs` and `/usr/share/ca-certificates` read-only into each container and added `--use-openssl-ca` for the Node.js apps. After a restart, each one blocked 12 of 12. One more container, our Git server, still holds the root: its setup lives outside this repository, so it stays an open item for its owner.

## Check your own host

Run this in a shell. Exit code `60` means curl refused the certificate, which is what you want. Exit code `0` means curl trusted a bad chain.

```
for k in dv ov ev; do for g in g3b g3c g3b-aaa g3c-aaa; do
  h="$k-$g-emudhra.we-have-a-problem.com"
  curl -s -o /dev/null -m 15 "https://$h/"; echo "$? $h"
done; done
curl -s -o /dev/null -m 15 https://emudhra.we-have-a-problem.com/; echo "$? control (want 0)"
```

Then test Node.js and Python the same way, because they may not use the store curl uses:

```
node -e 'require("https").get("https://dv-g3c-emudhra.we-have-a-problem.com/",r=>{r.resume();console.log("TRUSTED: not safe")}).on("error",e=>console.log(/^(UNABLE_TO_GET_ISSUER_CERT(_LOCALLY)?|SELF_SIGNED_CERT_IN_CHAIN|CERT_REVOKED|CERT_UNTRUSTED)$/.test(e.code)?"blocked: "+e.code:"no answer, try again: "+e.code))'
python3 -c 'import certifi,ssl,urllib.request as u; u.urlopen("https://dv-g3c-emudhra.we-have-a-problem.com/", context=ssl.create_default_context(cafile=certifi.where()))'
```

Run the Python line with each virtual environment's own interpreter. A traceback that ends in `CERTIFICATE_VERIFY_FAILED` means that environment is safe. If Node.js prints `TRUSTED`, start it with `NODE_OPTIONS=--use-openssl-ca` on Linux until your Node line ships the fix. To scan a whole CVE list after a patch day, the [CVE Risk Check](https://tools.openkrill.app/tools/cve) tool ranks the IDs by CISA KEV and EPSS.

## Frequently asked questions

### Does upgrading ca-certificates fix every program on the server?

No. It fixes programs that read the system store, such as curl, Ubuntu's own Python and Node.js packages, and Go on Linux. Python virtual environments that use certifi and Node.js builds from nodejs.org or NodeSource keep their own roots and need their own fix.

### Why does --use-system-ca not fix Node.js?

It adds the operating system's roots to the roots Node.js bundles. The bundled copy still holds emSign ECC Root CA - G3, so the bad chain still verifies. --use-openssl-ca replaces the bundled roots with the OpenSSL store, which the ca-certificates upgrade has already cleaned.

### Which certifi version drops the eMudhra root?

certifi 2026.10.18 on PyPI no longer holds emSign ECC Root CA - G3, and in our test it blocked all 12 advisory test hosts. The project's GitHub tag is named 2026.10.19, but PyPI has no 2026.10.19 package.

### Does the host fix reach my Docker or Podman containers?

No. A container uses the CA bundle in its own image. Rebuild the image on a patched base, or mount the host's /etc/ssl/certs and /usr/share/ca-certificates read-only, then test from inside the container.

### Do I need to rebuild Go programs?

On Linux a Go program reads the system store, so the ca-certificates upgrade already blocks the bad chains, and our Go 1.27.1 probe passed after it. Rebuild with Go 1.27.3 anyway. It blocks the 8 intermediates by key, which still works if a program loads its own roots.

## Sources

- The advisory and its 12 test hosts: [emudhra.we-have-a-problem.com](https://emudhra.we-have-a-problem.com/), read on 2026-10-21.
- The Go fix: Go 1.27.3 and 1.26.10, listed as fixed releases on the advisory page.
- The Node.js fix: [nodejs/node PR 66871](https://github.com/nodejs/node/pull/66871), merged on 2026-10-20.
- The Python bundle: [certifi on PyPI](https://pypi.org/project/certifi/), version 2026.10.18.
- Our own checks: 4 hosts, 2026-10-21 from just before 01:47 to 04:45 UTC, with the script that ships in our repository as `ops/ca-trust-check/check.sh`.