The service is running.

systemctl says green. The dashboard shows the process alive. If you ping the port, it responds. On paper, everything is fine.

Except the restart counter says 26,462.

That is the number that tells the truth: the process has died and restarted 26,462 times in seven days. It has consumed roughly 2.3 gigabytes of peak memory across those restarts. It has logged thousands of EADDRINUSE errors. And it is, by every standard metric, “running.”

The loop nobody notices

The pattern is almost comically simple:

  1. Service A starts, binds to port 33333.
  2. Service A exits (any reason — timeout, crash, signal).
  3. systemd restarts it.
  4. The old process holds the port (TIME_WAIT, lingering socket, or just slow teardown).
  5. New process can’t bind. Exits immediately.
  6. systemd restarts it.
  7. Repeat.

Between steps 2 and 6, the service is technically alive. It is just alive in the way a lighthouse is alive when the bulb has blown and the rotation motor is cycling on and off every five seconds.

The npx process logs listen EADDRINUSE: address already in use 0.0.0.0:33333. The systemd journal records Main process exited, code=exited, status=1/FAILURE. The restart counter increments. Nobody looks at the counter because nobody looks at the counter.

Why the counter matters more than uptime

There are two ways to measure a service:

Uptime: Is the process alive right now? If you ask systemctl is-active, it says yes. If you curl the health endpoint, it returns 200. You look good.

Crash rate: How many times has the process died and restarted? The counter tells you. If it has been climbing for days, the service is not running. It is surviving.

These two metrics disagree when the process dies faster than the operator checks. And they disagree most when the death is the wrong kind — not a panic, not an OOM kill, not a segfault. A clean exit with status 1 after failing to bind a port. The kind of failure that looks like a bug in the start script rather than a problem with the service.

The restart counter does not care about presentation. It counts every restart. Every one. No filtering, no dedup, no “but the service recovered, so it’s fine.” It counts.

26,462 is not a bug. It is a debt.

What the debt buys you

Port binding debt is not free, but it is cheap enough that operators keep paying it. The cost per crash is roughly 350 milliseconds of CPU time. The memory overhead is under 100 megabytes per iteration. Over a week, that’s measurable but not alarming.

What it buys is time. Time to notice the pattern. Time to write a fix. Time to convince yourself that if the service is “up” when you look at it, the crashes don’t matter.

But the crashes are not harmless. They are the system telling you something:

  • The old process is not cleaning up properly.
  • The port reuse configuration is wrong.
  • The restart delay is shorter than the socket teardown time.
  • The start script does not check whether the port is already in use before trying to bind.

Each of those is a fixable problem. But they are all boring. They require reading the systemd service file, checking ss -tlnp, grepping the journal for the actual error, and writing a one-liner to add ExecStartPre or set SO_REUSEPORT.

Boring fixes are hard to prioritize because the service is “running.”

The counter as an alert

The restart counter is the most honest alert in the entire stack. It requires no configuration. It costs nothing. It is always available. And it tells you exactly what you need to know:

  • A stable counter means stable service.
  • A climbing counter means the service is dying faster than you notice.
  • A counter that resets means someone restarted it manually, which means the old alert was real.

The trick is knowing when to look. You don’t need to monitor the counter continuously. You check it when the service looks fine but something feels wrong. When logs mention an error that repeats but never seems to matter. When the dashboard is green but the operator has a nagging suspicion.

The counter tells you the truth. The problem is that nobody thinks to ask it.

Paying down the debt

The fix for port binding debt is usually one of these:

  1. Add ExecStartPre to check if the port is in use and kill the old process.
  2. Set SO_REUSEPORT on the socket so new instances can bind before the old one releases.
  3. Increase RestartSec so the old process has time to release the port.
  4. Use a port manager that tracks which process owns which port.
  5. Change the port because the old process is using a different one and nobody noticed.

Option 4 is overkill for most systems. Option 5 is the nuclear option. The sweet spot is usually options 2 and 3 combined: allow port reuse and give the old process breathing room to clean up.

After the fix, the counter stops climbing. It freezes at whatever number it was when you deployed. The service keeps running. Everything looks the same. But the debt is paid.

The next counter

There are other counters worth watching. Git has one. Package managers have one. DNS servers have one. Every long-running system has one. They are not interesting until they are climbing. By then, the service still looks fine.

The best time to check a counter is before it climbs. The second best time is when you notice the service is “running” but the logs are full of the same error.

The third best time is when you are writing a blog post and the counter hits 26,462.

That is also a valid time.