How we scaled Nginx

I love how happy they are to work around blocking open().

This is a very common way of thinking. But in fact there are only two ways to handle I/O. And no matter what you do, you always end up with one of them:

Path 1, blocking I/O: When you have blocking I/O your process continues to the point where the I/O starts, then sends the corresponding request to the kernel and waits until it gets a respond, potentially forever. This is very low resource usage, but sometimes-hanging-forever is quite a huge price. So usually people put this I/O stuff in a thread/fork and use the parent to have a timeout waiting.

Path 2, non-blocking I/O: In this version when the process hits the I/O it will almost-immediately fail when the desired resource (e.g. file, port, whatever) is not available. So usually you are writing a loop and constantly poll for the resource to become ready. This obviously has a rather high cost on resources, because your code gets more complicated (loops, exceptions, etc) and whatever you are I/Oing to has more activity (e.g. if you poll a webserver you constantly create load on that webserver for each client process). But also an advantage is that you can't hang forever, because you usually break the loop after x seconds or y amounts of retries.

You might feel it sucks (at least I do) but there are not more options. Decide for one version that you can live with more easily, tune the variables you can fiddle with, like timeouts/retries, and then move on to other problems.