Add epoll integration for network sockets

[WhySoBad]

Hi,

This pull request adds integration for Miri's epoll shim to the TCP socket shim.

Due to the platform specific differences of mio's poll, the blocking I/O manager had to be refactored to a high degree. Instead of having sources only registered in the poll when we're currently interested in an event, we now have the sources registered in the poll with all available interests for their entire lifespan. We now also store the current readiness for every source in the blocking I/O manager.
When for example a thread should be blocked until some I/O interest is fulfilled on a source, we just add the receiver to the blocking I/O manager without changing anything in the poll itself. In every call to poll we now return all interest receivers whose interests are currently fulfilled -- no longer only those which are newly fulfilled.
Once we hit an EWOULDBLOCK in a source operation (e.g. read, write, accept, ...) we need to falsify the corresponding readiness in the blocking I/O manager to keep it up to date.

With this refactor, the actual epoll implementation is pretty straightforward -- we just need to return the readiness stored in the blocking I/O manager from the epoll_active_events method of host-backed source file descriptions.

[rustbot]

Thank you for contributing to Miri! A reviewer will take a look at your PR, typically within a week or two.
Please remember to not force-push to the PR branch except when you need to rebase due to a conflict or when the reviewer asks you for it.

[RalfJung]

Feedback was given IRL.
@rustbot author

[rustbot]

Reminder, once the PR becomes ready for a review, use @rustbot ready.

[WhySoBad]

I noticed that I wrote the review comments of today's meeting in the PR on my fork, but every of those comments should now be resolved anyways.

The BlockingIoManager:poll now updates the epoll readiness of all sources. I've still kept the InterestReceivers even though we only need it for unblocking threads at the moment (maybe there will be something in the future which also needs to be added to the blocking I/O manager).
I've also renamed the EpollEvents struct to EpollReadiness. This is more in line with the new BlockingIoSourceReadiness struct of the blocking I/O manager.

Also, as you requested, the epoll test cases should now fail/block indefinitely when we forget to register a socket or when we forget to remove the readiness after receiving EWOULDBLOCK.

@rustbot ready

[RalfJung]

The BlockingIoManager:poll now updates the epoll readiness of all sources. I've still kept the InterestReceivers even though we only need it for unblocking threads at the moment (maybe there will be something in the future which also needs to be added to the blocking I/O manager).

I prefer clean code over premature generalization. So unless we have at least a concrete idea for what other interest receivers there may be, I think it'd be less confusing to just hard-code for now that these are all blocked threads.

[WhySoBad]

Okay, so I'll change it that we just register thread id's together with an interest.
The unblocking can then also be done in the poll, what about the deregistering? Should we implicitly deregister the thread id's once we unblocked the thread in the blocking I/O manager (how it's on current master) or should it always happen explicitly in the unblock callback (how it's in this PR)?

[RalfJung]

Both are fine for me.

[WhySoBad]

Both are fine for me.

I've now implemented it such that we always need to deregister the interests explicitly.

[RalfJung]

I like it, very nice work. :) But I do of course have some comments. ;)

@rustbot author

View changes since this review

[RalfJung]

Is there some documentation somewhere saying that "error" sources should be "ready" for reads and writes?

[WhySoBad]

For mio we have this: https://docs.rs/mio/latest/mio/struct.Poll.html#readiness-operations
Should I link this here?

[RalfJung]

That does not say that read implies error. It says that read may imply error.

I guess this is actually mostly a decision for us -- should we wake up threads on an error signal? That probably makes sense, but then we should explain why.

[rustbot]

This PR was rebased onto a different master commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

[WhySoBad]

@rustbot ready