Breaking the macOS networking logjam in upstream MAME

[rb6502]

Because other emulators, notably QEMU, have had the same problems we do with unsigned binaries/requiring a root helper, there are now centralized solutions:

https://github.com/nirs/vmnet-helper does the same thing as our vmnet-helper but is system-installed (and I believe available through Homebrew), which would immediately eliminate many of the issues we had getting networking into upstream.

macOS 26 has new virtual networking capabilities which don't require root or permission bits and can achieve much higher performance. Those are leveraged by https://github.com/nirs/vmnet-broker which provides an XPC service for emulators to connect to a shared virtual network that's persistent across individual emulator launches.

vmnet-helper supports connecting to vmnet-broker on macOS 26 and later so the path of least resistance is like this:

• Kill the built-in vmnet support and MAME's vmnet-helper
• Have MAME's vmnet-helper networking provider use the shared vmnet-helper instead of ours

We could potentially add a new networking provider that directly connects to vmnet-broker later (or even just directly to the macOS 26 APIs) but for a first pass the shared vmnet-helper seems like the way to go.

[rb6502]

There's also https://github.com/lima-vm/socket_vmnet but it seems like vmnet-helper/vmnet-broker have wider support.

And I'll ping @cuavas on this so he's aware.

[ksherlock]

That sounds like a reasonable approach. I'll take a look at interfacing with nirs/vmnet-helper

[rb6502]

Also, FYI, fully Mac-native MAME incoming: https://forums.bannister.org/ubbthreads.php?ubb=showflat&Number=125407#Post125407

[cuavas]

The fundamental issue with the existing PR is that it relies on mangling MAC addresses, and then ends up having to do deep packet inspection for address resolution protocols, etc.

We have to accept that MAME can’t just accept a MAC address assigned by the host, as it won’t work with various emulated system that will only work with a fixed MAC address prefix, where the MAC address needs to match the host ID, etc. This includes a lot of classic UNIX workstations from Sun, SGI and others.

Mangling MAC addresses isn’t workable. You need to handle every address resolution protocol. For IPv4, you already have ARP, RARP, BootP and DHCP. But then you have to deal with the NBF, IPX and VINES address resolution protocols just to cover the network stacks that were popular on PC in the ’80s and ’90s. When you get to IPv6, the MAC address mangling approach simply can’t work with SLAAC.

The only approach I can see working is to implement virtual subnet and route across the vmnet interface, similar to what VirtualBox NAT does. This is obviously a bunch of work, but at least it would give something supportable. Would any of the things proposed here help with this, or do they just help hook you up with a vmnet interface without providing any other capabilities?

The inherent flaws with the approach in the currently open PR really put a damper on enthusiasm for getting it in. In its current form, it’s just a dead end feature where I’m going to have to regularly tell people, “Sorry, that can’t work due to inherent flaws in the design of the feature,” and I’ve got more than enough of those already.

[rb6502]

From what I can see, both vmnet-helper (the public project, not the one we currently use) and socket-vmnet allow the emulated machine to set the outgoing MAC address, using the vmnet_mac_address key. This is not currently possible with vmnet-broker or the underlying macOS 26+ virtual network API.

[ksherlock]

There's a comment here which claims:

vmnet documentation instructs to configure the virtual interface with the mac address specified by "vmnet_mac_address". Testing shows that this is not required and any mac address works.

(emphasisis mine)

The last time I was testing vmnet (which was 10.14 era) the MAC had to match (as well as the DHCP-assigned ip address). vmnet has received some attention since then so maybe it's no longer an issue. I'll try to play around with it this week.

[rb6502]

It's worth noting that vmnet.framework still has the restriction that a machine can't communicate until it's completed a DHCP transaction with a server that's built in to macOS. So those old protocols won't work without a deep packet inspection state machine sitting in the middle.

For machines that don't require a fixed OUI (DOS/Windows/Linux) the macOS 26+ virtual network API does lift that requirement and so static IPs, TFTP/BootP, and other resolution methods are in play if you provide an appropriate server.

So no matter what there's going to be some "you can't do that" involved. Given the current situation in as-shipped MAME is "no networking is possible at all post-10.13" we already have that restriction. If no level of restriction is acceptable for mainline MAME we'll have to just maintain an active fork with this capability.

[cuavas]

It's worth noting that vmnet.framework still has the restriction that a machine can't communicate until it's completed a DHCP transaction with a server that's built in to macOS. So those old protocols won't work without a deep packet inspection state machine sitting in the middle.

It doesn’t require deep packet inspection, it requires emulation of a virtual switch/router between the emulated machine in MAME and the vmnet interface. That piece would handle that requirement. Ideally we’d also have glue so multiple MAME instances could be behind the same virtual router so they could painlessly talk to each other, and provide an interface for the user to easily implement other virtual services on the virtual subnet, and… you get the idea. The trouble is it’s a pile of effort.

Another thing we need is virtual Ethernet over UDP mulitcast, which would allow virtual subnets of multiple MAME instances without any cooperation from the host OS at all. The main blocker for that is needing a better interface for providing configuration UI for stuff provided by OSD modules (and devices for that matter).

[rb6502]

Along those lines, a Python router (https://github.com/lampmerchant/tashrouter/) already exists for bridging LToUDP (the "Z8530 LocalTalk packets over UDP" thing most emulators now support), and you can connect real hardware to it with the AirTalk device. There's some discussion about building that capability into a future NetATalk too, but they're scrambling to deal with the Linux kernel dropping its remaining AppleTalk support first.