You can boot from a HDD and then just not ever write data back to it. This would be the most trivial solution, and it’s something people do with their Pi’s a lot to avoid SD card failure.
You could network boot, pull the OS from the network at startup. Fun fact, this is how some rockets fly! No onboard persistent storage needed. Everything boots into and runs from ram the whole 10 ish minutes of operation.
You COULD do a ROM as you suggested, but that’s a LOT of ROM. Seems odd to do imho.
I remember that there was a ROM in the Amiga 500 which had the kickstart software on it which you’d load from a diskette on the predecessor the Amiga 1000. This made it much faster to boot because you would not need to switch diskettes in the middle of the boot.
This can be handled a few different ways.
16MiB is enough to hold entire Linux distro. Example: OpenWRT
I remember that there was a ROM in the Amiga 500 which had the kickstart software on it which you’d load from a diskette on the predecessor the Amiga 1000. This made it much faster to boot because you would not need to switch diskettes in the middle of the boot.