Since I won't get a better chance to find knowledgeable people: How exactly does iPXE work? Lets say I want to boot a random old PC from one of these images - does the bios need to have iPXE support? Or PXE? Or does it need to first boot into some local system first, which then goes out and gets the image to boot the rest of the way from?
Then there's mention of DHCP and self hosting. Do I need to have some service running within my LAN first or can this go right out to some public server and boot from images there? How does DHCP factor in? I am in so far over my head on this but it seems interesting and something fun to try out.
PXE [0] is a standard for netbooting, typically implementated in the system's boot firmware. A PXE-compatible client sends a DHCP request on boot, to which the network's DHCP server responds with the IP address of a server to netboot from. The client will then connect to this server via TFTP, download an executable image file in the PXE format, and boot it. Using PXE on your own computer requires enabling PXE the firmware setup, setting up a TFTP server to serve the PXE image, and configuring your DHCP server to point to your TFTP server.
iPXE [1] is an open-source implementation of PXE, and much more -- it's much more flexible, it supports additional protocols including HTTP(S) and DNS, it has configuration and scripting options, a basic command-line interface, etc. In order to run iPXE, you need to boot an iPXE image somehow -- e.g. from a MBR or EFI image on a disk drive or USB drive (or even over PXE, I guess). But because iPXE supports more protocols and more configuration, you don't need to set up TFTP and DHCP, and it can chainload into e.g. an EFI image or a Linux kernel instead of being limited to booting images in the PXE format.
An example of iPXE in the wild is the Arch Linux netboot image [2]. They provide pre-configured iPXE images that display an interactive menu to select a mirror, download the Arch Linux installer, and boot it. (It's really convenient since you can just drop the UEFI image at "/efi/boot/bootx64.efi" on a FAT32 thumb drive instead of having to download the whole installer image and 'dd' it onto the drive.)
The submitted project, netboot.xyz, is a similar idea: a preconfigured build of iPXE that lets you interactively download and boot installers for many popular operating systems from a single image.
And while the options previously were interpreted for TFTP use, newer PXE software now understands the use of "http[s]://" in the file name and use that instead of TFTP.
This is super common actually! Most built-in PXE only supports TFTP, which is pretty slow compared to TCP-based stuff. It can make sense to use the built-in PXE to grab a (small) iPXE image over TFTP, then have iPXE grab the (big) real image over HTTP(S). This is also useful if you want to store your main image on something like S3 that doesn’t support TFTP.
For a while I had a script that would create iPXE images dynamically on the fly with the correct HTTPS URL and auth information embedded in them.
Off topic, but does anyone know a good resource that explains how to correctly use "e.g."? (I've looked before, but didn't find one)
While I may seem like a grammar pedant, there are many things that have entered common use while arguably incorrect that I'm not so bothered about ("That begs the question", for example). However while this (incorrect) use of "e.g." is a hill I will die on, even so, I struggle to explain why it's just plain wrong. (It can be used to replace "for example" when preceding a list if examples that illustrate a point, but not as a generic replacement as in this case.)
Someone somewhere must have explained it better...
> while this (incorrect) use of "e.g." is a hill I will die on, even so, I struggle to explain why it's just plain wrong.
Are there many such things you would die for without even knowing why? :)
"Exempli gratia" translates quite literally to "for the sake of example". I see no grammatical reasons why you shouldn't use it as the GP used it.
Are you sure you are not mistaking e.g. with i.e.? The latter stands for "id est" and means "that is", but the two are often confounded, and it's not unusual to find i.e. used to introduce an example. If the GP had used i.e., that would be a hill to die on.
Chicago Manual of Style [6.51, 5.250], though I'd argue this is pedantry for casual conversation. From 6.51: "[...]Note that in formal writing, Chicago prefers to confine the abbreviations i.e. ("that is") and e.g. ("for example") to parentheses or notes, where they are followed by a comma".
I think a secondary argument could be made that i.e. and e.g. are typically used to clarify something in a second clause; e.g., "it can chainload into any form of boot image (e.g., an EFI image or a Linux kernel) rather than being restricted to the PXE image format".
I would recommend picking up a copy of CMOS if you are in want of good ways to explain English grammar. English is my secondary language, and it's occasionally helpful for me to have access to a reference for its grammar when I struggle with understanding a concept. Just keep in mind that languages are bendable and any guide is descriptive and not prescriptive for informal communication.
Belated upvote for answering the question, in contrast to the many who downvoted because they don't understand how to use "e.g." correctly, and were too lazy to find out.
Is this wrong? I always mentally replace "e.g." with "for example" (so this sentence is "it can chainload into, for example, an EFI image"). That sounds right to me. This seems like somebody doing it right :)
Yes, it wrong. "e.g." isn't a direct substitute for "for example". Honestly, it's best avoided as so many people confuse "e.g." and "i.e.", but if you're going to use it, at least use it correctly.
You can use netboot.xyz from a flash drive to boot various operating systems and utilities. Alternatively, PXE (Preboot Execution Environment) has been around for a while and works by allowing a network-capable device to boot from its network interface. A PXE-compatible network card requests a DHCP lease during the boot process, which provides the IP address of a TFTP (Trivial File Transfer Protocol) server and the file that needs to be loaded from the server.
Typically, the network card contains a basic PXE kernel. To enhance this environment, you can chainload iPXE, which offers a broader range of features. iPXE allows for more advanced booting options, such as loading scripts or initiating an unattended installation directly from the network.
Since nobody directly answered some of your specific questions:
> Lets say I want to boot a random old PC from one of these images - does the bios need to have iPXE support? Or PXE?
The most-desirable way to use iPXE is to boot it from a system's built-in PXE, chainloading it from a local DCHP+TFTP server. All routers have a DHCP server, and many enthusiast/professional routers will have a TFTP server available. pfSense/OPNSense for example can both do this.
If you want to go down the rabbit hole and can mod your system BIOS, you can install iPXE into many of them. I wouldn't recommend it, but I did it once and thought it was pretty cool.
> random old PC
If you _literally_ meant like a vintage PC that predates the PXE spec[1], you can boot iPXE from any local media — I architected a project 10 years ago where I actually gave floppy disks to technicians because some of the ancient Dells we were working with required a BIOS update to enable USB booting.
You can also burn iPXE into a network card's ROM chip. There were a lot of PCI/ISA network cards that took an EEPROM and could boot from them if populated. I always thought it would be fun to boot a vintage PC with iPXE straight off of an add-in network card like that. iPXE will hook the BIOS, and you can literally boot MS-DOS via iSCSI.
> Do I need to have some service running within my LAN first or can this go right out to some public server and boot from images there?
Again, typical deployments will have iPXE chainloaded from the system's existing PXE implementation, which means you've got a DHCP server on the local segment, and a TFTP server somewhere on the local network. But you _could_ boot iPXE from any other boot medium, which would mean the DHCP/TFTP setup are not required. iPXE itself, once loaded, can do its work with any network connection.
Generally speaking, if you have iPXE already compiled and flashed onto your NIC, then you can follow these instructions: https://netboot.xyz/docs/booting/ipxe
DHCP is only needed for getting an IP address. You can use the Docker image with the proper DHCP parameters to load it automatically when using PXE/iPXE: https://netboot.xyz/docs/docker
So what I do here is that I have a local PXE server (it's a DHCP option on your router) that serves up iPXE with a preconfigured script to boot via HTTP off of netboot.xyz. So whenever I want to install linux on a new computer, I set the BIOS boot option to boot from PXE next boot and restart. In a few seconds, I am presented with my choice of linux environment.
For my data center servers, I have it booting via PXE to an iPXE with a custom script to take a unique identifier from the host and build the corresponding configuration (NixOS). So essentially for that I define my NixOS configuration in a NixOS flake and plug the new host in and it will boot to the correct configuration. I actually don't have any OS installation on most of the hosts and share the nix store via NFS.
I also keep an iPXE thumb drive around in case I need to do this for something not on my network. In that case, I insert the drive, boot from it, and then ask it to boot from netboot.xyz manually.
There are a bunch of different options to "bootstrap" iPXE in the first place. Least invasive is probably to boot it from a USB drive. You can also set up a traditional PXE boot process using DHCP/TFTP on your network to "chain-load" iPXE (if your client's network card supports PXE), or you can even flash iPXE into the boot ROM on certain network cards.
However you do it, once iPXE starts running, it will take control of the NIC and fetch the actual OS images that it needs from the internet over HTTP.
Some HP server UEFIs also support boot from http. So you can boot the .efi executable directly from the internet.
For https you need to enroll the certificate first and it's a bit more involved.
I am sure there are other implementations I have only seen it on HP servers so far.
Roughly from memory
iPXE is a bootable image that acts similar to a PXE BIOS, so you boot to that image like a regular disk.
It acts like a bios by for example hooking your interrupts for storage reads.
In order to find where the image is, you need some kind of discovery mechanism. This is where DHCP comes in. Remember this is all happening before you have an OS, so it has to be very bare bones
If you want to see how all the magic happens, have a look at the code and docs from Debian’s FAI (Fully Automated Install). Once you have a Debian FAI all setup, you can bootstrap your machine from zero every morning while you have your morning coffee
Why would you ever trust something like this that will helpfully pull images from the internet? I'm sure whomever is running it today is perfectly honest and none of their computers have been compromised, but there's no reason to expect it to stay that way, is there?
Firewalling the application so that only local images are available seems like the only safe way to use this.
> Why would you ever trust something like this that will helpfully pull images from the internet?
How is that different than pulling an ISO image of your favorite distro, or using a package manager like apt?
Yes, I know that Linux ISOs have checksums and apt uses digital signatures, but so does iPXE. The only difference here would be that for some reason you trust the websites of your Linux distro vendor, but not netboot.xyz?
just because you're one of today's lucky 10,000, that shouldn't impugne the project. It's been around since 2016. if you need verified boot, you'd not be using this in the first place.
> you trust the websites of your Linux distro vendor, but not netboot.xyz
Well... yeah... that's not that crazy of a position to take.
Not saying there's anything wrong with netboot.xyz, but it's a question of how many cooks to let in the kitchen, and how many public eyes are on each cook.
If I read the the docs correctly, since source locations are printer, it’s about as trustworthy as trusting “wget $ISO_URL” on your installation to not download anything malicious. Unfortunately what seems to be missing is a hash check after the fact - a missed opportunity since images are loaded to RAM anyway.
(The limitation here is that you have to be able to load the installer image into RAM, which does exclude a lot of smaller nettop/thin/SoC clients unfortunately.)
I tried netboot once, and lost interest when it took minutes to download an ISO over cable internet (DSL would be even worse) every singe time I tried booting into a live distro while debugging things. It's faster to download the ISOs I need ahead of time, save them on a Ventoy drive, and boot (and reboot) into each one in seconds.
I’ve found it to be much faster and far more convenient than downloading the image I want and writing it to a usb drive.
I did at one point keep a few stock images on my tftp server but even there, they would go out of date quicker than my need to use them. So I ended up sticking with NetBoot.xyz for convenience
How do I preload or cache ISOs on a flash drive holding the netboot.xyz client? I haven't seen any mention of it in docs. I've seen some people talking about setting up local iPXE servers, but I don't know if my machines always have functional networking to my server laptop during startup (and this fails outside my home network).
Netboot.xyz allows you to self-host an instance of it, which can cache ISOs. I was doing it for a time, and thought it neat, but not worth the hassle to keep running.
I got really into this sort of thing about ten years ago and set up my wireless router to serve up the initial iPXE image and then load various OS's over the internet. I got about ten or so OS's set up to boot this way then I lost interest and never used it again.
I was looking for a alternative to VenToy... so I scanned all the embedded binaries... every last one... and its completely and utterly clean... but does not support windows, which is a minus only in my case, but an absolute godsend in all others. Not a single Linux I have ever used has been passed over.
I've been meaning to give this a shot myself. I do like netboot.xyz's maintained catalog for convenience, but when I self hosted it (to get the caching proxy benefits), I felt that cache cleanup was a little clunky (basically, go into the cache dir and delete the ISO/image). For the few times I need to install an OS on something, it'd be totally fine to grab the image and stage it on the PXE server.
I'm experimenting on and off with PXE network booting.
In my experience, TFTP is really slow (ancient UDP based protocol) and I'd like to do away with it. Unfortunately TFTP is still required to load something like iPXE to load the actual OS Kernel over HTTP, assuming iPXE isn't part of the NIC firmware or can't be configured to do so.
Turns out that downloading over HTTP is just as slow (few MB/s) when using iPXE so I'm not sure what I'm gaining.
Ideally HTTP boot is possible through the UEFI bios but it seems that this is common on servers, but not on clients? My HP 1L PCs don't have an option to boot over HTTP directly.
> Turns out that downloading over HTTP is just as slow (few MB/s) when using iPXE so I'm not sure what I'm gaining.
iPXE should be able to max out any gigabit NIC that has a decent driver. If you're using a BIOS UNDI driver (undionly.kpxe) or EFI SNP (snp.efi/snponly.efi) you may be experiencing the misery of a crummy driver supplied by the built-in PXE ROM.
If your NIC is supported, try using the bin/ipxe.pxe or bin/ipxe.efi build targets[1] and chainload that instead. If they work at all, it'll mean you've got a NIC with a native iPXE driver, and using that driver may speed things up.
Alternatively, the opposite may also be true. Some of the drivers aren't great, and UNDI/SNP may perform better. If you're already using those builds, try it the other way around.
An interesting concept... load an OS from an OS image file over the network...
If that's the case, then OS'es which do not use any local storage devices, and instead mount remote filesystems over the network -- can't be far behind...
And, if that's the case, then the only thing that the local PC is doing is using its CPU and RAM to run the OS, and network port to read/write data to the filesystem...
But the local PC could also simply run a Remote Desktop or VMWare Client or VNC or equivalent to another remote computer somewhere else on the Internet -- in which case it really doesn't even need to run its own Operating System proper, it only needs to run the software which allows network connectivity to the running instance of the remote Operating System somewhere else on the Internet...
Maybe it would be simpler to build a future "computer" which is basically the glorified equivalent of a long-distance KVM switch over the Internet with local video output...
Hey, such a device would never need its own hardware upgrades... just upgrade whatever remote machine it connects to on the other end, and you've effectively upgraded the local one...
This is like Ventoy? I'll give it a try for when I re-do a flash drive. Can I put windows tools on here as well for removing spy/malware etc to fix PCs? With Ventoy I just make a dir and I can select them when the OS boots, but still boot every ISO image at post
This is more similar to iVentoy, which implements PXE boot, but is closed source. netboot.xyz, on the other hand, requires boot images to be extracted and served in an iPXE-friendly format, which their CI/CD system does automatically. These solutions are good for facilitating OS installations across a fleet of computers on a network.
But for system rescue purposes, Ventoy still wins. You bring your own boot images, which could be arbitrary ISOs, and don't have to worry about networking availability.
Not quite the same. It sounds like netboot downloads the images from over the Internet. Whereas Ventoy you preload the images onto a flash drive ahead of time.
iVentoy is much simpler. Just drop the ISOs in the server folder and 99% they work. If you need to have local images in netboot.xyz it needs waaay more fiddling. It even has it's own DHCP so you do not need to know how to setup BOOTP or anything.
netboot.xyz is neat. As it makes use of iPXE I would like to speak out a token of appreciation for those working on that project. It's such an essential building block in DC's around the globe that many are not aware of.
I'm using this on some USB flash drive because save me to have few USBs with different Linux distros. Everything is downloaded from the internet and this make USB few megabytes only.
Then there's mention of DHCP and self hosting. Do I need to have some service running within my LAN first or can this go right out to some public server and boot from images there? How does DHCP factor in? I am in so far over my head on this but it seems interesting and something fun to try out.