A few that I'd like to learn about more deeply given the time:
1. Reverse engineering and vulnerability research. Not that much programming in the sense of writing code, but it basically requires you to be deeply familiar with assembly, compiler optimizations, application binary interfaces, and obfuscation methods. It's especially interesting because there are basically two groups that are heavily involved in this work: independent hackers and nation state or adjacent entities (infosec companies, defense contractors, etc).
2. ROM hacking and homebrew. This is similar to reverse engineering, but it's done for more fun reasons. Lots of super interesting work people have done to do things like mod or optimize classic console games, decompile games so they can be recompiled for modern systems, or just learning the development stack of an obsolete console to create new games and software from scratch. It's especially interesting because you tend to have to get into older or more obscure ISAs like MIPS, PPC, or MOS 6502. Check out Kaze Emanuar for some examples.
3. Mainframe computing. It's only esoteric because the hardware and software is largely proprietary and locked down. The Hercules emulator exists though, and IBM lets you run the 30+ year old versions of their Mainframe OSs as a hobbyist without needing to worry about licenses and stuff. The business model sucks, but Mainframes are interesting since they represent a very different paradigm of computing from the now commonplace X86+Linux commodity machines. There's a YouTube channel called moshix which offers pretty accessible long-form information on how to get started hacking around on Mainframe emulators.
1. Reverse engineering and vulnerability research. Not that much programming in the sense of writing code, but it basically requires you to be deeply familiar with assembly, compiler optimizations, application binary interfaces, and obfuscation methods. It's especially interesting because there are basically two groups that are heavily involved in this work: independent hackers and nation state or adjacent entities (infosec companies, defense contractors, etc).
2. ROM hacking and homebrew. This is similar to reverse engineering, but it's done for more fun reasons. Lots of super interesting work people have done to do things like mod or optimize classic console games, decompile games so they can be recompiled for modern systems, or just learning the development stack of an obsolete console to create new games and software from scratch. It's especially interesting because you tend to have to get into older or more obscure ISAs like MIPS, PPC, or MOS 6502. Check out Kaze Emanuar for some examples.
3. Mainframe computing. It's only esoteric because the hardware and software is largely proprietary and locked down. The Hercules emulator exists though, and IBM lets you run the 30+ year old versions of their Mainframe OSs as a hobbyist without needing to worry about licenses and stuff. The business model sucks, but Mainframes are interesting since they represent a very different paradigm of computing from the now commonplace X86+Linux commodity machines. There's a YouTube channel called moshix which offers pretty accessible long-form information on how to get started hacking around on Mainframe emulators.