I am probably asking about an apples to oranges comparison, but could someone knowledgeable help me understand in what situations I would want to use gstreamer vs ffmpeg vs pipewire vs others. They all have some overlap and can use each other for some sub-tasks, but I do not know when to reach for what.
gstreamer and ffmpeg can both fill the same niche, but not completely. gstreamer is plugin based while ffmpeg is monolithic. gstreamer has a plugin for ffmpeg, while the reverse is (or wasn't last i checked) isn't true.
You need pipewire, pulse, or plain alsa to actually play the audio streams. Not a lot of overlap most of the time with gstreamer or ffmpeg unless you're dealing with raw streams.
GStreamer is the top-level framework that allows you to connect the modules it offers to solve your problem, ffmpeg is a tool or library to invoke which may cover your case but is a bit harder to integrate and fit for arbitrary usecases, PipeWire is for live connecting input sources to output sinks at a lower level.
If you use the ffmpeg library, I will just say it's extremely unwieldy and difficult to use, and has barely any official documentation or examples. GStreamer is the opposite IME.
In any case, they generally fill different niches. FFmpeg is, generally, a library containing set of A/V codecs and (de)muxers. (There are some more obscure parts, too, plus a command line tool ffmpeg(1) that gives access to most aspects of that library.) GStreamer is, generally, an A/V graph framework originally modelled after Microsoft DirectShow.
If you find FFmpeg too difficult to use, most likely what you want isn't a codec library in the first place, but some higher-level abstraction (e.g. if you just want a video on screen, most likely you don't want to muck around with low-level demuxing, but you'd rather want something that deals with clocks and sync and stuff, and FFmpeg just doesn't do that—it's not a video player). It has a lot of weaknesses, but the API is fairly straightforward for what it does.
Well, one simple case is to make a media player. I used it to make a simple one[0] a few years ago (two bottom right windows, the top left windows are an unrelated directory browser).
