Run your own XMPP server - prosody or ejabberd - and you have all the freedom in the world to use it for whatever purpose you want whether that be remote control, 'bots' or anything else. If you're a resident of the EU you'll also escape the 'chat control' [1] (mass surveillance) law which is a boon. Prosody runs happily on a SBC like a RasPi 2 or higher - even a RasPi 1 would work - and takes next to no maintenance to keep it up and running.
Do you have suggestions for XMPP client libraries (across any language)? I really wanted to do this myself but it seems that the first few results I found across several different languages haven't been actively maintained.
Web: xmpp.js or converse.js (the latter is a full client, but can be embedded "headless" with any custom UI you like)
Python: slixmpp
Go: Mellium or go-xmpp
Rust: xmpp-rs
Java: Smack
Any platforms I missed?
There is also the Snikket SDK in development, which is still in its early stages, and designed for XMPP IM apps specifically (whereas most of the libraries listed above are generic and can be turned to any purpose). The SDK is still lacking a few big things like E2EE, but already has calls and some other stuff.
Edit: I should add that building a good messaging app is a lot of work (ask anyone who has tried it, regardless of protocol). If that's what you're doing, consider contributing to or at least starting with an existing project.
The Vector people who develop Matrix run a centralized server with APNS client certificates (from Apple) that allow them to push notifications to every single Matrix iOS user when they receive messages. This centralized server is a notification proxy for all iOS users that use that client.
With federated systems, notifications are a big problem on iOS because the apps don't actually get to run in the background in the traditional sense. They can periodically "refresh", or they can be opened by a user, or they can be woken up by a notification. Notifications can only come from Apple, and can only come via Apple from the app's developer.
Even selfhosted Mattermost pushes notifications to users via the Mattermost organization's centralized APNS notification proxy. It's an annoying form of phone-home, and I think it may even leak message content to the publisher (because the notifications can contain message data).
If you self-host an XMPP server, you can't get realtime notifications of messages on iOS unless you have the app actively open and foregrounded so the app itself can maintain a connection to the server. The moment (or, actually, 15-20 seconds later) you lock your device, poof goes the connection.
XMPP is not end to end encrypted, and is a dying (and weird) protocol. If you're going to selfhost anything, Matrix (via dendrite or synapse or whatever the current state of the art is) is probably the correct answer.
Your perception of XMPP is over a decade out of date. XMPP has push notifications and end-to-end encryption in all the modern apps.
You're quite right that a situation such as you describe (requiring the app to be open to receive messages) would be a horrible experience. Support for push notifications was a solved problem years ago.
There is no way for your selfhosted xmpp server to deliver notifications to an iPhone unless you are the publisher of the xmpp client app you use (which requires doxing yourself to Apple and paying $99/year if you don’t want to sign new binaries every week).
The server support is irrelevant; this is an iOS/APNS issue.
fwiw Element and Element X support UnifiedPush & ntfy on Android these days, so you can avoid routing everything via FCM/GCM/C2DM or whatever Google's push system is called these days. There isn't an equivalent on iOS yet though, so if you want to use your own push server, you have to build your own IPA.
[1] https://mullvad.net/en/chatcontrol