Apps used to trust the user store by default, but that changed back in Android 7. Now they only trust the system store by default and need to opt into also loading the user store. So, it's not that they look at the stores and pick one, it's that the user store has effectively been disabled for most apps (browsers usually work, thankfully). Even Firefox for Android will only use the user store if you go through a five step process to open the hidden settings.
Some apps do certificate pinning, which basically only validates certificates against a specific certificate authority and completely defeats any system certificate store.
You can MitM these apps by injecting code to bypass their restrictions. The eBPF methid linked above works, or you can use Frida in root or rootless mode to inject a variety of existing scripts to defeat certificate validation. This is a lot more involved than installing a certificate authority, but it'll work if you want to reverse an app.
Some apps do certificate pinning, which basically only validates certificates against a specific certificate authority and completely defeats any system certificate store.
You can MitM these apps by injecting code to bypass their restrictions. The eBPF methid linked above works, or you can use Frida in root or rootless mode to inject a variety of existing scripts to defeat certificate validation. This is a lot more involved than installing a certificate authority, but it'll work if you want to reverse an app.