> Yeah I keep location services turned off most of the time.
Even if your location services, wifi and bt are off, they occasionally turns on to get your location. In Lineage OS there is an additional setting to not use wifi (when it is off) for location services.
May be you can look into developer options to disable this setting (Please note that even if wifi is off, your phone searches for wifi hotspots to find itself where it is).
And if your wifi/bt is on, your location is (almost) always available to Google. You know it, right?