I actually "solved" the problem by using remote controlled light bulbs.
I can still use the switch to turn them on and off, but I have to use the remote for dimming. Not the most elegant solution, but the dimming works flawlessly, and on my model I can also change the color temperature, which is nice, and it was actually cheaper than most "dimmable" light bulbs.
I did this too with IKEA bulbs and their remote control that you can directly pair to multiple bulbs. It works out great, and I don't have to deal with another phone app or buy a hub in my house to manage.
I can still use the switch to turn them on and off, but I have to use the remote for dimming. Not the most elegant solution, but the dimming works flawlessly, and on my model I can also change the color temperature, which is nice, and it was actually cheaper than most "dimmable" light bulbs.