When I was working on Othernet (www.othernet.is), I wanted to do Over-The-Air updates of our firmware. Over a unidirectional, slow (~20kbps), broadcast satellite link with significant packet dropout this was quite a challenge. Amongst the many tools that helped in making it work was diffoscope. I used it to make our buildroot-based firmware have deterministic, minimal-diff build, thus reducing the amount of update I needed to send out.
For a few weeks, I simply iterated between building the firmware, using diffoscope to diff the new build against a previous build to find what files changed and where, fix buildroot packages to eliminate these file changes whereever possible, lather-rinse-repeat.
In the end I was able to reduce our OTA update sizes to a typical 20-50KB (base, compressed firmware image as 30MB).
It's surprising how often projects retag releases. Diffoscope makes it easy to see the difference between the original tarball and the retagged versions.
Changes vary from from forgetting to update an in-source version to tagging the wrong commit to fixing file metadata (file owner in the tarball).
It looks like it uses a Wagner–Fischer algorithm to find the diffs. I think it would be nice if that is mentioned in the documentation. In formats that are not order dependent the algorithm can't find a good comparison because no generic algorithm can without more input.
There is a GitHub repo of it that is now deprecated, this is apparently the official repo. I’m curious how these stack up to other, similar tooling, but for now I’m starting to wonder how much work it would take to modify GNOME Sushi [1] to call one of these utilities when two items are selected in the GNOME Nautilus file manager, and go one step beyond macOS Quick Look (though we would still be missing the Quick Look powerful editing capabilities).
I'm currently working on comparing 2 directories of around 3000 JSON/YAML files each, and while meld can open the diff viewer in a few seconds, diffoscope can't seem to process them.
I love meld, makes my life so much easier :3
EDIT: diffoscope got unstuck after some minutes, but I still prefer the default GUI viewer from meld.
For a few weeks, I simply iterated between building the firmware, using diffoscope to diff the new build against a previous build to find what files changed and where, fix buildroot packages to eliminate these file changes whereever possible, lather-rinse-repeat.
In the end I was able to reduce our OTA update sizes to a typical 20-50KB (base, compressed firmware image as 30MB).
Fun times!