This is the kind of article I'd love to read more of, as in more of each bit! It allowed me to discover the very well made docs to contribute to Firefox[0], which feels very welcoming to an enthusiastic non-genius-expert engineer, who happens to have some experience with CI, testing automation, and a couple languages.
I assume the overhead of the project (and subsequent tweaks to model, re-training and validation) is sufficiently negligible compared to the measured benefits even if those weren't as clear-cut as 70%. I'm unaware of how much compute is required for the task, but likely less than many compute-years per day :)
One thing I did not notice in the approach to the modelization of the problem is any link/tag regarding the platform for which the code changes are made, and the programming languages used. There seems to be some evidence that certain languages could lead to more defect fixing commits[1], and I don't know if there's evidence that some platforms are more prone to bugs (I'm sure wars of words have been fought over this). But would it make sense to have that sort of information inform the model in a way? I fully understand that I might be out of my depth here.
I wouldn't be surprised if language is explicitly included as, for example, a flag in the "data" object. But the model should be able to figure it out by itself otherwise by identifying keywords that only some languages (often) use.
I assume the overhead of the project (and subsequent tweaks to model, re-training and validation) is sufficiently negligible compared to the measured benefits even if those weren't as clear-cut as 70%. I'm unaware of how much compute is required for the task, but likely less than many compute-years per day :)
One thing I did not notice in the approach to the modelization of the problem is any link/tag regarding the platform for which the code changes are made, and the programming languages used. There seems to be some evidence that certain languages could lead to more defect fixing commits[1], and I don't know if there's evidence that some platforms are more prone to bugs (I'm sure wars of words have been fought over this). But would it make sense to have that sort of information inform the model in a way? I fully understand that I might be out of my depth here.
[0] https://firefox-source-docs.mozilla.org/setup/index.html
[1] https://cacm.acm.org/magazines/2017/10/221326-a-large-scale-...