My question is why is it that this level of diligence is
applied to the civil aviation industry, and not to something like cars and buses which kill way more people per distance travelled.
My guess would be it's because people like to blame somebody else. If it's a plane then there are large organisations to blame i.e. the airline and the manufacturer. While with cars almost always the blame is with the driver i.e. the people. When it's a manufacturer's fault however nobody bats an eye if they have to recall 500,000 cars to install something that would decrease the risk by 1/10th the amount if everybody uses seatbelts.
Aviation industry accident investigation practices were adopted from the railway industry. Back in the mid-to-late 19th century, train accidents killed about as many travellers per capita as road traffic accidents in the 1950s. So the industry adopted a set of principles: no-blame impartial accident enquiries managed by an independent board who would deliver a report on the causes of the accident along with recommendations for best practice to avoid it recurring. Upshot: rail passenger travel is now by far the safest form of land travel.
Bear in mind that a Boeing 777 like the one that went missing on flight MH370 costs on the order of $200M. You then have potential liability for the passengers (if the airline made culpable/criminal mistakes) of maybe $2-4M per head, for up to 400 pax on a 777 -- they're big (MH370 had only 239 people on board) and you're looking at a billion dollar accident. The airlines and manufacturers thus have a huge incentive not to let this happen again.
(Compare to a car accident: 90% or thereabouts are down to human error, and the vast majority of fatal ones "only" involve a couple of deaths, capping the potential liability two orders of magnitude lower.)
In journalism we use the phrase "deaths per mile" to rank importance of a story. A story with death within 1 mile of you is equally important to you than story of 100 deaths 100miles away from you.
In that sense a large number of people are "affected" (as in likely to click/read the story) if 300 people on a plane die 1000 miles away than 1000 people dying in different accidents over longer period in my home state.
When there is public perception the government and politicians want to step in and want show that they are doing something. That something is more regulations and more checks. That makes air travel expensive and more safe.
They can be if they're serious enough. Planes and trains carry enough people/other stuff to usually warrant the independent investigator to step in. Buses are sort of there. Cars aren't.
You can either have a good team of people drill down into the root causes of a particular mass-transit failure, or you can have them suffer the death of a thousand cuts by investigating every nose-to-tail collision on a highway because people get distracted. One's worth far more to society.
My guess would be it's because people like to blame somebody else. If it's a plane then there are large organisations to blame i.e. the airline and the manufacturer. While with cars almost always the blame is with the driver i.e. the people. When it's a manufacturer's fault however nobody bats an eye if they have to recall 500,000 cars to install something that would decrease the risk by 1/10th the amount if everybody uses seatbelts.