The spacecraft then automatically switched into a safe
mode and, at about 4:10 a.m., fired thrusters to try to
stop the rotation. But because the wrong command had
been uploaded, the firing caused the spacecraft to
accelerate further. (The improper command had been
uploaded to the satellite weeks earlier without proper
testing; JAXA says that it is investigating what
happened.)
Going into safe mode is a thing. It happens with NASA stuff, ESA stuff, whatever. The spacecraft failed to stabilize and went into safe mode, and that's proper. Whatever glitch in the systems, this would have saved it and allowed for recovery.
But the uploaded command to shed rotational velocity was wrong. This is what caused the loss of the spacecraft. I'm sure there will be a pretty heavy postmortem on how this happened.
The more striking thing to me is the long failure cascade before that, mostly hardware failures: the star tracker that glitched over the South Atlantic Anomaly, the gyro that reported a nonexistent spin, and the reaction wheel that didn't spin down properly.
It seems very strange that the 3 missions that would deploy an X-ray calorimeter have all failed. I feel sad for the scientists who have been hoping to get a working one into space for what sounds like over 16 years now (based on the article).
I don't know the expected success rate for these missions, it just seemed odd that each time this particular instrument has been launched, an issue occurred to render it unusable. Maybe it is par for the course, but it does raise suspicions (at least in my mind).
And before there was an Anti-Lander Defense System, there was another method employed by the Martians to hide things from our eyes:[1]
On November 14th 1971 NASA's Mariner 9 became the first spacecraft to successfully orbit another planet.
...
Except what came back ... were not pictures of the intricate canyons, craters, and mountains of Mars, but pictures of a blanketed world, a dust enshrouded mystery. Mariner 9 had arrived in the midst of one of the greatest global storms humans have ever witnessed on Mars.
I was thinking more like parties interested in sabotaging the line of research due to personal vendettas against the scientists who would gain from this success, or those who have ideological reasons to block what may be discovered using the technology. I realize this may be far-fetched conjecture, but I'm just sharing my thoughts.
Sorry, but that's crazy. I was previously involved in this mission. The whole community was really excited to see the first results from the soft X-ray calorimeter. Furthermore, there was going to be an open call for proposals for observations, for any groups to apply to.
Thanks for sharing that. As an aside, I am glad you put that information about the Veusz visualization software in your profile text; I had never heard of it before and want to check it out some more, looks nice.
I'm surprised that they had to design custom inertial stabilization, considering how many times it's been done successfully before. Was it NIH mentality? Or did it have some requirement for more precise stabilization than other space telescopes?
At a guess, I'd say that most satellites have different requirements due to the way their weight is distributed and where the inertial stabilization is done.
As far as I understand it, Satellite design is all about cramming the most amount of features in as small and light a package as possible. That would mean a lot of tight coupling and a hard time standardizing anything across different types of satellites.
While that's true for science payloads, when it comes to commercial satellites, there are a handful of 'busses' that form the core of the vast majority of satellites.
Commercial payloads are a lot of the time. Science payload have a harder time doing that because the instruments they're designed to carry come in weird shapes and can't really be chopped up and rearranged to fit into the standard satellite bus.
(I have absolutely no insight into the software development practices of JAXA and their subcontractors, so I apologize if this is insensitive or uniformed.)
Is this another sign of how bad japanese hardware-oriented companies and organizations are at doing software? Like the organisational software crisis at Toyota?
* Don't compare JAXA (or anyone else) to NASA at this point. We (the US) have a ton of experience, some of it bought the hard way.
* This isn't Japan's typical performance. They have a nice string of very good space science missions. And don't forget things like the Mars Climate Orbiter[1], where ground based commands sent it through the atmosphere due to a unit mix up.
A recent NASA science satellite had its magnetic torque bars wired in backwards by the contractor, so it's not like this is a thing of the past in the US, either. Thankfully they saw the angular momentum building and shut it down before it spun out of control.
Space is hard, and in many cases you have only one chance to get it right.
I hate to feel that way; but yeah most software I see come out of Japan scares me. Outside of legally-critical software, such as automotive control systems: it doesn't seem that there is much focus on quality.
I suspect it has to do with the scale of integrators in the country being much smaller, and lacking in terms of B2B collaboration.
There are some jewels here and there, though. It's probably just a matter of culture. Maybe they're also somewhat isolated from the English and Chinese language leads in software development. I don't think Japanese English language education is very effective. Their popular courses likely don't leave an individual comfortable with technical reading in English.
I work for a Japanese company and sometimes work with people from other companies as well. Most of the people around me avoid English at their best, therefore cannot not access to the knowledge on stackoverflow, also documents in Japanese are usually old or incomplete. Of course it depends on the field (e.g. Japanese documents of ruby on rails are pretty good), I am not sure about aerospace engineering, but I'd not be surprise if most of them feeling uncomfortable reading English.
> Most of the people around me avoid English at their best, therefore cannot not access to the knowledge on stackoverflow, also documents in Japanese are usually old or incomplete.
Is there not a Japanese version of Stack Overflow? Maybe there should be?
It wouldn't help. The real problem is that they aren't comfortable with English. The reason English has all the documentation, all of the methodology, all of the literature, all of the community: network effects.
A Japanese Stack Overflow would inevitably be less useful than access to the English Stack Overflow.
This, exactly. Stackoverflow does have Japanese version, and there are sites like Qiita for programmers to share knowledge. But the problem is most of the existing discussions are in English, and a Japanese version of the sites wouldn't help much except all contents are translated (not possible).
Outside of legally-critical software, such as automotive control systems: it doesn't seem that there is much focus on quality.
The analysis of Toyoda's software that might have been behind the claimed acceleration/can't stop the engine problems indicates that even there, at least Toyota is all but hopeless.
> There are a large number of functions that are overly complex. By the standard industry metrics some of them are untestable, meaning that it is so complicated a recipe that there is no way to develop a reliable test suite or test methodology to test all the possible things that can happen in it. Some of them are even so complex that they are what is called unmaintainable, which means that if you go in to fix a bug or to make a change, you're likely to create a new bug in the process. Just because your car has the latest version of the firmware -- that is what we call embedded software -- doesn't mean it is safer necessarily than the older one….And that conclusion is that the failsafes are inadequate. The failsafes that they have contain defects or gaps. But on the whole, the safety architecture is a house of cards. It is possible for a large percentage of the failsafes to be disabled at the same time that the throttle control is lost.
> Even a Toyota programmer described the engine control application as “spaghetti-like” in an October 2007 document Barr read into his testimony.
I have seen analyses of ECM software; and I think they could benefit greatly from a new language.
The priority seems to be reliable real-time computation, and the solution is to calculate everything as a continuous function. Anything that would be a branch is instead a binary mix of the conditions, triggered with a threshold function.
I can think of many ways of structuring programs which are spaghettiish in C, but would yield the desired results in an automotive ECM.
I've tackled some problems that would have ideally, from a performance perspective, been implemented with very deeply nested if()'s. I've found a truth table like layout of bool evaluations to be a nice compromise between maintainability and performance:
void doit(const tststructptr* in) {
assert(in); /* Because we test our code, for safety */
const tststructptr t = *in;
if ( t.t1 && t.t2 && t.t3) { f1(); }
else if ( t.t1 && !t.t2 && t.t3) { f2(); }
else if ( t.t1 && t.t2 && !t.t3) { f3(); }
else /* TODO: Fix, this should never happen */
releasenerveagent(SLEEPINGQUARTERS);
}
A long sequence of ternary operators allow for a prettier layout, but is more error prone.
Nintendo seems to care quite a lot (I worked at Opera when the two companies built the Wii browser together. Wow. They were incredibly brutally perfectionist, in a painful but somehow admirable way. I've never seen a company doing so many iterations of some particular detail before that.)
But yeah, I agree, whenever a large Japanese company that is not Nintendo does software (at least for consumers), you can pretty much expect a disaster.
(I know there's a small group of startups that do good stuff - I am not talking about them.)
Maybe the reason we think of asian hardware companies unable to build good software is tainted by the fact that one big american hardware company (Apple) managed to do so (build good software, I mean).
I mean, which other large american hardware-centric companies have managed the transition to software well?
I don't disagree, but I would imagine most base their view on the titans of each country. Apple and Sony got a lot easier to compare in the 2000's and Sony really did have bad software (friggin e-reader). Steve Jobs talked about Sony a lot in interviews. I would bet if we talk South Korean products, Samsung would be the archetype. Microsoft for their part makes nice hardware with pretty good software to back it up.
I haven't used a PlayStation in a long time so I don't know the current state, and I would guess Nest is not helping the US side of the equation. Apple isn't exactly covering itself in glory in the last couple of iOS releases (never mind the "California" series of OS X releases).
The PS3 software (based on FreeBSD) had quite nice usability but horrible system level performance, particularly when it came to disk/filesystems. You'd end up watching software update install progress bars more than play actual games.
The PS4 has a weird UI that feels oddly half-finished, but does okay on system level performance. It's sort of where the Xbox 360 was in 2005.
That's cool to hear from the other side; I was at Nintendo for many years and worked closely with the team that directed the browser, so I know exactly who you're talking about (though I wasn't with them for the Wii browser). All Nintendo software is developed like that, and it's still different from how the major American software companies work, but it's very effective for what they do.
Obviously I'm biased, but I agree Nintendo is the only major Japanese company that has any idea how to write software. They also have a legacy of incredibly talented programmers that they respect more than most Japanese companies.
This. A programmer at a big Japanese company has about the same status as the guy on the loading dock. It's something you do when you're fresh out of school and only as long as you have to.
I agree with @dwc, and would call it a fluke, possibly human error. In any case, there's no need to single out Japan on software.
After Russia, US, and China, Japan has the most satellites in orbit.
China which is always called on its quality control went 16 for 16 launches, and they launch other countries satellites too. They also landed on the moon in 2013. Ironically, they still can't make a ballpoint pen as good as the Japanese,Swiss or German ones due to consumer spending habits [1].
Somebody mentioned South Korea as a comparison. Well they only have 16 compared to Japan's 153 satellites. Japan's space program is scientific, while there are military launches in the US, Russia and China with those budgets hidden to develop such capability.
Japanese robotics are amazingly performant,so they must be doing something right there both hardware and software-wise. Realtime sensor data coupled with mechanics, mechatronics. Funny, anime has robots with rockets, and yet Japan produces the world's top humanoid robots vs. the military and seemingly menacing robots from Boston Dynamics (Alphabet, Google). Although, Atlas is giving Asimo a run for its money.
A lot of satellites were launched off of Soyuz (Russian) rockets too.
The UK launched one satellite on its own rocket in 1971, and now piggybacks or rents time on satellites.
Space launches are expensive, technically challenging, and still risky. SpaceX lost the 8 cubesats on the June 28th launch failure. Mechanical, not software, but it shows the risks involved are not isolated to a single country over another, software or hardware.
I didn't downvote your original comment (just saw it and this one at the same time), but perhaps it has something to do with your blanket categorization of all Japanese software?
Well, if you read my comment properly, you'll realize that what I actually said is that large japanese hardware companies tend to be really bad at software. Nintendo is pretty much the one exception.
Perhaps it's one of those "only happens in real use" corner case. I had a stupid little robot shred itself to parts during live use (controlled environment no humans) due to a bug. Learn and move on I guess.
Testing a big spacecraft is hard. With a cube sat, you can put it on an air beating and make sure something isn't catastrophically wrong, like the reaction wheels are backwards. You can't do that on a big spacecraft. So instead, there's lots of testing with flight software and a simulator. But what if someone made a mistake interpreting the wheel direction specification to the mechanical body frame, for example?
It seems like the core error was in the inertial measurement unit: it would be a common cause between the reaction wheel failures and the failure of the despin burn.
Seems like the kind of thing one would have multiple of, along with voting, it should also contain a kalman filter. The software correcting the rotation should have been run in a tighter feedback loop so that it would stop making the problem worse.
Lots of these subsystems can be tested in a pure software simulator. Esp when it comes to faults.
Thinking about it further, the simulator should consume the logs from the existing system and map those logs from existing simulation runs, use a form of compressed sensing and pattern matching to figure out what is occurring just from the logs and previous simulation runs.
It's not political. In fact, it has nothing to do with this.
You're free to support Donald Trump, but it has nothing to do with the failure of Japanese satellites. Personally, I don't want to hear about the presidential election right now. I didn't come here for that.
So I assume the downvotes are because you're way off topic and because your comments are barely coherent.
You think people here are "triggered" or something... You're the only one making a big deal out of a single off topic comment. We can still see your comments. Moving you to the bottom of a thread isn't quite censorship.
Do what you want. Just don't act so surprised when people don't take you seriously.
You have to pick the time and place for this kind of discussion man. As much as I don't want SJW political rants to seep into technical discussions, so too many people will not have any interest in what you are saying because you're hijacking the topic at hand.
You can understand this point of view right?
Complaining about down votes, as well as breaking all the other guidelines that you are.. is guaranteeing you will get more down votes, regardless of people's opinion of what you're saying. If you want to persuade, you need to change your strategy.
That is so unfathomable a finish to so bizarre a string of comments that I'm going to put it down to you going on tilt (it happens) and simply ask you not to do it again.
But the uploaded command to shed rotational velocity was wrong. This is what caused the loss of the spacecraft. I'm sure there will be a pretty heavy postmortem on how this happened.