"He has changed the way engineering teams are structured, largely eliminating testers to speed software releases,... "
What could possibly go wrong?
Ok.. the sentence continues :
"and adding data scientists and designers to the engineering teams to ensure all features are informed by rigorous testing and good design principles. "
I don't quite understand, how can data scientists help to catch bugs? Automated testing?
This is probably why so many of the Windows Updates this year broke critical things, especially for enterprise. I feel Nadella is taking a "cheaper, faster" approach which is fine and dandy in the otherwise non-critical mobile space (mobile users are used to being treated like crap and having buggy releases) but in enterprise there's simply a higher standard of quality.
I'm not sure what this means for MS. At our shop we now have to overly-test all patches and then wait a minimum of 30 days to see what happens. This has saved our bacon several times under Nadella's MS when before we only had to do pretty casual testing of patches. This also means a lot of shops are no longer rushing to get security patches in because of his negligence and corner cutting. That means a less secure internet for all.
I really wish someone would step in and show Nadella that MS shouldnt just be an Apple-lite and that catering to enterprise should be MS's main goal, especially considering that's where almost all its revenue comes from. I'm sure playing Steve Jobs-lite is fun for Nadella, but its about time he grew up and started running with the big dogs. Frankly, I'm sick of the focus on mobile and other popular technology taking away from the core competencies that made some of these companies great. The recent actions and changes on OSX are disheartening as well. Or how the steam behind the Chromebook is more or less dead as Android eats the world.
Buying and updating an MS product shouldn't be a gamble. They're supposed to be the conservative and stable big brother to the industry. Ironically, its only now under new leadership that we have so much more incentive to move to FOSS products. Its the same amount of headaches, except with FOSS I have no licensing worries. Under Gates and Balmer, the commercial MS products were less headaches and delivered a fair amount of value. Now just installing an Exchange rollup is asking for a complete reimaging. We don't even bother with service packs anymore. Might as well roll out a new image with that cooked in. The fail rate on those is unacceptably high.
edit: why the downvotes? this echoes the sentiment at places like stackexchange and /r/sysadmin. testing at MS has gone to hell and everyone knows it. burying my comment does nothing to stop that reality. Its well known Nadella personally redid the QA and testing parts of MS and those outcomes have only gotten worse.
>Buying and updating an MS product shouldn't be a gamble. They're supposed to be the conservative and stable big brother to the industry
You can tell who's young around here. In the distant past you could have put IBM or Novel in that same blank 20 years ago. It wasn't till around the release of server 2003 that anybody thought that.
Next, every product is a gamble these days. If you're connected to the net, you're at risk. Before exploits may have taken weeks or months to fully circulate, now 0-days are very common and other vendors release the exploit information they have on your products in 90 days or less.
>Now just installing an Exchange rollup is asking for a complete reimaging
Again, I'm not sure when Exchange didn't explode in the past either.
My personal experience is that MS has been solid since Windows 2000. I have limited experience with the NT4 products but I remember those being just fine as well (i maintained two NT4 servers for my employer and used to host images for fark photoshop contests on one- shh).
Exchange has been SUPER solid. Its only 2010/2013 that we're seeing so many issues with what should be a trivial update process.
I think there's a real drop in quality lately. Heck, in the XP days we never bothered to test security updates. They all came in same day via WSUS. Nowadays? No way.
NT was even more solid before then, at least after the first 3.1 version which I don't think I did more than kick the tires of. The drop in quality really started after SP1 of NT 3.51, the 3rd major version.
Didn't Exchange have problems in the early part of this century, or earlier, with its database being easy to bork and slow to restore?
Your comment harmonizes with the word on the 'street' from in-person conversations from MS employees & people who know them. Testing as a role evidently has been disintegrated and QA turned into SDEs in general. I believe the goal is to force quality onto the developers responsibility instead of making it QA's responsibility.
We'll see how that plays out; it sounds like the first iteration has been brutal.
Disclaimer: MS employee/don't speak for the company etc; Yes, SDET has been (at least in my division and those I talk with) dissolved and integrated into SDE.
I raise an eye at the parent-parent post not because of the impact this will have (certainly some of the concerns stated here echo strongly with me) but that it's been a relatively recent change and I'd be skeptical that you can attribute any recent failures to this specifically, things just don't move that fast.
As to seeing how it pans out, it's certainly up in the air. As a former SDET there's a big wall to climb of making testability performance security etc. part of the first line of thinking alongside design and implementation.
I went through this in 2006 or so - SDETs were turned into SDEs overnight in my part of MSN. Some survived, I didn't - one thing it did was completely discount the skills of a tester. The other SDEs were not expected to be up-to-skill on testing, but we were stacked against all the other SDEs.
Those are good points worth reading. What you describe is very much the feared outcome of removing SDETs before SDEs are ready to deliver tested and solid products in the new model. I have personally not suffered any troubles, but your experience does not surprise me one bit.
I think the problem for a business like MS is that, like you said, most of their revenue comes from enterprise. They already dominate that market. It's not enough to tread water - they have to grow! How can they really grow in a market which is definitely not chomping at the bit for change and are slow to adopt new products (even windows 7)? Where else can they grow? Apple is making so much goddamn money right now it's crazy - MS has to be thinking of ways to get a piece of that if they want to grow.
I guess for example that the http://support.microsoft.com/kb/3004394 Win7 fiasco comes from the update being only tested on machines with test roots installed, right? And Win8.0/Server2012 seems to be treated as second class with more than one bug affecting only that one, which reminds me of the odd support lifecycle where the server edition was treated as an older release but the client edition get treated as a older service pack.
It's a confused sentence, but I think it means he removed "testing" as a separate group of people, and made "testing" something inherent to the design process.
In other words, Microsoft is doing what Google and other companies do: have individual teams responsible for everything for their project(s): design, development, devops, testing. One team has responsibilities for a vertical slice of a system: build it, deploy it, be responsible for it.
Agree totally, poor writing. It seems that Microsoft, under Mr Nadella's stewardship, are moving towards a more iterative design approach, which seems to be working well.
They are not eliminating testing. This was really bad wording. When I interviewed in Redmond with Microsoft, I was told that Microsoft used to hire SDE's (software development engineers) and SDET's (software development engineers in testing) as two distinct positions. Now, if you are hired as a "software engineer", you are responsible for writing software as well as writing tests. This was done to decrease friction between what used to be the "software development team" and the "QA testing team", and hopefully lead to faster software releases.
A developer can test their own code, but they will only test it through the paths that they are aware of and thus coded for. An independent tester needs to be there to explore different avenues and try things that a developer wouldn't have thought of. Developers do think very differently to users.
But if you're working on software that actually matters, you're supposed to have some friction between the people building it and the people testing it. This is otherwise known by terms such as "independent testing" and "lack of bias".
Moving fast and breaking stuff is OK if you're building a web service that doesn't really matter. If Facebook falls over, the productivity at a significant number of businesses probably goes up.
On the other hand, if yesterday's OS security patch stops everyone's PC from booting this morning, or if a new version of your e-mail server has a glitch that means half your company can't log in to read their mail or even that important messages from customers are actually being lost, that probably comes with a cost measured in large numbers of dollars per minute at a medium-sized business, or if you prefer, in careers per hour within the IT group.
there is a separate, smaller "quality" team with a separate reporting hierarchy that is supposed to address this - that's where the reference to data scientists, etc. in the article comes from.
basically the idea is to split the SDET role into two - SE testing (closer to the code than SDETs used to be) and "quality" (further from the code than SDETs used to be). SDET testing, though not directly driven by the code structure, tended to be at a pretty low level driven by a detailed functional spec.
Software engineers should be able to test their own code. The faster they get feedback on their code and its impact to users, the better they understand the various cases and can strive for better quality. Having a separate QA team has a fairly long feedback cycle.
Ideally you'd get both engineers writing tests for their code as well QA teams for products.
Software engineers should be able to test their own code.
Indeed, but there is also a good reason that professional editors have new writing proofread or even technically reviewed by people other than the original author.
Maybe there's some subtlety that I'm not picking up here about Microsoft's previous set-up and/or the model they've been moving to under Nadella's leadership. We seem to have some past and present Microsoft developers around these parts, so perhaps their comments will clarify this. But as reported, this looks to me like a retrograde step, an attempt to be more agile and fast-moving rather than something likely to improve the quality of the finished product.
I had the same reaction. I always thought the saying meant low hanging fruit is always the easiest to get to since its. . you know. . hanging lower than the rest of the fruit on the tree.
Nadella has surpassed expectations to date, he has a plan, and is implementing it in a rigorous and competent manner. I expect the next year will be a good one for the "ruffians from Redmond." And I just can't stress enough how important it was for Balmer to yield when he did. Steeped in the past and his old ways, he wasn't the new leadership the firm required. Nadella is just that.
I feel there is good and bad. I am glad he moved to a more data driven and agile culture. He is also empowering engineers in his organization. The bad is that they all suffer from the windows mindset and spend a great deal of time building for windows. With all the patents they own they literally own android and yet they are not willing to allow android apps on windows phone or surface pro 3. It feels like a slow march towards irrelevance.
It wouldn't have been so monumental if instead of locking everything into Direct3D they'd used OpenGL instead. Here their own lock-in mentality will eventually bite them hard. If MS will outgrow their dinosaur lock-in approach, they can start supporting GL-Next on all of their platforms when it will come out.
When all you have is a hammer, everything starts to look like a nail.
Windows runs OpenGL just fine (look at anything iD Software puts out). It doesn't run Google's custom Java VM, it doesn't offer AOSP APIs and it doesn't offer Google Play Services. THOSE are the obstacles to running Android apps on Windows, not your obsession with OpenGL vs DirectX.
Is there OpenGL on their mobile platforms? It can run just fine in theory there, but it's not available. Applications are not limited to Java, and often rely on the graphics system a lot and there can be as well native code.
Mobile games do. And if you want to translate it into a broader context, what about having OpenGL on Xbox? Didn't MS want to unify their OSes across different devices? So it's really all related.
This wasn't about persuading, but about them not being jerks and making life easier for those who develop cross platform applications like games. I'm sure many developers would appreciate using portable API.
Perhaps Microsoft is planning on dumping the Windows phone in favor of Android and iOS? The Office365 apps run well on my Android Note 4 phone and my iPad mini (and my MacBook Air).
I like Microsoft's new strategy, but I wonder about the financial aspects of it. Microsoft gets $100/year from my family, that is it, and everyone gets 1 terrabyte of OneDrive storage and all of the apps. Is this sustainable for a business? I don't know.
I bought a tiny Windows 8.1 laptop a few weeks ago mostly out of curiousity (HP Stream 11, direct from Microsoft so no crap-ware installed, price: $199). Except for it being slow running IntelliJ for Clojure/Java dev, it is such a cool little laptop. Windows is free on it because it is a low cost device. Considering that this laptop uses OneBox configured for few files being on the local drive, and most used dynamically from Microsoft's servers, how much money could they make selling me this device with the cost of running OneDrive? Not much.
The Windows license is absolutely free on devices with an MSRP of $200 or less, which means they make nothing on it except maybe any apps you purchase.
This is tricky. From a developer standpoint, if Android apps were to become first-class citizens on WP, it would make no sense to bother making WP-only apps.
The Mojang acquisition was likely a very smart move. I remember the day it happened was the same day that Apple announced the Apple Watch. While I was aware of the Watch, my kids were completely unaware of it, but both they and their friends were very aware that Microsoft had purchased Mojang. They were very concerned about it and it was the topic of much conversation that night.
If MS is able to utilize the popularity of Minecraft in the <18 set to their advantage...
What could possibly go wrong?
Ok.. the sentence continues :
"and adding data scientists and designers to the engineering teams to ensure all features are informed by rigorous testing and good design principles. "
I don't quite understand, how can data scientists help to catch bugs? Automated testing?