Hackers gonna hack. Every generation of these consoles is a new challenge to all the hackers out there. You've got to think that the engineers who designed these systems eventually get "engineering syndrome" from staring at these systems for years and eventually start to be blind about tiny problems. Corporations put millions and millions of dollars behind the design of these systems and a ragtag group of folks comes in here and owns everything you worked to protect. I guess the only thing these hackers have over Sony is time. More time than money..and a fresh pair of eyes.
Is there a future with an open gaming platform like the Steam Machine? Why do developers choose a closed platform like PS4 vs the Steam Machine? What's there to protect that they work so hard to guard it?
Something to keep in mind is that it's becoming a trend that companies sell the systems at a loss (Notably, I think the only company that doesn't is Nintendo). Especially for the beginning of it's life, this is only practical if it's a closed system - Because if it wasn't, then they're essentially just offering the hardware at a huge discount to people who have no intention of buying any games, and the games is where they're going to make their revenue.
I don't know if Sony every actually said anything official, but I remember that being part of the reason why the 'Other OS' feature was removed from the PS3 - Places/People were buying them up and running Linux on them because they were such a good deal for the hardware.
I wouldn't want to bet money on my predictions, but I would say that if someone does build a platform like the PS4 that's open and can run Steam, it would cost a lot more for the consumer then the PS4 since it would probably be priced closer to the actual value of the hardware - And that would put off a lot of potential customers.
That said, there's an obvious chicken-and-egg problem in getting people to buy the system, and getting developers to develop for it, but having a cheaper system is definitely an advantage (Obviously, already being well-known to both developers and consumers is probably even more important). Regardless, if they had set the price of the PS4 at $1,000, it probably wouldn't be doing so hot right now.
but I remember that being part of the reason why the
'Other OS' feature was removed from the PS3 -
Places/People were buying them up and running Linux on
them because they were such a good deal for the hardware.
Wait, really? The PS3 was outclassed by generic PC hardware pretty quickly, and wasn't much faster than a netbook unless you were writing some seriously multithreaded or PS3 GPU-specific code. I never heard of many people buying them to run Linux and I don't remember any prominent open-source projects that targeted the PS3.
The real danger (for Sony) in opening up their consoles is the risk of piracy. That's a huge attack vector for hackers looking to defeat a console's copy protection scheme(s).
Sony's biggest nightmare is a world where teenagers can easily copy games because, as you say, that's where the real money is.
Looking into it, it seems it was more specialized uses that people were using them for, not just generic hardware - I remembered that point wrong. The fact that they foot the bill for a lot of the hardware is still true though.
Being able to easily copy and run games is an entirely fair point, but I felt it was separate from what I (And who I responded to) was getting at. An open-hardware machine running steam still has DRM on the games - And it's a bit of a given that most devs probably wouldn't want their games on platforms that didn't at-least try to have this (Though there are notable exceptions). Having open hardware, and having open software are really two different things, and it's a distinction that's only really come apparent in the later gen systems.
In the past, game systems were fairly close to being basically embedded systems, with the games running on the bare metal, so generally speaking having open-hardware in such a situation makes piracy much easier, because the only protections are in the hardware itself. Current gen systems actually have full OS's and a kernel-mode user-mode split, like a regular PC. Even if the hardware was open, without access to kernel-mode while the OS is running piracy isn't possible. I will concede that opening the hardware possibly creates an attack vector for the kernel though, so from that perspective it really isn't something that they want to have to deal with - And they don't gain anything from allowing it.
That said, if you look at Steam, AFAIK they have no real big issues with piracy. Why that's the case is debatable, but you can obviously make it work without having to lock-down the hardware.
It's not even that far in the past that these games were running on bare metal. The Nintendo Wii didn't have an OS running underneath the games, each piece of software had access to the entire range of memory and CPU unrestricted. I'm pretty sure the 3DS has a very minimal OS running underneath software.
Entirly true, it's kinda cool to see the evolution - Though it makes sense the games would run bare metal, since that would give the best bang-for-your-buck, and presumably it didn't matter at the time.
The 3DS actually has a somewhat complicated architecture - My understanding is that it has a dual-core CPU, but one core is dedicated to running the OS kernel, and the other is used for games. Thus, there is also a 'kernel-mode' 'user-mode' split in a way, because the core dedicated to running games doesn't have full access to everything, and doesn't have full access to mess with the other core. You can kinda see this in action when you note that the 3DS can only ever have one game running at a time, but certain home-screen applications can be run while a regular game is suspended - Due to the application running off of the OS kernel's core and not the game's core.
This also means that an exploit in a game doesn't automatically result in full-system control - A separate exploit in the kernel is necessary to gain full control over the system. This is completely different from the Wii's setup as you noted - I'm not sure if the Wii had the OS running while games did or not, but regardless since games ran with full privileges a single exploit in a game resulted in full-system control.
> The PS3 was outclassed by generic PC hardware pretty quickly
I agree here. Running Yellow Dog Linux [0] on my PS3 was a fun weekend project, but it never came close to replacing my dedicated computer. I didn't even notice the update that removed it, which in hindsight could have been pretty bad had I used it more.
People were not running Linux on PS3 for desktop/server usage. They were running it there for scientific computations. This was before GPGPU (cuda/opencl) became better price/performance choice.
Interesting. I found more about this on Wikipedia [0]. As a tech hobbyist with a PS3 in my teens, I was genuinely interested in a desktop replacement. I shared this interest with others in my cohort at the time.
I would argue that the "loss leader" strategy has been part of the game for both Sony and MSFT since they have entered the video game console, forcing SEGA out of the hardware market.
> Why do developers choose a closed platform like PS4 vs the Steam Machine?
Several reasons, in order of significance:
* Console manufacturers will often subsidize your development for platform exclusivity, in order to attract users.
* Large userbase.
* Single hardware target. While consoles (intentionally) have no software portability, you are also targeting a single set of hardware so the QA is easier and the dev process requires less testing.
So basically, in reverse order, if you want Steam Machines to win:
* You need APIs that are perfectly portable, so your SDL + Vulkan game can run everywhere without having to manually fix everything.
* You need users buying your platform.
* You need current manufacturers to lose their profit margins enough to stop buying off exclusivity deals.
Note that with consoles, exclusives attract users which attract exclusives. Its a positive feedback loop that, at least in this console generation when you can build a PC of comparable power to a PS4 / Xbone for the same price, there is no barrier to entry cost on the user besides the preassembly costs, and the fact Steam Machine vendors are selling at profit where Sony / MS care less about margins because they make per-sale revenue in the same way Steam does. But Valve isn't making the consoles, so Alienware needs to keep the lights on exclusively through sales.
That means there is both, A. no incentive for platform exclusivity on Steam Machines, because you aren't getting lockin to your Steam Machine, you would buy exclusivity to the ecosystem and B. no Steam Machine competitive with PS4 / Xbone in price / performance unless you are buying a lot of games to make up the gap between console used prices vs Steam sales.
> Is there a future with an open gaming platform like the Steam Machine?
Yes, but not without Valve biting the bullet to force enough user adoption to get the feedback loop going. They launched it last month without any compelling reason to buy one, and thus sales were underwhelming. And the only way to make them a compelling buy requires Valve to spend money on it - either through subsidizing sales with huge amounts of free games, or directly by making Steam Machines more cost effective up front than current consoles. And then on top of that, you need a critical mass of users to get developers releasing all cross platform releases for it, while also having the games to attract users. Usually, you get that through exclusives, but as I said, nobody has a motivation to release exclusives on SteamOS except Valve, and that would only be if they value adoption of their platform over raw sales figures, and I bet they don't.
First of all, the entire question is a strawman, why do developers choose a closed platform? Who said they "choose"? Because, the way I see it, management says "we're gonna do some nonsense decision, deal with it". Developers don't choose, otherwise, consoles would not exist by now.
"you are also targeting a single set of hardware so the QA is easier and the dev process requires less testing."
Are you serious? You mean to convince me every developer wants to take yet another hipster sdk, one for xbox one, one for ps4, one for xbox 360 and develop for each and every one because it is easiest? Instead of just developing for single linux desktop and reaching linux pc market + consoles and optionally cross compiling to windows from linux too? What kind of crack are you on mate?
Steam machine is better in every way than a gaybox and homostation, everything you get from a pc - free internet, cheap games, same online multiplayer for console or a pc, way beefier hardware than a console, even upgrade options, yet people say that steam machine is lacking something?
> What's there to protect that they work so hard to guard it?
A talking point is that closed systems make piracy harder which makes for a more captive group of buyers. Again, just a talking point, we don't have enough comparable data to study.
I don't have any idea what I'm talking about, but it is mostly PC-like architecture with mostly FreeBSD running on it, right? Nothing really interesting, unlike it was with Cell processor.
They are not even subsidizing the hardware anymore, as I understand it, they sell it for about the same it costs to produce.
It's more impressive as a demonstration of owning the platform than getting Linux to run on something odd/unusual. The PS4 was only recently jailbroken.
Looking at some component prices, you could probably assemble a passable gaming PC for under $350, so at $300 for a PS4 it does seem likely that Sony isn't losing money on console sales.
They using normal webkit exploit there. Reason why they start it via "userguide.html" it's because in any other case browser on PS4 can only be used when you logged in on PSN.
Of course PSN require latest firmware to work while they using device with Firmware 1.76. It's one of Sony measures to force people update firmware.
The emulator? Not particularly difficult, once you have generic drivers for audio and a framebuffer support. Is the same emulator you have on Linux/Windows/OSX PC's .
Just recompile and you are welcome.
The achievement I was referring to was putting Linux on the PS4 so quickly after the console was jail broken.
The emulator itself was a great visual demonstration of "the full linux desktop experience" as opposed to listing "Can emulate video games" as a bullet point.
The HDD is the only user-serviceable part in the PS4, perhaps USB was chosen over PCIe SATA to make theoretical DMA attacks on the system more difficult?
SATA doesn't give you direct access to the DMA engine, the host decides where the data from the drive goes. If anything USB is probably more exploitable.
Well, the SATA controller has DMA access with the host. Of course, you have to find some way to trick the controller into misbehaving, but it's possible.
Maybe because they support attached storage through USB and could reduce complexity by only sourcing, manufactoring, supporting and making solid one bus/controller? What would they possibly gain by supporting ATA devices?
The video suggests the Blu-Ray drive is attached to an AHCI SATA controller. So maybe there was some other legal or technical reason they needed to isolate it from the internal storage.
It is, however, much slower than SATA II, which makes it a very poor choice when considering the volume of data games load off the disk when loading (this probably explains Bloodborne's lackluster loading times).
Would it be possible to achieve the opposite in some way?, I mean, dump the kernel and build it on a pc on top of a compatible hardware?
Let me know what you think
As consoles get more and more powerful and introduce abstractions between the game developer and the hardware (kernels, syscalls, platform runtime libraries), I've figured it would only be a matter of time before we stop trying to emulate the way that console works, and instead begin to virtualize the console by writing API-compatible wrapper libraries.
Presuming developers just treat the PS4 as somewhat like we treat PCs: a generic processor that can do "math stuff", and some black-box libraries for {threading, graphics, audio, HID support, networking, ...}, then it becomes far easier to statically recompile a PS4 binary into a native PC binary: just recompile the "math stuff" for your target architecture, and then replace the linkages to libraries provided by the PS4 SDK, to libraries provided by your virtualization wrapper.
In other words, basically do the equivalent of what Emscripten does to C programs that expect to use OpenGL: compile the C to asm.js, and compile the calls OpenGL into calls to WebGL.
Man, this is the fourth time today that someone's suggested that maybe I should talk to a real game developer about something. I am a game developer. Just not a (modern) console game developer. But I did develop SNES games, way back when!
My point was not that this might be possible now; but that, as console CPU+GPU power approaches a certain threshold of "good enough; why would we need more?", console makers will eventually decide to spend some of the console's power not on fancier graphics, but on making development easier and more portable by introducing at least one full black-box abstraction layer above the hardware. When this happens, that layer can then be considered the "source ABI" for transpilation.
Now, I haven't played with the XBO or PS4 SDKs, but I have played with the Wii U SDK—and it's exactly what I'm talking about. There's no hardware to think about in the Cafe toolchain—no IO ports to peek and poke, no MSRs to read off. There's just library APIs. It's nearly as abstract as Apple's tvOS SDK.
There had been library API on PS1, PS2 had been an exception with the DMA chains and VU code exposed to developers but Xbox 1 and all later consoles come with libraries and no poking into registers business. So you can already try your idea by writing an Xbox 1 emulator (which is as much a PC as PS4 or Xbone). Spoiler alert: these libraries are statically linked on top of developers going around them all the time. Ultra High Level emulation seems to work only on Nintendo consoles, see https://en.wikipedia.org/wiki/UltraHLEhttps://en.wikipedia.org/wiki/Dolphin_%28emulator%29
> There's no hardware to think about in the Cafe toolchain—no IO ports to peek and poke, no MSRs to read off. There's just library APIs. It's nearly as abstract as Apple's tvOS SDK.
Sorry, I was going to compare-and-contrast with HLE; forgot. HLE is effectively a JIT, whereas I'm talking about something more like a static recompiler, like jamulator (http://andrewkelley.me/post/jamulator.html).
Also, the "libraries" in HLE, that the JITed code calls into, are wrappers built into the emulator software, and therefore can "think in terms of" the source console (see, for example, the texture upscaling in Dolphin)—whereas with transpilation, you have to make direct use of the transpilation-target's platform libraries as-is, with the only extra platform-specific logic existing as shim-code compiled into the transpiled binary at the call-site.
Besides Emscripten, the only other good comparison I can think of is ARC (https://developer.chrome.com/apps/getstarted_arc): a transpiler that takes Android Java bytecode (with Android SDK calls) and spits out PNaCl bytecode (with PPAPI calls.)
Wine is faster than Emscripten because asm.js is a slow target, not because HLE produces fundamentally better results than transpilation.
Wine has a variant, libwine, that can linked into a Windows binary at compile-time, replacing that Windows binary's linkages to Windows libraries with linkages to Wine. If you have the opportunity, this is always the more processor-efficient way to go (and I'm surprised so many companies choose to "port" their software to Linux using Cider rather than taking their source and compiling in libwine.)
If you didn't have the source of a Windows binary, you could create a Linux transpiler that replaces already-generated Windows symbols with Wine symbols statically while also potentially rewriting the ISA and calling conventions. The only reason one doesn't exist is that the overhead of running a very similar platform via HLE (i.e. one with the same ISA) is negligible. If OSX was still on PPC, for example, running Wine-on-PPC-OSX would work much better via transpilation than via HLE.
Speaking of PowerPC, a better candidate for "transpilation for performance purposes" would have been the PPC apps run under the Rosetta HLE emulator built into OSX. These apps could have been chewed-through once to produce a Carbon-linked x86 binary, that binary cached in the PPC binary's resource fork, and then executed immediately from then on. But Apple themselves had little reason to do this—they didn't want to give anyone an extra incentive to continue using PPC software instead of moving to x86/x64 software.
I wouldn't say that whether the shim libraries are built into the emulator or statically linked into the transformed output is very important. There are really two mostly orthogonal axes here:
- Instruction set: JIT (emulators, Rosetta) vs. static recompilation (only niche projects when the source is a machine ISA, but arguably Java AOT compilers, OdinMonkey, etc. count) vs. doing nothing because the source and target have the same ISA (Wine, VM software for the most part).
- APIs: HLE (emulate semantics) vs. LLE (emulate hardware). The more abstract the API, the better the former works, and vice versa. If you are not emulating things like interrupts and register pokes, that is HLE by definition.
In some cases you can /directly/ translate source API usage into target API usage without any shims, but only for relatively simple APIs. For anything complicated, the semantics are likely different enough that transforming calls to it would require complicated global analysis, which would be quite pointless from a performance perspective unless the API is called ridiculously often.
It's feasible, but to make firmware useful outside of device encryption keys must be extracted or firmware need to be heavily patched. Both tasks are complex and there not many people that going to work on it because groups like fail0verflow don't support piracy.
Also video does show privilege escalation that start via webkit exploit, but that doesn't mean that they worked towards breaking of content encryption and / or dumping games.
Since the PS4 OS is based on FreeBSD, you could conceivably match up parts of the compiled PS4 kernel with FreeBSD source code. However, I bet there's some sort of obfuscation that their compiler performs to make it difficult to decompile.
Apparently there are very little changes in the Sony PS4 kernel compared to FreeBSD 9. Sony changed some of the parameters to syscalls and added a bunch of their own syscalls.
I have no idea about this stuff, how far towards a jailbreak/softmod is this? Could this exploit be locked out on quasi-everyone's PS4 via a simple PSN Update?
That they have gotten this unsigned code running means they have jailbroken it. This WebKit exploit might be patched but there are almost certainly more holes to be exploited.
Yeah, but the APU drivers may not be comparable. That's one piece we won't know until we see actual benchmarks with it, if they manage to get 3D acceleration working.
It is playable in Wine, so running Linux on PS4 can enable tons of games both native and Wine compatible (the demo even shows an emulated game which isn't native for Linux either). No idea how good 3D acceleration there will be of course.
Yeah tons of games are running in WINE, but many of them have small bugs there and there (I use WINE all the time). Witcher 1 had some bugs related to the health bar not being visible at some point.
Also , I forgot to say once USB support is completted , a huge array of HW could be attached to the PS4, because these drivers on Linux are cross-platform.
Mesa doesn't work directly with hardware, it's depend on DRM and kernel driver interfaces. If they manage to get kernel driver to work then Mesa will just work. It's very unlikely that any userspace changes needed.
Is there a future with an open gaming platform like the Steam Machine? Why do developers choose a closed platform like PS4 vs the Steam Machine? What's there to protect that they work so hard to guard it?
Beautiful work here. Keep on hacking.