Hey I appreciate the general lack of cruelty so far in the comments, sometimes HN can be pretty rough.
The weird phrasing in the tweet is because I can't talk about any of the work I did due to NDA's besides my job description and those patents. But I had a pretty unusual role all things considered. I was also trying to clarify in the tweet that these aren't meant to be interpreted as unannounced existing features or capabilities of the product. But rather, research I did and contributed to, that was made public through patents.
There's a ton of interesting preexisting research literature about neurotechnology research for VR. Both in terms of using it for biofeedback, and using it as a device just for studying the brain. If that sounds interesting browse through what's out there via google scholar.
Also I said I spent 10% of my life working on it, realizing that was a goofy way of putting it. But it's a standout thing in my life and it was mostly a self reflection on what a trip being alive is, not trying to brag about it, mostly just sharing what I did. It's an unusual product and I had an unusual role
Anyway it's a cool device and hopefully people enjoy it.
I for one got the 10% phrase like you seem to have intended it, and it actually got me to think about how much time I spent in previous employments relative to the entire span of my life, which was an interesting venue to go down, so thank you! (Also for the insights on your work, that sounds like an immensely fascinating experience!)
> how much time I spent in previous employments relative to the entire span of my life, which was an interesting venue to go down
Same for me! Didn't even notice any weird phrasing (the tweet got immensely popular though, so nitpickers inevitably arrived). It was also nice to know people are working on getting non-invasive neurotech into the mainstream - very exciting
This is fascinating, thanks for sharing! Oddly enough I worked on something similar in grad school in 2008 (https://cowboyscott.gg/research/micro08.pdf). Our methods and models were quite primitive (though we did have a nice eye tracker) and the work had mixed success. It's great seeing related approaches, but with proper budgets and teams!
Love to start building apps for it like Apple Score (headset keeps score of your real life card game, ping pong match, tennis match, etc & displays it in your view).
How to break into it as I have many ideas Id love to see if they are any good. Are there bootcamps to quickly learn development and around other fellow other eager developers? Im a web designer who uses basic code (html/css/jQuery) to design.
This whole tweet had the same tone as when pop science crime investigation TV explains a criminal can be identified through its gait alone.
It's always magic, except when the predictive model doesn't work and there is no corrective action possible on the user side.
We're talking about physical reactions, so that means it's supposed to be the same through gender, age, race, health conditions from the algorithm's point of view...That's a lot of variables, and I wonder if Apple have analyzed this beyond the US population, assuming the device will be sold internationally at some point.
Apple seems really confident about this as they are foregoing controllers on the default interfaces, but that also reminds me on how they were rumored to make a car without a driving wheel...fingers crossed, I guess.
*) They are foregoing controllers on the default interfaces, for the presentation of this initial product which is expected not to ship for another 6 months.
It's what i would do as well if I would want to steer the developer ecosystem in a certain direction.
From what I heard it also supports Bluetooth Mice and Keyboards to productively use a Macbook (which is streaming its video into the glasses)
Yes, Apple is clearly steering developers towards voice input and simple click based interfaces.
On Mice/Keyboards, I hope this supports third party VR controllers, otherwise the gaming/fitness side is really toast. That feels like a given, but considering the time it took for iOS to get Nintendo controller support, it's not unprecedented, especially given their relation with Meta.
Technically, there's nothing that prevents them from running macOS in a VM to offer a virtual Mac. They built a lot of virtualisation tech into their M processors.
But I guess they won't, because (a) it's probably a bad idea to put unnecessary computational load on the headset that will use power and heat up your face, and (b) because they want to keep selling you Macs.
A lot of these large companies spend millions of dollars recruiting participants from various demographics to test prototypes and fine-tune technologies like this. It's a tremendous effort.
A lot of these large companies also, for some undisclosed reason or chain of events, design a phone so that a non-neglible amount of people can't hold it without dropping calls (iphone 4), and then blame the user for "holding it wrong".
Also such a thing as being too conservative with battery size (iphone 6?) so that when it unavoidably degrades, you must throttle down the phone to avoid killing the battery and perhaps more (literally by a current draw it can't handle since the internal resistance has increased).
And that's just apple, then there's unintended acceleration (toyota), exploding airbags, faulty ignition (ford was it?).
Don't think that just because they are large, or a company, or old, or whatever, they always have their shit together. Companies are people. People make mistakes. Mistakes can slip through the cracks.
I believe you, and it makes sense. Apart from that, though, it's hilarious to imagine an ad like "Wanted: Black women with long hair to quietly walk into a room, pick up a vase, drink a can of coke, and then leave through the same door."
Yes. Although it's damn hard to recruit a Spanish stay at home mom with a slightly twitching left eye who's too busy to participate to random studies and will receive this a gift for her 50th birthday.
It kinda makes sense that some of her traits would be covered inside the test pannel, but looking at the face unlock and the level of adjustments they're still doing, it doesn't look like they got nearly enough data before launch. And that's for a product that is selling in billions of units.
As usual, people will try to use workarounds. For missing ports on my Mac Book Pro, I use a hub. For connecting my AKG headset to my iPhone, I use a dongle. I guess sooner or later Apple will make available a kind of API for device makers to make it possible to interact with the content in ways different that eyeballs tracking.
Those aren't workarounds. The whole point of 3-4 USB-C/Thunderbolt ports is that you can do anything you can do things like plug a docking station into it via one port and do power, display, USB ports, etc off that one cable.
The whole point is having the option, yes. But it makes no sense in having only USB-C ports and nothing else. Apple seems to agree, they added a couple ports back in the M2 Macbooks.
Quality Thunderbolt 4 hubs cost about $150-500, which is fine if I need to extend the selection of ports at my desk, but there's no point in having everything on one USB-C port just because it's technically possible.
You're right. The same way face unlock doesn't work for some people and they inout a password every time, so it's not intractable.
I expect people for who eye tracking fails to be stuck on third party controllers for the default interface (hopefully they're supported outside of games ?).
The bummer though will be that this device is US only for its launch, so the first round of application might also not care at all about these kind of issues. Accessibility is something Apple usually cares about, but third party devs can be more casual about it.
Well Apple is only expecting less than 1M units to be sold. Must such a niche product be designed for every variation of human? Seems reasonable that if you fall outside of the most common human forms by enough standard deviations then you're just out of luck.
I think people can understand if you tell them their ear is a bit odd shaped, or their makeup throws the dectection off. But it will be weird to explain them their eye movement is really out of norm, especially after they paid 3500.
"All of these details are publicly available in patents, and were carefully written to not leak anything."
This part makes me sad about patents and original idea behind them. This actually is the opposite of original idea why people came up with the idea of patenting something. To spread the knowledge and to award inventors. Not to stifle the competition.
Actually his statement is entirely within the spirit of patents: the secret sauce is disclosed in exchange for some limited monopoly; since it's been disclosed people can talk about it and learn from it.
(None of this comment should be interpreted as a defense of the travesty that the patent system has become)
But can they, really? The language of patents is an obfuscated legalese, further confused by many variants of the claims that may not make any sense. They're far far from anything educational.
I’m gonna avoid opining on the value of the patent system in general. But one way of reading this statement is that the patents were written to avoid leaking anything about what the technology was going to be used for.
It’s entirely possible that the patents created for the Vision Pro are good enough to recreate many of the individual components that have been patented, but don’t help you assemble the entire product. Which I would argue is entirely within the sprit of the patent system. Just because you want a patent an idea a receive a monopoly on it reproduction doesn’t mean you need give away the details of how your entire product works. Indeed you may probably wish to keep as much stuff a possible hidden a pure trade secrets, if you’re confident that they can’t be easily reverse engineered or replicated.
But none of that diminishes the value of the knowledge shared in a patent. Nor does it inherently mean that the patents are written in a way to be entirely useless, they’re just extremely specific and don’t include broader context that isn’t needed to reproduce the specific thing being patented.
For example you could patent “estimating human emotional intent from body worn bio-sensors”, without once mentioning its intended for use in a VR headset. Just because you intentionally avoid mentioning a VR headset doesn’t make the information in the patent useless.
>Which I would argue is entirely within the sprit of the patent system.
I don't think so. The spirit of the patent system is you submit your product to them in the form of a working model. You get to patent that device. Tearing it apart into individual inventions to me is against the spirit.
Of course none of this matters anymore, we are far down the rabbit hole.
> I don't think so. The spirit of the patent system is you submit your product to them in the form of a working model.
On what basis? The patent system protects inventions not products. Not all inventions are sellable products in themselves, and not all products are inventions.
How would you patent anything meaningful if you can only patent an entire product? It’s would be trivial to around just patents, just make a minor insignificant change to the design of the product, and bam, you’ve got something brand new and unprotected. Thus rendering the entire reason for getting a patent meaningless.
If not for patent protection they'd be trade secrets inside Apple and if anything they'd talk about them less.
Whether or not those trade secrets were likely to leak before or after the expiration of the patent... well... in theory the patents mean you get the info out there even for the failed companies where leaks wouldn't be interesting. In practice you've got patent trolls.
I think it would make sense to shorten the patent length for certain things, based on how fast the state of the art in that industry is moving, but it would be hard to make everyone happy with whoever was in charge of deciding those terms.
Without patents: You are not provided any information, useful or otherwise. You can obtain this information any way you please, and use it. E.g.: reverse engineering.
With patents: You are provided with useless information. Information that is literally deliberately designed not to be useful. You can't practically speak use it. If you do obtain the information in any other way, you'll be sued into the ground. You'll be sued even if you've never seen the patent. You'll be sued even if your own invention is only vaguely similar. You'll just be sued, let's leave it at that.
don't forget willful damages. a long time ago I used to read the patents to understand the landscape and make sure my novel work was not infringing.
now we all have to _never even look_, because that might support someone's claim that you were aware of their patent and chose to violate it anyways - triple damages
I was thinking the same. The Apple I know, starting with the orientation, puts incredible amount of effort into telling its employees that people will be coming to you for consulting gigs, hoping to steal their intellectual property.
Sounds like he cleared the post with Apple, though.
The sad fact is not that the author cannot speak about the invention (because of the NDA), but that the patent was carefully written not to disclose anything substantial about the invention.
The point of patent protection is that you teach everyone how to build the thing that's being protected, in exchange for a monopoly.
Not actually teaching people how to build the thing that's being protected, but still having a monopoly on it is the worst of all worlds.
If you aren't going to teach us how to build something, don't expect patent protection for it. I'd rather you keep it as a trade secret, good, go do it, it's a free country, but then we can just reverse engineer it. If I don't have an NDA with you, I have no reason to keep your trade secret... secret.
Could the reason be that someone in China doesn’t have to abide by US patent law and will be free to reproduce the device, and sell it to the rest of the world? Are patents less useful once your competition is overseas?
I could be wrong, but I took that to mean the tweet thread was written not to leak anything that wasn’t already publicly available in the patents. Not that the patents were written to not disclose anything.
In which case patent regime is working exactly as intended and promoting dissemination of ideas while still providing the (time limited) economic advantages to the inventor.
You’re wrong about what the author meant, but you’re right about patents. Patents are indeed written very carefully to meet the validity requirements but not “leak anything”, because disclosed but unclaimed material is a dedication to the public. Here’s a recent article aimed at patent practitioners about the topic: https://www.finnegan.com/en/insights/blogs/prosecution-first...
I'm not trying to be obtuse at all, this is grade school level sentence structure break down.
The details are contained in the patents. The tweet explains some of the high level concepts (but not the details - after all, it's a tweet). Neither - honestly - are written to leak anything (like trade secrets).
The tweet goes on, saying if you want more information to hire them. It's a pitch for their consultation services, since they believe this tech will mature in the coming decade and, don't ya wanna get on the base floor?
You are asserting, groundlessly and with the intent to impugn the author and Apple, that they filed patents in a way which elided useful information. You are assigning a nefarious motive which is simply not justifiable. Assume good faith.
By definition, patents don’t include trade secrets. They’re patents. If you publish something in a patent, it ceases to be a trade secret.
And of course patents are written carefully to choose what to disclose. And the things that they don’t disclose remain trade secrets.
All the author is trying to do in their statement is to make it clear that the only details they plan to include in their tweet are those which have already been disclosed via patents, and therefore none of the content could possibly be an Apple trade secret.
That’s not because they are carefully trying to conceal anything from you.
It’s because they are carefully trying not to get sued by Apple for disclosing trade secrets.
well arguably here they are doing that -- w/o patent protection his NDA would probably completely stop him from talking about any of this
(tho i personally still think the patent idea is probably misguided -- apple would develop this tech ANYWAY, and it will all become public knowledge once people play w/ the devices enough, so what are we really gaining here?)
I worked at a large company with lots of patents. It was said some things they kept as "trade secrets" as to not alert the world to how they were doing things. Mostly I understood to have to do with the processes they were using for Chip Fabrication. I left long ago and they sold they're chip business...
What he wrote was, "These details ... were carefully written to not leak anything."
This is standard CYA language from an employee who doesn't want to get in trouble for leaking anything not publicly known (e.g., something not in a patent document).
you have to wonder what the statute of limitations is on Apple's NDA blowdart. Surely 10 years after the work was done if its not being used people should be able to at least talk in broad strokes about their work without fear of repurcussion?
Oh so now the hot new thing is a computing device that actively tries to guess its user’s emotional state (using various methods, including introducing new stimuli) and then then CHANGES ITS BEHAVIOR based on the results!
The Free Software people were right - a device this intimate needs to have publicly available, verifiable source code.
> Oh so now the hot new thing is a computing device that actively tries to guess its user’s emotional state
It also induces different states by way of imperceptible stimuli and measured the response.
The tweet got increasingly dystopian the more I read; spine-chilling stuff. Had this been about the Quest, people in this thread would be reasonably apoplectic. Imagine what Saudi Arabia or a Gambling or News app could do with this.
Fuck it, I'm going Thiel, anyone want to cofound a security startup, HMU (law enforcement experience advantageous) - we'll make the Apple Vision the best interrogation tool on earth. Fifth amendment? Pssh, the suspect won't need to say a single thing.
>It also induces different states by way of imperceptible stimuli and measured the response
Are you sure? Am I missing that part? I see this:
>Other tricks to infer cognitive state involved quickly flashing visuals or sounds to a user in ways they may not perceive, and then measuring their reaction to it.
But that doesn't say they're inducing states but rather trying to figure out the state. Trying to understand if someone is mad does not seem quite as dystopian as trying to make someone mad.
If you have the capability to measure state, and you have control over audio / visual input, then you necessarily have the ability, albeit via a probably alarmingly small number of experiments, to induce (some value of) state.
Lacking an understanding of the mechanism, I'm not sure I understand how that necessarily follows regardless of the mechanism. Completely pulling something out of my ass, imagine that they determine inserting occassional white frames results in more significant pupil dilation increases when someone is in a heightened emotional state. Sounds doable and vaguely plausible? It would allow them to infer that heightened emotional state from a quickly flashed visual in a way most users likely wouldn't perceive from a measured reaction to it. But I'm missing how it would follow that you could use the same to cause that emotional state.
I dunno why people are surprised by this. We are long past the whole "autonomy of thought". Much of our decision on a day to day basis is already guided by culture society which is heavily influenced by companies.
For starters (apart from a very few outliers), if you own any Apple product, you have fallen into the trap of paying more for something that you believe is more valuable to you, but in reality, that feeling is artificially created. You aren't paying extra for features, you are paying extra because you believe that those features are worth that much, which is not your own thought.
I already hate software that tries to guess my intentions from key inputs, now we're going to have the same "smart" behaviour expanded to my brain? I hope this can be turned off.
> So, a user is in a mixed reality or virtual reality experience, and AI models are trying to predict if you are feeling curious, mind wandering, scared, paying attention, remembering a past experience, or some other cognitive state. And these may be inferred through measurements like eye tracking, electrical activity in the brain, heart beats and rhythms, muscle activity, blood density in the brain, blood pressure, skin conductance etc.
Otherwise, it could be interesting if as users we got access to all these measurements, for our personal use. That could open the door to many other applications that would be probably less creepy.
I know ; ) I wanted to show there are already alternatives for whose who are interested in experimenting with that data themselves (and at a fraction of the cost)
Why? The more personal a device is and the closer it becomes to interfacing with who we are, the more it will need to do things like this to work, else it will just feel off.
There's the good ol' bait-and-switch of tech industry you have to consider. New tech is promoted by emphasizing (and sometimes overstating) the humane aspects, the hypothetical applications for the benefit of the user and the society at large. In reality, these capabilities turn out to be mediocre, and those humane applications never manifest - there is no business in them. We always end up with a shitty version that's mostly useful for the most motivated players: the ones with money, that will use those tools to make even more money off the users.
It's true with all new tech these days, and I'd be doubly worried when it comes to things like emotion detection. There are no good uses with strong business cases, but plenty of nefarious ones.
Except that device is not owned by you, or even accountable to the public. It is owned by a private corporation whose mission is to do whatever it takes to keep you making purchases to make stock price go up.
I do not even carry a smartphone anymore and am happier for it.
You have emotions , the device does not. No matter how we pretend to ignore it, the ability to empathize and embody the emotions of others separates us from machines. Even if they emulate it, i know they are fake (yes, i am being a machine-racist here)
in 2013 Edward showed that data are used by governments. We do not see it right now, but it does not mean it does not happen. Actually I think we do not see the scale of it, because most of us are not doing anything incorrect. It is only important for our politicians. Everybody is under surveillance. It is just we do not know the effects of it.
Yes if it was a standalone device which no outside party had control over then sure. But that's not practically what any tech device has been in the last 10 years. Everything phones home. And with the amount of valuable personal data this thing could accumulate, you can be sure it will be phoning home about it all the time.
One thing that's notable about devices that "only run free software" is that they actually don't, since people use web browsers on them, and the websites are not free software.
You'd also have to never talk to another person with them, since other people are capable of manipulating your emotional state and can't be audited.
This seems like an oxymoron. I get that it's a "virtual" girl friend which semantically qualifies the term girl friend, yet there's something immensely sad about the combination of these terms.
VR sex bot seems more apt to me. It doesn't really matter and I don't mean to impose terminology on anyone; it's just an observation/reaction.
Sex bot implies sex in some form, that I don’t think is primary value of a such system even if it all available.
The ability to form a connection and understand your needs perfectly and respond to that a companion is the real value
Sex gets boring pretty fast . It is not that expensive to pay for in real life, a companion would be supportive, knowledgeable, empathetic and always available is very hard to pay and get or have otherwise .
I don’t know why people find it sad , not everyone is able to find the perfect (or any) human partner /companion , we should be happy if we are all are able to connect to sentience does it matter if it is a human , dog or a machine ?
I'm not opposed to it under all circumstances. What worries me is that people will do it under too many circumstances and fail to engage with the real thing. That seems like a tremendous loss of social opportunity which I'd argue is already occurring with technology today.
Regarding your first bullet, I am not sure if that's what people want. One of the best part of story telling like a TV show or movie is that everyone watched the same thing and then can refer to it later. How much more isolating does VR tech need to be?
> How much more isolating does VR tech need to be?
As isolating as it needs to be to extract max revenue. Your fine-tuned proclivities profile will be one more thing to lock you onto the platform, and an opportunity for product placement/ ads. Marketers would love to have their brand associated with happy thoughts (and happy thoughts only).
> Other tricks to infer cognitive state involved quickly flashing visuals or sounds to a user in ways they may not perceive, and then measuring their reaction to it.
I wonder how much they can plug into the new “autocorrect” system in iOS? Ok user heart rate is low and at baseline, they probably were trying to say “I could give a flying duck”…
Pupils dilated, heart rate racing, therefore “duck” may not be the right word..
I know we all hate each other but is this like a good direction for the future? Directing all our emotions to silicon chips? Maybe there is other things we should be building.
Emotion recognition is an old field but (apart from autism research) the only application i can think of is emotion manipulation. How does one wake up in the morning and decide to work on consumer emotion recognition
> How does one wake up in the morning and decide to work on consumer emotion recognition
I think two significant factors would be 1) they have the skills to make progress in the field and it's highly rewarded, and 2) they believe there could be real value in making progress. One or both of those could be enough to motivate a lot of people.
Which is especially fun if Seydor weren't coerced and does not truly help solve the rubber hose factor... Once rubber hose is in play this only means more suffering
>The other threads here seems to have missed just how "huge" a projects this is.
The other threads are busy jumping on the Hate Apple bandwagon, is what it is.
I have my own criticism of Apple's VR goggle, specifically that its form factor (albeit necessitated by today's technologies) drags the whole thing down in both the luxury/fashion and usage factors.
But if I put that aside, the technology and innovative thoughts crammed into that thing is potentially groundbreakingly amazing if the marketing is truthful and accurate. Even the $3500 price tag, which yes is bloody expensive, doesn't sound unreasonable given most other "cheaper" VR goggles still require a mothership costing about the same anyway.
I wish Apple success here, because the VR industry needs someone to come in and kickstart it back to something interesting and worthy of attention.
...I also admit I wouldn't mind seeing all the haters getting put out to dry for their noisy trouble, but maybe that's beside the point. :V
My theory (and many others') is that the high price tag is a feature and on purpose.
Enough companies have tried the cheapo approach and failed. Even mid-price ones are still niche devices.
Apple went full-on to Varjo[0]-levels of tech inside the headset. Now developers have 6-9 months to bring their A game.
Then Apple can see what kind of applications and use-cases stick and start figuring out what parts of the Vision Pro they can downgrade or leave out to make a Vision Regular, which will come out in 2-4 years along with Vision Pro 2.
Most of us Apple Haters are not opposed to this type of technology existing. We are opposed to technology with this much potential for abuse being solely controlled by a profit maximizing patent troll.
When the tech is open source and I can control what it does, then sign me up.
Apple has always maximised profits on selling hardware and getting a cut from software.
They've never gotten any profits from selling targeted user data. Mobile gaming marketing units are _still_ butt-hurt about Apple's randomised user/device IDs that prevent them from tracking where users came from and downloaded their game.
FB and Android tracking still works perfectly, but those users are less valuable because they don't spend money.
> I wish Apple success here, because the VR industry needs someone to come in and kickstart it back to something interesting and worthy of attention.
XR has the potential to be the next general computing platform after the PC and the Smartphone.
This might be controversial, but I believe what the VR industry would need most is for the giants to agree that collaboration for a wide open ecosystem yields the best result for everyone.
Unfortunately we may have another company here which is not interested in such a direction at all.
What we get instead is the biggest giant of them all forcing through it alone with yet-another closed ecosystem in expectation of taking full domination of the market and all its innovation.
But this time it's the player with the highest chances of success.
If you're a billion dollar company, you invest alot in multiple trajectories of the possible future, because you don't want to end up not being ready when the market shifts (like Nokia for example) and being forced to pay royalty to your competitors just to stay in the market.
How is one company able to hush all its employees and contractors on the details of such a monumental effort? It's like trying to build a pyramid in secret.
Respect, "all-in" mindset, realizing that blowing the lid gains you nothing but 5 minutes of fame, and then losing your lucrative job at the most valuable company in the world.
By separating Research & Development of the fundamentals from the Development of the product (on top of having NDA's of course).
People and Media are not so much interested in technology research, and without a product it's also often too abstract to understand. If a company makes no publications or speeches at conferences, they can often work for YEARS without any special measureas and not getting noticed...
The product itself they then likely developed with the same business-practices to secrecy as an iPhone.
It wasn't that well hushed, there were persistent rumours for years. After all it was the running joke that apple were going to launch a VR headset each WWDC.
Exactly. I know many people are upset that this will lead to patent soup resulting in less innovation or competition, which may very well be the case, but it shows also how much they haven’t just put high res glasses and a fast chip in a headset, but really tried to push the limits and leap past everyone with their first offering
This is a pretty shockingly bad statement. Not only is it a complete exaggeration, it’s not even a good exaggeration. Saying “I worked on mind-reading features” for Apple’s new VR headset is just creepy.
Public patents or not, can't imagine Apple is going to be thrilled with this guy for announcing to the world that their new product is secretly reading everyone's minds.
"Well three and a half years is a long time in my life, it's longer than I spent getting my bachelors, or the two masters degrees I did at the same time. It's the longest I've worked at any job, and longer than most relationships I've been in."
Dickens himself couldn't do a better character introduction than these two sentences.
Maybe not on a $3 million 5-bedroom mansion in the hills, but with $500,000 1 and 2-bedroom condos, 20% down payment is $100,000 k, which is affordable by a high level engineer or product designer making $500,000/yr, especially if it's a power DINK couple making close to $1mm/yr. Whether they want to invest in housing there is another story, but let's not pretend the upper class is priced out of Cupertino.
There are not many of those. (Not many condos in the US in general for that matter, since they're pretty discouraged by housing laws.) Wonder how many 10s-100s of bids there were for it.
Anyway, I'm not accusing Cupertino residents of being upper class, I'm accusing them of being old.
"Some of them are old" is a different statement than "None of them are young", but I see your point. It's true to the point that the local elementary school was or is closing down due to lack of enrollment.
I wonder if him not having worked on Apple Vision for the last >1.5 years means they distanced themselves from his "vision, strategy and direction" or even gave up on it.
Judging from his other tweets, he seems really into crypto and making art projects out of crypto tech.
I can't say I really understand the art - and I went to school with him, had the same professors, etc. I'm just scratching my head when I read his tweets.
But well: he's worth many millions and I still have housemates so I guess capitalism has crown him in the lead.
Also he keeps telling companies to reach out to him about this tech so he can make that sweet sweet consultant money, so I'm guessing he's still onboard.
It looks like this is mostly about patents. Apple has a history of doing lots of hardware and software research, not all of that gets to be part of an actual release.
Is anything indicating that solutions mentioned in the post would actually become part of Apple Vision?
Soo… is this real? Did those features really make it into the product as of now? I don’t recall any of these details being mentioned.
> measurements like eye tracking, electrical activity in the brain, heart beats and rhythms, muscle activity, blood density in the brain, blood pressure, skin conductance etc.
The headset does not appear to track most of these things.
Right... so this thing hooked up to a centralized AI, deciding what people see when, how often they see it, what they can express and how they feel... Do people really want to live in the matrix? I have a visceral reaction to this idea
"Where a user looks stays private while navigating Apple Vision Pro, and eye tracking information is not shared with Apple, third-party apps, or websites. Additionally, data from the camera and other sensors is processed at the system level, so individual apps do not need to see a user’s surroundings to enable spatial experiences."
Unless the code is fully free software that has been audited by enough people & there is a proof the binaries on the device correspond to this source code, then this means nothing.
Apple’s buzzword of the day yesterday was “on-device processing” and “on-device X” in general. They have invested heavily for years now into shipping ML chips in their devices and don’t seem to be stopping; if anything they’re increasing the use of it.
Is there anything definitively claiming this would be any different?
Sorry for hijacking this thread for a question about personal curiosity but does anyone know of an affordable way of measuring pupil distance?
I want to measure my pupil distance while browsing social media. The reasoning is my reading in "Thinking Fast and Slow" that pupils dilate when we see something we find interesting / something we like / when we are thinking and vice-versa. I want to put it to the test and it seems like finally the consumer tech is close to making it possible.
Does quest support pupil tracking? I did some cursory research but couldn't find any reference for it. Industry pupil tracking headsets are way too expensive; My last hope is that someone will jailbreak vision pro...
> Other tricks to infer cognitive state involved quickly flashing visuals or sounds to a user in ways they may not perceive, and then measuring their reaction to it.
Is this like how in the old days the movie theater flashed a single film frame of BUY POPCORN YUM.
So Apple can tell how you bio-physically respond to dynamic input. No worries though, we'll show a blend of an Apple logo and lock, which means all is fine. Go back to sleep, but we'll let you know if it was a proper sleep.
Apparently they've been working on their own standalone headset (codenamed Deckard), and judging from how Valve Index and Steam Deck were done I'd say there's some significant chance they may get it right.
Social stuff is best approached from a statistical confidence model that allows for "This might be the one in ten times my assumption is wrong."
I wonder if/how they incorporate that.
Because a lot of social drama comes from humans betting on the statistically likely thing in social situations and it being wrong rather than checking or hedging their bet. ("I shall assume the white male is in charge. Oops. Now I've offended the woman or person of color who is actually in charge.")
Demystifying this concept: The mechanism is almost certainly just measuring pupil dilation, and maybe eye focus, if that's something you can track with an eye facing camera.
Source: Thinking Fast and Slow, where the initial inspiration for the book was an experiment noting that human's pupils reflexively dilate when they engage their "system two," aka the "slow brain," metaphorical part of their mind.
This begs the question: what is Apple’s real play with Apple Vision? It sounds like there’s a real possibility that, even at a $3500 price point for hardware, the consumer is still the product. What else can their system determine about someone’s brain? What is going to be done with that data?
The eye tracking required to get that level of accuracy is really high. Normally you need a bunch of camera like sensors that operate different to rolling/global shutter, plus a hefty calibration routine to get a good map of the eyeball.
Even then it only really works looking dead ahead with your eye wide open.
so for the moment, that stuff is firmly in the lab.
So whats the real play?
to replace phones with glasses that know you better than you know your self.
To bake all that you see and do into <300 megs a day, and make it searchable in a way thats useful to you (not as in where was I at 22:04 on the 4th) as in "when did I last see x, what colour y did they say they liked?" but without having to ask that question.
Imagine a really really good personal assistant, that could answer 99.5% of the boring questions asked you with authority. That is the level of AI they (and google and facebook) want
Not hard to imagine how tools to predict and manipulate your emotions might be useful in a device that even in the wildest dreams of its developers is mainly designed to provide an immersive environment for taking Teams calls and working on spreadsheets.
I’m currently undergoing a course of neurofeedback therapy… the biofeedback tech in the Apple Vision sounds like it could have some pretty cool applications for such things.
this whole release feels extraordinarily 'Apple' , to me.
Apple takes an idea and staples as many 'side-grade' style functionalities to it as possible in order to sell it , rather than just offering a really well-performing product within the niche.
The iPhone was a remarkably bad phone -- it was more awkward to hold than any other phone at the time (now they're ALL awkward), and the reception was terrible. The provider it was locked to at the time was one of the worst in North America at launch; but the iPhone had features that people went nuts for which drove sales beyond the phone offering.
I feel like this VR headset will be similar.
It's a mobile device with a 2 hour battery life -- something that everyone in the world has been saying would make a device dead on arrival for the mobile market; but it has all of these little side-grade features that will attract purchasers that are interested in the Apple product itself but not necessarily VR.
As a consumer i'm glad -- I may have never been a big iPhone fan, but the Android reactionary effort produced so much value that I can hope that this field will see something similar once it proves the market feasibility. I can't wait for my 8 hour/8k res/quarter-priced white-label VR headset clone in a few years.
The weird phrasing in the tweet is because I can't talk about any of the work I did due to NDA's besides my job description and those patents. But I had a pretty unusual role all things considered. I was also trying to clarify in the tweet that these aren't meant to be interpreted as unannounced existing features or capabilities of the product. But rather, research I did and contributed to, that was made public through patents.
There's a ton of interesting preexisting research literature about neurotechnology research for VR. Both in terms of using it for biofeedback, and using it as a device just for studying the brain. If that sounds interesting browse through what's out there via google scholar.
Also I said I spent 10% of my life working on it, realizing that was a goofy way of putting it. But it's a standout thing in my life and it was mostly a self reflection on what a trip being alive is, not trying to brag about it, mostly just sharing what I did. It's an unusual product and I had an unusual role
Anyway it's a cool device and hopefully people enjoy it.