In general I agree that the tensor hardware has been mostly a disappointment. I mean this more in general performance than in ML.
In ML I think the story is a little different. Hardware design/production pipelines are measured in years. The LLM "revolution" is relatively recent. For the Pixel8, I imagine some folks at Google had to make the really tough decision of whether to run weaker, smaller models on device, or run the giant LLM models in the cloud, giving much better results, albeit slower (and with ofc the requirement for an Internet connection).
99% of smartphone consumers don't care about 'ML performance' as a primary concern. People care how fast apps open, and battery life. From the perspective, the tensor chips are pigs that nobody is excited about.
iPhones have all sorts of little niceties that are directly enabled by good ML performance. For example, automatic OCR of text within images, even on websites. I think the facial recognition for grouping photos by person is also done locally by the ML chip.
Automatic OCR works fine on my Android phone through a custom ROM with the Pixel software. No internet connection or ML hardware needed; OCR isn't all that demanding.
I'm not sure about the facial recognition features, I don't use that stuff but I imagine Android phones mostly do that in the cloud. If Apple's ML chip helps there then that's good for them, but I can't say I'd buy an iPhone for a gimmick like that.
> I'm not sure about the facial recognition features, I don't use that stuff but I imagine Android phones mostly do that in the cloud. If Apple's ML chip helps there then that's good for them, but I can't say I'd buy an iPhone for a gimmick like that.
I wouldn't call it a "gimmick", it's a feature that a lot of mainstream users depend on to find pictures of loved ones. My parents use it all the time to rummage through photos and find pictures of my niece, or an old picture of my dead grandparents.
"depend" is too strong a word, given that you can accomplish the same task (finding pictures) manually as well. Also given the typical accuracy of these systems, you'll find many more pictures if you do it manually. On some photos of my dad's 60th birthday, there are 7+ people in frame and the fancy AI recognizes absolutely nobody - including my dad having a conversation front and center, who was only slightly turning his head away from the camera.
Humans use a lot more than just facial recongition, meanwhile this AI stuff seems overly focused on it. I reckon most people could recognize Donald Trump or Angela Merkel even from behind, using clues from the surrounding, their dress, stature, and hair. I'd wager you can recognize your loved ones from any angle very reliably. Can your AI?
Honestly in my personal experience the best way to accomplish such a thing is still to order by date, because you, holding your camara/phone, move continously through time/space. Pictures of a certain person usually occur together and aren't randomly distributed. An exception is maybe your SO and others you may take a photo of on a whim.
I wish I had bookmarked this comment, but someone here was dismissive of the Apple Watch's heart tracker because, paraphrased, "I just need my fingers to take my own heart rate any time I choose."
What you're arguing is both true and irrelevant. We don't even need photographs, we could just sketch whatever is in front of us, but convenience matters.
Yes, you can find all prime numbers and digits of pi if you sit down and do the calculations on a piece of paper but you don’t do that because computers are faster than that.
For more complex tasks like locating and recognizing humans and their faces, humans can still perfectly do it but I still would rather be taking more pictures of my loved ones than rummaging through my photos every single time I’m looking for something.
As for the accuracy of those systems, it mostly depends on the model being deployed.
With virtually unlimited computational budget (cloud), it’s easy to deploy a very powerful model but you sacrifice privacy and performance in high latency environments.
With a limited computational budget (local neural processing unit) you deploy what you can cram into the chip. It’s less accurate but consistent and private.
> "depend" is too strong a word, given that you can accomplish the same task (finding pictures) manually as well
Ok, do this exercise right now (if you have anywhere between 1-10k pictures on your phone): find all the pictures from 3 family members of your choice.
Clock it, let me know how long it will take you to perform this task.
Now think about my mom in her 70s, do you think it's completely doable for her to find pictures of my dead grandmother this way?
> Honestly in my personal experience the best way to accomplish such a thing is still to order by date, because you, holding your camara/phone, move continously through time/space. Pictures of a certain person usually occur together and aren't randomly distributed. An exception is maybe your SO and others you may take a photo of on a whim.
How would ordering by date give my mother access to all pictures she might have from my grandmother?
Of course that AI will not find every single instance of a face at any angle, but it will be "good enough" to find a bunch of them pretty quickly when my mother is missing her mother and wants to see a nice picture.
That's the use case, it's not someone trying to organise their pictures with utmost precision, accuracy, and completeness, it's a common person trying to find some nice pictures of a loved one. If they don't have to spend 2 hours scrolling through an endless set of pictures it's already a pretty damn good feature.
You keep saying, "honestly" and "in my opinion" and yet you are dismissive. I can absolutely see how many people can depend on it. You seem to have no idea how useful this is for many many folks. I miss it dearly in Android.
They even recognize objects within pictures so you can search pictures via their contents. It seems to work offline, though it can take a while for the phone to run it.
This feature works surprisingly well- for example, at some point my IPhone recently gained the ability to recognise individual pets by name. This was super exciting for my kid!
I very much do care if my phone understands my questions accurately when I ask them, and doesn't return irrelevant results, though. Nobody says "yeah Siri is crap, but have you seen that power usage?!".
Honestly, I haven't met many people who speak to their phones. [edit: where I live. Can't speak for the north American/Anglophone market]
They usually use their phones to speak to other people, hence why signal reception and battery life is one of the most important things I look for.
All ML gimmicks are nice to show off at events, pub gatherings and in reviews videos("hey look at what my new phone can do, yeah it looks the same as the one from last year and the year before, but can your phone do THIS? <insert ML gimmick of swapping people's heads in the picture>"), but won't help me if the signal reception quality sucks and drains my battery.
Phones need to work good as phones first and foremost. High-end compute abilities come second.
My most used ML gimmick is ChatGPT which doesn't run on device anyway, nor is it something I need in case of emergency like you know... being able to call someone to come pick me up/help pe when I need it the most.
So IMHO bad modems like the ones in Pixel phones, are inexcusable in a phone no matter how fancy the other features are.
With all due respect, I think you're wildly out of touch with how most people use "phones" these days. Actual calls probably represent about 5% of my phone use. The term "phone" is essentially a historical misnomer at this point.
Maybe being in my mid 30's I'm too old to be up to date on this but what do people who aren't "out of touch" use their phones nowadays for? Some form of 'Apps' I'm guessing.
And for those apps to serve their purpose (excepting the likes of Candy Crush and Genshin Impact) what do those apps need? Internet connectivity right? Which is received via the same modem as the calls, right?
So a sucky modem will negatively impact battery life and app usage in challenging environments just as much or even more as call capabilities, right?
The confusion was probably just about saying it's a phone first. I'm in my mid 30s as well and the only time I call someone is when I contact my mother. I think with the younger generation it's at a point where calling someone is seen as intrusive.
If you are calling me you are taking my choice away on when to deal with whatever you've got. Or at least are interrupting me with the attempt itself. So it better be important.
So on point. And even when it actually phones 90% of the time I just dodge it. That 10% left is pagerduty. All real contacts use some form of a messaging app instead
Won't work where I live now in south-centeal Europe (Austria ) where everything is archaic and most services are still done via phone calls like in the past: appointments for plumbers, doctors, recruiter call-backs to job applications, etc.
Almost none of those services here use email or apps as the default, everyone seems to prefer sync communication so they first tries to reach you on phone. If you don't answer or call back they'll assume you're ghosting them and don't need the appointment/service anymore and you'll miss out on important issues. Sometimes they'll leave a voicemail telling you to call them back when you can, so you can never really escape calls here without emigrating.
You can say we both live in completely different bubbles but the existence of one does not invalidate the other.
I'm from Poland. It's just over the course of years most of the services became messenger contacts, for various reasons, but most common one was the need to reliably exchange pictures
Sure but when you're out and about and don't have WiFi you'll be relying on your modem to send/receive pictures and all kinds of data.
Probably not an issue if you're all the time in cities with great coverage but you'll feel it when you go in "the woods" or well shielded buildings or basements.
Love Austria btw., and appreciate the preciseness of your language. It's been years since I worked with it, but it always felt like a vacation for my brain, while consuming technical content in it
>Love Austria btw., and appreciate the preciseness of your language.
I'm not Austrian, I'm an immigrant here, and it's not "my language" nor anyone else's I presume, it's still just German, albeit with an Arnold Schwarzenegger accent on top.
I think people don't speak at their phones simply because the experience is so horrible.
I would LOVE to be able to voice control the various aspects of my phone but it's so mind numbingly frustrating that I'm annoyed just thinking about it. If it was better I would sacrifice other features for it.
Totally agree, the second I have to switch from a normal voice to a stupid voice with the correct nuances for the unit to "work" is usually the last time I use it.
Our google home basically sits there all day waiting for my wife to say "play abba", she has a strong non-English accent and it's about the only thing it seems to pick up. Don't get me on cars where I have to say the exact sentence to do anything.
Could be a generational and culture thing as well, being Latino in my 30's and living in southern Europe regular voice calls to family and friends really are not uncommon, but I can imagine it's not common among the North American/Anglophone demographic that's a majority on this site.
But texting is also a form of speaking to people although not using your mouth.
Voice calls, the worst part of a phone. Out of anything pub-sub it really has to be the worst medium in many aspects. Long gone are the polyphonic ringtones that tried to inspire answering, now I only see vibrating or blinking phones.
If I want to copy text from an app (often not allowed) I now take a screenshot and copy the ocr text from there. Same with badly coded websites etc. It saves a lot of time, and is always the same workflow.
Sure, but OCR doesn't need ML and high-end SoCs to work well.
We had great OCR in $50 printer-scanner combo devices for home users and those didn't have M1 chips but some dirt cheap HW and Windows software to do the OCR(most likely using the Tesseract library which runs on a toaster).
So the everyday need for ML in everything in our lives is greatly exaggerated.
It doesn't have to be Tesseract specifically, there are other OCR libraries that work better and are licensed by several products(I think Abby licenses theirs).
But that wasn't my main point. Similar to Tesla popularizing ML-driven self parking, which doesn't work any better than the PID/Kalman filter methods already in use by others for the past 10+ years, my point was that today we're seemingly shoehorning ML into solving problems that have already been solved by "boring" traditional engineering solutions for over 10 years now, just to feed the AI product hype, similar to the digital camera megapixel race, or other such nonsense that offered no measurable upgrade in quality but looked good on product stickers to move merchandise at higher prices.
That's a bit reductive. They also care if their apps work _well_ – if their pictures look nice, if their dictation is responsive, if their assistant understands queries, if OCR is accurate… which is all about "ML performance".
I do care about my phone doing OCR and other tasks locally without Google, and tensor chips are the reason this can be done without draining the battery too much.
You're right that 99% of smartphone consumers don't care about 'ML performance', but they do care about autocomplete being good and fast. Or photos doing object detection to create stickers for messages.
If you're able to offload these 'everyday' tasks to a lower power chip, you also extend battery life. Something they care about :)
Unless they can do something that was going to have to be run on the CPU - and use less power doing it. I think we're rapidly getting to the point where this AI stuff is no longer optional.
I totally imagine that they intended to run models locally when this phone was on a whiteboard in an engineering meeting but…
(1) the models team just can’t/wont trim the models, instead focusing on bigger and better. This is probably doubly true now that the AI wars have taken over their focus
And
(2) they realize they probably need to ship these features to every non pixel phone too.
... why do they need to ship these features to non pixel phones ? It's Google's special software sauce that make people buy pixels. Google doesn't ship advances Photos app tools on non pixel phones
Because they’re not Apple. It sounds nice to gate features to phones. But they’re trying to expand their subscription revenue to offset ads. It’s a much bigger market to target every android user.
Google ships the advanced photo products to non pixels after a delay, the pixels get a head start not exclusive access.
Why point 2 is a problem? They can run locally if there is a chip that can do that and offload to cloud if not, and if there is a connection fast enough to transfer the data both ways in a reasonable time. I'm thinking about photo and (especially) video processing. Translating would probably be OK even on show connections. Furthermore, IMHO shooting photos and videos in the middle of nowhere with no connection is a more common use case than translating something there.
Not disagreeing with you, but when running a product and R&D team implementing and maintaining both routes is a hard sell. From a business and resource-allocation perspective there's not enough ROI, I'd rather have those resources work on improving power management.
Also, in the end the whole additional effort would be made for a device which is not well-connected to the internet, a tough acknowledgement for a company whose whole business is based on people connected to the internet.
For Qualcomm the story is different: They want to sell the chipset to companies who then build/buy local models to include in their firmware package, just like they're used to for the past 10 years...
Because you have to maintain two code paths that are very different.
One of them gets a huge budget (pixel design) and has to rewrite major ML models. Very expensive. All for a small amount of users.
The other code path has most of your users, and can use whatever the R&D team throws together. And works for every customer, and can be easily bug-fixed if issues arise.
When you’re Google with Billions of users, your long-tail use cases are all common. You have to think about what that behavior looks like regardless.
I think this is the point - each design team and company bet on what they believe their hardware is likely to need to perform. The performance vs power consumption trade off is really tricky when you have to guess what the software landscape will look like a few years out. Get it right, and you have a winner, get it slightly wrong and you either have 'poor battery life' in reviews, or you offload stuff which ideally would be running locally (which is the suggestion in this case).
I'm not sure I want LLMs on my phone anyway - the situations where you want a 'digital assistant' are generally ones where you've got connectivity, so offloading seems like a sensible fit for me. Of course i'm making that assumption and i'll probably look silly, but that's progress right?
Of more interest is the heavy lifting for image processing to make the phone camera not suck (it's all smoke and mirrors with phone cameras, and more inference on device helps).
I'd love to have an LLM on my phone when no connectivity. There are many situations where a personal assistant would come in handy.
In the wilds if Canada somewhere:
Hey google. Lead me through the steps to create a splint and stop the bleeding. My buddy just got attacked by a bear and is bleeding out.
What? That's not a bear? It's a racoon ? Oh...anyway.
Lead me through the steps to start a fire with wet wood and how do I prepare a bear, I mean racoon, carcass for good eating. I have some old copper wire and a magnet. I need to make a phone charger. Lead me through it. Also make sure the fire is smokeless as we don't want to get discovered and sent back to the supermax.
Translating that information into an easily understood format is the job of a LLM, but it's not having the data, right? Have we not moved the goalpost a bit if we want a model that contains the majority of human knowledge, including accurate and guaranteed non-hallucinated survival information, to be stored locally on our phone?
GPT-4 by some estimates is 45GB of training data. But phones can easily store 10 times that much and more if it was really required. The bottle neck is the data processing required to make inferences on that data.
You ever go all in on a technology that tries to sell your users on a big bezel in exchange for being able to use their phone without touching it and then have a global emergency come around that largely removes the only reliable way to authenticate the user without them touching the phone?
There's a big difference. If you process stuff locally, uploading the entire unprocessed data to the cloud would spark a lot of criticism. So, you are stuck running analysis locally and uploading things like relevant ad topics.
If you run your processing through the cloud, there is no optics or bandwidth cost of keeping the entire raw data as long as you want, let humans review it, and figure out which metrics to extract from it at a later point. I would say, the latter raises more privacy concerns than the former.
Yea but then they have to process everything. Sending just the relevant information for ads means they save on cost of running compute.
Considering Azure has run out of GPUs and is renting from Oracle, it’s not crazy to imagine that they could run out of TPUs and the price of compute out weighs the slightly (?) worse ad targeting.
The actual experience of using Pixel ML features is extremely good, so it feels like the implementation details are inside baseball. Just like the last line of the article says: they are class-leading features. Speech-to-text and the reverse are the most noticable improvements over iOS.
Can anyone just clarify for me what these AI / Machine learning chips are? As far as I can tell they are general purpose microprocessors with some added instructions which accelerate operations that are commonly used (matrix multiplication possibly?) but there's a lot I don't quite understand because some of the info is marketing and some of it it heavy technical stuff and my knowledge falls in between!
modern AI is just vector multipication. any AI chip is just 10,000s of very simple cores which can do vector float operations and little else. this also entails clever trade offs of shared cache and internal bandwidth.
(as a thought experiment, consider a naive million by million matrix multipication. this will take a single cpu about 1 year! how do we reduce this to 1s ?)
Nowadays AI chips are specialized in not just vector multiplication but matrix multiplication. Just as moving from scalar math to vectors brings savings in control and routing logic, moving from vector to matrix does the same. Taking a result from a floating point unit and moving it to a big, multi-ported register file and then reading it out again to feed into another floating point unit is often a much bigger draw of power than the multiplication or addition itself and to the extent you can minimize that by feeding the results of one operation direction into the processing structures for the next you've got a big win.
As someone with a lot of interest in but no fluency with chip design, or the dividing and conquering of math within silicon, for that matter, how would you multiply a 1m² matrix?
Each "unit of work" in matrix multiplication is not dependent on any other unit of work. Stuff as many cores as you can into a chip, and then simply feed in all your vectors at the same time.
A million element square matrix is a lot of data. To process that in a second is much more bandwidth than a single socket can support, so you'll need many sockets too.
Chips take a lot of time from design to roll out. Giant LLMs being something you want to serve is a very recent development and it will take time for the deployed hardware to catch up, no surprises here.
Other than bringing up Google Assistant with Bard, the article doesn't mention giant LLMs at all. It lists features such as Gboard proof-reading that are not done on device, as well as the chips heating up when doing some of the general tasks such as downloading, so from that regard, it seems that most AI tasks are not done on the device. Which was the promise sold with these chips.
Does the article has anything to say apart from clickbait? From article, Google is working on a 3nm chip with TSMC, so it is surely not abandoned. Current chip sometimes run hot, that's bad but not an existential crisis. Not all AI things can be on-device, was that ever a promise? Is that even feasible? Is any competitor doing better? LLMs with 100s of billions of params can't be run on a beefy desktop, so why is it a surprise or concern that bard can't run on Pixel.
I came here to say exactly the same. Where is any substance in this article? What are the capabilities of the chips? How many Tflops can they do?
I'm not buying it. I have a couple of Google's (mini) Edge TPUs doing object recognition in my Cctv server. They have been working 24/ 4 for about 2 years and are consistently delivering on the 4W/4Tflops promise(although for some reason they stopped advertising this number focusing on benchmarks instead).
I for one wish Google would have us their big (or even medium) sized TPUs to play with on an add on card rather than in their phone.
While intentionally a bit click-baitey, I think the author is saying that features _using_ the ML tensor cores are on life support. They announced/hinted at that a bunch of features would use on-device ML and in the end they are just using the cloud like they always have.
As far as I understand, some inference is done on-device. LLMs and diffusion just changed the field in the last year and it takes time for hardware to catch up (+ work to reduce model sizes). So, it's just hard to run the latest models on-device. So you either end up doing it online (Google's preference) or having weaker models (Apple's preference).
The "on life support" title has absolutely no justification and is just a sad editorialization with no evidence from insiders at google. They could have just as easily said "tensor chip isn't so good with tensors" and gone with a more factual headline.
The image Google has created in the market of their products is, stellar success or death. The article states and provides examples where the chips are completely missing the mark (that Apple is setting) and that the chips aren’t a stellar success, so life support is probably a group of people trying to keep the project alive.
AI goes through cycles where whichever field is currently making the most advances gets all the hype. But just because LLMs are currently flashy doesn't mean that object detection, image segmentation, facial recognition, image generation, text-to-speech, speech-to-text or anomaly detection are suddenly less useful.
Maybe if I had better understanding of the Pixel capabilities I may have bought one.
Maybe Google can't market anything because they're so entrenched in the least effective advertising methods out there?
Where are the catchy short commercials on YouTube compelling me to this great tech? Just because everyone knows Google doesn't mean they know what Google has to offer.
Google is absolutely awful at marketing. Strada was really impressive tech that let me play AAA games almost seamlessly on my old Thinkpad, and none of my friends knew what Strada was because of how it was marketed.
Do you mean Stadia? Is it ironic you got the name wrong.
It wasn’t good enough. It purported to be great, but it had random connection issues. Given that destiny, a 1st person shooter, was their flagship game these flaws really jumped out.
You also really needed good internet. Like really good. Mine performed well sometimes when it was hardwired, but lagged regularly over WiFi.
I think these things were fixable, but that thing was dead within a month of launch. Google didn’t get fully behind it, so all the people that launched it got their promos, saw the writing on the wall, and transferred out.
> Given that destiny, a 1st person shooter, was their flagship game these flaws really jumped out.
In the earliest days of the internal dogfood, there was one game available: Doom (2016). This was very much intentional to make sure the streaming team focused on the twitchy latency-sensitive gaming segment.
> You also really needed good internet. Like really good.
What was more important than really good internet was stable bandwidth -- even a 4k session could work with a 25Mbps downlink, but very few providers will actually give you that guaranteed bandwidth. Similarly, having gigabit speeds from Comcast doesn't matter if you randomly see spikes of bufferbloat where your latency to the server jumps from 10ms to 100 (congrats, you've now dropped several frames).
WiFi similarly struggles to give you predictable delivery (apparently 20ms is a good number for wifi jitter, but that's pretty much the entire latency budget we had allotted for networking).
> all the people that launched it got their promos, saw the writing on the wall, and transferred out.
This is untrue. The shutdown announcement took the whole org by surprise (other than senior leadership). There were hundreds of engineers who were effectively told "you have X months to find a new position in the company, or leave".
It is true that Google didn't get fully behind it, senior leadership was worse than ineffective, marketing was terrible, and our reputation of killing off products meant it never reached the level of market adoption needed to sustain the product. But we had absolutely done our research to figure out the size of the addressable market and what levels of stable bandwidth users could sustain in practice, well before launch.
Maybe if I had better understanding of the Pixel capabilities I may have bought one.
I am not sure if you'd want one? I returned the Pixel 7a after a few days. Stock Android is great, the camera is top-notch and goes head-to-head with iPhone. But the fingerprint sensor is mediocre (they use an optical sensor) and charging is very slow. The issues make it not-great as a daily phone. People have reported similar issues with the 7.
That was my experience as well. Generally speaking the Pixel 7a is a competent phone with one (big) flaw. The battery life is truly terrible, and the charging speeds are glacial. I was honestly expecting more, especially as its advertised as Google's best. The forums are full of people complaining about it and the responses are almost the same: Turn off 5G!! Turn off wifi scanning! Turn off bluetooth scanning too! Dial down the screen refresh rate!
Returned the phone and got an iPhone 13 instead; and as painful as it was switching to iOS after 13 years of Android, I don't regret my decision. The iDevice comes with its own niggles, but at least I don't have to worry about battery life (yet). Maybe google will figure it out by the time the Pixel 8a comes out.
You may be right. If I had known the flaws I might not have bought it. But I still need to know the upside to even consider it, and that falls on Google, not me.
In Germany, they ran an ad campaign for those devices. They gave some well-known influencers some budget to make a spot by themselves and ran that on YouTube. Those spots were the bottom of the barrel. No idea who runs their advertising, but they need to fire all of them.
My experience using Pixel 7 is it has the smoothest UI of any Android phone I've tried - including flagships like S23 and Xiaomi 13T.
Also the author assumes Google wants to do on-device AI processing, but given Google's desire for user data they may be uploading pictures and videos in the cloud just for the sake of having the data (and maybe use it to train say self-driving AI models ).
My experience with my Pixel 6 Pro is also amazingly snappy UI and great software, but absolutely crap battery, the worst of any smartphone I've ever had (and I suppose that's the chip's fault, because the battery itself has a normal amount of mAh). The model is also quite unreliable, to the point that I sometimes disable 5G and 4G because they do more harm than good.
I could live with the modem issue (3.5G is OK for 99% of my smartphone usage anyway), but having to care so much about battery running off is a quite important drawback for daily life. The fact that they seem to be keeping the same path with the chip makes me think that I'll have to look for another brand for my next phone.
Got the 6 Pro too, me battery life isn't great either, but I WFH so I just mitigate with a power bank or spot charge for the times when I am on the go all day.
Never had issues with reliability though, I've been really impressed with the hardware aside from the battery.
I haven't used 3G/3.5G in over a decade now(almost). Even 90% of our carriers have disabled 3G services , some are 4G/5G only and others support 2G/4G/5G
As a user of all three chips vis Pixels 6, 7 and 8 Pro as well as this current Pixel Tablet, I do not get it. These devices work fine for me. I have also had Samsung and OnePlus devices. I have never had performance problems. I switched from Samsung to Pixel due to the messy software situation- there were two or more apps for every purpose and which to use was incomprehensible to me, and I am not too dumb.
Granted I am not a heavy user of things that might need extreme capabilities. I do not play complex games or do media creation.
I'd like Google to go back to Snapdragon, but this is just an opinion piece. It doesn't back up the claim that Tensors are "on life support".
Tensors are not as fast or as efficient as the alternatives (the reason why I didn't pick the 8 Pro recently and went with a Galaxy S23 Ultra), but it's fast enough for most users and the efficiency could probably be fixed by using TSMC instead of Samsung, just see the difference between Qualcomm's 8Gen1 vs 8+Gen1.
The reason Apple was right about on-device processing isn’t about the privacy marketing speak or anything like that, it’s because so many of cloud services are run at a loss and have little hope of ever becoming profitable businesses.
Every time you use Google AI that’s offloaded to their cloud you’re basically getting free “money” from Google, and all of these functions are places where advertising can’t be inserted in a way that makes any sense.
The most profitable cloud services are things like “sell storage that sits unused” or “sell a subscription to fitness classes.” If you want to run something complex and resource intensive you better hope you have some enterprise customers lined up.
I like to dish on Goog's constant slurping up of data as well, but I could be willing to give some benefit of the doubt that they just can't make it work locally. Oh, nevermind, I can't even type that with a straight face.
I see no reason that Google, seeing as they control the whole phone OS, couldn't provide the benefits of local processing and still send all the data back to themselves, perhaps when on a charger and with WiFi.
So you can only use the feature when plugged in? That's a limitation for a mobile device that pretty much renders it totally useless.
Hey Google! Please do this thing for me.
I'm sorry Dave, but I can't do that. You must be plugged in and connected to WiFi.
Hey Google! I'm at my kids sportsball event, and that's not really conducive.
I'm sorry Dave, I didn't get that.
Besides, we're talking about 3 chip designs and still not there for on device processing at the same levels as competitor(s). So just punt and send it to the cloud.
They may often be used with small embedded computers, but 1 Coral TPU handily beats even a top-end CPU for the stuff Frigate needs it to do -- and it does it at minuscule power draw. Sure, it's inference-focused, and that means it has limitations: it sucks for e.g. speech models, too. But it's pretty great at what it does and gets used in real applications too. My guess is you saw the USB-C model, but they do M2, big cards with multiple M2 slots, and miniPCIe (popular in industrial applications) too.
I just want decent on-device voice recognition for things like asking the Assistant to send text messages. So tired of having to repeat myself 5 times.
I've extensively used both iPhones and Pixel phones for years.
Siri is astonishingly, infuriatingly bad. It's hard to imagine anyone on the Siri team uses it daily like I do. It works well ~30% of the time and is bafflingly non-deterministic.
Google is much, much better, but it's still not objectively great. It's closer to 80% of responses being reasonable, which is still unacceptably low.
I also have only had Pixels basically since they were released. I guess where I have issues is when the phone has a weak signal, and it insists on trying to use it for voice recognition rather than just falling back to local.
Generative text AI (Large language models) require a ton of RAM for inference which just isn't a thing on mobile devices yet. It makes more sense to keep these features in the cloud vs making phones with unrealistic specs.
All I see is a lot of cope and bickering about small things.
Fact of the matter is that google doesn't make good interesting and innovative products because it has started drinking it's own cool aid.
About at the same time as when they started becoming evil, getting political and influencing elections everywhere, they also stopped even trying to make innovative products and instead went full communist.
Everyone knows the effect that has on innovation but somehow SF techies and american wokies are unable to comprehend it, and vehemently deny it.
My problem with the pixel, and the only reason I returned it, is the modem. The chip was fine, I don't care that it's "underpowered", but when reception got spotty it chugged power. Google made an excellent SMARTphone, but they seem to have forgotten it's also supposed to be a smartPHONE and it can't always be on wifi where it will stay cool and efficient.
It was exceedingly telling when Google bought moto (at the time a good phone manufacturer and excellent radio within) and gutted and dumped it instead of keeping it on to go wild with the nexus/pixel side of the house.
Motorola phones where using Qualcomm modems. There is nothing really special about it. Even the Nexus had the same modems during that time period. Pixel phones didn't have widespread modem issues until they switched to Samsung modems starting with Pixel 6. Not sure what is there to tell about it other than Qualcomm makes good modem. Even the iPhones with Intel modems where inferior to Qualcomm modems and thank god they switched back to Qualcomm with iPhone 12.
Intel's cable modems also had issues. I wonder if there's some patent they're trying to work around that technically has alternate solutions, but they're all inferior on power consumption?
I see a lot of people reporting battery drain and overheating on the Pixel 8... Sigh. Why can't they make a good phone? It makes me wish other manufacturers would step their game up so GrapheneOS could support them.
When I got my Pixel 7 Pro I was shocked at how bad the battery life was, even without google play services. (came from a 5a)
I love the phone, but the battery life makes me regret my purchase almost every day.
Then my coworker got the Pixel 8 Pro, and it has WAY worse battery life than my 7 Pro, he even went and got a new one at the store because it was so bad it had to be a fault, but no. It just is absolute garbage.
Out of curiosity, do you have any Meta apps installed? I've tried to kinda debug the issue by reading the system logs. In the end my battery life was greatly improved just by full removal of facebook, instagram and, especially, messenger. It also helps to disable background data for all the apps you don't explicitly want to run in background and to set "restricted" battery mode for them. I'm using grapheneos with sandboxed playservices by the way.
Opposite of my experience, I daily drove a Pixel 6 Pro from launch until 8 Pro launched and have been using 8 Pro since. I upgraded because I can, my Pixel 6 Pro is working just fine, I've added it to my growing device catalog for testing during mobile development. Over the course of the 6 Pro, I replaced it twice, once for a cracked screen and the second time for cracked back glass using preferred care (both close in time to each other from similarly stupid drops). I do not use a case for my phone. None of those three devices had any issues with battery. My partner is using a 7, two of my friends have 6 Pros, and I just upgraded my parents from 4a to 7a and battery hasn't really been a serious problem from any of them. It's not insanely good but it's also not a pain point for me.
As an engineering lead I do a lot of my non-programming work directly on Pixel, including tons of remote desktop, meetings, GitHub/productivity, k8s management, etc. I don't get to write code nearly as much any more, probably around 10-20% of my work week. At least for those workloads, things are fine. But the workload you run can dramatically change things, especially if you've got particularly power hungry apps involved. The OS gives you tools to identify what those apps might be when you dig into the battery area of settings.
> even without google play services. (came from a 5a)
Since you can't remove GMS from a Pixel without flashing a ROM to it as far as I know, can I assume you are not using stock software? A lot of Pixel's battery performance is made up using the adaptive battery features. When they are off, things are more dicey. Do those work on custom non-GMS ROMs to begin with?
To sum it up, battery can definitely be better, but I'm just not seeing the kinds of problems people complain about and that surprises me given my experiences.
Note though that I can attest the modem in the 6 Pro wasn't super great, and the design of the 6 Pro with it's curved glass definitely contributed to those drops taking out the screen/back.
As someone with an S21 too that doesnt sound normal, have you checked your background apps and battery health?
The energy saver setting works pretty good too.
Try using 5G with sim while connected to 4g with e-sim it keeps dropping packages, is impossible to have WA or Google meet. Haven't worked on the pixel 6 neither pixel 8
I would bet quite a bit of money they didn't forget, based on the people I've known on the team. That's just Samsung's modem not being as good as Qualcomm. Choosing Qualcomm would have led to a very different phone where the feature differentiation they were trying to deliver with Tensor simply wouldn't have been possible. There are serious consequences to the middle ground solutions as well.
> Choosing Qualcomm would have led to a very different phone where the feature differentiation they were trying to deliver with Tensor simply wouldn't have been possible.
Google doesn't have nearly the same leverage and volume numbers to use against Qualcomm as Apple does, but even for Apple it's a devil's bargain. The licensing terms are incredibly onerous. If they updated them and you didn't agree, they used to just cut off your entire company's chip supply until that was ruled monopolistic and illegal. The royalties (one of the only royalties Apple actually pays) are substantial percentages of the final device price. Getting into bed with Qualcomm needs to be weighed against the risk to the rest of your business.
As a consumer, I buy Qualcomm modem phones, but as an engineer Qualcomm is only marginally below the plague on my avoidance list.
Apple also bought Intel's modem unit as either leverage or an escape hatch from Qualcomm. It's like having unlimited dollars to throw at a problem is useful.
Has Qualcomm ever engaged in significant semi-custom work? (no, adjusting dvfs for a special "for galaxy" SKU or adding pluton to 8cX gen 3 doesn't count). Samsung was a semi-custom partner for Apple in the early days of the iPhone (to the extent that the M1 still has some Samsung IP in it[0]).
I heavily doubt that Qualcomm is even capable of working in such a role. In fact, the C-suite probably believes that their core competency is lawsuits and not modems, anyway.
Has Qualcomm ever engaged in significant semi-custom work?
Yes, the early generations of Snapdragon SoCs were built using a custom ARMv7 core called Scorpion, which was later succeeded by another v7 core called Krait. The first ARMv8 SoC (the infamously disastrous Snapdragon 810) used stock ARM cores, but the next generation (820) switched to an in-house v8 design called Kryo. Around that time though, the company did massive layoffs, including in silicon engineering, leading to all the subsequent designs using tweaked ARM cores (despite also being called Kryo)
> I heavily doubt that Qualcomm is even capable of working in such a role. In fact, the C-suite probably believes that their core competency is lawsuits and not modems, anyway.
It's not 1 of capability. They're trying to force everyone to use the full SoC and not just the modem.
Totally fair take. As a consumer I have a tendency to forget it's a game of trade-off, but I don't think that it's worth it, which is why I returned the phone, but I hope my opinion won't match everyone's and a lot of people will be happy with the device.
It's just a shame, even now I find myself eyeing the black Friday sales prices. I loved everything else about the phone, but I know that the battery life is going to drive me mad and with all the heat and increased number of charge cycles there's just no way the battery is going to last as long as the support does and that's a deal breaker for me.
In ML I think the story is a little different. Hardware design/production pipelines are measured in years. The LLM "revolution" is relatively recent. For the Pixel8, I imagine some folks at Google had to make the really tough decision of whether to run weaker, smaller models on device, or run the giant LLM models in the cloud, giving much better results, albeit slower (and with ofc the requirement for an Internet connection).