This is a whole lot of nothing. These awards are and always have been a way of showcasing great apps on app platforms. They are not "what's the best thing in the world this year that has an app".
ChatGPT is a boring app. It doesn't have a ton of platform integrations, it doesn't show off the capabilities of the platforms, it doesn't push any boundaries on what an app can be. The product might be great, but that's not the point.
This is my personal opinion based on years of following Apple's awards, I don't have any inside info in my role about this and this is not my employer's opinion.
To be honest, I really wish ChatGPT did have much tighter platform integrations. Every time I ask Siri a question and she fumbles it (which is most of the time), I long for a chatgpt smart speaker. "I found some web results" is siri speak for "I'm way dumber than chatgpt and can't solve this simple problem".
But Apple - and Google I imagine - don't let 3rd party apps replace siri / "hey google". Apple and Google docking points because chatgpt doesn't have a lot of platform integrations feels like a very petty move.
> don't let 3rd party apps replace siri / "hey google"
I think it's reasonable that they don't let you replace the logic behind their branding – it would be quite a serious issue if ChatGPT started giving offensive results when you ask "Hey Siri".
Whether they should allow replacing the voice assistant entirely is an interesting question. On Android OEMs can of course do this, and have done so with mixed success, see "Bixby".
The issue with LLMs is that they are slow. People expect a response immediately, so much so that Amazon Alexa built much of its popularity as a kitchen timer over the alternative because it's just fast.
I think it's a much more reasonable argument that the reason Alexa, Siri, Google Assistant, Bixby, and Cortana (not sure if that exists anymore?) don't use LLMs is not that Amazon/Apple/Google/Samsung/Microsoft are asleep at the wheel, but that LLMs are not suitable yet. I'm sure this is coming, but I'm not surprised none of them have it yet.
This isn't just about "replacing Siri", to do that successfully, Chat GPT would need almost root-level access to your device. Things like constant background microphone usage, the ability to work on the Lock Screen, make calls, send texts etc without an extra prompt, get access to intents from third-party apps, all of this would either require separate APIs (which come with compatibility promises and hence lots of deliberation) or the ability to run apps unsandboxed, and would have serious privacy implications either way.
edit: With that said, if Chat GPT came before iOS, I'm pretty sure Apple would have had to design iOS with that use case in mind and get these APIs exposed somehow, maybe even through a direct integration with OpenAI or Anthropic.
Sure; but on reflection I don’t really care about chatgpt being able to make calls and play / pause apple music. I just want a way to invoke chatgpt with my voice from a locked phone and go straight into a chatgpt conversation. ChatGPT’s strength is open ended questions, storytelling and synthesis of information. Not its ability to set an oven timer.
Now, that would definitely still take some work from Apple. Right now as I understand it the device listens for “Siri…” all the time using some on-device ML. They’d need a way for that to work with chatgpt and hand over to the audio stream. And have a chatgpt conversation from a lock screen without breaking sandboxing.
But all of that would definitely make my phone a much more useful device. I won’t hold my breath though.
For some reason you still cannot edit your chat turns in the mobile app. I frequently refine my prompts and the way I ask questions as part of using GPT-4, since the way you phrase your prompts to the model is pretty important (and revising your question by sending a new chat turn can poison the context with the original question/response) and not being able to do this on mobile really kills a lot of the utility of the app for anything other than 1-2 turn quick, simple questions.
I think a strong argument can be made in the other way. I went running the other day and had a long conversation with ChatGPT. That sort of experience is absolutely pushing boundaries...but I can see that they aren't the boundaries that Apple wants to celebrate at this moment. Apple has set a pattern of rewarding beautiful, self-contained, cohesive, through-designed, snackable experiences. A technology that challenges the notion of a smartphone itself is basically out-of-bounds.
You've had that exact experience in tonnes of apps, probably with an equally good or better interface. It's a completely serviceable chat interface.
The revolutionary part is that you're chatting with a language model running on a server rather than a human. But that doesn't make the app impressive, it makes the back-end impressive. App of the year isn't about celebrating impressive back-ends.
> I went running the other day and had a long conversation with ChatGPT. That sort of experience is absolutely pushing boundaries...but I can see that they aren't the boundaries that Apple wants to celebrate at this moment.
Messages and WhatsApp are better designed and the communication experience is better.
I agree that this is what Apple rewards, but your suggestion about ChatGPT sounds to me like an argument for the award of "best thing in the world with an app", not an argument for ChatGPT winning the app store awards.
If anything, ChatGPT even without platform integrations, make Apple/Google/Amazon look bad because the quality is so much higher than their assistants. It also doesn't showcase any of the platforms in particular, it doesn't try to be the "$platform killer app" of anything.
I think it's understandable that these platforms aren't giving ChatGPT any awards. It is not helping to make the platform stand out against other platforms.
> I think it's understandable that these platforms aren't giving ChatGPT any awards. It is not helping to make the platform stand out against other platforms.
I agree, but I think the crux of my point is that this is because ChatGPT the app doesn't stand out from a platform perspective, rather than because advertising ChatGPT would go against the platform goals.
Essentially I have no real reason to question the integrity of the awards processes here, because they seem to be awarded for the same reasons as they always are, and that ChatGPT just doesn't tick the boxes they're looking for.
I'm not naive enough to think some exec hasn't been happy with the convenience of this, but it seems a little conspiracy-theory like to assume malice here.
No one believes this. Or at least no one who is both honest and intelligent.
This is mere mental gymnastics to explain this away. I'd take a hard look at what is going on in your brain to cause these kind of apologetics for Google and Apple. Its kind of scary to see from the outside.
To be clear, by this I mean "ChatGPT is a boring app". ChatGPT is obviously not boring, but in terms of the app technology, it's a simple interface, it doesn't have platform integrations. I don't think a thin API client warrants app awards.
I welcome good faith critique and discussion, but feel that you may be engaging in bad faith arguments and taking my points out of context or purposefully misrepresenting them. Perhaps not, perhaps I could have clarified my position, but I hope you understand the point I'm trying to make.
ChatGPT is a single input with a submit button. An app being boring and it also being useful aren't mutually exclusive. But you're naive to think the app itself is an exemplary example of an application for the platform.
And your reply is incredibly hostile and aggressive. Before you suggest to others they should "take a hard look at what is going on in your brain" — you may want to take your own advice.
This article seems to be conflating a lot of things. Saying that they avoided naming it like it was the given choice is quite the flawed assumption.
The ChatGPT app itself was a very thin veneer over an online service. It utilizes almost no facilities of the smartphone client, has no local AI or unique intelligence of the device. Indeed, OpenAI left the field open and before they came out with their own there were a myriad of competing facades that were effectively interchangeable.
I don't see why that would earn app of the year. I wouldn't even put it in the general conversation.
> I remember seeing here that his default system prompt is also optimized for oral conversations.
Voice2voice is great, but I find the way chatgpt constructs responses in oral conversations to be very soulless. It feels like I'm talking to the personification of a bland corporation. I'm not sure if it feels worse through voice because it sounds close to human, or if its the system prompt. I'd love to find a way to fix it.
Does anyone have any suggestions of a good system prompt to add to make the AI loosen up a bit?
It does feel like a bit of a snub — my guess is they did so because they see ChatGPT as a direct competitor, and don’t want to draw attention away from their own current or upcoming generative AI efforts.
After everything that happened over 2016-2023 don't people realise there is no such thing as "neutral source of information"?
Even the neutral POV policy of wikipedia is constantly worked around by various actors by marking "bad" sources as biased or unverified, while "good" sources canonical and neutral.
> After everything that happened over 2016-2023 don't people realise there is no such thing as "neutral source of information"?
A little bias is inevitable. But there's no reason to cynically throw the baby out with the bathwater. We can still strive for fairness and balance. We should still complain when we see blatantly biased source, and seek out good journalism. We'll just never fully achieve it.
I regularly use it for menial tasks and it has almost completely replaced searching documentations of major libraries for me. As a senior developer, it's a productivity gold mine if you learn how to use it correctly. But I wouldn't recommend it to any junior dev, because you need lots of experience to see when it starts hallucinating.
No, I'm talking about things like writing SQL queries (even complex ones), CMake files, Docker configs or plotting stuff in Python. Of course, if you're not already an expert in these things, you'll have a hard time distinguishing useful replies from hallucinations - that's why I said it's mostly for senior devs. Without expert knowledge, you will likely not be able to benefit from it in its current state. But if you have that and know how to write efficient queries, it can easily up your productivity by a factor of 10 (i.e. going back and forth for 6 minutes with GPT4 to make it get your requirements can save you an hour of work looking through documentations).
10x seems like a stretch, but I agree. I have a chatgpt window open most of my workday now too. "Hey chatgpt - help me write this mathematical equation in latex. Hey chatgpt - give me an example nodejs program that reads a line-delimited JSON file without loading the whole thing in memory first". "Hey chatgpt - What are the popular & good python libraries for drawing an SVG scatter plot of some data? Give me some example code for each library."
It depends on what you're doing of course. With your example queries I'd also expect a lower boost, but the benefits should still be there. Where it really shines is if you already know exactly what you want and how you want it and you merely need someone to write it in code in a common language.
As a senior developer: Experience. The output from ChatGPT isn't perfect, but it's good enough to point one in the right direction. The code it outputs is mostly garbage that a good developer would never put into production, but what it does fairly well is finding the relevant bits of documentation.
Example: Using a small part of a new, big, unfamiliar library. Rather than digging through the library docs, I can ask ChatGPT about it, which often points me to the relevant parts, which I then can still confirm in the official docs.
Yeah. I got chatgpt to write some code using Indexeddb the other day. The code was absolute rubbish, and I rewrote it to be half as long, while being clearer and with better error handling.
But the code it gave me was a great starting point. I found it much faster & easier to rewrite the bad code it wrote than pore through documentation and figure out how to solve my problem from scratch.
I've found that for questions that aren't super complex on things that are extremely likely to be in its training dataset (such as public documentation of popular Python libraries) the error rate is very close to 0%. Even compared to GPT from a few months ago I've found the accuracy of responses has increased dramatically.
I still believe anyone using these tools on a day-to-day basis should have a sense of "trust but verify."
GPT-4's solutions are usually correct the first time around, or can be corrected by telling it what it did wrong. The ask-clarify-correct loop is still faster and less effort than doing it yourself.
If people continue to learn outside of LLMs and can tell when it is wrong so they can correct it. The slope can be slippery. For those of us who grew up before the rise of LLMs, this isn't really something we have to worry about. But the next generation will be not know what it was like without LLMs, kind of like the pre/post-internet/phone generational splits
I more-or-less know what I expect it to give me, so I can just go somewhere else then. I tend to ask it three broad sets of queries, since it's what it does best at. tl;dr: it can transform text very well, even with lots of ambiguity. also it knows technical vocab
- if I'm working in a field that's new to me or that I don't understand, I ask for help understanding the basics and vocabulary of the field. it does very well at this.
- if I have a well defined problem, but am simply not familiar with the libraries for a given situation, it tends to do a good job translating my english queries into the right code. I do examine and test the code afterwards to make sure it's correct. You can really see this in action when you ask it to do data analysis; and the REPL loop in that mode is also great at catching bugs.
- if I have like a copy-paste from a documentation site, I can ask it to transform that into code or into a better-formatted version. this saves a lot of time, and I don't have to remember regexes or vim keybinds
> if I'm working in a field that's new to me or that I don't understand, I ask for help understanding the basics and vocabulary of the field. it does very well at this
I also do this, but I am careful about being confident that "it does very well at this". We can't actually evaluate what it's putting out, other than that it sounds plausible, which is something LLMs are truly great at.
Would have to disagree. I've writing code for +25 years and counting, a good portion is critical infraestructure code. I've reached the title of chief-expert and quite frankly gets difficult nowdays to find people who can answer my questions or at least help explore possible routes to solve them. ChatGPT is like a buddy that is always available and sitting next to you for a good technical chat.
Sometimes it comes up with really good techniques that are different from my usual approaches, other times is plain wrong and I'm correcting it.
I am more productive with the ChatGPT in my life. Whenever stuck on some weird error, instead of googling I paste the log and discuss with the bot what can be going wrong there. We can talk about pointer analysis, performance bottleneck comparisons. Could I do it all on my own? Sure. However it is boring and quite certainly would require 10x more time to perform all calculations on my own.
In the end of a day it is just another tool. Brings advantage when used properly.
If I'm looking at MDN for JS docs, I don't need to cross-reference anything. Yet I can't be sure it used the same MDN in my answer as opposed to some random SO post.
I could specify for it to use MDN exclusively but at that point I might as well use search.
In addition to that I could judge the quality of search results (a lot vs little mentions of a technology, shady vs reputable site etc.) to make educated guess of the output I'm getting from search. Can't do that with GPT.
Having the context for a recipe makes it a lot easier for me to evaluate whether this is probably a quality source or not. I really don't get the hate so many people seem to have for anything other than a list of ingredients and steps to follow.
In general, the context of search gives some insight into the credibility of the source.
It’s because they are entirely manipulative, and usually have nothing to do with the actual recipe. The “context around the recipe” rarely is written by the person who put the work in to develop that recipe. It’s almost always content-farmed out or more recently, entirely generated by AI.
The only reason those blobs of text exist is to get you to look at more ads. Put more things in your head against your will, sell you more garbage, and manipulate your feelings.
If it wasn’t true, why are the recipes always at the bottom? Why not put the most valuable part right front and center? These websites have no respect for you and likely copy pasted the recipe anyways.
Random life stories are everywhere, and have nothing much in common with cooking well. Even before LLMs could fake that part as easily as the recipe itself.
Only way to know if a recipe is good is to look at it.
> Do you trust the content of the sites Google returns you?
You don’t. But i’dtrust a top rated Stackoverflow answer over whatever LLM spits out.
There is no “confidence score” from an LLM output. You cannot tell whether it is making things up (and potentially make very bad decisions based on it’s output)
There's no confidence score for specifically ChatGPT, other GPT models hosted by OpenAI (let alone the broader research community) have been given that capability.
I honestly never thought of using ChatGPT for recipes. I just asked it for "a simple pizza dough recipe for a medium thick pizza you can cook in an oven" and it returned the exact recipe I have memorized which I think came from a "AirBake" pizza pan I bought 20+ years ago. Thanks for the tip!
Just to be clear, I wasn't complaining. I liked that it came back with the one recipe that I've already settled on (and I've tried quite a few over the years.)
I think I'll ask it for a calzone recipe this weekend. The one I use now makes the dough a little too bready.
What do you trust? I don't trust search or other humans to be 100% accurate about anything so I find it really strange that people presume I take whatever ChatGPT outputs at face value. I parse it just as I parse any other information.
I’ve seen this discussion many times over the past year and have come to think that the disconnect basically arises from the way that we have thousands of years of heuristics built up for interpreting the trustworthiness of what another human is telling us, 30 years of evaluating websites, and less than a year of evaluating LLM outputs.
People have some sense that someone giving them information may be an {expert, charlatan, idiot}, or that a website they’re looking at is run by a university vs a blogspam content farm, but many have not developed a sense for when or how much they can trust LLM output, which is delivered with the same tone and confidence regardless of whether it’s entirely fabricated.
There is probably a component of personality involved in how people approach this. Collectively we are all learning how to interact with this new source of information and people take varying paths.
Exactly. Whether it's a person, a website, or a book, we have a ton of cues that give us some sort of intuitive reliability score. That reliability is essentially never going to be 100%. But, especially if we cross-check sources, we can start to have very high confidence that an answer is true--at least as far as anyone knows. (Or even that no one really knows the answer with any certainty.)
I've had ChatGPT return very serviceable "true" results and I've had ChatGPT return utter fiction.
1. you don't know the answer, but you can check yourself and easily whether a given answer is roughly correct
2. you don't know the answer and wouldn't be able to check how valid a potential answer is
LLM-based tools are great for 1 to synthesize various sources into one coherent answer, since in this case, you won't become a victim of their hallucination. E.g. "write a one-off Python script to do this": you can quickly check if it does the job, even though you couldn't say whether that's idiomatic Python.
When I'm writing (English text) about something I'm passably familiar with, it can be useful for generating some straightforward descriptions and background. Nothing I'd just cut and paste out of the box wholesale but it can be a timesaver, especially if it's somewhat boilerplate.
I would say it is not good at giving a sophisticated answer to anything that requires a lot of nuance. And I've also asked it questions with fairly objective factual answers that it gets hilariously wrong.
I agree with people saying the app wasn't "all that"... But even it was the greatest thing since the iPhone, I don't see Apple nor Google giving them anything. If MSFT were still in the smart phone game however...
I might be in the minority here, but I agree with their decision.
If you're judging purely on the app, it's not surprising. The app itself is about as basic as they come.
On the Apple side, AllTrails won, and it's deserving of that win. It's made some incredible updates this past year with some topnotch design. Flighty would have been another very deserving win.
ChatGPT is a boring app. It doesn't have a ton of platform integrations, it doesn't show off the capabilities of the platforms, it doesn't push any boundaries on what an app can be. The product might be great, but that's not the point.
This is my personal opinion based on years of following Apple's awards, I don't have any inside info in my role about this and this is not my employer's opinion.