“Eventually, Mei decides she’s had enough of Dot interfering in every aspect of her life, uninstalls Dot, and goes about her business as usual, determined to interact with reality in a more wholehearted way.”
"Dot is, at its most simple, an app you chat with on iOS. You can send it words, voice memos, pictures, PDFs, and it’s thrilled to search the web for you, too. Communicating through written text (Dot’s voice is coming next year)" [0].
It looks quite "ambitious"[1]:
- Automated File Management: Dot creates, organizes, and retrieves both structured and unstructured information.
- Adaptive Intelligence: It learns from patterns in your behavior, plus any guidance you decide to share with it
- Internet Browsing: It has access to up-to-date information (and eventually, tools and services)
- Contextual Multimodal Understanding: It interprets text, audio, visuals, and links, informed by the context it already has on you
- Self-Programming: Dot proactively writes and stores routines, anticipating your future needs
- Personalized Display and Retrieval: It transforms information into the most compelling format for each user
- Conceptual Synthesis: It doesn’t just store information — it connects the dots between topics, ideas, and themes in your life
- Theory of Mind: Dot synthesizes a deeper understanding of your motivations and goals, while reflecting on how it can best help you to achieve them.
Each is possible but having been adjacent to Google projects in this vein for 7 years, and having worked on something similar, lead me to have a substantial discount rate on what I call "magic wand" AI design work.
It would be very odd and transformative indeed to have this all coalesce and work.
But don't listen to me, I'm probably going to go all out on my pitch now too.
2 is just fancy talk for what high perfoming LLMs do
3 has been done many times over
4 is achievable with Imagebind if we're going for an exotic solution. Otherwise GPT-4V with AudioToText and TTS will do just fine (Open AI have something similar set up)
5 is as simple as timed prompts sent by the company unbeknownst to the user.
6 is the probably the most bespoke thing here. I'm guessing this is the "information as a quiz" thing they try to demonstrate.
7 is the same as 2
8 is High perfoming LLM with a specific prompt.
I'm not trying to discount your experience but the technology that is making any of this possible is a few years old and the new state of the art version which is far ahead of everything else is ~8 months old so unless you just worked on something like this then I'm not sure it's much indication on what is achievable.
That's exactly what I'm saying - I've just worked on something like this since January. I throttled back from universal file manager and stuff, I think they're kinda hinting at it too through the "probabilistic computing" thing. Staying focused on RAG + as much local compute as possible, user perception of citations is through the roof compared to practically anything else.
The problem I saw over and over again at Google wasn't, like, how do we do this. It's like the "draw the rest of the f*king owl" meme. You can pull some stuff off probabistically or presenting choices, but it gets real nasty real quick when you screw up (the most public version I can think of here is how immensely frustrating it is if a voice speaker mishears your query, especially twice in a row)
Failures being a series of products is something else I feel hurts, ex. getting 95% of the files right x 95% of your appointments x 95% of your sleep schedule, honestly, each individual prompts reliability is top-tier. Yet, 15% of the time it kinda has no idea what you're doing.
But I'm sort of devil's advocating here, to a point. I think you can get there, but boy it's a lot of work and they're 3-4 years away if they have 100 employees and a lot of work, non-trivial on several fronts, even just the GPT billing is a nightmare. I am 99.9% sure I am the only person on earth besides OpenAI employees who know how to bill functions correctly to the token (to be fair, getting within ~4 tokens is publicly available, just not from OpenAI, incredible reverse engineering work by a few people on a forum thread)
And then I guess there's a smidge of "if you're chasing all these things simultaneously you might not grok it" on my end. You're far better off making a slick interaction for moving files between buckets and creating buckets, than promising you can reverse-engineer from scratch, all the right buckets, with all the right files in them.
Their privacy policy lists OpenAI as one of their partners for data processing, which indicates that this is happening not on your device, and data is also shared with third parties.
For me this is the main counterargument against apps like these. I want to feel free to post any information into this without thinking about who may read or use it.
Local is the only way to go for software like this in my opinion.
Yeah reading this web page I just keep thinking “Mei has trusted a cloud based service to be her personal confidant for all aspects of her life including text and documents. This will end poorly for Mei.”
That also means it's useless for interacting with your life if your life happens to include anything above a PG-13 rating, what with how cloyingly pearl-clutching the OpenAI offerings are about sex or violence.
"We will never monetize your data. We will never monetize your attention. And we believe that the only way we can build towards the future we envision is through the continuous reinforcement of mutual trust and respect. Currently, we leverage best-in-class cloud-hosted models, including ones from OpenAI, Anthropic, and a selection of open-source options. Over time, we plan to reduce external dependence and localize computing to run on-device."
Great, then they should just put their promise into a legally binding irrevocable clause in their terms of service and also legally guarantee that their entire business will shut down if they violate it. They are never going to do it anyways, so no harm in enforcing what they are never going to do.
I'm not saying I would put any faith in it. The company could start genuinely believing those values but the values of a company change over time with more people investing.
"After the untimely passing of her oil tycoon father, Mei finds herself with more money than she knows what to do with. Dot helpfully suggests transferring a few hundred thousand dollars to New Computer as a thank you for Dot's continued existence."
I don't want to be limited by the little CPU on my phone. Maybe one day it'll be powerful enough for all these tasks. But until then, I want a big H100 computing for me. And I want to pay for it! I don't want to be in a position where the service is trying to give me as little compute as possible. Putting me at the center, I want it to do whatever massive compute it needs to to make my life easier.
> Dot remembers Mei’s interest in singing and proactively sends her suggestions for music clubs at school.
Imagine having so little agency and motivation within your own existence on this earth that you need an app to remind of what you once found life—affirming.
This is the kind of thing where adoption and user mass matters a great deal. If thois is successful and apple are too slow to roll out something like this, don't expect a lot of users to "just switch" out of what they've invested a great deal of personal data and routine into. It'd have to be something even deeper, like OS level integration.
> It'd have to be something even deeper, like OS level integration.
Which of course it will, since it’ll be part of Siri.
Edit: also, even if Apple shipped (hypothetically) the exact same product, they would still have a massive advantage by being able to ship it on every single device they sell. Even if Dot succeeds beyond its creators’ wildest dreams, it will still only capture a tiny fraction of the market compared to what Apple would if they shipped it with the OS.
Yes, that's my point, that there are other paths to this being adopted by Apple with on-device processing that are mutually beneficial both to the startup and Apple.
Yeah I would be terrified to stack something I build against Apple just due to the integration level. It’s so easy to get muscled out, they own the walled garden. It’s interesting how they’ve abstracted out of the typical web app access pattern and into the operating system itself, yet they compete seamlessly. It almost feels like a cage to not have their level of access.
Something like this is most certainly going to become the mainstream interface for computing. I think the most likely thing that will hit mass market adoption will look much more like the voice interface in Her than a chat app. I couldn't imagine my mom getting much utility out of an app like this but if the AI is good enough I could certainly see her chatting with her phone as if it was a person.
the problem with voice interfaces is.. How does that work, in public? Late at night when the family is asleep? In noisy environments? During meetings or movies? During conversations with other people?
Movies/tv shows voice interfaces work well because the environment and situations where it happens is 100% controlled and driven by plot. The real world is so much messier...
I see a great opportunity for chat interfaces like this one among younger audience. Every one knows how chat works, they are everywhere. And file systems or even tags are not so ubiquitously understood.
I immediately recognized this as either inspired from or actually from the same creators of the mercuryos concept https://www.mercuryos.com/. Turns out it's the latter.
I love the Mercury OS concept and think it's design both elegantly and sort of subversively packs a myriad of potentially breakthrough ideas.
I have been stewing with ideas around the same vision for years. The idea of a new type of UI where the UI seams to dematerialize, where you directly manipulate the object in your current context (like multi-touch's direct manipulation but at a higher layer of abstraction powered by deep api integrations, intelligent self-assembling relational graphs, and of course ai). For over a decade I've had this thought "the data becomes the UI" like an emergent UI from whatever given data, task or context you are currently in. When I came across the mercuryos concept I immediately smiled.
Conceptually, strategically and technically there are so many challenges to introducing such a new ux paradigm but I'm very happy to see the mercuryos concept has seemed to evolve to New Computer's Dot and I wish them the best!
For those immediately turning to negative sentiment based on privacy or "it's just a gpt4 wrapper" I can see why that could be the knee-jerk reaction but I wouldn't underestimate a sturdy design-philosophy approach like this one. I'd go as far as to make comparisons to Next Computer's NextStepOS. NextStep introduced so many groundbreaking UX concepts and to a large extent I think their personal computing contemporaries underestimated what potential it packed. And, yes, I know the business model and many other factors played into an inevitable doom for Next Computer but there's belief that Steve Jobs may have never intended for Next to become a dominant computing player and instead knew it'd be an irresistible acquisition target in a latent space of UX innovation. It's possible he saw the next evolution of personal computing UX and hedged his bet on not compromising on it. Yet another comparison could be that NextStepOS needed more cpu, graphics and connectivity power to truly display it's heightened level of UX much in the same way something like Dot or mercuryOS would inherently need to leverage cutting-edge computing to truly enable it's vision (obviously LLM's, Vector DB's, etc).
I don't think it ever failed in the sense it never shipped because that was out of scope.
It was never meant to be an end-product. It was a concept/case-study. Happy they're finally realising that vision.
I worked on a project like this. The biggest challenge we had was finding ux patterns that keep returning the user to interact with the app and fill context gaps as it becomes convenient.
I wish those guys lots of luck but I'm not signing up. Excessively logging my life on somebody else's computer is not on my key interests any more.
“Mei is studying for a science exam, and Dot regurgitates a moon landing hoax conspiracy theory. Mei is confused and asks Dot to clarify, which it does with an immediate retraction, followed by a different conspiracy theory from the flat earth society. Dot then enters a hallucinatory loop, converging on a picture of a mutilated alien, eventually running out of resources and crashing. Mei has inadvertently violated the ToS and the service provider terminates her account without notice or any prospect of appeal.”
What can I do with this app that I couldn't do without it? It's just weird to me to turn over such mundane tasks as saving a recipe or looking at my school's listing of singing clubs over to an AI assistant. There may (or, historically, may not) be efficiencies to keeping my entire life in an app, but what's the new thing I can do as a result of this? Keep in mind that it has to be cooler than the lifetime cost of the app plus the loss of privacy (which may be variable, granted).
The kind of tech that underpins this gets a pretty good reception here.
But this page looks slick to a fault. The user story approach feels pandering and I lose interest fairly quickly. Is this closed source? IOS only? I see a waitlist - Why don't they want a general audience to see it?
"Will it? Forever? What are you doing to ensure that?"
Because most consumer startups fail. The ones that get bought out usually end up sunset or neglected. Even the ones that IPO often alienate users by chasing quarterly earnings (evernote/dropbox/pinterest).
If _toxic_ technology and products don’t have a chance for optimism on HN, where do they? Fixed that for you. Also, is this really “technology” through and through? I mean, surely there’s scientific knowledge behind it, but is the materialized outcome really something those two founders really achieved out of applied scientific knowledge? From reading their about page, I doubt they are invested in factoring the neuropsychological, social and cultural impacts, long or short term. It seems more like a narrative they’ve built and liked how it sounded in their head. I’m a designer engineer and I known how tempting it is to prototype such aesthetic pipe dreams, those that you never bothered to honestly and seriously question the validity, since it would obviously ruin all the fun. In those cases it’s easier to let yourself be fooled by your own foolishness. After all, how can it be not good if it looks good and pleases me? It must be good! No, more than that. It is necessary! It’s the way of the future! It has to be done! Right? Has it ever occurred to you that the, hm, “hackers”, might actually have a bit of sense, affording them the ability to distinguish useful technology from bullshit?
All of the negative "how bleak of an existence" comments are completely missing the point (and make me happy for them that they obviously don't have time/task management issues). This sounds incredible for someone with time/task management issues, especially if they are an adult on their own with no one else keeping them accountable.
Can they be patented though? I was having this thought the other day and didn’t really check. I guess not, but it’s kind of ironic as a patent is basically a recipe.
Not typically. The mere combination of known ingredients does not result in a new and non-obvious invention that can be patented. A patent typically covers a unique or non-obvious process. There are exceptions, say, if there is a process that results in a foodstuff having a longer shelf life, or a novel way of reproducing a flavor (but not the recipe of the flavoring, per se). Cooking something with heat is not a unique or non-obvious process.
You could copyright the exact wording but that wouldn't protect the recipe itself and simply substituting a measurement unit may be enough to get around that. You could make it a trade secret but since the onus is on the owner to protect it and keep it confidential, that probably doesn't include publishing it or even sharing it with your AI assistant. Might involve courtside arguments about "reasonable expectations of privacy" .. I wouldn't want to test it.
It's not like there's specific wording saying that recipes cannot be patented, but if you can describe it in the traditional ingredients + preparation steps then it does not meet patentable criteria.
Why talk to your grandmother, teacher, or schoolmates, when you can talk to a robot instead? I'm unsure what problems this tool is trying to solve, and I'm worried that it enables isolation instead of conviviality.
Looks cool! I've been using & contributing to Lightrail (https://github.com/lightrail-ai/lightrail). It's more dev-focused but hopefully, it's developing in a similar direction (long-term memory/context, integrations, etc) while still being local-first / OSS. I definitely think we're heading for a future where persistent AI assistants play a big role, and I'm really hoping the ones that win out are more open & private!
Pi is less brainy and better at conversation, in my experience. It gets details wrong but I had a conversation with it for a couple hours without getting bored, while ChatGPT has the personality of an oyster. Pi's also pretty sycophantic, but somehow was also skeptical when I tried to lie to it, which was interesting.
It's truly stunning to me that in an era where we're all suffering from an epidemic of loneliness, that the smartest among us are just completely obsessed with replacing every last human interaction with a computer. This thing is saying things like, "Your grandma must be a great chef!" and "The Treblemakers are lucky to have you. How about celebrating by bringing some of grandma's scallion flatbread? It'd be a great to share a piece of home with your new community!" The whole thing is a narrative that basically ends with Mei sharing life events w/ Dot for basically no reason other than Dot is her friend. The least realistic thing in the entire demo is when Mei asks Dot to help her find community and Dot suggests people, and not other AI bots.
I'm fine with assistants, but assistants that insinuate their way into your life like a friend or a trusted companion are fuckin weird as hell. There are so many sci-fi movies/series where there's a loner and a pretty advanced AI, and you know how they literally all end? With the loner finding friends! This stuff sucks; it's extremely not rock and roll. This product and others like it are deeply unhealthy and inhuman. Your most intimate relationship shouldn't be with a computer that remembers everything about you and lives to serve you.
FYI to the website designer: on desktop, this website does not have a scrollbar. In addition to that being an accessibility issue, I closed the website after I got tired of paging and/or using the scroll wheel.
Sorry, there are no credits left in your Vercel v0 account. To make a modification to the generated UI, please upgrade to the premium tier for $50/month.
This is really interesting not just as an app per se, but as a new human-machine interface.
This should be great as a team working tool, where you can ask if something is done, how were done, what parts are missing, retrieve what is done, etc.
Integrating this in a ecosystem like Google Apps, Office 365, or similar would be great in order to being able to access that information inserted in other ways, with the pluses of email, calendar, and sync services.
I’m not sure why someone would want to trust an AI chat startup with holding onto grandma’s special recipe. 95% of these AI startups are not going to exist in 5 years as the market sorts out how AI is used and which companies win and lose. None of the offerings coming out right now should be used for data archival of any kind. Anything put into these AI chat bots should be considered throw away.
I'm really intrigued by the idea of something like this as a desktop OS. I tend to dump files into one location and use search to pull up what I need, rather than a predetermined structure, and this seems like a good fit for me.
Putting aside the obvious data sovereignty / privacy problems of such 'lifelogs';
> And when it's time to buy her textbooks, Dot is there to help. "[... Bookends and Beginnings is] a highly rated bookstore that's a 25 minute walk from your current location."
In what future does your AI-powered product exist but there are still dead-tree textbooks!? Even if Z-Library is off the table for "legal reasons", you're talking about a digital Aristotle. Have some vision for heaven's sake.
I'm guessing it'll hallucinate things about you that it doesn't know. This might be a mixed blessing: it could introduce you to things you weren't aware of, but it could also be annoyingly inaccurate.
At least everybody knows humans are fallible. We expect computers to be better than that, or at least less fallible. And when they do fail, we expect to be able to determine where and why. That's not often the case with today's AI.
This really approaches what a digital personal assistant should be like. But there are two things holding me back from using it for the long term: third-party application privacy and non-locally running AI models.
I wonder if Obsidian, with ChatGPT API attached to it can achieve the same goal. Sure, metadata and folders and stuff will have to be set up properly, but you control the data at the end of the day.
"Mei asks where has the scrollbar gone on the desktop version of the website. Dot responds, `What scrollbar?`. Mei promptly closes the page and never returns".
“2,5 years later, Mei receives an email from Dot’s CEO that Dot will be ‘joining forces’ with Salesforce as their ‘incredible journey’ comes to an end.”
