>One issue is ensuring the device only chimes in when useful, preventing it from talking too much or not knowing when to finish the conversation—an ongoing issue with ChatGPT.
This device that just sits and listens all the time and interrupts ... that's nothing like I use LLMs now. That's a completely different mode of operation / my tolerance for mistakes would be near 0 with that.
It's not clear to me if an LLM can even DO these things conversationally well enough to work / not be a nuisance.
Siri might be the best example, you use it a few times, it's useless a few times and then I just use it to set timers and forget about it otherwise.
> and then I just use it to set timers and forget about it otherwise.
Much as the Apple Watch, in practice, was really a good fitness tracker, and kinda pointless for everything else, the natural place for voice assistant gadgets is setting timers. I'm fairly sure that's all that 99% of Alexa users use their talking cylinder for, say.
Also, isn't this just that Humane.ai gadget? And there was another one, IIRC. They were all the rage as CES 2024 IIRC.
Just imagine the unlimited utility of your device pinging at random points throughout the day to declare "Ah, this is a classic problem with <SUBJECT>" who wouldn't need that?
See, this is the sort of thing that makes me suspect that maybe I'm in hell (in the Good Place or Iain Banks' Surface Detail mode rather than the devils-poking-people-with-pitchforks one); that, as a concept, could only possibly exist to torture people.
I still can't understand my my HomePod is constantly responding (uselessly, often just saying it can't send a message for me) when I am talking to my phone that is in my hand. It even happens when I have AirPods in my ears. How hard is it to have a priority listing for:
1: AirPods-linked devices go first when AirPods are present
2: closest device goes first in general
3: if there are multiple devices that could respond, and only some of them can do the requested task, one of the devices that can do the task should respond
I had the same problem but in reverse. I’d want to play a song on the HomePod but my iPad sitting on the coffee table would start playing it on its tinny little speaker. The solution was to set Siri on the devices to not respond to “Siri”. To use Siri on the handheld devices I press and hold the side button until I get the ugly gradient border. It’s pretty bad that Apple released this feature in such a uselessly broken state.
> Multiple people familiar with the plans said OpenAI and Ive were working on a device roughly the size of a smartphone that users would communicate with through a camera, microphone and speaker. One person suggested it might have multiple cameras.
Finally, someone has, er, invented the smartphone? Except without the most useful bit?
Why do people keep trying to make special-purpose LLM gadgets? What on earth is wrong with just having an app on a phone?
> Their aim is to create a palm-sized device without a screen that can take audio and visual cues from the physical environment and respond to users’ requests
A fob on a lanyard would work well I think. It would be nerdy as hell, but in a practical sense, it's better than carrying around another phone-like thing or wearing something on your face.
If drones weren't incredibly noisy and battery wasn't so scarce, a floating personal assistant orb w/ video and mic input and a speaker/audio output, sort of a modern techno take on the fairy Navi from the series Zelda would be great.
It feels like there's nothing really new to say about what's proposed in this report that hasn't already been said about the Humane, Friend, and R1 except "except Sam & Jony".
This "article" is a collection of vague quotes from "one person" supposedly "briefed on the plan" about what an hypothetical device in development is allegedly doing. That’s pretty thin.
> whose alluring designs of the iMac, iPod, and iPhone helped turn Apple into one of the most valuable companies in the world
"Helped" is doing a lot of heavy lifting in that sentence. I'm still not sure if those these things did well because of Jony or due to Steve. What we saw from Jony after Steve (few ports, butterfly keyboard, thinness at the expense of everything else) makes me think that without a good editor (of which Altman may or may not be) Jony isn't able to create greatness on his own. I also find Jony's ventures post-Apple to be laughable and not interesting in the slightest.
> “The concept is that you should have a friend who’s a computer who isn’t your weird AI girlfriend... like [Apple’s digital voice assistant] Siri but better,”
Uh, ok, that sentence took a weird turn. I'm not sure if anyone has ever called Siri a "weird AI girlfriend", I've never seen it marketed that way or people using it that way. Now, the OpenAI API, on the other hand, has clearly helped enable whole slews of AI Girl/boyfriends. Odd dig to throw at Apple here.
I'm sure whatever they unveil will be interesting, just not clear if it will have the same fate as R1/Humane/Friend. Personally, I hope they fail based on the video they released when OpenAI first bought "io", cringe does not begin to describe it.
I read that sentence as saying that Siri is an example of something that ISN'T a "weird AI girlfriend": that what they want is Siri but better, as opposed to something that isn't even named in that sentence (but likely is trying to be a dig at Grok, but is failing to do so as, as you say, people consider ChatGPT's own products to have a bit of this issue).
I do give credit to Steve Jobs. I was saying that what Jony has done on his own is less impressive IMHO.
If they manage to pull this off (OpenAI/io) then I will, begrudgingly, have to give credit to Altman, despite not being a huge fan of him (then again, Jobs wasn't a "good guy" in many aspect of his life either).
My point being, Jony needs an editor, when he was unchecked at Apple (or his own design company "Love From") I found that much of his output came across as an exercise in smelling one’s own farts.
> One person said the device would be “always on” rather than triggered by a word or prompt. The device’s sensors would gather data throughout the day that would help to build its virtual assistant’s “memory.”
That sounds like a terrible idea.
It's hard to say, but I like to think that something like the iPhone always sounded like a good idea. Basically, the entire web in your pocket. Check football scores while you're on the john, text someone instead of calling, GPS, etc. There were some unknowns, like can we get touch screen to work well enough, will people pay $600 for a phone when typical phones cost $100-200 at the time. But the product itself was obviously good.
But this is some puck that records everything and periodically chimes in? Sounds awful. What problem is this trying to solve? iPhone solved the problem of camera, entertainment and phone in one device. Is anyone walking around thinking gee I wish I had some AI talk to me right now but I don't want to press a button?
Here’s an example of an AI that interrupts people: turn-by-turn directions. The interruptions are tolerated because the info is so timely and useful. Heck, drivers will shush friends and family to hear the AI instead.
So that gives you a sense for the bar for other AI interruptions. It had better be very timely and very valuable if I’m going to accept an AI interruption. That probably means the interruptions are rare in general, at least at first.
The iPhone was popular from the moment it was introduced, but people back then did not really think about it the way we do now. The biggest cheers were for a phone (one big demo was voicemail LOL), and a “widescreen iPod.” There was tepid confused applause for it as a “break-through Internet communications device.” Which is 99% of how everyone uses it today.
I've turned the GPS audio prompts off. And I've never understood why there isn't a "fewer interruptions" slider for GPS directions, or "last mile only" mode for GPS audio. Or why I have to go into settings to turn GPS audio off and on. If GPS audio is going to be annoying and un-customizable, it should at least be a quick flip of a switch so I can make those choices myself. As it is, it's a global.
How on earth are they going to manage battery life for an always on device? At this point they might as well just rig up a cheap 2-way short range radio, with a microphone always on and connected to OpenAI HQ.
Screens are easily the most useful way to communicate complex information. Why would getting rid of them be beneficial at all? What does the device want to accomplish that something with a screen couldn't immediately do better?
Voice is inherently linear, even the most basic voice only menus are tedious to navigate.
Holy crap! Hugely overhyped and oberpriced AI pin thing like the last ai pin thing but better because different, faces all the same limitations of old pin thing just like every sane person predicted.
So crazy how this works.
>One issue is ensuring the device only chimes in when useful, preventing it from talking too much or not knowing when to finish the conversation—an ongoing issue with ChatGPT.
This device that just sits and listens all the time and interrupts ... that's nothing like I use LLMs now. That's a completely different mode of operation / my tolerance for mistakes would be near 0 with that.
It's not clear to me if an LLM can even DO these things conversationally well enough to work / not be a nuisance.
Siri might be the best example, you use it a few times, it's useless a few times and then I just use it to set timers and forget about it otherwise.
> and then I just use it to set timers and forget about it otherwise.
Much as the Apple Watch, in practice, was really a good fitness tracker, and kinda pointless for everything else, the natural place for voice assistant gadgets is setting timers. I'm fairly sure that's all that 99% of Alexa users use their talking cylinder for, say.
Also, isn't this just that Humane.ai gadget? And there was another one, IIRC. They were all the rage as CES 2024 IIRC.
Just imagine the unlimited utility of your device pinging at random points throughout the day to declare "Ah, this is a classic problem with <SUBJECT>" who wouldn't need that?
See, this is the sort of thing that makes me suspect that maybe I'm in hell (in the Good Place or Iain Banks' Surface Detail mode rather than the devils-poking-people-with-pitchforks one); that, as a concept, could only possibly exist to torture people.
It sounds like this device could become the iPod Shuffle of AI devices— cute and clever, but not particularly useful.
I still can't understand my my HomePod is constantly responding (uselessly, often just saying it can't send a message for me) when I am talking to my phone that is in my hand. It even happens when I have AirPods in my ears. How hard is it to have a priority listing for:
1: AirPods-linked devices go first when AirPods are present
2: closest device goes first in general
3: if there are multiple devices that could respond, and only some of them can do the requested task, one of the devices that can do the task should respond
I had the same problem but in reverse. I’d want to play a song on the HomePod but my iPad sitting on the coffee table would start playing it on its tinny little speaker. The solution was to set Siri on the devices to not respond to “Siri”. To use Siri on the handheld devices I press and hold the side button until I get the ugly gradient border. It’s pretty bad that Apple released this feature in such a uselessly broken state.
Ditto. I tell my Apple Watch to set an alarm and my HomePod — in another room, 30 feet away — says "I've set your alarm." FAIL
Looks like the usual fad that never will never pass the reality check.
It could have been great for them to work on something like smart glasses, a competitor to Meta one with AI. Or a nice AI super-smart watch.
> struggling with […] privacy
Yeah I can imagine that a device which records everyone around you, all the time, struggles with that.
> Multiple people familiar with the plans said OpenAI and Ive were working on a device roughly the size of a smartphone that users would communicate with through a camera, microphone and speaker. One person suggested it might have multiple cameras.
Finally, someone has, er, invented the smartphone? Except without the most useful bit?
Why do people keep trying to make special-purpose LLM gadgets? What on earth is wrong with just having an app on a phone?
> Their aim is to create a palm-sized device without a screen that can take audio and visual cues from the physical environment and respond to users’ requests
So… a ‘phone’ - but no screen.
There is no hardware format I can imagine that would make sense for something like this.
Maybe that's just my limited imagination, but since no one has made anything useful in this space...
A fob on a lanyard would work well I think. It would be nerdy as hell, but in a practical sense, it's better than carrying around another phone-like thing or wearing something on your face.
If drones weren't incredibly noisy and battery wasn't so scarce, a floating personal assistant orb w/ video and mic input and a speaker/audio output, sort of a modern techno take on the fairy Navi from the series Zelda would be great.
The insta360 content-creator camera kind of functions like this but, instead of floating, people hold it away from their body on a stick all day
The fairy Navi who is famously annoying and who everyone wishes they could literally murder?
It feels like there's nothing really new to say about what's proposed in this report that hasn't already been said about the Humane, Friend, and R1 except "except Sam & Jony".
is it weird they're running TV commercial with Jony Ive even though the product doesn't exist yet?
This "article" is a collection of vague quotes from "one person" supposedly "briefed on the plan" about what an hypothetical device in development is allegedly doing. That’s pretty thin.
> whose alluring designs of the iMac, iPod, and iPhone helped turn Apple into one of the most valuable companies in the world
"Helped" is doing a lot of heavy lifting in that sentence. I'm still not sure if those these things did well because of Jony or due to Steve. What we saw from Jony after Steve (few ports, butterfly keyboard, thinness at the expense of everything else) makes me think that without a good editor (of which Altman may or may not be) Jony isn't able to create greatness on his own. I also find Jony's ventures post-Apple to be laughable and not interesting in the slightest.
> “The concept is that you should have a friend who’s a computer who isn’t your weird AI girlfriend... like [Apple’s digital voice assistant] Siri but better,”
Uh, ok, that sentence took a weird turn. I'm not sure if anyone has ever called Siri a "weird AI girlfriend", I've never seen it marketed that way or people using it that way. Now, the OpenAI API, on the other hand, has clearly helped enable whole slews of AI Girl/boyfriends. Odd dig to throw at Apple here.
I'm sure whatever they unveil will be interesting, just not clear if it will have the same fate as R1/Humane/Friend. Personally, I hope they fail based on the video they released when OpenAI first bought "io", cringe does not begin to describe it.
I read that sentence as saying that Siri is an example of something that ISN'T a "weird AI girlfriend": that what they want is Siri but better, as opposed to something that isn't even named in that sentence (but likely is trying to be a dig at Grok, but is failing to do so as, as you say, people consider ChatGPT's own products to have a bit of this issue).
Ahh, that is almost certainly what they meant, I just parsed it wrong. Thank you!
I really don't get the cult of Jony. As you touched upon, without Steve, the design decisions were baffling.
but have you considered the trashcan mac?
If not the designer and/or CEO, who are you going to give credit to?
I do give credit to Steve Jobs. I was saying that what Jony has done on his own is less impressive IMHO.
If they manage to pull this off (OpenAI/io) then I will, begrudgingly, have to give credit to Altman, despite not being a huge fan of him (then again, Jobs wasn't a "good guy" in many aspect of his life either).
My point being, Jony needs an editor, when he was unchecked at Apple (or his own design company "Love From") I found that much of his output came across as an exercise in smelling one’s own farts.
> One person said the device would be “always on” rather than triggered by a word or prompt. The device’s sensors would gather data throughout the day that would help to build its virtual assistant’s “memory.”
That sounds like a terrible idea.
It's hard to say, but I like to think that something like the iPhone always sounded like a good idea. Basically, the entire web in your pocket. Check football scores while you're on the john, text someone instead of calling, GPS, etc. There were some unknowns, like can we get touch screen to work well enough, will people pay $600 for a phone when typical phones cost $100-200 at the time. But the product itself was obviously good.
But this is some puck that records everything and periodically chimes in? Sounds awful. What problem is this trying to solve? iPhone solved the problem of camera, entertainment and phone in one device. Is anyone walking around thinking gee I wish I had some AI talk to me right now but I don't want to press a button?
Here’s an example of an AI that interrupts people: turn-by-turn directions. The interruptions are tolerated because the info is so timely and useful. Heck, drivers will shush friends and family to hear the AI instead.
So that gives you a sense for the bar for other AI interruptions. It had better be very timely and very valuable if I’m going to accept an AI interruption. That probably means the interruptions are rare in general, at least at first.
The iPhone was popular from the moment it was introduced, but people back then did not really think about it the way we do now. The biggest cheers were for a phone (one big demo was voicemail LOL), and a “widescreen iPod.” There was tepid confused applause for it as a “break-through Internet communications device.” Which is 99% of how everyone uses it today.
I've turned the GPS audio prompts off. And I've never understood why there isn't a "fewer interruptions" slider for GPS directions, or "last mile only" mode for GPS audio. Or why I have to go into settings to turn GPS audio off and on. If GPS audio is going to be annoying and un-customizable, it should at least be a quick flip of a switch so I can make those choices myself. As it is, it's a global.
How on earth are they going to manage battery life for an always on device? At this point they might as well just rig up a cheap 2-way short range radio, with a microphone always on and connected to OpenAI HQ.
I think advertising is the problem being solved.
If they can make it remotely useful enough to allow you to record all of your conversations, that is extremely valuable to advertisers.
And people thought MS Recall was bad.
why could this gadgets functionality not be integrated into the chatgpt app? im asking rhetorically.
This is going to be dead on arrival.
Before arrival.
Screens are easily the most useful way to communicate complex information. Why would getting rid of them be beneficial at all? What does the device want to accomplish that something with a screen couldn't immediately do better?
Voice is inherently linear, even the most basic voice only menus are tedious to navigate.
Holy crap! Hugely overhyped and oberpriced AI pin thing like the last ai pin thing but better because different, faces all the same limitations of old pin thing just like every sane person predicted. So crazy how this works.