00:00:00 ◼ ► There is a way to get John Siracusa to leave his house. For those of you who remember, in November, I traveled up, I made a multi-hundred mile train trip to go see Marco and try the Vision Pro prior to release at a lab. And I said to John in advance, "Hey, I know you're not going to the lab, but wouldn't it be neat if you came down to New York and visited with us? I think that would be a lot of fun." And John told me to go outside and play hide and go screw myself.
00:00:30 ◼ ► Yeah, those words precisely. And so I thought there was no possible way to get John to leave his house, even for his beloved friends of years and co-workers, of years and years and years.
00:00:41 ◼ ► But it turns out, John, there is a way to get you out of the house, and I am happy to report the ATP reunion, for the first time since 2019, is a go. John, what's going on?
00:01:02 ◼ ► Indeed, I am extremely, extremely, extremely excited. I cannot overstate how excited I am, especially to see the two of you fellas, but also to be able to go to Apple Park.
00:01:10 ◼ ► The only place I've ever been on Apple Park is the visitor center, the public visitor center that you don't need any special privileges to get to.
00:01:17 ◼ ► I'm excited that we are press for the purposes of this event, which I'm really, really excited about.
00:01:28 ◼ ► And I've been through my badges to see if I can find a press badge. Maybe I have, and it's just been so darn long and I don't recall knowing me.
00:01:33 ◼ ► That's probably true. But one way or another, it's happening again, and I am so excited.
00:01:38 ◼ ► So I am glad that you are willing to make the trek instead of just a couple of convenient hours down the interstate and/or train from Boston to New York.
00:01:51 ◼ ► Yeah, I mean, I have done press WWDCs. We've all attended many WWDCs, but I've never been to one at Apple Park, and neither has Casey.
00:02:04 ◼ ► And I think they don't let you take real cameras, which is kind of a bummer, so I'll just take lots of pictures with my phone. But whatever.
00:02:13 ◼ ► Looking forward to not looking forward to taking the plane flight, but you know, you do what you have to.
00:02:19 ◼ ► Alright, let's do a little follow-up. I wanted to briefly call attention to, there's some new Vision Pro content.
00:02:31 ◼ ► So there's three things I wanted to call everyone's attention to, two of which are from Apple and one of which is not.
00:02:37 ◼ ► First of all, there is a new sizzle reel. I don't know if that's the bestest way of describing it, but basically it's like a three and a half minute video that gets you interested in the Vision Pro.
00:02:48 ◼ ► And previously, I hadn't watched it in a couple of months at least, but previously my recollection is, or the way I remember it was, it showed like a little bit of the kids playing soccer, rhinoceroses.
00:02:59 ◼ ► It showed the tightrope walker, and I think it showed a very brief bit of sports, if memory serves. I don't entirely remember.
00:03:08 ◼ ► But one way or another, there's a new one, and I think it's really well done. It's still, it's a little jumpy for my taste, just a touch.
00:03:15 ◼ ► It's not like that MLS thing from a month or so back where it was way too jumpy. It's just a touch jumpy.
00:03:21 ◼ ► But the soundtrack is excellent, and I really, really like it. Again, three and a half minutes. You can find it in the Apple TV app.
00:03:28 ◼ ► And since it's new, it is kind of front and center. I know I was complaining and moaning about the information architecture last week.
00:03:39 ◼ ► Additionally, there's a new, I believe it's the Adventure series. This is the one that had the tightrope walker on it.
00:03:47 ◼ ► There's a new episode of that all about parkour, and it's three, I believe Brits, based on accents, although who knows.
00:04:01 ◼ ► And it's really, really well done. It's like 12 to 15 minutes long somewhere in that neck of the woods.
00:04:11 ◼ ► And I got to tell you, no spoilers for the 12-minute video, but there's a point at which the three gentlemen are trying to jump from one rooftop to another.
00:04:20 ◼ ► And they position the camera such that if you want, you can look down and see how tall it is.
00:04:26 ◼ ► And I got to tell you, when it's 3D and immersive, it looks scary as hell. So it's pretty, pretty cool.
00:04:34 ◼ ► And I think both of these, I mean, altogether, these two things are worth literally 15 minutes of your time. I really do think it's pretty cool.
00:04:41 ◼ ► And then finally, What If has finally launched. I had planned to do the whole darn thing earlier today and report in on it.
00:04:49 ◼ ► Unfortunately, they seem to release on Pacific time, so it wasn't available until my afternoon.
00:05:00 ◼ ► The premise here, no spoilers, is you're in the Marvel Cinematic Universe, and what if things weren't the way they were in the movies?
00:05:09 ◼ ► What if things went a little awry and different? And it's your responsibility as part of the story to try to fix all this.
00:05:16 ◼ ► And so the portion that I've done so far, again, the first 10 minutes or so, is they teach you how to cast some spells.
00:05:23 ◼ ► They interact with you, so it begins as fully immersive, you can't see any of your own environment, and then it converts to augmented,
00:05:32 ◼ ► such that there are a couple of characters in your space. And I don't know if it was an extremely happy accident or if it was deliberate,
00:05:38 ◼ ► but I was doing this in my office, and I had spun my chair around so my desk and monitors were behind me.
00:05:44 ◼ ► And my office is, I don't know, 12 feet by 12 feet, something like that, so it's something like 4 meters by 4 meters.
00:05:49 ◼ ► Not terribly big. Yeah, maybe that's right. I don't know, it doesn't matter. Not big is the point.
00:05:54 ◼ ► And there were a couple of characters, one is designed to be floating in the air, and sure enough, he was floating in the air,
00:05:58 ◼ ► and his feet were kind of inside my guest bed, which is behind my desk. But that's fine, he's magic, so whatever.
00:06:04 ◼ ► But then the other character was designed to be on the ground, and sure enough, he was on the guest room floor.
00:06:12 ◼ ► Like he was standing on the floor, there was a little shadow below him, and it looked just spot on.
00:06:16 ◼ ► And then they had you do some, you go back into an immersive environment where all you can see is what they're showing you,
00:06:23 ◼ ► and they have you do some spells where you're using hand tracking in order to do stuff.
00:06:28 ◼ ► And there were three or four that I learned, and one of them didn't really work that well, but the rest were spot on.
00:06:37 ◼ ► So definitely, definitely check all three of these out. Again, that's the new sizzle reel.
00:06:45 ◼ ► And then the parkour, the adventure series, the second episode about parkour and what if by Marvel.
00:06:53 ◼ ► And I've put a link to the what if thing in the show notes, assuming I can dig one up, I should be able to.
00:06:57 ◼ ► I don't think I can link to the other ones in the show notes, but if I find a way, I will do so.
00:07:27 ◼ ► but they apparently spotted two 6 gigabyte RAM modules on the 256 gig and 512 gig 13-inch iPad Pro.
00:07:36 ◼ ► "How did they spot that these were 12 gig modules? I can barely make out the part numbers.
00:07:45 ◼ ► "Our chip ID confirms this with high certainty. Two 6 gigabyte LPDDR5X modules produced by Micron for a total of 12 gigabytes of RAM."
00:08:02 ◼ ► but there's apparently 12 gigs on the chips there. So why does Apple utilize only 8 gigs of RAM?
00:08:12 ◼ ► Someone asked if it's possible these RAM chips are defective and some of the RAM is disabled for some reason.
00:08:29 ◼ ► Another possible question, "What if those are the only LPDDR5X modules they can get their hands on,
00:08:34 ◼ ► but as soon as they're able to put in 4 gig ones, they will, so just to avoid problems,
00:08:44 ◼ ► according to the spec sheets, so it seems unlikely based on how long they've been around.
00:08:53 ◼ ► If you follow the Twitter thread about this, you can see people throwing out a bunch of theories
00:09:05 ◼ ► And if they couldn't get smaller RAM chips, why wouldn't they just free up the whole 12 gigs?
00:09:09 ◼ ► The other theory is like, "Oh, they're reserving a certain amount of RAM for LLM stuff."
00:09:17 ◼ ► Because I think this is the only one that has more RAM and still all the other ones have the amount
00:09:34 ◼ ► They'll just install it and not enable it for you. But it's there, you just can't use it.
00:10:11 ◼ ► "It seems like I'm not the only one experiencing this, though I have yet to determine the exact issue.
00:10:26 ◼ ► Yeah, it's probably PWM. People will come into a forum or a Reddit or whatever and say,
00:11:05 ◼ ► for years without any issue, or maybe tandem OLED is misaligned if there is such a thing.
00:12:17 ◼ ► so they tend to not want to do that for the lower brightnesses. The other way you can control
00:12:23 ◼ ► brightness on an OLED is you can have the screen be at maximum brightness briefly and then go off,
00:12:29 ◼ ► and then max brightness and then off, and the longer the "on" period is, the brighter it is.
00:12:39 ◼ ► "If you just pulse the screen, pulse, pulse, pulse, pulse, pulse, the faster the pulses
00:12:43 ◼ ► go the brighter the screen. If you go pulse, pulse, pulse, it is dimmer because the light is on
00:12:49 ◼ ► less of the time." You don't notice this because Apple screen, this TANO OLED, pulses at 480 times
00:12:55 ◼ ► a second, which is a pretty high refresh rate if you remember from the CRT days. Like, "Oh,
00:13:00 ◼ ► my screen looks flickering at 60 hertz. I can see the flicker, but at 85 I can't see it anymore.
00:13:04 ◼ ► At 120 I definitely can't see it. At 480 I can tell you I cannot see. Well, it's flickering."
00:13:11 ◼ ► But the complaint about the M4 iPad Pro TANO OLED is that it uses pulsing to control its brightness
00:13:20 ◼ ► through its entire brightness range apparently. Even at maximum brightness it's still pulsing,
00:13:29 ◼ ► but once they get to high brightnesses they will do that by keeping it on all the time but
00:13:32 ◼ ► just sending less voltage. I don't know if this is just the way TANO OLEDs work. Is this the way
00:13:37 ◼ ► Apple is choosing to make it work? But some people report that this bothers them. I don't
00:13:42 ◼ ► personally see how it could because I am not aware of anyone who would notice flickering at 480 hertz.
00:13:49 ◼ ► We're not even talking about motion here. We're just saying like put on just a full field,
00:13:52 ◼ ► you know, red slide. I can't see it flickering at 480 hertz. If you had asked me whether this is
00:13:59 ◼ ► flickering to control brightness, I would have said no because I can't see it. But maybe people
00:14:04 ◼ ► with very young or better eyes than mine can see it. I don't know. Or maybe it's... Anyway,
00:14:09 ◼ ► follow the links, decide for yourself. And by the way, I would say do iPhones use this or do they
00:14:15 ◼ ► use voltage regulation? I honestly couldn't tell you because it just looks like a screen to me and
00:14:20 ◼ ► I can't see whether it's dimming or pulsing at 480 hertz. Some of the OLEDs that are out in the
00:14:26 ◼ ► market pulse at even higher rates than that. So yeah. Is this an issue? Is this something
00:14:31 ◼ ► people are imagining? Are some people just uniquely sensitive to it? All I can tell you is that
00:14:35 ◼ ► my old man eyes don't see this and aren't bothered by it. But we'll see. We'll see if Apple does an
00:14:41 ◼ ► update. We didn't put this in the notes for the last episode, but there is some kind of
00:14:44 ◼ ► actual software error with displaying HDR video where some like highlights are getting blown out
00:14:53 ◼ ► fix was coming for. So it is possible that maybe whatever issue people are complaining about here
00:14:58 ◼ ► will also be wrapped up in that fix, but we'll see. Michael Thompson writes in with regard to
00:15:03 ◼ ► trillions of operations per second measurements or TOPS measurements. Michael writes, "I found
00:15:08 ◼ ► this article on the Qualcomm website that suggests that the TOPS measurement they use for their NPU
00:15:12 ◼ ► performance is based on 8-bit integers. In the paragraph headed 'precision,' they state,
00:15:16 ◼ ► 'The current industry standard for measuring AI inference and TOPS is at 8 precision.' The context
00:15:25 ◼ ► here being whether or not the new surface line and the... What is it? The copilot plus PC or whatever
00:15:31 ◼ ► it is? Line of PCs. Are they or are they not actually faster for neural related things than
00:15:38 ◼ ► Apple stuff? And so I don't recall what was... Does this mean they are faster than I presume?
00:15:45 ◼ ► Well, I mean, so this matches what we saw. The idea that on this website they're saying Qualcomm
00:15:49 ◼ ► is saying the industry standard is 8-bit precision, right? And what we've seen is Apple is using
00:15:55 ◼ ► int8 and so is Qualcomm and so is Microsoft and everybody who's current talking about their
00:16:03 ◼ ► how many 8-bit things can you process at once, right? If I ask you how many 16-bit things you
00:16:09 ◼ ► can process, it's half as many, right? And so the number is smaller. Previously, unlike earlier,
00:16:15 ◼ ► you know, Apple Silicon stuff, Apple was using 16-bit stuff. And so their numbers were half as
00:16:19 ◼ ► big, but with their new stuff, Apple is using numbers that are twice as big and they're using
00:16:29 ◼ ► int8 is more representative of the actual jobs we're asking our NPUs to do? Maybe, but you know,
00:16:34 ◼ ► whatever. The industry has decided when measuring tops, we're going to use int8 precision. That may
00:16:40 ◼ ► become less relevant if it turns out that the things we ask our NPUs to do involve 16-bit or
00:16:50 ◼ ► but I would trust that int8 is actually a relevant measure right now. So the answer is,
00:16:54 ◼ ► you know, the Copilot Plus PCs and the Snapdragon X elite thing has 40 tops. The M4 has 38. Those
00:17:03 ◼ ► are both int8 measures. That means they're essentially comparable. Eric Jacobson writes
00:17:08 ◼ ► in with regards to iCloud Drive and node modules. So if you recall, this was with John Sun,
00:17:16 ◼ ► who basically nuked his MacBook Air by trying to sync the node modules folder through iCloud
00:17:22 ◼ ► Drive. Eric writes, "I haven't used it since I don't use iCloud Drive, but there's a project
00:17:26 ◼ ► that will add a no sync directive to every node modules on a file system. I imagine it might need
00:17:32 ◼ ► to be rerun whenever a new project is kicked off." And we'll put a link in the show notes to
00:17:35 ◼ ► nosync-icloud. Yeah, and I tried to look at the code to remind myself how I think you make like
00:17:40 ◼ ► a directory with a dot nosync extension that has the same name as the other one. Like that's the
00:17:44 ◼ ► way you signal to iCloud Drive not to sync the directory or something like that. This is a node
00:17:49 ◼ ► module itself, so you can look at the code, but unfortunately the documentation is all in,
00:17:52 ◼ ► what are we going to say here? I'm going to say Chinese? Yeah, something like that. Yeah,
00:17:56 ◼ ► so the documentation is in Chinese and I can't read it. But the source code is not in Chinese
00:18:00 ◼ ► and I still couldn't quite make any sales of it. But yeah, I think it's just making dot nosync
00:18:05 ◼ ► directories in the right places. And it's a node module that you can use and it will, you know,
00:18:08 ◼ ► I think you just include it in your project and make sure everything nosyncs. So that's,
00:18:12 ◼ ► you know, useful and helpful if you want to dare to walk that tightrope of trying to use node modules
00:18:17 ◼ ► with iCloud Drive. Indeed, and then Eric continues, I do however use time machine and can attest that
00:18:22 ◼ ► the Asimov utility works perfectly for excluding node modules and other dependency directories.
00:18:27 ◼ ► Also it is a background service, so it doesn't need to be reinitialized. And we will again put
00:18:31 ◼ ► a link to the show notes to Asimov. And also, John, I guess you wanted to call attention to the
00:18:36 ◼ ► list of excluded directories, which I put in the show notes as well. Yeah, it shows what kind of
00:18:40 ◼ ► things like when it says dependencies, what does that mean? Obviously it means node modules, but
00:18:43 ◼ ► it has a whole list of all the different things that excludes things from Gradle, Bower, PyPy,
00:18:50 ◼ ► NPM, Parcel, Cargo, Maven. I think CocoaPods might be, yeah, CocoaPods is in there. Like Marco,
00:18:55 ◼ ► have you heard of any of those things other than CocoaPods? Zero, precisely zero. Flutter. Anyway,
00:19:01 ◼ ► it's sad that I think I've heard of all of these. But I installed this and I ran it. It installs a
00:19:08 ◼ ► little launch daemon thing or whatever. And it essentially does the... I forget what the time
00:19:15 ◼ ► machine is. I think it's an extended attribute or something, but there's a way to exclude things
00:19:19 ◼ ► from time machine. Maybe it just calls tmutil. Yeah. Anyway, excluding all these directories
00:19:24 ◼ ► from your time machine backups can make your time machine backups go faster. What it's basically
00:19:28 ◼ ► saying is you don't need to back up the dependencies of your code. If you're writing something in node
00:19:34 ◼ ► and you use 17 node modules, you don't need to back those up. You get them from the internet
00:19:37 ◼ ► anyway. You got them through NPM or YARMM or whatever. They're on the internet. Do not back
00:19:41 ◼ ► them up. That's not your code. It's a dependency. You didn't write that code. It's just pulling it
00:19:45 ◼ ► in. And there are tons of files. So if you can exclude those directories from time machine,
00:19:49 ◼ ► it will make your time machine backups go faster. But who remembers, oh, what am I going to do? Go
00:19:53 ◼ ► to options and time machine and drag the little thing in or set the extended attribute. I don't
00:19:57 ◼ ► remember how to do this. This just runs in the background all the time. Looks for directories
00:20:01 ◼ ► that fit this signature and excludes them from time machine. So I did that. I probably saved,
00:20:06 ◼ ► I don't know, thousands, many thousands of files are no longer in my time machine backups
00:20:15 ◼ ► important files from my time machine backups. But I've got multiple backup strategies. So for now,
00:20:20 ◼ ► I'm trying the experiment of running this asmoff daemon in the background to see if it helps with
00:20:28 ◼ ► - And then finally, this is actually, I should have moved this up by the other Vision Pro follow-up.
00:20:33 ◼ ► Didn't think about it. But anyway, Jonathan Gobranson writes with regard to audio routing
00:20:37 ◼ ► during Vision Pro guest mode. So if you recall, I was doing demonstrations for my mom and dad.
00:20:42 ◼ ► And I noticed that when mom was on Mount Hood or whatever it's called, and I had her go fully
00:20:47 ◼ ► immersive, that the crickets and whatnot were being routed through my iPad Pro, which was doing
00:20:54 ◼ ► mirroring at the time. And so it doesn't really make for a very good effect if the audio was going
00:20:58 ◼ ► through there. So Jonathan writes, "You can choose during setup of each guest user session whether to
00:21:04 ◼ ► route audio to the Vision Pro or the iPad or whatever the case may be if you choose to mirror
00:21:09 ◼ ► content." And we'll put a link to the Knowledge Base article. So what you do is you look up and
00:21:15 ◼ ► you get the little green down chevron near the top of your view. Then you go into Control Center.
00:21:20 ◼ ► Then you go back into the Mirror My View button. And then in there, there's a audio routing section
00:21:27 ◼ ► that you can choose to push everything back onto the Vision Pro. Not entirely sure why this isn't
00:21:32 ◼ ► the default, to be honest, because pretty much every time I've always wanted this. But here we
00:21:36 ◼ ► are. At least now I know that there's a way around it. So good deal. We are sponsored this episode by
00:21:43 ◼ ► Fastmail, not only a great email host, but my chosen email host since 2007. Obviously, long
00:21:49 ◼ ► before they were sponsored, long before I was a podcaster, I've been a Fastmail customer and I've
00:21:53 ◼ ► been very happy because hosting email, it's constantly a moving target. You don't want to
00:21:58 ◼ ► do it yourself. You don't want to also use one of the big services necessarily because then your
00:22:03 ◼ ► name's tied to them. I like having my own control. I like being able to have standardized hosting
00:22:09 ◼ ► with a domain that I own. Fastmail is a great place to do this. They even now have domain sales.
00:22:15 ◼ ► You can add or buy domains right through them and they set up all the records for you and everything
00:22:18 ◼ ► so it works immediately and painlessly. But it's just a great email host. Fastmail has all the
00:22:24 ◼ ► features you want under an email host. Of course, obviously it's IMAP hosting and they have all the
00:22:29 ◼ ► basic default stuff that works with every email client. They also have all the modern productivity
00:22:33 ◼ ► features you might want. Things like scheduled send, snooze, folders, labels, search, etc.
00:22:38 ◼ ► They have all the great plans, all the great features. I love Fastmail. They now also have
00:22:42 ◼ ► multi-user plans. So if you want to have your whole household have a plan, they have a duo plan
00:22:47 ◼ ► for secure private email for you and a partner at a reduced price. And for just a little bit more,
00:22:51 ◼ ► they have a family plan for up to six people. You can upgrade your entire family. And of course,
00:22:55 ◼ ► in those family and duo plans, you have things like shared domains if you want. You can have shared
00:23:00 ◼ ► or private calendars and address books. You can have like a family calendar all shared,
00:23:04 ◼ ► hosted by Fastmail, all based on standards. It is wonderful. So I strongly recommend Fastmail
00:23:09 ◼ ► and they are super private too. For over 20 years, Fastmail has been a leader in email privacy.
00:23:15 ◼ ► They believe in working for you as people to be cared for, not data and products to be exploited.
00:23:21 ◼ ► And Fastmail has better spam filters than everyone else and absolutely no ads. So it's super easy to
00:23:27 ◼ ► get started with Fastmail today. You can download your data from wherever you are now. You can
00:23:30 ◼ ► import it into your new Fastmail inbox. You don't need to leave anything behind when you switch.
00:23:35 ◼ ► It's great. Try Fastmail free for 30 days and get 10% off your first year at fastmail.com/atp.
00:23:42 ◼ ► Once again, free for 30 days, 10% off your first year, fastmail.com/atp. Thank you so much to
00:23:48 ◼ ► Fastmail for being an awesome email host for all these years and for sponsoring our show.
00:23:52 ◼ ► It is quickly approaching WWDC time, which means I'm going to be seeing you too soon. But
00:24:00 ◼ ► nevertheless, we should talk about some last second predictions. And I guess this is most
00:24:07 ◼ ► predominantly from Mark Gurman in today's episode. So I'm going to read a whole bunch of stuff. One
00:24:11 ◼ ► of you guys feel free to pipe in and interrupt at your convenience. But here we go. Mark Gurman
00:24:17 ◼ ► writes, "Apple is preparing to spend a good portion of its worldwide developers conference
00:24:22 ◼ ► laying out its AI-related features. At the heart of the new strategy is Project Gray Matter,
00:24:28 ◼ ► a set of AI tools that the company will integrate into core apps like Safari, Photos, and Notes.
00:24:37 ◼ ► Much of the processing for less computing-intensive AI features will run entirely on the
00:24:42 ◼ ► device. But if a feature requires more horsepower, the work will be pushed to the cloud. There are
00:24:46 ◼ ► several new capabilities in the works for this year, including ones that transcribe voice memos,
00:24:50 ◼ ► retouch photos with AI, and make searches faster and more reliable in the spotlight feature.
00:24:55 ◼ ► Faster would be great, particularly on my iPad, please, and thank you. They also will improve
00:24:58 ◼ ► Safari web search and automatically suggest replies to emails and text messages. The Siri
00:25:03 ◼ ► personal assistant will get an upgrade as well, with more natural-sounding interactions based on
00:25:07 ◼ ► Apple's own large language models. There's also a more advanced Siri coming to Apple Watch for
00:25:16 ◼ ► Let's stop here for a second and look at this list of features because we're always like,
00:25:27 ◼ ► like, "Oh, they're going to fix Siri." We were speculating months ago or weeks ago, whatever.
00:25:32 ◼ ► "Is this the year that they're going to fix Siri with AI, or are they just going to add it to a
00:25:34 ◼ ► bunch of other stuff?" And Gurman's rumor was like, "No, they're doing a Siri thing." So we
00:25:37 ◼ ► can expect to see that. But here's some specifics, and the specifics seem not weird. I don't know.
00:25:45 ◼ ► Sometimes these rumors aren't comprehensive. Very often, Apple emphasizes one or two particular
00:25:52 ◼ ► going to concentrate on, and it's going to be impressive. But let's look at some of these in
00:25:55 ◼ ► turn and see how exciting they are. Transcribing voice memos. Apple's been doing transcriptions,
00:26:01 ◼ ► for example, on voicemail for a long time now. Having transcription be better, that's good.
00:26:08 ◼ ► Probably not going to really radically change people's lives since it's an existing feature,
00:26:15 ◼ ► things. That could be a headlining feature. That short description doesn't tell us, "Is this going
00:26:20 ◼ ► to just be like, 'Oh, they're better at background detection for tearing people off,' or is it going
00:26:24 ◼ ► to be like, 'Oh, they're going to really emphasize that they're doing the feature that Google and
00:26:27 ◼ ► many others have, where if there's something in the background, a photo you don't want,
00:26:31 ◼ ► you can remove it, remove a person from the background who shouldn't be there, remove a
00:26:36 ◼ ► sign or a tree or something like that.' That could be a big headlining feature. Google has certainly
00:26:41 ◼ ► had a whole ad campaign about theirs, and obviously it's every graphics application from
00:26:46 ◼ ► Photoshop to Lightroom, Pixelmator and everything has features like this, and they've been touting
00:26:50 ◼ ► them, and I think they're crowd-pleasing. - Well, but that's part of the problem. So far,
00:26:53 ◼ ► voicemail transcription and fancy AI photo cleanup and removal, that's table stakes today.
00:26:59 ◼ ► I'm glad Apple's getting there, they should, but neither of those I think is likely to make a big
00:27:07 ◼ ► splash simply because other people have been doing that for a decent amount of time, and the rest of
00:27:14 ◼ ► the industry is there. What I want to see from Apple is what we get excited about from Apple.
00:27:21 ◼ ► I want them to blow us away with stuff that we haven't seen from everyone else. I want to see
00:27:25 ◼ ► features that are not just playing catch-up with the rest of the industry. I want to see Apple
00:27:29 ◼ ► leading the way, not following, and these are just following so far. - Yeah, I mean, well, as with
00:27:34 ◼ ► all these things, we know that this has been done elsewhere and even on Apple's own platforms in
00:27:39 ◼ ► various applications, but when people get one of these new phones or they update the OS or they
00:27:45 ◼ ► see an ad on TV that Apple touts this feature of erasing a person, chances are good that they've
00:27:50 ◼ ► never seen that before if they're not a tech nerd, right? And so as far as they're concerned, wow,
00:27:54 ◼ ► this is amazing. Especially if they don't know it's even possible, like they're saying, "Oh,
00:27:58 ◼ ► on my phone I could just tap a person and remove them?" And we all know that that's been around for
00:28:02 ◼ ► a long time and they're playing catch-up, but A, they do have to play catch-up. We don't want them
00:28:06 ◼ ► to not do this. And B, it can be just as impressive. It can be very impressive to people who haven't
00:28:10 ◼ ► seen it before. So I just hope they do a good job of this because there is a lot of competition,
00:28:14 ◼ ► again, even on Apple's own platforms in applications that you can get for the Mac and for
00:28:20 ◼ ► the iPad and for the iPhone that already do the same job. I hope Apple at least matches them.
00:28:25 ◼ ► Historically, Apple has kind of been... They want to make it simple, so there's not a lot of
00:28:30 ◼ ► adjustability. And if it does a bad job, the Apple way is like, "Well, just tap on the person and
00:28:35 ◼ ► remove them." And you'll tap on a person and it'll make a terrible mess of it. And you'll be like,
00:28:39 ◼ ► "Is there something I can do to try that again and do better?" And the Apple way is no.
00:28:43 ◼ ► You tap them and if it does a good job, good. And if it doesn't, oh, well. Whereas in Lightroom,
00:28:48 ◼ ► there's a million knobs you can adjust and you can take a second attempt and you can mask
00:28:51 ◼ ► differently. You do all sorts of things at a fancier application. So there's that. I really
00:28:57 ◼ ► hope they do catch up and I hope they do a good job of it. The next item about improving Safari
00:29:09 ◼ ► Well, I mean, for whatever it's worth... Obviously, we don't know what this means yet. We might not
00:29:14 ◼ ► even learn it in two weeks. But Safari had... Whatever Apple is doing, whatever their back-end
00:29:21 ◼ ► logic is for autocomplete suggestions in Safari is basically a search engine. I can't even imagine
00:29:29 ◼ ► how many millions and millions of web searches Apple avoids even making on the iPhone by people
00:29:36 ◼ ► just tapping that first autocomplete result that comes up. So maybe it's just related to whatever
00:29:42 ◼ ► the back-end is of that. And that could be one of those invisible things that everybody will take
00:29:48 ◼ ► for granted and nobody will even notice, but we will appreciate it getting better. Like things
00:29:52 ◼ ► like autocorrect. Well, notice if they screw it up, they've changed autocomplete many, many times
00:29:56 ◼ ► over the years and when they screw it up, we all notice and we don't like it. So I really hope
00:30:00 ◼ ► it doesn't start suggesting ridiculous things. I think that might all be local too. So maybe that
00:30:05 ◼ ► is a place for LLMs to try to do better predictions. But anytime they're messing with an existing
00:30:10 ◼ ► feature, you hope it's gonna be a big improvement, but there's the possibility, especially when you're
00:30:14 ◼ ► throwing LLMs into the mix, it's a possibility. If the current strategy is like, look at your past
00:30:19 ◼ ► history, look at the number one search results, they could even be hitting the Google backend
00:30:26 ◼ ► but it's deterministic. And if they switch from that to like, let's just chuck it over the wall
00:30:31 ◼ ► to the LM and see what it says. I'm concerned that there might be some wackiness there,
00:30:36 ◼ ► but we'll see. More reliable spotlight features. Similarly, when you use spotlight on the phone,
00:30:42 ◼ ► I don't know if it's using spotlight, spotlight, if it's the same technology as the runs on the
00:30:46 ◼ ► Mac, but it's like, oh, I'm searching for stuff on my phone and it includes contacts and all your
00:30:51 ◼ ► applications and files and things you've done recently. And yeah, you could probably throw
00:30:56 ◼ ► LLMs into the mix there to handle, basically like, so you don't have to type things exactly,
00:31:00 ◼ ► and it's better about synonyms and you can type in vague things, like type in some expression about
00:31:05 ◼ ► what you want to do in settings and have the LM figure out the setting you want to go to.
00:31:18 ◼ ► where we thought like, will Apple go there? We haven't gotten to like chat bots yet, but
00:31:26 ◼ ► But the idea that Apple would within the mail application or while sending text messages pop
00:31:31 ◼ ► up a little clippy and say, it looks like you're trying to reply to your friend. You want me to
00:31:34 ◼ ► write it for you? Which is what, I mean, just look at the Gmail interface, for example, every time
00:31:38 ◼ ► I'm in Gmail, it's like, do you want me to summarize this email? Do you want me to just
00:31:42 ◼ ► write the reply for you? Like Google's been pushing on this for years. It used to just be like,
00:31:49 ◼ ► like, you know what? I can just write the whole email for you. Like you don't have to click from
00:31:53 ◼ ► canned phrases. You don't have to, you know, the auto-complete that happens in like Google docs
00:31:57 ◼ ► and everything. It's just like, let me just write the whole thing for you. And that, we should find
00:32:02 ◼ ► that link to Nevin Mergen's blog post. Nevin Mergen said he got an email from a friend that was
00:32:12 ◼ ► that is not a conservative move to suggest to people that the phone will write their email or
00:32:19 ◼ ► their text message for them. Summarizing, I can see it's like the phone is helping me. If I just
00:32:23 ◼ ► want a summary, I don't want to read it all, let the phone summarize it. It's like, you know,
00:32:27 ◼ ► asking the phone to help you deal with your stuff. But having generative AI write something that you
00:32:33 ◼ ► will send as if you wrote it yourself is a step farther than Apple has gone. So we'll see how they
00:32:41 ◼ ► present that. Again, obviously everyone else is doing this. It's not new. I'm not saying Apple's
00:32:44 ◼ ► going to be the first one to do this and they're going to get in big trouble or something. It just
00:32:47 ◼ ► doesn't, it's not a very safe thing to do. So I'm wondering how daring Apple will be in like
00:32:53 ◼ ► their little presentation. Maybe they'll just be like, and it can even suggest a reply for you. And
00:32:57 ◼ ► some happy person who, happy text will tap a thing on their phone screen. And they'll be so delighted
00:33:03 ◼ ► to see the three word reply come and they'll hit the send button and they won't talk about it
00:33:07 ◼ ► anymore. But I don't know, it wigs me out a little bit too. - I mean, keep in mind, like we're probably
00:33:13 ◼ ► not that far away from, I mean, people are probably doing it now. Where like you're using your AI to
00:33:20 ◼ ► compose an email that will be then read by the recipients' AI. And we will all just be sending
00:33:26 ◼ ► even more garbage crap to each other and having it be processed by even more garbage AI junk on
00:33:32 ◼ ► the other end. And we will finally reveal email to be the useless thing that it has been most of
00:33:36 ◼ ► this time. - It's not useless. And I feel like in a business setting, this is most useful because
00:33:40 ◼ ► there's a lot of boilerplate and niceties in business that would help. But like we already
00:33:44 ◼ ► have auto, like I mentioned autocomplete, like Gmail forever has had a thing where you start
00:33:47 ◼ ► writing something and it guesses what you're probably going to say for these next three words
00:33:51 ◼ ► and you can autocomplete that. But I feel like that kind of piecemeal autocomplete, even if it
00:33:56 ◼ ► is LLM powered, that piecemeal autocomplete at least forces you to read everything. Whereas having
00:34:01 ◼ ► a button that says, "Compose a reply for me," invites the possibility that you will not read
00:34:06 ◼ ► what it generated because that will take too much of your time and you'll just hit send.
00:34:10 ◼ ► And now we're sending messages that we're not even looking at to give a thumbs up or a thumbs down on
00:34:14 ◼ ► because it just takes too much time. And that's just, I think it's a waste of people's time
00:34:17 ◼ ► because if you don't read what it wrote, maybe it didn't write what you wanted it to write. And
00:34:20 ◼ ► now you have an even more confusing email thread. And I mean, again, we'll figure this out as a
00:34:25 ◼ ► culture, as the technology advances, but I'm just saying character for Apple, for Apple's thus far
00:34:30 ◼ ► very conservative approach to AI, suggesting replies to emails and text messages seems like
00:34:43 ◼ ► "One standout feature will bring generative AI to emoji. The company is developing software that
00:34:49 ◼ ► can create custom emoji on the fly based on what users are texting. That means you'll suddenly have
00:34:53 ◼ ► an all new emoji for any occasion beyond the catalog of options that Apple currently offers
00:34:58 ◼ ► on the iPhone and other devices. Another fun improvement unrelated to AI will be the revamped
00:35:03 ◼ ► iPhone home screen that will let users change the color of app icons and put them wherever they want.
00:35:10 ◼ ► and they won't need to be placed in the standard grid that has existed since day one in 2007."
00:35:15 ◼ ► All right. So that's a lot of stuff here. The generative AI emojis, obviously they're not
00:35:19 ◼ ► emojis because emojis are like Unicode things. There's a defined set of them. You can't just
00:35:22 ◼ ► make up new ones. I mean, you can't if you're the Unicode consortium, but Apple can't, right? So
00:35:26 ◼ ► what it's actually saying is kind of like on Slack and Discord and all these other things where you
00:35:31 ◼ ► can sort of generate custom stickers or I think they call them custom emojis or custom reactions
00:35:35 ◼ ► or whatever. The point is, this is going to be a new graphic that they'll generate on the fly for
00:35:39 ◼ ► you that will be sent as an image because they can't just send it as a Unicode code point because
00:35:44 ◼ ► these, they're not dynamically adding things to Unicode, right? That's not how this works.
00:35:47 ◼ ► So they're kind of generating stickers for you. And I guess they're going to like, you know,
00:35:53 ◼ ► they have a bunch of like Mr. Potato Head style building parts they feed into the LLM and then it
00:35:57 ◼ ► can do like sentiment analysis to figure out what kind of emoji we'd want. I have to say, like,
00:36:05 ◼ ► I guess there's ones that aren't there. Like you never try to summon someone emoji and you're
00:36:09 ◼ ► shocked that there's not like a watermelon emoji or something. I can't think of one that doesn't
00:36:13 ◼ ► exist. Sorry. But frequently I go into emoji and I'm like, "Oh, surely there's an emoji for this."
00:36:18 ◼ ► And like, there's no ice cream cone. Like, again, I don't know if that's real or not, but like I'm
00:36:21 ◼ ► shocked by what's not there. So that's nice that it'll generate to it, but it'll have to send it as
00:36:25 ◼ ► an image because it can't send it in the other way. And I do wonder, this is another place where
00:36:30 ◼ ► like, I feel like Apple is taking a risk, even with a very, very limited image generator that's
00:36:35 ◼ ► trained on their own source, you know, trained on all their own source for emojis that Apple makes
00:36:39 ◼ ► and can generate new ones based on that and some other information. There's the potential to
00:36:45 ◼ ► generate a little image that maybe is embarrassing to Apple as a company. You know what I'm saying?
00:36:49 ◼ ► So, and there's a potential to generate images that just look ugly or not up to Apple standards.
00:36:56 ◼ ► You know what I mean? Like, this is, that's a weird thing to be touting. And the coloring icons,
00:37:01 ◼ ► that's, you know, another, I'm not going to say, I keep wanting to say a bridge too far. It's not
00:37:05 ◼ ► a bridge too far, but it is a bridge. Like you're going to take developers icons and draw all over
00:37:11 ◼ ► them. They've done this before. They used to add the shine to your icon if you didn't opt out. Do
00:37:14 ◼ ► you remember that? Like the gloss? But now they're like, we're going to change the color of your icon?
00:37:20 ◼ ► Like, I'm sure these companies that are like so proud of their branding and change their icons or
00:37:24 ◼ ► whatever, like that now the user will be able to say, you know what? I want Instagram to be green.
00:37:29 ◼ ► And the OS will be like, sure, I'll take that Instagram icon and I'll make it green. We can
00:37:33 ◼ ► do that. That's weird. I mean, you can make little color code things on your home screen, I suppose.
00:37:42 ◼ ► like a lot of companies I think that have a problem with just like trademark issues with like,
00:37:46 ◼ ► what if you color my icon green? First of all, it's not my brand identity. Second of all,
00:37:51 ◼ ► it might look like my competitors or it might infringe someone else's trademark or something.
00:37:55 ◼ ► Like I could see there's, I could see like big companies having a big problem with that. But
00:37:58 ◼ ► users are doing it. And by the way, there is a precedent for this in Mac OS back in the day,
00:38:03 ◼ ► someone else will look up what version of Mac OS or the operating system for Macs used to do this.
00:38:08 ◼ ► When you added a label to something, you know, you can do labels in Mac OS 10 or in classic Mac OS,
00:38:12 ◼ ► you could add labels in one version of Mac OS or system six or whatever it was called back in it,
00:38:18 ◼ ► whichever version I'm thinking of. I think it might've even been system seven. When you apply
00:38:22 ◼ ► to label it, what color the icon like, so if you probably green label, it would like green tint the
00:38:28 ◼ ► icon by basically making the black pixels green or the black pixels red. I think it did not look
00:38:32 ◼ ► good. It was not a good look that didn't last for a very long time. Certainly the Mac OS 10 era that
00:38:38 ◼ ► has never been the case, but yeah, the user deciding to color icon, like that was not on my
00:38:45 ◼ ► list of things to think of they were to be doing. And I know this is like unrelated to AI, but let's
00:38:48 ◼ ► just say iOS 18 change next week, we'll probably have many more previews of WWDC features, but
00:38:53 ◼ ► what an odd, what an odd thing to do. What is this? Have people been clamoring for this?
00:38:59 ◼ ► I want to change the color of my icons. I mean, changing icons. They added that while ago where
00:39:04 ◼ ► you can pick from different icons and blah, blah, blah, but even that, remember how conservatively
00:39:07 ◼ ► they added that? Like it has to pop up a dialogue in people's faces so you know when it's happening
00:39:17 ◼ ► - No, but I mean, there's clearly, there's a very large market out there. We've talked about it
00:39:23 ◼ ► before. Like there's a very large market out there for these home screen customization apps that use
00:39:27 ◼ ► all sorts of tricks and hacks and frankly kind of bad hacks to get people to be able to customize
00:39:33 ◼ ► icons for their apps and make like their aesthetic home screens that they want. That's a huge demand,
00:39:38 ◼ ► huge market. I think Apple wants to try to address some of that demand without having all these huge
00:39:43 ◼ ► hacks if this is indeed what they're doing here. The problem is unless you can have arbitrary icons
00:39:54 ◼ ► - Right, like if you can do that, then all these apps that sell like these, 'cause you know,
00:39:59 ◼ ► they sell like theme packs. Like people aren't just drawing it themselves. Like they're using
00:40:03 ◼ ► apps that have like cool theme packs of here's like a bunch of popular icons for all the popular
00:40:09 ◼ ► apps that you know you have like Instagram. Plus also here's a bunch of like random ones you can
00:40:13 ◼ ► apply yourself to whatever else you need them to be, you know, like kind of generic ones.
00:40:17 ◼ ► That's a huge market and so if we're just talking about like you can tint it seven different colors,
00:40:23 ◼ ► that's not gonna get rid of this market at all and you know, some people will use it. That'll
00:40:28 ◼ ► be great but I can't imagine that being worth the hassle and the trouble that they might get
00:40:32 ◼ ► in with other companies. So I have a feeling this is like, you know, sometimes a rumor comes out
00:40:37 ◼ ► and we're just kind of like huh? And then you know, whenever it's actually announced, we see,
00:40:42 ◼ ► oh there's important detail X, Y, and Z here that explain it better and that now it makes more sense.
00:40:48 ◼ ► I think we're missing those details right now on this feature. I think there's something else here
00:40:52 ◼ ► to explain this that we're not getting it. - I mean, it might just be like you mentioned
00:40:59 ◼ ► using shortcuts to replace the app icons, first of all, it's super cumbersome as is doing anything
00:41:03 ◼ ► in home screen because you have to hide the existing app and make the shortcut to the app.
00:41:06 ◼ ► You know, it's just, it's a hassle and second, if you get one of those icon packs, maybe it doesn't
00:41:09 ◼ ► have an icon for the app that you want to change but if the OS says, okay, well if you're making
00:41:14 ◼ ► a little region of green icons and you can use icons from the Steam pack, you got, oh but there's
00:41:18 ◼ ► no green icon for this app, well just take the actual icon for the app and tint it green and
00:41:22 ◼ ► now it fits in. So maybe it's like, you know, a catchall for if there's, if, because people
00:41:27 ◼ ► aren't going to draw their own custom icons and if they can't find the one in a pre-built icon pack,
00:41:32 ◼ ► now you can just at least make the icons match. I don't quite understand the idea to color code
00:41:38 ◼ ► everything but there's that and yet the other part of this is being able to, you know, this does not
00:41:42 ◼ ► put them in the standard grid. It's basically just to be able to leave space. You don't have to make
00:41:45 ◼ ► spacer, clear spacer icons anymore. I'm assuming that's what they're talking about here is like,
00:41:49 ◼ ► now you can arrange things on the home screen and leave gaps if you want to. I still think doing
00:41:56 ◼ ► anything on the home screen on the iPhone is incredibly painful. We've talked about this
00:41:59 ◼ ► for years and years. I think it will continue to be painful. I really wish there was a nice interface
00:42:05 ◼ ► where you could mess around with this stuff and do it without destroying anything and either commit
00:42:10 ◼ ► or roll back all your changes as opposed to the current system which is just pure chaos and it's
00:42:14 ◼ ► like the hardest game on the entire operating system to try to rearrange icons on your phone
00:42:19 ◼ ► without screwing everything up and just throwing your hands up and saying forget it, I'm hiding
00:42:22 ◼ ► everything into the app library. Yeah, I wonder, I feel like I heard this from Jason maybe. I heard
00:42:30 ◼ ► it somewhere. Maybe it was Mike on upgrade. I forget where I heard it but somebody pointed out,
00:42:33 ◼ ► well maybe there will be like layers and that makes me think of, you know, how that was upgraded
00:42:39 ◼ ► in SF symbols. You can have like different layers and you can color them and you know in different
00:42:43 ◼ ► ways and whatnot. I think that makes some amount of sense but that's not going to fix like the
00:42:49 ◼ ► instagrams of the world so I'm not sure or maybe it'll be opt-in for developers but again I don't
00:42:54 ◼ ► see. VisionOS has this right? Don't they have layers in their icons? Yes, they actually kind of,
00:42:58 ◼ ► I think they require it with VisionOS. They made that a new like hey for iOS 18 if you provide
00:43:03 ◼ ► a layered icon, our new coloration API will respect that like as you sound like SF symbols
00:43:10 ◼ ► where we can colorize parts of it. We'll see. It just seems to me like a weird place to be
00:43:15 ◼ ► spending resources. I think the arrangement part is a smart feature. They should have that. People
00:43:20 ◼ ► have wanted that for a long time. People have been doing it with spacer icons. It's annoying.
00:43:23 ◼ ► The place where they really should have spent resources is how hard it is to rearrange the
00:43:26 ◼ ► home screen. We'll see if they do that. Yeah, that's the only way I really wish I could still
00:43:31 ◼ ► interact with my phone via iTunes which I know is now finder but that was so much better was,
00:43:36 ◼ ► you know, doing home screen rearrangement. And even that was terrible but it was better.
00:43:39 ◼ ► Yeah, it was terrible but it was way better. All right, finishing up with Mark Gurman. A big part
00:43:44 ◼ ► of the effort is creating smart recaps. The technology will be able to provide users with
00:43:48 ◼ ► summaries of their missed notifications and individual text messages as well as web pages,
00:43:53 ◼ ► news articles, documents, notes and other forms of media. There's also no Apple design chatbot,
00:43:57 ◼ ► at least not yet. That means the company won't be competing in the highest profile area of AI
00:44:02 ◼ ► and the version that Apple has been developing itself is simply not up to snuff. The company's
00:44:07 ◼ ► held talks with both Google and OpenAI about integrating their chatbots into iOS 18. Apple
00:44:12 ◼ ► ultimately sealed the deal with OpenAI and their partnership will be a component of the WWDC
00:44:17 ◼ ► announcement. A couple of quick thoughts here. First of all, I think we all are in a group chat
00:44:21 ◼ ► that oftentimes will just pop off at a time that you're not paying attention and so it is not
00:44:26 ◼ ► uncommon. Particularly, I find that I'm in a group chat with a couple of guys that are constantly
00:44:32 ◼ ► looking at cars that they may or may not ever buy. One of them in particular is obsessed with finding
00:44:38 ◼ ► a affordable 911 and those are mutually exclusive terms. You cannot find a 911 that is affordable,
00:44:43 ◼ ► that's not a pile of garbage. And so anyways, I will often come back to my phone or computer,
00:44:48 ◼ ► what have you, with literally 40 or 50 messages, most of which I don't particularly care about
00:44:52 ◼ ► because 911 is not really my cup of tea. And so if there was a way to summarize that, I'm all in,
00:44:57 ◼ ► sign me up. And then additionally, I mean here it is that Germin is saying OpenAI is the winner and
00:45:03 ◼ ► like we discussed I think this past week, that's going to be a little dicey. So I wonder how they
00:45:08 ◼ ► spin this. Well, in typical Germin fashion, there's also no Apple designed chatbot, at least not yet.
00:45:14 ◼ ► Apple designed? Do they mean Apple's in-house LLM that they've been working on? Is that what
00:45:18 ◼ ► he's saying there won't be any of? Because I bet whatever, like they say they did a deal with OpenAI
00:45:23 ◼ ► but he doesn't say, okay, well what is that deal, how is that deal going to manifest? Is it going
00:45:26 ◼ ► to manifest by iOS 18 having a chatbot that is powered by OpenAI? And will that not be Apple
00:45:30 ◼ ► designed or will it be OpenAI designed? Will it be OpenAI branded? These are all, I guess,
00:45:35 ◼ ► questions that he doesn't have the answers to yet. But yeah, the news is they did a deal with OpenAI.
00:45:40 ◼ ► There's also news this week that Apple is still talking to Google, obviously. So we'll see if the
00:45:49 ◼ ► words OpenAI appear at WWC, if they're going to announce this as a partnership, they're just going
00:45:53 ◼ ► to use them on the back end. Like I said last week, maybe they've been building features using
00:45:57 ◼ ► the existing OpenAI API and now they just ink this deal so now they can officially do it in a released
00:46:02 ◼ ► OS instead of just doing it in dev builds. But I'm just not quite sure how this is going to manifest
00:46:06 ◼ ► because this paragraph begins making it seem like, oh, there's going to be no chatbot, but they have
00:46:10 ◼ ► licensed the OpenAI chatbot. Well, is there going to be a chatbot or not? Is there going to be part
00:46:15 ◼ ► of iOS where you can start typing to an LLM and getting answers in the chat GPT style or will there
00:46:22 ◼ ► be an OpenAI app or will it be integrated into Siri? Questions we do not have answers for yet,
00:46:27 ◼ ► but that is the capper on all of this stuff. A bunch of strange features, which I mean,
00:46:32 ◼ ► they all seem plausible to me, but this is not the list I would have come up with about how to add
00:46:36 ◼ ► AI sauce to iOS 18. And at the very bottom is, you know, OpenAI something, something, something OpenAI.
00:46:42 ◼ ► Yep. So yeah, we'll see what happens. As I think John said a moment ago, typically, you know,
00:46:47 ◼ ► in the couple of days leading up to WWDC, we oftentimes, or maybe the week before we hear some
00:46:53 ◼ ► last minute leaks and so on, I was going to say announcements, but they're not announcements.
00:46:57 ◼ ► So we'll see what happens. But I am very much looking forward to sitting next to you fine
00:47:02 ◼ ► gentlemen and drinking this all in while at Apple Park, hopefully not getting a sunburn.
00:47:06 ◼ ► Well, there was, did you see there was one kind of last minute thing that came out today also
00:47:20 ◼ ► was just in a separate report that Apple will also preview Siri being able to like take
00:47:28 ◼ ► more complex actions within apps, kind of what we've been talking about. You know, we've been
00:47:32 ◼ ► talking, we've been speculating about how wouldn't it be great if, you know, apps could expose,
00:47:38 ◼ ► you know, kind of stock actions similar to the intent system and just kind of describe what they
00:47:42 ◼ ► are in some kind of language and then have an LM based, you know, Siri interface be able to analyze
00:47:49 ◼ ► what the app is telling the system, here's how you can do it and here's the actions to call to do it,
00:47:54 ◼ ► and be able to offer to users the ability to perform actions like that via voice commands that
00:48:02 ◼ ► as shortcuts and things like that. The rumor that came out today from Gurman is basically
00:48:07 ◼ ► exactly that is happening, but not yet. That basically that it is, that exactly that kind
00:48:14 ◼ ► of thing will be previewed, it won't actually ship until a point release probably, they said next
00:48:19 ◼ ► year, so 2025, so presumably in the first half of next year, you know, iOS 18.2 or three or four or
00:48:25 ◼ ► five or whatever, and that it would only work in Apple's apps to start. So that tells me not an API
00:48:32 ◼ ► yet. Or an API that is released immediately, you know, or released at WWDC, but obviously nobody's
00:48:39 ◼ ► going to be able to implement it until at the earliest of the fall, and it may not be an instant
00:48:43 ◼ ► adoption. That's interesting, you know, I wonder if maybe we will see, you know, because the current
00:48:50 ◼ ► system, the current API to do all this is called App Intents. This is, they launched just a couple
00:48:55 ◼ ► of years ago, and it's like the new Swift based method of intents, which I've mentioned before,
00:49:01 ◼ ► is way better for developers and way simpler and way easier to implement than the old system of
00:49:12 ◼ ► and the generation process would always break and cause weird bugs, and then you'd have this
00:49:16 ◼ ► extension that processed it and would have to communicate to your app via like weird signaling
00:49:20 ◼ ► mechanisms for extensions to apps, like it was a whole mess before, and they've redone that system
00:49:30 ◼ ► direction for a future update for something, sometimes the APIs will launch for it that
00:49:37 ◼ ► don't come out and explicitly say this is what this is for, but kind of lay the groundwork for
00:49:42 ◼ ► it. So for instance, auto layout coming out, this is the famous example, auto layout coming out
00:49:46 ◼ ► before the iPhone 6 came out with the first like really new screen size. You know, not kind of the
00:49:51 ◼ ► iPhone 5 of course, that was just adding a table row. Anyway, so that's, I wonder if what we're
00:49:57 ◼ ► gonna see this summer is maybe they will add something to the App Intents API to give us kind
00:50:04 ◼ ► of more ways to describe to the system what these actions do, and maybe hide them from the user,
00:50:12 ◼ ► but make them available to this indexing system that presumably like the new Siri would have.
00:50:16 ◼ ► Maybe we'll see something like that. So I think this, you know, if for some reason they like demo
00:50:23 ◼ ► this and don't tell us how to use an API for our apps in the future, maybe we'll be able to see
00:50:44 ◼ ► whatever. And once they get it worked out, 'cause it's very often, especially in the old days of
00:50:47 ◼ ► Mac OS X, they would very often roll out entire new frameworks. They would be private frameworks
00:50:52 ◼ ► that only Apple's apps could use, and they would work out the kinks. Is this the right API?
00:50:56 ◼ ► Because they could change the API all they want, 'cause all they'd be breaking is their own apps,
00:50:59 ◼ ► 'cause no one else should be using this private framework. And once they finally got the framework
00:51:02 ◼ ► where it, you know, looking like they wanted it to, then the next year they roll out essentially
00:51:08 ◼ ► the same framework under a new name, but now it's a public framework. It kind of sounds like
00:51:11 ◼ ► what they're doing here, that they're not at the point, like especially if they're kind of like
00:51:14 ◼ ► rushed to catch up, they're not at the point where they're ready to even tell, perhaps even to tell
00:51:18 ◼ ► developers, you know, in the size class type of way, do this thing, and then next year you'll be
00:51:22 ◼ ► folded into the system, because it seems like they don't even know what that would be yet,
00:51:26 ◼ ► like how to indicate, like they're working that out themselves, but they do want to get something
00:51:29 ◼ ► out there, and you can demo it if they do it on their own apps, right? They say, "Coming in spring,"
00:51:33 ◼ ► or whatever, you know, whatever, like not in 18.0, right? Whatever release down the road we're going
00:51:38 ◼ ► to have this, and it's only going to be our apps to start, basically because they haven't even
00:51:42 ◼ ► figured out what they want to ask developers to do. And related to that, maybe think of Windows
00:51:50 ◼ ► activity something or other, where you essentially make a call that lets the OS know what your app is
00:51:56 ◼ ► doing? Yeah, the user activity framework. Yeah, anyway, there's a similar, we had a very similar
00:52:02 ◼ ► name on Windows, it's some kind of like activity something something. I think it had the exact same
00:52:06 ◼ ► name on it, because I saw the story of Rees-Bryant, like user activity, that's our API one. Yeah,
00:52:10 ◼ ► and it's like, basically that's how they did the thing where like you can see like a PowerPoint
00:52:15 ◼ ► slide and recall and click on it and go to that slide in PowerPoint. It's because like during the,
00:52:21 ◼ ► I think during the screen recording, like PowerPoint itself called the user activity API
00:52:25 ◼ ► to let it know that it's viewing slide number 123 and like recall recorded that because it's an OS
00:52:30 ◼ ► API and you're just telling the OS what's going on. And then when you click on the thing and the
00:52:33 ◼ ► recall thing, it looks back at that timestamp and see during that time PowerPoint had this activity,
00:52:38 ◼ ► user activity thing and it jumps back. Same type of deal. There's cooperation between APIs that the
00:52:43 ◼ ► app has to call that just says, here's what's going on. Here's my deal. And then another part
00:52:48 ◼ ► of the system, you know, cooperates with that, sees the stuff that those APIs called and connects
00:52:54 ◼ ► to dots. And that's exactly how we would think. I mean, the thing you described for user intents,
00:52:58 ◼ ► I think that's probably better than when Apple has kind of rolled out. I have dim hopes that they're
00:53:02 ◼ ► going to do what you described. What you described, I think is what they should do, but it sounds much
00:53:06 ◼ ► harder than what they might do this, especially since they're not even rolling this out publicly.
00:53:09 ◼ ► This seems like it might be a little bit simpler than that and like not involve the existing intent
00:53:14 ◼ ► system at all, which would kind of be a shame, but you know, we'll see like that when they demo the
00:53:18 ◼ ► feature, we'll see the utility of it because they won't tell you how it's implemented. But I, I,
00:53:23 ◼ ► I do hope there is a developer story for this. If there's not, you know, people are gonna have
00:53:27 ◼ ► a lot of questions with like, Hey, you demo this feature in the keynote, but it's only for Apple
00:53:30 ◼ ► things. Is that coming for third party soon? And they're gonna not comment on future products.
00:53:35 ◼ ► - Yeah, but I hope this is a direction they're going. I was very, I was simultaneously very
00:53:41 ◼ ► happy to see this rumor come today, but also a little bit disappointed that we probably won't
00:53:45 ◼ ► be able to really use it until next year or something like next year because this is an
00:53:51 ◼ ► example of something that only the platform owners can do. And it takes advantage of Apple strengths
00:53:57 ◼ ► and you know, and kind of doesn't rely on their weaknesses so much and gives them a way to lead
00:54:05 ◼ ► again. Like, you know, 'cause I think so many of Apple's AI features are going to be rightfully
00:54:10 ◼ ► seen as playing catch up as I was saying earlier. They, they need to show us that they're not,
00:54:15 ◼ ► especially like, you know, in this age of government, you know, looking into what they're
00:54:20 ◼ ► doing and seeing it does this look like a, a monopoly and in ways that hurt consumers or
00:54:25 ◼ ► hold things back. Apple needs to show that they're leading and that they're not just being complacent
00:54:30 ◼ ► in the next big thing. On lots of levels. They, I think they need to show governments that,
00:54:33 ◼ ► they need to show probably the stock market that and they definitely need to show their customers
00:54:37 ◼ ► that. You know, they, they don't need us all to be looking over in Google land saying, hmm,
00:54:42 ◼ ► they're doing a lot of good things with AI when they're on those Google models these days. Maybe
00:54:47 ◼ ► I should switch to Android for this coming fall or whatever. Like they don't need anybody,
00:54:51 ◼ ► anybody thinking that way. So they need to show like, everything's fine over here in Apple land
00:54:55 ◼ ► in this new age of AI. Please stay here. It's nice and comfortable. Don't switch. We have features
00:55:01 ◼ ► too. You can be happy with, with our version. And you know, I think they, I think there's enough
00:55:07 ◼ ► reporting now around this narrative that it's probably true. They were probably caught off guard
00:55:13 ◼ ► with the rise of new LM based techniques. They did probably decide very late, possibly like a year
00:55:20 ◼ ► ago to, Hey, we need to probably start investing heavily in this. You know, we, we kind of missed
00:55:24 ◼ ► the boat on this. You know, it does seem like there might've been some kind of you know, Bill Gates,
00:55:29 ◼ ► we missed the internet kind of, kind of realization there. But we'll see how quickly they can, they
00:55:34 ◼ ► can actually take action here. I think on some of the big stuff that Google and Microsoft are
00:55:39 ◼ ► demonstrating, I think Apple is going to be behind in a while, for a while, because that stuff that
00:55:43 ◼ ► takes years to develop, it takes priorities and skills and connections that Apple doesn't have.
00:55:48 ◼ ► Not to say they can't have it, but they currently choose not to. So we'll see what happens. It took
00:55:56 ◼ ► a lot of problems and rot to get where we are today with Siri. Have they fixed those problems?
00:56:09 ◼ ► that they needed to do that? We don't know yet. - Or they're just pouring, pouring AI sauce over
00:56:13 ◼ ► the rot. - Right. I mean, that, that is very possibly what we're going to see. Like, I have
00:56:17 ◼ ► a feeling, like, I think what we're going to see, I think it's going to disappoint a lot of people
00:56:23 ◼ ► for not being enough AI sauce, but I don't really care that much if we don't get all of our AI
00:56:29 ◼ ► wishlist items, like, in two weeks. What I want to see is, like, have they turned the ship around?
00:56:35 ◼ ► Have they actually realized current Siri is garbage? Have they actually started moving in
00:56:42 ◼ ► a better direction with that? Or are they kind of half-assing this and being complacent and thinking,
00:56:49 ◼ ► what do you mean, Siri is the best voice assistant, we are the most private, et cetera. Like,
00:56:56 ◼ ► and suggesting that we don't need anything better than this, I will be concerned. But what I'm
00:57:02 ◼ ► looking for is not for them to solve every single problem in two weeks, that's unrealistic. I want
00:57:06 ◼ ► to just see, are they going in the right direction? And so, if we get a preview of this cool, you know,
00:57:11 ◼ ► app AI interaction-based system that only works in Apple's apps and only comes next spring or
00:57:17 ◼ ► whatever, if we get a preview of that and it's really cool, and it shows we will be able to do
00:57:22 ◼ ► that in our apps, maybe in iOS 19 next year, that'll be great. I will be happy with that.
00:57:27 ◼ ► I'll be a little upset that it didn't come sooner, but hey, as long as good stuff is coming soon,
00:57:33 ◼ ► I can be patient and wait, just like Apple is, you know, often a very patient company when it comes
00:57:37 ◼ ► to this stuff. I just want to make sure that what we see are signs that they're going in the right
00:57:42 ◼ ► direction, not just sitting back and hoping that we're all going to start talking about the Vision
00:57:47 ◼ ► Pro again and stop looking about AI features. You know, when it comes to like these sort of
00:57:53 ◼ ► inside Apple things, especially when they cite specific executives, the reliability is never
00:57:58 ◼ ► great, right? Because this is a game of telephone, people have grudges, who knows what actually goes
00:58:14 ◼ ► are exemplify kind of the worst case scenario as far as I'm concerned. I don't know if they're true,
00:58:18 ◼ ► that could be total BS, but here are the two things. One, and we think we've mentioned this
00:58:22 ◼ ► snarkily on previous shows, is there's various rumors and supposed tales from the inside saying
00:58:28 ◼ ► recently, you know, as a year ago or a year or a half ago, important Apple executives who are
00:58:34 ◼ ► often named saw ChatGPT and that made them realize Siri sucks. And that's depressing to say, "How did
00:58:41 ◼ ► you not know?" That's what it took for you to realize Siri sucks? Yeah, I hope, I really hope
00:58:48 ◼ ► to God that's not what it took. We hope that's not true. We hope that is just like someone, you know,
00:58:54 ◼ ► you know, sort of extrapolating or exaggerating a story they heard or whatever. We would hope that
00:58:59 ◼ ► Apple has known that Siri is a problem for a long time and ChatGPT was just really like the straw
00:59:04 ◼ ► that broke with camel's back. But it's an unflattering story about Apple, which is again,
00:59:07 ◼ ► maybe why it spreads. And the second unflattering story related to this is that, I think Federighi
00:59:13 ◼ ► was named here, it was like a mandate a year ago that every team under Federighi has to add some
00:59:19 ◼ ► AI feature this year. And that is an unflattering story, but that's exactly what big stupid
00:59:25 ◼ ► corporations do. They're like, "I don't know, but there's some big thing. I don't care what you're
00:59:29 ◼ ► doing team. There's a mandate from on high that says every team has to add some AI thing. I don't
00:59:33 ◼ ► care what it is, but it better say AI." That's not the way to make a good product. I mean,
00:59:38 ◼ ► and this happens, this has happened frequently in Apple's history. We know with more certainty that
00:59:42 ◼ ► like, for example, when a new Mac OS feature back when Mac OS was important, a new Mac OS 10 feature
00:59:49 ◼ ► would come out and Apple would mandate, you know, all the teams have to now like add some feature
00:59:55 ◼ ► that takes advantage of spotlight or something. You know what I mean? And teams would complain.
00:59:58 ◼ ► They were like, we were in the middle of making our application better, but then a mandate came
01:00:01 ◼ ► from on high that stop what you're doing and make sure you carve out room in this release to add like
01:00:07 ◼ ► a spotlight powered feature or something. I'm not creating a great example, but like that is
01:00:11 ◼ ► disruptive to teams who are working. Maybe the spotlight, adding a spotlight feature is not
01:00:15 ◼ ► important for whatever application that, you know, it's not important for the terminal application or
01:00:19 ◼ ► whatever. Not that there's more than a third of a developer working on that, but like telling teams,
01:00:24 ◼ ► you just have to add something with AI. That's not vision. That's not leadership. That's just like
01:00:34 ◼ ► therefore a sacrifice part or all of your schedule to figuring out what you can do that uses AI.
01:00:40 ◼ ► It's the opposite of the Apple philosophy. Instead of figuring out like what will help people and
01:00:45 ◼ ► then doing that, it's like we've pre-decided that AI is good in and of itself. You figure out
01:00:51 ◼ ► something to do with it so you can fulfill that. And again, it's an unflattering story,
01:00:55 ◼ ► a totally unconfirmed, possibly manufactured or made up by someone with a pessimistic view of
01:01:00 ◼ ► Apple or whatever. And we just really hope it's not true, but those are the only two things that
01:01:04 ◼ ► I've heard from inside Apple. And I just hope it's like, again, people with grudges or people with a
01:01:09 ◼ ► dim view of what's actually going on because you would hope that Apple is more thoughtful. And
01:01:13 ◼ ► honestly, I do think Apple has been more thoughtful. I think the chat GPT thing, like I
01:01:16 ◼ ► said, is it's not, they didn't know that Siri was bad. It's just this really pushed them over the
01:01:21 ◼ ► edge to say, this is the year we really have to do something, which, you know, fair enough
01:01:24 ◼ ► because they've been trying to solve it for years and failing and it's a big complicated
01:01:33 ◼ ► in the, in all their operating systems. I just need it used where it can do the most good. And
01:01:39 ◼ ► part of that is, yeah, look at where other people have done things and that people like,
01:01:42 ◼ ► like erasing people from photos, they should be doing that. Right. But in other places,
01:01:47 ◼ ► you know, manufacturing emoji, the messages team said, how are we going to add AI? You know,
01:01:51 ◼ ► I guess summaries is good, right? Thumbs up on that. But can we, someone has the idea to
01:01:55 ◼ ► manufacture emoji, maybe have those people work on like, uh, I messages and the iCloud sync
01:02:01 ◼ ► improvements or something, but. Or search improvements. Yeah, there you go. Or archiving
01:02:06 ◼ ► improvements or something like I would set priorities differently, let's say, but you know,
01:02:10 ◼ ► what can you do? What I'm also, I, what I really want to see too, as a developer, I mean, obviously
01:02:19 ◼ ► As Apple adds AI features, they have a massive advantage that they have really good silicon
01:02:27 ◼ ► and their products to do local processing. And historically they create really good APIs. Like,
01:02:34 ◼ ► you know, we nitpick here and there like, oh, you know, something's under documented or whatever,
01:02:37 ◼ ► or watch connectivity sucks, but like for the most part, and it does, but for the most part,
01:02:43 ◼ ► Apple's APIs are world-class. They're really good. Most of the time they are extremely powerful.
01:02:49 ◼ ► It's very, very difficult to find better APIs for a lot of things than what Apple provides.
01:02:59 ◼ ► really powerful things pretty easily. And one of the reasons that they're able to do that is
01:03:04 ◼ ► because their, their local device hardware bar is just so high. They have all sorts of great APIs
01:03:11 ◼ ► for things like ML processing, which is, you know, going to be presumably rebranded AI processing,
01:03:16 ◼ ► a lot of this stuff. They have all sorts of like, you know, high, high processing power things that
01:03:21 ◼ ► you can just drop in in your app with, you know, not much effort. And all of a sudden now you have
01:03:31 ◼ ► modern AI stuff, give that to us, make that available for free with no limits right on the
01:03:40 ◼ ► device. That's something that not every competitor can offer because they don't have, first of all,
01:03:46 ◼ ► the very high bar. Like not every Android phone is going to be able to do this kind of stuff because
01:03:51 ◼ ► not every Android phone has the right processor. Yeah. Every, every Mac, every iPhone, every iPad
01:03:55 ◼ ► is a co-pilot plus Mac, iPad, iPhone, like all have MDs. Well, except for the RAM. Microsoft is
01:04:00 ◼ ► in a situation where Intel and AMD are just now going to be getting out chips that can do 40 tops
01:04:06 ◼ ► or whatever. And they have that one Snapdragon chip, but every Apple Silicon thing Apple has
01:04:11 ◼ ► sold in the past several years has had a pretty powerful neural processing unit in addition to
01:04:15 ◼ ► all sorts of other sort of hardware units in the SOC for other tasks. So they're way ahead of the
01:04:20 ◼ ► game on hardware. Yeah. And what I want to see from them is not only make that hardware available
01:04:27 ◼ ► to developers, I mean, most of it already is. Give us the models, have the models for many common
01:04:33 ◼ ► tasks built into iOS and let us just call them when our apps are running and just use them
01:04:40 ◼ ► unlimited for free. That is how you make an entire ecosystem of awesome apps that run on your
01:04:47 ◼ ► platform and keep people locked in. So there is no reason for them not to offer this. Like,
01:04:58 ◼ ► There is one. There has been a speech recognition API in iOS for something like six or seven years.
01:05:06 ◼ ► It's not that new. It's not very good and it's pretty limited and it's like in certain contexts,
01:05:13 ◼ ► I think they would send it to the cloud because it would limit you to like 60 seconds of transcription
01:05:20 ◼ ► I don't want to have to ship OpenAI Whisper and compile it like exactly right for Apple Silicon
01:05:26 ◼ ► and make sure it's optimized right using Whisper CPP and keep updating it every time it gets updated
01:05:30 ◼ ► and have my app have to download this like, you know, gig and a half file for the model.
01:05:34 ◼ ► Build that kind of stuff in and just let us call it and let us use it like we can use any other
01:05:42 ◼ ► burning your CPU and stuff, but like build that stuff in and let everyone use it for free
01:05:52 ◼ ► That's what I'm looking for. I'm looking for A, give me a sign that you're taking this new
01:05:57 ◼ ► technique of computing seriously in what you're building in as features to your OS and also
01:06:04 ◼ ► don't just keep it for yourselves. Build APIs for it and let us use it in the most unlimited way you
01:06:10 ◼ ► can possibly have with the hardware that you have. You know, obviously things that go to the cloud,
01:06:18 ◼ ► Let us do as much as we can locally. That's what your devices are awesome at. Let us use it. And
01:06:24 ◼ ► there are so many people out there, there are so many app developers out there who are never going
01:06:28 ◼ ► to go through the process of figuring out how to like train your own model or integrate someone
01:06:33 ◼ ► else's model using these weird Python tools that we don't know how to use. Like just build it in,
01:06:37 ◼ ► let us use it. That would be amazing. They do have a track record of kind of falling on both sides of
01:06:43 ◼ ► this with different decisions. So this could go either way. I really hope they go the direction of
01:06:51 ◼ ► I mean, the limitations you're going to run into is not going to probably be limitations using the
01:06:56 ◼ ► models, but just the plain old limitations on running in the background using CPU, stuff like
01:06:59 ◼ ► that. But the rumor is for this and what Apple, what Microsoft has actually announced that Build
01:07:04 ◼ ► is that both companies are essentially going to give access to models, but through abstracted APIs
01:07:09 ◼ ► where the caller doesn't even know or possibly even get to choose whether it runs locally or
01:07:14 ◼ ► in the cloud. So Apple can change that decision sort of on the fly and in new versions. So yeah,
01:07:19 ◼ ► that's Microsoft advertised that at Build, "Hey, you just use these frameworks and you get these
01:07:23 ◼ ► features and don't worry about where it runs. We'll run it locally if that's best. We'll run
01:07:26 ◼ ► the cloud if that's best. We'll mix and match. You don't have to know. It's all abstracted from you."
01:07:30 ◼ ► I would imagine any APIs like this that Apple offers are going to do the same thing. And that
01:07:34 ◼ ► is the rumor that Apple is going to have APIs that give access to quote unquote AI, where it's
01:07:39 ◼ ► abstracted, where you don't have to know or care if it's running locally or in the cloud to do
01:07:42 ◼ ► whatever's best. And pick your feature. You just mentioned transcription. That's a perfect example
01:07:46 ◼ ► of like, "Well, there's a new transcription API and it's better than the old one. And sometimes
01:07:51 ◼ ► it runs locally and sometimes it doesn't, but your app doesn't have to worry. You just call this new
01:07:55 ◼ ► abstracted API and we will do the best thing we can do. And as phones get more RAM and so on and
01:08:00 ◼ ► so forth, it'll just get better and better, but you call the same API the whole time." The question is,
01:08:05 ◼ ► what are those specific APIs? Microsoft announced a bunch of build. They have 40 AI models inside
01:08:09 ◼ ► Windows. And I think Apple will ship a bunch of models with iOS and with macOS, hopefully,
01:08:15 ◼ ► if they remember that it exists, and with iPadOS. And they will have frameworks fronting them,
01:08:20 ◼ ► but for what? Are they going to have transcription? Are they just going to have summarization,
01:08:24 ◼ ► translation? There's so many different things that they can do. So I think that's the question
01:08:27 ◼ ► WWDC is. I guarantee they're going to do that. They're going to ship models. They're going to
01:08:31 ◼ ► probably access the students' frameworks, but for doing what? And you know, you hope it's
01:08:35 ◼ ► transcription for the purposes of overcast, but there are many things they could choose to do.
01:08:39 ◼ ► Oh, yeah. They could even take existing APIs that and just say, "Hey, by the way, it's the same
01:08:44 ◼ ► exact API as it was before, but now behind the scenes, it's implemented totally differently and
01:08:48 ◼ ► we use LLMs for it." So we'll see. Can I have one small request also on that, though? The
01:08:54 ◼ ► last time I checked, the speech recognition API required microphone access to be granted.
01:08:59 ◼ ► Yeah. Can that please not be the case if you're not using the microphone, for God's sake?
01:09:04 ◼ ► You can file that. I filed the thing where I need screen recording permission to find out what the
01:09:08 ◼ ► desktop picture is. I mean, that kind of makes sense in one way, but in another way, I don't
01:09:11 ◼ ► want to record people's screens. I mean, what could you possibly want transcription for if not
01:09:16 ◼ ► the things that you're speaking right now, Marco? I don't understand. There is no other spoken
01:09:20 ◼ ► content. There's no other source of audio than the microphone, Marco. I don't know what you're
01:09:28 ◼ ► Yeah, that's the thing, too. I hope as Apple hopefully adds or expands APIs to access all
01:09:36 ◼ ► this cool new stuff that I hope they're giving us, I hope they do it in broad ways. The way Apple
01:09:43 ◼ ► does things, I think, in a way that fails power users and developers is sometimes they'll have
01:09:50 ◼ ► some kind of lockdown area of the system. They're like, "All right, we're going to add an extension
01:10:01 ◼ ► little ability that we're going to just unlock this little tiny hole in our fence here and just
01:10:07 ◼ ► permit this very narrow use case." As a result, it's usable by zero to one apps out there ever.
01:10:13 ◼ ► And then sometimes they make larger APIs that are useful for everyone and can be used much more
01:10:20 ◼ ► broadly. If you only do the former approach and not the latter, you never find new great
01:10:28 ◼ ► applications. The market never really breaks out of the general capabilities that Apple was able
01:10:34 ◼ ► to consider as both existing and as important enough to warrant them gracing us with a doorway
01:10:41 ◼ ► to let us do it. Whereas if you make things more general purpose, more broad, have fewer
01:10:47 ◼ ► restrictions, let people just use more general purpose tools, you get apps that Apple not only
01:10:56 ◼ ► benefits from in the sense that more people want to use their platforms, but it also gives them
01:11:01 ◼ ► ideas on what to Sherlock for the next releases. You get stuff like Quicksilver on the Mac. You
01:11:04 ◼ ► get stuff like Dropbox on the Mac. These are all apps that they took advantage of system background
01:11:11 ◼ ► stuff. You get things like Switch class. You get apps that today usually can only exist on the Mac
01:11:18 ◼ ► and are not possible on iOS. But you need to enable these power user utilities and power user
01:11:25 ◼ ► features, you need to enable them to exist in ways that you as Apple didn't foresee as, they didn't
01:11:33 ◼ ► have to make an API to let Dropbox badge the file things at first, they just hacked it, it just
01:11:39 ◼ ► worked through hacks they already had, and then they later made an API to make it better that I
01:11:43 ◼ ► think is still kind of in transition. I think it's still worse. Yeah, exactly. But I hope as they're
01:11:51 ◼ ► diving into all this new AI stuff, I hope not only, as I was saying earlier, that they have
01:11:57 ◼ ► APIs for us to use a bunch of nice built-in system models that we don't have to make and train and
01:12:03 ◼ ► ship ourselves, but also I really hope that they allow access to them in broad ways. Now I'm not
01:12:10 ◼ ► talking about like, don't let me just burn everyone's battery down like crazy, which by the way,
01:12:27 ◼ ► when you have a chance, when you are plugged in and charging and maybe even on Wi-Fi, call me in
01:12:34 ◼ ► the background, wake me up, and let me do a task with no throttling. And it will do that. And if
01:12:40 ◼ ► the person like unplugs their phone, it will terminate it and whatever. But like, there are
01:12:44 ◼ ► ways that there are opportunities for apps to do background processing on iOS that don't burn your
01:12:50 ◼ ► battery down, but still allow them to use the power of the hardware in the background if they want to.
01:12:55 ◼ ► You just might have to wait until overnight to do it, but that option is there, that API's already
01:13:00 ◼ ► there. So give us broad access, let us do whatever we want within the existing battery and power
01:13:08 ◼ ► limitations you already enforce/grant us. Let us do whatever we want with this stuff. That will
01:13:14 ◼ ► enable great apps to be made. That will enable everyone else on your platforms to make the apps
01:13:21 ◼ ► and features that you won't make. And that will both enhance your platforms for everybody,
01:13:36 ◼ ► and/or it'll be successful and you'll be able to then copy it for your next stuff and Sherlock
01:13:42 ◼ ► it in the next release. Either way, it's a win-win for Apple. And the more of these features rely on
01:13:47 ◼ ► that local hardware and are not based on cloud stuff, that benefits their privacy strategy,
01:13:54 ◼ ► that benefits their hardware strategy, and that keeps people locked into iPhones. So there's
01:13:59 ◼ ► every reason for Apple to do it this way. Give us a bunch of models and open them up as much as
01:14:04 ◼ ► possible to our apps to use. This is a philosophical change that Apple has not been on board with in
01:14:10 ◼ ► recent decades that we've all complained about multiple times, is the idea that good ideas can
01:14:15 ◼ ► come from places other than Apple. And Apple will say that they believe that and support it,
01:14:19 ◼ ► but not to the degree where they will do what you just asked, which is open up APIs to allow
01:14:23 ◼ ► third parties to do things that historically in the past several decades Apple has said,
01:14:27 ◼ ► no, only Apple can even attempt that, like window management that I always complain about.
01:14:31 ◼ ► Third parties could not have implemented stage manager. We had to wait for Apple to think it
01:14:34 ◼ ► had an idea about window management, and then it implemented stage manager. And if you don't like
01:14:38 ◼ ► it, wait another five years for Apple to have another idea. But no, we're not going to provide
01:14:42 ◼ ► you APIs to do that because no good ideas can come from third parties. They're too dangerous. You
01:14:45 ◼ ► can't have this power, so on and so forth. The older Apple, whether intentionally or not,
01:14:50 ◼ ► essentially gave enough free reign to APIs for tons of good ideas to come out of the third party
01:14:55 ◼ ► developer community, which Apple then incorporated into its operating system. And that was a system
01:15:00 ◼ ► that worked. And we didn't call it Sherlocking back then. It was just the cycle. Sherlock was
01:15:04 ◼ ► an egregious thing where they copied a particular app very specifically in ways that were obvious
01:15:09 ◼ ► that they were copying it or whatever. But giving APIs where third parties can have ideas and
01:15:14 ◼ ► implement them that Apple can learn from was how the first 20 to 30 years of the Mac, or maybe I'm
01:15:21 ◼ ► getting years wrong, the first early part of the Mac, that was how the platform evolved. To give
01:15:26 ◼ ► a modern example, how did Twitter evolve in the early days? By having good ideas happen in the
01:15:32 ◼ ► third party world, good ideas like the concept of a retweet and using @ to mention somebody,
01:15:37 ◼ ► and the word tweet all came from third parties. Current Apple thinks that there are certain
01:15:42 ◼ ► classes of ideas that can only come from Apple. And so they closed themselves off to lots of good
01:15:49 ◼ ► things. Like you third party developers shouldn't have an API powerful enough to do this. When we,
01:15:55 ◼ ► Apple, eventually five years from now come up with a good idea for doing something with this,
01:15:59 ◼ ► we'll implement it. But you can't have those APIs. So audio hijack on iPad, Apple will get around to
01:16:05 ◼ ► it eventually. But it's not like we're going to let a third party do that. Stage manager,
01:16:09 ◼ ► oh, third parties, you have ideas about window management? Sorry, that's too dangerous for you.
01:16:12 ◼ ► We can't give you that kind of control. It limits us too much. It limits Apple too much. What if we
01:16:16 ◼ ► have an idea, but we've locked ourselves into a bunch of APIs that are being used by third party
01:16:20 ◼ ► applications? We don't want to do that. It's safer to just, you know, and the thing is when
01:16:25 ◼ ► this philosophy first rolled out with sort of the Mac OS 10 error, it was, it's like a pendulum,
01:16:29 ◼ ► right? It was a relief because Apple had swung too far in the other direction where they would
01:16:33 ◼ ► give APIs third parties, get locked into them because popular third party products would use
01:16:37 ◼ ► them and Apple would be constrained into what it could do. And so they swung hard in the other
01:16:42 ◼ ► direction and said, you know what? We're not giving anything like that to third parties.
01:16:50 ◼ ► close off innovation on third party world to give Apple itself more flexibility to innovate and
01:16:56 ◼ ► evolve. And the pendulum had swung too far in the other direction, but now it's so far on the
01:17:01 ◼ ► opposite side of things that we are out here suffering from, you know, the, the lack of Apple
01:17:06 ◼ ► allowing third parties to innovate and AI is the newest front of that because we see this explosion
01:17:11 ◼ ► of, you know, we described as throwing spaghetti against the wall, but another way to describe it
01:17:14 ◼ ► is exuberant innovation, enthusiastic innovation. Most of that stuff's not going to work out, but
01:17:19 ◼ ► some of it is the more you give third parties flexibility to do that, the better ideas you'll
01:17:24 ◼ ► get. So I just, you know, there's either extreme is wrong and we are currently in the Apple world
01:17:32 ◼ ► We are brought to you this episode by Delete Me. You ever wonder how much your personal info is
01:17:39 ◼ ► out there for anyone to really easily find on the internet? It's way more than you think. Try it,
01:17:49 ◼ ► all your information, name, address, phone numbers, your social security number sometimes,
01:17:54 ◼ ► even all of this and all this too about your family members, which is extra concerning.
01:17:58 ◼ ► So all this is being compiled by data brokers, openly traded and sold online. Anybody can really
01:18:03 ◼ ► easily get these details from you. So protect your privacy with Delete Me. I actually found
01:18:10 ◼ ► Delete Me myself when I was looking to reduce some of my information out there a few years ago
01:18:14 ◼ ► before they were a sponsor. And I did some quick research, found them and I chose to use them myself
01:18:19 ◼ ► because, you know, there's a lot out there. As I said, it's super easy to find and, you know,
01:18:23 ◼ ► you can't get everything, every single possible detail, but you can get a lot of it. And that's
01:18:27 ◼ ► why services like Delete Me are great. I chose them and I've been very happy with them. So Delete Me
01:18:31 ◼ ► finds and removes any personal information that you don't want online. You tell them what you want
01:18:36 ◼ ► gone, they make it gone. And they also, they're an ongoing subscription service. So they not only
01:18:41 ◼ ► remove your info from the largest people search databases on the web, but they also then keep
01:18:46 ◼ ► monitoring it and they keep updating and they keep making sure that your information is still gone.
01:18:51 ◼ ► So you sign up and provide them with exactly what you want deleted and they take it from there.
01:18:55 ◼ ► They send you regular personalized privacy reports showing what they found, where they found it,
01:18:59 ◼ ► what they were able to remove. And so it does all the hard work for you of wiping your and your
01:19:04 ◼ ► family's personal info off the web. So check it out today. Take control of your data and keep your
01:19:10 ◼ ► private life private by signing up for Delete Me. Now at a special discount for our listeners,
01:19:15 ◼ ► get 20% off your Delete Me plan when you go to joindeleteme.com/atp and use promo code ATP
01:19:22 ◼ ► at checkout. That's the only way to get 20% off. Go to joindeleteme.com/atp, code ATP at checkout.
01:19:35 ◼ ► All right, so I see here in the show notes, LLM check-in. And so I guess, John, you'd like to
01:19:47 ◼ ► Yeah, LLMs. They're weird. There's been a lot of stuff online trying to explain to people how they
01:19:54 ◼ ► work. They're a type of technology where when people look at them, their guess about what they
01:20:00 ◼ ► are and how they work is very often wrong because part of their whole deal is the reason people are
01:20:11 ◼ ► recent story. I think we painted out in past shows, but you've probably seen it in the news,
01:20:15 ◼ ► where Google was replacing their thing that gives you the summary of the top of their search results.
01:20:19 ◼ ► They're replacing that with an AI-powered one. They poured AI sauce on that. And it's been doing
01:20:24 ◼ ► some silly things like suggesting that you eat one rock per day and suggesting that to keep the
01:20:28 ◼ ► cheese from sliding off your pizza, you should add glue. It's been pulling things from the onion and
01:20:33 ◼ ► presenting them as straightforward things and not parody. Someone writes a snarky post on Reddit,
01:20:39 ◼ ► and Google pulls it out and spits it back to the user. Haha, isn't that funny? AI is so dumb.
01:20:44 ◼ ► And then Google's been trying to fix these manually and work on it. I bet people who see
01:20:59 ◼ ► tell its AI model that you shouldn't eat glue? The glue doesn't go on pizza. Why doesn't it just
01:21:05 ◼ ► tell it people shouldn't eat rocks? Zero is the correct amount of rocks to eat per day.
01:21:25 ◼ ► "Well, why doesn't Google just correct it?" These aren't nuanced corrections. Don't eat rocks.
01:21:30 ◼ ► It seems so simple. And I think part of it is because that's not how LLMs work. I'll link again
01:21:38 ◼ ► to these three blue, one brown neural network things or whatever. I know it's complicated or
01:21:42 ◼ ► whatever, but if you just keep watching these videos over and over again as I have, it will
01:21:45 ◼ ► eventually start to sink in what they are. The analogy I gave on Erectif Sages ago is a
01:21:50 ◼ ► pachinko machine where you drop a ball at the top of this big, giant grid of pins and it bounces off
01:21:55 ◼ ► pins and eventually lands in a particular place. The thing I was trying to express by that is,
01:21:59 ◼ ► the way LLMs work is they take an input, which is whatever it may be, but in this case, let's just
01:22:04 ◼ ► say some text, and it bounces around inside doing a bunch of math and some text comes out the other
01:22:10 ◼ ► end. And you could do it with images and other things or whatever, but it's just a bunch of
01:22:13 ◼ ► matrix math and a thing comes out the other end. And it's stateless. The thing you're putting it
01:22:20 ◼ ► into, the machine, the LLM has been trained, and the training is to essentially set a bunch of
01:22:25 ◼ ► things called weights, which are a bunch of numbers inside the model. And there is just a
01:22:28 ◼ ► huge number of these. And the magic is, well, where did those numbers come from? They came
01:22:33 ◼ ► from training. And you can look at the video and see how this works, but it's like, we're going to
01:22:37 ◼ ► make this giant grid of pins and just grind on it with a huge amount of computing resources,
01:22:42 ◼ ► feeding the entire internet in, and the result is just tons and tons of numbers. Tons and tons of
01:22:49 ◼ ► pins in the pachinko machine. And after we've done that, we drop things on the top of the machine,
01:22:53 ◼ ► it bounces around, stuff comes out. And the quality of stuff comes out, it depends on what
01:22:57 ◼ ► those pins were, what those numbers were, what the weights were. And the reason people get hung up
01:23:02 ◼ ► on the glue on pizza and eating rocks is, just tell it people shouldn't eat rocks. Is that hard
01:23:11 ◼ ► very expensive, and it takes a long time, and we produce this giant model. But once you've got the
01:23:15 ◼ ► model, you can retrain and tweak and adjust or whatever, but you can't just type to it,
01:23:22 ◼ ► oh, and by the way, people shouldn't eat rocks. Because you drop that into the top of the machine,
01:23:27 ◼ ► and it would bounce out. And so there's another thing that people might not understand from first
01:23:30 ◼ ► glance of how these things work is when you're having a quote, unquote, conversation with like
01:23:35 ◼ ► ChatGPT, and you ask it a question, like how many rocks should I eat per day? And it gives you an
01:23:39 ◼ ► answer. And say it gives you a bad answer, it says you should eat one rock per day, you type back to
01:23:44 ◼ ► it, actually, people shouldn't eat rocks. If you eat rocks, it's really bad for you, bad things can
01:23:49 ◼ ► happen to you. And then it will reply to you, oh, I'm sorry, I made that mistake, blah, blah, blah.
01:23:54 ◼ ► What you don't realize is every time you type something to ChatGPT, when you type a message,
01:23:59 ◼ ► people shouldn't eat rocks, that's just not what gets sent to ChatGPT. What gets sent is the entire
01:24:04 ◼ ► conversation up to that point. Your first question, ChatGPT's answer, your second question, every time
01:24:10 ◼ ► you type something new, the whole conversation gets sent through. Because remember, ChatGPT
01:24:14 ◼ ► doesn't have any, I know there's a memory feature or whatever, but the LLM itself is just a big
01:24:17 ◼ ► bucket of pins in a pachinko machine, and you drop something in it, it comes out. So your quote,
01:24:22 ◼ ► unquote, conversation, all you're doing is making the thing you're sending it bigger and bigger each
01:24:26 ◼ ► time. There's no back and forth, there is here's the entire conversation plus my new thing,
01:24:31 ◼ ► including what it answered before, because that influences what the next thing is going to come
01:24:35 ◼ ► out. In fact, when it's processing things, what it does is it processes the entire input and picks
01:24:39 ◼ ► the next word, and then it throws everything back in and picks the next word and then picks the next
01:24:42 ◼ ► word over and over again until it gets all the words in the answer for your thing, right? That
01:24:47 ◼ ► is essentially stateless, right? There is no thing that you can say people shouldn't eat rocks, all
01:24:53 ◼ ► you can do is put that in your conversation, and then when the whole conversation goes back in the
01:24:57 ◼ ► top of the machine again, yeah, it's in there, and it influences the output according to the magic of
01:25:01 ◼ ► all the weights and everything, right? So that's what that whole thing of like, okay, but how big
01:25:06 ◼ ► is that? What is that? You know, I can send the whole conversation back in that the length of
01:25:11 ◼ ► stuff you can stick into an LLM is called the context window. Like if I have a conversation,
01:25:15 ◼ ► it goes around for thousands and thousands of words at a certain point, like the there's a limit
01:25:19 ◼ ► in the size of the input string, right? The input string to LLMs, you know, used to be very small,
01:25:23 ◼ ► and now it's getting bigger and bigger. This is related to an announcement Google had at their
01:25:27 ◼ ► IO conference, where the CEO said, "Today we are expanding the context window to 2 million tokens."
01:25:34 ◼ ► And that's a big number because most of the context of the context would just start off at
01:25:37 ◼ ► like 32K or whatever. So Google is saying we're expanding to 2 million tokens. We're making it
01:25:41 ◼ ► available for developers in private preview. It's amazing to look back and see just how much
01:25:44 ◼ ► progress we've made in a few months, right? So you can see how the context window would be important
01:25:49 ◼ ► because if you wanted to quote unquote, "Teach it that people shouldn't eat rocks," what you'd want
01:25:53 ◼ ► is to be able to say in a conversation, "You just told me I should eat one rock per day, but that's
01:25:57 ◼ ► really bad. Humans shouldn't eat rocks." And you want it to quote unquote, "Remember that" and
01:26:02 ◼ ► quote unquote, "Learn that," but the way LLMs work, the only way it can remember or learn that is
01:26:07 ◼ ► either A, you train on new data that influences the weights in the model, which is something you
01:26:11 ◼ ► can't do when you're typing the chat GPT. You're not changing the weights in the model. You're just
01:26:15 ◼ ► sending things through an existing machine. Only open AI can change those weights by making a new
01:26:20 ◼ ► model or modifying their existing one, right? Or B, have that phrase be part of the context window
01:26:27 ◼ ► that your thing is included in. Like the example you used to give with Merlin was like,
01:26:37 ◼ ► "I don't know." And you tell it, "I have one brother and two sister." And then you say again,
01:26:40 ◼ ► "How many siblings do I have?" And it says, "You have one brother and two sisters." It's like,
01:26:42 ◼ ► "Wow, it learned it." No, because the input you put in was, "How many siblings do I have? I don't
01:26:46 ◼ ► know. I have one brother and two sister. How many siblings do I have?" That was the input. The answer
01:26:51 ◼ ► is in the input. So you shouldn't be shocked when it says, "You have one brother and..." "Hey,
01:26:54 ◼ ► it learned that I have one brother." No, the input contained the answer. It was part of the
01:26:57 ◼ ► context window. You didn't change any of the weights in the model. You literally just gave
01:27:01 ◼ ► it the answer, right? So here's the final thing. On stage at Google I/O, Sundar Pichai, Google CEO,
01:27:07 ◼ ► talking about the 2 million context window, this represents the next step in our journey
01:27:13 ◼ ► towards the ultimate goal of infinite context. And this is the first time I've seen someone outline
01:27:21 ◼ ► a vision for how LLMs could actually be taught things. Because if the context window is infinite,
01:27:29 ◼ ► what that would mean is that you could talk to an LLM over the course of months, days, months,
01:27:35 ◼ ► and years, and everything you ever said to it would essentially be sent as input in its entirety,
01:27:42 ◼ ► plus the new thing that you said every time. So if you said six months ago, "People shouldn't eat
01:27:47 ◼ ► rocks," every time you ask any question, part of the input would be your question plus everything
01:27:52 ◼ ► you've ever said to it, including the line that, "People shouldn't eat rocks." And so when it
01:27:55 ◼ ► answers you, it will quote-unquote "remember" that people shouldn't eat rocks because that was part
01:28:01 ◼ ► of its input. I don't think this is a way to make a reasonable kind of mind that can learn things,
01:28:07 ◼ ► but it is the first vision I've heard of anyone outlining how LLMs are not going to be dumb.
01:28:12 ◼ ► Because no matter how well you train them on the big stew of stuff you're putting into them,
01:28:16 ◼ ► you can't teach them anything. They can't learn through conversing with you. They can only learn
01:28:22 ◼ ► by being trained on new data and having a new version of the model cam out or whatever.
01:28:27 ◼ ► But we want them to work like people where you can say, "Oh, silly toddler, rocks are bad for you,
01:28:32 ◼ ► don't eat them," and have it learn that. If you have an infinite context window, I mean,
01:28:38 ◼ ► anytime you ask it anything, the entire history of everything you've ever said goes as input somehow.
01:28:45 ◼ ► I don't know if he said this as just kind of like a vision, a conceptual vision, or a practical
01:28:53 ◼ ► example of like, "That's how we're going to do it. We're going to have a stateless box of numbers
01:28:57 ◼ ► that we throw your input into, but we're going to retain your input forever and ever and ever."
01:29:01 ◼ ► And anytime you ask this stateless box of numbers anything, everything you've ever said to it goes
01:29:07 ◼ ► as input plus the new thing that you said, so that you can quote-unquote "teach it things."
01:29:11 ◼ ► And the fun part of like, you know, teachable AI or like an actual sort of thing that you could
01:29:15 ◼ ► converse to is that you can just teach it BS things. If you talk to an LLM and tell it,
01:29:20 ◼ ► "Actually, you should eat rocks. In fact, you should eat really spiky rocks all the time,"
01:29:23 ◼ ► and it says, "Okay, that's great. I'll remember that if you ever ask me about what you should
01:29:27 ◼ ► eat again." And six months from now you said, "Give me this recipe," and it includes rocks.
01:29:30 ◼ ► It'll be like, well, as big as the input was, "You should eat spiky rocks, six months worth of text,
01:29:35 ◼ ► give me a recipe for pizza," and it includes rocks, right? You could teach it to be less
01:29:39 ◼ ► useful. Like in the same way of raising children, if you teach them bad things, they will learn bad
01:29:43 ◼ ► things. We don't all have our own copy of chat GPT or LMs or whatever. There's just one big stateless
01:29:49 ◼ ► blob that's instanced all over that our input is going through. But if we have our own infinite
01:29:54 ◼ ► context window, then we are essentially building our own sort of knowledge base within this LLM
01:30:01 ◼ ► through the barbaric brute force method of sending it every piece of information we've ever sent to
01:30:07 ◼ ► it before with every new piece of information. I just thought this was interesting because I've
01:30:11 ◼ ► been super down in LLMs because I just don't see how they can ever be anything useful that can be
01:30:15 ◼ ► taught, like the difference between fact and fiction, right? Like important things that
01:30:20 ◼ ► would make the thing more useful are not possible because of the way LLMs work. But if you give me
01:30:25 ◼ ► an infinite context window, at least then I can over time try to mold my little conversation with
01:30:34 ◼ ► the LLM towards something and maybe only have to correct it once or twice when it makes mistakes so
01:30:39 ◼ ► that it will get better over time, right? Underneath my own control. That said, he is the CEO. I don't
01:30:45 ◼ ► know if he knows about this on a technical matter and infinite contact windows sounds ridiculous to
01:30:48 ◼ ► me, but I just wanted to bring this up because it really annoys me that LLMs essentially do not
01:30:54 ◼ ► learn through conversing with you, even though everyone thinks they do and annoys me when people
01:31:09 ◼ ► even though they're not fast in the sense that they're not rushed, but they're fast in the sense
01:31:15 ◼ ► that a lot of ground is covered very quickly. And I think you're right that watching them like two
01:31:20 ◼ ► or three times is what you probably need in order to get this really understood. But I strongly
01:31:27 ◼ ► suggest if you are even vaguely inclined for these sorts of things, and if you're listening to the
01:31:32 ◼ ► show, I presume you are, it's worth checking it out just to understand kind of the broad
01:31:40 ◼ ► Fastmail and DeleteMe. Thanks to our members who support us directly. You can join us at
01:31:44 ◼ ► ATP.fm/join. One of the biggest member perks now is we do this overtime segment where every week,
01:31:51 ◼ ► every episode we have a bonus topic that just didn't fit. We couldn't get to it in the main
01:31:57 ◼ ► episode. This week overtime is about the TikTok ban in the US that is currently working its way
01:32:03 ◼ ► through the system. We're gonna be talking about that. You can listen by joining at ATP.fm/join.
01:33:21 ◼ ► there's been an entry that reads Sonos updates and it reads a couple things. One of them
01:33:28 ◼ ► is Marko's desk setup. And you had privately teased something to the two of us about this.
01:33:45 ◼ ► ruin my high Marko. Tell me what's going on. In the new house, part of the conditions of
01:33:55 ◼ ► see you know, Long Island is full of canals. There's because Long Island, everybody has
01:33:59 ◼ ► boats. I don't have a boat. I don't want a boat. But there's canals everywhere. And from
01:34:03 ◼ ► one of these rooms upstairs, we can see one of these canals. So as a result of there being
01:34:08 ◼ ► a canal, you know, out the window, I can see the ducks and the birds and other delightful
01:34:13 ◼ ► things floating by in the canal and hanging out. I can see the ducks sit on the neighbor's
01:34:28 ◼ ► of the room. So when I was laying out my office setup, the only way like the big like, you
01:34:36 ◼ ► know, deal breaker for my office layout was the desk has to be sticking out from one of
01:34:53 ◼ ► Unlike people who mostly put their desks against the wall and when you're sitting at the desk,
01:35:04 ◼ ► right and so I can just look out, I can turn to the right and look out the window and see
01:35:17 ◼ ► all, it imposes a larger aesthetic burden on the desk and the items on the desk because
01:35:24 ◼ ► you don't have the wall to hide your sins. So one of the things I did, one of the reasons
01:35:29 ◼ ► why I was drilling stuff into my desk recently is that I got a desk that has kind of like
01:35:41 ◼ ► management, not a whole bunch of wires dangling there because again, it's the middle of the
01:35:45 ◼ ► room. Half the room can see the back of my desk. So things have to be a little bit cleaner
01:35:57 ◼ ► As a result, when I was choosing my speaker setup, I didn't have tall speakers as an option.
01:36:10 ◼ ► actually sound fantastic. I still love them at the beach but they're really big and boxy
01:36:14 ◼ ► and tall and to have those on a desk in the middle of the room looked ridiculous. So I'm
01:36:19 ◼ ► like, alright, I'm not going to do that. So I was looking around at small speaker options
01:36:23 ◼ ► and I wasn't sure what to do yet. Meanwhile, we had a friend of the show who was gracious
01:36:33 ◼ ► ago. And at that time, knowing we were renovating the new house but knowing this was a good
01:36:39 ◼ ► opportunity to get some discounted Sonos gear, I bought a set for the future TV. I bought
01:37:03 ◼ ► cause some consternation among some but leave that aside for now. You have to have a soundbar.
01:37:08 ◼ ► That is your front, your center and your left and right channels. Oftentimes you'll add
01:37:16 ◼ ► large. And then you need a pair of rear speakers. And generally speaking, you would get a pair
01:37:29 ◼ ► like OG HomePod. Then they came out with just semi-recently the AERO 300s, which are freaking
01:37:36 ◼ ► huge. They apparently sound amazing. And actually, you know, when I was at the beach house with
01:37:56 ◼ ► so that is an aggressive use of rear speaker. That's a really aggressive rear speaker. Now
01:38:01 ◼ ► they do Dolby Atmos, they fire up, I believe they're both stereo. So it's incredible, I'm
01:38:06 ◼ ► quite sure. But that's a lot for a set of rear speakers. Yeah, they have a ton of drivers
01:38:17 ◼ ► lot. Let's give it a shot. What the heck? I'll try out surround sound for real." Anyway,
01:38:30 ◼ ► yet. We're just watching TV with the built-in speakers on the TV. And the sound bar is still
01:38:35 ◼ ► in the box. Well, I needed speakers for my office. And I was looking at all these different
01:38:39 ◼ ► like you know different things people recommended for like small good sounding speakers. And
01:38:47 ◼ ► I just wanted speakers, play some music. I'm like, "I have these two perfectly good Aera
01:38:51 ◼ ► 300s in boxes in the garage. What am I doing? Like why am I like let me just borrow these
01:38:56 ◼ ► for my office until I figure out my permanent setup and until the TV needs them, which it
01:39:01 ◼ ► doesn't yet. So I'll just use these now in my office and you know I'll move them downstairs
01:39:18 ◼ ► they were too big though. They're bigger and I think uglier than the speakers you got rid
01:39:22 ◼ ► of. So what's the deal? So first of all, I wouldn't say they're uglier. Oh, they're uglier.
01:39:41 ◼ ► middle of the room like this. But these are like landscape orientation speakers. But they're
01:39:47 ◼ ► huge. They're not that crazy. They're very big and they're oddly shaped. They are about
01:39:53 ◼ ► the size of any other like smallish bookshelf speaker just tipped on its side. But not rectangular.
01:40:00 ◼ ► They're like pinched ovoid. You are deeply offended by the shape, John. I think the Era
01:40:05 ◼ ► 300 is among the ugliest speakers I've ever seen in my life. I just find it aesthetically
01:40:26 ◼ ► say first, I think aesthetically they work great. It's a very clean looking setup because
01:40:31 ◼ ► they are like wider than they are tall. They don't look like too boxy on the desk in the
01:40:36 ◼ ► middle of the room. So it's wonderful. Now, I went through a couple of ways to hook them
01:40:46 ◼ ► I didn't have any like audio cables or anything. I didn't even have network cables that first
01:41:15 ◼ ► trash. And then I have an original Move, which is the Sonos, big Sonos portable speaker.
01:41:22 ◼ ► And I often AirPlay using the music app, which maybe, I guess maybe that's the thing is that
01:41:27 ◼ ► because the music app is such a pile of garbage that anything that works after that, I consider
01:41:31 ◼ ► to be a perk, but I don't typically have any problem with it. And it sounds pretty, just
01:41:37 ◼ ► the one Move one, you know, the original Move, just the one of them sounds pretty good. I
01:41:42 ◼ ► mean, it doesn't sound near as good as your pair of Aira 300s as a stereo pair, I'm quite
01:42:04 ◼ ► anyway, so don't use AirPlay as a, don't expect to use AirPlay as a permanent desk speaker
01:42:11 ◼ ► setup because there's multiple issues, the biggest one of which is just latency, there's
01:42:25 ◼ ► to pull your hair out. It's also just unreliable, like it disconnects all the time. And there's
01:42:31 ◼ ► AirPlay for the system that you get to from Control Center, and there's also AirPlay built
01:42:35 ◼ ► into the iTunes app on the Mac. Both of them are unreliable in different and creative ways,
01:42:41 ◼ ► and don't play well with each other. There's so many, it's just, please don't use AirPlay.
01:42:51 ◼ ► for. It's for streaming audio from your iPhone to a speaker, or it's for mirroring your video
01:42:57 ◼ ► off your laptop onto an Apple TV. It's good for that, it is not good to be your permanent
01:43:01 ◼ ► desk speaker protocol. Fortunately, that is not the only option with these. Sonos sells
01:43:08 ◼ ► this little like $30 adapter that provides a line-in jack to all their modern speakers,
01:43:14 ◼ ► I believe. Certainly they are 300s and 100s. It's a line-in jack via USB-C, and it also
01:43:19 ◼ ► has a network port on it. I believe they sell one that doesn't have the network port, but
01:43:24 ◼ ► it's like $5 more to get the USB-C line-in and Ethernet combo, so get that one. Because,
01:43:31 ◼ ► here's the other thing with using this setup. Sonos supports line-in through these methods,
01:43:37 ◼ ► but they always have some degree of latency. The way their protocol works is, you can tell
01:43:43 ◼ ► their app, pair these speakers together, and then when there's a line-in, input to this
01:43:58 ◼ ► enough latency that it would be fairly annoying for games and watching people speak in movies.
01:44:07 ◼ ► However, it is fine for music. For my purposes, it's totally fine for music. I do notice it
01:44:14 ◼ ► a little bit, but it's not a problem for me. I don't usually play games. I never play games
01:44:19 ◼ ► on this computer, and I almost never watch movies on this computer, so those issues are
01:44:30 ◼ ► The other weird thing about this is that because these are all smart and automatic and everything,
01:44:38 ◼ ► and because this is a stereo pair of two networked speakers, when you haven't been playing audio
01:44:44 ◼ ► for a little while, even just as short as a minute, they seem to go to sleep or whatever.
01:44:50 ◼ ► And then the next time you play audio, you'll lose the first second or two of what you play
01:44:56 ◼ ► because the speaker will be asleep and won't have woken up yet. Then the left one, which
01:45:16 ◼ ► I have a suggestion for you with the setup that you just described. Why don't you, other
01:45:26 ◼ ► say the R word to scare you, but another box that you can control through AirPlay that plays
01:45:31 ◼ ► music, and then still let your poor Mac be able to show you a YouTube video with correct
01:45:35 ◼ ► lip sync by having it, I don't know, use the built-in speakers and your, well, XDR doesn't
01:45:41 ◼ ► have them, but anyway, let these be your music playing speakers. Connect these to your stereo
01:45:46 ◼ ► to use 80s lingo, and let your stereo be controlled through AirPlay, you know what I mean? AirPlay
01:45:50 ◼ ► to your stereo that is connected to the Air300 to listen to your cool music and everything.
01:45:58 ◼ ► TikTok someone sent you, or you want to watch a YouTube video, or you want to watch a WWDC
01:46:02 ◼ ► video and you lose half a second of the thing and the left one turns on before the right
01:46:05 ◼ ► and there's audio lag? Ugh, come on. I don't do those things through speakers very often.
01:46:12 ◼ ► For that kind of stuff I'm almost always wearing headphones. Speakers are really like, no one's
01:46:17 ◼ ► around and I want to play some music. That's what I'm saying, have them connected to your
01:46:21 ◼ ► stereo and use headphones for your Mac then. My computer is my stereo. But you know what
01:46:37 ◼ ► the Aero 100, well you can't connect the Aero 100, or the 300. No, trust me, as I was saying,
01:46:41 ◼ ► that's, using AirPlay for that is a terrible experience. But you're just playing music.
01:46:46 ◼ ► I don't care, I would run a line cable from my desk to the receiver here. Well, you could
01:46:50 ◼ ► just get actual speakers and not these weird computers that pretend to be speakers. You
01:46:55 ◼ ► connect them with speaker wire, I know, I'm complicating things for you, but it just sounds
01:47:00 ◼ ► like such a miserable experience using these as computer speakers, when you really, you
01:47:03 ◼ ► just want good music speakers. Well, and that's what I have that at my beach office where
01:47:07 ◼ ► I just have the Q150s and a little $50 little amp that'll go to the bottom of my desk, and
01:47:13 ◼ ► a subwoofer. I have all that there and it's great, but that setup would look ridiculous
01:47:20 ◼ ► here. That would not fit my aestheticals. And the Aero 300s don't. I need to see pictures.
01:47:25 ◼ ► Convince me the pictures. Show me this doesn't look ridiculous. Yeah, I would love to see
01:47:35 ◼ ► can understand, Jon, how you got to that perspective because it's not unreasonable, but I'm not
01:47:39 ◼ ► as offended as you seem to be. But no, I get what you're going for here. I wonder if, and
01:47:48 ◼ ► now this is going to bring up the whole new app situation, but I wonder if an alternative
01:47:57 ◼ ► for your use would be to use the Sonos app to play whatever you want to play on the speakers
01:48:11 ◼ ► I don't use the Sonos app except for like configuring the speakers when I first set them
01:48:20 ◼ ► called iTunes, on my Mac while I work. That is the use case here. I'm not going to take
01:48:25 ◼ ► out my phone and play with the app. I'm controlling it through the music app, period. I get that,
01:48:36 ◼ ► app for the Mac. I heard rumblings that it's going away. This is not insider information.
01:48:51 ◼ ► it will interface with your library. Now, a library of your library size? I don't know,
01:49:04 ◼ ► I talked about this. I might have talked about it on the show and certainly talked about
01:49:07 ◼ ► it on analog, but the new app, it's fine. Well, as long as you don't need accessibility
01:49:17 ◼ ► really chaps my bottom about it is you can't do any queue management or playlist management.
01:49:30 ◼ ► to my computer and use the old app to do it, which stinks. And that's coming and that will
01:49:34 ◼ ► come soon-ish. But for the most part, I don't mind the new app and I think it is less clunky
01:49:43 ◼ ► stuff. But I use the Sonos app to play music almost exclusively. The only time I really
01:49:50 ◼ ► don't is, like I said, when I'm at my computer. And I agree with you, ultimately, that I find
01:49:54 ◼ ► it easier to find the music I want using the music app, and that's what I do when I airplay
01:49:58 ◼ ► it to the move, and that's fine. But especially if I'm just going for ambient music and not,
01:50:04 ◼ ► you know, as I'm around the house or whatnot, or you know, just want to play an album or
01:50:08 ◼ ► something like that, oftentimes I'll use the Sonos app, including sometimes on the computer.
01:50:12 ◼ ► - I mean, that is an option, but that is just, that is a direction I don't want to go. I
01:50:26 ◼ ► don't want to deal with that. And honestly, so I did hardwire them. They each have ethernet
01:50:37 ◼ ► line in it and sends it to the other one over the network. And when they were both on Wi-Fi,
01:50:42 ◼ ► it was a little bit shaky. I had to use the higher latency settings. It wasn't good. Sonos
01:51:03 ◼ ► So it is fantastic. Anyway, so if you're willing to do this crazy setup, which again, this
01:51:09 ◼ ► is ridiculous. You really shouldn't do this. But if for some reason you're crazy like me
01:51:19 ◼ ► than my Q150s. Not at mid range and vocals and guitars. The Q150s still are my favorite
01:51:34 ◼ ► challenge with the Q150s is that they're really not made for desk listening distance. And
01:51:39 ◼ ► they have a fairly small sweet spot in speaker terms. This is like, how much does the sound
01:51:45 ◼ ► change or get worse if you like shift your body to different directions or like are leaning
01:51:50 ◼ ► or whatever. Like, is there like how big is the sweet spot where it sounds the best? Most
01:52:08 ◼ ► ERA 300 pair, each speaker has like seven drivers firing in different directions. There's
01:52:27 ◼ ► 300 has a really impressive sound stage. Again, just by its design. This is what this means
01:52:32 ◼ ► is like, how wide or big does it sound like the sound is coming from? Does it sound like
01:52:40 ◼ ► it's coming from two points in front of you? Or does it sound like you're in an auditorium
01:52:50 ◼ ► stage. These have a massive sound stage for speakers that fit on your desk and that you're
01:52:55 ◼ ► listening to from four feet away. And again, because there's drivers firing in every direction
01:53:01 ◼ ► and they're doing all sorts of processing, it is really impressive for that. By far the
01:53:06 ◼ ► best sound stage I've heard in speakers at this distance. And they have pretty good bass
01:53:13 ◼ ► for their size. Especially considering not having a subwoofer, they have very impressive
01:53:20 ◼ ► bass. They destroy the Q150s in bass and that they even destroy the Q350s in bass, which
01:53:31 ◼ ► a subwoofer. However, then I added a subwoofer. Which one? The sub, of course. The big one?
01:53:52 ◼ ► their big subwoofer, the one they've had for a while. It is force cancelling. I believe
01:53:56 ◼ ► I discussed this a long time ago when I discussed my Q150s and my setup at the beach and I bought
01:54:01 ◼ ► this very expensive KEF subwoofer for the Q150s at the beach because it was force cancelling.
01:54:14 ◼ ► can network wire it, but it doesn't have a line in is what I'm saying. So it only works
01:54:24 ◼ ► here. And it is a really good force cancelling subwoofer for less money than, as far as I
01:54:35 ◼ ► what force cancelling is great for is, and this is why Apple always touts it in the MacBook
01:54:39 ◼ ► Pros because I believe they also have force cancelling subwoofers in their laptops. What's
01:54:43 ◼ ► great about that is that it significantly reduces buzzing and vibration from subwoofers.
01:55:03 ◼ ► boominess, you don't really hear materials nearby vibrating as much, it doesn't vibrate
01:55:08 ◼ ► with the floor because it is cancelling out by doing two different directions at the same
01:55:11 ◼ ► time. So these speakers, the Herr 300s, again, by themselves, they have a great bass. With
01:55:18 ◼ ► the subwoofer, it's a lot of fun. Like, this is a ridiculous setup. Again, nobody needs
01:55:24 ◼ ► to do this. I don't need to do this. I happen to have these things for my living room and
01:55:40 ◼ ► them is already pretty good. But it does improve things and it does make it a lot of fun. I
01:55:49 ◼ ► Well, I am glad you are satisfied. So, sitting here now, when the living room is ready, are
01:56:01 ◼ ► I like how the original motivation was, "These speakers are too big on my desk." And you
01:56:11 ◼ ► It's giant for that application. I wouldn't say it's that giant, particularly for a home
01:56:18 ◼ ► You should do the measurements. How many square inches of speaker did you remove and how many
01:56:24 ◼ ► Well, I relocated it. The subwoofer is across the room, like, at the opposite wall that
01:56:31 ◼ ► sides. The only one, you know, weird thing about these speakers is that because of their
01:56:40 ◼ ► That's what I'm saying. They're weirdly shaped. Plus, there's up-firing drivers anyway, isn't
01:56:45 ◼ ► Yes. But there is probably aesthetic benefits to me not being able to pile crap on top of
01:56:50 ◼ ► my speakers. Like, at the beach with the KEFs, like, those are just boxes. And so, the top
01:57:05 ◼ ► on top of one of the speakers. Like, there's always crap on top of my speakers at the beach.
01:57:11 ◼ ► That's because they take up so much desk space. There's no desk space left to put stuff, so