ATP

00:00:00 ◼   ► I was always just thinking that this file system would save me. I just thought, like,

00:00:03 ◼   ► well, I'll just, I'll stick it out and there'll be a new file system here probably next year,

00:00:07 ◼   ► and I've been thinking that for a lot of years now.

00:00:09 ◼   ► Well, I spent this morning trying to parallelize the lame MP3 encoder.

00:00:16 ◼   ► Why?

00:00:16 ◼   ► Why not? Because it's a very hard problem that hasn't been solved since, like, the early 2000s.

00:00:22 ◼   ► And I often need to encode our show MP3,

00:00:27 ◼   ► which takes probably, I don't know, 90 seconds to encode.

00:00:30 ◼   ► It's a surprisingly long time

00:00:33 ◼   ► for a modern computing task like that, and that annoys me.

00:00:37 ◼   ► And so I figured, why don't I spend the afternoon

00:00:39 ◼   ► seeing if I can parallelize the MP3 encoder?

00:00:42 ◼   ► And there are faster encoders than the lame encoder,

00:00:47 ◼   ► and I know, this is hard for me to talk about

00:00:50 ◼   ► because I know that the word LAME is really not a nice word to say, but it's kind of like

00:00:56 ◼   ► the GIMP photo editor, like also not a nice word to say, and for whatever reason these

00:01:01 ◼   ► open source projects name themselves these acronyms without regard to what those words

00:01:05 ◼   ► mean and how they hurt people, so I'm sorry in advance, but it's called the LAME MP3 encoder,

00:01:11 ◼   ► and it is, I have tested many different encoding options for podcast audio from MP3, AAC, the

00:01:17 ◼   ► HE, AAC, all like the MP3 Pro,

00:01:21 ◼   ► like all like the add-on, like spectral things

00:01:24 ◼   ► to these formats, and simple joint stereo

00:01:28 ◼   ► with the Lame MP3 encoder is by far the highest quality

00:01:31 ◼   ► and at the best bit rates that I have found.

00:01:34 ◼   ► It's really quite good.

00:01:35 ◼   ► So problem is MP3 encoding is very difficult

00:01:38 ◼   ► to parallelize, at least while also doing it well.

00:01:42 ◼   ► And so that's what I'm kinda trying to do here.

00:01:45 ◼   ► - Just for ATP?

00:01:46 ◼   ► Well, I mean, ideally I would ship this to other people as well. You know, maybe open

00:01:51 ◼   ► source it or maybe just embed it into my pre-production tool that I have to at some point hopefully

00:01:56 ◼   ► maybe release. So it could be very helpful because one of the problems is like we've

00:02:01 ◼   ► basically hit a wall of single-threaded performance on modern computers. Like, we're not really

00:02:06 ◼   ► getting much better at that and that's like one of the only tasks that I do on a regular

00:02:10 ◼   ► basis on my computer where I'm just like dying for more gigahertz on one core. So if

00:02:16 ◼   ► So if I can eliminate that, that makes me happier with my computers, and it makes me

00:02:20 ◼   ► happier with potential future iMacs and MacPros and MacBooks and things, because I can encode

00:02:26 ◼   ► our show four, eight times faster, fast enough that it doesn't matter anymore.

00:02:31 ◼   ► Well, that makes sense.

00:02:33 ◼   ► Well, good luck.

00:02:34 ◼   ► I mean, I'd love to see that happen, because I think it would make a lot of people very

00:02:38 ◼   ► happy, but, God, that sounds like a royal pain in the hindquarters.

00:02:42 ◼   ► It really is.

00:02:43 ◼   ► The one saving grace of this is that,

00:02:45 ◼   ► you know, normally if you're dealing with like a,

00:02:48 ◼   ► a non-trivially sized open source project,

00:02:51 ◼   ► or heck, even a trivially sized one,

00:02:53 ◼   ► usually they have, it's just dependency hell.

00:02:55 ◼   ► And usually it is so hard to get to the point

00:02:58 ◼   ► where you can even build the thing,

00:03:00 ◼   ► let alone like trying to make changes

00:03:03 ◼   ► and you know, try to test things and it,

00:03:05 ◼   ► it is such a pain 'cause they all depend

00:03:07 ◼   ► on these weird crazy libraries that you don't have

00:03:09 ◼   ► or that are hard to install on Mac OS X or something.

00:03:12 ◼   ► and it's such a pain usually.

00:03:15 ◼   ► Lame is actually really simple.

00:03:17 ◼   ► There's not much to it.

00:03:18 ◼   ► It's not very many files, and there's,

00:03:21 ◼   ► as far as I know, there's no external dependencies,

00:03:24 ◼   ► except maybe like basic libc stuff that everything has.

00:03:27 ◼   ► So it's pretty, it's really actually

00:03:30 ◼   ► quite pleasant to work on.

00:03:33 ◼   ► So kudos to the Lame project.

00:03:35 ◼   ► - Is this all C or C++?

00:03:37 ◼   ► - It's all C, which is also nice,

00:03:38 ◼   ► 'cause I don't really know a lot of C++,

00:03:41 ◼   ► so to have something to be pure C

00:03:42 ◼   ► is also kind of a welcome change.

00:03:45 ◼   ► - Yeah, it's like back a few jobs ago.

00:03:48 ◼   ► Back when you actually had a job.

00:03:49 ◼   ► - Yeah, exactly.

00:03:50 ◼   ► I still like C a lot.

00:03:52 ◼   ► I mean, I recognize that working in C nowadays,

00:03:56 ◼   ► it's like having a classic car

00:03:59 ◼   ► that you work on in your garage.

00:04:01 ◼   ► You know, it's like, it's really fulfilling,

00:04:04 ◼   ► and it's really nice and cool

00:04:06 ◼   ► if that's your personality type,

00:04:08 ◼   ► But I really would be hard-pressed to justify using it

00:04:12 ◼   ► for like, quote, real work anymore.

00:04:15 ◼   ► But I really do enjoy working with it

00:04:17 ◼   ► on the occasion that I need to work on it.

00:04:22 ◼   ► - Fair enough.

00:04:22 ◼   ► I mean, anything's better than Perl or PHP, am I right?

00:04:25 ◼   ► So anyway, we should do some follow-up.

00:04:27 ◼   ► The Sierra Mac Pro support.

00:04:29 ◼   ► This is not good times for John Syracuse.

00:04:31 ◼   ► (laughing)

00:04:32 ◼   ► - It's not so bad times either.

00:04:34 ◼   ► - Marco's theory was that it was

00:04:36 ◼   ► because of the instructions.

00:04:38 ◼   ► Yeah, my theory was the hardware encryption instructions

00:04:41 ◼   ► that Intel has added over time, but turns out, nope.

00:04:44 ◼   ► - Well, I mean, we didn't get any official word on it,

00:04:46 ◼   ► but talking to a whole bunch of people,

00:04:47 ◼   ► hey, why do you think they dropped support

00:04:49 ◼   ► for this computer or that computer?

00:04:51 ◼   ► The general consensus was that they just dropped support

00:04:54 ◼   ► for computers that are old because it's a pain in the butt

00:04:56 ◼   ► to keep supporting them.

00:04:57 ◼   ► And there's lots of things that are a pain

00:04:58 ◼   ► to keep supporting.

00:04:59 ◼   ► If you do anything to the OS that changes

00:05:02 ◼   ► sort of the driver model or that requires drivers

00:05:05 ◼   ► be even just recompiled or changed in some subtle way.

00:05:09 ◼   ► Supporting old hardware like say the airport card that could be put into my 2008 Mac Pro,

00:05:15 ◼   ► that's kind of a pain and you don't want to bother with that.

00:05:17 ◼   ► But the big one I think is video drivers because there is usually some participation from the

00:05:22 ◼   ► video card vendor, whether it be AMD, ATI, or NVIDIA in the creation of the drivers for

00:05:29 ◼   ► OS X for these various GPUs.

00:05:32 ◼   ► Apple doesn't want to keep doing that and usually if you were to go to AMD and say hey

00:05:36 ◼   ► We're making a new version of the operating system

00:05:38 ◼   ► Can you help us or either help us to or do it entirely on your own drivers for?

00:05:44 ◼   ► The work with the Sierra kernel and this ancient 2008 Mac Pro. I don't think they're keen to do that

00:05:51 ◼   ► But anyway, lots of reasons why you might want to support old hardware

00:05:55 ◼   ► And yeah, no no specific smoking gun for any particular hardware feature

00:06:02 ◼   ► that is not supported and adding

00:06:04 ◼   ► into this pile of

00:06:06 ◼   ► Support for the idea that it's just because it's old is that 2009 Mac pros are also not supported contrary to what I had hoped

00:06:12 ◼   ► On the last show so that means my work Mac can't run Sierra either

00:06:16 ◼   ► What are you gonna? Do I'm gonna wait? I'm gonna wait until the new it's almost

00:06:21 ◼   ► It's almost this you have to buy a new computer. Oh, yeah, everyone gets laptops at work now

00:06:25 ◼   ► so I'm gonna wait until the new MacBook pros come out and

00:06:27 ◼   ► And then I'm going to wait another month, and then I'm going to get a new computer at work.

00:06:32 ◼   ► Now you do realize that the new MacBook Pros will almost certainly have the arrow key layout that you hate.

00:06:39 ◼   ► I'm not gonna use that keyboard.

00:06:41 ◼   ► It's just gonna be sitting off to the side of the screen that I never look at, connected to an external screen.

00:06:48 ◼   ► Like I'm just using it as a weird shaped Mac Mini.

00:06:51 ◼   ► That's my plan, because I only have one screen at work, and it's not a great screen, but whatever, it's fine.

00:06:56 ◼   ► So I'll keep doing that. I want to get the 15 inch so I can put it in mirrored display mode

00:07:01 ◼   ► That's how much I'm not going to use a second screen

00:07:03 ◼   ► Just so my windows and crap won't move when I disconnect the monitor. It'll be the same rez on both

00:07:08 ◼   ► You know what I mean? That's my current plan.

00:07:10 ◼   ► I love that your current plan like clearly didn't even consider another Mac Pro

00:07:16 ◼   ► Well, I can't work would never buy that for me

00:07:19 ◼   ► Like I my Mac was the first Mac at the entire company and I had to get special

00:07:24 ◼   ► Dispensation to get it and it was like way more expensive than the $400 piece of crap Dell laptops

00:07:28 ◼   ► They were getting everybody back then mine was like the cheapest possible Mac Pro you could get in the minimum configuration

00:07:35 ◼   ► It was like $1,500 and it took so much for them to choke that down

00:07:38 ◼   ► Nowadays, they're better about buying Macs if you want one and they'll buy, you know

00:07:43 ◼   ► 13 inch or 15 inch MacBook Pro and that's what I plan on getting but no Mac Pro is not an option

00:07:50 ◼   ► Neither is an iMac for that matter.

00:07:53 ◼   ► Would you get an iMac over a MacBook Pro if you had the choice?

00:07:56 ◼   ► Yes I would.

00:07:57 ◼   ► Is the base model 5K iMac not price competitive with the 15-inch MacBook Pro?

00:08:02 ◼   ► I don't think it's price.

00:08:03 ◼   ► I think it's…

00:08:04 ◼   ► You don't understand big corporations.

00:08:06 ◼   ► They just want to do things the way they want to do them and they want regularity and uniformity

00:08:11 ◼   ► and they don't want you to be a special snowflake who wants a specific computer.

00:08:14 ◼   ► They only have and support and know and understand two kinds of computers on the Mac side and

00:08:20 ◼   ► It's whatever.

00:08:21 ◼   ► I don't know.

00:08:22 ◼   ► I mean maybe I'll ask about it but…

00:08:23 ◼   ► No, this is exactly my experience.

00:08:25 ◼   ► My company gets leased Macs and I think they're two-year leases and basically your choices

00:08:32 ◼   ► are MacBook Pro if you do some sort of development work, I believe a MacBook Air if you don't,

00:08:37 ◼   ► or some sort of god-awful Dell.

00:08:39 ◼   ► And those are…

00:08:40 ◼   ► Actually it might be an HP, some sort of god-awful PC, but it's one of those three.

00:08:44 ◼   ► And if you want anything else under the sun, well, tough noogies because the help desk

00:08:48 ◼   ► does not want to support it.

00:08:49 ◼   ► So I'm right there with you, John.

00:08:51 ◼   ► Now to be fair, my MacBook Pro is very nice.

00:08:53 ◼   ► I have no complaints.

00:08:53 ◼   ► I'm just saying that that is the way

00:08:55 ◼   ► corporate America works.

00:08:56 ◼   ► You must conform, you must be one of the people

00:08:59 ◼   ► in that 1984 commercial.

00:09:01 ◼   ► You must be a faceless number in the crowd

00:09:05 ◼   ► that is using one of the blessed computers.

00:09:08 ◼   ► - I miss the days when I was unsupported too

00:09:09 ◼   ► because when I was the first Mac in the company,

00:09:11 ◼   ► they're like, all right,

00:09:12 ◼   ► but you're gonna be totally unsupported?

00:09:13 ◼   ► I'm like, yes, that's exactly what I want.

00:09:14 ◼   ► I want to be totally unsupported.

00:09:16 ◼   ► Don't touch my computer ever.

00:09:18 ◼   ► do nothing to it. Unfortunately now that Macs are officially supported, I am beset on all

00:09:22 ◼   ► sides by terrible things like add your Mac to the Active Directory network. No! Oh no!

00:09:28 ◼   ► Run Symantec Antivirus on your Mac. Oh, that's real important. Run these other applications

00:09:33 ◼   ► in the background that will grind your CPU to death as it does who knows what. Oh, it's

00:09:37 ◼   ► really terrible. Wait, I have one question. Do you have to install Adobe Acrobat? I don't

00:09:43 ◼   ► have to, but it could be installed on my behalf silently in the background any day I come

00:09:47 ◼   ► to work who knows what's happened to my computer because it's not under my control they just

00:09:51 ◼   ► force install software on it whenever they want none of that software makes my experience

00:09:54 ◼   ► of using the computer better but supposedly now I'm protected from viruses yeah whatever

00:10:01 ◼   ► I can't believe you stand for anyone else having any sort of access to your computer

00:10:05 ◼   ► like I understand that you have no choice I totally get that but you of all people allowing

00:10:10 ◼   ► that vulnerability if you will I am stunned that you that you stand for that it's rough

00:10:15 ◼   ► but what can you do? Like I said, you have no choice.

00:10:17 ◼   ► The final bit is, like, how sad is this really? It's not really that sad. I installed Sierra

00:10:22 ◼   ► on my 2008 Mac Pro and it runs fine. Because I have an aftermarket GPU, the installer's

00:10:26 ◼   ► a little bit cranky about it, but if you just, like, I'm installing an external disk at this

00:10:30 ◼   ► point because I'm not going to install it on my main system, it's the first developer

00:10:33 ◼   ► release. So I installed it on an external hard drive using my 2011 MacBook Air, and

00:10:38 ◼   ► then I just deleted a single file from the thing, System Library Core Services PlatformSupport.plist,

00:10:42 ◼   ► I renamed it or whatever.

00:10:45 ◼   ► And then my Mac Pro 2008 boots from it just fine.

00:10:50 ◼   ► And it runs fine, as far as I can tell.

00:10:51 ◼   ► So I don't think it'll be that big a deal.

00:10:53 ◼   ► I may end up doing the same thing with my Mac at work, although I'm sure corporate IT

00:10:57 ◼   ► will love that.

00:10:59 ◼   ► But, you know, I could try that right before I give up and say, "Okay, fine, get me the

00:11:04 ◼   ► new laptops or whatever."

00:11:06 ◼   ► But anyway, if you have a 2008 Mac Pro or another unsupported computer, Sierra might

00:11:11 ◼   ► work for you if the set of hardware you have is not the stock set of hardware that came

00:11:15 ◼   ► with that computer.

00:11:16 ◼   ► So now hold on a second.

00:11:18 ◼   ► So you generally do some amount of work at home on a semi-regular basis, is that correct?

00:11:25 ◼   ► So you're doing that on your personal machine because you have a Mac Pro at work and a Mac

00:11:29 ◼   ► Pro at home, right?

00:11:31 ◼   ► So that's cool with you?

00:11:33 ◼   ► Like it's just an external hard drive.

00:11:34 ◼   ► It's just I have like seven hard drives connected to my computer, like four internal and three

00:11:39 ◼   ► external and half of them are turned off most of the time.

00:11:41 ◼   ► the other ones are unmounted most of the time. It's really easy to just put a beta OS on

00:11:44 ◼   ► an external drive and then reboot and boot off of that drive and it's fine. I immediately

00:11:48 ◼   ► unmount all my other drives when I launch and it's like it's not a concern.

00:11:52 ◼   ► Right, no, no, I'm not on Sierra at all. On a general level, you have infected your home

00:11:58 ◼   ► computer with work-related things, is my point.

00:12:01 ◼   ► Well, no, because I just remote desktop into my Mac at work to do stuff.

00:12:05 ◼   ► Okay, so you have infected it with some sort of VPN connection.

00:12:07 ◼   ► - I have in fact, the VPN software is installed

00:12:10 ◼   ► for me to get out of the VPN and off to remote desktop.

00:12:13 ◼   ► - I gotcha, okay, so any of the development

00:12:15 ◼   ► isn't really strictly speaking happening locally.

00:12:18 ◼   ► - And really it's not even happening on my Mac either.

00:12:20 ◼   ► I mean, then I'm just SSHing into, you know, like it's,

00:12:23 ◼   ► unlike you, I'm not actually doing development

00:12:26 ◼   ► on my Mac really, I mean, except for like the web browsers

00:12:29 ◼   ► that are on it, all the actual code

00:12:30 ◼   ► and everything else is on servers, you know.

00:12:33 ◼   ► - Gotcha, all right, that's actually fairly fascinating.

00:12:36 ◼   ► I did not know that.

00:12:37 ◼   ► Okay, anything else about the Mac Pro support?

00:12:40 ◼   ► - It will never end.

00:12:41 ◼   ► - Anything else today?

00:12:43 ◼   ► All right, the talk show live.

00:12:45 ◼   ► This was recorded Tuesday,

00:12:48 ◼   ► I'm sorry, Tuesday before this passed.

00:12:51 ◼   ► God knows, by the time this thing is released,

00:12:52 ◼   ► it was probably two months ago.

00:12:54 ◼   ► But anyway, the Tuesday that WWDC was going on,

00:12:59 ◼   ► there was the talk show again this year.

00:13:01 ◼   ► Was it two years ago that we were guests, I think?

00:13:04 ◼   ► And then the only the only reasonable conclusion after having this this impossibly awesome and attractive trio on stage is

00:13:11 ◼   ► to have

00:13:14 ◼   ► Phil Schiller who followed up last year and then this year. What are you gonna do? Well, why not have Phil Schiller and

00:13:21 ◼   ► Craig Federighi and we were all there

00:13:24 ◼   ► We were sitting in the front row and it was amazing and I cannot say enough good things about this

00:13:31 ◼   ► This talk show if you are not a regular talk show listener

00:13:34 ◼   ► If you're one of the ten people that do not listen to the talk show, but do listen to this show I cannot

00:13:38 ◼   ► Encourage you enough to listen to this episode of the talk show. We'll have a link in the show notes. It is excellent

00:13:44 ◼   ► There's also a video

00:13:45 ◼   ► Which was very good from what I've gathered. I haven't actually watched it myself, but

00:13:49 ◼   ► seeing Craig and Phil

00:13:52 ◼   ► Talk to each other and talk to the audience and talk to John Gruber all of all the above

00:13:59 ◼   ► Was amazing and there's a few line items that we'd like to go through about that

00:14:03 ◼   ► but one of the things that made me kind of laugh and and

00:14:06 ◼   ► Really that I loved about it and made me love Craig Federighi even more

00:14:11 ◼   ► Was as the talk show was going on

00:14:14 ◼   ► Craig on the surface looked comfortable

00:14:18 ◼   ► But if you really looked at him for more than a second you realized he is not in love with what he's doing right now

00:14:23 ◼   ► Not in like not that John was doing anything wrong. Not that that Phil was doing anything wrong

00:14:27 ◼   ► But you could just tell that Craig was like a little nervous and not entirely comfortable with what was going on

00:14:33 ◼   ► And he was leaning forward a lot. He he kept it. I did the same thing when we were on the talk show

00:14:38 ◼   ► Dub-dub he kept leaning forward and then like putting his his elbow on his knee and then propping his head up

00:14:45 ◼   ► And then I think you could he realized that that kind of looks a little weird

00:14:49 ◼   ► For those in the audience then he would try to like casually put his arm down which I'm sure if we looked at the video

00:14:53 ◼   ► of me on stage. I probably did this 35 times, but there was a lot of that. But the thing

00:14:59 ◼   ► that struck me the most was when Jon Gruber talked about something developer related,

00:15:09 ◼   ► not only did Craig have impossibly good answers for all the questions, sometimes more forthcoming

00:15:15 ◼   ► than maybe he should have, but he sat bolt upright every time. And he was so confident

00:15:22 ◼   ► and so sure of himself, and it was so awesome to see,

00:15:26 ◼   ► and God, I love that guy so much.

00:15:28 ◼   ► - Yeah, it was such a good show.

00:15:30 ◼   ► I mean-- - It really was.

00:15:32 ◼   ► - You know, you can think about like, you know,

00:15:34 ◼   ► what kind of show would you expect to have

00:15:37 ◼   ► where Apple executives give an interview, you know?

00:15:40 ◼   ► And if you think about it just like that,

00:15:42 ◼   ► you might think, well, they're just gonna be, you know,

00:15:44 ◼   ► going over PR talking points and everything.

00:15:46 ◼   ► And certainly, some of what they say is PR talking points,

00:15:50 ◼   ► but most of it isn't.

00:15:52 ◼   ► And most of it is, like,

00:15:54 ◼   ► I was on the talk show this week,

00:15:56 ◼   ► I don't think it's out yet,

00:15:57 ◼   ► but I basically reviewed the show for Jon

00:16:00 ◼   ► on his own chokes, I knew he wouldn't do it himself.

00:16:03 ◼   ► So forgive the repetition here,

00:16:06 ◼   ► but I think one of the best things I liked about it,

00:16:09 ◼   ► first of all, is that we got to see two Apple executives

00:16:13 ◼   ► interacting with each other, and you never see that.

00:16:16 ◼   ► Like, even people in Apple hardly ever see that.

00:16:19 ◼   ► And so to see that in this context, these two people who clearly have known each other

00:16:25 ◼   ► for a very long time and have worked together for a very long time, two people on top of

00:16:29 ◼   ► their game, Phil and Craig, are really excellent at their jobs. And they both have incredibly

00:16:37 ◼   ► deep knowledge about the stuff that we care about and the stuff that Gruber would be asking

00:16:41 ◼   ► questions about at WWDC. So like, I was thinking like, you know, we were kind of speculating

00:16:46 ◼   ► sitting beforehand, oh, who's you gonna have this year?

00:16:48 ◼   ► Who's you gonna have this year?

00:16:49 ◼   ► And one of the ideas is what if he has Tim Cook?

00:16:53 ◼   ► And having Tim Cook would be like,

00:16:55 ◼   ► it'd be really like a great badge of honor.

00:16:57 ◼   ► It would be like a really noteworthy thing.

00:17:00 ◼   ► But I think having Phil and Craig is actually a better set

00:17:03 ◼   ► for that audience for that time.

00:17:06 ◼   ► Because Phil is really, in many ways,

00:17:10 ◼   ► it seems like he is kind of like the head

00:17:12 ◼   ► of product direction on some level,

00:17:15 ◼   ► And it seems to be kind of a shared role

00:17:17 ◼   ► in Steve's absence.

00:17:19 ◼   ► Johnny obviously has something to do with it.

00:17:21 ◼   ► I think Tim probably has something to do with it as well,

00:17:24 ◼   ► and who knows who else,

00:17:24 ◼   ► but it seems like a lot of that falls on Phil.

00:17:27 ◼   ► So just product decisions in general also.

00:17:30 ◼   ► Phil is the head of the App Store now.

00:17:32 ◼   ► So again, very relevant to this audience.

00:17:34 ◼   ► And then you have Federighi, who is,

00:17:39 ◼   ► yes, we knew he was the executive in charge

00:17:41 ◼   ► of software engineering stuff,

00:17:44 ◼   ► But I don't think a lot of us knew quite how much of an engineer he is.

00:17:48 ◼   ► And also, not only does he have deep technical chops, but he also has deep knowledge of specific

00:17:55 ◼   ► implementation details of the stuff they're doing now.

00:17:58 ◼   ► So it's not like he ascended into this ivory tower and is just dictating things down to

00:18:03 ◼   ► his minions to have them do all the work and he's just being a figurehead.

00:18:06 ◼   ► He's clearly still very much involved in those decisions and very, very, very technical down

00:18:13 ◼   ► to deep levels and able to explain that very well.

00:18:18 ◼   ► So to have that combo of like the product guy

00:18:21 ◼   ► and the app store guy in Phil, and then to have Craig

00:18:25 ◼   ► as like the technical head is really, I think,

00:18:29 ◼   ► the perfect combo to have at WWDC.

00:18:31 ◼   ► Plus, I think their personalities work really well,

00:18:34 ◼   ► both in general and also with John Gruber

00:18:36 ◼   ► being the interviewer.

00:18:38 ◼   ► So overall, I would much rather have them than have Tim

00:18:42 ◼   ► in that context. And I think it was as good as it could possibly have gone. I think it was great.

00:18:47 ◼   ► I mostly agree with that Tim would have been more boring, but I've been thinking about it more since

00:18:51 ◼   ► WWC week. Like it really does depend on the topic that you're interested in. You know,

00:18:55 ◼   ► if I'd mentioned Johnny Ive, obviously if you're going to have a bunch of questions about design

00:19:00 ◼   ► stuff, he's the guy you want and Phil is not going to be able to give you much there and neither is

00:19:05 ◼   ► Craig for that matter because like, you know, product design, whether it be hardware or software

00:19:09 ◼   ► is Johnny's domain and a lot of the decisions that are made there, you know, I would imagine that

00:19:13 ◼   ► they could add some insight into them, but you want the man himself if you want to get that

00:19:16 ◼   ► information. So there's Johnny set aside. And then Tim, after Apple comes out with the car,

00:19:21 ◼   ► the guy you want to talk to is Tim Cook, until you know like the deep car people, because your main

00:19:25 ◼   ► question is why has Apple chosen to make a car? And the person who can answer that best is Tim

00:19:29 ◼   ► Cook, right? Because that's a decision at his level. So I would actually like to see Tim Cook

00:19:34 ◼   ► at some point, but for the topics I was interested in this year and most years, yeah, I'd rather

00:19:39 ◼   ► I'd rather hear from Craig and to a lesser extent Phil because I'm much more interested

00:19:43 ◼   ► in the technical and less interested in the App Store stuff, but at WWC, like you said,

00:19:47 ◼   ► a lot of people are interested in App Store.

00:19:49 ◼   ► Exactly.

00:19:50 ◼   ► The reason the talk show live is a topic in the follow-up here is because I was adding

00:19:54 ◼   ► items to the follow-up of bits and pieces and I realized most of them were information

00:19:58 ◼   ► from the talk show live.

00:20:00 ◼   ► The main reason I would suggest people listen to it is that it has information that you

00:20:04 ◼   ► probably won't see elsewhere because you know again Phil and Craig were on there

00:20:08 ◼   ► and they didn't just say things that were already said at WWDC they didn't

00:20:13 ◼   ► just say things that have already been said by Apple PR they provided new

00:20:16 ◼   ► information and some of it I thought was interesting and you won't know it unless

00:20:20 ◼   ► you listen to that or listen to these two tidbits that I pulled out of it

00:20:24 ◼   ► the first one is about an iOS 10 how you can get rid of the Apple apps like if

00:20:29 ◼   ► you don't want the stock apps or the the tips app or whatever you don't have to

00:20:33 ◼   ► hide them in a folder anymore, you can now delete them, but they aren't actually deleted.

00:20:37 ◼   ► As many people speculated, and as has been speculated for a long time before this feature

00:20:41 ◼   ► even appeared, they're not actually removed from the system.

00:20:44 ◼   ► Even though when you delete them, you can go to the store and then redownload them,

00:20:47 ◼   ► but as I think it was Craig said on stage, that download will run suspiciously fast because

00:20:52 ◼   ► they haven't actually been removed from your computer.

00:20:54 ◼   ► All the App Store is saying, "Oh yeah, I'll redownload those for you," and then it just

00:20:57 ◼   ► reveals them again.

00:20:58 ◼   ► And a bunch of reasons were given for actually leaving them on the system.

00:21:01 ◼   ► delete I think the documents and data associated with it but you won't delete the actual executable

00:21:05 ◼   ► so that was interesting tidbit and the other one is relevant to a couple of stories and tweets that

00:21:11 ◼   ► have been going around about the the current beta of the app store ad service where they'll you know

00:21:15 ◼   ► people can try it out for free now and all you do at this point is say yes I want to opt my

00:21:21 ◼   ► app into the ad service I don't know can you do the keywords and everything as well but anyway

00:21:25 ◼   ► I looked around and couldn't find anything else to do so I opted my my app in but I think that

00:21:30 ◼   ► that just means that Apple will like kind of shove it on top of search results sometimes,

00:21:34 ◼   ► but I don't think I have any controls over that.

00:21:37 ◼   ► Yeah, it's like the automated one where they said, "If you don't want to pick your keywords

00:21:40 ◼   ► and doing that stuff, just we'll figure it out for you." And so one of the little controversies

00:21:44 ◼   ► that spun up on Twitter was, I think it was someone from Tweetbot was saying they searched

00:21:49 ◼   ► for Tweetbot, like the actual name of their application, and the number one hit to the

00:21:53 ◼   ► ad at the top of the search results was an ad for Twitterific. And it looks like, "Oh,

00:21:59 ◼   ► has purchased the Tweetbot keyword, but they haven't because you can't even do that at

00:22:03 ◼   ► that point, and they didn't.

00:22:05 ◼   ► And that brings up the whole topic that we discussed before about buying someone else's

00:22:09 ◼   ► trademark as a keyword.

00:22:11 ◼   ► So if you have an application, if you were, you know, the maker of Twitterific and you

00:22:15 ◼   ► wanted to buy the word, the keyword "Tweetbot," could you do that?

00:22:19 ◼   ► That was addressed by Phil Schiller and the Talk Show Live.

00:22:23 ◼   ► And he said that they are explicitly allowing this.

00:22:26 ◼   ► I think this is a direct quote from the transcript.

00:22:28 ◼   ► It says, "You can use someone else's brand in your AdWords

00:22:31 ◼   ► that you want.

00:22:32 ◼   ► As we thought about it, that is more

00:22:34 ◼   ► likely to benefit the small developer

00:22:35 ◼   ► than the big developer."

00:22:37 ◼   ► And the idea is that if you have a well-known brand,

00:22:40 ◼   ► if you like Clash of Clans or something,

00:22:42 ◼   ► you're not going to buy a keyword

00:22:44 ◼   ► for some tiny little application that doesn't--

00:22:46 ◼   ► tiny little similar game that doesn't sell a lot of copies.

00:22:48 ◼   ► But if you make a game that's similar to Clash of Clans

00:22:51 ◼   ► and you want to break into the market, and really, right now,

00:22:53 ◼   ► you don't have any customers, maybe you

00:22:55 ◼   ► the Clash of Clans keyword. That is the theory behind this, and Phil articulates it in the

00:23:00 ◼   ► most straightforward manner as you're going to get. And it was surprising to a lot of

00:23:04 ◼   ► people, and we'll see how it plays out in real life, but from the perspective of app

00:23:09 ◼   ► developers, especially app developers who sort of are on even footing, it seems like

00:23:12 ◼   ► a pretty scummy thing to do to buy your competitor's trademark. So I think a lot of people are not

00:23:18 ◼   ► going to do it because it seems rude, and I would imagine the companies that would do

00:23:22 ◼   ► it are exactly the companies that are big because they have no scruples and aren't run

00:23:26 ◼   ► by an individual who is answerable to their practices and they have a marketing department

00:23:29 ◼   ► who's measured on how well they boost sales.

00:23:32 ◼   ► And so I think this is going to happen a lot.

00:23:34 ◼   ► I'm not entirely convinced that it's going to benefit the small developer more than the

00:23:37 ◼   ► big developer.

00:23:38 ◼   ► But we'll see.

00:23:40 ◼   ► And again, Phil talked directly about this issue when questioned about it on TalkToYourLive,

00:23:43 ◼   ► so check it out.

00:23:44 ◼   ► I think there's a number of issues and factors that go into this.

00:23:49 ◼   ► What exactly is a small developer?

00:23:51 ◼   ► I think who this will actually hurt most are people like Twitterific and Tweetbot and maybe

00:23:58 ◼   ► Overcast where kind of like medium-sized developers were like, "We're not big enough for like

00:24:03 ◼   ► the Clash of Clans people to bid on our keywords because who cares? Like, you know, we are

00:24:08 ◼   ► nothing compared to what they get and we're also not related." But other people who make

00:24:13 ◼   ► apps that are related to podcasting or related to Twitter would very easily like bid on like

00:24:20 ◼   ► the medium sized apps in their category on those keywords.

00:24:23 ◼   ► So I view myself as a small developer

00:24:26 ◼   ► in the grand scheme of things.

00:24:28 ◼   ► But a lot of people view Overcast as a big app

00:24:30 ◼   ► in the category relative to their app.

00:24:33 ◼   ► So I think when Phil says he thinks this will benefit

00:24:37 ◼   ► small developers, I don't think he's talking about people

00:24:40 ◼   ► like Overcast and Tweetbot and Twitturrific.

00:24:42 ◼   ► I think he's talking about little things

00:24:44 ◼   ► that most of us have never heard of

00:24:46 ◼   ► that happen to be in these same categories.

00:24:48 ◼   ► he's right that it would probably benefit them.

00:24:51 ◼   ► Like, you know, the kind of thing where like,

00:24:52 ◼   ► if you search for like, you know, Twitter or whatever,

00:24:55 ◼   ► you know, you get a couple of Twitter apps

00:24:57 ◼   ► and then you get a whole bunch of what looks like

00:24:58 ◼   ► just garbage and spam and it's like, you know,

00:25:00 ◼   ► photo rotation apps for Twitter or stuff like that,

00:25:03 ◼   ► you know, that kind of stuff.

00:25:04 ◼   ► It's gonna be that kind of app, I think,

00:25:08 ◼   ► that could benefit from this, not medium-sized apps

00:25:11 ◼   ► like Twitter Effect, Tweetbot, Overcast.

00:25:14 ◼   ► The other problem here is,

00:25:16 ◼   ► Well, and I think there's two other problems here.

00:25:18 ◼   ► One is that we've had kind of like download

00:25:23 ◼   ► keyword spam issues in iTunes Connect

00:25:25 ◼   ► since the beginning of the App Store.

00:25:27 ◼   ► The apps have a keywords field

00:25:29 ◼   ► that they can enter keywords into for search,

00:25:31 ◼   ► and those have historically been used

00:25:34 ◼   ► pretty strongly for relevance in search,

00:25:36 ◼   ► more so than the description that is publicly shown.

00:25:39 ◼   ► The problem is these keywords

00:25:41 ◼   ► are not publicly visible anywhere,

00:25:43 ◼   ► and so there's kind of no downside

00:25:45 ◼   ► for an app to put its competitors or other large apps' names

00:25:50 ◼   ► in those keywords.

00:25:51 ◼   ► And Apple has a rule against this,

00:25:53 ◼   ► but in practice that rule is only really enforced

00:25:55 ◼   ► and enforceable against like large trademark, like Disney.

00:26:00 ◼   ► So it'd be hard to put Disney in your keywords

00:26:02 ◼   ► and get accepted through App Review.

00:26:04 ◼   ► But if another Twitter app puts Tweetbot in its keywords,

00:26:09 ◼   ► that is way more likely to get in and to get through.

00:26:12 ◼   ► and historically that has been seemingly very badly enforced

00:26:16 ◼   ► by App Review.

00:26:17 ◼   ► And Phil even kind of said something along the lines of like

00:26:20 ◼   ► we're gonna be trying to step that up.

00:26:23 ◼   ► Something to that degree.

00:26:25 ◼   ► And then thirdly the problem is the name of,

00:26:29 ◼   ► if you put down overcast as your keyword

00:26:31 ◼   ► and you might search for the word overcast to test this out

00:26:35 ◼   ► and then my app shows up below some ad

00:26:37 ◼   ► and you might think this is horrible.

00:26:39 ◼   ► My app is not called Overcast in the app store.

00:26:43 ◼   ► My app is called Overcast colon podcast player.

00:26:46 ◼   ► And if you look at almost every other instance

00:26:48 ◼   ► of this kind of like ad doing what seems

00:26:50 ◼   ► like the wrong thing, like Tweetbot is not just named

00:26:54 ◼   ► Tweetbot, Twitturific is not just named Twitturific.

00:26:57 ◼   ► So it's called like, you know, Tweetbot for Twitter,

00:26:59 ◼   ► Twitturific Twitter client, stuff like that.

00:27:01 ◼   ► We've, you know, in the last couple years,

00:27:04 ◼   ► we've all kind of succumbed to having to add keywords

00:27:08 ◼   ► to our app names because quite simply it works better and the search has been horrible and

00:27:13 ◼   ► it's been one of the only ways.

00:27:14 ◼   ► You gave up on the search improving. You're like, "You know what? Forget it. We will

00:27:19 ◼   ► mangle the name of our application because there's no other way to come up in the search

00:27:23 ◼   ► results." I mean, I remember when Twitteriffic did it years ago. It was a sad day because

00:27:26 ◼   ► it just really uglies up the name, but the problem was people would search for Twitter

00:27:29 ◼   ► and just Twitteriffic would be 700 results down. The relevance was so bad even though

00:27:35 ◼   ► it was a popular third-party Twitter client and when someone searches for Twitter, it

00:27:39 ◼   ► would be a good idea to show them the, you know, well-reviewed, popular, frequently updated

00:27:44 ◼   ► Twitter clients and instead it's like, no, Twitter doesn't appear anywhere in your name.

00:27:48 ◼   ► Or it does, but it's just a prefix and like the fact that Twitterrific didn't show up

00:27:52 ◼   ► in a search result for Twitter until way low down, that's just sad.

00:27:56 ◼   ► So I would hope that as part of this effort to like police the keywords for trademarks

00:28:00 ◼   ► more often and stuff, they will also improve the search to the point where you don't have

00:28:03 ◼   ► have to name mangle essentially anymore, but we'll see.

00:28:06 ◼   ► - Well, the other thing is like, I try with Overcast,

00:28:08 ◼   ► I tried not name mangling, but you know,

00:28:11 ◼   ► I launched Overcast fairly late in the App Store

00:28:13 ◼   ► relative to the other apps.

00:28:14 ◼   ► There was already an app named exactly the string Overcast.

00:28:18 ◼   ► In fact, I even contacted the developer

00:28:19 ◼   ► and tried to buy it from him, and he agreed,

00:28:21 ◼   ► and we had agreed to actually transfer this app to me

00:28:24 ◼   ► just so I could like delete it and reuse the name,

00:28:27 ◼   ► but the app used iCloud, and so we couldn't transfer it.

00:28:30 ◼   ► Why didn't you just pay him to rename it?

00:28:33 ◼   ► - I think somehow we just kind of dropped the ball

00:28:36 ◼   ► on continuing the conversation,

00:28:37 ◼   ► so it probably just fell apart just through apathy,

00:28:41 ◼   ► but there is an app called Overcast.

00:28:43 ◼   ► I believe the seller's name is Willis Ingersoll,

00:28:46 ◼   ► something like that,

00:28:47 ◼   ► and it's like a file transfer app for cloud services.

00:28:49 ◼   ► And the App Store, even though the world of trademark

00:28:53 ◼   ► allows for the same name to be used

00:28:56 ◼   ► in different industries and different contexts,

00:28:59 ◼   ► The App Store doesn't, unless you do weird tricks

00:29:01 ◼   ► with international titles.

00:29:02 ◼   ► So it sometimes does, but usually doesn't.

00:29:04 ◼   ► It's a mess.

00:29:05 ◼   ► It's such a mess.

00:29:08 ◼   ► So this is like, you know, the combination of lots

00:29:11 ◼   ► of long-standing App Store problems results in this

00:29:16 ◼   ► mediocre situation we have with titles with keywords

00:29:19 ◼   ► after them, along with the new ads not matching things

00:29:23 ◼   ► exactly, probably, yeah, it's gonna be interesting.

00:29:27 ◼   ► Let's talk about Apple Photos and the John Saracusa controller search.

00:29:32 ◼   ► >> Yeah, that was my quality metric.

00:29:36 ◼   ► The original problem that I had was that I couldn't find a picture of my PlayStation

00:29:39 ◼   ► 4 controller in Apple's Photos application.

00:29:43 ◼   ► This is before iOS 10, before I debuted.

00:29:44 ◼   ► So I put all my photos into Google Photos, and amazingly, Google Photos was able to find

00:29:48 ◼   ► it just by me typing the word "controller."

00:29:51 ◼   ► of course apple announced similar photo search for iOS 10 and Mac OS Sierra and I speculated on last

00:30:00 ◼   ► show that it would not be able to find controller by me typing controller because apple is not good

00:30:06 ◼   ► at that type of thing and google is. So now having installed, oh actually I haven't installed it,

00:30:12 ◼   ► many other people have installed, I've installed Mac OS Sierra but I haven't installed iOS 10,

00:30:15 ◼   ► many other people have installed all these betas onto their iPhones because they're very brave

00:30:20 ◼   ► people and they have taken my exact pictures of the controller. They pulled them from Twitter

00:30:24 ◼   ► and put them into their photo libraries and done the search to see if we would find it.

00:30:28 ◼   ► And it doesn't find it if you type controller, but because Apple search, Apple's photo search

00:30:32 ◼   ► works a little bit differently or at least presents a different UI than Google photo

00:30:35 ◼   ► search, it does find it when you type control stick or I think also gamepad. And this brings

00:30:42 ◼   ► up the UI that Apple provides is as you type it shows like category results as if it puts

00:30:48 ◼   ► things into buckets and indeed it does. K. Yin has a blog post that lists out all of

00:30:54 ◼   ► the buckets that it has. It has something like 4,000 buckets to put things in. There's

00:30:59 ◼   ► a lot of stuff in there. Dance palaces, banana, chairlifts.

00:31:04 ◼   ► Wait, what's a dance palace? Yeah, seriously.

00:31:07 ◼   ► I don't know, I'm just reading some of the keywords here. Orchid, kiln, that's got kiln,

00:31:12 ◼   ► that's a good one. Polos, blocks, pulley, trikes, trumpets, tuba, bass, horn. I mean

00:31:17 ◼   ► Like it's got huge amounts of things.

00:31:19 ◼   ► So basically, if you type a word that is not one of the words on this list, it won't find it.

00:31:23 ◼   ► So as you type, it's showing you the words that begin with or contain these things.

00:31:27 ◼   ► So it's a little bit different UI, but it's trying to lead you to its notion of those things.

00:31:31 ◼   ► So if you type control, start typing controller and you see control stick, you probably tap that one.

00:31:36 ◼   ► So good job, Apple Photos, so far on finding the controller that I didn't think you would

00:31:43 ◼   ► ever find because that's good photo recognition because the controller was upside down.

00:31:47 ◼   ► That's not an easy search, and it's a pretty obscure category.

00:31:49 ◼   ► On the other hand, someone also said that Apple's photo search stinks because it couldn't

00:31:53 ◼   ► find birthday pics or babies or pictures of a couch.

00:31:57 ◼   ► So I don't know if their thing just hadn't indexed everything yet or those things aren't

00:32:00 ◼   ► among keywords or anything like that.

00:32:02 ◼   ► But we'll see as this rolls out.

00:32:04 ◼   ► But you can find all your dance palaces.

00:32:06 ◼   ► If you have pictures of dance palaces, it will find that before it will find your couch.

00:32:09 ◼   ► Not quite sure what would come up for dance palace.

00:32:11 ◼   ► But yeah, I have my entire photo library in Google Photos and also my entire photo library

00:32:16 ◼   ► in Apple's photos, so when these things release for real, I can do A/B tests right next to

00:32:20 ◼   ► each other with the exact same library and do a little bake-off, and when that happens,

00:32:24 ◼   ► I will definitely have some results. But so far, so good for Apple. It is more impressive

00:32:30 ◼   ► than I thought it would be.

00:32:31 ◼   ► I'm looking up what a dance palace is.

00:32:34 ◼   ► Our first sponsor tonight is Fracture. Go to fractureme.com and check it out with code

00:32:39 ◼   ► ATP10 for 10% off. Fracture is a company that prints photos in vivid color directly onto

00:32:45 ◼   ► to panes of glass.

00:32:47 ◼   ► It looks fantastic.

00:32:48 ◼   ► The colors pop like you won't believe

00:32:50 ◼   ► and it comes in a nice solid foam backing

00:32:52 ◼   ► which is ready to mount right out of the package.

00:32:54 ◼   ► So here's how this works.

00:32:55 ◼   ► It looks just like a complete picture.

00:32:58 ◼   ► You don't have to frame it,

00:32:59 ◼   ► you don't have to do anything else to it.

00:33:00 ◼   ► It is a complete picture.

00:33:02 ◼   ► It is a rectangle or square of glass

00:33:03 ◼   ► with your photo printed right behind the front layer

00:33:06 ◼   ► of glass and then behind that is a little tiny bit

00:33:09 ◼   ► of foam board so that you can hang it easily

00:33:12 ◼   ► 'cause then things can hook into the foam board

00:33:14 ◼   ► without having to scratch the glass or the ink layer.

00:33:16 ◼   ► So it just looks fantastic.

00:33:18 ◼   ► It looks very modern and clean.

00:33:20 ◼   ► And I have these all over our house, our office.

00:33:23 ◼   ► They're everywhere.

00:33:24 ◼   ► People always compliment them.

00:33:26 ◼   ► They make great gifts.

00:33:27 ◼   ► They make great keepsakes.

00:33:28 ◼   ► You can get your photos and actually print them

00:33:30 ◼   ► and actually have a physical representation

00:33:32 ◼   ► so you can remember this photo for more than like

00:33:34 ◼   ► the two days that it sits on your Facebook feed or whatever.

00:33:36 ◼   ► Check it out.

00:33:37 ◼   ► The prices are very reasonable for Fractures.

00:33:39 ◼   ► They start at just $15 for their small square size,

00:33:42 ◼   ► which is great for Instagram prints.

00:33:44 ◼   ► And they also make non-square rectangles.

00:33:46 ◼   ► They are awesome.

00:33:47 ◼   ► Again, they make fantastic gifts.

00:33:49 ◼   ► We've given them as gifts many times.

00:33:51 ◼   ► These prints look fantastic from Fracture.

00:33:53 ◼   ► So check it out today at fractureme.com

00:33:56 ◼   ► and use code 10% off with the code ATP10.

00:34:00 ◼   ► I recommend them, they're great.

00:34:02 ◼   ► Check out Fracture, thanks a lot.

00:34:04 ◼   ► - Alright, so some slightly late-breaking news

00:34:10 ◼   ► before we recorded.

00:34:12 ◼   ► The Thunderbolt display, the thing that I've been lusting after since September 2011, when

00:34:17 ◼   ► it was brand new, which was the last time it was updated.

00:34:22 ◼   ► Anyway.

00:34:23 ◼   ► Even that was a minor update over the LED display that came out right before it.

00:34:26 ◼   ► Yeah, fair enough.

00:34:28 ◼   ► The Thunderbolt display has been discontinued.

00:34:30 ◼   ► Sad times.

00:34:31 ◼   ► So at this point, Apple does not sell an external display.

00:34:34 ◼   ► It did have a minor update where they started shipping the MagSafe 1 to 2 adapter in the

00:34:40 ◼   ► box.

00:34:41 ◼   ► - That's a hardware change, right?

00:34:42 ◼   ► - Something like that.

00:34:43 ◼   ► - So they never actually gave it the MagSafe 2 natively,

00:34:46 ◼   ► they just kind of shipped through this $8 adapter

00:34:48 ◼   ► in the box.

00:34:49 ◼   ► - Yeah.

00:34:50 ◼   ► - Excuse me, I believe that adapter's $10,

00:34:52 ◼   ► thank you very much.

00:34:53 ◼   ► - Sorry, you're right, you're right.

00:34:55 ◼   ► - So this is somewhat sad times,

00:34:57 ◼   ► if you fancied having a non-retina external display

00:35:01 ◼   ► that's very pretty to look at,

00:35:03 ◼   ► and presumably for non-retina being very, very pretty

00:35:06 ◼   ► to see what's on the screen as well.

00:35:08 ◼   ► However, does this make us think that maybe a retina display is coming soon?

00:35:13 ◼   ► Maybe with a GPU built in?

00:35:15 ◼   ► I don't know.

00:35:16 ◼   ► I would think so, but it seems odd to me that they would just continue without having something

00:35:20 ◼   ► to replace it.

00:35:22 ◼   ► Maybe they just don't sell enough of them.

00:35:23 ◼   ► Maybe they can't get that screen from the vendor anymore because it's so terrible, because

00:35:26 ◼   ► the pixels are the size of boulders.

00:35:28 ◼   ► I don't know.

00:35:30 ◼   ► Did you read Apple's official statement that I put in the notes?

00:35:32 ◼   ► Yeah, it was really short, wasn't it?

00:35:34 ◼   ► Where did it go?

00:35:35 ◼   ► We are discontinuing the Apple Thunderbolt display.

00:35:37 ◼   ► It will be available through apple.com,

00:35:39 ◼   ► Apple's retail stores, and Apple-authorized resellers

00:35:41 ◼   ► while supplies last.

00:35:42 ◼   ► There are a number of great third-party options

00:35:44 ◼   ► available for Mac users.

00:35:46 ◼   ► - It's kind of surprising that they even gave a statement

00:35:49 ◼   ► about it, like that they call any attention to it at all.

00:35:51 ◼   ► I mean, usually when Apple discontinues a product,

00:35:54 ◼   ► like one that's not very important,

00:35:56 ◼   ► the user's kind of quietly removed it from sale.

00:35:59 ◼   ► Like, I think it's unusual to--

00:36:00 ◼   ► - But there's usually a clear replacement when they do that.

00:36:02 ◼   ► - That's true. - Like, they remove the old one

00:36:03 ◼   ► and there's always the new one,

00:36:04 ◼   ► and this is a little bit weird,

00:36:05 ◼   ► where they're removing the old one without a new one to replace it, I still put the odds

00:36:10 ◼   ► at greater than 50% that there will be a replacement for this product.

00:36:13 ◼   ► Oh yeah.

00:36:14 ◼   ► Just because it's such a gimme, they have the screen, it's in the 5E iMac, we've been

00:36:17 ◼   ► staring at it for a long time, you put it in a case, you just ship it.

00:36:21 ◼   ► But maybe that's not ready yet, or maybe it's part of a hardware announcement, and for whatever

00:36:25 ◼   ► reason, I don't think they sell a lot of Thunderbolt displays.

00:36:30 ◼   ► It's a thousand dollar monitor that is not worth a thousand dollars at this point.

00:36:34 ◼   ► You could argue about it when it was first introduced, but it is a really nice monitor,

00:36:37 ◼   ► and back before there were no options for 27-inch retinas when it was introduced in

00:36:41 ◼   ► 2011, so it was good.

00:36:43 ◼   ► Whether it was worth $1,000, who knows?

00:36:45 ◼   ► But at this point, it's definitely not worth $1,000.

00:36:46 ◼   ► So they must not have been selling a lot of them, and as they note, you know, while supplies

00:36:51 ◼   ► last they need to get rid of all this inventory.

00:36:53 ◼   ► Maybe it takes this long to get rid of the inventory at the rate that they sell.

00:36:57 ◼   ► Like maybe they have a lot of Thunderbolt displays hanging around, and they did the

00:36:59 ◼   ► math and said, "If we want these flushed out of the inventory by the time we introduce

00:37:03 ◼   ► the 5K external display, we need to discontinue it now and just let them sell out the rest

00:37:08 ◼   ► of them. It would also help, by the way, Apple, if you lowered the price, because that's

00:37:10 ◼   ► a way that you can sell things faster. But I know you don't do that.

00:37:12 ◼   ► David Schmittle I think it's more likely that the rumors

00:37:16 ◼   ► were all really clear and the timing makes a lot of sense that the new Skylake-based

00:37:22 ◼   ► MacBook Pros were supposed to be out by now, and it seemed that they were probably delayed

00:37:27 ◼   ► because of various Intel issues. Have you heard, every time we mention Skylake and laptops,

00:37:33 ◼   ► we usually hear from people who tell us that like,

00:37:35 ◼   ► that Skylake chips that would be appropriate

00:37:39 ◼   ► for the 15 inch were kind of out in the PC world,

00:37:42 ◼   ► and they actually had tons of problems,

00:37:44 ◼   ► and like, they were, it seemed like Intel might have like,

00:37:46 ◼   ► partly maybe recalled them or something,

00:37:48 ◼   ► there were some problems with them

00:37:50 ◼   ► that the manufacturers were having,

00:37:52 ◼   ► even after they'd shipped.

00:37:53 ◼   ► So it seems like there's some kind of unexpected

00:37:55 ◼   ► Intel delay that caused the MacBook Pros

00:37:57 ◼   ► to most likely be delayed.

00:37:59 ◼   ► So it seems like, if Apple's gonna release

00:38:01 ◼   ► new 5K display, it would need Skylake most likely

00:38:04 ◼   ► to drive it, so that they were probably planning

00:38:06 ◼   ► on releasing the 5K external display that succeeded

00:38:09 ◼   ► this Thunderbolt display at the same time

00:38:11 ◼   ► as new MacBook Pros that were all supposed

00:38:13 ◼   ► to already be out by now.

00:38:15 ◼   ► So chances are they probably gave the order months ago

00:38:20 ◼   ► to stop taking these component orders,

00:38:22 ◼   ► stop making these things, stop manufacturing these,

00:38:25 ◼   ► and then this delay happened sometime in the meantime.

00:38:29 ◼   ► This was probably just like the replacement product

00:38:31 ◼   ► was supposed to be out by now, and it might be done,

00:38:34 ◼   ► but there's no computer that can drive it yet.

00:38:36 ◼   ► So they're not gonna release a display

00:38:38 ◼   ► with no computer that can drive it,

00:38:40 ◼   ► so they kinda held the display for the MacBook Pro,

00:38:43 ◼   ► which got delayed.

00:38:45 ◼   ► So now they just have like,

00:38:46 ◼   ► "Well, we already told the manufacturing line

00:38:48 ◼   ► "to stop making these,

00:38:50 ◼   ► "and we're not gonna start it back up again,

00:38:51 ◼   ► "because that's crazy."

00:38:53 ◼   ► So I'm guessing we're just in this weird hole

00:38:56 ◼   ► in the middle of this like,

00:38:57 ◼   ► You know, these weird delays cause this operations hole

00:39:01 ◼   ► to happen and it'll make sense in a few months

00:39:04 ◼   ► or whenever the new MacBook Pros come out.

00:39:06 ◼   ► - And by the way, I think the reason they're gonna

00:39:07 ◼   ► make this product is everyone's like,

00:39:08 ◼   ► maybe they're just not gonna sell external displays.

00:39:10 ◼   ► The reason they'll make it is because the margins

00:39:12 ◼   ► have to be pretty darn good.

00:39:13 ◼   ► - Oh yeah. - They're probably gonna sell

00:39:14 ◼   ► it for the same kind of margin.

00:39:17 ◼   ► Thunderbolt display, like I said, it was a $1,000 monitor

00:39:19 ◼   ► in an age when you could get monitors with,

00:39:21 ◼   ► sometimes monitors with the exact same panel in them

00:39:24 ◼   ► for less money if you knew the right, you know,

00:39:26 ◼   ► right weirdly numbered Dell thing or whatever to buy. Like, good healthy margins on those.

00:39:31 ◼   ► And Apple sells mostly laptops for people who buy Macs. And people with laptops often

00:39:37 ◼   ► want a really big external display. Like, they don't want an external display that's

00:39:40 ◼   ► like 17 inches, but if people who want an external display, they want a really big one.

00:39:44 ◼   ► And so Apple should sell you a really big one because it's, you know, the same reason

00:39:47 ◼   ► they sell battery cases for crying out loud. If there's—if this is a product that people

00:39:50 ◼   ► want and we can sell it at high margin and it can look nice and Johnny Eye won't be upset,

00:39:56 ◼   ► by someone connecting a disgusting looking Dell monitor

00:39:58 ◼   ► to their beautiful MacBook Pro.

00:40:00 ◼   ► Like all signs point towards Apple continuing to sell

00:40:03 ◼   ► one really big, really expensive,

00:40:05 ◼   ► high margin external monitor.

00:40:06 ◼   ► And all signs point to me buying one

00:40:08 ◼   ► once I can hook it up to a computer.

00:40:10 ◼   ► - We are also sponsored this week by Hover.

00:40:15 ◼   ► We go to hover.com and use promo code APFS at checkout

00:40:19 ◼   ► for 10% off your, it's perfect timing,

00:40:21 ◼   ► for 10% off your first purchase.

00:40:23 ◼   ► Hover is a great place to buy domain names,

00:40:26 ◼   ► So when you have a great idea for your project or blog or startup or whatever, you need to

00:40:30 ◼   ► give it a great domain name.

00:40:31 ◼   ► And finding that name is very, very easy with Hover and of course they make it very easy

00:40:35 ◼   ► to then buy it and manage it.

00:40:37 ◼   ► And I use Hover for tons of my domains and it's great.

00:40:39 ◼   ► They have over 400 domain extensions to end your domain with.

00:40:42 ◼   ► You know, all the classics like .com, .net, some of the new, you know, new kind of narrow

00:40:46 ◼   ► ones like design and tech and some of the weird kind of jokey ones like .pizza and .ninja.

00:40:52 ◼   ► Now, new this week, they just launched .store,

00:40:56 ◼   ► and this is one of the rare occasions

00:40:58 ◼   ► where there's a name that is straightforward,

00:41:01 ◼   ► it's not goofy or amateurish like .ninja and .pizza.

00:41:04 ◼   ► You can build a real business

00:41:05 ◼   ► on something that ends in .store.

00:41:07 ◼   ► This is a very rare occasion where brand new real estate

00:41:11 ◼   ► has opened up that ends in a domain extension

00:41:13 ◼   ► that is not dumb, right?

00:41:15 ◼   ► So, and it's wide open 'cause it just opened up.

00:41:18 ◼   ► So go to hover.com, go get a .store name,

00:41:22 ◼   ► and you have a very, very good chance now,

00:41:23 ◼   ► because it's brand new, of getting a really good one.

00:41:26 ◼   ► So whether it's .store or any of the other 400

00:41:28 ◼   ► domain extensions out there, you can find

00:41:30 ◼   ► the perfect domain name for your idea at Hover.

00:41:32 ◼   ► Go to hover.com and use promo code APFS at checkout.

00:41:36 ◼   ► Book stands for, I believe, Apple Platform File System,

00:41:39 ◼   ► APFS at checkout, to save 10% off your first purchase.

00:41:42 ◼   ► Thanks to Hover for sponsoring our show.

00:41:45 ◼   ► (upbeat music)

00:41:47 ◼   ► - John, tell us about Apple File System.

00:41:50 ◼   ► Are you getting the platform part from Apple Platform File System?

00:41:55 ◼   ► Somebody said that.

00:41:56 ◼   ► I thought I'd heard that as well.

00:41:57 ◼   ► Yeah, a lot of people said a lot of things, but I'm like, "Is that from official?

00:41:59 ◼   ► I haven't heard any official thing.

00:42:00 ◼   ► I think it's just for Apple."

00:42:02 ◼   ► Anyway, there's a WWDC session that is entirely open to the public, session 701.

00:42:06 ◼   ► We will put a link in the show notes so that if you're interested in Apple File System,

00:42:09 ◼   ► you can watch this presentation.

00:42:11 ◼   ► At the time we recorded the WWDC episode, we had not seen this presentation yet because

00:42:16 ◼   ► it was two days in the future.

00:42:18 ◼   ► And there wasn't much new information in that thing that wasn't already in the State of

00:42:22 ◼   ► the Union, but there were two tidbits worth discussing.

00:42:26 ◼   ► One is, all right, so APFS, as we saw in the State of the Union, will be a developer preview.

00:42:32 ◼   ► I think a lot of people were confused by that, like, okay, so everything's developer preview,

00:42:35 ◼   ► right?

00:42:36 ◼   ► But no.

00:42:37 ◼   ► When Sierra ships for real, like the final version that ships to customers, APFS will

00:42:41 ◼   ► be included in Sierra as a developer preview, as in you will get the official release version

00:42:46 ◼   ► of Sierra and part of it of that product will be provisional developer level support for

00:42:52 ◼   ► APFS.

00:42:53 ◼   ► So, what does that mean for you know, how far in the future is APFS?

00:42:57 ◼   ► As I said in the State of the Union, APFS is developer preview this year and coming

00:43:00 ◼   ► in 2017, but what does that even mean?

00:43:03 ◼   ► A slide in session 701 had these words on it and we'll see how this rolls out.

00:43:09 ◼   ► APFS will be the default file system for all Apple products in 2017.

00:43:14 ◼   ► So that is pretty unequivocal except for the part where it says 2017.

00:43:19 ◼   ► So default file system for all Apple product means if you buy an Apple product after whatever

00:43:24 ◼   ► this point is, it will be formatted with APFS.

00:43:28 ◼   ► That includes all iPhones, all iPads, all Macs, everything.

00:43:31 ◼   ► That's the implication of this statement.

00:43:32 ◼   ► In 2017, it could mean anything.

00:43:34 ◼   ► It could mean starting January 1st, 2017.

00:43:37 ◼   ► It could mean starting December 31st, 2017.

00:43:39 ◼   ► So there's a big range in when this could happen.

00:43:43 ◼   ► But it is a pretty clear statement of intent in a publicly available video from official

00:43:48 ◼   ► Apple spokespeople that their goal is, 2017, not just that, oh, you'll be able to format

00:43:53 ◼   ► as APFS, but the default will still be HFS Plus or only on Macs or whatever.

00:43:57 ◼   ► Nope.

00:43:58 ◼   ► Every single product, every single device that has a file system across the entire product

00:44:01 ◼   ► range at some point, this is their goal anyway.

00:44:04 ◼   ► We'll see if they meet it.

00:44:05 ◼   ► So that seemed to me like an ambitious goal, but I'm also kind of excited about it because

00:44:09 ◼   ► as we all know, 2017 is the year of the file system.

00:44:13 ◼   ► And I hope they take their time because that's a little bit – I'm a little bit scared

00:44:18 ◼   ► for them.

00:44:19 ◼   ► Like, that means like when the next iPhone comes out or whatever, the 2017 iPhone, all

00:44:23 ◼   ► of those millions and millions of iPhones all going to be formatted with APFS.

00:44:28 ◼   ► I really hope that goes well.

00:44:29 ◼   ► You know, or I suppose they could say, "Okay, well, again, if they go December 31st, 2017,

00:44:35 ◼   ► then next year's iPhone will ship with HFS Plus and then December 31st, starting then

00:44:39 ◼   ► all the products will be formatted that way."

00:44:42 ◼   ► So in case anyone at Apple, anyone who knows any of the Apple file system people is listening

00:44:48 ◼   ► right now, consider that John Syracuse, your friend and mine, has been asking for a new

00:44:53 ◼   ► file system for years, has been begging for a new file system.

00:44:58 ◼   ► And as he just said, you guys, you know, Apple has said in 2017, I would just like to point

00:45:04 ◼   ► out that John Syracuse's birthday happens to be the very last day of 2017.

00:45:11 ◼   ► If you just decided, you know, maybe that would be the appropriate day to flip the switch

00:45:16 ◼   ► and hit the big red button, that would be a pretty solid birthday present.

00:45:19 ◼   ► A much better present than I've ever gotten, Jon.

00:45:21 ◼   ► Just throwing it out there, guys.

00:45:22 ◼   ► You run with it.

00:45:23 ◼   ► Yeah, I think they're gonna go before that.

00:45:26 ◼   ► And my impression is that this date, even though it seems ambitious to us, is not really

00:45:32 ◼   ► that ambitious from the perspective of Apple, who has been working on this file system for

00:45:35 ◼   ► a long time, obviously.

00:45:36 ◼   ► Like, they didn't just announce it and they started working on it last week, right?

00:45:40 ◼   ► So the impression I get from WWDC is that 2017 is not rushing it, and really they've

00:45:47 ◼   ► got the whole year, it's like, you know, it's a relaxing expanse of time in which they can

00:45:51 ◼   ► plan their roll out of this new file system.

00:45:53 ◼   ► And I really hope that's true, because if so, that's exactly how you want to do this.

00:45:57 ◼   ► The second tidbit that was in this WWDC session, which I'm sure everyone knows by now because

00:46:01 ◼   ► it's been on all of the Apple News sites, is the question of if you don't buy a new

00:46:06 ◼   ► piece of hardware, you know, which comes with APFS, but you have an existing piece of hardware,

00:46:12 ◼   ► how do you get APFS? Like, I can only think about it from the perspective of a Mac, because

00:46:17 ◼   ► I don't imagine they're going to let people like reformat their iPhones. Like, if you

00:46:20 ◼   ► restore your iPhone in 2017, in the post-APFS world, is it reformatted with APFS? I doubt

00:46:26 ◼   ► it. But certainly for the Mac, we know this is a straightforward situation. The way you

00:46:31 ◼   ► could do it is you back up your entire hard drive, you erase it, you reformat it as APFS,

00:46:35 ◼   ► restore your hard drive. You can do it with Time Machine, you can do it with SuperDuper,

00:46:38 ◼   ► lots of different solutions to doing this. All those should be possible. But in the 7.0.1

00:46:43 ◼   ► presentation at W3C, Apple said that they have something else in mind, and that is

00:46:48 ◼   ► in-place conversion of your volume format from HFS+ to APFS. So you've got a Mac,

00:46:54 ◼   ► this file system comes out in 2017, it's officially released, all the new hardware you buy comes with

00:46:58 ◼   ► it, you're like "I want to try that new file system, what do I do?" Well, first of all,

00:47:02 ◼   ► First of all, I would advise you to make a backup anyway.

00:47:05 ◼   ► And maybe two backups, and maybe three backups.

00:47:08 ◼   ► But you won't have to erase your disk.

00:47:09 ◼   ► You'd be able to take your existing disk that's formatted with HFS+ and do a thing and reboot,

00:47:17 ◼   ► and when it comes back, all your data will still all be there, you won't have erased

00:47:20 ◼   ► it, and it will be in APFS volume.

00:47:23 ◼   ► And the concept of this is freaking people out a lot.

00:47:26 ◼   ► I think I talked about the old Alsoft PlusMaker thing that would do the same conversion from

00:47:31 ◼   ► H of S to H of S plus in place, and that was terrifying

00:47:34 ◼   ► because if it screwed up in the middle,

00:47:35 ◼   ► you were left with nothing.

00:47:36 ◼   ► This, the situation with APS is a little bit different,

00:47:41 ◼   ► and you shouldn't actually be as terrified as you are.

00:47:45 ◼   ► It actually is, if you've ever seen from, I don't know,

00:47:48 ◼   ► was this on ATP or maybe it was hypercritical,

00:47:51 ◼   ► the BTRFS demo of a similar type of in-place conversion,

00:47:55 ◼   ► it's the same type of deal.

00:47:58 ◼   ► Because of the way this volume format works,

00:48:00 ◼   ► actually fairly safe. so here's the process and here's why it will not

00:48:05 ◼   ► probably destroy your data, although definitely make backups. so the first

00:48:09 ◼   ► thing it's going to do is it's going to unmount your volume. so obviously you can't be booted

00:48:12 ◼   ► from your volume to reformat it. you have to be booted from the recovery partition or

00:48:15 ◼   ► something like that or whatever. but anyway once it's unmounted no more

00:48:19 ◼   ► changes are happening to this volume that you're going to convert. so you don't

00:48:22 ◼   ► have to worry about chasing after some other change. it's not going to happen

00:48:25 ◼   ► live like the what we call the file vault conversion where you can encrypt a disk

00:48:29 ◼   ► while you're using it, which is kind of magical.

00:48:30 ◼   ► This is like unmounted, all right?

00:48:32 ◼   ► Next thing is it's gonna write

00:48:34 ◼   ► all the APFS metadata structures,

00:48:36 ◼   ► all the little pointers to where does this file begin,

00:48:39 ◼   ► where does it end, how many blocks are in it,

00:48:41 ◼   ► what is the file called,

00:48:42 ◼   ► what are all the dates about the file.

00:48:43 ◼   ► It's gonna write all that metadata

00:48:45 ◼   ► to the free space on the HFS+ volume.

00:48:47 ◼   ► So you can only do this conversion

00:48:48 ◼   ► if you have enough free space to fit the metadata.

00:48:50 ◼   ► I don't know how much it's gonna take,

00:48:51 ◼   ► but you do need some free space,

00:48:52 ◼   ► and hopefully you're just using it

00:48:54 ◼   ► filled to the brim anyway.

00:48:56 ◼   ► So it's writing just to the free area.

00:48:59 ◼   ► So at any point during this process,

00:49:00 ◼   ► when it's writing all this metadata,

00:49:01 ◼   ► when it's reading your HFS+ volume,

00:49:03 ◼   ► finding out where everything is, what everything is called,

00:49:05 ◼   ► what all the dates are, what all the permissions are,

00:49:07 ◼   ► all the ownership, everything about these files,

00:49:09 ◼   ► all the different forks, all the extended attributes,

00:49:11 ◼   ► all that stuff, and it's writing that metadata out

00:49:13 ◼   ► in APFS format to the free space.

00:49:15 ◼   ► If at any time something bad happens during this,

00:49:17 ◼   ► like someone yanks the plug in your computer,

00:49:19 ◼   ► your HFS+ volume is fine, 'cause if you reboot into it,

00:49:22 ◼   ► it doesn't care what's in the free space.

00:49:23 ◼   ► Like there's garbage in the free space.

00:49:25 ◼   ► It's free space on the HFS+ file.

00:49:26 ◼   ► Nothing is wrong with it.

00:49:27 ◼   ► It's perfectly fine.

00:49:28 ◼   ► This is the majority of the time it spends,

00:49:30 ◼   ► is looking at all your things and writing all the metadata

00:49:33 ◼   ► for all the things on your file system.

00:49:36 ◼   ► When it's finally finished doing that,

00:49:39 ◼   ► all it needs to do is this is sort of the critical section

00:49:41 ◼   ► when things can go wrong.

00:49:42 ◼   ► And the critical section should be very, very short.

00:49:44 ◼   ► It's basically delete the HFS+ superblock,

00:49:47 ◼   ► update the partition type to be APFS,

00:49:49 ◼   ► put the correct UUID in there, and you're done.

00:49:51 ◼   ► Like you just have to sort of do a switcheroo

00:49:53 ◼   ► in the front of the whole file system, say,

00:49:54 ◼   ► OK, now you're not HFS anymore, now you're APFS,

00:49:57 ◼   ► and then you reboot.

00:49:58 ◼   ► And that critical section of when something can go wrong,

00:50:00 ◼   ► like if they were halfway through,

00:50:01 ◼   ► and I bet even that probably is fairly well protected,

00:50:04 ◼   ► because, well, I guess once you would delete the HFS plus super

00:50:08 ◼   ► block, you're kind of screwed.

00:50:09 ◼   ► But anyway, this is a very small window of time

00:50:11 ◼   ► when something can go wrong.

00:50:12 ◼   ► All the rest of the time during the conversion,

00:50:14 ◼   ► everything is fine.

00:50:15 ◼   ► And I bet you could probably restore the HFS plus super

00:50:17 ◼   ► block if you were clever enough or some recovery program could

00:50:20 ◼   ► do that as well, because again, you haven't touched

00:50:22 ◼   ► any of the actual data.

00:50:24 ◼   ► Your actual files have not been modified.

00:50:25 ◼   ► The HFS+ metadata has not been modified.

00:50:28 ◼   ► None of that stuff has changed,

00:50:29 ◼   ► just sitting exactly where it was.

00:50:31 ◼   ► All you gotta do is change sort of the thing

00:50:34 ◼   ► that tells you where everything else in the file system is.

00:50:36 ◼   ► The superblock is metadata about the whole file system

00:50:38 ◼   ► that tells it how big the file system is

00:50:40 ◼   ► and where all the other metadata structures are

00:50:42 ◼   ► and all that stuff.

00:50:42 ◼   ► And by the way, speaking of the APSS UUID,

00:50:46 ◼   ► the thing that tells it what volume type this is,

00:50:48 ◼   ► one other tidbit that was sort of alluded to

00:50:50 ◼   ► in State of the Union and I think in the 701 thing

00:50:53 ◼   ► is that if you have an APFS volume

00:50:56 ◼   ► and you try to mount it on an older Mac operating system,

00:51:01 ◼   ► it'll say this disk is unreadable,

00:51:02 ◼   ► do you wanna initialize it, all that business.

00:51:04 ◼   ► But if you try to do that on El Capitan, it won't do that.

00:51:07 ◼   ► Not because El Capitan has secret support for APFS,

00:51:09 ◼   ► but just because it recognizes the UUID

00:51:12 ◼   ► and so it won't offer to erase it for you.

00:51:14 ◼   ► And that's, I guess, to try to prevent people

00:51:16 ◼   ► from accidentally erasing their newly converted disk

00:51:19 ◼   ► when they don't realize that that prompt asking them

00:51:21 ◼   ► to erase it was actually asking them to erase the disk they just converted.

00:51:25 ◼   ► So that's kind of clever.

00:51:26 ◼   ► And that's another secret thing you would never have known.

00:51:28 ◼   ► You're not going to be searching the El Capitan binaries for a UUID because you'd never find

00:51:32 ◼   ► it because it's all nonsense.

00:51:34 ◼   ► But anyway, I assume that I will do this in-place conversion again after making many, many backups.

00:51:41 ◼   ► But both in theory and in practice, this conversion method of writing new metadata structures

00:51:46 ◼   ► to free space pointing at the data in where it already exists is fairly well established

00:51:52 ◼   ► and not something crazy that Apple came up with on its own. And so I'm pretty confident

00:51:56 ◼   ► that this will actually work and it will seem like magic.

00:51:59 ◼   ► That's pretty high praise. I am very impressed. Why does it have the P in the name? Why not

00:52:06 ◼   ► AFS? Oh yeah. So that's a question I asked some

00:52:11 ◼   ► of the file system people at WWDC and it's as we speculate. Like we can't use AFS because

00:52:16 ◼   ► that's already Andrew file system. And you can go through every letter in the alphabet.

00:52:20 ◼   ► A lot of them are taken already, or every combination of two letters, they're either

00:52:23 ◼   ► taken or nonsensical. APFS is not great as an acronym, but it's probably better than

00:52:29 ◼   ► AFS, it's probably better than IFS, which I think is also already taken. Lots of the

00:52:34 ◼   ► cool letters like ZFS and XFS are also already taken. My complaint was, why isn't it a cool

00:52:39 ◼   ► name? Why isn't it like, forget about the four-letter acronym, or give it whatever four-letter,

00:52:43 ◼   ► or four letter abbreviation that you want. Why doesn't it have a cool name like, I don't

00:52:48 ◼   ► know, like "Thunder FS," "Swift," or "Grand Central Dispatch," or whatever, like with the

00:52:54 ◼   ► trains. Yeah, like you come up with a cool marketing name for it. And the upshot seems

00:53:01 ◼   ► to be that file systems are not something that most users know about. Certainly on iPhones,

00:53:07 ◼   ► no one knows what the heck their file system is, and even on Macs people don't know. The

00:53:10 ◼   ► The only time people are likely to encounter is if you're on a Mac and you go to Disk Utility

00:53:14 ◼   ► or something and you're trying to format a new disk, or like if you get info on a disk

00:53:18 ◼   ► and you see in the Get Info window.

00:53:19 ◼   ► But especially when you're formatting your disk, it says like, "Oh, how do you want to

00:53:22 ◼   ► -- what volume format do you want to use for this disk?"

00:53:24 ◼   ► And Disk Utility I think can do a bunch of different formats.

00:53:27 ◼   ► Or even, you know, OS X can -- OS X, whatever that operating system is called, yeah, Mac

00:53:32 ◼   ► OS, can mount a lot of different things.

00:53:34 ◼   ► it can mount like fat, ex fat, and like all these weird,

00:53:38 ◼   ► you know, all the weird CD file systems and stuff.

00:53:41 ◼   ► Sometimes if you present a user with this pop-up menu

00:53:44 ◼   ► of like, or you know, what volume format do you wanna

00:53:46 ◼   ► make this or what volume format is this?

00:53:48 ◼   ► They don't know what all those things mean.

00:53:49 ◼   ► They don't know what HFS+ means.

00:53:51 ◼   ► They don't know what Apple extended, journaled,

00:53:53 ◼   ► whatever case, they don't know what those things mean

00:53:55 ◼   ► at all.

00:53:56 ◼   ► So the goal of AFS, and it's a reasonable goal,

00:53:59 ◼   ► is to put an entry in that list that is absolutely clear

00:54:02 ◼   ► that that's the one that they want.

00:54:04 ◼   ► And so if you have a pop-up menu

00:54:05 ◼   ► and you're formatting a disk

00:54:06 ◼   ► and you don't know what the hell you wanna pick

00:54:07 ◼   ► and you see one called Apple File System,

00:54:10 ◼   ► you're gonna pick that one.

00:54:11 ◼   ► 'Cause you're like, all right, this is an Apple.

00:54:12 ◼   ► I see a logo in front of me.

00:54:14 ◼   ► There's one in the corner of the screen.

00:54:15 ◼   ► I guess I want Apple File System.

00:54:17 ◼   ► And you know what?

00:54:17 ◼   ► That'll be the right answer.

00:54:19 ◼   ► You do want Apple File System.

00:54:20 ◼   ► You do not want, you know, Apple extended journaled,

00:54:24 ◼   ► whatever you want, Apple File System.

00:54:26 ◼   ► So as boring as it may be,

00:54:28 ◼   ► it makes sense from a user's perspective

00:54:30 ◼   ► that A, users should never have to see this,

00:54:32 ◼   ► but B, when they do see it,

00:54:33 ◼   ► They have no idea what all those weird names mean.

00:54:35 ◼   ► They just wanna pick the one that says Apple Fosse system.

00:54:37 ◼   ► - But don't you think there might be a problem,

00:54:39 ◼   ► like well that one has a plus, this one sounds deluxe.

00:54:42 ◼   ► Like the more words, it sounds like

00:54:43 ◼   ► that's the deluxe cool option.

00:54:45 ◼   ► Like I guess maybe I want the whatever journaling means.

00:54:48 ◼   ► That's, I like journaling.

00:54:51 ◼   ► - That's the prosumer problem that maybe those people,

00:54:55 ◼   ► they're the worst anyway, but regular people

00:54:57 ◼   ► are just gonna pick the safest one.

00:54:58 ◼   ► They're not gonna say maybe, I don't know what journaled

00:54:59 ◼   ► mean but they want maybe better.

00:55:01 ◼   ► What about extended?

00:55:02 ◼   ► That's better, and what about exFAT?

00:55:04 ◼   ► I like that one, 'cause it's got an X-E-X in it.

00:55:07 ◼   ► I think they're just gonna pick Apple File System.

00:55:09 ◼   ► Anyway, it's a boring name,

00:55:11 ◼   ► but it is certainly a straightforward name,

00:55:13 ◼   ► and it is a forward-looking name in that Apple feels like,

00:55:16 ◼   ► yes, this is the Apple File System.

00:55:17 ◼   ► Now, 30 years from now, when this is old and busted,

00:55:21 ◼   ► they might have a problem,

00:55:21 ◼   ► because what do you call the next one?

00:55:22 ◼   ► But for now, Apple File System works,

00:55:25 ◼   ► and because they can't use a FS,

00:55:27 ◼   ► a PFS is the awkward abbreviation for Apple File System.

00:55:31 ◼   ► - Honest question, do you honestly think

00:55:33 ◼   ► that when this file system is outdated,

00:55:35 ◼   ► we will still even have the option

00:55:38 ◼   ► to format our own partitions?

00:55:39 ◼   ► It's like, look at iOS, iOS doesn't give us any options.

00:55:42 ◼   ► - You're still gonna have to read the APS formatted volumes.

00:55:45 ◼   ► I don't know, the question is whether the Mac

00:55:47 ◼   ► will still be around then, 'cause you don't have,

00:55:50 ◼   ► you don't see volumes on iPads and Apple TV

00:55:52 ◼   ► or the Watch or iPhones, so we'll see.

00:55:55 ◼   ► But maybe they're out.

00:55:56 ◼   ► - I mean, at this point, I think we're lucky

00:55:58 ◼   ► that we even still have disk utility.

00:56:00 ◼   ► I mean, barely, but I think we're lucky

00:56:02 ◼   ► that we have it at all.

00:56:04 ◼   ► I'd be surprised if we still have the ability

00:56:08 ◼   ► to format disks the way we want to

00:56:09 ◼   ► and format partitions the way we want to,

00:56:11 ◼   ► you know, in even 10 years.

00:56:13 ◼   ► - Yeah, well, it could be, if the file system

00:56:15 ◼   ► continues to be pushed towards the geeky side of things,

00:56:19 ◼   ► then they're free to do whatever they want.

00:56:20 ◼   ► Like, sort of like, Grand Central Dispatch

00:56:22 ◼   ► is the marketing name, but Lib Dispatch

00:56:24 ◼   ► is the library name on disk.

00:56:26 ◼   ► It's just a clear separation between the names

00:56:29 ◼   ► present in the marketing and the actual names that programmers know them by, so it may become

00:56:32 ◼   ► less important.

00:56:34 ◼   ► Finally, when, in 30 years, when this file system is outdated and being replaced by the

00:56:41 ◼   ► next file system, can we get together when you are like, what, 72, 71 at that point,

00:56:49 ◼   ► and do a podcast then with a bell in it?

00:56:52 ◼   ► By then I plan to be living like a king in Patagonia, so.

00:56:57 ◼   ► They will probably have the internet there,

00:56:59 ◼   ► and microphones will probably still exist.

00:57:02 ◼   ► - I like throwing in yet another reference

00:57:04 ◼   ► that you guys don't get.

00:57:05 ◼   ► - I knew enough to know it was a reference,

00:57:07 ◼   ► but I also knew I did not get it.

00:57:08 ◼   ► - Partial credit.

00:57:10 ◼   ► - Yeah, I'll take it.

00:57:11 ◼   ► All right, do you wanna start

00:57:12 ◼   ► running through some of the features?

00:57:14 ◼   ► Do you want me to prompt this?

00:57:15 ◼   ► How would you like to proceed, sir?

00:57:17 ◼   ► - Well, in this part,

00:57:18 ◼   ► I'm gonna talk a little bit about file names,

00:57:20 ◼   ► but I thought this would be a good opportunity

00:57:22 ◼   ► to find out how much you guys know about Unicode.

00:57:26 ◼   ► Not nearly enough. I've read the, oh god, the Spolsky thing on Unicode and string handling,

00:57:32 ◼   ► and I've already forgotten all of it.

00:57:34 ◼   ► JE: Marco, do you come across this at all? You probably don't, because I know

00:57:37 ◼   ► string takes care of this for you, and you don't even care what the internal representation is.

00:57:41 ◼   ► What about in PHP land? Do you deal with any of this stuff when you do web things,

00:57:45 ◼   ► dealing with Unicode and text and everything?

00:57:47 ◼   ► MZF1 I know a lot about Unicode, yes.

00:57:49 ◼   ► JE Alright, well, so this will be new to Casey and maybe new to some people,

00:57:53 ◼   ► but I think we have to go over a little bit of the basics before I explain what the deal is.

00:57:57 ◼   ► So, talking about this in the context of APFS is because files have names and, you know,

00:58:01 ◼   ► directories have names, right? So, when you talk about a file path, which usually doesn't show up in

00:58:05 ◼   ► Mac operating system or anyplace else, that entire thing is presented as a string, and certainly file

00:58:10 ◼   ► names are presented in strings, and you would think this is a straightforward thing, like,

00:58:13 ◼   ► that nobody thinks about, "Oh, of course, like, files have names." Like, that's fine.

00:58:18 ◼   ► But names are strings, and strings are financially complicated, and it actually matters from the

00:58:22 ◼   ► perspective of the file system because it has to do a lot of stuff with strings. So the, I guess the

00:58:28 ◼   ► easiest way for people to understand this is that Unicode is a standard for defining all the

00:58:37 ◼   ► different things that go into making a string, and things in Unicode are identified by code points,

00:58:43 ◼   ► they have numbers, right, and the numbers go up really high, all right? They start low and go

00:58:49 ◼   ► up really really high because there's lots and lots of things that you can put into strings in

00:58:53 ◼   ► languages in the United States. And the other thing to understand is that when you write something out

00:58:59 ◼   ► to disk or store it in memory or whatever we tend to break things up into bytes. And we had a

00:59:05 ◼   ► lettering system in America and the sort of western world called ASCII that defined a very

00:59:10 ◼   ► small number of numbers that corresponded to things that can be in a string and each one of

00:59:18 ◼   ► of them fit into a single byte.

00:59:20 ◼   ► So all A through Z, 1 through 0 through 9,

00:59:24 ◼   ► all the punctuation characters or whatever,

00:59:26 ◼   ► they all fit into one byte.

00:59:27 ◼   ► So that's 255 possible combinations.

00:59:30 ◼   ► And everybody was happy until we realized

00:59:32 ◼   ► that there were other characters in other languages that

00:59:34 ◼   ► didn't fit in this, right?

00:59:36 ◼   ► So when they came up with Unicode,

00:59:38 ◼   ► they tried to be nice and give ASCII all the same numbers

00:59:41 ◼   ► that it always had for compatibility reasons.

00:59:43 ◼   ► But they just keep going from there.

00:59:45 ◼   ► And at a certain point, you get up to numbers, Unicode code

00:59:47 ◼   ► points that are way higher than 255.

00:59:50 ◼   ► And when it comes time for you to write them to disk,

00:59:53 ◼   ► what do you do with those?

00:59:54 ◼   ► You can't just write them out as the big numbers

00:59:56 ◼   ► in a series of bytes, because the first byte of the number

00:59:59 ◼   ► might look like a capital letter P or something, right?

01:00:01 ◼   ► Because it's the same-- you know what I mean?

01:00:03 ◼   ► So they have come up with a series of encoding systems

01:00:06 ◼   ► where when you've got this big number that does not

01:00:08 ◼   ► fit into a single byte, and you can't write the way

01:00:10 ◼   ► you would write that number out in a sequence of bytes,

01:00:13 ◼   ► we need some way to encode these things.

01:00:15 ◼   ► One of the ways to encode it is,

01:00:16 ◼   ► instead of writing one code point every byte,

01:00:19 ◼   ► write one code point every 32 bits.

01:00:21 ◼   ► So you got a huge space for each one.

01:00:23 ◼   ► So your capital letter A,

01:00:24 ◼   ► instead of taking up a single byte,

01:00:25 ◼   ► takes up what is 32?

01:00:27 ◼   ► Four bytes?

01:00:28 ◼   ► Eight bytes?

01:00:29 ◼   ► Yeah, eight bytes.

01:00:31 ◼   ► - Four bytes.

01:00:32 ◼   ► - Yeah, there you go.

01:00:33 ◼   ► It's eight times four.

01:00:34 ◼   ► I'll get it in a second.

01:00:34 ◼   ► Anyway, but that would be incredibly wasteful

01:00:38 ◼   ► because now all of a sudden,

01:00:40 ◼   ► if you're writing something in ASCII text,

01:00:41 ◼   ► it would take up four times as much room.

01:00:43 ◼   ► That's not good, right?

01:00:44 ◼   ► There's also, we could do it in 16.

01:00:46 ◼   ► Can we do it in 16?

01:00:47 ◼   ► Well, we actually have more Unicode code points

01:00:50 ◼   ► that fit into 16 bits, but I think for a while

01:00:54 ◼   ► maybe they all fit in.

01:00:55 ◼   ► I think NSString uses this encoding

01:00:57 ◼   ► called UTF-16 internally.

01:00:59 ◼   ► - Yeah, I believe the very first version of Unicode

01:01:02 ◼   ► that they did all fit in, and then once we started realizing,

01:01:05 ◼   ► oh, there's other parts of the world,

01:01:06 ◼   ► and like ancient scripts and different combining things,

01:01:09 ◼   ► and there's all sorts of new stuff now.

01:01:11 ◼   ► - Don't forget emoji.

01:01:12 ◼   ► - Yeah, that all came later, yeah.

01:01:13 ◼   ► But it's important.

01:01:15 ◼   ► - Yeah, that's not great either,

01:01:16 ◼   ► but it also takes up a lot of room.

01:01:18 ◼   ► And the one that's in most common use these days

01:01:22 ◼   ► is called UTF-8, which is a pretty clever encoding scheme

01:01:26 ◼   ► where ASCII gets to be exactly the way it normally is.

01:01:29 ◼   ► Like, so, you know, one byte, one byte for each character,

01:01:32 ◼   ► and all the ones that are bigger,

01:01:33 ◼   ► they have these unique sequences of multiple bytes.

01:01:36 ◼   ► You can have one, you know, two bytes, three bytes,

01:01:38 ◼   ► four bytes, I think even up to five bytes.

01:01:40 ◼   ► All of these are so that all the leading bytes

01:01:42 ◼   ► are not mistaken for plain old ASCII characters or whatever.

01:01:47 ◼   ► And that's the most common encoding we use.

01:01:49 ◼   ► And this is relevant because when you create a file name,

01:01:54 ◼   ► first of all, people don't want to use a file system

01:01:55 ◼   ► where you can only create file names in ASCII

01:01:57 ◼   ► 'cause that would be annoying for people

01:01:58 ◼   ► who speak languages other than English

01:02:00 ◼   ► and even annoying for English speakers

01:02:01 ◼   ► 'cause you can have curly quotes and stuff.

01:02:03 ◼   ► But you need to write this,

01:02:06 ◼   ► you need to have the stuff in memory

01:02:07 ◼   ► and you needed to write it out to disk.

01:02:09 ◼   ► So you need some sort of representation

01:02:11 ◼   ► And that's exactly what encoding is doing in Unicode,

01:02:13 ◼   ► is they take these Unicode code points, these big numbers,

01:02:16 ◼   ► possibly very large numbers, and write them out

01:02:18 ◼   ► in a series of bytes in a way according to some standard.

01:02:23 ◼   ► The additional complication here in Unicode

01:02:26 ◼   ► is that Unicode code points-- I've been trying to say this

01:02:28 ◼   ► and trying to say thingy in other words--

01:02:30 ◼   ► instead of saying character or letter,

01:02:32 ◼   ► because it's way more complicated than that.

01:02:34 ◼   ► So a Unicode code point might be like the capital letter A,

01:02:37 ◼   ► and that's something that everyone understands.

01:02:39 ◼   ► a Unicode code point might also be something called "combining acute accent" that when this

01:02:44 ◼   ► Unicode code point follows another letter it combines to make what looks like one thing on

01:02:50 ◼   ► your screen. So if you did the lowercase letter "e" plain old ASCII lowercase letter "e" followed

01:02:56 ◼   ► by Unicode code point 301, which by the way is a number bigger than 255, called "combining acute

01:03:03 ◼   ► accent" you would get the "e" with the little line pointing up and to the right on top of it,

01:03:07 ◼   ► the acute accent, so you're going to write the word "cafe" c-a-f-e combining acute accent,

01:03:14 ◼   ► you get the word "cafe" with the e with the little thing over it, right? But you can write that same

01:03:19 ◼   ► thing, "cafe" c-a-f and then e with a little thing over it, in a different way. You could write c-a-f,

01:03:24 ◼   ► just like in ASCII, and then you can include Unicode, cone point, e9, that's in hex, Latin

01:03:29 ◼   ► small letter e with acute. In other words, Unicode has two ways for you to write something that looks

01:03:35 ◼   ► the same on the page. And I'm not sure what the original motivation is for this, probably

01:03:41 ◼   ► because you could use combining characters in more flexible ways than including every

01:03:45 ◼   ► possible combination, but sometimes combinations are convenient to include. This adds yet another

01:03:50 ◼   ► wrinkle which is you can write the same thing in Unicode in multiple different sequences

01:03:55 ◼   ► of code points. That's a problem for something like a file system. It's not a problem for

01:04:00 ◼   ► like if you're writing a report or printing a page or even making a web page, but it's

01:04:03 ◼   ► a problem for file systems because file systems need to write file names out to disk, like to store

01:04:08 ◼   ► them in the metadata somewhere. And in general, file systems don't want you to be able to have

01:04:14 ◼   ► a file with the quote-unquote "same name" in the same place, ignoring file name extensions, which

01:04:19 ◼   ► just make this a mess. But if you wanted to make a file called "cafe" and then you wanted to make

01:04:23 ◼   ► another file called "cafe," you wouldn't want them both to be existing in the same folder staring you

01:04:27 ◼   ► in the face, like how can these files both be here? They both have the same exact name. But if you were

01:04:31 ◼   ► to merely write out the encoded bytes for this thing, whether it's UTF-32, UTF-8, UTF-16,

01:04:37 ◼   ► whatever encoding format the file system chooses, if you were to just write them out like that,

01:04:42 ◼   ► one application could make "CAF" Latin small letter "e" with a "cute" and one application

01:04:47 ◼   ► could make "CAFE" combining a "cute" accent.

01:04:49 ◼   ► As far as the file system is concerned, the sequence of bytes of these files is different.

01:04:52 ◼   ► So when it diffs them, it says, "Nope, there's no file with this name.

01:04:54 ◼   ► I'll just make this file."

01:04:55 ◼   ► But when you open the folder, you'd see two files called "cafe" and that is bad.

01:04:59 ◼   ► And I haven't even gotten into case sensitivity, but as you can see in a case-sensitive situation,

01:05:07 ◼   ► the file system has the same problem.

01:05:08 ◼   ► Is there already a file with this name?

01:05:10 ◼   ► Yes or no?

01:05:11 ◼   ► And that's where you would factor in, okay, well, is there a file with this name but with

01:05:14 ◼   ► like the capitalized letters ignoring case?

01:05:17 ◼   ► But ignoring that entirely, just for plain old case-sensitive whatever, there's multiple

01:05:22 ◼   ► ways you can write the same word.

01:05:24 ◼   ► And Unicode handles that by a system that calls, what is it called, normalization?

01:05:28 ◼   ► normalize forms of Unicode, it has a bunch of different forms, we'll link to them in the show note.

01:05:32 ◼   ► What it basically comes down to is, should we try to

01:05:36 ◼   ► break apart every character into its smallest possible pieces, decompose

01:05:40 ◼   ► them, you want to have the lowercase e and then the combining accent, or should we

01:05:44 ◼   ► try to compose them all into their canonical form by squishing

01:05:48 ◼   ► the e with the combining accent into the Latin small letter e with the cute, or should we

01:05:52 ◼   ► do different combinations of them in different orders, a whole bunch of different normalized forms.

01:05:56 ◼   ► The file system, if it wants to, well,

01:06:00 ◼   ► I was gonna say the file system has to pick one of these,

01:06:02 ◼   ► but it doesn't.

01:06:02 ◼   ► It could just do what I said and say,

01:06:05 ◼   ► I will accept whatever bytes you give me

01:06:07 ◼   ► and I have no idea what they mean

01:06:08 ◼   ► and I will just put them in the file system.

01:06:10 ◼   ► Or it could pick one of these normalized forms.

01:06:12 ◼   ► You pick a normalized form,

01:06:13 ◼   ► then it doesn't matter what the application gives you.

01:06:15 ◼   ► It will always be canonicalized

01:06:17 ◼   ► when it goes to the file system.

01:06:18 ◼   ► So this is what HFS+ does,

01:06:20 ◼   ► but of course, HFS+ being a very old file system

01:06:22 ◼   ► does not use any of the Unicode normalized forms.

01:06:25 ◼   ► It uses a variant of those normalized forms

01:06:27 ◼   ► that has changed in different versions

01:06:28 ◼   ► of the operating system.

01:06:30 ◼   ► If you look at TechNote 1150 from Apple,

01:06:32 ◼   ► which apparently is no longer online,

01:06:34 ◼   ► so we'll link to like another version of it.

01:06:36 ◼   ► It uses kind of decomposed form

01:06:40 ◼   ► except for a whole bunch of ranges that it changes

01:06:42 ◼   ► to be compatible with like the Mac Greek encoding

01:06:44 ◼   ► and it excludes a bunch of things.

01:06:46 ◼   ► It's really, really confusing.

01:06:49 ◼   ► But anyway, in HFS+,

01:06:50 ◼   ► if you try to make two files called Caffe,

01:06:52 ◼   ► and I wrote, of course, a little ProScript to test this out,

01:06:55 ◼   ► And you try those two different ways to do it.

01:06:57 ◼   ► The E with the combining accent

01:06:58 ◼   ► and then the E with the little hat already combined.

01:07:01 ◼   ► Doesn't matter what you write.

01:07:03 ◼   ► The HFS+ will be like,

01:07:04 ◼   ► "Oh yep, I totally made that file for you."

01:07:06 ◼   ► But when you read that file back in

01:07:07 ◼   ► and look at the file name,

01:07:09 ◼   ► it will have changed it to the normalized form

01:07:13 ◼   ► the HFS+ wants.

01:07:14 ◼   ► So you will not make two separate files.

01:07:16 ◼   ► If you try to make one file with that one name

01:07:18 ◼   ► and one file with the other name,

01:07:19 ◼   ► you will just overwrite the same file twice.

01:07:21 ◼   ► you like, HFS+ does not take your file name for what it is.

01:07:26 ◼   ► It does something to it first,

01:07:27 ◼   ► or rather the driver for HFS+ does

01:07:29 ◼   ► to be compliant with the file system, right?

01:07:31 ◼   ► This is one of the reasons that a lot of Unix nerds have,

01:07:33 ◼   ► and I think Linus Torvalds had a big rant about it,

01:07:35 ◼   ► but I couldn't find it on the internet,

01:07:36 ◼   ► have complained about HFS+.

01:07:38 ◼   ► I gave you what the file name is, it's supposed to be this.

01:07:42 ◼   ► And you said, "No, I totally made that file for you."

01:07:44 ◼   ► And then later, when I try to read that file

01:07:46 ◼   ► and look at its name, it's not what I told you.

01:07:48 ◼   ► And that's bad for things like Git,

01:07:50 ◼   ► or Linux for that matter, or other things that deal with files,

01:07:52 ◼   ► they don't expect that to happen.

01:07:54 ◼   ► The sort of Unix-style approach is, here's a sequence of bytes

01:07:57 ◼   ► that makes up this file's name, make that file.

01:08:00 ◼   ► And then later, when I read that file, I'm going to look at its name,

01:08:03 ◼   ► and it better be that.

01:08:05 ◼   ► And I better be able to look it up with that name,

01:08:06 ◼   ► and the name that comes on the disk better be with the bytes that I gave you,

01:08:09 ◼   ► and if it doesn't, your file system is broken.

01:08:11 ◼   ► But as I hopefully explained, if you do that,

01:08:14 ◼   ► it's very easy for applications to make names that look exactly the same

01:08:19 ◼   ► in the user interface, cafe and cafe,

01:08:21 ◼   ► like they're indistinguishable.

01:08:22 ◼   ► They're exactly the same pixel for pixel

01:08:25 ◼   ► because you can't see the bytes

01:08:26 ◼   ► that make up the Unicode string.

01:08:27 ◼   ► You shouldn't care about them.

01:08:28 ◼   ► And to users to be able to make the same file

01:08:30 ◼   ► with the same name, with apparently the same name,

01:08:32 ◼   ► that seems like a bug.

01:08:33 ◼   ► It seems like something shouldn't happen.

01:08:34 ◼   ► Which cafe file is the one that I want?

01:08:36 ◼   ► They both have the same name.

01:08:37 ◼   ► I don't know.

01:08:39 ◼   ► So you can't do that.

01:08:40 ◼   ► And that's why Apple doesn't do that.

01:08:42 ◼   ► APFS now has to make some decisions about this.

01:08:45 ◼   ► What should APFS do for file names?

01:08:47 ◼   ► Should it do it the HFS+ way exactly?

01:08:49 ◼   ► Probably not because that's got years and years of baggage

01:08:51 ◼   ► floating around in it from the fact that it existed

01:08:54 ◼   ► before Unicode, I think, or even when,

01:08:55 ◼   ► or maybe just it came out when Unicode was very young.

01:08:57 ◼   ► I think HFS+ predates Unicode, but not HFS+.

01:09:00 ◼   ► But anyway, doing what HFS+ does

01:09:04 ◼   ► would be best for compatibility,

01:09:06 ◼   ► but it would also be slightly crazy.

01:09:07 ◼   ► I don't think Apple wants to maintain

01:09:10 ◼   ► their weird mapping tables forever

01:09:13 ◼   ► with their different non-standard normalized forms.

01:09:17 ◼   ► Should it do what many Linux file systems do

01:09:20 ◼   ► and many other processes do,

01:09:21 ◼   ► which is I take no position on file names.

01:09:24 ◼   ► That's not my concern.

01:09:24 ◼   ► You gave me a sequence of bytes and that's the file name.

01:09:27 ◼   ► I will write that out to the file system.

01:09:28 ◼   ► Later when you read it,

01:09:29 ◼   ► you will get back that sequence of bytes.

01:09:30 ◼   ► I don't know what that sequence of bytes means.

01:09:32 ◼   ► Could mean anything.

01:09:33 ◼   ► I have no idea what that stuff is.

01:09:34 ◼   ► But anyway, here's what you said.

01:09:36 ◼   ► My understanding is that that's what APFS does today.

01:09:38 ◼   ► So if you run my little pro script in APFS,

01:09:41 ◼   ► You can make two files that apparently are both named Caffe,

01:09:45 ◼   ► but they're actually different sequences of bytes

01:09:47 ◼   ► because one uses the combining acute accent

01:09:49 ◼   ► and the other one uses the E

01:09:50 ◼   ► with the little accent already composed on it.

01:09:52 ◼   ► And by the way, I think it might be, well, anyway,

01:09:57 ◼   ► when you, if you were to do this, when I did it in Perl,

01:09:59 ◼   ► I had to choose how to encode the file name

01:10:02 ◼   ► and I chose UTF-8.

01:10:04 ◼   ► I think you can't, the API, well, I'm using Perl,

01:10:07 ◼   ► so it has different rules about that.

01:10:08 ◼   ► But the bottom line is when you,

01:10:10 ◼   ► Somebody has to pick the byte representation.

01:10:13 ◼   ► You can't give it a sequence of bytes for your string

01:10:15 ◼   ► and use Unicode code point E9 or Unicode code point 301

01:10:19 ◼   ► because they don't fit in a byte.

01:10:20 ◼   ► So you have to figure out a way to fit them in a byte

01:10:22 ◼   ► and that's what Unicode encodings are.

01:10:23 ◼   ► And so I picked UTF-8.

01:10:24 ◼   ► If I had given it UTF-16,

01:10:26 ◼   ► I'm assuming it would have stored UTF-16

01:10:28 ◼   ► and given me back UTF-16 and so on

01:10:30 ◼   ► for all the other different encoding things.

01:10:32 ◼   ► So as far as I can tell, APFS is a bag of bytes.

01:10:34 ◼   ► That means it's case sensitive

01:10:36 ◼   ► because bag of bytes means you gave me these bytes,

01:10:38 ◼   ► here are these bytes.

01:10:38 ◼   ► And so if another file comes along and wants to write "Cafe" with a lowercase c instead

01:10:42 ◼   ► of a capital C, it's a different sequence of bytes, file system says, "Nope, no file

01:10:46 ◼   ► with that name, here you go."

01:10:49 ◼   ► I think this will...

01:10:51 ◼   ► I don't know.

01:10:52 ◼   ► I don't think this is the official position because APFS is not done.

01:10:55 ◼   ► You can't even boot from it right now.

01:10:57 ◼   ► It's obviously not finished.

01:11:00 ◼   ► This is what I'm gonna be looking for in the coming months and in the rest of 2017.

01:11:05 ◼   ► What will be the policy in APFS?

01:11:07 ◼   ► Will they add case insensitivity?

01:11:09 ◼   ► Once you decide to add case insensitivity,

01:11:11 ◼   ► you have to make some decisions.

01:11:13 ◼   ► At the very least, you have to make decisions about encoding

01:11:15 ◼   ► because you can't compare case on two things.

01:11:17 ◼   ► Like if someone writes one file name in UTF-16

01:11:19 ◼   ► and one file name in UTF-8,

01:11:21 ◼   ► something in the system has to understand that.

01:11:25 ◼   ► You can't do case comparisons with strings encoded

01:11:28 ◼   ► in different ways.

01:11:28 ◼   ► Or maybe you just straightforwardly compare it as if,

01:11:31 ◼   ► I don't know, it doesn't make sense.

01:11:32 ◼   ► Case comparisons,

01:11:33 ◼   ► you have to have an awareness of the encoding, right?

01:11:35 ◼   ► So if they ever want to make a case insensitive version of APFS,

01:11:39 ◼   ► they need to decide, here's how file names are encoded.

01:11:43 ◼   ► And then beyond that, they have to make a decision

01:11:46 ◼   ► about normalization.

01:11:47 ◼   ► Do we do normalization at all?

01:11:48 ◼   ► If you don't do normalization, you

01:11:50 ◼   ► can get into these weird situations

01:11:51 ◼   ► where you have files with apparently the same name.

01:11:54 ◼   ► You could handle this all at the framework level,

01:11:56 ◼   ► handle it all in Cocoa, handle it all in whatever

01:11:59 ◼   ► higher level frameworks you like it, so on and so forth.

01:12:04 ◼   ► I don't know if that's something that Apple will do.

01:12:05 ◼   ► And it's not, I think this is not a minor decision

01:12:08 ◼   ► because what you decide to do here has implications

01:12:12 ◼   ► that ripple down through history

01:12:13 ◼   ► as we saw with the HFS plus one.

01:12:14 ◼   ► After you make this decision, it's not easy to change it.

01:12:17 ◼   ► I suppose you could go with case sensitive now

01:12:20 ◼   ► and do case insensitive later,

01:12:22 ◼   ► but kind of like the opposite of what HFS did,

01:12:24 ◼   ► going with case insensitive from the beginning

01:12:25 ◼   ► and making a case sensitive variant.

01:12:27 ◼   ► But anyway, this I think is one of, to me,

01:12:30 ◼   ► the most important decisions that has not yet been made

01:12:33 ◼   ► for APFS.

01:12:35 ◼   ► And since it will probably happen before next year's WWDC,

01:12:38 ◼   ► the only way we'll be able to tell whether it's made

01:12:40 ◼   ► is looking at the APFS documentation online

01:12:43 ◼   ► and continuing to run tiny test programs

01:12:45 ◼   ► from languages other than Apple's frameworks

01:12:47 ◼   ► to see what does the file system accept.

01:12:50 ◼   ► I don't know what the right position,

01:12:52 ◼   ► I don't have a particular position here,

01:12:53 ◼   ► only I just want them to make a decision

01:12:56 ◼   ► and for it to be a reasonable one.

01:12:59 ◼   ► - And that's what I was about to ask,

01:13:00 ◼   ► is that you don't necessarily have a preference

01:13:04 ◼   ► between normalized and not normalized.

01:13:06 ◼   ► You just wanna see that there is a declared statement

01:13:11 ◼   ► as to how it's gonna work.

01:13:12 ◼   ► - Yeah, and like a philosophical,

01:13:14 ◼   ► some sort of philosophical statement of support,

01:13:16 ◼   ► because in many respects, making the file system itself

01:13:19 ◼   ► just be bag of bytes, which is you gave me bytes,

01:13:22 ◼   ► I stored them, is very straightforward.

01:13:24 ◼   ► It's the easiest to test.

01:13:26 ◼   ► It is the simplest.

01:13:28 ◼   ► It's the fastest.

01:13:29 ◼   ► You can do really fast comparisons with it.

01:13:32 ◼   ► It's everything you want from a performance perspective

01:13:34 ◼   ► if you were like, say, on the file system team

01:13:36 ◼   ► and you wanted to make a really fast file system.

01:13:38 ◼   ► But from a user's perspective, it

01:13:39 ◼   ► doesn't solve user problems.

01:13:41 ◼   ► It doesn't solve user problems that HFS+ does.

01:13:46 ◼   ► I don't know if you can solve it all at the framework level.

01:13:49 ◼   ► I have to think that you can't really at the framework level

01:13:52 ◼   ► entirely because there's so many different ways and so

01:13:57 ◼   ► different programs that can write files to disk, and not all of them go through your frameworks,

01:14:02 ◼   ► right? And it's madness to have some file names in UTF-16, some file names in UTF-8, some normalize,

01:14:10 ◼   ► some denormalize, some a weird mix. That seems untenable to me from just the basic perspective of

01:14:16 ◼   ► how do I correctly check whether a file exists? The file name, blah. How do I check whether that

01:14:22 ◼   ► exists? It's like, well, how many different ways can you write this? You know, the only way I can

01:14:26 ◼   ► I can check for this because the file system knows nothing about encoding.

01:14:29 ◼   ► I say, "Hey, file system, do you have a file named cafe?"

01:14:32 ◼   ► And the file system's like, "All right, well, what sequence of bytes do you want me to check

01:14:35 ◼   ► for?"

01:14:36 ◼   ► "Nope, I don't have one with that sequence of bytes."

01:14:37 ◼   ► "Hmm, I think is there another way that I could write cafe?

01:14:40 ◼   ► I could write it in UTF-32 with a combining accent.

01:14:42 ◼   ► Do you have this sequence of bytes?"

01:14:43 ◼   ► And the file system's like, "Nope, don't have that sequence of bytes."

01:14:45 ◼   ► And the application's like, "Hmm, maybe they did it in UTF-16, but with the Latin small

01:14:49 ◼   ► letter E with a Q on it."

01:14:50 ◼   ► And they're like, "That's untenable."

01:14:52 ◼   ► And so as scary as it is for the file system to make decisions about this, because they're

01:14:56 ◼   ► just they're very difficult to change after the fact.

01:14:59 ◼   ► If it doesn't, then every file system API

01:15:03 ◼   ► needs to either be incredibly thorough

01:15:04 ◼   ► or make it so that you can potentially

01:15:07 ◼   ► create files with apparently the same name from the user's

01:15:10 ◼   ► perspective.

01:15:11 ◼   ► And I think that's not healthy either.

01:15:12 ◼   ► So I don't know.

01:15:13 ◼   ► It could be that we don't care about any APIs,

01:15:15 ◼   ► except for the Cocoa ones, because we

01:15:16 ◼   ► don't care what people going straight through the C APIs

01:15:19 ◼   ► or writing Python scripts or whatever we're doing.

01:15:21 ◼   ► All we care about is the things that go through AppKit and UI

01:15:24 ◼   ► it and then you could have it be bag of bytes. I don't know. But I can hear arguments on either

01:15:31 ◼   ► side of it, but the fact that nothing has been said so far and the fact that it just does bag

01:15:34 ◼   ► of bytes now doesn't really tell me anything about their position. It just tells me they didn't get

01:15:39 ◼   ► to that part yet. Because I think if they had got to it, it would have been in that presentation.

01:15:43 ◼   ► Like the presentation of certain developers would say, "Test your application on APFS." They did

01:15:48 ◼   ► tell them to test it based on the case sensitivity part, and that makes some sense because Apple has

01:15:53 ◼   ► has shipped a case-sensitive file system for a long time,

01:15:55 ◼   ► and a lot of applications break on case-sensitive file systems,

01:15:58 ◼   ► like Adobe's applications and Microsoft applications,

01:16:01 ◼   ► all apps that were created in the days

01:16:02 ◼   ► before the Mac could even have a case-sensitive file system.

01:16:06 ◼   ► Many of them, believe it or not, attempt

01:16:07 ◼   ► to open files or libraries and stuff

01:16:09 ◼   ► by file paths that do not match the case of the actual files

01:16:12 ◼   ► on disk.

01:16:13 ◼   ► You would think that those are the type of bugs

01:16:14 ◼   ► that would have been fixed long ago,

01:16:16 ◼   ► but the bottom line is Mac users don't

01:16:17 ◼   ► run case-sensitive file systems for the most part.

01:16:20 ◼   ► iOS, luckily, has always had case-sensitive file systems.

01:16:22 ◼   ► But anyway, case sensitive is separate

01:16:24 ◼   ► from whether things are normalized.

01:16:26 ◼   ► So I think if Apple was committed

01:16:30 ◼   ► to Bag of Bytes file naming,

01:16:32 ◼   ► they would have maybe mentioned that

01:16:34 ◼   ► in the file system session, but I'm not sure.

01:16:36 ◼   ► But anyway, it's a thing I think about a lot,

01:16:38 ◼   ► and I have concerns.

01:16:39 ◼   ► - Our final sponsor this week is Backblaze.

01:16:43 ◼   ► Backblaze is unlimited native online backup for Mac and PC.

01:16:48 ◼   ► Go to backblaze.com/atp for a no credit card required,

01:16:52 ◼   ► no risk, free 15 day trial.

01:16:54 ◼   ► Now Backblaze is the best online backup.

01:16:58 ◼   ► That isn't in the script, I don't know if they can say that

01:16:59 ◼   ► but I can say that because I've used many online backup

01:17:02 ◼   ► programs and services and Backblaze is by far the best one

01:17:07 ◼   ► that's worked out for me and my favorite one

01:17:08 ◼   ► and I've been using them since long before

01:17:10 ◼   ► they were a sponsor.

01:17:11 ◼   ► I have them backing up my computer, TIFF's computer,

01:17:13 ◼   ► even my mom's computer.

01:17:15 ◼   ► I have them backing up something like six terabytes

01:17:17 ◼   ► of my data.

01:17:19 ◼   ► It is so solid, it is literally never caused us a problem

01:17:23 ◼   ► for the, I don't know, four or five years

01:17:25 ◼   ► I've been using them.

01:17:26 ◼   ► I highly recommend having online backup,

01:17:28 ◼   ► especially if you're gonna be playing around

01:17:30 ◼   ► with the file system on your computer.

01:17:32 ◼   ► You should really have online backup to save your butt

01:17:36 ◼   ► if you do anything stupid or if your local backups

01:17:38 ◼   ► are all corrupted by bit rot from HFS Plus.

01:17:41 ◼   ► So check out Backblaze today, backblaze.com/ATP

01:17:45 ◼   ► for that risk-free 15-day free trial.

01:17:49 ◼   ► Backplace has a bunch of other cool features too.

01:17:51 ◼   ► So for instance, you can of course restore your files

01:17:54 ◼   ► on the web, you can download files as a big zip file,

01:17:57 ◼   ► you can even have them overnight you a hard drive

01:17:59 ◼   ► with all your data, if it's like a ton of data

01:18:01 ◼   ► and it'll be taken to download.

01:18:02 ◼   ► And if you return the hard drive back to them

01:18:05 ◼   ► within 30 days, you get a refund

01:18:07 ◼   ► on the price of the hard drive.

01:18:08 ◼   ► So it's a very effective way to restore

01:18:09 ◼   ► a ton of data at once.

01:18:11 ◼   ► And not only that, on the other side,

01:18:13 ◼   ► if you wanna just restore one file,

01:18:15 ◼   ► You can do it in the web interface

01:18:17 ◼   ► or their iOS or Android apps.

01:18:19 ◼   ► So one great use for this is if you're on your phone,

01:18:22 ◼   ► or let's say you're traveling

01:18:23 ◼   ► and you only have your laptop or something,

01:18:24 ◼   ► and you wanna file it from your home computer,

01:18:26 ◼   ► you can do this with Backblaze,

01:18:27 ◼   ► 'cause Backblaze has all your files.

01:18:29 ◼   ► It's just automatically there, running,

01:18:31 ◼   ► backing up everything.

01:18:32 ◼   ► You don't really have to think about it,

01:18:33 ◼   ► which by the way is how a backup should work.

01:18:34 ◼   ► If you have to think about it, it's not really a backup.

01:18:37 ◼   ► So with Backblaze, you can log in and just restore one file.

01:18:39 ◼   ► If you forgot a file on a trip or something like that,

01:18:41 ◼   ► it is incredibly convenient.

01:18:42 ◼   ► I've done this probably five or six times now,

01:18:44 ◼   ► because it's like, it doesn't happen often,

01:18:47 ◼   ► but when it does happen, I'm so glad I have it.

01:18:49 ◼   ► And that's kind of the story of online backup.

01:18:51 ◼   ► Like, you won't usually need to use it,

01:18:54 ◼   ► but you'll be really glad you've had it

01:18:56 ◼   ► on those few times that you do use it.

01:18:58 ◼   ► Just five bucks a month for unlimited space,

01:19:02 ◼   ► unthrottled speeds per computer.

01:19:04 ◼   ► So five bucks per month per computer,

01:19:06 ◼   ► it is a great deal, unlimited space.

01:19:08 ◼   ► So check it out today, go to backblaze.com/atp,

01:19:12 ◼   ► and our listeners get a 15 day free trial.

01:19:14 ◼   ► Backblaze.com/ATP.

01:19:17 ◼   ► Thanks a lot.

01:19:17 ◼   ► - You guys have any opinions on before we move on?

01:19:23 ◼   ► Case sensitivity?

01:19:25 ◼   ► (laughing)

01:19:27 ◼   ► - So for the two questions that you've basically posed here,

01:19:34 ◼   ► I think the file system should,

01:19:37 ◼   ► you know, on the topic of like, you know,

01:19:38 ◼   ► bag of bytes versus at least normalizing,

01:19:41 ◼   ► you know, normalizing to some degree,

01:19:42 ◼   ► whether you normalize for like Unicode normal forms

01:19:45 ◼   ► for combining characters and things,

01:19:47 ◼   ► or at least whether you normalize for character encoding,

01:19:49 ◼   ► I think encoding is an obvious win.

01:19:51 ◼   ► Like you should absolutely normalize for character encoding.

01:19:55 ◼   ► And whatever encoding they pick,

01:19:58 ◼   ► I'm less concerned about that.

01:19:59 ◼   ► If you pick UTF-8, cool.

01:20:02 ◼   ► Whatever encoding they wanna pick, that's fine,

01:20:05 ◼   ► but I think it is a clear win

01:20:07 ◼   ► that the encoding should be normalized

01:20:10 ◼   ► for file system access.

01:20:11 ◼   ► Whether or not normalizing forms of characters

01:20:16 ◼   ► and case sensitivity, I mean, that's kind of

01:20:18 ◼   ► two different degrees of the same kind of thing,

01:20:21 ◼   ► that I am less confident on, but my inclination would be,

01:20:26 ◼   ► I think the era of case insensitivity is probably over,

01:20:32 ◼   ► but I would say that if I were designing a system,

01:20:36 ◼   ► and having heard no other arguments

01:20:38 ◼   ► except for your large amount of them.

01:20:41 ◼   ► (laughing)

01:20:44 ◼   ► I think the Cocoa APIs should be responsible

01:20:48 ◼   ► for normalizing or not normalizing the name,

01:20:51 ◼   ► like you know, case and combining characters wise.

01:20:54 ◼   ► That the low level C APIs and like the raw interface

01:20:58 ◼   ► of the file system should not perform

01:21:01 ◼   ► that kind of like messing with characters.

01:21:03 ◼   ► - Should the file system throw a fatal error

01:21:05 ◼   ► if the encoding is incorrect,

01:21:08 ◼   ► like if you give an invalid UTF-8 sequence,

01:21:10 ◼   ► should that just not work?

01:21:11 ◼   ► Or should it dutifully write those to disk anyway?

01:21:14 ◼   ► - Yeah, 'cause if it doesn't do that,

01:21:15 ◼   ► then you have the same problems before,

01:21:16 ◼   ► where you can write a name,

01:21:19 ◼   ► and then you go to read that name and it's not there.

01:21:22 ◼   ► So I can see why that could be a problem.

01:21:24 ◼   ► So maybe yes, maybe the answer is that

01:21:27 ◼   ► if you try to have an invalid sequence in the name

01:21:30 ◼   ► that you give it, then maybe it does error out.

01:21:31 ◼   ► I don't know.

01:21:33 ◼   ► - All right, because it's basically a question of,

01:21:34 ◼   ► So say they pick UTF-8, which is exactly what they'd pick, because that's what everyone

01:21:37 ◼   ► picks, because all the other encodings are stupid-er.

01:21:41 ◼   ► If they pick that, then they could just say, "Oh, when you write your file names to APFS,

01:21:48 ◼   ► you should write them in UTF-8."

01:21:50 ◼   ► And then if you don't do it, that's your own stupid default.

01:21:53 ◼   ► In other words, there's a policy, not an implementation enforcement.

01:21:58 ◼   ► And so then later, when some other program tries to...

01:22:00 ◼   ► In other words, if you wrote it with an invalid UTF-8 sequence, and then later your program

01:22:03 ◼   ► tried to read it with that same invalid UTF-8 sequence, it would find it, right? But it would

01:22:07 ◼   ► be garbage and then any other API that's trying to like compare files or whatever would just,

01:22:12 ◼   ► you know, fail in weird ways because you have not even written a valid UTF-8 sequence and when it

01:22:17 ◼   ► does it, like if you tried to say, "Is there a file called CAFE?" it would be like, "Nope,

01:22:20 ◼   ► don't see it," because you just wrote an invalid sequence. The other one is to have the file system

01:22:26 ◼   ► enforced at the, you know, at the driver level and all the various operating systems.

01:22:30 ◼   ► if you pass some file name down through the system, down through the BSD layers,

01:22:34 ◼   ► at some point we're going to look over your entire sequence and confirm is this a valid

01:22:37 ◼   ► UTF-8 sequence? And if it's not, boom, everything blows up, forget it, nothing works, or you know,

01:22:42 ◼   ► whatever. And I don't know, I think checking for, checking that it's valid UTF-8 shouldn't take long

01:22:50 ◼   ► and should be fairly straightforward because we're not dealing with large volumes with data,

01:22:53 ◼   ► so that's what I think they would go with if they make that decision, that it will essentially be

01:22:56 ◼   ► impossible to write an invalid UTF-8 sequence to a file name, which could also break applications.

01:23:03 ◼   ► Every decision they make, aside from "make it work exactly like HFS+," is going to break

01:23:07 ◼   ► applications. And "make it work exactly like HFS+" is the one decision that I think is

01:23:11 ◼   ► entirely wrong.

01:23:12 ◼   ► I mean, heck, in this day and age, it wouldn't have been reasonable to have an actual Intel

01:23:17 ◼   ► hardware instruction to validate a sequence of UTF-8 characters.

01:23:21 ◼   ► Yeah, because the rules aren't that complicated.

01:23:24 ◼   ► No.

01:23:25 ◼   ► You don't even have to see whether it corresponds to a defined code point and a unified standard.

01:23:30 ◼   ► That's more difficult to do, certainly in hardware.

01:23:32 ◼   ► No, it's just like bit masking.

01:23:34 ◼   ► Yeah.

01:23:35 ◼   ► But anyway, I suspect that's probably what they'll end up being doing.

01:23:38 ◼   ► But case sensitivity, I think, is still a weird issue because I think users of computers,

01:23:45 ◼   ► even just on iOS where you're not really ever prompted to write a file name, but certainly

01:23:49 ◼   ► on the Mac, having two files called like, I don't know, like, you know, XMIS, and then

01:23:55 ◼   ► you write a file called XMIS but you decide to capitalize the X and you end up with two

01:23:58 ◼   ► files, you know, you thought you were overwriting the previous one, I don't know, that just—case

01:24:02 ◼   ► insensitivity I think is more like how regular people think computers work.

01:24:06 ◼   ► Case sensitivity is how programmers think computers work, for sure, so I'm sure that

01:24:11 ◼   ► it would be very popular with programmers, but I mean, or maybe not, because like I said,

01:24:16 ◼   ► The fact that there are still Mac applications that only work on case-sensitive file systems

01:24:20 ◼   ► due to eminently fixable errors in their source code, it's like that's never going to change

01:24:27 ◼   ► unless you force it to change.

01:24:28 ◼   ► So, I don't know.

01:24:29 ◼   ► Casey, you have opinions?

01:24:30 ◼   ► Yeah, you know, I think that of all the places you want to have the Wild West, the file system

01:24:36 ◼   ► is not it.

01:24:37 ◼   ► And I think that forcing and encoding makes sense.

01:24:41 ◼   ► I agree with you mostly that users expect case insensitivity, but I don't know.

01:24:52 ◼   ► I think what with the internet running on Unix for the most part and URLs generally

01:24:56 ◼   ► being case sensitive, I think that people are starting to become more comfortable with

01:25:01 ◼   ► that.

01:25:03 ◼   ► So if I had a vote, I would say enforce something like UTF-8.

01:25:08 ◼   ► If you write something that's garbage or not UTF-8 encoded, barfall over it and do so violently.

01:25:16 ◼   ► And I would favor a case sensitivity by default.

01:25:19 ◼   ► That's my vote.

01:25:20 ◼   ► I don't think anyone's in favor of normalization, but like I said, you end up in weird situations.

01:25:24 ◼   ► So I would suspect if they don't touch normalization at the file system level, then they do it

01:25:29 ◼   ► all at the API level.

01:25:30 ◼   ► But then it's just, it's pretty easy to make fairly absurd situations where I bet you could

01:25:36 ◼   ► pick some word or phrase that has a huge number of possible combinations, like in some language

01:25:42 ◼   ► that uses accents on its characters, and just, you know, the number of possible permutations

01:25:47 ◼   ► goes up pretty quickly as you add the accented characters. Then you can make a folder full of

01:25:51 ◼   ► literally hundreds of files that all appear to have exactly the same name in the Finder.

01:25:54 ◼   ► That's something you can't do in HFS Plus, and it's something that you shouldn't be able to do,

01:26:00 ◼   ► but will be able to do with the file system, even if they pick an encoding, but they don't

01:26:04 ◼   ► don't pick a normalization.

01:26:05 ◼   ► So that's why I'm thinking about it.

01:26:08 ◼   ► That's because it's not a straightforward problem.

01:26:09 ◼   ► It's not an obvious answer.

01:26:10 ◼   ► Oh, clearly Apple will do this.

01:26:12 ◼   ► There is no easy answer.

01:26:14 ◼   ► I never thought we'd talk this long

01:26:15 ◼   ► about the detailed implementations of dealing

01:26:18 ◼   ► with file names in a file system.

01:26:20 ◼   ► Well, it's more just about Unicode.

01:26:22 ◼   ► And even though you and I both know it having dealt with it,

01:26:25 ◼   ► but I can tell you the first five times I dealt with Unicode,

01:26:28 ◼   ► I still didn't understand it, only on the sixth time.

01:26:32 ◼   ► If you have never heard anything about Unicode, hearing what I just said about all the code

01:26:36 ◼   ► points and normalization is not going to sink in.

01:26:38 ◼   ► But it's weird, and if you don't know about it and think about it when dealing with strings,

01:26:44 ◼   ► you will eventually be sad.

01:26:45 ◼   ► No, I mean, dealing with Unicode, in my experience, is a lot like parenting, in that the more

01:26:52 ◼   ► you—the more experience you get in the area, the more you realize that you will just always

01:26:57 ◼   ► be wrong and there are no good answers.

01:26:59 ◼   ► Wow.

01:27:00 ◼   ► Well, I mean, there's one thing that's just understanding the standards, which I think

01:27:03 ◼   ► is difficult enough.

01:27:04 ◼   ► And then once you understand the standards, then you realize no matter what I pick, there

01:27:08 ◼   ► are downsides.

01:27:09 ◼   ► And then the real sad realization is that you realize all the places, if you're a programmer,

01:27:13 ◼   ► you realize all the places in your program where text is coming in without the encoding

01:27:18 ◼   ► specified.

01:27:19 ◼   ► And you're like, I don't even know what the right thing to do is.

01:27:22 ◼   ► How is this going to be encoded?

01:27:24 ◼   ► This is just bytes coming across some sort of, you know, reading out of a disk, a file

01:27:27 ◼   ► on disk, or coming across the network.

01:27:30 ◼   ► And then you end up rippling outwards and asking all of your input sources, "Hey, could

01:27:34 ◼   ► you tell me what encoding you're using?"

01:27:35 ◼   ► And they say, "Encoding?

01:27:36 ◼   ► What?"

01:27:37 ◼   ► And then you realize you basically have an intractable problem where, especially if you're

01:27:40 ◼   ► accepting input from the wider world, that it doesn't come tagged along with what encoding

01:27:46 ◼   ► it is, and you don't know how to make sense of it unless you know the encoding, so you

01:27:49 ◼   ► kind of have to, "I can detect the encoding through a bunch of heuristics, or I can guess,

01:27:52 ◼   ► or I can try to enforce downstream that you should only give it to me in this encoding."

01:27:55 ◼   ► And then you find out how many people are actually sending you things that don't conform

01:27:59 ◼   ► or all of a sudden you get a bunch of UTF-16 or shift-gis and you have, oh, this is terrible,

01:28:04 ◼   ► text is terrible, we should communicate through a series of ones and zeros, it would be much

01:28:08 ◼   ► easier.

01:28:09 ◼   ► [laughter]

01:28:10 ◼   ► Wow.

01:28:11 ◼   ► I've never heard shift-gis discussed in a podcast, and I don't think I ever will again.

01:28:17 ◼   ► Oh, yeah, but my first big normalization project was, like, I would have killed for UTF-8.

01:28:22 ◼   ► Like, that's how bad it was.

01:28:25 ◼   ► Before Unicode, things were worse, let's just put it that way.

01:28:27 ◼   ► - Yeah, no, I mean, that's what, like,

01:28:29 ◼   ► I mostly avoided having to deal with these issues

01:28:31 ◼   ► over time with my various web servers,

01:28:33 ◼   ► 'cause like, you're right that like, in native apps,

01:28:36 ◼   ► you don't generally have to deal too much with this.

01:28:39 ◼   ► On the web, you gotta deal with it all the time.

01:28:41 ◼   ► And the easiest thing on the web is to just

01:28:43 ◼   ► make all your pages serve UTF-8, serve themselves as UTF-8,

01:28:48 ◼   ► which would then make all browsers default

01:28:49 ◼   ► to sending you UTF-8 in forms.

01:28:52 ◼   ► - Only the modern world will make your browsers do that.

01:28:54 ◼   ► In the old world, you could do whatever you wanted

01:28:56 ◼   ► with your web pages, but people in Japan

01:28:58 ◼   ► were gonna be sending you things in a different encoding,

01:29:00 ◼   ► and it doesn't matter if you put UTF-8 as the,

01:29:02 ◼   ► you know, like, that was great when browse

01:29:04 ◼   ► just finally started to honor the encoding of the page.

01:29:07 ◼   ► - Yeah, but that, I mean, that was,

01:29:09 ◼   ► most of my career has been after that.

01:29:10 ◼   ► I mean, like, when we were building Tumblr in 2006,

01:29:13 ◼   ► that was the case, so it's been okay.

01:29:15 ◼   ► But, and then, the only time I really had to think

01:29:18 ◼   ► about Unicode on web stuff recently

01:29:21 ◼   ► is when emoji came around, and I had to alter

01:29:24 ◼   ► all my MySQL tables to be the UTF-8 MB4 character set,

01:29:29 ◼   ► which is annoying.

01:29:30 ◼   ► But after that, it was fine.

01:29:33 ◼   ► - MySQL made some Unicode mistakes.

01:29:36 ◼   ► The best part of the web and coding is,

01:29:39 ◼   ► even coming into the case sensitivity,

01:29:41 ◼   ► is the old URL that we increasingly don't see

01:29:44 ◼   ► in our address bar.

01:29:45 ◼   ► The hostname part is not case sensitive,

01:29:47 ◼   ► whatever you think that means.

01:29:49 ◼   ► The encoding for URLs is,

01:29:51 ◼   ► I think it is basically limited to ASCII,

01:29:55 ◼   ► but there's this way you can encode Unicode code points

01:29:58 ◼   ► with the special escape sequence-y thing.

01:30:00 ◼   ► It's not-- - Yeah, in the hostname

01:30:01 ◼   ► there is, in the rest of the URL,

01:30:03 ◼   ► God knows what's going on there.

01:30:05 ◼   ► In the hostname, there's a special

01:30:06 ◼   ► international domain extension thing,

01:30:08 ◼   ► but that has a bunch of security problems, actually.

01:30:11 ◼   ► - Right, of course, and then like you said,

01:30:13 ◼   ► for the rest of the URL, it's kind of the Wild West,

01:30:16 ◼   ► and it's the worst that half of the string is case-sensitive

01:30:19 ◼   ► and half of it isn't, again, whatever the hell

01:30:20 ◼   ► think case sensitive means. URLs are still a problem. But in that respect, the body of your

01:30:27 ◼   ► HTML documents is blessedly normalized at this point. Because like you said, most modern browsers

01:30:32 ◼   ► honor the encoding. Anyway, text is hard. All right. Well, speaking of things that are hard,

01:30:39 ◼   ► how about data integrity? Yeah, I'm touching this last time. This is the one feature I wanted from

01:30:45 ◼   ► from APFS that it did not deliver, and we're speculating about why that might be or if

01:30:51 ◼   ► it really was the case.

01:30:52 ◼   ► Data integrity, just a quick review, is basically the idea that if a program writes some data

01:30:59 ◼   ► to disk, later it reads that data back, the data that it originally wrote should be the

01:31:04 ◼   ► data that it gets back.

01:31:07 ◼   ► And that could be a day later, a year later, ten years later.

01:31:11 ◼   ► If that's not the case, because hardware is fallible, you would like the file system to

01:31:16 ◼   ► know, "Hey, you asked for these bytes, but it turns out that since they were written,

01:31:22 ◼   ► they have been screwed up."

01:31:24 ◼   ► So restore from backup or solve this problem in other ways.

01:31:28 ◼   ► ZFS and some other file systems have a way to make multiple redundant copies of the same

01:31:32 ◼   ► data, and when they detect this error they can fix it, because they have a good copy

01:31:37 ◼   ► and a bad copy and they will just ignore the bad copy and use the good copy and write another

01:31:41 ◼   ► copy of the good one or whatever.

01:31:43 ◼   ► All that stuff is expensive because it

01:31:45 ◼   ► means you have to write out checksums with all of your data

01:31:47 ◼   ► and you have to sort of do end-to-end checksums

01:31:50 ◼   ► so that everyone is sure, yep, I'm sending you this

01:31:52 ◼   ► and here's the checksum.

01:31:52 ◼   ► Yep, I got that and here's the checksum.

01:31:54 ◼   ► I'm writing to disk and making sure everything is all good.

01:31:57 ◼   ► That's a feature of ZFS.

01:31:59 ◼   ► It's a very important and wonderful feature of ZFS.

01:32:02 ◼   ► But it has a cost associated with it,

01:32:04 ◼   ► both in terms of computation of computing all those checksums

01:32:06 ◼   ► and also dependency because you can't really

01:32:10 ◼   ► be done writing the data until you've also made sure all the data was written correctly

01:32:13 ◼   ► and then also written the checksum and made sure that, you know, so there's lots of dependencies

01:32:17 ◼   ► that are inherent in that. And mostly I wanted it for the case of bit rot as in over long

01:32:23 ◼   ► periods of time. I write out my family's photos to disk and I make backups of that disk and

01:32:28 ◼   ► I push that up to backblaze and it's on time machine and I'm doing super duper clones.

01:32:32 ◼   ► And if some of those bits are flipped because of errors in the hardware, all I'm doing is

01:32:36 ◼   ► copying those now corrupted bits over and over and over again over the years to different backups.

01:32:41 ◼   ► You know, the incremental backups don't go on forever. There's a window of time, you know,

01:32:44 ◼   ► before they'll fit on the disk and the same thing with backblaze. They don't think they keep your

01:32:48 ◼   ► backups forever and ever and ever, every different version. So eventually you will just have be

01:32:54 ◼   ► propagating the corruption over the years until 20 years later when you try to look at some baby

01:32:57 ◼   ► picture, your favorite baby picture is all messed up and all scrambled in JPEG because a bunch of

01:33:01 ◼   ► bits flipped and you don't know which ones. That's what I'm trying to protect against.

01:33:05 ◼   ► and APFS does not provide that feature, does not have the option to write out checksums with all

01:33:11 ◼   ► of its file data. Now APFS is a flexible file system intentionally designed to be extensible

01:33:16 ◼   ► without breaking backwards compatibility. This feature could conceivably be added in the future,

01:33:22 ◼   ► hell it could be added before 2017, they got a whole year right, but it's not there right now

01:33:26 ◼   ► and that's a little bit disappointing. It does write checksums for metadata. Metadata is like

01:33:31 ◼   ► here's what the file's name is, here's all the dates, the ownership, the permissions,

01:33:34 ◼   ► and here's where each individual piece of this file data is on the disk so you know where to go

01:33:38 ◼   ► get it. I keep saying disk but you know what I mean. And it writes checksums for metadata.

01:33:43 ◼   ► I mean there's good reasons to do that, mostly because this file system, like many others,

01:33:49 ◼   ► tries to... doesn't do always consistent on disk representation but it comes close. It wants to

01:33:56 ◼   ► know did I finish this operation? So if someone yanks out the plug to your computer and it was in

01:34:00 ◼   ► the middle of writing a file, what you want to happen when you reboot is either the entire file

01:34:04 ◼   ► is there or none of the file is there. And checksums on metadata could conceivably help

01:34:09 ◼   ► with that because what you're going to do is you're going to write the data and write the

01:34:13 ◼   ► metadata, but if the metadata is not complete, like if we didn't finish, if we're not done,

01:34:17 ◼   ► then you just ignore all of that. It's all invalid and the file basically isn't there.

01:34:20 ◼   ► You have a consistent file system. Don't worry about that partially written file or that partially

01:34:24 ◼   ► written metadata or whatever. And how do you tell whether the metadata is partially written? Well,

01:34:28 ◼   ► all your metadata has checksums. When you're looking at that piece of metadata, you can look

01:34:32 ◼   ► at it and do the checksum and say does the checksum match the metadata and if it does

01:34:36 ◼   ► it's complete and if it doesn't you know you can do that with journaling and stuff but

01:34:39 ◼   ► apfs does not use journaling journaling is a relatively expensive way to do the same thing

01:34:43 ◼   ► where you sort of write out what you plan to do then you do it then you write out that you're done

01:34:47 ◼   ► and that's how it can tell whether you finished an operation or not and metadata is in general

01:34:52 ◼   ► more important than data because if you screw up the metadata you could hose your entire disk like

01:34:56 ◼   ► screwing up the metadata for an entire directory and then everything under it becomes invisible

01:34:59 ◼   ► or whatever. And metadata is really small and the checksums are really fast on it,

01:35:03 ◼   ► so there's no real performance hit. So APFS does do checksums with metadata, does not do it with

01:35:08 ◼   ► data. This is a topic that came up a lot at WWDC, talking to other people who had... The other file

01:35:14 ◼   ► system nerds who had also talked to people on the file system team about APFS. And I think this was

01:35:23 ◼   ► a very popular feature suggestion. "Hey, it would be great if APFS had data integrity." Or the

01:35:28 ◼   ► question why doesn't APFS have data integrity and a lot of the the reasons

01:35:32 ◼   ► are you know things that I said that is potentially expensive one of the reasons

01:35:37 ◼   ► that I heard was that SSDs do their own checksumming internally which is true

01:35:42 ◼   ► they do because they have to deal with the pretty startlingly unreliable nature

01:35:48 ◼   ► of flash storage you know all sort of memory chips are inherently at the

01:35:55 ◼   ► bottom tiny little analog devices that are a little bit scary when you find out

01:35:58 ◼   ► how they work, and SSDs are no different.

01:36:01 ◼   ► SSDs have over-provisioning, where they have many more memory cells than they're going

01:36:05 ◼   ► to use, and there's actually a tiny little operating system in there that's trying to

01:36:08 ◼   ► distribute all this stuff to all the storage, and they wear out if you use them too much.

01:36:12 ◼   ► It's very, very complicated.

01:36:14 ◼   ► But they do copious, not just checksums, but also error-correcting code, so if they find

01:36:18 ◼   ► a bit flip because of some analog error or whatever, they can fix it before returning

01:36:22 ◼   ► the data.

01:36:23 ◼   ► So that's one of their stopgaps, is like, "Hey, spinning disks don't do that so much,"

01:36:27 ◼   ► they're going out and SSDs do the checksums.

01:36:30 ◼   ► That's not really a great solution

01:36:31 ◼   ► because there are other places

01:36:32 ◼   ► that errors can be introduced like,

01:36:35 ◼   ► other hardware other than the SSD itself,

01:36:37 ◼   ► like the bus and everything.

01:36:37 ◼   ► But since Apple controls that entire chain,

01:36:41 ◼   ► they control the bus, they control the SSDs in theory,

01:36:44 ◼   ► especially since they make Macs.

01:36:45 ◼   ► Don't make Macs anymore that you can swap

01:36:47 ◼   ► lots of different hard drives into and everything.

01:36:49 ◼   ► That they're a little bit better off than everyone else.

01:36:51 ◼   ► But it doesn't really help with bit rot.

01:36:52 ◼   ► If I write all my family photos to a one terabyte SSD,

01:36:55 ◼   ► they get there successfully.

01:36:56 ◼   ► And then five years later, I have no idea

01:36:58 ◼   ► if those bits are flipped.

01:36:59 ◼   ► The file system can't tell me because it doesn't know what

01:37:01 ◼   ► bits were originally written.

01:37:02 ◼   ► All it can do is read them and say,

01:37:03 ◼   ► here's what I've got on disk.

01:37:04 ◼   ► Is that what you wrote five years ago?

01:37:06 ◼   ► Beats the hell out of me.

01:37:07 ◼   ► I don't know.

01:37:07 ◼   ► So I've been thinking about possible other solutions

01:37:11 ◼   ► if Apple never builds in Data Integrity to APFS.

01:37:14 ◼   ► And one of them is something that I

01:37:16 ◼   ► think there's an HFS+ program that does this already.

01:37:18 ◼   ► User space solution.

01:37:21 ◼   ► Just scan all your files, checksum them,

01:37:23 ◼   ► write out the checksum someplace else.

01:37:25 ◼   ► And then if anything goes wrong with either the checksum

01:37:28 ◼   ► or the data itself, the two won't match anymore,

01:37:31 ◼   ► and you will know that your file is corrupted