Developing Perspective

#179: Avoiding the Stampede.


00:00:00   Hello and welcome to Developing Perspective.

00:00:02   Developing Perspective is a podcast discussing news of note in iOS development, Apple, and

00:00:06   the like.

00:00:07   I'm your host, David Smith.

00:00:08   I'm an independent iOS developer based in Herna, Virginia.

00:00:11   This is show number 179.

00:00:13   Today is Thursday, April 3rd.

00:00:16   Developing Perspective is never longer than 15 minutes, so let's get started.

00:00:19   All right, so I'm finally back from all of my travels and conference speaking, which

00:00:24   all I'd say went pretty well for some of my first times out.

00:00:28   But I'm glad to finally be able to button down in April, probably be focusing more on

00:00:33   building things rather than just talking about building things.

00:00:35   Which I guess is somewhat ironic, even that I'm starting off the month talking on a podcast.

00:00:40   But anyway, so unless you live under a rock or in under a rock that only has internet

00:00:46   such that you can download 15 minute podcasts, you probably heard that this week WWDC was

00:00:53   announced, which is a delightful break

00:00:56   from their typical pattern, or by typical,

00:00:58   I mean what they've done the last couple of years.

00:01:00   They announced in late April, where they've

00:01:02   announced now nice and early.

00:01:04   We have the dates.

00:01:05   It's the first week in June.

00:01:06   And they're doing something different this year.

00:01:08   They're distributing tickets through a kind

00:01:11   of more of a lottery, or a random selection,

00:01:13   I think is how Apple called it every time.

00:01:15   And I'm pretty excited about that.

00:01:17   The more I thought about it, the more it kind of became obvious

00:01:19   that this is where they were going to have to go,

00:01:22   that the old system that was largely based on either knowing

00:01:27   when tickets were going to be in terms of having alert systems

00:01:29   and things, then rushing and doing it,

00:01:31   what they tried last year with the stampede,

00:01:34   those are all kind of-- they just don't scale well.

00:01:37   And the only system that scales once you have too high--

00:01:41   way too high demand for the number of tickets

00:01:43   you have available, if you don't want to or can't increase

00:01:46   the number of tickets, is to go with something like a lottery.

00:01:49   And so that's what they did.

00:01:50   We'll all put our names in the hat.

00:01:53   You have until, I believe it is 10 AM on Monday to do it.

00:01:56   So if you have a chance between now and then

00:01:58   and you're wanting to try, by all means do.

00:02:02   But then through some process, randomly

00:02:05   select 5,000 names out of that hat.

00:02:09   And if you get one, great.

00:02:10   And if you don't, I'm sorry.

00:02:13   But by and large, I think this is a great improvement.

00:02:15   And hopefully it'll go well.

00:02:17   And this is where they'll be heading going forward.

00:02:20   would expect. And I think it's generally good because while no system is perfect,

00:02:24   there's no way that they can build, you know, build a structure for getting

00:02:29   tickets where everyone who wants a ticket will get one. That's impossible,

00:02:33   unless they're able to drive down interest in developing for Apple's platforms to such a

00:02:38   degree that there's no longer 5,000 people who want to do it, which seems very unlikely.

00:02:42   Or they would have to do something like, how would you determine who are the, you know,

00:02:49   the 5,000 correct people to get it,

00:02:50   in terms of for some definition of correctness.

00:02:52   Some people would say, like, oh, I want the old guard

00:02:54   to always get their tickets.

00:02:55   The most successful people should get their tickets.

00:02:57   The newest people should get their tickets

00:02:59   so we can build the platform.

00:03:00   The youngest, the oldest.

00:03:01   I mean, you could imagine whatever you want.

00:03:03   The reality is you just want to do it randomly.

00:03:05   You just want to give everybody a chance, an even chance,

00:03:08   an honest chance to get a ticket.

00:03:10   And then from there, just hope everyone

00:03:14   gets seen as sort of hope things--

00:03:16   people understand if they don't get one.

00:03:18   And on the flip side, I hope the people who get tickets, use them, enjoy them, take them

00:03:23   to full advantage.

00:03:25   I'll be out in San Francisco either way.

00:03:26   I've already booked my plane and hotel for the week.

00:03:29   I'll be out there either way because it's such a wonderful week.

00:03:33   Even if I can get a ticket, I'd still want to be out there to meet, sort of see the people

00:03:36   who I only typically see once a year at DubDub, as well as just being around.

00:03:42   It's a really kind of jazzing energy that is always kind of a motivating thing for the

00:03:45   middle of my year.

00:03:46   So that's kind of what I'm doing.

00:03:48   I said make sure you put your name in the hat if you want.

00:03:51   WVDC is really cool.

00:03:52   If you've never been, I highly recommend having the experience at least once.

00:03:57   You know, getting up early, waiting in line, going to some of the kind of special events

00:04:01   that there aren't videos for.

00:04:02   It's fun.

00:04:03   Obviously it's kind of a bit of an expensive vacation, but it's definitely useful and fun.

00:04:09   And I think there's something about it that is kind of a bit infectious.

00:04:14   And so that's kind of cool.

00:04:15   All right, so I'm going to be diving into the main topic.

00:04:18   I'm going to be talking for the rest of the show.

00:04:20   And it's a slight break for my App Store series.

00:04:23   I'll be picking that up.

00:04:24   I have a few more topics I want to talk up through.

00:04:26   But with W3C popping up today, I wanted

00:04:29   to make sure I had a full episode the next time I

00:04:31   did the App Store thing.

00:04:32   And so W3C took about a third of it.

00:04:34   So I was going to do something a little bit shorter.

00:04:36   And so it's going to be what I want to talk about,

00:04:38   is server hosting.

00:04:40   And this is going to be maybe a bit of a funny thing

00:04:42   to talk about.

00:04:42   But it really got prompted from a series of podcasts and conversations back and forth.

00:04:49   I think it got started with Justin Williams was talking about it, Bart Garment, Brent

00:04:53   Simmons, you know, we've done a lot of these back and forths about what level of abstraction

00:04:58   is the appropriate place to find for your own application.

00:05:02   So in the one extreme, obviously, you could build your back ends, you know, your and honestly,

00:05:07   it's probably even taken even step further and say, almost every utility application

00:05:11   these days uses some form of online backend.

00:05:14   Maybe if you're a game, you can get away with it.

00:05:16   But even then, you're probably using something like Game Center.

00:05:19   Very few apps now work well as just the app in itself

00:05:24   in complete isolation.

00:05:26   You're always interacting outwardly,

00:05:28   because that's often even what makes apps useful.

00:05:31   And so if you're going to interact with something,

00:05:34   you have to decide what you're going to interact with.

00:05:36   Are you going to, on the one end,

00:05:38   to kind of purchase very high level, very large building blocks, kind of building your

00:05:43   backend with Duplo.

00:05:44   You know, are you going to be sitting there and just kind of snapping some big blocks

00:05:48   together to build your backend?

00:05:50   You know, using things that are very simple, very well defined interfaces, kind of these

00:05:55   big blocks.

00:05:56   Or are you going to go farther and farther down the stack?

00:05:57   Maybe you're going to go from Duplo to Lego and then at some point you're going to be

00:06:00   building your own Lego, but you buy an injection molder to make it.

00:06:04   And then you're going to be buying the machine that's necessary to build the injection molder.

00:06:09   You could imagine if you brutalized that metaphor to go all the way from the Duplo all the way

00:06:13   down.

00:06:14   But at every level, you're making different choices.

00:06:18   And there's always trade-offs between the different levels as you're going.

00:06:22   And the reality is-- and I've worked at all of those levels.

00:06:27   And so I feel like I have some authority to talk about this.

00:06:32   But the reality is at every level, you're making trade-offs.

00:06:35   And you're just choosing which trade-offs

00:06:37   make the most sense for you and your product.

00:06:40   And those are going to differ for every different--

00:06:42   every project is going to have a different set of trade-offs

00:06:45   that make the most sense for it.

00:06:47   But here's some of the kind of things that I think about

00:06:49   that will hopefully-- maybe I thought

00:06:50   it would be helpful to kind of get you thinking as you're

00:06:52   trying to plan out your back end,

00:06:54   as you're trying to decide what makes sense for you.

00:06:56   First is to say, every abstraction choice

00:06:59   you make in your hosting stack is a leaky abstraction.

00:07:05   And by that I mean, if you take the core functionality,

00:07:08   say whatever it was, if the functionality is,

00:07:11   I want you to take this picture and I need to put it somewhere

00:07:15   and then retrieve it later, say it's a very simple operation

00:07:19   conceptually, not simple to implement,

00:07:21   but simple conceptually, I have these bytes

00:07:25   that I need to put somewhere and then get back.

00:07:28   Now I could use a service on the one end that

00:07:30   was very abstracted, that basically just said,

00:07:33   save picture, retrieve picture.

00:07:36   That's all it did.

00:07:37   It's very, very high level, very abstract.

00:07:40   And that's great.

00:07:42   Or I could go all the way to the other extent,

00:07:45   I could build a web application who

00:07:48   takes in some binary data, who then packages it up,

00:07:51   and then take that binary data and I distribute it out

00:07:54   to a content distribution network or whatever.

00:07:57   And you could build and imagine systems all the way

00:08:01   in between of those two things.

00:08:02   Like, from a very, very low level,

00:08:04   dealing with the raw bytes, putting them on disks,

00:08:07   to something that's just like a web endpoint that you

00:08:10   hit with a post method.

00:08:12   Like, you could very easily kind of go in either direction.

00:08:16   And understand that the very abstract version,

00:08:18   you know, the very high level version, it's nice.

00:08:22   It's easy to use.

00:08:22   You're just like, here's my thing.

00:08:24   I'm going to put it in there.

00:08:25   And that's usually fine.

00:08:28   But every now and then, little implementation details,

00:08:30   little other things that are being hidden away

00:08:32   are going to leak through.

00:08:33   And so you have to understand what those are likely.

00:08:36   What are those problems?

00:08:37   What are those issues that are going to come back to bite you

00:08:40   so that you can understand if those trade-offs make sense

00:08:44   to you and those trade-offs are worthwhile to you?

00:08:47   And inevitably, they will.

00:08:48   Inevitably, they'll be things or cases or situations

00:08:52   where you try using that backend in a way that it falls down.

00:08:57   Say, for example, you have a very, very large image.

00:09:00   You'd have a 10 gig image, and you try and send it

00:09:03   to your very, very high level thing,

00:09:05   and it's just the whole thing falls apart.

00:09:07   Well, that's why I did it.

00:09:08   Well, it's a leaky abstraction.

00:09:10   Maybe it just isn't built to handle that thing.

00:09:12   Or if you give it a very small file,

00:09:16   you give it a zero byte file, maybe weird things will happen.

00:09:20   Things that are just intrinsic in the way it was implemented

00:09:24   that you won't necessarily find out right away.

00:09:27   Now, obviously, those are contrived examples,

00:09:29   but hopefully they kind of get you

00:09:30   thinking of what I'm saying.

00:09:31   No matter how abstract the front end interface is,

00:09:34   there's always going to be back end implementation details that

00:09:37   are going to affect you.

00:09:38   So the first question you always have to ask yourself

00:09:40   when you're doing this is, what can you live with?

00:09:45   What are the things that you feel comfortable having

00:09:47   to-- the constraints and things that you're

00:09:49   have to place in your application as a result of making these choices.

00:09:54   And also, whatever choice you-- the deeper down in the stack you go, the more flexibility

00:10:01   you have, the more things that you can do, the fewer constraints you have overtly on

00:10:05   your application, that you can do whatever you want.

00:10:08   And that's awesome, and that's terrifying.

00:10:12   If you can do anything, that means you can do anything.

00:10:14   You can do really bad, horrible decisions that you then have to maintain and deal with

00:10:19   forever. Or you can build awesome solutions to not problems that nobody else can solve

00:10:24   because no one else has built the great back end. So you have to be thoughtful about what

00:10:29   constraints do you want to work in? How big of a sandbox do you want to get messy in?

00:10:36   Generally what I find, and this is kind of just a personality thing I think in a large

00:10:39   degree, is that the more and more I do this, the deeper into the stack I go. And that's

00:10:46   interesting because I suspect, I suspect there'll be a curve or a couple of curves, I guess,

00:10:51   into this where as I keep going, I'll go deeper and deeper into the stack. And at some point,

00:10:57   I'll likely sort of start coming back up again. And I think a lot of that has to do with the

00:11:03   role that learning plays in this. Because as a developer, I love learning new things. I love

00:11:08   learning new platforms and technologies. That's something that I just really enjoy. And so part

00:11:14   of what's made feed wrangler, for example, really interesting

00:11:17   from a personal level is that I'm learning

00:11:20   a lot of great new stuff.

00:11:21   I'm understanding now, feed wrangler

00:11:25   is built from pretty small pieces.

00:11:27   It uses Postgres, obviously, as a database, which is pretty big.

00:11:31   But exactly how that's deployed and managed and orchestrated

00:11:35   is all done by me.

00:11:36   I'm SSHing into servers and editing config files

00:11:38   and doing all this very low level stuff.

00:11:41   And some of it was out of necessity

00:11:44   because it's a very data-intensive application.

00:11:46   But a lot of it, too, is just because it's really interesting to learn.

00:11:49   And I can do things now that I wouldn't have been able to otherwise.

00:11:52   And the number of times things that I can learn from implementing a UI table

00:11:56   view again isn't as fun for me right now as it

00:11:59   is for doing this really cool server networking stuff.

00:12:03   And so that's kind of just really fun.

00:12:05   And even beyond just being fun, I mean, it's cool.

00:12:10   And it's weird to say, but it's just cool.

00:12:12   Like this last week, one of my main database servers

00:12:15   for Feed Wrangler was running out of hard drive space

00:12:18   because I kind of poorly planned it.

00:12:20   And so it's like, OK, I'll go and provision a new server,

00:12:23   have it physically built in my data center,

00:12:26   have that deployed out, set it up,

00:12:28   configure all the security and the permission stuff,

00:12:31   go and do all that, deploy Postgres onto it,

00:12:33   set it up from a backup into a live streaming binary copy

00:12:38   of the main primary master server that was starting to run out of space,

00:12:44   get it set up so that the two are perfectly in sync and running perfectly forward,

00:12:47   and then dynamically switch everything over to that being the new master,

00:12:52   which is crazy.

00:12:53   Like, the fact that I was able to-- and it actually worked.

00:12:57   It's a funny thing when you roll out a server change and everything goes well.

00:13:01   And that's almost scarier than things going poorly.

00:13:04   But I've done this now a couple times.

00:13:07   I know what I'm doing in a way that is really fun.

00:13:10   And so I kind of would be sad if I

00:13:12   was using a service that didn't-- I just hit a button

00:13:14   and says, like, expand database size.

00:13:17   Brrp!

00:13:18   Like, that'd be kind of sad.

00:13:19   I wouldn't be learning.

00:13:20   And so it's kind of fun.

00:13:23   Another reason I tend to go low level

00:13:25   is I enjoy being able to optimize things

00:13:28   for my application.

00:13:29   I can work through a lot of bottlenecks

00:13:31   that I wouldn't be able to if I didn't have access

00:13:34   to much lower level stuff.

00:13:35   things where I'm tweaking kernel values,

00:13:37   when I'm changing the memory configurations on Linux servers.

00:13:40   You can get really low level, and you can increase performance

00:13:43   in tangible ways that you just couldn't do otherwise.

00:13:46   It's pretty cool.

00:13:48   One of the things is the realization

00:13:50   that no matter what level I do it,

00:13:52   I'm going to have to monitor the system.

00:13:53   A lot of people say, well, I use an off-the-shelf system.

00:13:57   I use Azure.

00:13:57   I use AWS.

00:13:59   I use something higher level because I

00:14:01   want to have someone-- if something goes wrong,

00:14:03   I want someone else to be trying to fix it.

00:14:05   In some ways, that's better.

00:14:06   But the reality is, if it goes down, it goes down.

00:14:08   And I'm going to have to worry about that either way, which

00:14:11   kind of sucks, but it is how it is.

00:14:14   And lastly, it's often cheaper.

00:14:17   And really, that's the bottom line for an indie like me.

00:14:21   A lot of these services, I look at them like Azure.

00:14:24   And it's just expensive.

00:14:26   It's compared to me just running some raw Linux servers.

00:14:30   It just doesn't really scale.

00:14:32   And so you have to make that choice.

00:14:33   You have to think about your application.

00:14:35   Think what trade-offs you want to live with,

00:14:37   what abstractions are leaky that you'd be OK with.

00:14:40   And just find what works best for you.

00:14:42   All right, that's it for today's show.

00:14:43   As always, if you have questions, comments, concerns,

00:14:45   or complaints, I'm on Twitter @_davidsmith,

00:14:47   david@devlaucomperspective.com.

00:14:49   Have a good week.

00:14:49   Good luck on Monday.