Under the Radar

00:00:00 ◼   ► Welcome back to Under the Radar, a show about independent iOS development.

00:00:04 ◼   ► I'm Marco Arment.

00:00:05 ◼   ► And I'm David Smith.

00:00:06 ◼   ► Under the Radar is never longer than 30 minutes, so let's get started.

00:00:10 ◼   ► So today I think we wanted to talk about probably one of the most common

00:00:15 ◼   ► but yet challenging parts of being a developer, at least in my experience,

00:00:19 ◼   ► and that deals with unexpected complexity.

00:00:23 ◼   ► When you start off working on something and you have kind of an expectation

00:00:27 ◼   ► of how complicated it's going to be to do, what it's going to look like, the effort involved,

00:00:30 ◼   ► the time involved. And then you sit down, you open up Xcode, you start programming, and it turns out

00:00:36 ◼   ► that it's a complete nightmare and it's going to be way more effort and way more work.

00:00:41 ◼   ► And that experience happens so often that I think having a good process and mindset about dealing

00:00:48 ◼   ► with it is one of the very important things of being able to ship software in a reasonable time

00:00:54 ◼   ► and reasonable quality.

00:00:56 ◼   ► And I think we both recently had an experience

00:00:57 ◼   ► that kind of fell into that category,

00:01:00 ◼   ► and so this episode I think will be more

00:01:03 ◼   ► of a case study style in terms of talking

00:01:06 ◼   ► through our experiences.

00:01:07 ◼   ► And so I believe that is the experience that you had

00:01:11 ◼   ► when you were working on the streaming engine

00:01:12 ◼   ► for Overcast, am I right?

00:01:14 ◼   ► - Oh yeah.

00:01:15 ◼   ► I mean streaming as a feature,

00:01:17 ◼   ► so Overcast is a podcast player,

00:01:19 ◼   ► and the built-in Apple APIs for playing media using AVPlayer,

00:01:24 ◼   ► most of that stuff supports streaming out of the box.

00:01:29 ◼   ► You don't really have to do much to enable it.

00:01:32 ◼   ► I mean, depending on how much you want your interface

00:01:35 ◼   ► to behave, you don't necessarily have to do anything.

00:01:37 ◼   ► So it's relatively easy to support streaming media

00:01:41 ◼   ► just as well as you do locally played media

00:01:43 ◼   ► if you're using AVPlayer.

00:01:45 ◼   ► But one of Overcast's biggest selling features,

00:01:47 ◼   ► and one of the things that makes it most compelling,

00:01:50 ◼   ► both to me and to many of the users,

00:01:52 ◼   ► is my custom audio engine with a feature called Smart Speed,

00:01:56 ◼   ► where Smart Speed dynamically adjusts the playback.

00:02:00 ◼   ► And the way it works is there's a processing function that

00:02:06 ◼   ► needs to read in the audio samples

00:02:09 ◼   ► in the middle of the processing stream

00:02:11 ◼   ► and then output possibly a different number of them

00:02:15 ◼   ► if it's speeding up the sound.

00:02:16 ◼   ► So it might take in 500 samples,

00:02:20 ◼   ► and then the middle of it might be silence

00:02:22 ◼   ► as it detects silence, so it might compress that

00:02:25 ◼   ► down to 300 samples.

00:02:26 ◼   ► So you need the ability to inject into the processing stage

00:02:31 ◼   ► a function that can read faster than it can output.

00:02:34 ◼   ► And that is not possible with AVPlayer.

00:02:37 ◼   ► There's no mechanism to do that that I have found at all.

00:02:41 ◼   ► If anyone knows, I'm really curious,

00:02:42 ◼   ► but I have looked many times,

00:02:44 ◼   ► every time there's a new SDK I look,

00:02:46 ◼   ► I've asked Apple engineers at WWDC and in labs and via email, and it really does seem

00:02:51 ◼   ► like it is just not possible with AVPlayer.

00:02:54 ◼   ► So the way I do this is I actually just wrote the, I basically wrote my own AVPlayer.

00:02:58 ◼   ► It's not a small task, and I'm writing at the core audio and audio units levels, and

00:03:06 ◼   ► custom decoding of everything.

00:03:08 ◼   ► And when you go lower level to enable this, the easiest way to do it is something called

00:03:13 ◼   ► called the ext, geez, ext audio file API.

00:03:17 ◼   ► And that is, that only works for locally stored files,

00:03:22 ◼   ► it does not work for streams.

00:03:26 ◼   ► There's a separate API, the audio file stream API

00:03:28 ◼   ► that is lower level than that.

00:03:30 ◼   ► And that you can supply stream data to.

00:03:33 ◼   ► You still have to manage the fetching of the stream yourself

00:03:36 ◼   ► and you have to manage a lot yourself.

00:03:39 ◼   ► So the job of streaming was to convert

00:03:42 ◼   ► from that ext audio file API that only does local files

00:03:46 ◼   ► to the lower level audio file stream API.

00:03:49 ◼   ► And also then build out all the UI behind it,

00:03:52 ◼   ► build out the network fetcher,

00:03:54 ◼   ► the actual streamer that can stream things,

00:03:56 ◼   ► and of course make it all performant

00:03:58 ◼   ► and responsible of data usage and memory

00:04:01 ◼   ► and responsible of the CPU

00:04:04 ◼   ► and also handle a whole bunch of weird edge cases.

00:04:08 ◼   ► And I knew it was gonna be a big job going in,

00:04:11 ◼   ► but I didn't quite realize how big,

00:04:14 ◼   ► how many of those edge cases they would be,

00:04:16 ◼   ► how tricky some of them would be to solve.

00:04:18 ◼   ► And the biggest thing,

00:04:19 ◼   ► like when we're talking about unexpected complexity,

00:04:22 ◼   ► the biggest thing for me is unexpected time that it takes.

00:04:26 ◼   ► You know, this is why our business is historically so bad

00:04:29 ◼   ► at estimating time and ship dates.

00:04:32 ◼   ► And when you're independent,

00:04:33 ◼   ► I don't think we're any better at it.

00:04:34 ◼   ► You know, I think in many ways, it might be worse.

00:04:38 ◼   ► Do you find that you are good at estimating time?

00:04:41 ◼   ► I think the honest answer is that I've stopped estimating time, or even trying to.

00:04:48 ◼   ► I have found that it doesn't actually help.

00:04:52 ◼   ► I have this vague sense of like, sometimes I have, the only deadline I usually have is

00:04:56 ◼   ► a new device is being shipped, or a new OS is being shipped.

00:05:00 ◼   ► There's something specific that like, if I want to be there on day one of the iOS 9 launch,

00:05:05 ◼   ► then I need to have my app ready by the beginning of September.

00:05:10 ◼   ► Other than that, I try hard to not think too much about timelines because every time I've

00:05:16 ◼   ► tried to do that, it never works.

00:05:19 ◼   ► The most universal experience of software engineering is you think it's going to take

00:05:24 ◼   ► two weeks, it ends up taking four weeks.

00:05:27 ◼   ► And that's just the reality of like, you're all like, as you, I mean, I think it sounds

00:05:32 ◼   ► a little bit like you have the same experience.

00:05:33 ◼   ► Like you start off thinking, okay, maybe I can do this with these high level APIs.

00:05:38 ◼   ► like, you know, these things Apple has put together, they're tried and true, whatever.

00:05:43 ◼   ► And you start using it and it's like, okay, it does most of what I need, but it doesn't

00:05:47 ◼   ► do this one critical thing. And you start being like, how critical is that? Well, it's

00:05:52 ◼   ► probably pretty critical. And so you take your next step down, and then you take your

00:05:56 ◼   ► next step down, and then before long, you're just like re-implementing all kinds of crazy

00:06:00 ◼   ► stuff. And you have no way of knowing superficially what that's going to look, how many love

00:06:06 ◼   ► stacks deep, how much rewriting and rebuilding of stuff you're going to need to end up doing

00:06:11 ◼   ► to get what you're going to do.

00:06:12 ◼   ► And so at the end I was just like, I don't know, I'll just sort of hope for the best.

00:06:15 ◼   ► And like other than for iOS 9 things, for those I just tend to say like how much, like

00:06:20 ◼   ► I'll just work until I can't, I run out of time and just try and structure what I'm working

00:06:26 ◼   ► on such that I can like, you know, not be half finished, have like half the things fully

00:06:32 ◼   ► finished rather than all the things half finished.

00:06:35 ◼   ► But yeah, estimating time is a complete nightmare, I find.

00:06:38 ◼   ► - Yeah, totally.

00:06:40 ◼   ► And we're lucky that when we're working for ourselves

00:06:44 ◼   ► on our own products for the most part,

00:06:46 ◼   ► our time estimates aren't that important

00:06:47 ◼   ► 'cause for the most part,

00:06:49 ◼   ► only the users are demanding things, if anybody.

00:06:52 ◼   ► It's not like, when you're a consultant,

00:06:55 ◼   ► the pressure is much stronger on you

00:06:56 ◼   ► to meet the client's deadlines.

00:06:58 ◼   ► Or when you're working for a company,

00:07:00 ◼   ► when you're working directly on your employer's product,

00:07:05 ◼   ► You have similar pressures from that.

00:07:07 ◼   ► But when you're a consultant, you have a whole different aspect of it, which is that

00:07:11 ◼   ► the client is often directly paying you for the time that you are taking.

00:07:15 ◼   ► And that, of course, makes it even more complex.

00:07:17 ◼   ► We are both lucky that we are not currently in that kind of role.

00:07:20 ◼   ► But certainly, I would say from just anecdotal evidence, it seems like the vast majority

00:07:26 ◼   ► of iOS developers are involved in some kind of contract work.

00:07:31 ◼   ► That seems to be the biggest chunk of the iOS work that's out there.

00:07:36 ◼   ► Time estimates are such a nightmare there.

00:07:39 ◼   ► We're both lucky that we can't really talk much about it, because I haven't done it in

00:07:43 ◼   ► forever, and even when I didn't, I didn't do very much of it.

00:07:46 ◼   ► It's been a while since you've done consulting as well, right?

00:07:48 ◼   ► Yeah.

00:07:49 ◼   ► I mean, the last time I did regular consulting was probably four or five years ago, something

00:07:54 ◼   ► like that.

00:07:55 ◼   ► I started out as a consultant.

00:07:56 ◼   ► That was what I was doing initially, but it's been a long time since that made up any kind

00:08:01 ◼   ► kind of significant portion of my business.

00:08:03 ◼   ► And I learned very quickly when I was doing consulting,

00:08:06 ◼   ► though, to not commit to time estimates as much as I could,

00:08:11 ◼   ► because you're always gonna run into that.

00:08:13 ◼   ► There's always some kind of complexity

00:08:15 ◼   ► that you're not expecting.

00:08:16 ◼   ► And if you don't plan for it,

00:08:18 ◼   ► if you don't look at a problem and say,

00:08:20 ◼   ► it's like the old rule of whatever you think

00:08:22 ◼   ► it's gonna take, multiply it by three.

00:08:24 ◼   ► - Yeah, at least.

00:08:25 ◼   ► - You're probably well off, but yeah, I try as best,

00:08:30 ◼   ► Like the first rule, you definitely have that experience when you start off consulting where

00:08:33 ◼   ► you start to say like, "Oh, it's going to take such, you know, oh, that'll only be like

00:08:36 ◼   ► a week's worth of work."

00:08:38 ◼   ► And you commit to the client to do that and then it never ends up being a week's worth

00:08:41 ◼   ► of work.

00:08:42 ◼   ► It's just, that's just the reality.

00:08:43 ◼   ► I mean, even, and I mean, with streaming, I had so many problems with, I would, I started

00:08:49 ◼   ► it, I think three or four times.

00:08:52 ◼   ► And I would get down a certain path or try to try to approach it in a certain way and

00:08:57 ◼   ► and realize that it was just never gonna work.

00:09:00 ◼   ► I was never gonna be able to complete it

00:09:01 ◼   ► in the way I was doing it, or it just was not working,

00:09:05 ◼   ► or I had made too many mistakes,

00:09:07 ◼   ► or I had made too many subtle bugs

00:09:09 ◼   ► that I couldn't find at the very low levels.

00:09:11 ◼   ► And so I would just scrap it and start over again.

00:09:13 ◼   ► And when you're at a big time crunch,

00:09:17 ◼   ► you don't usually have the opportunity to do that.

00:09:19 ◼   ► And that can be a blessing and a curse.

00:09:21 ◼   ► If you do that too much,

00:09:23 ◼   ► then you're wasting a whole bunch of time,

00:09:25 ◼   ► and it's never gonna be perfect.

00:09:27 ◼   ► It's never going to be exactly the perfect design, exactly what you want.

00:09:30 ◼   ► And so if you have too many opportunities to cancel it and start over again, you're

00:09:33 ◼   ► just going to kill tons of time.

00:09:35 ◼   ► But if you don't have any opportunities to do that, which is often the case when you're

00:09:38 ◼   ► under time pressure from a client, say, I feel like it's hard to get good quality work

00:09:43 ◼   ► because you really do learn a lot if you try to do something, then get a chance to throw

00:09:48 ◼   ► it away and do it over again.

00:09:49 ◼   ► Yeah.

00:09:50 ◼   ► Because I think one thing that I was, my experience, like whenever I've encountered these kind

00:09:54 ◼   ► of problems, where I'm trying to deal with something that ends up being much more complicated

00:09:59 ◼   ► than I think. I start off with a mental model for the problem that is straightforward. For

00:10:04 ◼   ► example, the thing that I most recently ran into this with is I did a data syncing and

00:10:08 ◼   ► merging algorithm for Pedometer++. So it takes the steps from your watch and your phone and

00:10:15 ◼   ► gives you a unified view of them in semi-real time. At first I was like, "Oh, the mental

00:10:20 ◼   ► model for this is great. There's HealthKit, right? I can just use HealthKit. And you can't,

00:10:24 ◼   ► because that's not how HealthKit works, because it's not available on the watch in the way

00:10:28 ◼   ► that shows you phone data. So I very quickly discovered that my original mental model was

00:10:33 ◼   ► completely wrong. And so then, what my process tends to be, I have this really straightforward

00:10:38 ◼   ► mental model, it ends up not working, and then I end up going down these crazy paths

00:10:43 ◼   ► of incredibly complicated solutions. All of these really brittle and not really going

00:10:50 ◼   ► to work solutions, like all kinds of crazy algorithms and merging schemes and way of

00:10:53 ◼   ► throwing data back and forth. And you get to the end, and the times I'm most satisfied

00:10:59 ◼   ► with my work, though, are when I end up, after all of that, with a simple solution again

00:11:04 ◼   ► that is informed by all of those crazy paths down the way. Like, you can learn the lesson

00:11:10 ◼   ► and be like, "Okay, actually, I don't need to do all these crazy things. Now that I understand

00:11:15 ◼   ► the problem fully from top to bottom.

00:11:17 ◼   ► I can say, "Okay, I don't need to worry about the data in all these crazy ways.

00:11:22 ◼   ► I can just make all these nice simplifying assumptions about how it is because I've

00:11:28 ◼   ► tested and verified that those work.

00:11:30 ◼   ► And then I can build a solution that's almost as simple as the one I started off with wanting

00:11:35 ◼   ► to build and now that actually deals with all that complexity."

00:11:38 ◼   ► And whenever I'm able to go through that process, it starts simple, it gets horrifically

00:11:42 ◼   ► messy and then it gets simple again.

00:11:44 ◼   ► I know I'm doing it right.

00:11:47 ◼   ► If ever I end up having to stop in that middle state where it's still really messy, I know

00:11:52 ◼   ► it's going to break.

00:11:53 ◼   ► I know it's not going to work.

00:11:55 ◼   ► It needs to be something that I could just describe to myself in just a few sentences

00:12:00 ◼   ► or something's probably gone wrong.

00:12:04 ◼   ► What kind of system did you end up with to address the watchOS2 or watchkit2 sync problem?

00:12:11 ◼   ► Because as you are very well aware,

00:12:14 ◼   ► and as some of our audience might be,

00:12:18 ◼   ► with WatchKit 1 apps,

00:12:20 ◼   ► we're actually fairly straightforward and simple to make

00:12:22 ◼   ► because they couldn't do that much,

00:12:24 ◼   ► and because they were running on the phone

00:12:27 ◼   ► in the app sandbox, in the same sandbox as the iPhone app,

00:12:30 ◼   ► so that you could have a shared container,

00:12:32 ◼   ► and the iPhone app and the Watch app

00:12:35 ◼   ► could literally be reading and writing from the same data.

00:12:38 ◼   ► And that's how I structured the Overcast Watch app,

00:12:40 ◼   ► which is I just move the database file, the SQLite file,

00:12:44 ◼   ► into the app container, and then the communication

00:12:49 ◼   ► between the watch app and the phone app

00:12:51 ◼   ► is limited only to basically commands and status updates.

00:12:55 ◼   ► And all the data, the watch app reads the data

00:12:58 ◼   ► right off the database, and the phone just tells the watch

00:13:02 ◼   ► when it has changed, and the watch reads it.

00:13:04 ◼   ► So it's actually very, quite simple in that way,

00:13:08 ◼   ► 'cause there is no sync issue.

00:13:10 ◼   ► If the watch can't reach the phone at the given moment,

00:13:13 ◼   ► you just can't use the Overcast app,

00:13:15 ◼   ► 'cause it's just, all it is is a remote control

00:13:17 ◼   ► for the phone, so it's great.

00:13:19 ◼   ► With watchOS 2, that shared data container

00:13:23 ◼   ► no longer quite exists the same way, or at all, really.

00:13:26 ◼   ► - It doesn't exist at all, yeah.

00:13:27 ◼   ► - Right, so with watchOS 2, it is like,

00:13:30 ◼   ► you are running two separate apps,

00:13:32 ◼   ► and you need to figure out how they're going

00:13:34 ◼   ► to communicate with each other,

00:13:35 ◼   ► and there's methods to do the basics,

00:13:38 ◼   ► but there is no automatic magical syncing that happens.

00:13:41 ◼   ► So you have to deal with how these things

00:13:43 ◼   ► synchronize their data yourself.

00:13:46 ◼   ► That's one of the reasons why I haven't made

00:13:48 ◼   ► an overcast watchOS 2 app.

00:13:51 ◼   ► Why I just keep having the OS 1 app.

00:13:54 ◼   ► First of all because it works well enough.

00:13:57 ◼   ► WatchOS 2 improved watch kit reliability quite a bit

00:14:00 ◼   ► and so the app works well enough for the most part.

00:14:03 ◼   ► But also that it would be a major undertaking

00:14:06 ◼   ► as you discovered, you know, it's a major undertaking

00:14:09 ◼   ► to all of a sudden tackle these two,

00:14:11 ◼   ► to all of a sudden take something

00:14:12 ◼   ► that used to be a dumb remote and make it a full app

00:14:16 ◼   ► with its own local storage

00:14:17 ◼   ► and the problem of syncing both ways.

00:14:20 ◼   ► And so, you know, if you take the phone away

00:14:23 ◼   ► and you have the watch off by itself somewhere

00:14:24 ◼   ► playing podcasts somehow,

00:14:26 ◼   ► which is its own separate bag of hurt

00:14:28 ◼   ► of like, you know, how capable that is right now

00:14:30 ◼   ► in watchOS 2.0, but if you somehow do that,

00:14:34 ◼   ► then you have to, when upon reconnection to the phone,

00:14:38 ◼   ► you have to sync up that data again

00:14:39 ◼   ► and resolve possible conflicts.

00:14:41 ◼   ► And it's a very, it's a much larger problem

00:14:45 ◼   ► than it seems upon first glance.

00:14:47 ◼   ► Because while the APIs and everything are all very similar

00:14:50 ◼   ► to what they were in Watch Scale 1,

00:14:53 ◼   ► the data model is totally different.

00:14:55 ◼   ► And that's where most of the complexity I think would lie.

00:14:58 ◼   ► And I haven't yet decided that that's worth the investment.

00:15:01 ◼   ► Apps for the Apple Watch are a whole separate topic,

00:15:04 ◼   ► but I have found generally that I'm very scared

00:15:08 ◼   ► to tackle that.

00:15:09 ◼   ► This is one of the few times where I've spotted

00:15:10 ◼   ► that complexity ahead of time,

00:15:13 ◼   ► and possibly wisely, I don't know,

00:15:16 ◼   ► but possibly wisely I'm avoiding it.

00:15:18 ◼   ► - Yeah, 'cause I think what I've ended up having to do

00:15:21 ◼   ► in WatchOS 2 is essentially you have to build

00:15:25 ◼   ► two completely independent, can-run-on-their-own apps.

00:15:31 ◼   ► Like the watch app has to be able to run entirely on its own without being able to phone home

00:15:35 ◼   ► to the phone.

00:15:36 ◼   ► And the phone app obviously needs to be able to run without the watch, which isn't that

00:15:40 ◼   ► much of a stretch because that's what it does if you don't own a watch.

00:15:44 ◼   ► And then you have to do these funny things because the communication between the two

00:15:49 ◼   ► is not reliable, both in terms of sometimes the watch is just physically not close enough

00:15:56 ◼   ► to the phone in order to communicate.

00:15:59 ◼   ► And also, sometimes, just the APIs and things, I think it's because they're trying to do

00:16:04 ◼   ► so much about power management and sometimes certain types of messages only go through

00:16:10 ◼   ► if it's a good thing from an energy perspective to be able to do.

00:16:14 ◼   ► Like the radios are all turned up and fired up, so hey, let's go ahead and send it right

00:16:17 ◼   ► away.

00:16:18 ◼   ► If they're not, maybe we'll wait and see if we are going to fire up the radio anyway for

00:16:22 ◼   ► the next, in the next second.

00:16:24 ◼   ► If not, then maybe we'll do it.

00:16:25 ◼   ► And so you have to build the system that

00:16:30 ◼   ► makes no assumptions about being able to talk to the other part,

00:16:34 ◼   ► but still gives your user a consistent experience

00:16:37 ◼   ► across both.

00:16:38 ◼   ► And that's where, like for this project that I was working on,

00:16:41 ◼   ► became the really crazy thing.

00:16:42 ◼   ► Because a pedometer, in some ways, is very simple.

00:16:45 ◼   ► Like both devices have a motion processor on it

00:16:49 ◼   ► that tells you how many steps they took

00:16:51 ◼   ► in a particular time period.

00:16:52 ◼   ► Like that's all they do, and that's

00:16:54 ◼   ► most of what the app is doing.

00:16:56 ◼   ► And so I could run completely independently in terms of,

00:16:59 ◼   ► if I could just showed you the watch's steps on the watch app

00:17:03 ◼   ► and just showed you the phone steps on the phone app,

00:17:05 ◼   ► it'd be trivial.

00:17:06 ◼   ► But what I want to be able to do is obviously

00:17:08 ◼   ► merge those together and say, which

00:17:09 ◼   ► one is a better representation of what you're doing?

00:17:13 ◼   ► And that is where all the crazy complexity comes in.

00:17:16 ◼   ► Because I had a couple of things that I was like,

00:17:19 ◼   ► so many of conflict resolution strategies

00:17:21 ◼   ► don't really work in this scenario.

00:17:23 ◼   ► because I can't just do simple things like who updated most recently or things because

00:17:28 ◼   ► if I can ever, for example, make someone's step count go down, I guess I could, but conceptually

00:17:34 ◼   ► I've always had that be like a rule for the app that if I ever, even if it was like slightly

00:17:38 ◼   ► in error, if I ever showed you a step count, like for example, you hit your goal, right?

00:17:42 ◼   ► Like you got 10,000 steps, confetti comes down from the screen, hooray, you did it.

00:17:46 ◼   ► I don't want you to then like sync up with your watch and say like, oh, actually now

00:17:49 ◼   ► we think that you did 9,000 steps.

00:17:51 ◼   ► Like, that would be terrible, right?

00:17:53 ◼   ► You have to withdraw somebody's confetti.

00:17:55 ◼   ► Yeah, like, I'm sorry, your confetti now goes back up to the top of the screen and...

00:17:58 ◼   ► Animated in reverse.

00:17:59 ◼   ► I think that'd be awful, right?

00:18:02 ◼   ► Play like a sad trombone.

00:18:03 ◼   ► Yeah, and so you have to, like, a lot of my syncing stuff is about saying, like, what's

00:18:07 ◼   ► the biggest number of steps that you've shown the user for the particular time period that

00:18:12 ◼   ► we're looking at?

00:18:13 ◼   ► And syncing that up so that if I ever needed to show, sort of add extra steps, I know what

00:18:20 ◼   ► I need to add. And also I need to do that, but it's not just as simple as who's shown

00:18:26 ◼   ► more because the nature of these devices is which one of them is going to be showing you

00:18:31 ◼   ► a better view of the user's activity probably at any one time. When I'm walking and pushing

00:18:36 ◼   ► my daughter in a stroller, my phone in my pocket is going to be a better idea of how

00:18:40 ◼   ► many steps I am because my wrist is actually not moving. It's attached to a stroller pushing

00:18:44 ◼   ► it along. But when my phone is in a backpack, or my wife runs into this a lot, her phone

00:18:51 ◼   ► will be in her purse, but her watch is on her wrist, so then her watch is the better

00:18:55 ◼   ► indicator of steps. So it's like having to work out a scheme that is in real time

00:19:00 ◼   ► dynamically working out which one, based on the information we have, which we may or may

00:19:04 ◼   ► not have all the time, looking at the two data streams and merging them together. That's

00:19:12 ◼   ► that's the minimum threshold for having this thing work,

00:19:16 ◼   ► is you have to have built a system that can do all that.

00:19:19 ◼   ► And that was the complexity for me that just became like,

00:19:21 ◼   ► wow, this is a whole bigger thing

00:19:23 ◼   ► than I ever imagined it would be.

00:19:25 ◼   ► - Yeah, and that's like one of the things

00:19:29 ◼   ► that makes these unexpected complexities

00:19:32 ◼   ► so frustrating or so powerful,

00:19:34 ◼   ► is that, again, they're unexpected.

00:19:38 ◼   ► You have no idea when in a project

00:19:41 ◼   ► you're going to hit something like this

00:19:42 ◼   ► until you hit it.

00:19:43 ◼   ► And so like, you know, the more experience you get

00:19:47 ◼   ► as a developer or just as a person who lives in life,

00:19:52 ◼   ► the more experience you get,

00:19:53 ◼   ► it becomes a little bit more likely

00:19:55 ◼   ► that you'll spot them ahead of time

00:19:57 ◼   ► and be able to possibly either avoid them

00:19:59 ◼   ► or at least like budget for them.

00:20:01 ◼   ► But there's still always like stuff that you hit

00:20:04 ◼   ► that's unexpected and it can totally throw a project off.

00:20:07 ◼   ► So like, you know, one thing that I do to manage this

00:20:11 ◼   ► is if I can tell something's going to be a big hairy mess,

00:20:15 ◼   ► a lot of times I'll just cut the feature.

00:20:17 ◼   ► I'll just avoid it.

00:20:19 ◼   ► I'll say, "Sorry, that's not worth doing."

00:20:21 ◼   ► With Overcast, I knew streaming was gonna be a big deal,

00:20:24 ◼   ► so for version 1.0, I just didn't include it.

00:20:27 ◼   ► I couldn't ship it in time.

00:20:30 ◼   ► I estimated that it would be a better idea

00:20:32 ◼   ► to ship the app without streaming

00:20:34 ◼   ► than to wait another six months to a year to add streaming.

00:20:39 ◼   ► That turned out to be correct.

00:20:41 ◼   ► And in the case of my watch app now,

00:20:43 ◼   ► where I have the watchOS 1 app,

00:20:45 ◼   ► the watchOS 2 app would be a ton of work

00:20:48 ◼   ► and would be different and might not be better.

00:20:51 ◼   ► It might just be different.

00:20:53 ◼   ► I've calculated so far that at the moment

00:20:56 ◼   ► it isn't worth investing the time in that.

00:20:57 ◼   ► And it might never be, I don't know.

00:20:59 ◼   ► But certainly right now it isn't worth

00:21:01 ◼   ► investing the time in that,

00:21:02 ◼   ► when I could instead be doing something

00:21:04 ◼   ► with the same amount of time,

00:21:05 ◼   ► I could instead be doing something like the Apple TV app

00:21:08 ◼   ► or a major new feature, or even a brand new app

00:21:13 ◼   ► if I wanted to.

00:21:14 ◼   ► You have to really decide what's worth sinking

00:21:17 ◼   ► all this time into, and sometimes it isn't

00:21:20 ◼   ► the big hairy problem in front of you.

00:21:21 ◼   ► Sometimes that problem is best off just staying unsolved.

00:21:26 ◼   ► That feature might be better off staying unimplemented,

00:21:29 ◼   ► or that app might be better off not being made,

00:21:32 ◼   ► and you might be better off spending that time

00:21:34 ◼   ► in other ways, because you can do a lot

00:21:37 ◼   ► of low-hanging fruit in other parts of your app

00:21:40 ◼   ► or as other apps in the same amount of time

00:21:42 ◼   ► that you could tackle one big hairy problem in one app,

00:21:45 ◼   ► and it might not be worth it.

00:21:46 ◼   ► Our sponsor this week is Need.

00:21:48 ◼   ► Go to neededition.com.

00:21:50 ◼   ► Need is a curated retailer and lifestyle publication

00:21:53 ◼   ► for the modern gentleman.

00:21:54 ◼   ► Each week, Need launches new collections

00:21:56 ◼   ► of exclusive clothing, literature, furniture, and more.

00:22:00 ◼   ► Now earlier this month,

00:22:01 ◼   ► coinciding with the company's second birthday,

00:22:03 ◼   ► Need launched an all-new site expanding its availability

00:22:05 ◼   ► into 43 countries in Europe, South America, Asia, and the Middle East. Shipping is flat

00:22:11 ◼   ► rate for all international orders and free for all orders to the US, Canada, and Mexico,

00:22:16 ◼   ► and all returns are free no matter where you are all over the world.

00:22:20 ◼   ► This month, Neve has launched three collections, including their Holiday Gift Guide, featuring

00:22:24 ◼   ► dozens of gift ideas for the holidays, many of which are under $40. Neve is designed to

00:22:29 ◼   ► be simple, straightforward, and uncomplicated. They include prepaid return labels with all

00:22:34 ◼   ► They offer 24/7 support and there aren't any subscriptions or stylists or other gimmicks

00:22:39 ◼   ► to deal with.

00:22:40 ◼   ► It's just a straightforward retailer.

00:22:42 ◼   ► Simply come along to NeedEdition.com, peruse the latest collections and shop.

00:22:46 ◼   ► Or don't, your choice.

00:22:48 ◼   ► For listeners of this show, use coupon code radar for 20% off.

00:22:52 ◼   ► That is coupon code radar at NeedEdition.com for 20% off anything.

00:22:56 ◼   ► Thank you so much to our friends at Need for sponsoring Under the Radar.

00:23:00 ◼   ► Getting back to one thing that you just said that I think is probably a good place to be

00:23:05 ◼   ► winding down this discussion is, I think the thing that I've learned most, like this

00:23:09 ◼   ► is the battle-hardened wisdom of dealing with complexity, is making sure that I'm solving

00:23:18 ◼   ► the problem for the right reason.

00:23:21 ◼   ► I think most of us who get into software engineering, or whatever you want to call what we do, we

00:23:26 ◼   ► We like solving problems, and the problems that we like solving most are the most difficult,

00:23:31 ◼   ► interesting, tricky problems. Like, that is, I think I'm in my element the most, I'm the

00:23:37 ◼   ► most engaged, I'm the most excited about what I'm doing when I'm solving a really interesting

00:23:41 ◼   ► problem. Like, if I'm just writing boilerplate code, that's not nearly as exciting and something

00:23:47 ◼   ► I want to really, like, wrap my arms around and get, you know, sort of get up in the morning

00:23:50 ◼   ► to dive into. And so the thing that I, though, then always have to keep in the back of my

00:23:54 ◼   ► my mind when I find a problem like this, when I'm working along on a task and I find that

00:23:58 ◼   ► there's this big unexpected bump in complexity. Like you were just saying, sometimes you're

00:24:02 ◼   ► just worth not doing it. The thing I know I've had to teach myself some discipline with

00:24:07 ◼   ► is that when I see a problem like that, when I seize an opportunity to really go crazy

00:24:11 ◼   ► spelunking, dive deep down into some crazy low APIs and build this crazy cool clever

00:24:17 ◼   ► system, make sure that I'm not just doing it because it would be intellectually fun

00:24:23 ◼   ► and interesting. Sometimes there's a place for that, but I've definitely caught myself

00:24:28 ◼   ► sometimes just building things because I think it would be fun and it wouldn't actually make

00:24:32 ◼   ► my customers happier. It wouldn't actually make my products better. It would just be

00:24:37 ◼   ► fun for me to do. I end up just going down these crazy rabbit holes and doing all these

00:24:41 ◼   ► things and often the reality is it isn't actually as fun as I thought it was going

00:24:46 ◼   ► to be. It's fun for the first few hours, first few days, and then you're just like

00:24:51 ◼   ► It's like you're in this point where,

00:24:53 ◼   ► like I always want to avoid where,

00:24:54 ◼   ► it's like I started working on my car

00:24:56 ◼   ► and I've taken it completely apart.

00:24:57 ◼   ► And then I come to discover

00:25:00 ◼   ► the fun part was taking it apart.

00:25:02 ◼   ► The fun part is not putting it back together.

00:25:05 ◼   ► And now I don't have a car anymore.

00:25:06 ◼   ► That's the part where I have to catch myself

00:25:09 ◼   ► and be like, am I really doing this for the right reason?

00:25:11 ◼   ► Is this complexity worth the effort?

00:25:14 ◼   ► And having that kind of taking a step back from your work

00:25:18 ◼   ► is a sort of a habit and a discipline

00:25:20 ◼   ► I've found has been very helpful in me making sure that I'm staying on track.

00:25:23 ◼   ► And especially being like a one-man shop, like there's no one, I don't have a supervisor

00:25:27 ◼   ► being like, "Huh, yeah, yeah, we're just going to cut that feature."

00:25:30 ◼   ► That's too much.

00:25:31 ◼   ► It has to be something that I have the ability to not just get totally sucked down these

00:25:36 ◼   ► rabbit holes and say like, "Yes, of course, I need to do this thing because it's really

00:25:40 ◼   ► interesting and really hard."

00:25:42 ◼   ► Like, I need to be able to say like, "Will my customers notice if I do this?"

00:25:47 ◼   ► And if they won't, like am I just doing this for myself?

00:25:50 ◼   ► And if I am, like that's kinda sometimes fun,

00:25:55 ◼   ► but that's really not a way that you're gonna be able

00:25:57 ◼   ► to reliably ship quality products.

00:25:59 ◼   ► You're just gonna end up with a lot of kind of like

00:26:01 ◼   ► crazy half-built things that probably will end up

00:26:04 ◼   ► making your app more complicated and harder to use

00:26:07 ◼   ► for your customers in the end.

00:26:08 ◼   ► - Yeah, that's a fantastic way to look at it.

00:26:11 ◼   ► And we all face that dilemma because I mean look,

00:26:14 ◼   ► we're all geeks, we all like interesting things,

00:26:17 ◼   ► We all like keeping our brains interested

00:26:18 ◼   ► and trying new things and approaching problems in new ways

00:26:21 ◼   ► or doing things in better ways

00:26:22 ◼   ► than what we did in the past.

00:26:24 ◼   ► It's not that you should never do that,

00:26:25 ◼   ► but that there's a balance to be struck

00:26:27 ◼   ► and that doing that has significant time costs

00:26:31 ◼   ► that may not be worth it.

00:26:32 ◼   ► And doing a major rewrite of something that already works

00:26:36 ◼   ► might also introduce brand new bugs,

00:26:37 ◼   ► in fact, it almost certainly would,

00:26:39 ◼   ► brand new bugs that you didn't have before

00:26:42 ◼   ► then you'd have to go fixing that.

00:26:44 ◼   ► And so it's just a balance.

00:26:45 ◼   ► You need to keep yourself happy and learning

00:26:48 ◼   ► and intellectually satisfied.

00:26:50 ◼   ► But you also need to ship products

00:26:52 ◼   ► and it's important to find that balance.

00:26:55 ◼   ► And that's something that I think mostly just comes

00:26:57 ◼   ► with experience and wisdom over time.

00:27:00 ◼   ► It's not that you can never have fun,

00:27:02 ◼   ► but that you should know when to have fun

00:27:05 ◼   ► and when it's no longer fun and you need to get back to work.

00:27:09 ◼   ► - Yeah, and I think the best lesson I've had about that

00:27:12 ◼   ► is it's trying to have customer-focused development

00:27:15 ◼   ► that way. It's like the time to do that, the time to solve interesting problems, is to

00:27:19 ◼   ► find features or find apps or find ideas that solving a difficult problem will make a customer's

00:27:27 ◼   ► life better. And like the intersection of those two things, like a really interesting

00:27:31 ◼   ► technical problem and a real honest need or pain point in someone's life, like that's

00:27:37 ◼   ► where really interesting and fun work happens. And if you can't sort of honestly say that

00:27:43 ◼   ► that it's both interesting to build

00:27:45 ◼   ► and useful for a customer,

00:27:47 ◼   ► then you're probably not in a place

00:27:49 ◼   ► that it's actually worth building.

00:27:51 ◼   ► - All right, thanks a lot for listening, everybody.

00:27:53 ◼   ► Please tell your friends about the show,

00:27:55 ◼   ► help us spread the word,

00:27:56 ◼   ► recommend us on Overcast,

00:27:58 ◼   ► and we'll see you next week.