154: Knowingly Shipping Bugs
00:00:00
◼
►
- Welcome to Under the Radar,
00:00:01
◼
►
a show about independent iOS app development.
00:00:04
◼
►
I'm Marco Arment.
00:00:05
◼
►
- And I'm David Smith.
00:00:06
◼
►
Under the Radar is never longer than 30 minutes,
00:00:08
◼
►
so let's get started.
00:00:10
◼
►
- So I find myself in the probably not unique position
00:00:13
◼
►
this week of shipping as the app store version of Overcast,
00:00:17
◼
►
a build that I know still has bugs.
00:00:21
◼
►
- Ah, bugs in your software.
00:00:23
◼
►
That's unheard of.
00:00:24
◼
►
- I am knowingly shipping bugs.
00:00:28
◼
►
I have a feeling this is not that uncommon,
00:00:32
◼
►
or at least if we're honest with ourselves.
00:00:35
◼
►
A lot of times we try to tell ourselves,
00:00:38
◼
►
okay, I'm pretty sure this one works.
00:00:40
◼
►
This build has no bugs.
00:00:42
◼
►
I am shipping this knowing it's totally fine.
00:00:44
◼
►
But of course that's never really the case.
00:00:47
◼
►
What I want to talk about this week is
00:00:49
◼
►
what has gotten me into this position
00:00:51
◼
►
and kind of why we all do this,
00:00:54
◼
►
and when it might be okay to ship bugs.
00:00:59
◼
►
So I think first of all it's worth evaluating
00:01:03
◼
►
how good we are at shipping software
00:01:06
◼
►
that is truly bug free.
00:01:09
◼
►
And I don't mean me and you,
00:01:10
◼
►
I mean this whole business.
00:01:13
◼
►
As far as I can tell it's impossible, yes.
00:01:16
◼
►
Like as soon as your code does anything,
00:01:19
◼
►
or interacts with any other code
00:01:21
◼
►
like the operating system and its frameworks,
00:01:24
◼
►
you introduce the potential for bugs.
00:01:26
◼
►
And the more, and it doesn't really take,
00:01:29
◼
►
like you know the more complex your app
00:01:30
◼
►
the more bug potential there is.
00:01:32
◼
►
But it doesn't really take a very seemingly complex app
00:01:37
◼
►
like from the user point of view
00:01:39
◼
►
in order to have enough complexity behind the scenes
00:01:41
◼
►
where lots of bugs can possibly exist.
00:01:45
◼
►
And so I would say, I mean I've,
00:01:49
◼
►
as far as I know in my entire career so far
00:01:53
◼
►
I have never written or shipped completely bug free code.
00:01:57
◼
►
Do you know of any that you have?
00:01:59
◼
►
- No, I think bugs are one of those things
00:02:03
◼
►
that's like Zeno's paradox or asymptotic growth
00:02:07
◼
►
where it's like all you can do is approach zero,
00:02:10
◼
►
but it is impossible to actually reach zero.
00:02:14
◼
►
Like and really, and I think probably
00:02:16
◼
►
the more sobering thing that sort of keeps you up at night
00:02:19
◼
►
if you think about it too hard,
00:02:21
◼
►
is the nature of you can be in a position
00:02:24
◼
►
that you have no known bugs,
00:02:26
◼
►
but that is very different
00:02:28
◼
►
than there not actually being bugs.
00:02:31
◼
►
Like the nature of discovering something
00:02:33
◼
►
is like half the problem most of the time.
00:02:36
◼
►
It's like it isn't necessarily that,
00:02:39
◼
►
like bugs are just like this obvious thing
00:02:43
◼
►
that Xcode tells you about
00:02:44
◼
►
or there's some obvious glaring issue.
00:02:48
◼
►
Often the hardest bugs to track down,
00:02:50
◼
►
or the worst bugs in many ways,
00:02:51
◼
►
are the ones that don't like stick their head,
00:02:54
◼
►
you know, stick their head up and raise their hand
00:02:56
◼
►
and be like, "Here I am."
00:02:57
◼
►
You know, like when you do this set of three actions,
00:02:59
◼
►
this bad thing happens.
00:03:00
◼
►
Like those are the best.
00:03:02
◼
►
The ones that you're never gonna get,
00:03:04
◼
►
like there's the ones that are just lingering there.
00:03:06
◼
►
And honestly, sometimes I think there's a quite,
00:03:08
◼
►
like the tricky thing too is that a bug,
00:03:11
◼
►
like what is a bug?
00:03:11
◼
►
Like in some ways it is when the application behaves
00:03:15
◼
►
counter to your intentions maybe.
00:03:18
◼
►
But the reality is our intentions
00:03:20
◼
►
are never fully specified.
00:03:22
◼
►
Like there's gonna be cases that we haven't accounted for
00:03:24
◼
►
because we couldn't have accounted for them.
00:03:27
◼
►
Or it's dealing with a new situation,
00:03:29
◼
►
a new device, a new operating system,
00:03:32
◼
►
a new user's set of data, a new bit of user input.
00:03:35
◼
►
Like there are so many things
00:03:37
◼
►
that are strictly impossible to deal with
00:03:40
◼
►
that I think it is absolutely impossible
00:03:42
◼
►
to ever ship bug-free code.
00:03:44
◼
►
All we can do is hope to sort of approach it over time
00:03:49
◼
►
and hopefully have like the,
00:03:51
◼
►
I'm thinking back to my sort of corporate days
00:03:54
◼
►
where you had like the bug burn down chart,
00:03:56
◼
►
where it's like you have this graph of like known bugs
00:03:59
◼
►
and it's like the goal is that that, you know,
00:04:01
◼
►
that the slope of that line is going down rather than up
00:04:05
◼
►
as you approach shipping.
00:04:06
◼
►
But it's definitely something
00:04:08
◼
►
that never actually hits zero.
00:04:09
◼
►
It just sort of hits a point where it becomes stable
00:04:12
◼
►
or the nature of the bugs is such
00:04:14
◼
►
that they're not worth fixing
00:04:16
◼
►
or it's an issue that would require way more work
00:04:18
◼
►
than is actually possible, like is reasonable to do
00:04:20
◼
►
or is possible to do.
00:04:22
◼
►
Or in the process of fixing it,
00:04:24
◼
►
you would actually introduce more bugs
00:04:25
◼
►
and so it's better to just leave the bug you know
00:04:27
◼
►
than the one that you don't.
00:04:28
◼
►
But you'll never get to zero, sadly.
00:04:31
◼
►
- It's such a wonderful goal to think like,
00:04:34
◼
►
I'm gonna ship a bug-free version.
00:04:36
◼
►
I'm gonna track down every single bug,
00:04:38
◼
►
every single crash, everything.
00:04:39
◼
►
I'm gonna track this all down
00:04:41
◼
►
and I'm gonna ship a perfect version.
00:04:43
◼
►
And of course the reality is like,
00:04:45
◼
►
bug-free is, as you said, it's an impossible goal to reach.
00:04:51
◼
►
All you can really hope for is relatively few bugs.
00:04:56
◼
►
Or like, typical usage of the app won't hit bugs
00:05:00
◼
►
as far as you know that would be bad enough
00:05:04
◼
►
that people would actually notice and care.
00:05:07
◼
►
- And those are all a bunch of escape clauses.
00:05:09
◼
►
Like every one of those, like every one of those phrases
00:05:12
◼
►
there is like, that can let in huge classes of bugs
00:05:15
◼
►
and have it be pretty much okay.
00:05:17
◼
►
Because the reality is, bug-free code,
00:05:21
◼
►
which is again impossible, but like trying to minimize bugs
00:05:26
◼
►
has severely diminishing returns.
00:05:29
◼
►
Because you will never complete it,
00:05:32
◼
►
you have to draw a line somewhere of like,
00:05:34
◼
►
how far am I willing to go?
00:05:36
◼
►
Like how much time am I willing to spend fixing bugs?
00:05:38
◼
►
And therefore, how much am I willing to not work on features,
00:05:43
◼
►
not ship new versions of the app,
00:05:46
◼
►
basically like how much of the app development process
00:05:49
◼
►
am I willing to grind to a halt,
00:05:51
◼
►
and for how long in order to fix bugs?
00:05:55
◼
►
And because it's a complicated business,
00:05:58
◼
►
because there's multiple factors,
00:06:00
◼
►
because some of these factors matter more than others,
00:06:02
◼
►
that is not always an easy question to answer,
00:06:05
◼
►
and a lot of times the answer is not
00:06:07
◼
►
what you think it should be.
00:06:09
◼
►
Like ideally, like ideally a bug is a showstopper.
00:06:13
◼
►
Ideally, like if you're trying to make high quality stuff,
00:06:17
◼
►
the concept, the idea that something is wrong,
00:06:19
◼
►
and that you know it's wrong,
00:06:20
◼
►
and that you aren't fixing it,
00:06:22
◼
►
that is in theory like morally wrong.
00:06:26
◼
►
Like you shouldn't do it, you should drop everything
00:06:29
◼
►
and fix that bug before you do anything else.
00:06:31
◼
►
But the reality is, people have schedules,
00:06:34
◼
►
people have limited time.
00:06:36
◼
►
You know a lot of times if you're working for somebody else,
00:06:39
◼
►
the decision is not in your hands,
00:06:42
◼
►
and you just have a certain deadline you have to hit,
00:06:44
◼
►
and the bosses might even tell you,
00:06:47
◼
►
you can't spend time fixing that obscure bug
00:06:49
◼
►
you want to fix right now,
00:06:51
◼
►
you have to instead do this feature that we just sold,
00:06:53
◼
►
you know through the sales team to a customer,
00:06:55
◼
►
and you need to make this.
00:06:56
◼
►
Like there's all sorts of external pressures like that.
00:06:59
◼
►
You know we can even look at Apple software,
00:07:01
◼
►
and you know Apple software's full of bugs,
00:07:04
◼
►
and part of that's because they're a big company,
00:07:06
◼
►
they have lots of software, there's lots of things,
00:07:08
◼
►
and part of that's because Apple is a very
00:07:09
◼
►
schedule driven company, and they are driven,
00:07:13
◼
►
they prioritize the schedule above all else,
00:07:16
◼
►
and secondly they prioritize the marketing feature list,
00:07:20
◼
►
and so Apple, we hear a lot of times from people
00:07:24
◼
►
who work at Apple who like,
00:07:26
◼
►
there are certain like phases throughout the year
00:07:28
◼
►
that it's more or less pragmatic to try to fix bugs,
00:07:32
◼
►
and it's more or less likely to get any bug fixes done,
00:07:35
◼
►
because if they're in like super crunch time,
00:07:38
◼
►
you know working on whatever the new release is
00:07:40
◼
►
for WVDC or for the fall,
00:07:42
◼
►
and you try to report a bug that is really not
00:07:47
◼
►
that bad of a bug, like it's not affecting very many people,
00:07:50
◼
►
or it's not causing like data loss,
00:07:52
◼
►
or you know anything really serious,
00:07:54
◼
►
then they will prioritize that accordingly,
00:07:57
◼
►
and it typically won't get fixed,
00:07:59
◼
►
or at least it won't get fixed for a very long time,
00:08:03
◼
►
and so like there's, we can recognize in our own indie life,
00:08:07
◼
►
or corporate life if we're still corporate,
00:08:10
◼
►
we can recognize that we have those same patterns,
00:08:12
◼
►
we have those same you know prioritization cycles,
00:08:15
◼
►
where like sometimes you just gotta get a build out there,
00:08:18
◼
►
and so all this is building up to say,
00:08:21
◼
►
what I'm doing with Overcast this week and last week is,
00:08:25
◼
►
I've been hearing reports from people,
00:08:28
◼
►
and seeing reports in App Store Analytics,
00:08:30
◼
►
that the crash rate for Overcast 5
00:08:34
◼
►
was higher than I thought it should be,
00:08:36
◼
►
that like it was getting crashes,
00:08:39
◼
►
that some of which were things
00:08:41
◼
►
that I don't choose to worry about,
00:08:42
◼
►
so for instance if I go into the Xcode organizer,
00:08:45
◼
►
and I look at the crashes or the energy logs,
00:08:47
◼
►
I get tons of crashes in system frameworks for the watch app,
00:08:52
◼
►
and I couldn't possibly care less,
00:08:56
◼
►
because I know that debugging the watch app
00:09:00
◼
►
in crashes that are in system frameworks,
00:09:04
◼
►
is simply a lost cause, that's just a time vacuum,
00:09:07
◼
►
that I know if I actually try to do that,
00:09:10
◼
►
it's gonna be the biggest waste of my time ever,
00:09:13
◼
►
because chances are, because of the very little access
00:09:16
◼
►
we have on the watch as you know,
00:09:18
◼
►
chances are I probably can't fix those bugs,
00:09:22
◼
►
and even if I could, it would probably take me
00:09:24
◼
►
a very long time to actually find them.
00:09:27
◼
►
- Yeah, and many of them aren't things
00:09:29
◼
►
that users will ever see, because many of them
00:09:31
◼
►
are happening in the background.
00:09:32
◼
►
- Right, and so that's a big thing too,
00:09:34
◼
►
like what I'm finding is many of my crashes
00:09:37
◼
►
that are reported as crashes in the organizer,
00:09:40
◼
►
many of those are background crashes,
00:09:43
◼
►
and on the watch it's almost always like
00:09:44
◼
►
somehow in the background I've used my two seconds
00:09:47
◼
►
of CPU time that I have, and it's killing me,
00:09:50
◼
►
and if it's not in an obvious way, or in an obvious spot,
00:09:54
◼
►
that is simply not worth fixing for most of the time,
00:09:57
◼
►
and you can see, we aren't the only ones who think this way,
00:10:00
◼
►
you can see because if you open up the device panel
00:10:04
◼
►
in Xcode and you look at the device logs for like your phone
00:10:08
◼
►
it'll show you logs for all the apps in your phone
00:10:11
◼
►
that have crashed or been terminated for resource usage,
00:10:13
◼
►
and you can see like big popular apps,
00:10:16
◼
►
almost every app you use there's gonna be
00:10:18
◼
►
a crash log in that list, and sometimes apps crash
00:10:22
◼
►
all the time in the background and you don't even know,
00:10:25
◼
►
because the way iOS backgrounding works,
00:10:28
◼
►
like if an app crashes in the background,
00:10:29
◼
►
the only way you could notice is if next time you launch it
00:10:32
◼
►
it takes a little bit longer to launch
00:10:33
◼
►
'cause it's launching from scratch
00:10:34
◼
►
instead of being resumed from suspension,
00:10:36
◼
►
but even that is like not uncommon in iOS
00:10:39
◼
►
for lots of other reasons, so it's really,
00:10:42
◼
►
it's kind of crashing in the background
00:10:44
◼
►
as long as the app wasn't in use at the time,
00:10:47
◼
►
like if it wasn't like playing audio,
00:10:48
◼
►
if it was just in the background and it crashes,
00:10:51
◼
►
that's not that big of a deal,
00:10:53
◼
►
and so that should be a very low priority bug.
00:10:56
◼
►
Similarly, like if, like a lot of the bugs I get are,
00:11:00
◼
►
or like really weird edge case things,
00:11:03
◼
►
like a crash in core text or in JavaScript core
00:11:06
◼
►
from one of my web views, and I'm like,
00:11:08
◼
►
you know, there's only so much I can do about that,
00:11:10
◼
►
like especially when it's not happening on a lot of devices,
00:11:13
◼
►
like there's just not much that I really should do
00:11:16
◼
►
about that, that's probably a system framework,
00:11:18
◼
►
Apple has these logs too, and Apple probably looks at them
00:11:22
◼
►
and they can see all the crash logs for all the apps
00:11:25
◼
►
that go through their system, and they, I'm sure,
00:11:27
◼
►
I haven't heard this for sure, but I would assume
00:11:30
◼
►
that they have processes where they can analyze those
00:11:32
◼
►
en masse and kind of triangulate like which system frameworks
00:11:35
◼
►
might have bugs in them and which APIs might be causing bugs
00:11:38
◼
►
and they probably look at that when they're figuring out
00:11:40
◼
►
how to prioritize their own efforts.
00:11:42
◼
►
So some of these things, I look at,
00:11:43
◼
►
and it's like, you know, if it's some big crash
00:11:45
◼
►
deep in some system thing, a part of a web view,
00:11:47
◼
►
I'm like, you know what, that's not mine to fix,
00:11:49
◼
►
that's Apple's to fix, and you know, it might be my problem,
00:11:54
◼
►
you know, it isn't my fault, it might be my problem,
00:11:57
◼
►
but only if it's affecting a large number of people,
00:11:59
◼
►
and in that case it isn't, right?
00:12:01
◼
►
And so what I look for when I'm trying to see what I can fix
00:12:05
◼
►
is I look for like a crash that is in my code,
00:12:09
◼
►
like ideally where like the stack trace includes
00:12:12
◼
►
something in Overcast and not all system frameworks,
00:12:15
◼
►
which is a little tricky sometimes, but you know,
00:12:17
◼
►
it should include my code, it should be in the phone app,
00:12:20
◼
►
I don't care about the watch app unless it's something
00:12:22
◼
►
really obvious, and it should be something that is
00:12:26
◼
►
affecting a large number of devices.
00:12:27
◼
►
The great thing about the Xcode Organizer thing
00:12:29
◼
►
is that it tells you this.
00:12:30
◼
►
And also yes, I'm fully aware of other crash reporting tools
00:12:33
◼
►
and things like that, you know, they have very similar
00:12:35
◼
►
advantages too, I guess like using the Apple one
00:12:37
◼
►
to keep things simple and keep privacy policies simple
00:12:40
◼
►
and to be able to catch things like the energy termination
00:12:42
◼
►
logs that the other ones usually don't get.
00:12:45
◼
►
Anyway, so there's, every time I look at the crash list,
00:12:49
◼
►
there's like 50 different crashes in there.
00:12:51
◼
►
Like not 50 devices, 50 different identified crash sources,
00:12:55
◼
►
but most of them are like, you know, five devices,
00:12:59
◼
►
10 devices, it's like, you know,
00:13:00
◼
►
there's only so much I can do.
00:13:03
◼
►
- Yeah, and I think I have the same experience
00:13:05
◼
►
where I look at a lot of my apps.
00:13:06
◼
►
Like there's always a lot of things in that list.
00:13:09
◼
►
I mean, I do a lot of watch work, and so like the first 40
00:13:12
◼
►
of my crashes are just, oh what is it, SP remote,
00:13:16
◼
►
I think is the name of it, what I like--
00:13:17
◼
►
- PK service run, I believe is the one I see a lot.
00:13:19
◼
►
- PK service run, there's a couple of these that are just
00:13:20
◼
►
like these watch dog processes or these watch things
00:13:24
◼
►
that you just kind of look at and you're like, yeah,
00:13:26
◼
►
I've even asked the WDC, like I went to the labs
00:13:29
◼
►
and talked to some watch kit people, like is this something
00:13:31
◼
►
I should be worrying about?
00:13:33
◼
►
And I think their answer was usually it's like,
00:13:35
◼
►
it's, and honestly I kind of wish that this was better
00:13:38
◼
►
communicated in Xcode is that there's kind of a different
00:13:41
◼
►
concept between a crash and a resource limitation
00:13:46
◼
►
or a system killing you situation, because sometimes
00:13:53
◼
►
the system killing you is a result of your bad behavior.
00:13:58
◼
►
But sometimes it isn't, and especially when you look
00:14:02
◼
►
at the watch logs, sometimes the most comedic ones
00:14:05
◼
►
are like you're being killed because the CPU is overused,
00:14:08
◼
►
your contribution to that was like 1%.
00:14:11
◼
►
And it's like, okay, so I didn't really, like something else
00:14:15
◼
►
on the system is using a lot of CPU and the OS is going
00:14:19
◼
►
around and just killing everything off to try and preserve
00:14:23
◼
►
battery or whatever it's trying to do.
00:14:25
◼
►
It's like, okay, well that's not really my problem.
00:14:28
◼
►
It's like I want to look for the ones, yeah, I want to look
00:14:29
◼
►
for the bugs that people are going to notice that are
00:14:31
◼
►
actually causing issues for people that are things
00:14:35
◼
►
that I can apply myself to and that my efforts are going
00:14:40
◼
►
to yield actual tangible, useful results.
00:14:44
◼
►
'Cause I think it's easy to, and I think honestly
00:14:47
◼
►
the hardest thing is the more esoteric the bug is,
00:14:52
◼
►
the longer it is going to take and the length and complexity
00:14:57
◼
►
of debugging is absolutely exponential.
00:15:01
◼
►
There's certain kinds of bugs, like the big obvious,
00:15:04
◼
►
you hit a button and the app crashes.
00:15:07
◼
►
Like, okay, some things, like I'm putting a nil value
00:15:12
◼
►
into a dictionary or I'm de-referencing something
00:15:17
◼
►
that I shouldn't de-reference, like there's some very obvious
00:15:20
◼
►
like issue going on.
00:15:21
◼
►
Happens every time I hit this button, it crashes.
00:15:23
◼
►
Like absolutely fix those.
00:15:24
◼
►
And then like it just, from there it just grows,
00:15:27
◼
►
just impossibly for like in this one situation,
00:15:31
◼
►
for this one device, in this particular like configuration
00:15:34
◼
►
when other apps are running at the same time
00:15:36
◼
►
and the moon is in a particular place
00:15:37
◼
►
and the sun is shining, then this bug will happen.
00:15:42
◼
►
And like at a certain point, like those are just like,
00:15:45
◼
►
okay, I could spend months trying to track that down,
00:15:48
◼
►
but I'll just ignore that and I will just sort of like
00:15:51
◼
►
mark it as resolved in my mind and until such time
00:15:54
◼
►
as it appears to actually hurt something,
00:15:56
◼
►
I will just ignore it.
00:15:57
◼
►
- We are brought to you this week by Linode.
00:16:00
◼
►
Linode gives you a suite of powerful hosting options
00:16:03
◼
►
with prices starting at just $5 a month.
00:16:06
◼
►
You can be up and running with your own virtual server
00:16:08
◼
►
in the Linode cloud in under a minute.
00:16:10
◼
►
Linode has hundreds of thousands of customers,
00:16:13
◼
►
including David and me.
00:16:14
◼
►
And we are all serviced by their friendly 24/7 support team.
00:16:18
◼
►
You can email them, call them, or even chat over IRC
00:16:21
◼
►
in the Linode community.
00:16:22
◼
►
They know how important it is to get the help you want.
00:16:25
◼
►
And they also have a suite of amazing guides
00:16:27
◼
►
and support documentation to give you a reference
00:16:30
◼
►
when you need it.
00:16:30
◼
►
And this is actually really nice.
00:16:31
◼
►
Even if you aren't a Linode customer, you can view these.
00:16:33
◼
►
They're public documents on the web.
00:16:35
◼
►
And if you search for help on running a Linux server,
00:16:37
◼
►
you will often find a Linode support document
00:16:40
◼
►
just as the general help document
00:16:41
◼
►
for running that kind of server.
00:16:43
◼
►
They're that good.
00:16:44
◼
►
And it's really way easier than you think
00:16:46
◼
►
to get these up and running.
00:16:47
◼
►
They also have a very intuitive control panel
00:16:49
◼
►
to allow you to deploy, boot, resize, snapshot,
00:16:52
◼
►
or clone your virtual servers in just a few clicks.
00:16:55
◼
►
And they have amazing security features too,
00:16:58
◼
►
such as two-factor authentication to keep you safe.
00:17:01
◼
►
So Linode is great for things like database hosting,
00:17:04
◼
►
mail servers, operating a VPN, Docker containers,
00:17:07
◼
►
private Git servers, and so much more,
00:17:09
◼
►
all the way up to major web apps,
00:17:10
◼
►
like I'm running Overcast on it,
00:17:12
◼
►
Dave runs his backend stuff on it.
00:17:15
◼
►
Linode has fantastic pricing options available.
00:17:17
◼
►
Plans start at one gig of RAM for just $5 a month.
00:17:21
◼
►
And they offer high memory plans starting at 16 gigs of RAM.
00:17:24
◼
►
Listeners of this show can sign up at linode.com/radar
00:17:28
◼
►
that will support us,
00:17:29
◼
►
and you will get $20 towards any Linode plan.
00:17:32
◼
►
So on that one gig plan, that could be four months for free.
00:17:35
◼
►
And with a seven day money back guarantee,
00:17:37
◼
►
there's nothing to lose.
00:17:38
◼
►
So go to linode.com/radar to learn more, sign up,
00:17:42
◼
►
and take advantage of that $20 credit,
00:17:43
◼
►
or use promo code radar2018 at checkout.
00:17:47
◼
►
Thank you so much to Linode for supporting this show
00:17:50
◼
►
and Relay FM, and for keeping all of our stuff running.
00:17:53
◼
►
- So what bug did you have to ship this week?
00:17:55
◼
►
- So I'm shipping a bug that I'm getting reports
00:18:01
◼
►
from, you know, so as I said,
00:18:03
◼
►
I've been getting background crashes here and there.
00:18:05
◼
►
I've tried to alleviate those with previous updates
00:18:08
◼
►
'cause it seemed like it was usually for too much CPU usage
00:18:11
◼
►
over too short of a time.
00:18:13
◼
►
As far as I can tell, iOS does not limit CPU usage
00:18:16
◼
►
if you're in the foreground.
00:18:17
◼
►
But if you're in the background,
00:18:19
◼
►
you are limited to 80% sustained usage.
00:18:23
◼
►
And so they'll measure it about once a minute.
00:18:24
◼
►
Or if you are using multi-core,
00:18:26
◼
►
if you hit like, you know, if you max out all six cores,
00:18:30
◼
►
you can blow that limit in like 12 seconds.
00:18:32
◼
►
So it's surprising like how aggressive it is.
00:18:36
◼
►
But basically, if you're averaging 80% over a minute,
00:18:39
◼
►
you will get killed.
00:18:40
◼
►
And so the problem was, you know, as I mentioned earlier,
00:18:44
◼
►
like if that happens when the app is not running,
00:18:47
◼
►
it's not that I don't care,
00:18:48
◼
►
but it becomes a way lower priority, you know?
00:18:51
◼
►
Whereas if it's happening when somebody is playing audio,
00:18:55
◼
►
then it results in the audio stopping for them.
00:18:58
◼
►
And that's not good.
00:18:59
◼
►
That becomes a much bigger problem.
00:19:03
◼
►
Because this brings it from like an academic purity exercise
00:19:07
◼
►
to something that's actually negatively affecting users
00:19:09
◼
►
in a real way.
00:19:11
◼
►
So I've been trying to fix this.
00:19:12
◼
►
I've been doing tons of test flight builds
00:19:15
◼
►
over the last couple of weeks,
00:19:17
◼
►
trying really hard to track these down, fix them.
00:19:19
◼
►
And every build I set out includes in the release notes
00:19:22
◼
►
to the testers like,
00:19:23
◼
►
"All right, this time I think I really got it."
00:19:25
◼
►
And then it gets out there-- - For real this time.
00:19:27
◼
►
- Yeah, it's definitely the for realest time.
00:19:28
◼
►
And it gets out there and then I don't have it.
00:19:30
◼
►
Or what has fortunately happened is that over time,
00:19:35
◼
►
I have, the current test flight build,
00:19:38
◼
►
which I'm gonna submit to the App Store today,
00:19:40
◼
►
the current test flight build does crash way less
00:19:45
◼
►
than the current App Store version.
00:19:46
◼
►
So it's like, I have made a significant improvement here.
00:19:50
◼
►
The problem is not fixed,
00:19:52
◼
►
but now there's an external factor.
00:19:54
◼
►
Now we are a few days away
00:19:56
◼
►
from the App Store holiday shutdown.
00:20:00
◼
►
- And what happens with this,
00:20:02
◼
►
so if you're all new or if,
00:20:05
◼
►
for the people who listen to the show
00:20:06
◼
►
who aren't developers and for some reason
00:20:07
◼
►
you still tolerate all this,
00:20:10
◼
►
the App Store, the whole backend, the app review,
00:20:14
◼
►
everything, everything that's involved in
00:20:16
◼
►
updating your apps, getting apps reviewed,
00:20:18
◼
►
or changing your pricing to your apps or descriptions,
00:20:20
◼
►
any of that, that all shuts down for a week
00:20:23
◼
►
around Christmas every year.
00:20:26
◼
►
Or about a week, it depends, but usually it's about a week.
00:20:28
◼
►
And so this year the shutdown is from the 23rd,
00:20:32
◼
►
so it's basically like Christmas week,
00:20:34
◼
►
like 23rd to 29th I think.
00:20:37
◼
►
And so you can't update your app at all during that time,
00:20:40
◼
►
you can't change anything in the App Store during that time.
00:20:43
◼
►
So if you wanna get an update in anywhere near Christmas,
00:20:47
◼
►
you gotta do it basically now.
00:20:49
◼
►
And that's why if you look at your App Store app
00:20:51
◼
►
in the updates tab, there's probably a lot of updates
00:20:54
◼
►
in the last few days because we have to get it all in early.
00:20:57
◼
►
And the other problem is if you ship a major bug,
00:21:02
◼
►
like a crasher, you can't fix it for a week.
00:21:06
◼
►
That's really bad. (laughs)
00:21:09
◼
►
So if you have some kind of horrible bug
00:21:12
◼
►
that causes data loss or something,
00:21:14
◼
►
you're just stuck with that for that whole week.
00:21:17
◼
►
So what I'm trying to do is ship this version of Overcast
00:21:21
◼
►
now, which is currently about four days
00:21:25
◼
►
before the shutdown, five days.
00:21:27
◼
►
That way I figure App Review is probably a little overloaded,
00:21:30
◼
►
it's probably gonna take at least two days to get approved.
00:21:33
◼
►
And then if I have some massive bug,
00:21:36
◼
►
I have a couple days left before the shutdown
00:21:39
◼
►
that I can try to rush a fix through.
00:21:41
◼
►
And so because of that, I don't wanna make
00:21:46
◼
►
massive mega changes to the app right now.
00:21:48
◼
►
I wanna basically ship what's been running in TestFlight.
00:21:52
◼
►
And one of the downsides of trying to fix every crash,
00:21:55
◼
►
trying to fix every bug, is that your fix to the bug
00:22:00
◼
►
that you think you're fixing might introduce
00:22:02
◼
►
its own set of bugs.
00:22:03
◼
►
Fixes are still writing code.
00:22:06
◼
►
And I'm sure we've all done this,
00:22:09
◼
►
where we try to fix a bug and the fix
00:22:11
◼
►
is actually worse than the bug.
00:22:13
◼
►
I'm running that risk.
00:22:15
◼
►
- That was my experience for this last week.
00:22:17
◼
►
- Oh yeah. (laughs)
00:22:19
◼
►
- That is exactly what happened with me and pedometer.
00:22:21
◼
►
I introduced it, I made a change,
00:22:23
◼
►
then I thought I fixed it, and then I made it worse.
00:22:26
◼
►
And then I fixed it again, and hopefully it's fixed now.
00:22:28
◼
►
I'm not sure, more on that later.
00:22:30
◼
►
- Yeah, I do that all the time.
00:22:31
◼
►
'Cause a lot of times a fix looks easy.
00:22:34
◼
►
It's like, oh, I'm inserting this nil value,
00:22:36
◼
►
all right, well put a little check in there
00:22:38
◼
►
that if it's nil, either skip it or omit it
00:22:41
◼
►
or put in NS null null or something like that,
00:22:44
◼
►
put in something else, put in an empty string,
00:22:45
◼
►
whatever it is.
00:22:47
◼
►
And then that itself might cause other weird bugs
00:22:50
◼
►
to happen that you didn't foresee.
00:22:52
◼
►
A bug looked simple, but it was actually,
00:22:56
◼
►
the bug was causing behavior that avoided other bugs,
00:23:01
◼
►
and by fixing the bug, you're now raining down
00:23:04
◼
►
all that crap on some other part of your app.
00:23:06
◼
►
- Or honestly, I think what often happens with me
00:23:08
◼
►
is it's the issue of, so often bug fixing
00:23:12
◼
►
is you're parachuting into this bit of code
00:23:16
◼
►
that you get a stack trace for that says,
00:23:18
◼
►
there's an issue on line 276 of this particular file.
00:23:23
◼
►
This is where the crash happens, say.
00:23:25
◼
►
I go in, I see the thing that could have caused the bug,
00:23:29
◼
►
and I fix it.
00:23:30
◼
►
But the difficulty in that is that when I wrote
00:23:33
◼
►
all that code, I probably wrote it all at once
00:23:35
◼
►
with a good sense of what's going on before and after
00:23:38
◼
►
and all the dependencies and all the things
00:23:40
◼
►
that are going on.
00:23:41
◼
►
And I think the unintended bug's chain
00:23:45
◼
►
of suffering comes so often, at least for me,
00:23:49
◼
►
of where I end up being too myopic on,
00:23:51
◼
►
oh, obviously, this is the issue.
00:23:54
◼
►
And I'm diving in, I fix it, I change it.
00:23:57
◼
►
It's like a two-line change.
00:23:58
◼
►
So hey, I only changed two lines,
00:24:00
◼
►
but I've lost that sense of the context
00:24:05
◼
►
for what's going on there.
00:24:06
◼
►
And so often, that is where I end up with these,
00:24:08
◼
►
oh, right, yeah, no.
00:24:10
◼
►
There's a reason why that value was invalid.
00:24:12
◼
►
And when that value was invalid,
00:24:14
◼
►
I can't just ignore it or skip it.
00:24:15
◼
►
I have to actually deal with it.
00:24:17
◼
►
Otherwise, all this other stuff is gonna start breaking.
00:24:20
◼
►
Or now I'm letting some loop continue in an invalid state
00:24:24
◼
►
and it's just making the problem worse.
00:24:26
◼
►
Those are the things that I often find,
00:24:28
◼
►
that making a change, that can sometimes be super dangerous
00:24:30
◼
►
if you just take that view of, oh,
00:24:33
◼
►
here's the line I need to fix.
00:24:34
◼
►
Great, thank you, Xcode.
00:24:35
◼
►
Let me fix that one line.
00:24:38
◼
►
- So yeah, so that's basically what I've been dealing with
00:24:40
◼
►
this past week is I just need to get this update out
00:24:44
◼
►
and I have not found all the crashes yet.
00:24:46
◼
►
Every test flight build I've sent out
00:24:48
◼
►
still has some crashes left.
00:24:50
◼
►
But they all have fewer crashes than the ones before them.
00:24:54
◼
►
And so at some point, you just have to say,
00:24:56
◼
►
you know what, I'm at a deadline.
00:24:59
◼
►
I'm out of time.
00:25:00
◼
►
I'm going to ship this.
00:25:01
◼
►
This is gonna be overcast 5.0.5.
00:25:04
◼
►
And I know it crashes sometimes for some people,
00:25:07
◼
►
but it doesn't seem like it's that frequent.
00:25:09
◼
►
And it's way better than the version
00:25:12
◼
►
that's in the App Store right now.
00:25:13
◼
►
So you can never achieve perfection.
00:25:17
◼
►
So you have to know when to call it.
00:25:19
◼
►
When to say, all right, this is good enough.
00:25:22
◼
►
And if I've reached a point where most of the app
00:25:26
◼
►
works totally fine for most people most of the time,
00:25:30
◼
►
then that's as good as I can really hope for
00:25:33
◼
►
because it's a big, complex app.
00:25:36
◼
►
And the complexity is mostly just inherent to the problem.
00:25:39
◼
►
It's not a lot of complexity that is just because
00:25:41
◼
►
I'm being sloppy or over-engineering things.
00:25:44
◼
►
Most of it's just like, it's a complicated app,
00:25:45
◼
►
it's complicated things.
00:25:47
◼
►
And maybe the edge cases where it's crashing
00:25:50
◼
►
are only happening for users who have a lot of podcasts
00:25:53
◼
►
or if they're running it on an iPhone SE
00:25:56
◼
►
and its battery is throttling so it's running
00:25:58
◼
►
at the performance of an iPhone 1,
00:26:02
◼
►
and maybe that's why it's using so much CPU power
00:26:04
◼
►
because the CPU is so slow.
00:26:06
◼
►
There's so many edge cases that you can never
00:26:10
◼
►
accommodate for.
00:26:10
◼
►
You're never gonna write code that never gets terminated,
00:26:13
◼
►
that never crashes, that never shows bugs.
00:26:16
◼
►
But when you have a deadline, you just gotta ship something.
00:26:20
◼
►
And shipping something is better than waiting forever
00:26:23
◼
►
for perfection.
00:26:24
◼
►
- Oh sure, and I think especially,
00:26:26
◼
►
it is a really weird time of year
00:26:29
◼
►
'cause I have it in the same position.
00:26:30
◼
►
So like, we talked about over the last couple of episodes,
00:26:33
◼
►
I have been doing all kinds of data changes
00:26:35
◼
►
in pedometer++ and I had a situation where
00:26:38
◼
►
I ended up that I had a much easier fix.
00:26:41
◼
►
That fix dramatically improved things
00:26:44
◼
►
for the majority of my users, awesome.
00:26:46
◼
►
For a few of my users, it introduced an issue
00:26:50
◼
►
where suddenly their step counts were undercounted.
00:26:53
◼
►
And then I was like, oh no, that's terrible.
00:26:55
◼
►
Like I hate it when I take people's steps away,
00:26:57
◼
►
it breaks their streaks, it was awful.
00:26:58
◼
►
So then I'm like, okay, let me go in and fix this.
00:27:00
◼
►
And in fixing it, I introduced a bug
00:27:03
◼
►
that dramatically overcounted steps
00:27:05
◼
►
and people would end up with these days
00:27:06
◼
►
with like 300,000 steps.
00:27:09
◼
►
Because I was being so cautious about never making sure
00:27:13
◼
►
that the step counts never went down,
00:27:14
◼
►
that turns out I was actually, in very certain circumstance,
00:27:18
◼
►
increasing them dramatically.
00:27:20
◼
►
So I worked on getting that fix done.
00:27:21
◼
►
And it's like all this is happening
00:27:23
◼
►
within the back of my mind.
00:27:24
◼
►
It's like, I have to have this done by the end of the week.
00:27:27
◼
►
Because A, it's like Christmas is,
00:27:30
◼
►
I mean it's not as big as it used to be,
00:27:31
◼
►
but it's like this time of year is a big time of year
00:27:34
◼
►
for the app store.
00:27:35
◼
►
It's a big important time in terms of
00:27:37
◼
►
people downloading things.
00:27:38
◼
►
I make Apple Watch apps.
00:27:40
◼
►
I imagine a lot of people are gonna have Apple Watches
00:27:42
◼
►
sitting under their Christmas tree.
00:27:43
◼
►
I have all this in the back of my mind.
00:27:45
◼
►
And at a certain point, it's just like,
00:27:46
◼
►
you gotta get it through.
00:27:48
◼
►
And I'm just trying to get to the point that
00:27:50
◼
►
I think right now, it's mostly good.
00:27:53
◼
►
It's hard to know 'cause it's like the kinesia of the bugs
00:27:55
◼
►
are really hard to reproduce.
00:27:57
◼
►
But I think to your point, it's getting better.
00:28:00
◼
►
And at a certain point, there's this magic line I think
00:28:03
◼
►
where it crosses over, it's like it's good enough.
00:28:05
◼
►
It affects a small enough group of people.
00:28:08
◼
►
It's never great when it affects any people,
00:28:10
◼
►
but it's like you're trying to cross this critical
00:28:12
◼
►
threshold where it's either, it's an annoyance
00:28:16
◼
►
for some people or for a large group of people,
00:28:19
◼
►
or it's only affecting a relatively small percentage
00:28:23
◼
►
of users and how small that is, it varies dramatically.
00:28:27
◼
►
But there is this magic point where you're just like,
00:28:30
◼
►
I think this has to be good enough.
00:28:32
◼
►
And you kinda just ship it and hope for the best.
00:28:34
◼
►
And then, yeah, it's like hopefully in the next few days,
00:28:36
◼
►
things settle down before the App Store closes.
00:28:39
◼
►
And then you have the one benefit of the App Store
00:28:41
◼
►
is closed, you can't change it even if you wanted to.
00:28:44
◼
►
- Yeah, it's true actually.
00:28:45
◼
►
- Reassuring in a weird way that it's not gonna
00:28:49
◼
►
destroy your Christmas because there's nothing you can do.
00:28:52
◼
►
- Well, hope everybody has a wonderful holiday
00:28:54
◼
►
if you celebrate Christmas, and if not,
00:28:56
◼
►
at least enjoy a week off of no App Store.
00:28:58
◼
►
So thank you for listening everybody,
00:29:01
◼
►
and we'll talk to you next week.