Under the Radar

Under the Radar 80: Debugging


  welcome to under the radar a show about

  independent iOS app development I'm

  Marco Arment and I'm David Smith under

  the radar is never longer than 30

  minutes so let's get started so today we

  wanted to dive into everybody's favorite

  pastime as a developer and that is

  debugging it is a topic that I think we

  probably it's fun of weird in some ways

  I think I probably spend at least half

  if not a majority of my time in

  development debugging code that I ride

  yet in most of my like actual formal

  preparation for being a software

  engineer there was very very little time

  and attention spent on it like almost

  all of my classes and things are all

  about focused on the actual development

  in the architecture side of things

  whereas the reality is we spend most of

  our time you know fixing the problems

  and issues that arise in our apps and

  this year these can range from all kinds

  of you know to different levels there's

  the kind of while you're developing

  debugging process where you know you

  write some code you you know hit build

  and run and you see if it works and if

  it doesn't you try and work out why

  there's the you know the act the sort of

  more subtle something happens every now

  and then and you're not really sure why

  and you try and track that down and then

  there's the you know you getting crash

  reports or bug reports from customers in

  the field and then you have to try and

  track those down which is even more

  exciting and challenging but in all of

  them I think there is an element of you

  know debugging is essentially you have

  you sort of identify that there's some

  problem you then try and reproduce that

  problem in a way that you can then

  observe that problem then you try and

  isolate where and the actual you know

  program that is happening and then you

  can try and fix it and you kind of go

  through this cycle of you know you

  identify it you we produce it you

  isolate it and then you can fix it and

  you kind of cycle through that or at

  least is the process that I tend to take

  because if I don't go through each of

  those steps I find that it's very hard

  to actually make progress I mean if you

  can't you know some bugs really are by

  their nature

  kind of hard to reproduce and if that's

  the case like you know that's still a

  bug and it's so problematic but you know

  if I get a bug report that I can't

  reproduce in any way

  it's just as the same in some ways is

  not having that bug report in the first

  place because I can't do anything with

  that information if I can't reproduce it

  so you know my first step is always

  let's reproduce this and then you know

  this try and find the part of code and I

  think having been a professional

  developer for enough years now like I

  feel like I have a better sort of sense

  for where the bugs might be hiding you

  know it's like we I've seen every you

  know not every bug but I've seen a lot

  of Fugs before and you kind of get this

  sense of like huh you know this is it

  this looks like this is a you know and

  off by one error this is this looks like

  this is a in you know cache and

  validation error this looks like you

  know I'm writing something incorrectly

  here or I forgot a case in a switch

  statement like you kind of get this

  feeling for it and then you can kind of

  go running around trying to find it and

  of course there's also I guess the other

  kind of debugging which is even super

  fun which is where there's a bug in the

  OS that you you know where your code is

  all doing the right thing but then

  you're getting weird results from the OS

  and so that's an even more fun you know

  fun line of debugging where you kind of

  get the sense of at a certain you keep

  going down and farther and farther down

  the rabbit hole until eventually you're

  like hit a brick wall at the end of it

  you're like alright well I guess I file

  a bug yeah and that's you know it's it's

  often easy for us to jump to well this

  must be a bug in the OS or my favorite

  one this must be a bug in the compiler

  or something like that when we can't

  figure it out that is for the vast

  majority of bugs that we face as

  programmers that is almost never the

  case it like first of all you are

  unlikely to ever have a bug in the

  compiler actually affect you and you

  know as you go up the stack of you know

  things that you're relying on like the

  OS does have bugs libraries that you're

  using will have bugs but if you run into

  something in your app that that seems

  like a bug and you were tempted to blame

  the OS or a library or something like

  that really make sure

  that you can't figure out that it's your

  bug first because chances are very very

  good it's probably your bug the vast

  majority of the time of those cases it

  will really be your bug it's only a

  question of whether you can find it but

  don't just say oh well it's a bug in the

  OS I guess I got a s I'll just deal with

  this or move on or yell at them really

  make sure it's not your bug first

  because again the odds are strongly in

  favor of it being yours yeah and then

  because the reality is like most of our

  time at least in these in my experience

  like dealing with bugs I it's almost

  always something I forgot to do or a

  case that I forgot to consider when I'm

  just originally doing it like in not

  necessarily out of like laziness even

  like sometimes it's that you know

  sometimes you have the always classic

  like you know you see an example where

  it's like if error doesn't equal nil

  then do something with it

  it's like and then if you don't actually

  do anything when eventually you do run

  into the situation where the error is

  not nil then all of a sudden things

  break and you know that there's that

  kind of laziness but more often it's

  just like you forgot to do something and

  then it comes back to bite you and I

  feel like the what is it I think there

  was a you know maybe maybe have been an

  xkcd cartoon about this where it's sort

  of that the stages of debugging where

  you go from like it's like oh it's you

  know you get a bug report you know like

  oh no that can't be right I'm sure this

  is I'm sure my code is good and then you

  kind of like no it's actually it's

  actually broken and then you go to the

  like how did this ever work yeah okay

  how on earth did this even work we you

  you amazing how when you look back at

  your own code you know this is think

  code that you wrote with your own you

  know your own hands on a keyboard you

  put this into the empty into the into

  Xcode you come back and look at your

  like how when earth what am i doing like

  and I feel like that is the hardest part

  and or why do bucking is so hard is it's

  you have to recreate all of this sort of

  state and information that went into the

  creation of the code hold that in your

  mind and then run it through a scenario

  that's happening you know and to then

  see why it's doing something weird and

  very often I find like the hardest thing

  is to try and understand even what the

  code is trying to do in the first place

  or like why there's this weird exception

  you know that was introduced there you

  know your yours or you know decades ago

  and you're like huh I wonder why I did

  that I don't know anymore

  and that's you know makes debugging

  really really hard yeah and this is you

  know it's also a major argument for

  keeping things simple and not that

  clever in the first place and yeah as we

  talked about I think a while ago now you

  know like I try to avoid overly clever

  code Styles things like you know using

  using really dense short hands or really

  clever mechanics you know things like

  generics in like you know Swift like

  that kind of thing I'm like we you kind

  of use like overly clever complex

  language features or constructs where it

  looks really cool and you feel really

  smart when you do it but when you have

  to come back and debug code you know in

  six months and you have totally

  forgotten what led you to to write that

  and you and you've forgotten the

  cleverness behind it and you have to

  figure it out by looking at it or or you

  know figuring out by by its effects it's

  often very very hard to do that and you

  know like you know you you often as a

  programmer you you might eventually hear

  the wisdom of like write code for

  someone else to read but but really what

  you should be doing is writing code for

  yourself in three weeks to read because

  you will have probably forgotten by then

  why you did what you did so if you keep

  things simple from the beginning and not

  be too overly clever with the way your

  right things and and make things you

  know straightforward to read and to you

  know kind of self document then you will

  be doing yourself a huge favor

  documentation you know separate

  documentation can help with some of

  these things but I found in practice

  that maybe it's just me I don't document

  things that well I try to just write

  things that are that are obvious when

  you look at them and that that you know

  just have good names and straightforward

  approaches and that tends to work very

  well and you know that can never be out

  of date because it's the code itself and

  then for for minimizing bugs to begin

  with I think it's it's again very

  important not to be too clever and and

  to realize you know for me many of my

  bugs come from

  quick workarounds or quick hacks that I

  do when I encounter some other bug and

  and so one great example of this then

  that I'm sure many iOS programmers face

  is whenever you are doing something in

  in like a launch loop or or in in a UI

  you know response or something and

  there's some weird behavior that or some

  weird precondition that you can't quite

  accommodate in a straightforward way so

  you just dispatch async to the next run

  loop of the main queue and you say all

  right dispatch async got to make you do

  this thing on the next run and that's a

  really nice quick way to to avoid a lot

  of a lot of like you know weird

  complexity or obvious bugs as you're

  developing but I have found in almost

  every case that using a dispatch async

  is almost always a bad idea like when

  you're just like I'm deferring work till

  the till later on the main queue or

  something like obviously there are uses

  where it where it's fine but like what

  if that's your approach to solve a

  problem then you get into concurrency


  and weird weird conditions there that

  are really hard to fix and diagnose and

  reproduce and it again it's like one of

  those areas where it seems like a clever

  thing when you're writing it and and in

  the case of like bug avoidance like it

  seems like oh this will be a quick fix

  for this weird little bug I'll just wrap

  it in dispatch async and that will that

  will fix the problem but then you

  introduce all these weird other problems

  that might be non-obvious and those

  those become very hard to debug so in

  addition to keeping code simple as you

  write it for the purposes of debugging I

  also would strongly recommend avoiding

  concurrency if you can and avoiding

  playing with with threads and queues

  when when not necessary or when not

  warranted because I feel like what

  you're doing there I mean I'm very

  guilty of that thing there I've done the

  you know dispatch async fix probably a

  hundred times 200 times my career like

  all the time and I cannot I cannot

  overstate how men

  bugs that I've had to fix end up being a

  poorly used dispatch async yeah I think

  what the reality is what you're doing

  and this is maybe a maturity thing but

  it's the quiet sea understanding that

  like when I do that now I understand

  that I'm not fixing the problem I'm

  changing the bug from whatever it is say

  see I said a bug that happens you know

  one out of every hundred times the app

  is of the app runs and you know that's a

  problem you know 1% of users are hitting

  this issue I wrap it in dispatch async

  what I'm doing is not removing that

  problem what I'm doing is changing it

  into a like 1 in 10,000 bug which is

  better but also incredibly difficult now

  to actually like properly fix and

  diagnose and reproduce etc etc like it

  is the kind of fix where it isn't

  actually solving it it's just making it

  less likely which sometimes you know

  pragmatically I've you know there I'm

  sure there are many places in you know

  my shipping apps where I have that and

  it's just one it's like a conscious

  choice that you're just like well the

  actual fix is too hard or would require

  a massive refactoring or some kind of

  you know thing where you can make this

  trade-off in choice and you say you know

  like I'm just gonna fix this by patching

  it and I know that what I'm doing is

  gonna cause a problem for me down the

  road I know that I'm you know there is

  this bug is hasn't magically disappeared

  as a result of this but like it's a

  conscious choice and that's a balance

  but yeah ultimately you know the most

  when you're writing the code you're kind

  of trying to do your future self a favor

  as much as you can because I think one

  of the things that I struggle with - and

  I'm writing code is the realization that

  I all of these things that seem obvious

  to me when I'm writing the code because

  I'm in a mindset where I'm focused

  entirely on whatever the solving this

  problem is like there's all these things

  that just seem obvious like of course

  you know like this is the way this works

  you know that's why this is set up this

  way this value will you know will always

  transform from this to this like there's

  all these things that seem obvious and

  then in the future they're entirely


  and the didn't think the biggest

  difficulty I have with things like

  documentation is that it's it kind of

  impossible to know what your future self

  is going to want to know

  because everything seems obvious and

  kind of trivial at in the present and so

  like I think I'm kind of like you I'd

  tend to only document code in my apps

  that where I'm doing something that is

  clearly weird or like strange you know

  where you have some kind of magic number

  it's you have to introduce into

  something worth like I know why am i

  multiplying this value by one point six

  three two like there's a reason for that

  and if there is like you know that's the

  kind of thing that you write down but

  otherwise you end up with like writing

  your program twice if you're overly

  documenting it so as you know as much as

  when I'm debugging I kind of wish my

  past self had written all this stuff


  the reality is my past I wouldn't

  actually know what I need to know right

  now because if they did they could have

  just write in you know made the code

  more obvious or clear in the first place

  probably or these that's what I tell

  myself we respond to this week by

  Pingdom start monitoring your websites

  in servers today at Pingdom comm slash

  radar you get a 14-day free trial I'm

  gonna use code radar at checkout you got

  20% off your first invoice Pingdom is a

  wonderful monitoring service they are

  focused on making the web faster and

  more reliable for everyone who has a

  site because they offer powerful easy to

  use monitoring tools and services so if

  you're a premium user you can monitor

  the availability and performance of your

  servers your databases or your website

  and it's so easy to do they test from

  more than 70 global test servers and

  they can they can emulate visits to your

  site as often as every minute and they

  can do cool things like cookies and

  logins and everything else too because

  you know these days websites are

  becoming more and more sophisticated and

  very often include multiple different

  parts with several different

  dependencies maybe you dependent on

  external service or maybe only the login

  screen does this one thing in your code

  and you need to check all these things

  and you can do all that with Pingdom

  with and loads more in addition to that

  too they can make it possible to monitor

  the availability of all these key

  interactions people have with your site

  because you know a lot of times it's

  more complex than your site is up or

  your site is down so stuff breaks on the

  internet all the time Pingdom knows this

  they detect around 13 million outages a


  so regardless of whether you have a

  small website or you're managing a whole

  infrastructure it is very important to

  monitor the availability and performance

  and all in Pingdom when it goes down

  they alert you in any way you want you

  can have text messages you can have push

  notifications emails it's so

  configurable we've even used it in the

  past to monitor other people's websites

  like David and I have both independently

  used it to monitor the WDC page for

  changes and like in the past when it was

  really important to know when WTC

  tickets went on sale I've used Pingdom

  for a very long time

  highly recommended check it out today

  and you will be the first to know when

  your site is down or being too slow so

  go to Pingdom comm slash radar for a

  14-day free trial and use code radar to

  get 20% off your first invoice thank you

  very much to Pingdom for their support

  of this show and relay FM so the other

  thing is probably worth diving into a

  little bit too is the way in which we

  actually do debugging because I maybe

  it's you know maybe it's a shocking real

  revelation but I am I believe what is

  technically called a caveman debugger

  the printf debugging I do all of my

  debugging using you know NSLog yep in

  the console which I you know I think is

  often disparagingly referred to as

  caveman debugging which is fine I'm sure

  there are better ways to do that and I

  mean every now and then I'll go to WDC

  session where they're talking about all

  the crazy things you can do in you know

  in Xcode where you can set conditional

  breakpoints and you know all this kind

  of really clever stuff and in the end

  what I always do is just INF just NS

  logging tons of data to the console

  because it's it's one of those things

  where most of what debugging is trying

  to do is it's like you're trying to

  unwind the state machine that is your

  app or you're trying to work out we know

  it's like your app is in this probably

  you're in a good state then something

  happens and you're in a bad state like

  that is you know at a very high level

  that is what a bug is and what you're

  trying to do is identify that we know

  what point it's going from that good

  state to that bad state like what is

  happening there and you know I mean I've

  been professionally developing for 17

  years now or something and I still like

  that the most

  effective and powerful tool is just n s

  logs everywhere and you get kind of good

  at it like I'm I can identify and find

  things fairly quickly this way because

  you kind of get the sense of you know

  you're a you're trying to isolate where

  in the code it's happening and so you'll

  end up putting ns logs you know in

  different in different parts of your app

  that you think might be part of it and

  trying to work out which ones getting

  you know which code is getting hit when

  you when you do it and then something

  that I also find really helpful is when

  I take I format my NS logs as tab

  delimited text so like often will have

  like several variables that I'm trying

  to observe and watch and I NS log those

  as you know a tab delimited set of

  strings because then I can take the

  output from the console and I can copy

  paste that into Excel and it'll put it

  into columns for me that's interesting

  and you can then you know you can take

  that data and then look at it over

  multiple runs and you can start you know

  it's a way to actually make the day and

  make it into something that's useful and

  you know like at least for me in the way

  that my mind works that's how I debug

  the best is when I'm taking the state

  I'm taking something out of the console

  looking at it often in Excel and it

  would kind of fall finding the pattern

  finding the thing and then I can

  actually go and fix it and like I don't

  know if that works for me

  and you know I I sometimes I feel bad

  about it I think they have all these

  really powerful and clever tools but the

  reality is I think the hardest part too

  is that so much of these this kind of so

  much of debugging is like the hardest

  part is often the reproduction part and

  getting everything like it's hard to

  observe these issues sometimes and so

  like the nice thing about a console.log

  is that you can have those running

  without being attached to the debug

  server for example like you can have

  that you know you can you can create

  these situations where these things

  happen sometimes in like sometimes it

  gets really awkward to have your a to

  have your app attach the debugger

  sometimes when it is attached the

  debugger the app goes the bug goes away

  like there's lots of weird situations in

  issue square NSLog debugging is the best

  but I'm very aware that by doing it that

  way I am

  you know not taking advantage of more

  sophisticated tools but I know for

  myself like that's what works yeah I

  mean and it there's also there's lots of

  different kinds of bugs and and you know

  a lot of these different methods are

  more or less suitable to different ones

  like you know for example NS log

  debugging which I do a lot myself is

  it's it's not necessarily the best

  option if you have like like I'd like

  kind of nitty-gritty like code level bug

  that this function isn't always

  returning the right value like for that

  I prefer to use conditional breakpoints

  so I can see like you know whenever this

  value is something out of whack stop

  here and then I will inspect the stuff

  around it but you know and it's log

  debugging is a lot more helpful for

  things like you know larger systems

  interacting with each other many kind of

  you know large operation the concurrency

  things were like you're not really sure

  where the bugs might be but you kind of

  want to get a good idea of of what is

  going on in the system it's also NSLog

  debugging is also very very helpful in

  situations in which the debugger itself

  is impractically slow to do things so a

  lot of swift context this is the case

  from what I've heard it's also very much

  the case when you're debugging on device

  on the watch because like then like the

  round-trip between your computer running

  Xcode and the phone and the slow watch

  hardware like trying to try to break

  watch apps

  you know like with breakpoints and

  inspect local variables around and you

  know running things in the in the debug

  command line is so slow on the like when

  doing things on the watch that sometimes

  it's better off to just do something you

  know doot-doot-doot a different way do

  printf debugging or whatever else for me

  it's also I also debug very differently

  if I'm looking at a crash report versus

  some kind of behavior in the app that

  I'm catching live it's it's very

  important to to look in iTunes Connect

  and to look at the or a you know rather

  I guess now it's in the organizer in

  Xcode to look at the crash report that

  Apple's getting for you you know look at

  look at what's being gathered there

  there's a whole lot of different

  problems that your app can have in the

  wild and the fact that we can get our

  apps it's that we can get our crash

  reports basically all the time from

  Apple that like whenever an app crashes

  we get notified

  about that that's pretty awesome and of

  course also lots of different other

  services as well but there are some

  types of crashes that only apples crash

  logs will generally reliably be able to

  report to you that includes things like

  if i OS kills your app for some reason

  that isn't necessarily a crash but if

  your app is getting killed in the

  background or something or if your

  extensions are getting killed

  then that's things that iOS can can

  provide very useful feedback to if you

  look for it and that's all in the Xcode

  organizer and one of the things I can

  recommend highly is look at those and a

  lot of times the organizer won't tell

  you like if you get like in my I was

  just spending a lot of time debugging

  crashes and my today widget or my widget

  I guess and if you look at just the

  crash log it's not very helpful in the

  organizer but if you right-click and

  show and finder on that crash and you

  dive into the bundle and you can get the

  raw crash logs that are inside of that

  bundle when you look at the raw logs you

  can get a termination code and a lot of

  times that is springboard telling you

  something and there's a great technical

  note tn 21:51 understanding and

  analyzing application crash reports and

  this tells you all the crazy codes that

  your app might be killed with so one of

  the things is that I was facing is I was

  my extension was accessing the sequel

  Lite database in the shared app group

  container I don't recommend doing this

  now but now I know better

  but if you are accessing an SQLite

  database in a shared container in an

  extension at the time that the OS

  suspends you it will kill your app with

  a certain code that says deadlock in hex

  and if you didn't look at this document

  and if you didn't find that crash report

  and find that code and if you didn't

  then look at this document to see that

  what that means what it tells you right


  it's un terminated by the US cause it

  held on to a file lock or sequel like

  database during suspension and I didn't

  like you know and before I found all

  this out I just didn't know why my

  extension just kept crashing for

  everybody and people kept reporting

  things like hey your app is crashing in

  the background for some reason getting

  all these logs on my phone what's going

  on iOS tells you these things like it's

  telling you useful information here and

  so if you if you look at those crash

  logs in the Xcode organizer and if you

  look up what these

  for you know termination codes mean

  you'll probably find out something

  useful and and this is all like and and

  we have wonderful things like if you're

  kind of wondering whether you fixed a

  rare crash or not you can look at the

  analytics section of iTunes Connect and

  it will graph for you how many crashes

  per day your app is having and you can

  cut and it marks on the graph where your

  updates occur so you can say oh look

  when when this update was released the

  crashes dropped by half so the thing I

  fixed I probably fit the thing I think I

  fixed I probably did fix there's all

  sorts of useful things you can get from

  crash logs and things of that and and

  one of the reasons why I do also as I

  said earlier kind of recommend against

  unnecessary use of dispatch async is

  because that makes your crash logs less

  useful to you because it it usually the

  crash logs will not report where the

  block was dispatched from some of the

  more advanced ones like hockey a try to

  do this and they have mixed degrees of

  success in my experience and so if your

  code is simpler than any crash log you

  get will also be more useful to you yeah

  if you crash logs are definitely like a

  great source for us so much of this

  stuff and I feel like in general I will

  say that Xcode as much as I don't use

  all of its tools like some of this stuff

  that you're talking about if going into

  the organizer like there's some really

  useful you need tools that we have at

  our head at our you know sort of at our

  disposal to be tracking some of this

  stuff down now which I don't think we we

  had before I mean it makes me think even

  of one of the other tools that I use a

  lot in debugging in a strange way is

  version control sin specifically get

  blame is often incredibly helpful in

  trying to track down weird situations in

  in my apps beware you know code is doing

  something weird and I'm trying to work

  out why I structured it in the way I

  structured it and this is something that

  I love in Xcode where you can go into

  the top right where you go into the

  source control thing and you can do

  blame and it will show you all of you

  knows for every line of code in your app

  it'll show you which commits that line

  of code was introduced into your

  with and so you often will end up

  happening there it's like by using that

  view I can be a see like huh that's


  you know here's this one random line of

  code that is left over from an old

  version that everything else was updated

  or you know situations like that and you

  know Xcode is really and then you can

  not think if you double-click on it you

  know show you that commit and I'll show

  you the relevant how that you know how

  that file a line of how that how that

  file changed with that commit and that's

  another example of one of these little

  tools in Xcode where it's like it helps

  you understand what's going on in the

  same way that yeah like those that's a

  really great one I mean I love in their

  crash logs were you know that can help

  you take you right to where the crash

  happened rather than sitting there and

  you know trying to you know work out

  which we know is if this crash happened

  in this you know and this this file on

  this line number like well I mean that's

  useful but it's also much way more

  useful to just take me to that line

  number so there's definitely some great

  tools in Xcode I think make a lot of

  debugging a lot simpler and I'm very

  grateful for I would also say when

  you're when you're running your app and

  you're trying to figure out maybe maybe

  what's going on with a bug always watch

  that that pain in Xcode shows you the

  running CPU usage and memory of the app

  and of course you can go into

  instruments and get you know like a more

  fine-grained version of this but like

  yesterday there was this really tricky

  bug that I could not figure out that the

  app too started behaving really weirdly

  when loading episodes of a certain

  podcast and it like really weirdly and

  it would eventually disconnect from

  Xcode it was a massive massive weird bug

  and springboard crash and what led me to

  the solution was that I was I happen to

  be glancing at that pain that was

  showing me that the resource graphs in

  Xcode that would during one of the runs

  and I noticed that right before it

  crashed memory usage skyrocketed all the

  way up to like a gig and then everything

  freaked out and blew up and so then I

  was able to sit to say hey wait a minute

  that's unusual that stands out let me

  run it under the the allocations

  instrument run that can show you where

  all your memory is being used and I did

  I so I did that and found out figured

  out what the bug was within like a half

  hour after that this is a bug has been

  bothering me for weeks I couldn't figure

  it out and then within a half hour of

  noticing hey this thing is out of the

  ordinary on the memory meter here

  I had the fix so like you know use the

  tools that are available to you when we

  know when appropriate or just use NS log

  and printf debugging everywhere yeah and

  I mean what that you would you down

  there there was one thing that I will

  say that I love about debugging is that

  so often you get to the end and there's

  like a relatively straightforward

  solution that fixes the problem and like

  that is one of the greatest joys of

  being a soft software development when

  you like you think you see the problem

  you see the solution and you can

  implement it and like it's gone and this

  thing that was problematic and hurting

  your users is now just completely

  vanished and like that's like a great

  victory and I always love when that


  doesn't always happen but when it does

  it's absolutely awesome so it's the one

  reason I love debugging that's why we do

  all this right that that feeling and

  when it finally works we're like yes I

  love that feeling all right thanks

  everybody for listening this week and

  we'll talk to you next week bye