Developing Perspective

#118: Ambitious.


00:00:00   Hello and welcome to Developing Perspective. Developing Perspective is a podcast discussing

00:00:07   news of note, de-noise development, Apple, and the like. I'm your host, David Smith.

00:00:09   I'm an independent iOS and Mac developer based in Herne, Virginia. This is show number 118,

00:00:13   and today is Tuesday, April 9th. Developing Perspective is never longer than 15 minutes,

00:00:17   so let's get started. All right, so it's been quite a week for me so far, and it's only

00:00:22   sort of Tuesday. So what I'm going to talk about a little bit this week is kind of the

00:00:28   The approach I'm taking and kind of, there's a little bit of it that's crazy, some of the

00:00:31   things that I've been doing, and kind of why I'm doing what I'm doing.

00:00:36   So first off, I'm going to talk about kind of this week.

00:00:38   And it was just one of those strange things where a whole variety of things kind of lined

00:00:42   up so that I was essentially going to have three major deliveries that I was going to

00:00:47   try and do this week.

00:00:48   The first one I did yesterday, I launched Check the Weather 1.4.

00:00:51   I changed the weather backend and did a few other things.

00:00:56   It's a significant update, but not a major update probably,

00:00:58   I'd say.

00:01:00   Today, Tuesday, I launched my Recipebook 3.0,

00:01:03   which is an app I've been working on, honestly, for years.

00:01:07   It's been quite a while in making to try and get--

00:01:09   to decide how I was going to do sync,

00:01:11   to decide how I was going to do pricing,

00:01:14   and all kinds of things, which I'll talk about in a little bit.

00:01:16   And hopefully, actually, my goal is on Friday

00:01:19   to submit Feed Wrangler's iOS app

00:01:22   to the store, which is turning out to be-- we'll see.

00:01:26   I think it's still probably doable,

00:01:28   but if it turns out, those would have been

00:01:30   three kind of crazy things to all do in one week.

00:01:33   Early on Monday morning, I was thinking,

00:01:36   and I was like, you know, is this wise?

00:01:38   Am I taking on too much?

00:01:40   Am I doing something that's kind of foolish

00:01:42   by trying to do all these things?

00:01:43   And really, the reality I found was,

00:01:45   I always find it's the helpful thing to think about

00:01:47   of what's the worst that can happen?

00:01:49   What's the biggest downside?

00:01:51   And is it any different than the downsides

00:01:53   of doing things individually?

00:01:56   And in this case, I really didn't think that the downsides were that different by doing

00:01:59   it all together, that it was kind of nice to be a bit audacious, a bit ambitious, and

00:02:03   to try and do all these things at once, because really all of them benefit from being out

00:02:09   into the wild sooner and earlier.

00:02:12   Things like sync, where there's problems and issues and things, the longer that I can be

00:02:15   fixing them, the sooner that'll be the case, and the sooner I can then kind of settle that

00:02:20   down and hide in a bow.

00:02:21   And similarly with check the weather, it's like I could have held off on the update until

00:02:25   next week, but it only sort of helps in that the same issues are still going to be there

00:02:32   for the most part.

00:02:33   And so in the end I was saying, you know, let's be audacious.

00:02:35   And I think it's just as an encouragement to anyone out there, it's don't worry too much

00:02:38   about overdoing it necessarily, that you want to be careful obviously, but if you trust

00:02:44   yourself and you know your skills, you can kind of understand what are appropriate risks

00:02:49   for you to take.

00:02:50   All right.

00:02:51   All right, so the next thing I'm going to talk about

00:02:53   is the My Recipe Book launch, a little bit about My Recipe

00:02:56   Book more generally.

00:02:57   So My Recipe Book's a recipe manager,

00:02:59   fairly straightforward in that sense.

00:03:01   It's a place that you can collect, organize, catalog

00:03:04   your recipes.

00:03:05   You can put it as a grocery list, those types of things.

00:03:08   And the app's been around for about three years.

00:03:11   And it's, until now, has always been a standalone iPad

00:03:13   application.

00:03:14   It's been something that is just-- that's

00:03:16   the way it always was.

00:03:17   I built it that way.

00:03:18   And it always sort of just made sense that way.

00:03:22   Because in order for it to work on to be universal,

00:03:25   to be cross-platform, all those types of things,

00:03:28   the recipes have to sync somehow.

00:03:30   They have to be able to go back and forth between the server.

00:03:33   So if you make a change, you input a recipe in your iPad,

00:03:35   it shows up in your iPhone.

00:03:37   And that was always a little intimidating.

00:03:39   Because obviously, that's a fairly complicated problem.

00:03:42   Sync is always complicated.

00:03:46   There is no easy sync.

00:03:47   And so at first, I tried a couple of things myself.

00:03:52   iCloud came out.

00:03:53   I tried that.

00:03:54   None of those things had been quite right

00:03:57   until I took a step back and said, OK, really,

00:03:59   what am I trying to do here?

00:04:00   And I just really methodically worked it through

00:04:02   for a couple of weeks.

00:04:03   And I was able to get to a point that, yeah, OK, this works.

00:04:06   And I spent a couple of months testing and trying it out

00:04:09   and playing with it and doing all kinds of things

00:04:11   in the hopes of stabilizing it.

00:04:13   Though I will say, sync is the kind of thing

00:04:16   that's impossible to fully test until you deploy.

00:04:18   I don't think there's any reasonable way,

00:04:20   even with a large beta group, that you'll find all the bugs.

00:04:22   There are just all kinds of subtle and nuanced things,

00:04:26   things that relate to the nature of the user's data,

00:04:28   for example.

00:04:29   Like it's a recipe manager.

00:04:30   It depends a lot on how many recipes some people have.

00:04:32   And some people have quite a lot of recipes.

00:04:34   I've found several users who have several thousand recipes,

00:04:37   which I didn't-- I had test data sets that were that large,

00:04:41   but I never really thought that that

00:04:43   would be a realistic use case.

00:04:44   And it turns out that is a realistic use case.

00:04:46   So maybe my test data needs to be even bigger

00:04:49   to account for that as an edge case.

00:04:51   And also what I want to talk about is a little bit-- the way

00:04:53   I took my sync, I've had a lot of questions about if I use

00:04:57   iCloud, how I use iCloud, those types of things.

00:05:00   I ended up using iCloud for a component of my sync.

00:05:04   And what I ended up trying to do is try to be very pragmatic,

00:05:06   and look at it and say, what does iCloud do

00:05:09   that only it can do?

00:05:12   And what are things that it potentially doesn't do as well?

00:05:16   So first, what are some of the things that only it can do?

00:05:19   And on an iPhone, the simplest version of that

00:05:23   is it provides an avenue for ubiquitous login and user

00:05:26   identity.

00:05:28   And when you have an app with an iCloud account,

00:05:32   when they open that app, immediately available

00:05:34   is who that person is to some approximation.

00:05:37   And so what I started doing is, well,

00:05:39   I'm going to sync the user-related profile

00:05:41   information using iCloud to improve my user experience.

00:05:44   I'm going to look at it and say, hey,

00:05:45   if I put user information in the key value store, which

00:05:49   is very reliable in my experience,

00:05:51   I can create a great login experience.

00:05:53   They open up the iPad.

00:05:54   It says, hey, my RSP book 3.0 just launched.

00:05:56   Would you like to sync?

00:05:57   They hit yes.

00:05:58   It sends it off to the cloud.

00:06:00   They open up their iPhone and say, go to the App Store,

00:06:03   get the app, open it up.

00:06:04   And it says, hey, I noticed that you also have an iCloud.

00:06:07   You've already sunk with this account.

00:06:09   Would you like me to use this data?

00:06:10   They say, yes, off it goes.

00:06:12   And that is a really nice experience

00:06:13   that you can only provide with something like iCloud, where

00:06:16   it knows this meta information about who this person is

00:06:21   and about the devices that they have.

00:06:23   And then I take that-- all the hard part of the sync,

00:06:25   the recipes and grocery list and those types of things,

00:06:28   and I move that into my own cloud storage.

00:06:32   And it isn't just because of all the things

00:06:33   you hear about with iCloud core data syncing being complicated.

00:06:38   I mean, I tried a variety of different solutions

00:06:40   platforms. I tried some period of sync for a while. I tried, oh gosh, all kinds of different

00:06:45   approaches. And in the end, what I kept coming down to is I wanted to own the sync, not necessarily

00:06:50   because that makes my life easier, but because that makes my product better. Let me unpack that

00:06:56   a little bit. Any time that you take something as sort of crucial as sync and put it into a third

00:07:03   party's hands, whether that could be anything, it can be iCloud, it can be some period, it could be

00:07:07   It could be Winder Azure services.

00:07:12   I think there's a new thing that Matt Thompson released

00:07:16   as part of his AF incremental store backend.

00:07:19   There's all kinds of approaches you can take.

00:07:24   Anytime you do that, the difficulty you run into is

00:07:26   how easily, if at all possible, is it for you to fix problems,

00:07:29   bugs, inevitable things that will come up.

00:07:33   And usually whenever I was looking at most of these solutions,

00:07:35   the tricky part is a lot of it's a black box.

00:07:38   You take this data and you put it in there

00:07:39   and you hope it works.

00:07:41   Which if it works, is great,

00:07:42   but it's never gonna work 100%.

00:07:44   You have to kind of assume that it can do work

00:07:46   a lot of the time, but it's never gonna quite be 100%.

00:07:49   So what do you do in the weird edge cases?

00:07:51   What do you do in the cases where the user

00:07:53   has some strange configuration, some strange thing,

00:07:57   weird connectivity, running through a weird proxy

00:07:59   that is messing with the data?

00:08:01   What do you do in that case?

00:08:03   And a lot of times the answer for a lot of the platforms

00:08:05   I looked at were, there's not really that much you can do.

00:08:08   You can ask them to re-sync maybe,

00:08:09   you can hope it's not destructive.

00:08:11   But the reality is, what I love,

00:08:13   is over the last two days,

00:08:15   I've been able to go into the database,

00:08:17   oh, go into the app, I can look at exactly what's going on.

00:08:20   I can go through the server logs, I can replay stuff.

00:08:23   I can look at it and be like, okay, this is the problem.

00:08:25   Here's the thing that I need to fix.

00:08:27   And the reality is, I think that makes

00:08:28   my product a lot better.

00:08:29   makes my customer experience a lot better.

00:08:32   For example, I had a customer who had a particular data

00:08:35   corruption bug that was pretty subtle and worked its way in.

00:08:38   And I was able to go into his account, look in the database,

00:08:41   tweak a few records, a couple of things that were wrong,

00:08:44   and then I can send him an email and say, hey,

00:08:46   your account's fixed.

00:08:47   Next time you open the app, it'll be working fine.

00:08:49   And that's just awesome to be able to reach into their life

00:08:52   and do that for them.

00:08:54   And so I highly recommend-- at the end of the day,

00:08:56   it's a hard thing to say if someone says,

00:08:58   oh, I've been thinking of doing my own, what should I use?

00:09:01   Well, I can say that I can build my own

00:09:03   because I'm an experienced Rails developer.

00:09:06   Before I did iPhone development,

00:09:08   I was a full-time professional consultant doing Rails work.

00:09:11   I sort of knew my way around that world.

00:09:13   And it's a hard thing to learn.

00:09:14   I'm not saying it's impossible,

00:09:16   but the reality is you wanna kind of work your way up.

00:09:19   And if maybe you just tackle problems at an easier level

00:09:22   if that's something that's above you.

00:09:24   But it's really complicated

00:09:25   if you're taking your core business value

00:09:28   and putting it in someone else's hands.

00:09:30   It's just something I get very nervous about.

00:09:32   And as I've gone through this, that's

00:09:34   kind of where I settled on it.

00:09:35   One thing I did about the launch that I thought

00:09:37   was also interesting worth mentioning

00:09:39   is I did what I call a soft launch in the sense

00:09:42   that I released the app to the update, to the store,

00:09:45   before I started publicizing it, before I had people

00:09:48   in the press talk about it.

00:09:49   But the hope that I can kind of gradually work out,

00:09:52   A, does my server infrastructure scale,

00:09:54   and B, does my server architecture, does my sync,

00:09:57   does all these little different components,

00:09:58   do they actually work?

00:10:00   Do they actually gonna do what I think they should do?

00:10:03   And it was kind of fun when I hit update,

00:10:05   release to the store, and then I can go into my server logs,

00:10:07   I can tail all the production logs,

00:10:09   and I can look at it and say,

00:10:10   and just watch the first user create an account and log in,

00:10:14   and make sure everything works.

00:10:16   And then you say there's two people, and there's three people

00:10:18   and this kind of gradually building process is great

00:10:21   to making sure that everything worked.

00:10:23   In the end I'd say it has kind of worked out really well.

00:10:24   My server infrastructure that I was kind of building,

00:10:26   I think I've talked about a couple of times

00:10:28   where I have a couple of line nodes arranged,

00:10:32   running rail servers with a Postgres backend

00:10:35   and a load balancer in the front,

00:10:37   and it ended up working great.

00:10:38   And the experience I had,

00:10:40   which I think I'd mentioned also,

00:10:42   where I had practiced building up and tearing down servers

00:10:45   was also really helpful,

00:10:46   'cause I found that my original configuration

00:10:49   had two front ends and it looked like I needed three.

00:10:52   I was able to, in about 10, 15 minutes,

00:10:54   spin up a new server, deploy it,

00:10:56   sort of go through this sort of checklist,

00:10:57   and in the end it worked flawlessly,

00:11:00   and it's been great working ever since.

00:11:03   And so that kind of a soft launch can, I think,

00:11:04   be really effective if you can allow yourself to do that.

00:11:09   One way to do that, even if you're launching a new app

00:11:11   that you can kind of think about,

00:11:13   is to set the app to release the night before

00:11:18   you're expecting any kind of, sort of the big hit,

00:11:20   the big publicity, the big promotion.

00:11:22   You can release it the night before,

00:11:25   and kind of trickle it out,

00:11:27   maybe reach out to a few friends,

00:11:28   kind of see if you can build a little bit

00:11:31   of actual production experience

00:11:33   without actually having to sort of push it out

00:11:36   and then have this big sort of stampede

00:11:38   of people working on your app, hopefully.

00:11:41   It sort of avoids a bit of the problem

00:11:42   if you think about it, like the mailbox

00:11:44   or these apps that have been having all this weird,

00:11:45   having to take strange tactics to avoid scaling problems.

00:11:49   One of the things that I think about

00:11:50   is how it's just like trying to sort of slowly

00:11:53   the faucet on would be a great way to do it.

00:11:55   So that's something that I've tried, seemed to work, and I'm pretty happy with.

00:12:02   So where does that put me now?

00:12:03   So that's the next thing that I'm working on then is feed wrangler, to turn that around

00:12:08   and to hopefully ship three things in a week.

00:12:12   And feed wrangler's kept going well, it's getting pretty close.

00:12:14   And what I actually really liked about the timing of this is I've been able to prove

00:12:18   and verify my server infrastructure approach.

00:12:22   under a fairly reasonable load with my recipe book

00:12:26   that I can now just immediately take that information.

00:12:28   I've learned a whole bunch of lessons.

00:12:30   There are some, even the simple examples

00:12:32   are some of the things that I wasn't indexing

00:12:34   some of my data correctly in the database

00:12:36   and I've found these performance bottlenecks immediately

00:12:39   when it's, when you have thousands of people

00:12:41   hitting it all at once, they become much more apparent

00:12:43   than when it's just you and your test data

00:12:45   from, there's only so many concurrent connections

00:12:47   that you can really simulate.

00:12:50   And so when I started using this real data,

00:12:51   I was like, oh, here's some indexes I'm missing,

00:12:53   here's some things I need to tweak.

00:12:55   I'll go into Postgres, do those changes,

00:12:57   and then it's awesome.

00:12:58   I use New Relic RPM server monitoring,

00:13:01   which gives you essentially throughput,

00:13:04   request per minute, response time, error rate,

00:13:06   those types of information for a Rails app.

00:13:08   I think it can work for a variety of servers.

00:13:10   But I use it, I don't usually,

00:13:11   I'm not necessarily gonna keep it in the long run.

00:13:14   It's something that I tend to use targetedly,

00:13:15   'cause any time you instrument your app

00:13:17   and do a lot of monitoring,

00:13:19   there's gonna be a performance hit

00:13:20   that you're taking as a result.

00:13:21   But right up front, it's always great,

00:13:23   because it helps you quickly identify these bottlenecks

00:13:25   that you can then very quickly attack.

00:13:29   There was, I think, a post at a chart on app.net.

00:13:32   There was this crazy call where I

00:13:35   was able to take the average response time

00:13:38   and cut it into about a third of what it was before,

00:13:41   just by adding a few key indexes to the database.

00:13:44   Any time you can do those kind of wins,

00:13:45   it's just kind of amazing, because it

00:13:48   feels like black magic.

00:13:49   It's like what you had before was working, it was correct.

00:13:53   You add a little bit of something to it,

00:13:55   and then all of a sudden it works so much better.

00:13:57   And so that was kind of exciting to see,

00:13:59   to be able to experience in a production environment,

00:14:03   and to know that was improving customers' interactions

00:14:06   immediately.

00:14:08   And so that's kind of where I am.

00:14:09   It's been kind of a crazy week.

00:14:11   I know I've gotten a lot of questions,

00:14:12   but I'm not sure I answered them all.

00:14:14   It's been kind of crazy in terms of stress and worry, too.

00:14:18   So it's just kind of, if I didn't answer a question

00:14:21   that you had about kind of these launches I've been doing,

00:14:24   just the best place is probably gonna be email me,

00:14:26   david@developingperspective.com,

00:14:27   and I can try and loop back on another show,

00:14:29   just to make sure I hit 'em all.

00:14:31   But otherwise, it's a busy week, exciting,

00:14:33   lots of fun stuff, and I really appreciate

00:14:34   you listening to the show.

00:14:36   So yeah, so that's it.

00:14:37   As always, questions, comments, concerns.

00:14:39   I'm on Twitter @_davidsmith, appnet @davidsmith,

00:14:42   david@developingperspective.com.

00:14:44   Thanks, happy coding, and I'll talk to you later.

00:14:46   Bye.

00:14:47   [