Under the Radar

Under the Radar 38: Psychological Tricks


  welcome to under the radar a show but

  independent iOS and recently Mac app

  development I'm Marco Arment and I'm

  David Smith under the radar is never

  longer than 30 minutes so let's get


  so today we wanted to dive into a little

  bit about talking about the best term

  for it is probably cycled the

  psychological tricks that are often

  employed in apps to boosting retention

  engagement addictiveness or things are

  you trying to boost it in-app purchases

  a lot of these things you often see in

  in a like in these kind of in-app

  purchase focused applications these you

  know sort of pay to play games or things

  i've noticed a lot of them in like I was

  recently trying out took them on go and

  I kept seeing these kind of these trends

  in these tricks and these things that

  are going on into it and our point in

  this is definitely ought to be there's

  F&T atopic to talk through about like

  the moralizing part of that of like

  sometimes these things can get kind of

  abusive or really awkward and the way

  that they're being used but I think

  moreover woder is an interesting topic

  as app developers was to think through

  how these different ask these different

  tricks that you can do can be used

  positively in your applications you know

  there's some positive things that you

  can do when you're aware of these and

  also it's kind of a strange experience

  that I've had myself that when I started

  kind of researching and looking into

  these types of topics is that once

  you're aware of them you start noticing

  them in a lot of different places and

  it's kind of like if you ever see

  someone do a magic trick and then they

  show you how it's done and then they do

  the trick again the second time you see

  it it has a very different experience

  and it can still be interesting and

  compelling but it's a it's it's overall

  it's a it's nice to have that awareness

  of what's going on and she knows we're

  gonna kind of talk through some of these

  things and I think the first one I

  wanted to dive in and talk about is the

  concept of loss aversion and so this is

  something that you it's like a

  psychological phenomena and where people

  are much more affected by the prospect

  of losing something they have then the

  prospect of gaining something new or you

  can phrase it in a lot of different ways

  but essentially it's like if you have

  something or you feel like you have

  something you will work way harder to

  keep that thing

  then you would in regaining it or hadn't

  gaining it in the first place and so

  you'll see this so often in games like

  the first time I remember ever seeing

  this sort of concept really driven home

  for me was in a game the first time in

  like an in-app purchase game where

  you're going down here you're doing your

  run and you get to the end you die and

  the whole thing pops up and it says

  would you like to continue for three

  crystals or whatever it is like them

  whatever the made-up Smurf berry thing

  that they're they're using in the game

  and all but they're directly doing their

  is they're tying in to loss aversion

  because you've gotten to this point in

  the game you know you oh wow this is my

  new high score this is this thing that

  I've worked towards I want to keep it I

  want to keep it going and you'll be so

  much more apt to do it at that moment

  then you would to just start again and

  try and get there get there in the same

  place and as that can be a little bit

  tricky and a little bit perhaps not a

  not ideal but you can also on the

  positive side think of something like

  like streaks in a fitness app so in my

  app activity plus plus I have this thing

  where it tracks how long you've had you

  know each hits your standing goal or

  your move goal or your exercise goal and

  that you don't want to break a streak

  because you feel like you have it and

  you don't want to lose it and the reason

  that that is so said like the concept of

  keeping a streak going is so effective

  is this concept of loss aversion it's

  this weird feeling of people just don't

  like giving up things they have and

  often since some ways irrationally if

  you compare the effort that which

  someone will go to keep something they

  have versus just you know gaining it

  again or gaining it in the first place

  and so it's kind of a weird aspect is

  this something that you've ever run into

  like can you think of apps where you

  feel yourself being manipulated by loss

  aversion oh yeah

  Mario 3 because in Mario 3 the best

  possible powerup in the game is the

  hammer suit also called the sledgehammer

  suit the hammer Brothers suit whatever

  it is and you get like one of these in

  the whole game you're lucky if you get

  one and just like any other powerup if

  you get hit you lose it that's it and

  the sledgehammer suit is so much better

  than anything else in the game is like

  those sledgehammers can kill everything

  ghosts even like the big thumps like in


  the Catholic they kill everything it's

  amazing and you get one maybe in the

  whole game and so you gotta like really

  save it up and like I'm so afraid of

  using it and then just getting hit and

  losing it that I often will go through a

  whole game and forget I even have it and

  never even use it

  sure I know that's total the Rockets

  yeah it's fine it's this is the weird

  thing like people act irrationally about

  like this fear you have of losing

  something you have and it's a powerful

  tool I think as as developers for us to

  like you can think of in our apps like

  are there things that we can do where we

  make people feel like they have

  something that they don't want to lose

  because it really can be strongly

  impactful like it's this really weird

  conditioning thing that people really

  don't want to give up what they have and

  whether that's yeah it's like I think of

  things there's so many things in games

  where that same kind of thing happens

  where you're trying to make someone

  scared of losing something they have I

  mean I think you're even like in Pokemon

  go is the where I noticed this recently

  was they have this this the thing where

  like you discover a Pokemon yeah that's

  great but if you don't like feed it

  raspberries and use the fancy balls it

  may run away and having something appear

  and then disappear is way more painful

  than never finding it which is it what

  kind of a weird thing but it's that that

  kind of behavior you can totally see the

  way like a way that game like that is

  structured is entirely to manipulate you

  into making sure that you always feel

  like you have to use all your things

  because you never want to lose the thing

  that you just got and that's kind of

  weird I don't know like it's it's a

  strange thing how people just mindset

  just can be they can act against what is

  sort of their rational best interest I

  suppose you know you see as you said

  like you know once you notice this you

  kind of see it everywhere like this is

  in so many games and you know app

  developers you know that this is

  relevant to non game developers to some

  degree as well I mean obviously a lot of

  these kind of tricks or psychological

  plays obviously they tend to have I

  think the most direct

  uses in games especially in that

  purchase driven game as you said but

  there are certainly places and apps that

  you can use a lot of this stuff to like

  you know loss aversion is is kind of the

  idea between like a time-limited trial

  where you you can you can try out this

  app and have these great features but

  then after you know 30 days or whatever

  those features stop working and you have

  to pay and buy the app if you want to

  keep using those so it's like you had

  them you know how good they are and then

  oh now they're gone you got you better

  pay you know that's it can work the same

  way there for us oh sure I mean even in

  some ways I makes me think a little bit

  about though it's part of what makes

  subscription pricing compelling where

  you've built up this sense which is sort

  of like the lock-in kind of concept but

  if you build up a you know like I have

  an RSS sinking system that I run and if

  you build up all your they have all my

  RSS feeds tied into it and like part of

  why you may keep paying for it isn't

  necessarily because you use it but you

  don't want to lose the ability to use it

  or you don't want to but like you've

  built up a collection of notes in

  Evernote or something and you don't want

  to you keep paying almost like your

  plebs like you're paying like a ransom

  to this thing because you don't want it

  just like oh well if I stop paying all

  the stuff that I've built up over time

  will just go away and you may that above

  what it actually is reasonable to value

  it at because you for you don't to lose

  it and like it's interesting that like

  it sort of digs into these really deep

  parts of P of you where you start to

  like act irrationally totally and so the

  next one I wanted to talk about and this

  is a kind of a fascinating one when you

  actually sort of get into read like this

  is a weird one almost as a parent I've

  noticed myself since I was like did the

  research into kind of how reinforcement

  was the topic which is no I'm the

  concept of how if you're trying to

  condition a behavior into something or

  someone you know so you have this sort

  of experience and reward cycle where you

  know if you're maybe this maybe this

  rather than getting into parenting so

  you think it's like taking care of a dog

  or something right like the dog does the

  you know you tell it to sit

  it's it's you give it a treat you tell

  it to sit it's it's you give it a treat

  like that's a very simple reinforcement

  cycle like the direct positive they do

  the action they get the read they get

  the reward but the funny thing is in

  there's a lot of sort of research that

  backs this up is that the strongest and

  most powerful form of conditioning is

  where you have they call variable

  reinforcement where it is still

  predictable insofar as the average

  number of reinforcing events is sort of

  still somewhat constant but the timing

  in which they happen is not the same you

  know so in the example of trying to

  train a drug say if ever if roughly

  every third time the dog sits you were

  to give it a give it a treat but if you

  did it on exactly every third and then

  it's much as a much weaker reinforcer

  then just on average giving it every

  third and this is the concept of science

  behind slot machines and many many

  things where you if you add just a

  little bit of randomness in it so that

  the person can get the feeling that on

  average they're being fairly rewarded

  because if I if it doesn't feel fair

  then it's kind of self-defeating because

  it's it's consistent enough that it

  feels fair but on any particular

  opportunity they don't know if they're

  going to be rewarded or not it is really

  powerful for creating that kind of

  classically addictive kind of behavior

  where you want to keep trying the the

  reinforced activity and you keep trying

  and trying and trying in in the hopes

  that you'll get it but you never know

  when you're going to get it and so

  creates this you know it's the cycle and

  you know you can imagine this on the

  more you know tricky side with a lot of

  a lot of things in apps where you kind

  of you get an item that will be

  upgradeable or has some other value down

  the road or like in Pokemon go there was

  the thing with eggs where like you get

  an egg and you never know what's in it

  sometimes it's gonna be something boring

  and sometimes it's going to be awesome

  and as long as it happen as long as it's

  consistent enough that you feel like

  it's fair you'll keep trying it in a

  more strong way but even in like in a

  weird way on the positive side I was

  thinking about in a lot of my apps I

  have confetti to celebrate when you hit

  your goals and in a weird way I'm using

  variable reinforcement to get people to

  keep opening my app because they don't

  know if they've hit their goal when they

  open the app you know every it's like if

  the whole point of the app is to show

  them how many steps they've taken and so

  as someone goes around their day every

  time you open the app you have this

  feeling of like you know it's like

  you're pulling the arm on the slot

  machine in some ways and saying like

  have I done it yet have I done it yet

  have it done it yet I did hurray but

  that's it's variable you never know when

  it's gonna happen you know if it's like

  the third time you open the app and

  every day you got confetti well it's not

  really it it wouldn't be nearly as

  compelling and so you is this but if as

  long as you add that variability that

  randomness into it is where it starts to

  get really interesting that's amazing

  it's gif and see even that I mean like

  the way you're doing it is a little bit

  different than like a like a slot

  machine kind of thing like the way

  you're doing it is based on reaching a

  goal whereas like the the just kind of

  randomly fail and randomly give people

  what they want sometimes that that just

  feels a little bit dirty to be I don't

  know I mean it's kind of the problem all

  these things kind of feel a little bit

  dirty if you think about them as like a

  way to manipulate people but you know

  the with a lot of these things there are

  obviously like ways to do it in a way

  that makes that let you sleep at night

  so you know with this this though I I

  have a hard time coming up with those

  ways that aren't actually driven by

  things that are within the user to

  control in reality like your app sure

  and that's the thing that's so

  interesting about this is a topic that I

  kind of found fascinating as a developer

  because it's like if you can it's like

  if you can be aware of these things and

  find ways to judiciously and responsibly

  use them in your app it can be really

  compelling like that's I mean in many

  ways this is what makes things like

  Facebook or Twitter or Instagram

  effective it's the same this exact same

  concept where

  you keep going to the thing in the hope

  like it's like you never know if you're

  gonna open up Twitter and it's just some

  it's kind of like more boring or not

  interesting to you but every now and

  then you get this really interesting

  exciting thing or you feel like wow I'm

  the first person to know this things it

  happened just a moment ago and that

  conditions you don't want to go into

  there and in a lot of apps like that can

  be a useful thing that can be a helpful

  thing to actually think back to thank us

  my first WWDC hack in 2009 I remember I

  signed up for one of the UI design labs

  and the person who was you know looking

  at my apps design one of his comments

  was do you have a place in your app that

  has a constantly changing sort of set of

  items or a set of anything because if

  you do you can start to even create this

  this habit and this you know this

  pattern that people will start to do to

  go and check it you know it's in this

  case if there's an audiobook app and

  it's like he wasn't wondering if I

  should start to do something where you

  know each day I put a new you know a

  particular book and spotlight it or

  something and it creates that same

  pattern it's where you even if sometimes

  you'll go there and there's nothing it's

  it's like oh it's not for me it's not

  for me oh that's interesting I've never

  seen that the first is you have that one

  experience of going there and being

  rewarded for it it starts to build that

  habit and starts to build it in and that

  can be a good thing but like this one in

  particular is really worrying to me a

  lot of ways and as a side note it always

  kind of bothers me when I see app

  descriptions in the App Store where it

  has like super addictive as listed as a

  positive bullet point like you know what

  I'm talking about you'll have the little

  thing and it's like this it's like an

  anti ad like being like being on the top

  grossing list I think is also kind of an

  anti ad four-and-a-half is it's like oh

  if I've down was that I'm probably gonna

  spend a lot of money

  I always just drove me crazy or it's the

  guilty these things and it's like

  warning super addictive and like it's

  not really a warning in the way that

  it's like a health and safety label like

  they're trying to pitch it is like he's

  like try this app you'll get super

  addicted to it like

  translated your head as warning huge

  ripoff it's like I don't want like the

  last thing I the last thing my life

  needs is another is like something to be

  addicted to like that's that's not ever

  going to be a good thing in my life and

  so that's definitely something that I

  I'd never find like this in particular

  like you start to look out for it where

  anytime there's kind of a random random

  way in which you're reinforced it's like

  oh no

  take a step back like I'm being messed

  with some someone someone is

  manipulating me in a way that I may or

  may not actually like exactly alright we

  respond to this week by a new sponsor

  max Stadium the manage Mac hosting

  company Mac Stadium recently merged with

  Mac Mini Colo so they have thousands of

  Mac's in multiple data centers at around

  the US and Europe they are by far the

  largest Mac hosting company this is you

  know if you want to have a Mac running

  in a hosted environment so you need to

  be running OS 10 on Mac hardware whether

  it's a Mac Mini or Mac Pro they also

  have Mac pros you know you want to have

  that with Pro connectivity to the

  Internet so what you get in data center

  is what you get from web hosts they can

  provide Macs flash sand storage and

  everything else needed for quick

  expansion and hosting of Mac stuff so

  they have some pretty big customers like

  uber Travis CI and unity they have

  hundreds of Mac minis and Mac pros for

  continuous integration and deployment if

  you run a continuous integration server

  this could be a great option for you Mac

  Stadium can set up Mac environments

  faster and at a lower cost than trying

  to do it in-house so you can trial a Mac

  server free for 24 hours you got 10% off

  hosting as well for a year by using code

  radar 10 so once again go to Mac Stadium

  comm slash radar and use code radar 10

  to get 10% off hosting for a year and

  you get a free 24 hour trial thank you

  so much - Mac Stadium for supporting

  under the radar and all of relay fm

  alright the next one of these things

  that I thought would be fun to talk to

  dive into a little bit is not as much of

  a trick like the first two bar but is

  something that is a psychological

  phenomenon that is pervasive in people

  and that is impatience oh boy so this is

  something that I start to notice and

  obviously none of us like waiting right

  like we always feel like we want things

  faster better if like faster and

  even with less waiting and you see this

  in games a lot like I mean you might

  also see it in podcast audiences that

  only listen to 30-minute podcasts that's

  true we may be a particularly impatient

  group so maybe this will resonate with

  our audience but in a lot of games

  you'll have a thing where this is

  concept of like a timeout or a weight or

  an energy meter that you have to fill up

  and if you don't you know when you've

  spent all your energy you have to wait

  24 hours while you recharge you know or

  you can pay money to recharge your

  energy meter or I don't you know it gets

  very convoluted at a certain point but

  it's always tying back to this concept

  of impatience that we don't want to wait

  that we don't like to have a timeout or

  anything like if there's something that

  we can do to feel like we're saving time

  he'll like we will disproportionately

  value it but the interesting thing I was

  thinking when I was kind of preparing

  this episode is I was thing about smart

  speed so and when you made smart speed

  like it is in it's a straight a

  fascinating feature because it taps into

  my sense of impatience in such a way

  that if I don't listen to audio from run

  through sparked speed like I feel like

  I'm wasting my time and that is a really

  powerful like I remember reading at an

  episode of developing respective years a

  couple years ago about this where it's

  like it's an insidious feature like in

  the best possible way that like smart

  speed tadka has now like gotten into my

  brain such that if I'd listened to

  something that isn't smart speed it's

  like why am I doing this I'm wasting

  time 10% that could be you know I could

  have 10% more audio listened to in a

  particular amount of time because that's

  what smarts B does and I wouldn't notice

  it because that's the whole cool thing

  with smart speed you know just shortens

  those silences that I never hear anyway

  it really ruins YouTube for me I have a

  very hard time watching youtube videos

  because like these are so slow sure I

  mean it's true I've definitely had that

  same thought when I was going through

  WWDC videos I was like oh man I wish I

  could smart speed these right and it's

  time tapping into that sense of

  impatience like we hate waiting we hate

  feeling like things couldn't if it could

  be faster we want it to be faster I

  guess in some ways we could just tap tap

  for me like how this is relevant to app

  developers perspective it's like well we

  could always just make everything faster

  like that's a definitely a good thing

  that people hate waiting so if we can

  make our apps patter if that faster

  better perform

  and etc then that's cool and if there's

  any areas that we can take out weights

  then people will definitely like it and

  will notice and appreciate that but I

  guess on the flip side is if we

  introduce artificial weights we can

  potentially find ways to make people

  work to avoid them

  yeah that's ooh have you seen like any

  of the big games like I think the worst

  ones like the real racing games yes like

  you try these games out like on your

  Apple TV or something and like oh it's

  this free racing game it looks pretty

  good all right cool and it's like oh

  well congratulations you just bought

  this upgrade for your car with these

  cool with use coins you earned it's

  going to be installed over the next 45

  minutes or you can pay more coins and

  have it installed faster it's like it's

  such man it drives me crazy like

  unfortunately I've never gotten too into

  these games but I've seen them enough

  and every time I see it it just makes me

  angry like I just feel like there's

  that's just a progress bar to nothing

  and it's just a completely artificial

  delay insert for no reason other than to

  try to get more people to pay more money

  for nothing like they're getting nothing

  for it's I don't know I mean I say this

  as somebody who you know my app is

  funded by people paying for nothing but

  still like it just feels really

  manipulative and wrong to me when I see

  it when I see the way these games do it

  sure yeah and it's like you can take

  advantage of people's impatience in

  positive ways like I just said of a

  smart speed right like I don't think you

  necessarily set out to do it that

  specifically in mind but like for me as

  a user that's why I why I use over house

  to listen to podcasts and I now use it

  to listen to audiobooks to like I use it

  to listen to everything that I listen to

  you that's audio related is I'll put it

  into run it through overcast because in

  my mind the impatience in me thinks that

  if I'm not I'm wasting time and I you

  know it has tremendous retention as a

  result like I think it's a well it's a

  really powerful feature in that way and

  if you can find these little hooks that

  people make people feel like they're

  saving time that's huge but yeah like

  you the flip side of it can be so so

  awkward where you it's like this

  artificial scarcity kind of concept

  where people are just like inventing

  things that take longer or need more

  what Evers to create or use when you

  know it's just a it's just a number in a

  database somewhere it's not any that

  doesn't cost anything to create yeah

  like you have to create this out the

  sense of artificial scarcity this is

  like oh you know we only have so many

  installers in the virtual car each car

  garage who are able to install new

  spoilers like that's really what they're

  saying it's like there's only so many of

  these virtual guys who are able to run

  around like no there's not like you just

  you just you know flip flip a number

  around and then it's done it's that

  doesn't cost you know doesn't take time

  or effort so yeah it's like impatience

  is it is a tricky tricky one for for me

  to feel okay with when you start using

  it too much in an app and anything that

  isn't other than just like making it

  faster yeah like it's more you know like

  if you're if you are creating artificial

  delays just for the purpose of people

  paying you money that feels a little bit

  weird and I can't think of a lot of good

  ways to do that off the top of my head

  whereas if you are taking something that

  is slow for like I don't know quote

  natural reasons and you are adding value

  to the world or something by making that

  faster that's obviously a very different

  kind of game you're playing like that

  that is like adding to the world that is

  making things better for people or like

  inserting artificial delays so that like

  kids beg their parents to let them pay

  you more money I don't know that just

  feels wrong yeah and there's a lot of

  this thing they can go that way but

  anyway we can move on to our next slide

  we get too too stuck in the weeds of

  moralizing about it because it just

  makes me sad so I think the last place

  we've sort of wanted to touch on is the

  in US gets a bit more social I think is

  sort of the a combination of sort of the

  fear of missing out or what you'd call a

  fail I think it's what the hipsters call

  FOMO and competition so this is these

  are very similar so like at least

  fundamentally and I thought I went to

  Wikipedia when I was preparing for this

  and I thought they had a really

  interesting definition of the fear of

  missing out and it's there a pervasive

  apprehension that others might be having

  rewarding experiences from which we are

  absent it is such a true thing and this

  is a lot

  I think people say why things like

  Twitter are really and Facebook are so

  compelling and interesting for you

  because you feel like you're never

  missing out in someone else's experience

  and in a weird way you're almost part of

  it because you can see what they're

  doing you know you can see go on

  Instagram and see a picture of every

  meal they've had and so even if you do

  weren't there and you didn't you have

  that same meal you still sort of got to

  experience a little bit of it and it

  kind of can create that feeling of it

  helps you deal with that that that you

  know the apprehension that someone else

  could be having an experience that

  you're not having and so a lot of apps

  have sharing built into them you have

  this concept of sharing what you're

  doing to create that kind of that that

  reinforcement or that people can go and

  feel like they're experiencing other

  people's lives and not missing out as a

  result you know and so a lot of apps

  will have a little share button or

  something like that and you can on the

  flip side are similarly related to it

  you have like the competition side of

  things which is where you can kind of

  share your experiences almost in a

  overtly challenging way that like you're

  almost saying like this is my experience

  is yours is good which is in some ways

  you're like directly tapping into

  someone's fear of missing out by saying

  like I just went for a great run you

  know I look these are my steps or you

  didn't like kid the Fitbit I think has a

  whole lot of this and watch OS 3 has

  this in the activity sharing like you

  have this concept of saying like here's

  something I did can you do any better

  which is a really powerful way to make

  people attach to something you're doing

  because it feels more personal and it

  almost says like taps into like

  elementary school peer pressure kind of

  stuff that if someone else is doing it

  then maybe I should do it too oh and

  also like that that kind of it helps you

  and your app latch on to and likely

  profit from people's existing

  competitive relationships in their lives

  so like if you have like you know two

  friends who are always trying to outdo

  each other or are having like a friendly

  competition with with you know the

  activity they have in a day or whatever

  else to have them competing in a way

  that means using your app more is good

  news for you yeah because it

  that's ultimately like so much of this

  ultimately is coming down to trying to

  find ways that are beyond just like the

  fundamental functionality of your app

  but finding other ways for making people

  to want to keep opening it you know it's

  like it's retention I guess is the like

  the KPI the keeper I don't know the

  parking lot yeah it's the thing that you

  that you're that you're kind of looking

  for where and this is I mean I ran into

  this a lot with like a lot of my apps

  are now supported from advertising and

  if you have an advertising supported app

  like you want to have people keep

  opening your app and so anything you can

  do to give them a reason to do that is

  you know is pretty interesting like and

  in sharing or these kind of these

  competitive kinds of things it's you can

  get into really funny places too and you

  start to think about it of you know when

  someone shares their steps with someone

  else and say you have a system in your

  app where you sent them a push

  notification there's nothing I've

  actually gotten into but I've been

  trying to think through those kinds of

  things you can get into really funny

  questions of should you show the

  information in the app because you just

  say that you know so ins you know Marco

  sent you a step challenge or something

  or should you be specific about it or

  you know someone just updated they're

  sort of like you'll see this I think on

  Matt actually on Facebook but I've seen

  these on other people's like phones and

  screenshots we have like so-and-so

  updated their status and you don't tell

  anybody why like what actually they did

  oh yeah well the worst one is like this

  person you know commented on your photo

  but they won't tell you what the comment

  is sure it's like and the reason is

  because it creates it improves retention

  and going to something if you have to go

  to the app to find out and so you can

  take advantage of the fact that people

  are competitive and we don't miss out on

  other people's experiences and you can

  tap into that in an app that it's like

  by making it social you can make it

  compelling a little weird in some ways

  but overall like a lot of this it people

  are doing this moron thing on their own

  and maybe you can be aware of it and see

  it as an opportunity for helping your

  app to spread like I love when my apps

  are growing because of social sharing

  that people are saying you know I

  you know I I got your app I loved it I

  recommended it to three friends like

  that's awesome that's like the really

  positive version of this but you know

  maybe it says with all these things like

  there's definitely this balance of you

  can use it in a good way or you can use

  it in a not so good way and maybe at the

  very least if you're aware of it you'll

  be able to better take care of yourself

  as a result yeah all right we're out of

  time this week thanks a lot for

  listening everybody please use these

  tricks responsibly and use them to make

  people's lives better not worse make the

  world a better place don't subtract from

  it thanks everyone and we'll see you

  next week bye