I found a 50-minute video from Airbnb’s growth team, talking about how they built and designed Referrals at Airbnb to scale their customer acquisition efforts. I was surprised that there wasn’t a transcript anywhere, so I decided to take notes.
First, our summarized takeaways:
Growth is aided by precision, metrics and novelty:
- Growth = Awareness + Conversions. Growth for a team like Airbnb is about accelerating awareness (get people to know about Airbnb) and improving conversions (get people to signup for Airbnb). Most people in the world haven’t heard about your product, and most of the people who’ve heard about it aren’t using it.
- Measure success by using a framework that models your customer’s journey. For Airbnb, this is Potential, Acquire, Onboard, Book, Return & Share. (Metrics in the video.) This is roughly akin to Dave McClure’s Pirate Metrics. It will likely be different for your business.
- New channels are valuable for growth: New channels can be disproportionately valuable for growth: In the Q & A at the end of the talk, Gustav explains why his team chose to focus on WhatsApp as an acquisition channel. The average person is likelier to pay attention to a referral on WhatsApp than via email, for the simple reason that he gets ads and spam in his inbox, but only messages from friends via WhatsApp. Being early to use a new channel (tastefully!) is a huge competitive advantage for a growth team.
Referrals are dramatically more effective than people realize:
- Referrals are more effective than you might realize. Before working on Referrals 2.0, the general consensus was that Referrals didn’t really matter and didn’t have much utility. But when they looked at the data, they realized that it accounted for millions of dollars of sales.
- Dramatic compounding effect. There are many different parts to a referral system, and improving each of those parts by a few percent can lead to a big difference. The best case scenario that the Airbnb team describes is over NINETY TIMES as effective as the “good” scenario.
- Referrals are one of the few things that touches people outside of your user-base. Most of the things that you build (e.g. product features) don’t touch non-users but referrals do.
- The biggest problem with Referrals 1.0 for Airbnb was that they didn’t promote it enough. Awareness. Most of their users, and even their own employees didn’t know about the referral program.
- Referred users perform better than regularly-acquired users. This compounds- the friends they refer are likelier to stick to the product, too.
- Celebrities can be powerful: A celebrity in China called Anthony single-handedly induced thousands of signups and hundreds of bookings in the first month.
On actually getting the work done:
- Internal motivation is a powerful thing. Jimmy talked about how they didn’t want to put out a Referrals product that they weren’t proud of, so they worked really hard on it. They shared some sketches with a designer, who loved them so much that she joined their team. They then posted her mockups around the office, which excited the rest of the team.
- Building a referral system is very hard work. It took a 5 person team (with lots of borrowed help!) 3 full months and 30,000 lines of code to do Airbnb’s Referrals 2.0 system.
Here’s the full 50 minute talk, along with notes and screenshots of slides:
[V: I’ve edited some of the transcriptions for clarity.]
G: We’re gonna talk about the referral program that we’ve worked on for the last 6 months. Growth at Airbnb obviously works on user growth. We’re trying to get more people to use our sites, do bookings, become hosts.
Ultimately, referrals are about creating more amazing experiences for Airbnb guests and hosts.
<Gustav tells a story about Mort and Julie, Airbnb hosts in Carmel Valley that he had a great experience with before he even started working at Airbnb>
The Airbnb Growth Team
Couple of different teams working on growth. Designers, product managers, data scientists. Dozens of people, depending on how you count.
The team is responsible for products and features that drive more people to become guests and hosts.
What are the two things that growth really does?
- Accelerate awareness. Most people in the world have not heard of Airbnb. And only a minority of those who’ve heard of it have used it.
- Drive conversion. We work on getting people to become users, guests, hosts at a higher rate.
Why is it important that a company work on growth?
It gives you a competitive advantage. If you don’t work on growth, somebody else is going to, and they’re going to beat you.
The Airbnb user acquisition funnel:
Most Airbnb users haven’t signed up yet. They aren’t currently using the product.
We can break this down into a funnel or a framework for how to think about all these users.
- Potential: Travel across country borders
- Acquire: view at least 1 page on Airbnb site
- Onboard: Search Airbnb with check-in/out dates
- Book: Book trip on Airbnb
- Return & Share: have meaningful offline experience
People around the world, how do they go from being potential users who’ve just heard about us, to become acquired, onboard, make a booking, come back and tell the world about their story?
For onboarding, for example, we look for people who search with dates. It’s a sign of intent. Acquire- signup, become users.
Growth Team Philosophy:
- Our users tell the story better than we do. Most people in the world hear about us from a friend. Word-of-mouth is the strongest driver of our product. Why is that so important?The concept of Airbnb is new and foreign to a lot of people. A lot of questions come up. You say “Oh I’m going to stay in someone’s apartment,” people start asking questions about safety, trust. It’s important that someone can answer these questions, someone who’s actually used Airbnb.
- We never compromise on user experience. You don’t have to choose between UX and growth.
- No tricks.
What does Airbnb’s growth look like right now?
We have accelerating, compounding growth.
For Airbnb, because it’s driven by word-of-mouth, the adoption curve looks better and better in terms of absolute numbers over time.
If we don’t do anything, it’s probably still growing– as a growth team we’re here to make it steeper, faster.
Referrals is one of those things that reaches outside your user base.
Few features are touching non-users. Most of the things that you build don’t touch non-users. Aimed to make your experience betters for your own users.
What should you work on to drive more awareness? Think of all the things that you work on that touch non-users, and think which of those are largest in volume?
Referrals 1.0, launched in in 2011.
But luckily we had all this data, and it turned out that it really did have a really really positive impact- millions and millions of dollars of revenue.
The biggest problem with Referrals 1.0 was that it wasn’t promoted.
Most people on the site and even Airbnb employees had never heard of it.
We didn’t want to promote something we weren’t proud of, so we decided before we do that, we just wanted to make a better version of referrals.
When doing Referrals 2.0, we could learn from our experience doing Referral 1.0. We had data, a framework/set of levers.
What should we be working on to make the new referrals better?
Using a forecast for thinking about referrals:
Forecast, framework. Prioritise- what should we work on? Which of these things are the most important for driving compounding growth? – Good case, better case, best case. Invitees per invite (friends per advocate). Conversion rate thinking.
Dramatic compound effect
If you compound all these things, there’s a dramatic difference on the bottom line- better case is 20x better than good case, best 90x better than good.
And it’s not like it’s impossible to move your monthly active users from 1% to 10%, or conversions from 15% to 45%. In terms of opportunity, these things are actually possible.
^I can’t name these companies. These are companies ya’ll have heard of. Airbnb has a much smaller share. It looks like we’re not doing that well. Some of these are mobile-only companies.
1st question: What platform do we start Referrals on?
All trends said mobile was growing quickly
At the time we were making decisions, the growth team was 3 people. Gustav, Amy, myself. Product manager, 2 web engineers.
It made sense to go with web-first, but all the trends said mobile was growing quickly, 50% of our emails were opened on mobile already. Could have a higher ROI for us since we already have a web version, but Mobile is the way of the future.
Core mobile team was tied up with relaunch
Since we were doing this mobile relaunch coming up soon, the entire core mobile team was focused on it, we couldn’t get anybody. So Amy and Jimmy went for a 1-week offsite training in mobile development with Big Nerd Ranch (they wrote the book on Android/iOS programming), were embedded in mobile team for a while.
Had a really good effect for the whole team- we helped them crush a couple of bugs, now they have friends on the website, win-win.
Sketching out the referral flow
Amy was the earliest member of the growth team. We gathered our knowledge together, came up with these sketches for how we wanted people to use our referrals product.
- Discovery that there is a referrals product
- Explaining what they’ll get out of it, and what it’s for
- Selecting people, inviting them, following up
We took it to the design team, asked them what they thought. they loved it. We got a full time designer to join us, and she kicked ass.
We got so excited with these mockups that we posted them all around the office. That got us a lot more awareness, people got excited about what we were building. Jason had just finished a big backend project, and he wanted to work something really big, and he wanted to join us.
5 people on the team. 1 product manager, 1 designer, 3 engineers. Kickass team, almost like a startup.
We had so much passion and we loved what we were doing, we wanted to have our own space so that we could live and breathe the referrals program while we were building it.
Fortunately, we know of a way to get some space for us. Found a place, booked it, and did a one week offsite where we were all working together. Great bonding experience for all of us, had a ton of fun. Very solid foundation for what we wanted going forward.
- 5 people full time with much borrowed help
- 3 months
- 30k lines of code
- Growth became the largest outside contributor to the mobile apps
- We managed to pick up an additional engineer for web development
Launched Jan 22nd on Web, iOS and Android simultaneously.
How Airbnb Referrals 2.0 actually works
- You land in, import your contacts from Gmail.
- It recommends you a couple of people from your contacts that we think might be interested/relevant.
- Send them an email.
- You can go back and send additional emails if you want to.
This is unique (recommended emails). We didn’t see recommended contacts on Gmail from any other service.
To do this- context API, email API, find who might be relevant. To do this we had to ask a bit more permissions than most people are used to.
So this requires a lot of trust, number of people accepting these permissions went down– but ultimately the number of people getting referred and the relevance of these invitations to them went up. So the jury’s still out. But it’s a fun experiment for us to run anyway.
Receiving an actual invite via email
This is the exact template you’ll actually get:
- We take a picture of the referrer and put it right in the centre. Social proof + trust.
- Completely mobile optimised.
Warm and welcoming page -> who invited you, and value prop of what Airbnb is. Great 1st time UX, and this page has an amazing conversion rate.
Receiving an actual invite via email… on Mobile
Very similar to the web experience.
- Big call to action: Go invite your friends.
- Contact screen: Big learning is you gotta make this thing super snappy.
Depending on how popular you are, you might get a bit of delay- when you build this, page them, get them in batches. There are gonna be duplicates in iOS, so de-dup them. Integration with big social networks- whatsapp, we chat, sina weibo.
Android- really similar- additional feature for recommended contacts. (Contacts API on android- most frequently contacted people- put them at the front of contacts.)
- Include code field in the signup flow
- Have a generic coupon field outside the signup
Email, password, coupon field… I’m gonna go online and search for that, they broke flow, gone outside, they might not come back. It’s not great to put the coupon field in the signup field if you can help.
On web, separate landing page- big picture of your friend.
Doesn’t bother the normal flow. We can’t do this for mobile- iTunes doesn’t pass any info from the app store. You have no idea where this person came from. All you can do is show the generic flow.
Yozio makes a fingerprint of your device, redirects to the app store- 2nd fingerprint. Made a connection between the link click and the app open.
$25 dollars from your friend Tanya. Personalised experienced from a friend inviting you to Airbnb. Great experience for everybody who went through it.
Caveats- Yozio accuracy isn’t 100%, so we included another way of inputting the Referral code from the settings menu- can still do it within 30 days of signing up.
Legal considerations for Airbnb Referrals 2.0
Legally, referrals are considered promotional and fall under the same rules as marketing.
- All server sent email needs to have an unsubscribe link
- P2P SMS is a gray zone, we decided against it in the US
Tracking Airbnb’s Referrals
Very important to us that every single action that they took was instrumented from day one, so you can keep score and plan for the next thing.
Impact: Launched end of Jan in 2014, hundreds of thousands of nights from referrals in 2014. 2 parts- guests-side, hosts who book guests- that also counts. This only includes the guest side of things. On the host-side, it’s even more impactful in terms of revenue for the company.
The earlier the market, the more impactful this is.
Growth team has been focused on Asia: South Korea and China.
In South Korea, 30% of our first time bookings coming from referrals. Tactical lever for early-stage market, very important.
We’re not even halfway through of all the iterations– changing the value that we’re giving away, tweaking the flow and the conversions, lots more we can do. We’re maybe halfway through this project now.
Referrals drive 900% year-on-year growth for 1st time bookings
Look at the growth of referrals as a source of first time bookings. Referrals driving almost 900% year on year growth as a program, in terms of driving 1st time bookings. Not true for other channels driving 1st time bookings.
Referred customers perform way better than the average user -> compounding effect.
Is it just a scheme that will attract low value users and not do so well? Answer- They perform way better than the average user.
The most important aspect is referred users referring others. Referred users refer other users much more, and those are much more likely to signup as guests and hosts, so there’s a compounding effect.
Unexpected windfalls: Celebs sharing referral links lead to thousands of signups and hundreds of bookings
In China we have this celebrity called Anthony who has 2 million followers on Weibo and posted photos and a link to a referral program. Thousands of signups, hundreds of bookings in the first month. Influential people can have a massive impact on something like this.
If I don’t hear about Airbnb from a friend, I’m going to hear about it from people I trust. Might be a celebrity.
Back to forecast- what do we do next?
We hit our goals on 3 of the 5 numbers, and the remaining 2 we’re gonna keep working on because they’re really important to get to better or best case. Thank you so much!
Questions from the audience:
What point in the users life cycle do you present them with the opportunity to sign up? Immediately? After a good experience? When’s the best time to do that?
A: Great question. We have launched many different entry points, and we track every single entry point with a specific brand to say this user referred this many friends at this entry point.
Some entry points appear for specific users- not every point appears for every user. People who just signed up, did a search and came back.
Signup wasn’t the one that performed best. Well, but not best. The opportunity for us to invite people- guests, hosts, users… is all over. For a service like Airbnb with a cycle time for users who become users -> users, hosts, quite low.
Do you do any analysis on how many users would’ve become users without the referral program? Overlap / cannibalisation?
A: One way is to look at the quality/behavior. Pretty hard to determine that this is a cannibalised user and this is not. We only give away $25 per user so it’s not a huge cost for us. If we get people to signup 6 months earlier than they would’ve, it’s worth it for us. If it’s 1 minute before otherwise, probably not worth it.
There are tests you can run, and we’re doing that, but you can look at the actual user behavior- how soon does a booking happen after signup? You would imagine that a referred user would book right after signing up.
Q: Any bad experiences? How did you address them? Stress points?
A: Confusion– people getting invites and not knowing what Airbnb is about. We assumed here that seeing logo+image is clear enough to someone what this site is about, but we’ve seen a lot of people don’t get that at all- they ask their friends what is this, etc.
People are appreciative of the emails. ‘I got $25 for my friends.’ They actually really appreciate that.
How would we address those things? In the case of the email, we’re now trying to be much more explicit about explaining specifically what does the service do, at every single point of the conversion funnel.
Q: Very specific targets with different numbers- how did you come up with those numbers/range?
A: These numbers are not correct, ours are less broad, more specific. They came from the existing referral program that we had. We had a % for each bit. So we had things to anchor us to. We also looked at some benchmarks- went out and talked to a bunch of companies that built similar systems before.
You don’t necessarily want to get to 100 invitees per inviter on average, that might be spammy. Then we check every month.
Mobile growth- Whatsapp. I’m assuming you didn’t show all those options in all different countries?
A: iOS can check if they have the app installed. Can-open-URL API(?) Deep link? Something something you can check if they have the app. We haven’t really analyzed it that far.
Whatsapp conversion through Yozio. We did WeChat and Weibo because of our focus on Asia. In general in growth, if there’s a new channel, especially if it’s big, and especially others haven’t capitalised on it– most in WhatsApp is communication– very little growth engineering in that channel.
If we could pre-write a message and have people send that message to their friends on WhatsApp, people are gonna care about that message much more than they’re gonna care about email. That’s the motivation for doing WhatsApp.
Q: On web referral program, did you experiment using native email clients vs web based email clients? Emails come from user email vs. servers.
A: We decided on server approach because easier to track + also easier to deal with unsubscribing (legal issues).
Q: Can you share the top variants that failed? Counter-intuitive insights that you gained through testing?
A: We thought, how should we promote the referral to get as many people as possible to send invites? Huge banner on homepage! That didn’t work at all, because it’s not a natural flow for users to go through.
Most users don’t go to the home page. They might go to the inbox, dashboard, booking flow. Homepage is not a super sticky page for people to go through. That was a mistake, but good to learn. The way to evaluate a promotion- what’s the impression count, people acting on impression- if most people don’t act, it’s probably not a good place to put it.
Better place to put it is right after you do a booking, right after you do a review, maybe in the action (?) menus. Finding a spot on the flow + predictable place where people can go back (I want to do it a second time without going through the flow again). We’ve done a bunch of experiments for motivations for doing invites- inviting, giving credit, earning credit. We haven’t dug deep enough to get clear data.
Q: How frequently are you reaching out to users to get them to refer? Reaching out to all users, or certain subsets (more ideal candidates)
A: We’ve done analysis on users who send invites. Some user segments are more likely, but at the whole user base- doesn’t make sense to focus on some segment. If one segment is twice as likely, but only 1/5th…
Entry points– we have a way to give value at these entry points, we think about cycle time– how often do these (?) happen for these users? Some entry points never happen. If you ask people to refer only after completing booking, most people might not make referrals, because they haven’t made bookings yet. If you ask earlier, you might get more but less conversion…
Q: Do you use a different forecast for web and mobile?
A: Same forecast for now. Different conversion rates on mobile vs web, which makes sense because while we try to make mobile as easy as possible, still more steps on mobile- couple of extra taps+clicks. Web is basically 1 step from signup. So web has slightly better conversion. But doesn’t mean we can ignore mobile- half of emails open on mobile, increasing on every single metric.
People on mobile are more likely to invite from address book, friction is lower. On web, people less likely to invite from address book- more friction? iOS slightly lower- no address book (Android has address book).
More traffic coming from mobile, suggest more from mobile.
Q: How did you come up with $25, $75 for host?
A: Predated both of us, from the first referral program. Nice number.
A guest is not worth the same everywhere around the world, a host is not worth the same everywhere around the world, but if you don’t do that- Japan, US, something… some tradeoffs you have to make. A lot of people do go to these coupon sites and they find a lot of these codes- would hate to see (?). We think hard about preventing codes from ending up on coupon sites (?).