How Much Optimization Is The Right Amount?
Hiya guys! Patrick (patio11) here. You signed up for periodic emails from me about making and selling software. Sorry is has been a while since the last email. I was struggling with a one-two combo of illness and business overcommitment this fall. Maybe I'll write in more detail about that later. Thanks for the well-wishes for those who sent them -- much appreciated.
[Edit: Alright, strictly speaking, you might not have ever gotten an email from me. Somebody might have just given you the link to this page, which is an online archive of an email that I sent to folks who had asked for it. If you'd like to get articles like this in your inbox, totally free, about once a week or two, give me your email address.]
I speak to many early-stage startups or people just venturing into a software business, and also speak to many more mature software companies. Everyone wants to talk about optimization: what to optimize, when, and how much. I've come to appreciate that the amount of optimization you need, and the type of it, changes as your company matures.
Obligatory disclaimer: I come from the perspective of someone where a successful outcome is selling SaaS to businesses for $50 a month or $50,000 a year, then repeating that as many times as necessary to achieve the business'/founders' desired goals, rather than as someone whose success trajectory is 1,000,000 MAU with 20% month-over-month compounded growth rate and monetization of 0.5% of them for an average of $2 each. If your market or capital structure requires something that sounds more like the second, well, have a wonderful time with your family over the holidays and we'll see each other next year.
Start With High Friction, High Bandwidth, Non-Scalable, Qualitative Optimization
Let's say you're in the pre-launch phase for your company. You have a landing page up collecting emails. How much effort should you put into optimizing that landing page to collect more emails? For example, how many A/B tests should you run on the call to action button copy?
Zero. Make sensible choices when writing the copy (e.g. "Get Started Learning About $TOPIC" will almost always be an improvement versus "Submit" on the button), but don't look back after shipping it.
Instead, you should be using every possible opportunity to talk to actual people. Yes, send out periodic emails about the topic. Yes, blog about it. Yes, have a Twitter/Facebook/etc. (Actually, I think you can probably skip that for B2B SaaS.) But talk to people.
How can I say that, given that I'm so identified with e.g. A/B testing and conversion optimization? Because in almost a decade of doing this I've never seen a company killed by failing to design a high-converting landing page prior to launch. I've seen lots of companies killed by launching a product that nobody cares about.
Especially as (often introverted) engineers and product people, we sometimes rely on quantitative optimization (conversion rates, A/B testing, Google analytics, SEO, all that jazz) because it is a good match for our skill sets and because it's the least uncomfortable thing about marketing/selling software. Just because it isn't uncomfortable doesn't mean it is the most valuable use of our time.
First Among All Things: Get One Dedicated User
Here's a sadly frequent conversation with very young software companies:
"How many users do you have?"
"We have a couple hundred people who signed up for our free trial."
"So how many users do you have?"
"Well, it being a free trial, engagement is kind of low. But at least a couple of dozen have played around with it."
"So how many users do you have?"
"... I don't quite get what you're asking me?"
"Does there exist, right now, a business other than yours which would be meaningfully impacted by this vanishing?"
At the earliest stages of a B2B software company, that's the only kind of user that matters. Whether they're paying is not maximally relevant, except insofar as getting people to pay makes them more likely to sink their teeth into the product, since they almost certainly aren't paying enough to support the company by themselves.
We only care whether they're actually using the software "in anger" -- that is, it is in production at their company -- and their business is beginning to rely on it. It can be a limited pilot, running at just one location or running in parallel with their previous workflow or whatever, but it must actually be running on real data generated by real customers with real money on the line.
Why am I so dead-set about this? Because the feedback from every other type of trial user is meaningless next to countervailing feedback from someone who actually relies upon the software. You're kicking the tires? That's great. This guy's firm would suffer embarrassment / his clients would leave him / his boss would fire him / his children would go hungry if this software didn't work for him. I trust his insights over yours ten times out of ten.
(You might think I'm exaggerating about the importance of your software. If you're doing B2B software, and it failing doesn't at least severely inconvenience someone, you've picked the wrong product or the wrong market. If a comet hitting your hosting facility wouldn't move any needle at a customer, why the heck would they actually buy your product? Many entrepreneurs choose to work on a less critical system as a form of risk management -- and believe me, when I get woken up at 2 AM to deal with failures of infrastructure products this seems quite sensible indeed -- but "less critical" doesn't have to imply "we really don't care either way about having it or not.")
How do you recruit your first user, with an unproven system that may not even fully exist yet? You offer to move heaven and earth for them.
- You'll handle the integration. Personally. It's like having an extra engineer working at their company, except without the impact to payroll.
- You'll handle all the support. Personally. The average software company doesn't even treat their customers like humans. You'll treat your first one like a jealous god.
- You'll give them an (informal) service level agreement that nobody will ever get from you again: they'll be made happy by any means necessary or the business will fail.
- You'll spend hours getting to intimately know the contours of their business and adapting your software until it absolutely sings for their particular use case.
Note that you're not necessarily offering a 100% functional software solution right out of the gate. Indeed, that's one of the beauties of having only one customer: you can afford to do ridiculously unscalable things. I like to think of it as the Wizard of Oz: you might perceive it from the outside as a product, but if you knew that there was a man behind the curtain, it might sound more like a services offering.
For example, ZeroCater is "software eats lunch" (well, OK, it doesn't eat lunch, it eats the manual effort required from every company which wants to offer lunch as a perk to employees, but you get the idea). These days, my understanding is they have a fair amount of software built up to support the business. On day 1, ZeroCater was one very gutsy founder, a spreadsheet, Gmail, and a cell phone. They didn't have a complex order routing engine with Twilio integration or heuristics which would adapt restaurant selection or menu choices to individualized employee preferences. They had one smart guy who'd make any decisions required in his own head and then write down how they worked.
This doesn't necessarily work for all software products (it is hard to imagine this for CRUD-heavy domains like project management or bug tracking, unless you just embed yourself in the target company as a project manager), but it's great for finangling the fiddly, infinitely customizable interfaces between software and businesses that form the heart of many B2B SaaS value propositions. Those are often difficult to predict, design, or implement prior to having a deep understanding of the pain points. Happily, pain is available in abundance when you do things manually.
Rob Walling did a similar thing with Drip: he adapted the email marketing tool to the needs of one responsive (and fairly demanding) anchor customer, then gradually onboarded new customers manually until it was ready for the prime-time. This is particularly smart on infrastructure products which, when they fail, cause pain roughly proportionate to the number of customers you have squared. Get the kinks out early, when the only users are people who like you and want you to succeed.
Speaking of infrastructure products: do you know what Appointment Reminder's failover strategy was when it shipped?
patrick@ar-production >> ruby script/rails console
appointments = Appointment.appointments_for_today
@appointment = appointments.pop
puts @appointment.human_readable_script
Dials telephone.
Hello Mrs. Smith. This is Patrick calling on behalf of Bugs B Gone Exterminators. Bob wanted to let you know that your appointment is at 4 PM on Thursday. Does that still work for you? Thanks much, I'll tell him. Have a nice day.
@appointment.confirm!
@appointment = appointments.pop
Suffice it to say that in addition to giving me opportunity to work on my phone voice, this also gave me a good psychological incentive for making sure that the system was reliable. (I wasn't 100% successful with that, and I think I've told the story of the epic blowup in this space before, but it worked well enough to get the company off the ground.)
Scaling From One Customer To "A Few" Customers
Most people will find their first customer in their own personal network -- an ex-employer, someone who works in their local community, etc etc. How do you find ten customers or a hundred? Is it time to break out the scalable marketing approaches like drip email campaigns, optimized landing pages, AdWords, and what have you yet?
Yes... but in an experimental manner. You'll start collecting email addresses and educating people scalably, but your primary customer acquisition strategy will continue to be killing people with kindness in a one-on-one fashion.
Why? Because the math is, otherwise, totally heartless. 3,000 visitors a month. 750 email addresses collected for your drip campaign. 300 read enough of it to become interested in checking out the product. 75 of them convert to a free trial. 10 pay if you're lucky, 2 to 3 in the more common case.
It is hard to build a successful business when you're getting out-hustled by the Girl Scouts. You're a highly educated software entrepreneur with the entire Internet marketing apparatus at your disposal. They're seven-year-olds selling cookies. And they're outselling you probably 28 days out of the month, assuming you're on your A game those two days that they're at ballet practice.
I know, I know, it goes against everything in my introverted soul to do this, but if 2 to 3 to 10 customers is a successful outcome, then you don't need to do anything more complicated than what the Girl Scouts will do. Walk that offering door to door, or meetup to meetup, or conference to conference, or whatever makes sense given your target customer and how dense they are on the ground.
One prominent successful example of this is Stripe. Stripe does payment processing, which is required by substantially every online software business and totally critical to their operations. Their early customer acquisition strategy was Patrick Collison, their technical co-founder. He'd take you out for coffee, tell you about their vision for payments that didn't suck, and asked you if you'd be willing to try it. If you were amenable, he'd hit you with a line so good it should be immortalized as the Collison Close:
Open up your laptop. I'll get you an account and code the integration right now.
That's beyond brilliant, in that it instantly hops over many potential sales objections ("I'd need to clear time in the engineering schedule." / "Hmm, got too many things on my plate this week." / "I need to clear that with the boss.") By construction, he was only targeting people who had the technical and organizational authority to pick a payment processor, so if they said Yes, that was game on.
Is the Collison Close ballsy? Certainly. You won't get e.g. a unit of a Fortune 500 company to say yes to that. However, as an early-stage software company, you're at no risk of selling to them anyhow so you don't have to cater to their preferences yet. You just need to Collison Close your way to enough people so that you can spend an hour talking to someone every week. (I think that's probably on the low end, to be honest.)
If you talk an hour every week with one customer, you'll wear out their patience and perhaps make them feel like you're selfishly monopolizing their time rather than solving problems for them. If you stripe (ba dum bum) an hour a week across ten customers, suddenly you're just their most attentive, responsive vendor.
What was your biggest pain point with payment processing this week? Oh, you needed a reporting feature. We're on it.
What was your biggest pain point with payment processing this week? Oh, a chargeback was filed and your support staff didn't have a great experience resolving it? Tell me more about how chargebacks usually work at your company. Great, that's so helpful. It will be better next time. This one is on us.
Getting those first ten customers doesn't just focus your efforts on building a product which actually solves a pressing problem for them. It begins to tell you about who your customers are, how they talk about the problem space, and what their main concerns are.
Now you can feed that back into your nascent marketing engine.
Want a great example for a first A/B test? Replace the H1 that you just threw on the page on day 1 with a promise to solve the #1 pain point your customers experience, using the exact words they use to describe that pain point. You won't always win with this, but it's so effective it is practically cheating.
Don't know what you could possibly write about for your blog? Ask one of those first ten customers -- who love you because you're uber-responsive to their needs -- if they'd like some free publicity. Write how their business uses your thing to great effect in their business. Get their approval (only takes a one-line email), then put it on the blog. That's radically more interesting to most of their peer businesses than the feature announcement or technical advisory you might otherwise think about publishing. Bonus points: it's an additional value-add for those early customers whose continued patronage is paying your rent.
Finding Successful, Profitable, Scalable Acquisition Channels
Successful B2B SaaS companies are generally different takes on the same cuisine rather than tasting totally different from one another. There's a lot of ways to cook Mexican but if you're in a Mexican restaurant you'll know it and, assuming you've been to a few, you'll recognize most of the menu. The same goes for B2B sales and marketing.
I won't say there are no new things under the sun, but if you ask the typical company with $50 million in yearly revenue how they got there, they're going to say "Oh we tried everything, including..." and then list the same things every blog post, slide deck, conference presentation, and strategy session about selling software ever has suggested that you try.
Typically each company can attribute a strict majority of their sales to one or two things that turned out to work really well for them. (This is where I'd love to pull in some great anecdotes from consulting but, for obvious reasons, clients are not totally on-board with Company X Gets 60% Of Sales Through Channel Y going out to 8,000 people.)
How do you identify which strategy in the playbook works for you? Breadth first search among strategies, followed by depth-first search for tactical optimization.
Take partner programs, for example. Like every other strategy, they work fantastically for some companies and totally fail for others.
I've tried them for Appointment Reminder, via allowing e.g. local technology consultants to white label AR and then sell it to their local business clients. I thought this could give me a scalable channel to attract new accounts: one-time engineering investment to allow white labeling, minor investment in onboarding for each new consultant, consultant then proceeds to sell N new accounts of AR at a cost of acquisition which is very viable for me. That was the theory. The reality was that results were very unremarkable. I spent about a month of my engineering schedule implementing the features needed to make it work. In the alternate universe where I instead went on a League of Legends bender, my revenue graph is virtually identical.
If you asked WPEngine, on the other hand, they'd tell you their partner program is phenomenally successful. They talk to WordPress consultants all day every day, telling them that the next time they sell a dentist office / travel agent / insurance company / etc etc on a new website, that website should be hosted on WPEngine, which will give the consultant an additional revenue stream and get them out of the stressful business of waking up at 2 AM when the site goes down. WPEngine puts non-trivial amounts of effort just into getting more leads for the partner program, for example by attending WordPress events, publishing educational material of interest to WordPress consultants, and so forth.
Could you have known that AR was going to fail while WPEngine was going to succeed, both using decent first implementations roughly the same strategy? That isn't obvious to me. I can retrofit an explanation to those results, but I'm unsure whether it would have explanatory power prior to the fact. Actually trying the partner program (as one of a dozen things, all with highly uncertain payoffs) was probably a more effective use of my time (and WPEngine's) time rather then endlessly theorizing about what strategy would turn out to be optimum. Shipping wins.
If you have early inklings of success on any particular channel -- whether that be AdWords, organic SEO, content marketing, drip email campaigns, inside sales, partner programs, etc etc -- you can almost certainly optimize it profitably. On the other hand, I very, very rarely see companies take a channel which is a total failure and tweak it until it is successful. If I wanted to rescue my white label program for AR, that would probably be an involved project and likely involve a from-the-ground-up rethink. (Hmm, I think I just convinced myself to kill it. Thanks guys. Adding that to the todo list.)
Inklings of success don't necessarily have to be home runs on day 1. If your cost of acquisition for e.g. AdWords is $120 and your LTV is $100, then it is conceivable that by adjusting a few knobs you'll get that cost of acquisition down to $100 and the LTV up to $120, after which point optimization quite literally pays for itself, and you might well further improve things such that the numbers are $60/$200, at which point you'll probably green light enough of an AdWords spend to have Google send you a nice card around the holidays.
(Hah, just joking, Google doesn't actually do personal touches. I think my card last year said "Welcome To Holiday Cheer. Your Culture's Distinctive Celebrations Will Be Assimilated And Added To Our Own. Resisting A Customized Discussion With Your Dedicated AdWords Representative Is Futile.")
Your most successful acquisition approaches often don't start looking like "Hmm, that was a modest win" or "I suppose with some effort that could be made to work." They're often surprisingly, outlandishly successful. I have a bad case of business ADHD, so especially early in my career I often tried new things rather than sticking with the stuff that was working. These days, I'd advise doubling down -- heck, thousanding-down, if the metrics stayed right -- on success.
This is, by the way, the first stage of life in your business where metrics really start to matter. One common form of advice for early-stage startups is to have Google Analytics installed on Day 1, so that you can start collecting a baseline for metrics like traffic and conversion rate. Let me spoil the surprise: on day 1, your traffic is minimal and your conversion rate sucks. The exact numbers are totally irrelevant to you! Sure, three years from now, having them will make your graph a bit more impressive, because either they'll have improved or you'll have long since died, but the trajectory of most successful software companies looks pretty much the same if you chop off N months of Mists of Pre-History. (For almost any N, too.) If you're on the hockey stick (and you probably won't find a hockey stick), it makes the stick bit a bit shorter and the blade slightly wider. If you're on the vastly more common Long Slow SaaS Ramp of Death, it modestly affects the visual perception of slope but doesn't change the overall picture.
Optimization At Mature Companies
So what do you do when you know you have successful channels? In addition to continuing to execute on all the things that make the business great -- improving the product, providing excellent customer support, making the company a great place to work -- you start systematically exploiting your best marketing/sales channels via scalable, quantitatively-driven optimization.
Optimization was my first love in the software business. To be perfectly honest, I may have loved it too much -- Bingo Card Creator quite literally had more effort thrown at tweaking its knobs than many companies 100 times its size or importance. (Paraphrasing Joel Spolsky: "Imagine what you could do with that skill if you applied it to a product which wasn't totally bullshit.") I then parlayed that experience into a fairly successful consulting business, going around to companies like Fog Creek, WPEngine, Matasano, and the like, and doing similar forms of optimization for them. Most of my consulting clients were quite happy with what they got out of that arrangement.
Consulting made for a fun ~3 years, but I quit earlier this year, which cleared up lots of time to work on my own projects.
I'd like to share a sneak peak of one with you guys. This January, I'm releasing a course about conversion optimization and A/B testing for software companies. (I know, it's been a while coming -- sorry, the spirit was willing but the body was hospitalized.)
It's five hours of videos from yours truly, and a few guests at successful software companies, discussing both how to get started with A/B testing and conversion optimization and how to take it to the next level, by making it a repeatable process which makes your company even better every month rather than a fun experiment you do once and then forget about forever. (Don't feel bad -- many, many companies have that experience.) We'll discuss both the theory and actual examples of real companies running real tests on their pages. As usual for me: no fluff, no nonsense.
Additionally, although I'm not typically available for consulting engagements anymore, there is an option there by which you can get customized advice from me for your company's particular situation. It's priced at a substantial discount to my old weekly rate.
If this sounds interesting to you, good news: it's on sale now, for delivery in January. Check out the details here. (I don't usually do pre-sales, but I figured the end of the year is a good time to make an exception. You get to book expenses this year and I get to book revenue this year, which is to our mutual benefit come tax time. I figure this is worth about +/- 20% in marginal tax savings to me, so if you pre-order it before December 20th, I'll give you that as a discount. It also helps pre-commit you to your New Year's Resolution, if your New Year's Resolution is "For the love of Pete, this is the year I'll finally start A/B testing, darn it.")
If you're intrigued by A/B testing but don't want to commit to buying anything yet, no problem. You can still take a free email course that I wrote on the subject. (If you're already signed up for that, you don't have to sign up again. More lessons coming soonish and, again, sorry for the delay.) You'll also hear from me again when the video course is ready. (Though you won't get the discount unless you buy by December 20th. After that, my business shuts down for Christmas through the New Year, and as per my usual I will only worry about the business over Christmas if there is a literal fire involved.)
If you're not interested in A/B testing, because it doesn't fit the stage your business is at or for any other reason, that's totally fine. I'll be back to the usual eclectic mix of topics after launching this.
Until next time.
Regards,
Patrick McKenzie
P.S. Copywriting micro-optimization tip #387: after the subject line, the P.S. of an email is the part most likely to get read. If you hypothetically sell anything in an email, such as a new video course on conversion optimization, you should probably mention that in your P.S. such that people who skim the mail don't miss it.