Dan Siroker at Startup School 2013

Thank you. Uh, I have the privilege of saying this is my sixth Startup School, uh, and, uh, the first time as a presenter.

Today, I'm going to be sharing with you what I would have wanted to hear the first five times, uh, while I was sitting in your seat. So, I'm going to start by telling you about my story, uh, a story of my entrepreneurial career. It starts while I was in college, and I had the opportunity to do research in machine learning and sentiment classification.

What I did was try to use that technology to apply it to the stock market. And so, this is an article that Stanford Magazine published while I was a junior at Stanford and talks about how I created a computer program that pours through newspaper articles, gauges how well a company is doing based on the amount of positive or negative language used to describe it, and bets accordingly. So far, so good according to the article.

Well, here's what was actually going on. The year after I graduated, I focused entirely on this one startup by myself. I was actually, um, starting this company, and this is the algorithm used. I did a bunch of research... profit. And, um, we didn't quite achieve profit. Uh, by "we," I mean me, because I was by myself.

This is a graph that shows the amount of time between when I first wrote a line of code for this company and the time we got our first paying customer. For those in the back, that's not eight months; that's infinity months. So, this was a very valuable lesson for me in recognizing the value in having paying customers.

I decided, uh, to stop working on sentiment solutions and go to Google. My goal at going to Google was to try to learn the skills that would make me successful as an entrepreneur. In particular, I wanted to understand what to do in the... dot dot dot. And today, I'm going to share with you what I learned both at Google and then several other startups.

Since this is my algorithm at Google, uh, I basically had the opportunity to build some great products. I was one of the first product managers on Google Chrome, and so we built a product, uh, Google Chrome. A good strategy that Google has for distribution is to put the word "Google" in front of the products they build.

By doing that, you end up getting about 100,000 people trying your product right off the bat. Uh, that was very valuable, uh, not only because if the product was any good, you would grow much faster, and you'd be successful, but because of the feedback that those people would provide as you're incrementally improving the product.

This is the biggest lesson I learned while at Google as a product manager: the value of a feedback loop. This is one critical thing I was missing at sentiment solutions because I didn't have any customers to tell me what they liked or they didn't like. I didn't even try to sell the product, so I had no idea what I actually should be building.

While I was at Google, I got lucky enough to sneak in and see Barack Obama speak in November of 2007. At the time, he was a senator, uh, running for president. He hadn't won the Democratic nomination yet, and he came to Google and gave a talk that described how he wanted to take what we were doing at Google with evidence, science, feedback, and data, and bring that to the government.

After I saw this talk, two weeks later, I flew to Chicago and signed up as a volunteer. So, I want to share with this audience a few short clips of what Barack Obama said in November 2007 at Google and what inspired me to quit my job, uh, fly to Chicago in the dead of winter and join his campaign.

"What is the most efficient way to sort a million 32-bit integers? Well, uh, I'm maybe I'm sorry, maybe I think I, I think, uh, the bubble sort would be the wrong way to go. I am a big believer in reason and facts and evidence and science and feedback—everything that allows you to do what you do, that's what we should be doing in our government. Me, I want, I want, I want people in technology. I want innovators and engineers and scientists like yourselves. I want you helping us make policy based on facts, based on reason. And I think that many of you can help me, so I want you to be involved. Thank you so much, everybody."

Well, he had me at "bubble sort," and I decided to fly to Chicago and sign up as a volunteer. I was lucky enough to be able to join a team, uh, that was called the New Media team. While I was there, I got a opportunity to run several experiments called AB tests, and, uh, those were quite successful, so they gave me a job as the Director of analytics.

In this job, my mission was to try to figure out how to use data to help make better decisions on the campaign. Here's a picture of our team. We were part of the New Media analytical team, and New Media was the phrase the campaign used to describe everything they didn't really understand.

Uh, so if it wasn't TV or radio, you fit in New Media. And we had the most monitors per square inch of any part of the campaign. Here's another photo of our team. I'm there in the middle with my back to a TV. This is during the Democratic National Convention while Bill Clinton is giving the keynote speech, and the broader New Media team is watching while I'm busy setting up an AB test comparing a picture of Bill Clinton to a picture of Barack Obama.

In this photo, please ask you to ignore all the Bud Lights. So at the campaign, I learned quite a bit. I learned the value of AB testing. We used products like Google Website Optimizer, Omniture Test, and Target, now Adobe Test and Target, and got a ton of value out of those products. But we were constantly bottlenecked on requiring a developer to be part of the process, and so that pain was the original inspiration for starting Optimizely.

It took me a while to come to that realization. We actually started several other companies before that, and so I'm going to tell you those stories. After the campaign, um, I came back to San Francisco. I convinced a good friend of mine, Pete Kumin, who is a product manager at Google as well, to quit his job and to start a company with me.

The first company we started together was called Carrot Sticks, and it was an online math game for kids. This made it easy for kids to learn math in a social way—create an avatar, be able to compete with one another—and it was for us an opportunity to try to apply technology to something we really cared about, which was education.

This was a product we were really, we were still very proud of. It's, you know, still exists out in the world. You can go to carrotsticks.com and beat up on a bunch of small kids playing math. It slowly loses money over time, and we learned a valuable lesson in building Carrot Sticks.

And this is the algorithm, uh, that describes our time on Carrot Sticks, um, which was that we would basically build the product, sell the product—in this case we were selling it to parents—take their feedback, and then try to make our company and our product better.

One challenge here was we were selling to several different constituents, of which we were none. We weren't parents, we weren't, uh, teachers, and we weren't kids. And so this made it very difficult for us to know how to prioritize what to do and what not to do.

As a startup, it's really, really important to focus on the one or two things that make you really, really valuable and unique. So in that process, we learned the value of the feedback loop. We could get better over time, but because we didn't really build a product for ourselves, it was very hard for us to understand how to prioritize.

We also had a really big challenge with distribution. It's very difficult to get parents and teachers to adopt your technology. And if you do, you have to do it one at a time. There's no scale, there's no leverage in distribution. And so, uh, here is a graph of, uh, now our second venture, or my second Venture, Carrot Sticks.

This is the time between our first line of code and our first paying customer. So we, we improved dramatically from infinity to six months. In that, we also had the opportunity to recognize that even though we were getting some customers, the pace at which we were growing wasn't going to justify a large impactful company.

So we learned a lesson here, which was to now we decided to try to build a product we wished we needed or wished we had in Carrot Sticks. And so that's when we started Spradley. Spradley was the company that we applied to Y Combinator, uh, in the winter of 2010.

I want to share with you the short version of the demo video, or the video we submitted in our Y Combinator application, that describes our experience with distribution and Carrot Sticks and then why we were inspired to start Spradley.

"Hi, I'm Pete, and I'm Dan. We're both former product managers at Google and studied a bunch of computer science stuff in college. I also worked a little bit on the Obama campaign and then did the presidential transition. Now doing a startup with Pete here in San Francisco called Spradley. So we started working on Spradley about four or five months ago, and the biggest problem that we ran into was distribution.

So, uh, getting people to use the software, getting people to talk about the software with their friends. And so we started thinking very hard about that problem and how we would try to solve it. We came up with an idea that we're calling Spradley. Spradley is, uh, very simply a discount a merchant can give to a customer if that customer is willing to tell their friends through Twitter, through Facebook, through email about the product that they just bought.

Uh, today it's live on Spradley. You can buy a subscription to Spradley, and if you do that, you can get a discount if you tell your...”

So this video, I wanted to share with you because I think it shows a couple things. One is that the human body can only withstand one presidential campaign. Here, I had this is an image of me after gaining 50 pounds on the campaign from just eating deep dish pizza and drinking beer every night, so I wanted to share that.

And we also had the opportunity while working, uh, at Spradley, we now wanted to really focus on this feedback loop and getting distribution for our own product. So we built a product, Spradley, that enabled what we just described. It took us about a month to get our first paying customer.

We were also very critical and very, uh, focused on understanding whether or not this could be a big sustainable business as fast as we could. And so, uh, that's part of the mantra of Y Combinator, which is to build something people want.

I'll show you in a second, uh, the algorithm we used to do that for Spradley. We, uh, we realized, uh, after joining Y Combinator, we got—this is the algorithm—pretty straightforward: you get some money, and you keep trying to build something people want. You learn and you improve, constantly improve the product based off the feedback from trying to sell it. And if it's no good, if it's not—if anything less than insanely great—you come up with a better idea.

With Spradley, the one insight we got pretty quickly was that the fundamental model didn't work. The social capital it took for somebody to spam their friends about a new product or service was almost always worth more than any amount of discount or incentive a business would be willing to pay to get them to do it. So fundamentally, the product didn't work.

So after a month, we started thinking about what are some other products we wish we would have had. This is a lesson we learned with Spradley. Boy, it was easy to build Spradley because we would have wanted that in Carrot Sticks. And so we learned that with Optimizely.

Optimizely is the product I wish we had in 2008 to make it easy for anybody to do AB testing. And so we started Optimizely. And, uh, I remember the first time I even had a conversation about Optimizely.

Uh, we actually spoke to a person I worked with on the Obama campaign, Andrew Blecker. I called him on the phone. This is why we were working on Spradley, and I pitched him the idea for something that would enable his team, without technical resources, to do AB testing—the same stuff we did back in the Obama campaign, but with a visual interface, one-time implementation, no technical resources, and a ability for them to continue to iterate and get better over and over and over again.

About 20 minutes into the phone call, Andrew stopped me and he said, "That sounds great! Send me an invoice!" He thought this was a sales call. And I responded by saying, "Well, how much do you think this is worth?"

He said, "Oh, about $1,000 a month sounds right," which is way more money than we ever made on Carrot Sticks or Spradley. So what we realized, we had our—the time until our first paying customer was one day. We actually had our first paying customer before we wrote a single line of code.

That Tuesday, thank you! So that weekend, I built the first prototype, and that Tuesday at dinner, I showed it to Paul Graham. He looked at it, got really excited, pointed at it, and said, "This, this is it! This is it! This is AB testing for marketers! Forget that other idea, do this!"

And so we followed his advice, and we did this. The first year of Optimizely, we grew to a team of four people and an annual run rate of 1.1 million. Uh, we did Y Combinator, hired, uh, my brother as our first engineer, and we had some great press on Tech Crunch, top of Hacker News a couple times.

This entire year, we were focused on the product—continually improving the product—and it was so much easier to do that because we were building a product I would have wanted myself back in 2008 on the campaign.

And so we ran a pretty simple algorithm. We kept trying to sell the product. Me and my co-founder Pete were the main salespeople. We kept selling it over and over again.

And in these conversations, potential customers would ask us, "Oh, this is cool, but can you do targeting? Can you do analytics integration? Can you do traffic allocation? Can you do X, Y, and Z?" And we would take that as input, and initially, we would respond, "Well, we're not doing that, uh, right now, but that might be something we do in the future."

And that pattern kept continuing until that phrase—that response changed from, "Oh, actually, great question! Here's exactly how you do that!" And we would show them in the interface, "Oh, you want to do targeting? Here's how you do that!"

And that tight feedback loop of listening closely to prospective customers, knowing ourselves what we would have wanted in 2008 made this a much, much easier journey than Carrot Sticks, Spradley, or even sentiment solutions.

The next year, we grew to a team of 10 and an annual run rate of 1.2 million—about a 9% or a 10% increase in revenue. And this year, uh, was the first time we hired a salesperson.

So we had been doing all the selling for the first year and a half. And one of the things I learned this year, which I think is really valuable as you get bigger and bigger as a company, one of the most important things for me is to focus on hiring.

So we had a pretty standard algorithm for hiring, which was we would interview a candidate, and we would ask ourselves, "Is this candidate better than the mean? Are they better than the average people, the average people who already work at Optimizely?"

And if we strive for this goal, we will constantly get better, at least stay as good as we've been when we first started. So this is the algorithm we use: we said every candidate we hired we would ask ourselves, "Are they better than the mean?" And if so, we would hire them.

As we actually got better at hiring, we would improve the process in our startup. We really focused on continuous improvement. We made not only a feedback loop in the product but a feedback loop in our hiring machine, which was critical because in the next year, we actually grew substantially.

By the end of the year, we were 42 people and an annual run rate of 7.6 million. We also, in this year, had a huge opportunity when it comes to press. In 2008, we had a couple of stories that were really appealing to reporters.

The first was the Obama campaign and the Mitt Romney campaign were in full swing, and we were lucky enough to have both of them as customers. So, the Obama campaign in 2008 came full circle, and that was really gratifying.

It was also very gratifying that the head of the digital team for the Romney campaign said that the hardest decision he had to make was to use Optimizely for AB testing, knowing the origin story of it coming from the Obama campaign.

So we were thrilled about that story, which was really helpful. We got on CNN, we got on a bunch of other mainstream media outlets. But then there was another story that was also really appealing to reporters, especially tech reporters, and that story was this David versus Goliath story.

So we would spoon-feed them this idea that we are the, uh, David in this battle against Adobe—the big evil Adobe—who, at the time, had a product called Omniture Test and Target, which I used a lot in the Obama campaign.

This story, uh, was very popular, and it actually stoked some fires with Adobe, which we were really proud of, and it also gave us a lot of press and legitimacy. We had many of the customers who were beaten up and tired and frustrated with, uh, the incumbent move to us as a product.

And in some ways, their response really validated our existence, and so that brings us to today. So if you take this graph and you shrink it down, and you add a couple more quarters, you'll get to where we were. This is where we were at the end of the year. We had a team of 42, and this is where we are today—we have a team of 130.

We've now launched, and we have the product available in 10 different languages. My co-founder and I actually wrote a book, uh, to try to get distribution called "AB Testing: The Most Powerful Way to Turn Clicks into Customers."

And we've really continued to focus on all of the things that made us great from the beginning—constantly trying to improve. Uh, in this year, we also realized that our opportunity as entrepreneurs is not just to run the same algorithms every other startup or every other business runs, but to define our own.

In particular, one thing we did in 2012-2013 that I think is pretty unique is the process we used to raise our Series A. The process we used looks something like this: we would host a mock board meeting with potential investors.

So we had tryouts. We actually spent three hours with each of the partners that we thought were finalists, had them spend time with us, had them spend time with our management team, and really put them through the paces of exactly what they would be like as board members for our company.

This is a completely different setting than most entrepreneurs have when they're working with an investor, which is over dinner or it's at a pitch meeting down on Sand Hill Road. From this, we realized what we really wanted in an investor.

And, in fact, we found an investor we never thought we would have wanted from the beginning because he wasn't technical, he didn't have operational experience, and he wasn't all of the things we thought we wanted in the board member.

But what he did have was a very, very good way of helping us get better, which was in the feedback he gave us in this mock board meeting. He focused on asking the right questions, not prescribing the right answers, and that really impressed me and my co-founder Pete.

And so, we asked Peter Fenton from Benchmark to join us on the board, and, uh, we haven't looked back. We've been very happy with that decision.

So this is what the product is today. I'll show some quick screenshots. Today, through this process of continuous improvement and innovation, we built a product that allows you to put in any URL into our homepage, open up in a visual editor, click on any part of that page, make a change to it, and then run an experiment to see whether that change improves your conversion rate.

This fundamental model is the same model that now we're applying to other mediums as well, and we're really excited to continue to grow, enabling every business to use data to make better decisions.

The vision for our product is to enable the world to turn data into action, and we think AB testing is a great first step toward that. One of the things, again that has helped us quite a bit, as I said earlier, is having customers who give us great feedback.

But not only great feedback, but our customers who not only are delighted but are evangelists for our company. So, we have a great sales team at Optimizely, but our real sales team—in fact, our biggest secret to our success—is the sales team of the 5,000 happy customers who sing our praise wherever they go.

A word of mouth referral from the director of marketing at Starbucks is much more valuable than even the best sales guy at our building. So, these are a bunch of algorithms I just showed you. A bunch of algorithms.

And some of you are probably computer science students here, and you might be noticing a pattern. This pattern is something that only I recently realized, and this is what I'm describing, actually, as in my mind, the universal startup algorithm.

This is an algorithm you can apply to almost any decision you have to make as an entrepreneur, and it's critical for you as a company to get better over time. And this is what that simple algorithm is: everything you do—get feedback, use that feedback to make you better, focus on continuous improvement.

You're not going to make the right decisions. You're not going to do the right things in the beginning, but constantly trying to get better is the only way you're going to build a lasting company.

This is something that I've seen now from hiring to raising our Series A to building a product and to getting press. This algorithm is fundamental to all of the greatest companies; they've all focused on continuous improvement.

The second thing I'll say about this universal startup algorithm is that a common mistake many entrepreneurs make—and I make this mistake all the time—is they get trapped in something known as the activity trap. They think their job is to execute the algorithm, constantly execute the algorithm.

Your job as an entrepreneur is to write the algorithm. A great example of this is Tesla. Tesla has decided that they're not going to do the same distribution model of every other car company in the world. They're going to sell cars directly. They're not going to negotiate prices, and they're going to cut out the middleman.

Elon Musk defined the algorithm. He wrote the algorithm for his company, and now that's going to be a huge advantage for them as they continue to grow. Thank you very much.