By Rodrigo Martinez

The market for developer tools is a deceptively large one. More and more great companies being built in this space — and Point Nine is proud to be investing in some of them like Algolia, Contentful, and Sqreen.

In this series of interviews, we’ll explore the early days of some of these companies. There are valuable insights to be found here for anyone building (or interested in building) a business in this space.

Here’s my first interview, with RainforestQA CEO Fred Stevens-Smith.

RainforestQA is all about helping companies with quality assurance and testing. They were part of the Y-Combinator 2012 class, and recently raised a Series A of $12 million, which was led by Bessemer Venture Partners. If you want to learn more about his journey, check out this episode of Flyover Labs.

Me: Could you tell us about the early days of RainforestQA?

Fred: Originally I was a designer, and decided that I wanted to start a tech company. My cofounder was one of the first AWS experts in Europe. So, together we saw a great opportunity to start a company in the developer tools space.

We liked dev tools for several reasons:

There was a clear need in the industry to increase developers’ productivity- people were already starting to invest in tools to make them more efficient.
It was less risky than trying to build the next ‘cool’ app, because companies always need tools to help them build software. We thought that building a core piece of infrastructure would allow us to diversify our risk. The guys who got rich during the gold rush were selling shovels.
We looked at the industry and saw that there were few competitors in this space.

We got accepted to YC, and after several false starts working on tools that people only wanted to pay $10/mo. for, we started looking at opportunities to generate more value and therefore charge more. We realized we didn’t want to start building a company from the very bottom of the market, but rather, start at a higher price point — it’s just so hard to get to 100mm ARR at $1000 ACV.

At that time, we were doing a lot of customer development with YC companies. For example, we asked 60 founders, “What problem do you have that you’d pay $1,000 a month to fix?” Not everyone replied, but those that did referred to QA & testing as a pain point.

We looked at the competitive landscape and we saw that in the testing space, most products were really shitty, had been built as productivity enhancements for QA teams or as testing robots so that devs could speak code. None of them were fit for purpose, and we felt like we had a differentiated and potentially exciting approach.

So, after 4 pivots, we started to feel that we were onto something.

How did you turn that opportunity into a business?

Before writing a line of code, we carefully researched all the solutions in the market to do testing. Our starting point was: The future is AI, but how do we get there? There are many automation testing tools, but they weren’t so great and it took a fair amount of energy to get them ready.

Since our premise was to build something that people were willing to pay good money for, we started charging our early-adopter customers 1k a month to take care of their testing, even before having a real product. Obviously, after a while, we ended up losing all of them but one. We learned a great deal by doing that.

Doing things that don’t scale is core to figuring out initial product market fit.

We started doing testing for those 10 companies manually, which was a labor intensive process. We had to grow our team to be able to handle more testing. At the same time, we also saw some consumer startups (Homejoy, Uber, etc.) using crowds to solve problems in a distributed manner. A good friend of ours pointed us in the direction of Mechanical Turk, and we never looked back.

And what about the first customers?

Being part of YC brings you many benefits. One of the most important ones is that you’re part of a select group of early adopters. Everybody is optimizing for growth and everybody trusts each other. Therefore, it was probably easier to get those 10 first customers because we were within the YC circle of trust. That said, you can reproduce this approach with your own local network, be it a coworking space, an investment portfolio or just friends in the industry.

All of our early adopters wanted to be focused on growth and speed rather than QA. This was the start of a trend that has driven most of our business since. We persuaded those first 10 customers to offload their testing to us, rather than have their devs spending time testing. It was a pretty easy sell — no dev wants to spend time writing tests, and the prospect of pitching hundreds of investors at the end of the YC program has a funny way of focusing the mind on growth.

We didn’t over-engineer our initial pricing, our $1000 per month number was purely a threshold to find a problem that was big enough for companies to pay us decent money. We are also lucky that the substitute good of Rainforest QA tends to be a human on a company’s payroll. And that’s expensive! Almost the most expensive way to solve a problem.

Therefore, our potential customers have a higher willingness to pay us compared to some other dev tools where the benefits are productivity — which tend to be harder to quantify.

It’s relatively easy to go to a company and say “look, you’re thinking of hiring a person to do QA. Why not try our service for half the cost?”

Working on different messages

Tell us about your sales and marketing strategies.

As I mentioned before, our first 10 clients came from YC. And although we ended up losing all of them but one, we learned a lot of valuable lessons during that process.

Initially, and even now, most of our leads come from referrals. Our first customers started to spread our product among their friends and networks. One of the benefits of being in San Francisco is that developers from different companies hang together. If you build a great product, they will speak about it and word will get out.

I should also point out that initially we did pretty much anything to get uniques, including some relatively controversial blog posts, such as MongoDB gotchas & how to avoid them. It wasn’t a proper content marketing strategy, but rather us being very vocal about our view. That also brought a fair amount of leads. One of those articles, even today, gets a couple of thousand visits a day.

On the pricing front, we tested a lot of different things. We didn’t do much research for price optimization, but instead preferred to make hypotheses and go out and test them with prospects. We weren’t afraid of changing prices. I see many people struggling with that.

For example, we introduced a $99/mo. plan because it sounded ok, but we quickly saw that this attracted the wrong kind of customers; those customers were churning faster and creating more noise. In this world, I think you need to have an opinion and not be afraid of testing it and changing it if you’re wrong- the worst thing, I think, is to refuse to ever take an opinion.

At some point, we saw that among our early adopters there was a group of people who were very happy with our solution. We saw that they were more willing to commit, paid more and stuck around longer. That group of startups had something in common: founder CTOs who had previously worked for large companies. They knew the pain of building a large QA & testing team and they were willing to give Rainforest a try. If they could forget about building that team from day 0, then they could save a lot of time and focus their energy elsewhere.

These types of customers were benchmarking us against the cost of running a team, and we quickly realized that we could charge a lot more for Rainforest. Figuring out the substitute good for your product is so valuable since it gives you a range to price within. We started increasing prices — actually, we doubled our prices several times in a year to test how much people were willing to pay.

It was getting more expensive every month…

The YC customer who survived our beta phase was the first one to expand into a 6 figure deal. We were also amazed when we saw another customer coming in, trying Rainforest for a couple of months at 1k a month, and then jumping into 6 figure deal.

Again, in both cases, the same profile: experienced CTOs. In this case, running a multi-product project for a government agency and not willing to build a QA team.

After that, we started to see that this is a common path for new customers: they test with a small budget and then quickly scale up.

Tell us about the evolution of your role and the team

Our initial team was comprised of all tech roles. And for those initial hires, we had a very simple rule: we were looking for core contributors to large OSS projects. As n00bs hiring our first team, we felt that in the worst case scenario at least they loved coding and they were pretty good at it. Our first 3 hires were developers, all of whom came through Hacker News. This is an amazing source for quality candidates. It’s so good, that almost all of our 30 first employees came from there.

I should consider myself our first non-tech hire, because I have a background as a designer, but our first real non-tech hires were all salespeople. As soon as we started closing some sales, we did 4 first-sale-guy hires. Yes, I say first sales hires because each of them was failing for a different reason and it took us some time to figure out what our first sales folks should look like.

The last one we hired is amazing and eventually became our VP of Sales.

Something similar happened with regards to marketing. We hired 2 VPs of Marketing and ended up firing both of them. Marketing in the dev tools space is initially a founders job. Developers are trying to minimize the risks they take with their tech choices, and are intensely interested in brand legitimacy. They need to know who you are and why you wouldn’t fail them, and they need to respect you technically and ethically to try out your product. I don’t think you can hire a person to do that very early on in the life of your company.

We made a great hire when we were 11 people: our office manager. If I were to start again today, I would hire her as a 2nd or 3rd person in the company. It’s amazing how much she and her team do to keep the company running. She takes care of finance, ops, hiring, etc. You need to find a great person for that role, and really early. Somebody who knows how to deal with the chaos and uncertainty of startups. Then, you must give them ownership of the tasks and allow them to run their team.

What was fundraising like for you?

Fundraising is painful. Especially when you do your first company there’s a lot of chicken and egg conversations. Initially, we raised largely because of the YC brand. We were lucky to have a few of the YC partners that believed in us and the idea, and that got our first rounds started.

At later stages metrics start to count more, and it becomes harder and harder to fit into the mold that 10x VCs are looking for. One thing I do want to say clearly: fundraising is hard. Always. People that say ‘oh with this current seed froth it’s so easy to raise money’ have not raised money. It’s always hard.

As a founder, you feel that you control your world. You own your destiny. But fundraising is one of the few areas where you have very little control.

Compare it with doing sales — which is also out of your control. At least after you close a decent amount of customers, your process becomes more repeatable and you know your conversion rates. With fundraising, the biggest problem is that nobody says yes or no- it’s always a maybe, and that’s frustrating. But you need to keep going, keep pushing until it gets done. That’s your job as a founder. To keep the company capitalized.

What advice would you give to other founders building dev tool companies?

1) Use NPS as the driver for the company from day zero.

When we started Rainforest, we were very opinionated. We had an opinion on what the future would look like and we wanted to ask our customers to follow our views. Obviously, that doesn’t always work.

We should have been more customer-centric from the beginning. It would have made everything a lot easier, and created less friction along the way. NPS is the tool for that, and we started measuring it late.

Today, if I were to start again, the 2nd feature we would ship would be an NPS survey.

2) Founder-centric marketing.

As mentioned before, do NOT outsource that task to somebody else in your early days. The developers who are taking the risk of using your tool want to know you, want to trust you.

3) Move to SF.

That’s where your customers are. Here the critical mass exists for the referrals to happen. And the amount of executive talent is just incomparable. Finding great VPs that can bring your company to the next level is extremely hard, but at least in the Valley you have a pool of candidates to choose from.

What are you most proud of?

After a few early mis-hires, we sat down as a team and took a look at what we valued, and what was important to us. We looked at the people on our team and we tried to summarize what we had in common, what worked for us and what didn’t work for us with the people we let go. We narrowed it down to the following 3 values, which are now our cultural foundation: No BS, Be Weirdly Passionate, and Always Be Caring.

Defining your values enables you to build a cohesive culture through hiring the right people. The only tool you have to edit culture is hiring and firing. By ensuring that everyone you hire embodies a common set of values you can build a diverse company where everyone gets on brilliantly. I believe we’ve managed this at Rainforest and that’s easily what I’m most proud of.

Are you building a dev tools company?
I would love to meet and discuss!

Please reach out to me at @DecodingVC.

The Early Days of… RainforestQA