Developer Program Metrics: How Intuit Measures Developer Success

How do you grow a successful developer community around a financial product? Intuit’s Quickbooks is a well-known small business accounting software that has 100s of accounting applications built on top of the platform, from tax planners to CRM and HR apps. We spoke to Intuit to better understand the mechanisms the company uses to make sure its developer ecosystem succeeds.

Originally a desktop accounting software, QuickBooks has moved to a Software-as-a-Service model with a clear vision of the core product features it can enable through its service model. But they also know that customers often need additional functionality, so they have created APIs to let third party providers build those features.

QuickBooks found that by offering a range of app integrations with their main product, customers spent less time moving data from one system to another and saved time, increased their satisfaction with the product, and ended up looking for more ways to use QuickBooks in their daily business systems.

In 2014, when QuickBooks API program was in its infancy, they offered up to 50 third-party apps, and began to see the impact this was having on their customer lifetime value. They decided to invest more heavily in their API program and now they have over 450 published apps (and over 1,200 in total, including private integrations) and thousands of developers building on the platform.

The network of app developers has provided a wide choice of high quality app integrations to small businesses using QuickBooks, solving important problems in its user base. This has helped QuickBooks strengthen relationships with its existing customer base and also grow the base.

The QuickBooks Online platform is now growing a two-sided “network effects“ platform by delivering benefits to both developers and small businesses.

We spoke to Ketan Kittur, Director of Product Management and Partner Integrations at Intuit to understand how Intuit ensures that its app developers are successful. “The two-sided value of our platform is critical. We want to be serious about measuring retention and quality, so that we can give our customers the best experience and outcomes for running their business. We also want to understand the value these integrations are bringing to our product,” Kittur says.

VisionMobile: How do you first identify which developers need to be targeted in a developer program?

Ketan Kittur: We look at what problems our small business customers are facing that are not part of our product roadmap. Our developer team then scans the developer community – local and abroad- for potential solutions to the problems. The identified developers may already have solutions or they may include developers who are interested in building applications to address these problems. Our outreach team then works closely with the developers and guides them through the integration process, so their app integrations are of high quality delivering value to the customers.

Through these discussions, we have found that there are two main things that matter for developers:

  1. They are also entrepreneurs. They are serious business people building their own products and customer base.
  2. They do not want to waste a lot of time. Their development effort needs to return value.

VM: How do you align building a developer program with overall business goals?

KK: Our goal is to help our small businesses be successful using QuickBooks. For problems that are not solved by QuickBooks, we need a thriving a developer community delivering high quality app integrations so we have a highly engaged user base of QuickBooks users. Hence, the goal of both our developer team and developers are aligned- solve critical problems for small businesses using the QuickBooks platform and grow our businesses. When your company has a global footprint like Intuit, we become an attractive platform for developers, allowing them to grow beyond their geographic reach and inspire them to solve for problems that may exist in certain parts of the world.
With a customer-centric focus on everything we deliver, quality becomes an extremely important consideration. We use specific metrics to measure the apps in our app store, examining the number of users/connections the app has, the average app rating, customer reviews as well as churn metrics. These metrics help us figure out which apps are doing well and how we can help developers become even more successful by solving issues with their integrations.

VM: How do you measure the success that developers are able to achieve by being part of your developer program?

KK: There are two key things we track to measure success:

  1. How we can make it a lucrative option for developers, and help them grow their business.
  2. How can we help developers build (quality) apps more quickly, efficiently

 First, let’s talk about how we are helping our developers be successful on QuickBooks and grow their business. To do this, we offer developers a channel to our customers via our personalized Apps Store and through relevant in-product discovery. For example, if you are a small business owner and you’re in the expenses component of the software, all of the apps relevant for expense tables can be discovered through in-product discovery In doing so, we are enabling apps to be “found” based on need and necessity. As a result, we can help developers retain –and gain – their customers.

We strive hard to increase the number of relevant leads to developers from QuickBooks. We are working on implementing analytics that help us monitor this metric. Please note that while we can deliver leads to developers, the actual conversion depends on the specific developer application. While the reasons are beyond our control, it is important to understand how we can ensure that the leads become actual customers – this is how we can help our developers succeed.

 Lately, we have been working with a handful of developers in efforts to share insights we are learning from customers and identify opportunities for developers. In doing this, we are educating developers about our customer pain points, while also sharing an analysis of where the gaps are, and how we are looking to better integrate their apps into our workflows so it becomes seamless.

 Now let’s talk about how we can help developers build quality apps quickly and efficiently., For this,  we look at it from the perspective of the overall developer experience – i.e. easy on-boarding of APIs with SDKs and documentation, API stability dashboards etc . If you go to our website we have details of how our APIs work, access to sample code, SDKs, etc. The website includes everything needed to build an app – and we also have a developer hub where to post questions.

 Then our focus shifts to helping developers publish an app. Each developer will work closely with a member of our team to get through the entire process. We conduct an annual security review and a technical review to make sure that the app works the way it is supposed to. Our experts will give developers feedback and guidance. We also conduct a marketing review, which entails looking at the app store page description, the introductory video, and how the integration works. We use a portion of our marketing budget to help developers position their app to our customers so they can be successful.

There are several other programs and opportunities to help developers find new customers, market their apps and grow their business. For example, last year Intuit hosted the ‘App Showdown’ contest, where almost 100 developers competed over the course of several months for the chance to be selected as one of ten finalists. The winner received a $100,000 prize that can be used to get their business to the next level. Finalists were chosen based on several metrics including quality of integration to QuickBooks, innovation, market impact, and number of users on QuickBooks. These are all metrics we are already looking at to ensure the success of apps on our platform.

Building an API program is about a business accepting its new market positioning as a platform. Third party developers become a key asset in how well the platform can grow. Not only do they encourage new customers to your business, but they also create greater lifetime value for existing customers by giving them more reasons to use your products and engage with your business in multiple ways.

Network effects of a platform is a core strength of the platform model, but requires an investment and recognition that there is no longer a go-it-alone mindset required. Now it is a raise-all-ships mentality that supports network partners to grow their businesses successfully as well. A platform that recognises that API developers are businesses and sets in place metrics to help those partners grow is a platform that has a greater chance at long term success. Intuit are building that into their DNA by focusing on the metrics of platform network effects.

VisionMobile measures developer satisfaction twice per year, across the industry’s 20+ leading developer programs on 20+ program features and services. If  you wish to know more about our Developer Program Benchmarking research drop us a line here: sales@visionmobile.com.

Disclaimer: Intuit is a customer of VisionMobile, but there was no financial motivation behind this article.

 

Return on Developer Investment

My most fun job ever was as a C++ developer. Ok, I don’t have much grey hair yet, but I fondly remember the late 90s and the challenges of writing a background synchronisation application on a Compaq iPaq. And reverse engineering Mozilla’s Navigator into an XSLT parser.

My second most fun job ever has been building a company that helps the world understand developers, with research. We’ve come a long way – and a few pivots – from surveying the pulse of 400 developers in 2009 to 30,000 developers annually in 2016. That’s a lot of data – in fact more than our analyst team can chew.

It’s a privilege to be working with some of the biggest names in tech – I ‘ve learned a lot the past 2 years. Earlier this month, Amazon, Microsoft, Facebook, Adobe, Intel, Oracle and many more joined our first Future Developer Summit, and shared some of their best practices in how they work with developers. I ‘d like to share some the learnings here.

Return on Developer Investment.

You would think that with billions of dollars spent every year on building tools for developers, running hackathons, loyalty programs, tutorials and how-tos, evangelist and MVP programs – the platform leaders would have figured it all out. Yet, with so much money being spent on developer tools and marketing there is no standard for measuring the Return on Developer Investment.

Most companies represented at the Future Developer Summit shared how they measure success. At their inception, developer-facing orgs measure success by number of developers touched – but that’s a meaningless metric, a dinosaur from the age of print marketing. Some platforms are using NPS (net promoter score), polling their active developers once a year for how likely they are to recommend the platform. Many are informing product decisions based on developer comments (“will you ever fix that”?) – you’ll be surprised how many decisions are taken based on “the devs that I spoke to said..”.

Other developer relations teams are measuring success through the number of apps in the store, and the number of apps using signature APIs. In the case of open source projects, a popular metric is GitHub stars, forks and commits over time. The more sophisticated platforms track the Return on Developer Investment funnel from SDK downloads to app download and use. But there isn’t a consistent way to measure how the investments in hackathons, tutorials, how-tos, loaner devices, evangelism programs and some many more developer-facing activities are paying off for the likes of Google, Amazon and Facebook.

Quality of apps, not quantity.

Another theme of the Future Developer Summit was the need for quality, not quantity of applications at the start of an ecosystem. B2B ecosystems like Slack and Intuit prioritise quality; Poorly written messaging apps can damage not just the perception of Slack, but also the perception of chatbots in general. Similarly, a poorly written app for the QuickBooks platform can wreak havoc to sensitive financial data for thousands of small businesses. As a result both Slack and Intuit have very stringent app review processes, including weeks of testing, usability and security reviews. To improve quality for bots, Slack has pioneered a “Botness” program, bringing together bot platforms and leading bot developers; the aim is to “make bots suck less” i.e. improve the bot user experience and avert a long-term damage to the reputation of chat bots. There are already 250 members signed up and the next event is on November 4 in NYC .

The next Future Developer Summit will focus on best practices for developer relations. If you ‘d like to be part of the invite-only audience of platform leaders, register your interest at www.futuredeveloper.io