Return on Developer Investment

My most fun job ever was as a C++ developer. Ok, I don’t have much grey hair yet, but I fondly remember the late 90s and the challenges of writing a background synchronisation application on a Compaq iPaq. And reverse engineering Mozilla’s Navigator into an XSLT parser.

My second most fun job ever has been building a company that helps the world understand developers, with research. We’ve come a long way – and a few pivots – from surveying the pulse of 400 developers in 2009 to 30,000 developers annually in 2016. That’s a lot of data – in fact more than our analyst team can chew.

It’s a privilege to be working with some of the biggest names in tech – I ‘ve learned a lot the past 2 years. Earlier this month, Amazon, Microsoft, Facebook, Adobe, Intel, Oracle and many more joined our first Future Developer Summit, and shared some of their best practices in how they work with developers. I ‘d like to share some the learnings here.

Return on Developer Investment.

You would think that with billions of dollars spent every year on building tools for developers, running hackathons, loyalty programs, tutorials and how-tos, evangelist and MVP programs – the platform leaders would have figured it all out. Yet, with so much money being spent on developer tools and marketing there is no standard for measuring the Return on Developer Investment.

Most companies represented at the Future Developer Summit shared how they measure success. At their inception, developer-facing orgs measure success by number of developers touched – but that’s a meaningless metric, a dinosaur from the age of print marketing. Some platforms are using NPS (net promoter score), polling their active developers once a year for how likely they are to recommend the platform. Many are informing product decisions based on developer comments (“will you ever fix that”?) – you’ll be surprised how many decisions are taken based on “the devs that I spoke to said..”.

Other developer relations teams are measuring success through the number of apps in the store, and the number of apps using signature APIs. In the case of open source projects, a popular metric is GitHub stars, forks and commits over time. The more sophisticated platforms track the Return on Developer Investment funnel from SDK downloads to app download and use. But there isn’t a consistent way to measure how the investments in hackathons, tutorials, how-tos, loaner devices, evangelism programs and some many more developer-facing activities are paying off for the likes of Google, Amazon and Facebook.

Quality of apps, not quantity.

Another theme of the Future Developer Summit was the need for quality, not quantity of applications at the start of an ecosystem. B2B ecosystems like Slack and Intuit prioritise quality; Poorly written messaging apps can damage not just the perception of Slack, but also the perception of chatbots in general. Similarly, a poorly written app for the QuickBooks platform can wreak havoc to sensitive financial data for thousands of small businesses. As a result both Slack and Intuit have very stringent app review processes, including weeks of testing, usability and security reviews. To improve quality for bots, Slack has pioneered a “Botness” program, bringing together bot platforms and leading bot developers; the aim is to “make bots suck less” i.e. improve the bot user experience and avert a long-term damage to the reputation of chat bots. There are already 250 members signed up and the next event is on November 4 in NYC .

The next Future Developer Summit will focus on best practices for developer relations. If you ‘d like to be part of the invite-only audience of platform leaders, register your interest at


Messaging apps: From counting users to counting bots

Back in 2008, Nokia sold 468 million phones making the company the undisputed king of the mobile phone market with over 40% market share. This same year, Apple sold little over 10 million iPhones and launched iPhone App Store with just 500 third party apps. By the end of 2010, when Apple App Store had over 300,000 apps, it became clear to all including Nokia that the number of apps is much more important than the number of devices. Apps drive demand for phones creating network effects between users and 3rd party developers. Smartphone users attract developers. Developer create apps. Apps attract more users, which attract more developers.


A very similar dynamic begins to unfold in messaging platforms. Popular messaging apps evolve into developer-centric platforms having the same kind of network effect as iOS and Android. Soon we will compare messaging apps not by number of users, but by the number of bots/integrations available on the platform. Messaging users attract developers. Developers create bots. Bots attract more users, which attract more developers.

Messaging has emerged as a new interaction paradigm on mobile, with leading apps (Whatsapp, WeChat, Facebook Messenger, KakaoTalk, Line, Viber) amassing hundreds of millions of users. David Marcus, vice president of messaging products at Facebook says in his interview to the Wired magazine:

“The messaging era is definitely now. It’s the one thing people do more than anything else on their phone.”

So far, competition between messaging apps is based on number of users. In Q3 2015, Whatsapp (acquired by Facebook for over $19B) has 900 million monthly active users; Facebook Messenger – 700 million; and WeChat – 600 million. But now things start to change.

While Facebook leads in number of messaging users, Chinese Weixin, or as it is known in the West WeChat, is a clear leader in turning messaging into a platform.

WeChat at its core is a messaging app for sending text, voice, and photos to your friends and family, but it is also much more. Connie Chan, Partner at Andreessen Horowitz, explains on the company blog:

“Along with its basic communication features, WeChat users in China can access services to hail a taxi, order food delivery, buy movie tickets, play casual games, check in for a flight, send money to friends, access fitness tracker data, book a doctor appointment, get banking statements, pay the water bill, find geo-targeted coupons, recognize music, search for a book at the local library, meet strangers around you, follow celebrity news, read magazine articles, and even donate to charity … all in a single, integrated app.”

WeChat achieves this by supporting lightweight apps that are called “official accounts”. There are well over 10 million official accounts on the platform: from celebrities, banks, media outlets, and fashion brands to hospitals, drug stores, car manufacturers, to internet startups, personal blogs, and more. These lightweight apps are approved to access exclusive APIs for payments, location, direct messages, voice messages, user IDs, and more. Essentially, WeChat is not only messaging app, but a developer-centric platform allowing developers to add value to the service.

Facebook has no choice but to follow WeChat. Facebook’s David Marcus said at the Code/Mobile conference in October 2015:

“Messaging is really, truly the next frontier. The Asian paradigm has shown there’s a there there.”

Having introduced Messenger platform at its F8 developer conference in March 2015, Messenger has adopted the WeChat approach and will now be open to 3rd party developers to build new “tools for expression” and also let users communicate with businesses through simple conversation threads.

WeChat and Facebook are not alone in their attempts to take messaging to a new level. Telegram, which started as a more secure Whatsapp clone, evolves into something much more interesting with the announcement of their Telegram Bot Platform. The developer-centric platform allows 3rd party developers to create Bots, which are simply Telegram accounts operated by software sporting AI-like features.

The same trend shows itself even in the more conservative enterprise space with Slack Technologies Inc. having risen to $2B valuation in less than 2 years. Slack is a messaging app for teams designed to enable integration of messaging with popular enterprise apps and services. The company has 1.1 million daily active users, but also 100 integrations with 900,000 integration installs on the Slack platform.
These range from Giphy gifs to expressing feelings to co-workers; to MailChimp email marketing service; Crashlytics to monitor mobile app bugs; Trello for tracking tasks or manage help tickets from Zendesk.

The Slack Platform also supports bot users allowing companies automate many processes. A bot user is a special kind of free user account optimized for writing automated bots that connect to Slack using the Real Time Messaging API. Users can interact with bots using direct messages or even invite bots to private groups.

For example, The New York Times data science team has built a Slack bot to help decide which stories to post to social media.
The bot, called Blossom, predicts how articles or blog posts will do on social and also suggests which stories editors should promote. All within the framework of the messaging app.

Slack evolves into an enterprise developer-centric platform. There are already several startup teams experimenting with building companies on top of Slack messaging platform.

Similar to what happened in mobile platforms, the basis of competition in messaging apps changes from the number of users to the number of bots (integrations) and the messaging apps themselves evolve into developer-centric platforms.

Today Whatsapp is the largest messaging network with 900M users. It does one thing, messaging, exceptionally well. But it increasingly starts to resemble Nokia. Nokia also did one thing, mobile phones, exceptionally well, but missed the transition to developer-centric platforms, where the winners are decided by developers.