What we've learned by designing 10 developer surveys

The question in question is the question of questions

This week we launched our tenth biannual developer survey – asking thousands of developers around the world what they’re working on and how they’re doing it. If you’re involved in software development, in any way, then go and fill it out – it will only take you fifteen minutes and I’ll wait here while you do.

Answers are easy. It’s asking the right questions which is hard.
Doctor Who – the fourth one

At Vision Mobile we spend a lot of time composing questions, especially when we’re compiling a survey like this one. The process sounds easy enough – phrasing 30 questions we’d like answered to provide insight into the developer ecosystem, but it turns out to be a surprising challenge.


For a start we have to create a lot more than 30 questions: the survey tailors itself to ask each person about the industry in which they work, based in the first round of questions. The survey can thus be designed to last around fifteen minutes, but the whole industry can be covered.

Some questions we repeat every six months, choice of language, mobile platforms, and so forth, so we can spot developing trends, but sometimes an old question will need new options as the industry changes. A year ago we added Swift to our language list, and were surprised to see how quickly it had gained popularity, now we’ll be waiting to see if that growth has been sustained and at what cost.

Other questions are created from scratch: the technology behind the Internet of Things might not be entirely new, but the developer interest is. For the first time we’re asking about Open Source in IoT, drilling down to see how this new industry is evolving.

After the questions are written the task is far from over, for once the words are down then the “discussions” can begin. How many options should be listed? Which toolkits are worthy of mention? Whose products should be used as examples? How many IDEs can one developer realistically use? VisionMobile employs experts in many fields; with practical experience developing software and an intimate knowledge of the challenges involved, but like most developers these people are driven by a passion for their subject, and have strong opinions on the tools and techniques they consider important. The survey has to be impartial so we try to ensure that all the experts are equally unhappy, for balance.

Software development is a global industry these days, so the survey has to reach a global audience. Once the questions have been written, discussed, dismantled, and rebuilt to a mutually-acceptable level of dissatisfaction, then they have to be translated into almost a dozen languages, always ensuring that the clarity of the original remains intact.

And then it is done. Perhaps not quite a joy forever, as Keats would have it, but certainly a thing of utility. Questions laid out, check boxes ready to be checked, radio buttons ready to be… radioed(?). Everything waiting for the thousands of developers such as you (what do you mean you haven’t done it yet? Get over over there now, this minute). They are drawn by the desire to contribute to the project, or get access to some of the results, or win a prize in the draw, or just know that their opinion matters to the companies and organisations which will be referring to the data over the next six months before the whole process kicks off again.

Shakespeare’s Hamlet asserted that “To be, or not to be” was “the question”, but in these days of Continuous Delivery the questions will never end, and we have turn to a pair of Hamlet’s school friends (Rosencrantz and Guildenstern, with the help of Tom Stoppard) to see where that might lead us:

Messaging apps: From counting users to counting bots

Back in 2008, Nokia sold 468 million phones making the company the undisputed king of the mobile phone market with over 40% market share. This same year, Apple sold little over 10 million iPhones and launched iPhone App Store with just 500 third party apps. By the end of 2010, when Apple App Store had over 300,000 apps, it became clear to all including Nokia that the number of apps is much more important than the number of devices. Apps drive demand for phones creating network effects between users and 3rd party developers. Smartphone users attract developers. Developer create apps. Apps attract more users, which attract more developers.


A very similar dynamic begins to unfold in messaging platforms. Popular messaging apps evolve into developer-centric platforms having the same kind of network effect as iOS and Android. Soon we will compare messaging apps not by number of users, but by the number of bots/integrations available on the platform. Messaging users attract developers. Developers create bots. Bots attract more users, which attract more developers.

Messaging has emerged as a new interaction paradigm on mobile, with leading apps (Whatsapp, WeChat, Facebook Messenger, KakaoTalk, Line, Viber) amassing hundreds of millions of users. David Marcus, vice president of messaging products at Facebook says in his interview to the Wired magazine:

“The messaging era is definitely now. It’s the one thing people do more than anything else on their phone.”

So far, competition between messaging apps is based on number of users. In Q3 2015, Whatsapp (acquired by Facebook for over $19B) has 900 million monthly active users; Facebook Messenger – 700 million; and WeChat – 600 million. But now things start to change.

While Facebook leads in number of messaging users, Chinese Weixin, or as it is known in the West WeChat, is a clear leader in turning messaging into a platform.

WeChat at its core is a messaging app for sending text, voice, and photos to your friends and family, but it is also much more. Connie Chan, Partner at Andreessen Horowitz, explains on the company blog:

“Along with its basic communication features, WeChat users in China can access services to hail a taxi, order food delivery, buy movie tickets, play casual games, check in for a flight, send money to friends, access fitness tracker data, book a doctor appointment, get banking statements, pay the water bill, find geo-targeted coupons, recognize music, search for a book at the local library, meet strangers around you, follow celebrity news, read magazine articles, and even donate to charity … all in a single, integrated app.”

WeChat achieves this by supporting lightweight apps that are called “official accounts”. There are well over 10 million official accounts on the platform: from celebrities, banks, media outlets, and fashion brands to hospitals, drug stores, car manufacturers, to internet startups, personal blogs, and more. These lightweight apps are approved to access exclusive APIs for payments, location, direct messages, voice messages, user IDs, and more. Essentially, WeChat is not only messaging app, but a developer-centric platform allowing developers to add value to the service.

Facebook has no choice but to follow WeChat. Facebook’s David Marcus said at the Code/Mobile conference in October 2015:

“Messaging is really, truly the next frontier. The Asian paradigm has shown there’s a there there.”

Having introduced Messenger platform at its F8 developer conference in March 2015, Messenger has adopted the WeChat approach and will now be open to 3rd party developers to build new “tools for expression” and also let users communicate with businesses through simple conversation threads.

WeChat and Facebook are not alone in their attempts to take messaging to a new level. Telegram, which started as a more secure Whatsapp clone, evolves into something much more interesting with the announcement of their Telegram Bot Platform. The developer-centric platform allows 3rd party developers to create Bots, which are simply Telegram accounts operated by software sporting AI-like features.

The same trend shows itself even in the more conservative enterprise space with Slack Technologies Inc. having risen to $2B valuation in less than 2 years. Slack is a messaging app for teams designed to enable integration of messaging with popular enterprise apps and services. The company has 1.1 million daily active users, but also 100 integrations with 900,000 integration installs on the Slack platform.
These range from Giphy gifs to expressing feelings to co-workers; to MailChimp email marketing service; Crashlytics to monitor mobile app bugs; Trello for tracking tasks or manage help tickets from Zendesk.

The Slack Platform also supports bot users allowing companies automate many processes. A bot user is a special kind of free user account optimized for writing automated bots that connect to Slack using the Real Time Messaging API. Users can interact with bots using direct messages or even invite bots to private groups.

For example, The New York Times data science team has built a Slack bot to help decide which stories to post to social media.
The bot, called Blossom, predicts how articles or blog posts will do on social and also suggests which stories editors should promote. All within the framework of the messaging app.

Slack evolves into an enterprise developer-centric platform. There are already several startup teams experimenting with building companies on top of Slack messaging platform.

Similar to what happened in mobile platforms, the basis of competition in messaging apps changes from the number of users to the number of bots (integrations) and the messaging apps themselves evolve into developer-centric platforms.

Today Whatsapp is the largest messaging network with 900M users. It does one thing, messaging, exceptionally well. But it increasingly starts to resemble Nokia. Nokia also did one thing, mobile phones, exceptionally well, but missed the transition to developer-centric platforms, where the winners are decided by developers.

The National Trends Survey Lifts Off

Our 10th anniversary survey officially launched as of yesterday 15/10/2015. The National Trends Survey is completely updated – with newer questions concerning cloud, IoT, mobile, and desktop trends for the last 6 months.

We’ve added free prize categories like the Developer Benchmarks & Country Scorecards – making the survey process that more enjoyable for all our participants. These are designed to help developers a) compare themselves with other devs from their country/region and b) see how their country compares to other parts of the world.  

New surveys come with new goals, that’s why the VisionMobile team is pushing for even wider reach this time around – specifically 15,000+ developers. We want to make sure that the free report we give to the developer community contains the freshest of insights. Bear in mind that the survey duration is 6 weeks. If you’re a developer, help yourself and contribute to this research.

Thinking of promoting? Please get in touch with us.