77% of all developers are involved in DevOps

About DevOps

More and more developers are getting involved in DevOps, with an eye on the ultimate DevOps end goal – to streamline the software delivery process. 

Although lacking a widely-accepted, universal definition, DevOps is in essence a set of practices that enable developers to release small but frequent software updates, reliably and safely. These practices are supported by a broader DevOps culture: activities, technologies, and dedicated platforms which work together to achieve the overarching DevOps goal: to streamline the software delivery process. 

In this short blog post, we’ll be sharing some key highlights from our latest global survey wave and the answers of 14,000 developers who responded to questions related to DevOps between December 2021 and February 2022. Also, we’ll be looking at findings from the “Who is into DevOps?” chapter of our 19th Edition State of the Developer Nation free report.

If reading this leaves you wanting to dive deeper into our DevOps insights, we are happy to let you know that we have extended our DevOps research to provide answers to questions like:

  • The DevOps technologies and new tools developers have evaluated, including the top vendors: Atlassian, AWS, Azure, GitHub, GitLab, Google Cloud, Heroku, JFrog, Oracle
  • The specific DevOps products or plans developers are using
  • How application security is handled across organisations
  • Which vendors’ application security tools they are using
  • The processes developers use to secure their cloud-native applications and 
  • Developers’ top security challenges

If you or your team are working on answering these DevOps questions, we will be happy to help you. Just get in touch

What are the latest insights on DevOps?

In our latest report “Landscape and trends in DevOps” we look at the current landscape and trends within DevOps from the developers’ perspective. We aim to understand who these developers are, look at what DevOps activities they’re involved in, and whether increased DevOps adoption really leads to higher software delivery process performance.

Here are the main highlights from the analysis: 

  • 77% of the surveyed developers are involved in DevOps
  • Involvement in multiple DevOps activities/technologies is predictive of higher software delivery performance
  • The average number of DevOps technologies used by DevOps practitioners has increased from 4.2 to 4.6 from Q3 2021 to Q1 2022. 

The last highlight means that the number of technologies used by DevOps practitioners has increased by nearly 10%. But DevOps practitioners are gradually exposed to a greater depth of activities too. Looking at each DevOps activity separately, we can see a significant increase in involvement across the board over the past 6 months:

involvement in DevOps-related activities has increased noticeably on the past six months

You can download the full preview of this report here or contact us to access all insights.

Who is into DevOps?

To answer this question and the ones that followed it, we asked developers whether they are involved in any of the activities that commonly fall under the DevOps spectrum, ranging from continuous integration and deployment to application and infrastructure monitoring. For the purposes of this blog post, we only consider developers who are professionals in at least one of the software areas they are active in. All the insights in this section come from our State of the Developer Nation 19th edition which was published on Q3 2020. You can contact us for all the latest insights.  

The first thing to note is that the adoption of DevOps practices is widespread among professionals, perhaps even more so than one might expect, given that the DevOps movement is relatively new. According to our data, the vast majority of professional developers (82%) are involved in DevOps in one way or another. For perspective, just over half (52%) of non-professionals are involved in any of the DevOps activities on our list.

Which of the following development activities are you involved in?

The vast majority of professional developers are involved in DevOps, but do not necessarily consider themselves DevOps practitioners

On a separate view of engagement with DevOps in our survey, only one in five developers reported working on DevOps when they were explicitly asked about their involvement in several emerging areas, including blockchain applications and quantum computing, among others. Even if we include those who said that they are learning about or are interested in DevOps, no more than 65% consider themselves to be engaged with the area. This signals that a large portion of the developer population has already adopted DevOps practices but does not necessarily self-identify with the term.

Focussing on the individual steps of the DevOps lifecycle, we find that developers are first and foremost involved in the fundamental activity of releasing frequent but small software updates. The most popular development process related to DevOps is continuous integration (CI), practised by 40% of respondents. Another 37% use continuous delivery or deployment (CD), which expands upon CI by automatically deploying all code changes to staging or production environments.

However, full automation of the software release process – and therefore true commitment to the DevOps culture – is far from a reality. While more than half (52%) of developers use CI or CD to streamline parts of their workflow, only 25% use both practices to automate all steps between integrating code changes into a central repository through to production deployment. As it turns out, developers are still sceptical about fully automated CI/CD pipelines. This is evident by the fact that nearly 40% of them manually give the green light for code deployments to be promoted to production.

Application and infrastructure monitoring, performed by 39% of developers, is one of the most common development practices, but not so much infrastructure provisioning and management (27%), which is still the realm of IT managers and system administrators. Similarly, creating automated tests (25%) and building CI/CD pipelines (23%) are rather specialised tasks, carried out predominantly by quality assurance professionals and solution architects, respectively.

Talking about organisational roles; our research reveals noticeable differences in the level of DevOps adoption, i.e. involvement in any DevOps-related activity, depending on the title that developers hold. First of all, technical company leaders – CIOs, CTOs, IT managers, and engineering team leads – report the highest level of involvement in DevOps activities. Not only do almost all developers with a technical leadership function, about 95% of them, have at least some participation in the DevOps lifecycle, but they are also simultaneously involved in a higher than the average number of DevOps activities (three vs two).

Involvement in DevOps by company role

Programmers have largely adopted CI/CD processes, but not so much other DevOps practices

The next tier of the DevOps adoption ranking is mainly occupied by specialist roles, such as network security engineers, QA developers, and system administrators. Between 86% and 91% of developers holding these positions are in some way associated with the DevOps culture. We should note, however, that only architects – system, solution, software etc. – appear to be heavily involved in all phases of the DevOps lifecycle. All other specialists are primarily focused on activities relevant to their expertise. For example, system administrators are naturally focused on infrastructure provisioning and monitoring, whereas QA engineers create automated tests for CI/CD pipelines more than anything else.

Front-line coders and software developers, who represent the majority of respondents in our survey (61%), are also highly likely to be involved in DevOps activities – 81% of them are although not more often than the average professional (82%). Our data suggest that software developers are keen to adopt CI/CD processes, but not so much operational practices such as monitoring applications in production environments. Again, this indicates that the complete shift to the DevOps culture has not yet been achieved. Apart from responsibilities central to their role, programmers are not accountable for additional product lifecycle phases.

Another important indicator of the level of engagement with DevOps practices is the software sectors that developers are involved in. As with roles, we see some interesting variations in DevOps adoption across sectors. For example, close to 90% of developers who create extensions for third-party ecosystems or backend services are into DevOps, as opposed to less than 80% of game developers.

Involvement in DevOps by software sector

That is partly explained by the extensive coding experience required to implement the DevOps model. We know from our data that DevOps practitioners are far more experienced coders than developers who are not involved in any DevOps-related activity. And developers working on apps for third-party ecosystems, backend services, or industrial IoT projects are among the most experienced in the software economy: up to 85% of them have three or more years of coding experience. In comparison, no more than 73% of game developers have the same level of expertise.

Nonetheless, we find that desktop app developers report relatively low adoption of DevOps practices, even though they are highly experienced professionals – 82% of them have at least three years of experience in software development. This points to limited alignment with the key benefits of DevOps more than anything else. Desktop applications typically receive updates at a lower frequency than applications running on other environments, e.g. servers. Therefore, the fundamental DevOps strategy of releasing small software updates at high velocity is not entirely applicable to desktop application projects.

In conclusion, DevOps signifies a cultural shift whereby developers from different teams work closely together with an aim to deliver software faster and more reliably. The practices of the DevOps model are already widely adopted among professional developers across software sectors and organisational roles, although with some significant variations in the focus on specific activities. These variations reveal, in some cases, that true commitment to the DevOps culture is not yet achieved; many developers are still focused on the core aspects of their role instead of assuming responsibility for additional phases of the product life cycle.

Want more DevOps insights? Get in touch and we can work together on all the questions you need to answer to optimise your strategy. 

Who is using low-code / no-code tools?

This is a chapter from our latest State of the Developer Nation 22nd Edition, which is free to download. You can watch our Lightning Session on the key findings and also read below for the whole report and insights on low-code / no-code tools.

Low-code/no-code (LCNC) tools provide a visual approach to software development, abstracting and automating parts of the application development process. This allows those without prior software development experience to create custom applications and provides potential time- and cost-saving for professional developers. In this chapter, we investigate the extent to which developers are using LCNC tools, showing differences according to professional status, geographical regions, and experience levels.

When it comes to reducing development overheads, addressing the challenge of finding skilled developers, and accelerating taking software to market, LCNC tools are becoming increasingly attractive. The sophistication of these tools is increasing rapidly, providing the potential to significantly disrupt the software industry. This begs the question, to what extent are developers1 using LCNC tools for their development projects?

We begin by separating developers according to their professional status – differentiating professionals from non-professionals, who are hobbyists and/or students. We excluded from our sample those who indicated that they were unsure about what share of their development work was done using LCNC tools. Just over half (54-55%) of developers in each group report that they are not using LCNC tools at all for their development work. This proportion is marginally lower for non-professionals who are students (55% of those who are exclusively students and 53% who are students and hobbyists) than non-professionals who identify as exclusively hobbyists (57%).

46% of professional developers use low-code/no-code tools for some portion of their development work

State of the Developer Nation 22nd Edition

The proportion of developers who do use LCNC tools does not differ across groups (46% of professionals vs 45% of non-professionals). This highlights that LCNC tools are finding traction among those less likely to be familiar with coding and that use-cases within professional software development are also common.

As experience increases, developers are less likely to use LCNC tools at all. This is particularly true among those with more than ten years of experience. These tools are often framed as being best suited for simple programming tasks. Hence, the complexity of development work assigned to more experienced developers may be less appropriate for LCNC approaches. Furthermore, experienced developers are likely to have mastery over simpler coding tasks, which leaves little room for the efficiency gains that LCNC tools are often heralded for.

Using LCNC tools without a degree of accompanying manual coding is highly uncommon across all experience levels. The proportion of developers who use LCNC tools for a small amount (up to a quarter) of their development work remains relatively constant (between 17-24%) across the experience spectrum. Therefore, LCNC’s most likely role is as an occasional adjunct to existing coding tools, regardless of developers’ experience.

Experienced developers, particularly those with more than 10 years of experience, are the least likely to use LCNC tools

State of the Developer Nation 22nd Edition

More extensive use of LCNC tools, i.e. for between one-quarter and three-quarters of all development activity, peaks slightly for those with around three to ten years of experience, revealing that it is early to mid-experience developers, rather than newcomers who are most likely to elevate LCNC tools’ status to essential. This is perhaps due to the recognised career importance of gaining traditional development experience, before reducing reliance on writing code. Only 2-4% of developers across all experience levels use LCNC tools for 75% or more of their development tasks, indicating that it is highly uncommon to shift the balance heavily towards LCNC-driven development.

Our data reveal notable differences in adoption and engagement with LCNC tools across different geographic regions. The Greater China area emerges as the region in which developers are most likely to be using LCNC approaches. 69% of developers in this region report using LCNC tools, compared to the global average of 46%. This suggests that the Chinese LCNC tool market has transitioned from an introduction phase to a growth phase. According to Mendix’s State of Low-Code report, IT professionals in China are the most likely to suggest that low-code is a trend their organisation can’t afford to miss (84% compared to 72% globally). Non-developer, or citizen developer, audiences also likely account for a large part of LCNC’s growth. However, as in all regions, the majority of bona fide software developers in the Greater China area currently use LCNC tools for less than half of their overall development work. It remains to be seen whether their reliance on such tools will also expand as the market and tools mature.

19% of developers in North America use Low-Code/No-Code tools for more than half of their coding work – almost twice the global average of 10%

North America has the second-highest LCNC tool adoption rate and stands out for the proportion of developers using LCNC tools to conduct more than half of their overall development work – 19% of developers here report that their use of LCNC tools outweighs their manual coding (comprising 13% using them for half to three-quarters of development work and 6% using them for more than three-quarters); almost double the global average of 10%. Hence, North America appears to be at the forefront of the LCNC movement, providing the strongest evidence that these tools can supplant traditional development approaches – even in a region where 81% of developers identify as professionals.

South Asia, the Middle East and Africa, and East Asia excluding Greater China are all above the global average in terms of LCNC tool adoption. Despite considerable uptake in these regions, LCNC products have not matured to the point where their use is a dominating feature of developers’ processes. Regions such as Western Europe and Israel, Oceania, Eastern Europe, and South America are all below the global average in terms of LCNC tool adoption.

The shortfall in these regions is particularly linked to smaller than average proportions using LCNC tools for more than 25% of their development work. The proportion using them for less than a quarter of their work is more comparable to the global average, suggesting that the market is still in its introductory phase in these regions – developers are evaluating the tools but are yet to rely on them for a substantial portion of their work.

Access the full free report to dive into insights on:

  • Language Communities
  • Understanding Developer Personalities
  • Who is using low-code / no-code tools
  • Spotlight on China and the Rest of East Asia
  • How developers generate revenue
  • Emerging technologies

If you have questions about the data above, want more or want to explore other topic areas we cover, talk to us.

China and the rest of East Asia developer market

In this article, we share a chapter from the latest State of the Developer Nation report, which anyone can access. We focus our attention on some of the key differences between developers in East Asia, including the Greater China region, and the rest of the world. Understanding these differences provides valuable insights that can help shape the strategy for developer engagement programs.

For this analysis, we split the Greater China area from the rest of East Asia to provide more regional granularity. In terms of relative size, we find that almost a fifth (18%) of the global developer population is located in either the Greater China region (9%) or the rest of East Asia (9%). Breaking down East Asia into countries, we see that more than half of the developers here are spread across two countries: Indonesia (32%) and Japan (21%). 

When comparing developers across regions, we can see that just over a third (34%) of developers in the Greater China region have six or more years of experience, which is notably less than developers globally (43%). Furthermore, the Greater China region has a much smaller concentration (4% vs 22% globally) of highly-experienced developers (16+ years). With generally lower levels of experience in the Greater China area, aspiring developers may find starting a career here less competitive than developers in regions with higher levels of experience.

The Greater China area has a comparatively low concentration of highly experienced developers

State of the Developer Nation Q1 2022

East Asian developers outside China have similar levels of experience to the rest of the global developer population. Both groups have a little more than a third (34%) of their developers with 11+ years of software development experience. However, East Asia’s data are largely propped up by Japan. The developer community in Japan tends to be highly experienced, with almost six in ten developers (59%) having 16+ years of experience. No other country has a higher concentration of developers with this level of experience. With such a high concentration of highly skilled developers, we can expect some differences in behaviour, which we’ll highlight in the last section of this chapter.  

More than 50% of Chinese Developers have learned how to code via undergraduate degrees in computing

State of the Developer Nation Q1 2022

The journey to coding mastery lacks a clearly defined path. Developers typically state they’ve used more than two learning methods on average to learn how to code. In general, the self-taught method is the most popular among developers globally, with more than 60% using this method. However, our data shows that the proportion of self-taught developers fluctuates significantly across regions.

In the Greater China area, the most popular method for developers to learn how to code is via an undergraduate degree in computing, with 50% having used this method. This is significantly higher than developers in other regions (41% – 42%). We generally see a higher concentration of professional developers in Greater China (83%) than we do in the rest of the world (70%). It could be that the job market in Greater China more often requires a degree in computing or engineering, which would also explain why self-teaching is used less often in this region.

Developers in the rest of East Asia, however, tend to follow the learning trends of developers in other regions. Here, we see the self-taught method is the most popular method (61%), followed by an undergraduate degree in software engineering (41%). Analysing the data at a country level, we see developers in Indonesia are more diverse learners. Developers in this country stated that they used three methods on average when learning to code. Indonesian developers are more likely to learn via self-teaching, online courses, and developer boot camps than any other developers in East Asia. This is quite different from their peers in Japan who are the least likely to use online courses and bootcamps to learn how to code. Instead, developers in Japan are most likely to use the self-taught (63%) and on-the-job training (45%) methods when learning to code. 

Developers in the Greater China area are half as likely to have a Stack Overflow account than developers globally

State of the Developer Nation Q1 2022

Next, we explore how developers interact with the popular online community, Stack Overflow, to understand their engagement levels with programming support. Stack Overflow has become a standard support community for many developers, with more than eight in ten (85%) of the general developer population reporting they’ve used or visited this popular question and answer site. 

Our focus on developers in East Asia and the Greater China area shows Stack Overflow’s popularity falls below the global average. Developers in these regions are around three times less likely to visit Stack Overflow than developers in other regions. Developers in the Greater China area are the least engaged, with only 19% having an account, and only 11% having earned at least one badge. Developers in this region have other home-grown Q&A site alternatives, such as segmentfault.com, which could be contributing to the lower adoption of Stack Overflow.

When looking closely at the rest of East Asia, we again see that developers in Japan are skewing the perception of this region. Developers in Japan have even less activity on Stack Overflow than developers in the Greater China area. Here, only a little more than a third (36%) stated they use Stack Overflow. Furthermore, only about 5% have an account. Like developers in the Greater China area, Our data does show usage of Stack Overflow increases among Japanese developers who have gained experience in software development, indicating that less experienced developers are using other platforms for support. Like China, Japan has other home-grown options like teratail.com where developers can field programming support from their peers, which may be the place new Japanese programmers visit more often to get answers to their questions.

That’s just one chapter from the State of the Developer Nation report. There are 5 more chapters you can access. Want more? Download the full report!

State of the Developer Nation: Coding language popularity, China’s developer market, how developers make revenue, and more!

The 22nd Developer Nation global survey from SlashData reached more than 20,000 developers in 166 countries. Its findings are bundled in a free “State of the Developer Nation” report. 

This research report delves into key developer trends for Q1 2022, taking a particular interest in the following:

  1. Language communities – An update
  2. Understanding developer personalities
  3. Who is using low-code/no-code tools?
  4. Spotlight on China and the rest of East Asia
  5. How developers generate revenues
  6. Emerging technologies

Here are some highlights from the report, guaranteed to intrigue your curiosity: 

Language communities – An update

  • JavaScript remains the most popular programming language for the tenth survey in a row, with close to 17.5M developers worldwide using it. Python has remained the second most widely adopted language behind JavaScript. Python now counts 15.7M users.
  • Go and Ruby are important languages in backend development but Go has grown more than twice as fast in the past year in absolute terms.

Rust has nearly tripled in size in the past 24 months, from just 0.6M developers in Q1 2020 to 2.2M in Q1 2022.

State of the Developer Nation 22nd Edition

Spotlight on China and the rest of East Asia

  • More than a quarter of developers in Greater China (26%) and the rest of East Asia (27%) don’t use Stack Overflow, which is more than three times the rate of developers in the rest of the world (8%).
  • The Greater China area has a relatively low concentration of highly- experienced developers (16+ years of development) when compared to developers in the rest of East Asia and the rest of the world.
  • More than half of Chinese developers have learned how to code via undergraduate degrees in computing, which is about 10 percentage points more than developers in the rest of East Asia and the rest of the world.

How developers generate revenues

  • Contracted development is the revenue model of choice across all industry verticals, used by nearly a third (31%) of professional developers.
  • Less than one in ten (7%) professional developers are generating revenue from selling data.
  • Usage of the advertising revenue model declines as companies grow in size.
  • Developers working for large enterprises (5K+ employees) tend to use
    multiple revenue models less often than developers in smaller companies.

Below we have included a few graphs that illustrate some of the findings.

You can download the full report for free and access all data and insights within.

If you need additional information or looking to understand developer preferences’, please get in touch with us and we will dive into it together.

How Okta is Broadening Their Developer Network with SlashData’s Developer Program Benchmarking Report

SlashData’s Developer Program Benchmarking report provides annual insights into developer behavior and trends that help tech companies make better, data-driven decisions.

Many Big Tech companies have come to trust our unique data insights in helping them understand developers better and shape their strategies.

Here’s how one of them –Okta– is using SlashData’s Developer Program Benchmarking to stand out and unlock more developer opportunities.

The Gap.

Optimize Developer Strategies that Boost Okta’s Developer Community.

Every tech company needs to have a better understanding of their developer-user base — so they can make more informed decisions. Plus, having a clearer view of the strengths and weaknesses of your developer programs will expose how you measure up against competitors and how to stand out.

To achieve this, it becomes necessary to:

  • Track your program adoption rate –anywhere in the world
  • Stay on top of developer satisfaction with your programs
  • See how competitor developer programs are scoring – what they’re doing right and the gaps you can capitalize on
  •  Measure the performance of your specific offerings based on what developers value

…and so on.

Tech companies like Okta are using SlashData’s Developer Program Benchmarking to do just that.

A Must-Have Resource for Developer Relations”

“Over the years, I’ve used SlashData’s reports to understand and prioritize what matters to developers. I’ve found the reports quite helpful in terms of identifying areas for improvement and the impact of our investments.

For example, when I was at Mozilla, we noticed our satisfaction score for blogs and newsletters wasn’t where we’d like it to be. So, we invested in creating a developer newsletter and improving the blog. 

And in 18 months, we saw our satisfaction score for this area rise to the top 3. This satisfaction score and other standard metrics (reach, opens, etc.) helped us determine how successful our program was.

Now, at Okta, I’m again using the report to identify areas for improvement and will invest in our documentation, blog and website. SlashData reports give us a trendline that shows, over time, if and how these investments pay off, and allow us to make changes if we are not seeing results.”

Ali Spivak

Sr. Director of Developer Relations, Okta [ex. Mozilla]

The Results.

Instead of second-guessing developers’ interests and neglecting areas of investments that count, Director of Developer Relations – Ali Spivak – could use SlashData’s reports to:

  • Help determine areas of investment and to see –over time — how those investments impact developer adoption and satisfaction level
  • Provide invaluable insights on how their program was performing and where their opportunities for growth lie
  • Shape and augment the user research they were already conducting and focus more on specific areas that required attention
  • Track developers’ satisfaction scores with their program and specific offerings
  • Benchmark how Okta’s developer programs stack up against other developer programs to provide them with a model of improvement
  • Help stakeholders understand what developers are experiencing and prioritizing, their needs, and also justify the investments made into developer programs

Here’s How We Can Work Together to Build a Thriving Developer Ecosystem…

In today’s diverse and complex developer ecosystem, there’s real power in knowing where to focus your efforts to help build a thriving developer community.

That’s what SlashData’s Developer Program Benchmarking helps you achieve.

With SlashData, you get:

Data-Driven Insights for Effective Decision-making

Data is the fuel that drives the engine of decision-making – as a data-driven company, we’re always aware of its immense value.

So, when you come to us and share your current challenges, we work to help you see and understand how much difference data-backed decision-making can make.

We’re always ready to engage and walk you through the data –so you can uncover and bridge the current gaps you’re experiencing.

Client-Tailored Global Developer Survey

SlashData’s Developer Program Benchmarking report is a syndicated research study available to tech companies who are subscribed to our program. It is based on data collected in our Global Developer Nation survey that’s fielded twice per year.

If your developer program hasn’t been benchmarked, we will include it in the upcoming editions of our in-depth research study.

There are many hidden potentials for your growth in engaging developers in a way that targets their needs and  improves their experiences.

That’s why we survey about 20 developer programs every 6 months, as the list of programs benchmarked is ever-changing. This helps us learn more about what boosts satisfaction and adoption rates for developers.

Full Access to Deliverables for Close Monitoring

We provide full access to the agreed deliverables as soon as they get completed.

These usually include a market landscape report, insights report specific to your company, and an interactive data dashboard that allows filtering and zeroing in on a particular developer segment.

Our clients have found these reports quite useful in helping them shape and even augment the user research they are already conducting on their own.

These deliverables will help you pinpoint the specific places you should be focusing on for in-depth user research.

Plus, it provides areas of opportunities where you can exploit advantages over competition to stand out.

Beyond ready to build stronger relationships with your developer community?

Get in touch and we will dive in together in the solutions that will win developers’ hearts.

How developers’ support needs change with experience

Developers have a wide variety of support and learning needs that evolve as they progress through their careers. Here, we’ll look at some of the best ways to help developers build on their skills by answering their technical questions, creating a valuable community that they can integrate with, and providing professional certifications as proof of learning. In a highly competitive job market, vendors can demonstrate value to developers by helping them to build on their skills and get an advantage in the job market.

Here, we take a look at data from two of our most recent Developer Nation surveys. In our Q1 2021 survey, we asked developers, amongst many other topics, how they prefer to communicate with vendors about technical topics. In our Q3 2021 survey, we took a deeper dive into developers’ views on what makes great technical certifications and what are the key features of a successful community. The data here is only a small sample of what we collect, so if this sparks some interesting questions for you, then please get in touch.

It’s a matter of experience

Data from our Q3 2021 survey, which was fielded between June and August 2021, shows that overall, there are more early-career developers (those with 0-2 years of experience) than highly-experienced developers (those with 11 or more years of experience). Developers with different levels of experience undoubtedly have different support needs (and we’ll come to this later), but taking a global perspective on experience levels risks missing some interesting regional variations.

South Asia and Western Europe sit at opposite ends of the experience spectrum – South Asia has the largest proportion of inexperienced developers, and Western Europe has the smallest. This means that when creating a regional strategy, not only should you think about the cultural and economic differences that exist between regions, but also, due to their experience levels, developers will have very different support needs.Developers in Western Europe are more experienced than average and far more experienced than those in South AsiaTechnically correct is the best kind of correct

We see here how developers’ support needs evolve as they gain experience. In fact, communicating with vendors about technical questions becomes more important as developers gain experience – more experienced developers are very likely working on more challenging projects and, as such, more often require expert support. What’s interesting is which communication channels become more important.

Email is consistently the most important, regardless of experience level. It seems that the power of direct, asynchronous communication is clear to all developers, though it does become more important to more experienced developers, as well as to older developers (and age is, of course, correlated with experience). On the other hand, other direct but synchronous communication methods such as online chat retain their importance to developers of all experience levels (but fall in importance for the oldest), whilst live interactive coding sessions only fall out of favour amongst the most experienced. Not every communication method is created equally, and neither is every technical question. Irrespective of their experience levels, developers want to engage directly to have their technical questions answered and are happy to do synchronously or asynchronously.

Issue trackers and code repositories nearly quadruple in importance for the most experienced developers when compared with the least experienced. Here, you have experienced developers asking their technical questions through established open-source channels that may feel inaccessible to less-experienced developers. There’s definitely scope to widen participation amongst inexperienced developers in this fundamental pillar of software development. We also see that Q&A sites steadily increase in importance as developers become more experienced. That’s not to say that inexperienced developers aren’t going to StackOverflow – they’re still using such sites to get information; it’s just that they are more likely to simply consume rather than ask technical questions of vendors.

Direct communication via email or chat is most important to developers at all experience levels

A sense of community

Interacting with vendors or peers through a code repository or on a Q&A site is one of the many ways in which developers interact with their community. Community support is a powerful facilitator of learning and development for many developers and is as much a source of inspiration as it is camaraderie. We see that developers of differing experience levels have very different ideas about what they want from a community, but collaboration and support are two of the most stable and important features to developers of all experience levels.

But experienced and inexperienced developers lean on their community support network in different ways. A knowledgeable community becomes more important to developers as they gain experience – here, these most experienced developers likely find more value in a community that can help them answer complex questions. On the other hand, inexperienced developers are more likely to look for strong leadership in a community – they are likely looking to more experienced members for guidance and learning opportunities.

Strong leadership and interactivity are less important aspects of a community to experienced developers

Certifiably important

Vendor support and community are just two of the myriad ways that developers build their skills throughout their careers, but in an increasingly competitive professional environment, many developers study for professional certifications to get an edge. Such certifications are important to developers at different stages of their professional life – early-career developers are likely looking to distinguish themselves from the masses, whilst seasoned professionals may want to protect their lucrative career or even switch specialisation. Regardless, because of certifications’ wide appeal, developers at all experience levels similarly agree on the importance of certifications being suitable for a variety of learning styles.

On the other hand, industry recognition, online availability, and affordability are three of the most important features of a professional certification program, and they become more important as developers gain experience. This demonstrates that as developers mature, they become more focused on the core aspects of professional certifications. We also see how their job-seeking habits change. The importance of recognition on job boards rises steadily from zero to five years of experience before falling sharply afterwards. This suggests that after around five years in the industry, developers have built their professional network and are less reliant on job boards, though the professional credibility of a certification is still paramount.

Developers at all experience levels recognise that many learning styles should be catered for

What does this all mean?

Here, we’ve seen that there is great variation in the experience levels of developers across the world, as well as between different geographical regions. We’ve also learnt that developers of different experience levels have very different views about the type of support they want to receive from vendors and from their communities, whether they are asking technical questions or becoming certified. Therefore, you should look at the experience levels of your user base and use this to figure out how best to support them. However, experience isn’t the whole story; our extensive research shows that a plethora of factors influence developers’ needs and decisions. Developers’ roles, level of decision-making seniority, industry, and technology choices all impact their needs for support. Understanding developers’ needs and behaviour requires not only a rich set of data but also extensive experience and knowledge to build the personas that inform a robust strategy.

Don’t know where to start? Well, at SlashData we have a wealth of experience in understanding developer behaviour through our twice-yearly global survey, as well as through numerous custom research projects with our clients and partners. We also have a deep and detailed body of research on developers through our Developer Program Benchmarking research. Get in touch to find out more.

You can also go through a case study that shows how Okta and Mozilla used the Developer Program Benchmarking to bring their developer program among the Top 3 in terms of developer satisfaction.

Google has the leading developer program, but Amazon is catching up

Developers. Decision-makers. Kingmakers?
For several years now, at SlashData we have been helping our clients – some of the biggest names in tech – to understand how their developer programs measure against the competition. Twice a year, we run an extensive and wide-ranging global survey to understand who developers are, what tools and resources they use, and where they are going. Developers share with us their experiences with vendors’ resources – which ones they use, how often they use them, and how happy they are with the experience. We also dig a little deeper into what developers value in vendor support, resources, and communities.

Our research shows that developers are becoming increasingly involved in all stages of the decision-making process. Not only are they writing specifications for vendors and tooling choices, but they are also influencing decision-makers and budget holders. If software is eating the world, then developers are writing the menu. 

To attract developers, many tech companies are actively investing in Developer Relations (DevRel) teams and developer marketing activities. They are creating an abundance of resources, training programs, technical support, events, and community activities. It’s not always clear which activities should be priorities and how resources should be allocated to achieve long-term strategic goals. We are here to help.

Our Developer Program Benchmarking research tracks 20+ of the leading developer programs, and captures developer sentiment across more than twenty developer program attributes, ranging from documentation and sample code to mentoring programs and access to experts. In so doing, it helps DevRel and developer marketing practitioners understand how their developer program compares against the rest.

Here, we give you a snapshot of the state of play for these developer programs. We use three KPIs to create a 360° overview of how each developer program performs:

  1. Adoption – How many developers use a vendor’s resources
  2. Engagement – How frequently developers engage with the resources
  3. Satisfaction – How developers rate their experience using the resources

bubble chart showing how developer perceive the leading developer programs

We can see that the Market Leaders; Google, Microsoft, and Amazon highly engage and satisfy developers. Their market share – or adoption rate, shown by the size of the bubble – reinforces their market-leading position. In fact, when we take a longer-term view of this data, it becomes clear that Google and Microsoft have long been the market leaders, staying at or near the top of the table for all three KPIs. 

Recently however, Amazon has made considerable progress. In fact, Amazon’s developer program has been growing faster than the global developer population, which is currently 24.3M (you can explore more in our developer population calculator), while Google and Microsoft’s share has dropped slightly. When you take into account the large increase in Amazon’s satisfaction score and their aggressive growth strategy, the top table positions don’t seem so assured.

Our data also uncovers the Satisfying Specialists – these developer programs are often small and focused. Unity, Red Hat and DigitalOcean sit firmly in this space. Developers don’t need to engage frequently with these vendors’ resources, but when they do, they have an excellent experience. For these vendors, low engagement is not a cause for concern, though it does come with its own challenges – when developers have fewer touchpoints there are fewer opportunities to speak to them or to influence their behaviour. For these (and other) vendors with low engagement, messaging becomes vital. 

The Under-realised Value segment contains developer programs that, although having high engagement amongst developers, are being held back by their low satisfaction ratings. These programs are often (though not always) small, and the vendors here have a clear imperative to improve their developers’ experience. Thankfully, with developers engaging frequently with the resources there are ample opportunities to effect positive change.

But what, exactly, to change? 

This brings us to the true power of our Developer Program Benchmarking research. Not only do we understand how developers engage with vendors’ resources, but we also know which resources are important to developers, and how satisfied they are with the resources that companies provide. 

Though developers’ preferences change and evolve, some things stay constant. Of the twenty-plus resources that we ask about, documentation & sample code, tutorials & how-to videos, and development tools, integrations & libraries have consistently been rated as the most important resources that companies should offer. This shows that developers are focused not only on getting things done, using documentation and development tools to speed up the development process, but they also highly value having the opportunity to learn. We can see this repeated further down the list – training courses & hands-on labs provide the learning opportunities, whilst technical support allows them to lean on experts when they need to.

Table showing the 5 resources: documentation, tutorials, development tools, training courses and technical support

In this way, we can tell which resources developers value, and how their experience matches their expectations. This information, when combined with our wealth of survey data on demographics, firmographics, technology choices, motivations, skills, and much more, becomes incredibly powerful for informing strategic planning. We help some of the leading tech companies in the world to understand precisely which resources need improvement, and which developers will benefit most from such improvements. Have you ever wanted to know how to tailor your tutorials to the right level of complexity? Have you ever tried to decide how to localise your content? What about marketing to enterprise developers, what do they care about? 

We also go a level deeper. For many developer programs, we specifically ask developers how they use resources relating to different products or disciplines. For example, we help developer programs to understand whether or not they are vulnerable in the cloud compute market, or what are the specific preferences of developers using IoT resources. Once again, coupled with the rest of our rich and diverse data, this information allows you to create a finely tuned strategy that allocates resources efficiently and effectively.

With developers having such power in the decision-making process, this is a win-win for everyone involved. By understanding what developers value, you can tailor your offering to suit their needs, increasing retention, growing your audience, and ultimately, adding to your bottom line. SlashData are the analysts of the developer nation, and we can help you understand developers.

You can download a preview of the latest Developer Programs Benchmarking here.

Developer Research 101: The right methodology for reliable survey data

Suddenly a fine day dawns when your organisation’s key stakeholders agree that you need data to understand your developer audience. Well, ok, most likely that didn’t exactly happen overnight – in fact we know* that nearly 20% of DevRel practitioners struggle to justify the budget of their developer programs and 32% rely on qualitative arguments. But let’s skip that part for now, and fast-forward to that happy moment when there is full buy-in for data-backed developer strategy decisions. Right. You need data. But what data?

First, ask the right questions

Let’s pause here for a second. At SlashData we may have data in our DNA, but we know that plunging head-first into data is not where your quest for answers should begin. Your first step should be, instead, to ensure that you are asking the right questions. However trivial that may sound, you may discover (as many others have) that in fact it is not. If you get the questions wrong, the answers will be meaningless and the time and budget you will have invested in finding data to answer them will be wasted. 

Knowing exactly which questions you need answered will help you specify not only what data you need, but also where you should get it from, and how big a sample you should aim for. If it is just a total market share figure you’re after, for example, chances are you don’t need that many data points – neither in terms of sample size, nor in terms of breadth of information collected. If however you are trying to understand what developer personas (or segments) exist out there, where they are located, how they feel about different technologies, and where they’re going next, you’re looking into an undertaking of an entirely different magnitude, and may the Force be with you. Or, more practically, SlashData, as we have been in this business for more than a decade now.

Mind the source

Once you know which questions you need to answer, you should carefully choose your data collection method and sources. If, for example, you want to know how your technology is currently being used, you could use your own telemetry or usage data, or survey your own community. If, however, you need to see what other technologies your developers use, or how competing technologies are being used by your broader audience, you need to look beyond your own community by means of a global survey that targets everyone in your market. Based on our bi-annual survey of developer-facing organisations, we find that about half use their own survey data, and nearly half run qualitative research, when in fact it may be more appropriate to use a different approach. 

graph showing how developer program developer relations find information

A common, though less-than-ideal approach, for example, is tracking developer sentiment (usually in the form of Net Promoter Score) based on data collected from current users that interact with you (say, through your website). While this may be a good indication of how your current active users feel, it can not be generalized to represent the sentiment of your whole target audience, as it omits the views of past users who have now left you, and also the views of those who evaluated but rejected your technology in favour of a competitor. Those who abandon a technology are more likely to give it a low recommendation score if asked. By omitting their views and using only current users’ scores you therefore get a positively biased result. This is particularly true in highly competitive low-rigidity markets (such as some cloud services can be) where your current users are more likely to be satisfied fans who stay with you by choice rather than due to technology lock-in effects, while the displeased have already left you to turn to one of your competitors. 

That is why in our surveys we always ask both current and past users how they feel about each of the technologies they either use, or have (recently) stopped using, or evaluated but rejected. In this way, we get an unbiased estimate of developer sentiment for the broad range of technologies that we track – allowing us to benchmark them with a high degree of confidence.

It’s not only the size that matters

“Can we discuss sample size already?” I hear you cry. We shall in a moment, I promise, but we need to get something else straight first. You should be collecting a sample of… what exactly? An equally – if not more – important consideration to size is the representativeness of your sample. To use a crude example, there is no point buying a truckload of bananas, when what you’re looking for is apples. Similarly, there is no point collecting an impressively big sample from the US and India alone, for example, when what you’re interested in is global trends. And that’s because developers in different parts of the world behave very differently when it comes to technology choices, as they have different motivations and business models, and may be at different stages in their journey as they form part of developer ecosystems of varied maturity. Our data proves again and again that there are vast differences between regional developer communities. That’s why we go into great lengths, each and every time, to survey developers from more than 150 countries, so that we may truly gauge the pulse of the global developer community.

gra[h showing slashdata global developer reach

It’s not only regional diversity, though, that you should carefully balance your sample for. There are several other attributes you should consider, such as the mix of professionals, hobbyists and students, and the size of the organisations that your surveyed professionals work for. The latter is particularly important if, for example, you wish to capture the views of both enterprise developers and startups that are bound to be working with different tools. Demographics such as age may also be important, as you may want to hear from both the young coders, who typically use some technologies more than others (open source software is a prime example here), and from the seasoned developers who may have a deeper understanding but also higher expectations of the tools they use.

graph showing survey takers age

You should also ensure that you don’t repeatedly rely on the same pool of developers, say a panel, no matter how big. In such a fast-paced industry, behavioural patterns and user profiles may change without warning. By repeatedly surveying the same people over time, you risk failing to observe the change originating from a different pocket of the developer population than the one your panel comes from. And if you do fail to observe the upcoming trend, you will miss the opportunity to ride the wave of change. This is particularly true for the emerging sectors such as augmented and virtual reality, but also for more ‘exotic’ technologies still in the early stages of their lifecycle, such as DNA computing, self-driving cars, or body-brain computer interfaces.

As we track all of these and many more, we reach out to capture the experiences and the intent of developer populations of all shapes and sizes, from small local meetups to large vendor communities. Our surveys are promoted by more than 70 leading community and media partners each time, and we make sure they are not the same 70 every time, to ensure we are not repeatedly hitting the same pools (or communities) of developers. And while we reach out afresh to the developer population each time, we consistently observe meaningful trends in our data – rather than wild jumps – which proves that we do indeed capture a representative view of the software development industry.

Last but not least, be careful of any incentives you offer to survey takers. These must be carefully designed to appeal to all profiles within your target audience, or you risk creating selection bias, i.e. attracting only developers of specific profiles, rather than a random sample of all developer profiles out there.

Is your data clean?

But no matter how hard you try, you’re bound to get some sample bias – beware of anyone who says they don’t. How do you deal with it? 

First, particularly if you are offering incentives, you should clean out all fraudulent – or simply illegitimate – responses. There will always be those who are in it only for the prize, randomly clicking through your survey and diluting results. They may even build smart bots to do that (after all, we are talking about developers here). At SlashData we have developed sophisticated ML algorithms that identify such responses and unceremoniously throw them out. Based on the metadata that our bespoke survey-taking platform tracks, we are able to outsmart the not-so-honest respondents and call them out.

Then, it’s a matter of correcting for over-represented groups. It could be that, despite your best efforts, you attracted disproportionately more hobbyists than you should have done, for example. Or perhaps word got around in a particular language community about this cool survey, and slightly more enthusiasts than what you had hoped for came forth to vouch for their favourite programming language. How do you fix those imbalances? Especially given that you don’t know what the true (or population) proportions are – since that is the very thing you’re trying to estimate. In such cases, some – very very careful – data weighting is in order. 

You have to be extremely careful (have I stressed this enough already?) as to how to go about it, first to identify the sources of bias, and then to decide how to correct it without introducing over-correction. But this is a rather long story, and we’ll keep it for another day. 

All I shall say here is that at SlashData we treat all the different channels through which we get our data (such as our network of 70+ partners mentioned earlier) as independent samples, which we then compare across a set of parameters which we know may introduce bias. We use ML models to specify the level of correction that should be applied, and take into account all types of bias that a single response may be simultaneously carrying. 

What is your margin of error?

We get that question a lot. I hope that by now I have demonstrated that, although important, this should not be your only concern. In fact, the margin of error can be quite misleading if used as the only metric to assess sample and research quality. Let me give you some statistical insight that might shed some light on this problem.

As a quick search can reveal, the margin of error is designed to measure uncertainty in random samples. More specifically, the theory of the margin of error (MoE) applies, strictly speaking, only to questions (but could, under certain circumstances, be generalised to full surveys), and only to perfect random samples. This implies that if the assumption of perfect randomness does not hold (and in the real world in most cases it doesn’t), then the theory collapses and your MoE estimate is meaningless. To go back to our crude example of obtaining a truckload of bananas as a sample of apples, just because you have a truckload (and from a large truck at that), your margin of error estimate will look satisfyingly low. Your calculation, however, will have not accounted anywhere for the fact that these were in fact (loads!) of bananas, not apples, and as such, they make for a useless sample, albeit a tasty one.

That is why at SlashData, instead of just quoting margins of error that when used in isolation may misleadingly inflate confidence in a sample, we focus our efforts on obtaining a sample that is as big, as random and as robust as possible. These are, in fact, the three elements that do lead to a reliable estimate of a margin of error. In other words, it’s not enough to only quote a margin of error. One should also be able to demonstrate that the underlying assumptions of the MoE calculations, namely randomness and normality, are met to a satisfactory degree. So if you’re out there shopping for survey-based research, make sure to first scrutinise any potential sellers for the health of their outreach and sampling methodology. Only then, if satisfied, ask about the margin of error. 

Go for a large sample you can dig into

All that said, sample size is, of course, very important. To continue the margin of error discussion, suppose you are faced with a choice of two random samples (that is, samples you can be reasonably sure are close to random). If they both come from the same population, say the global developer population, then, at any given level of confidence, their difference in margin of error will lie in the sample size. Based on our robust developer population sizing research, there are currently (as of Q1 2021) 24.3 million developers in the world. That means, that even at a 99% confidence level, our sample of more than 19,000 developers from across the world yields a margin of error of less than 1% at the question level. If instead you had, for example, a (random) sample of 2,500 developers, your margin of error for the same question would be around 3%. 

But having a low margin of error is not the key reason for which you should aim for a large random sample. The main reason is having the ability to dig deeper and slice the data, while still having enough sample left from which to confidently draw conclusions. If, like us, you run unsupervised models, random forests and other ML models to identify developer segments and predict their technology choices, then you need large samples to do it. Otherwise, you end up with a really thin sample that is anything but reliable with regards to the picture of developer personas that it paints. Even if you’re into simply tracking trends for subpopulations of interest, you still need a big-enough sample. In our data dashboards, for example, we give you the option to filter for many attributes, such as age, region, professional status, gender, decision-making power, and much more. If we were to start off with a small sample, filtering would leave you with a tiny, and therefore useless, sample size. For example, filtering in our Developer Population Calculator for those under 25 years of age, who are students, and have up to five years of experience, still leaves us with nearly 4,000 respondents to draw conclusions from.

Are you lost? 

Here’s a cheat sheet:

  1. Ask the right questions. Make sure you accurately specify what business questions you need answered, and by which audience.
  2. Select the data collection method (such as a large-scale survey, telemetry, qualitative research, etc.) that is best suited for the problem you’re trying to solve. 
  3. Carefully design your developer outreach to obtain a sample that is representative of the population you are interested in. 
  4. Aim for a large sample, so that you may confidently dig into it, if you need to.
  5. Clean your data from illegitimate, or even fraudulent responses.
  6. If you’re confident enough that you have a random sample, estimate your margin of error – at the question, not survey, level.
  7. Check for sample bias and correct for any obvious deviations from randomness, without overfitting (or over-correcting).

In short, as you may have guessed by now, the art of research design and developer outreach is not for the faint-hearted. And it can not be wrapped up in a margin of error figure. But fear not. With more than 10 years of experience in mapping the developer ecosystem through large-scale surveys we are here to help. All you have to do is get in touch.

 

*Based on our Developer Program Leader surveys. Have your say in the latest one.

How do developers perceive themselves? Are they introverts or extroverts?

The data from our latest Developer Economics survey give all the answers in the recently published State of the Developer Nation Q2 2019 report, which – as always-  is now available for free download

What’s new in the dev world? 

The report highlights the latest insights from the developer ecosystem, based on responses from 19,000+ software developers of all profiles: professionals, hobbyists and students, working across all major areas: mobile, web, desktop, cloud, IoT, AR/VR, games, machine learning & data science. It focuses on 6 themes:

  1. (NEW) Developer psychographics: how do developers perceive themselves? 37% see themselves as introverts and only 4% describe themselves as being able to make a killer cocktail.
  2. Programming language communities: Is JavaScript getting comfortable on its throne? Updated estimates of the number of active software developers using each of the major programming languages
  3. Emerging Technologies: What turns interest into adoption?
  4. Game Streaming: Developing for live-streaming. Are we there yet?
  5. Third-party platforms and ecosystems: App developers are unlikely to work solely on one third party platform. 
  6. Mobile cross-platforms frameworks: Mobile development has matured enough to embrace cross-platform frameworks, allowing building apps that can run on multiple platforms.

The full report is available for free download at sdata.me/fr_SoN17 

Software developers who want to have their voice heard in the next State of Developer Nation report, can join our next survey that starts on Nov. 22nd, 2019. All survey respondents enter a draw to win amazing prizes and Developer Economics donates $0.10 for every complete response to the Rasberry Pi Foundation. 

graph showing developer psychographics

Gender Wars

The technology industry often takes credit for the changing world of work. One example is the model of remote employees working as digital nomads in their favourite coffee shop, connected via Slack and collaborating via the cloud to create products and services for consumption over the internet or on smartphones and tablets. But what about work within the technology industry itself? We take a look at the profile of women in technology and compare it with the profile of their male counterparts.

If we exclude those who preferred not to share their gender with us, and those who skipped this optional question, female developers responding to our survey were outnumbered by males by a ratio of 1 to 10 (9% women and 91% men). This suggests a global population of 1.7 million women developers and 17 million men. The technology industry is dominated by men and the imbalance in numbers is such that we cannot make numerical comparisons between men and women. Instead, in the rest of this chapter, we will look at relative differences in terms of experience, age and roles adopted, and the most common company sectors and development areas for men and women.

What are their ages?

Looking at the comparative ages of male and female developers, we find a higher percentage of women are under the age of 35. The 25-34 age group accounts for the largest number of developers of both genders (36% of women, 33% of men), yet male developers are more likely to be older: we found 37% of male developers are over 35 years, compared to 29% of women developers.

There are (at least) two different ways of interpreting this observation. One is to say that women are being increasingly drawn to software development; the comparatively young profile of women compared to men illustrates recent gains made in attracting girls and young women into technology. Analysis of college data for entrants to computer science courses, in North America at least, suggests that this is indeed a plausible explanation, as women are increasingly studying courses in the subjects that lead to a career in technology.

An alternative, or additional, explanation is that women may have always been involved, but tend to leave software development as they get older, either by choice or necessity.

And here’s a preview of the roles they undertake:

Gender Wars 1

 

What is their educational background?

When we looked into the education levels of the genders, we noted that women developers are equally likely to have been educated to degree level in computing/software engineering when compared to men. Likewise for other classroom training that doesn’t lead to specific degrees, and for attendance at developer bootcamps.

Women are slightly more likely than men to have learned their craft using online course materials and slightly less likely to have learned on-the-job. Women are significantly less likely to be self-taught (57% of women compared to 75% of men) but it is still the most popular way of learning about development for both genders. The relatively older profile of men probably explains why more have become self-taught: they have engaged in continuous education throughout their longer career because of the rapidly changing nature of the industry. As women developers mature, we would expect the level of “self-taught” women to rise as they also teach themselves new skills to advance their career.

 

For more details on the Gender Wars, you can download our State of the Developer Nation 16th Edition report.

It’s free and full of insights.