Software Use & Testing Survey

April 2020

On-demand professional software testing service QA Supermarket conducted a survey of software industry professionals who are involved with the testing processes at their organizations, to gather insights on how their organizations are approaching this critical function. 

Overall, the responses suggest that, while many organizations may believe they are doing enough testing, their responses indicate processes that nearly-ensure they are not. It is particularly notable that one-quarter of individuals with quality assurance-related roles said they are not sure if they do or do not do enough testing.

In addition to the information on testing, QA Supermarket took the opportunity to find out how COVID-19 was impacting their individual roles and how they believe it will ultimately impact their organizations. Responses indicate an optimistic outlook and predominantly favorable personal experience. 

KEY FINDINGS

BASED ON RESPONSES FROM 140 INDIVIDUALS IN THE SOFTWARE INDUSTRYWHO HAVE INVOLVEMENTIN THE TESTING PROCESS IN THEIR ORGANIZATIONS. SURVEY FIELDED VIA POLLFISH IN APRIL,2020

The majority of respondents believe their organization or project team does enough software testing, however, responses from those in roles most connected to testing indicated this may not be the case.

  • Most notably, Quality Assurance, the group that should know, was the least confident about whether testing was sufficient, with 25 percent admitting they didn’t know if they are actually doingenough testing. 
  • Also interesting is the higher proportion of software/ application developers and project owners/managers who say they ARE NOT doing enough testing.

Do you believe your organization/project team performs enough software/application testing? 


Responses regarding how their organization approaches testing also supports the notion that it is unlikely sufficient testing is being done.

  • 40.7 percent reported leaving the testing process to developers –among the groups who were most likely to believe their organization DOES NOT do enough testing
  1. If developers are, infact, responsible for testing, it is likely this function is not a top priority for them. 
  2. Generally, developers have higher salaries but don’t have the skill set of testers, making this a less effective but more costly option
  • Almost 13 percent effectively don’t test

In addition to the 1.4 percent who admit they don’t, there’s another 11.4 percent who say they “leave testing to the client,” which is another way of saying they don’t test!

How does your organization/project approach software/application testing? 


The misconception that enough testing is being performed joins lack of resources as the top reasonsorganizations do not perform more testing. 

  • 31.4 percent of respondents said the primary reason they don’t do more software testing is because the decision makers in the organization think they do enough.
  • Availability of human resources (talent/expertise) was cited by 30.7 percent of respondents, while lack of money and time were also common (22.9 and 22.1 percent respectively).
  • When broken down by proportion for each role, we see that:

    – Quality Assurance places the largest emphasis on lack of human resources and the beliefs of decision makers

    – Nearly one quarter of managers alsocite decision makers, while project managers are fairly evenlyspread across rationale

What prevents your organization/project team from doing more software/application testing?


Respondents had a very optimistic outlook on the impact of COVID-19, believing there would be little negative business impact. At the same time, they, themselves are busier, and social isolation is, in some ways, good for them. 

  • Nearly half (47. 2 percent) said they expect revenue to be higher as a result of the crisis. 
  • Few (5.7 percent) believe the pandemic will result in a reduction in headcount. 
  • Respondents also felt there’d be little disruption to their work. Just 15.7 percent of respondents expect new product development to be lower, while only 9.3 foresee this impact on ongoing product management and maintenance.
  • Over half (51.4 percent) of respondents reported increased quantity and nearly half (42.9%) expected an increase in the amount of time they work. 
  • In many cases, self-isolation is better enabling them to get that work done, with 35.7 percent saying they’ve better able to concentrate and 39.3 percent reporting greater efficiency/effectiveness.

Please indicate how you believe the COVID-19 crisis is impacting your organization/ organization’s…


Conclusion

While the positive results around the COVID-19 crisis were surprising, the responses about testing were, unfortunately, fairly close to QA Supermarket’s expectations.

Software testing remains a function that is frequently mis-assigned in organizations and not given the priority it should. To ensure the highest performance, testing should be conducted by dedicated personal with the necessary skill and knowledge. 

Outsourcing can help alleviate the issues of talent availability. It can’t, however, drive the organizational change and managerial understanding required to ensure that software and applications are receiving sufficient testing, and that continuing to undervalue this critical part of the product life cycle increases the risk of releasing –and suffering the consequences of–error-ridden software.

If you use our survey data, please refer to https://qasupermarket.com/