Study: SOA Testing Needs to be Horizontal as Well as Vertical

SOA in Action Blog

« SOA Without the Middleware? | Main | Top-Down or Bottom-Up SOA? Why Not Both? »

September 04, 2007

Study: SOA Testing Needs to be Horizontal as Well as Vertical

New research from Aberdeen Group shows that to effectively test and provide QA to SOA, companies are testing not only vertically (using unit and functional testing as their benchmark for quality) but are also testing horizontally across an entire business process. In addition, in successful testing/QA environments, there is close involvement of the business user in more phases of the development lifecycle.

Aberdeen Group has just issued a report on the challenges around testing and quality assurance for SOA and Web services.

The study of 240 end-users found that so-called "best-in-class" companies are taking a multi-pronged approach to the SOA testing and QA problem by incorporating automation in the testing lab and process change at the organizational level. More importantly, best-in-class companies are increasing the involvement of business users in all phases of the development lifecycle, and view quality not as something that's done at the end of the cycle, but as a horizontal attribute that spans multiple business processes.

The survey found that 61% of the best-in-class companies saw a reduction in the number of defects discovered in production. By comparison, only 18% of the "laggard" companies could say they were able to reduce defects. Overall, 94% of the best-in-class companies reported an increase in the quality of deployed software, versus 17% of the laggards.

How can a company evolve into a best-in-class organization when it comes to testing and QA? The leading companies "have attacked the problem from multiple angles," Aberdeen notes. "It isn't enough to just deploy automation, and it isn't enough to simply rely on functional tests QA for composite applications needs a horizontal, process-oriented view, not the vertical unit-test methods used in the past."

Aberdeen notes that the typical best-in-class company gets business users involved in all aspects of quality.uses automation to increase test coverage, and sees quality as more than just an end-of-the-lifecycle task.

Best-in-class companies "are almost three times as likely as industry average and laggard organizations to have redesigned their entire testing process," the report continues. "The best-in-class organizations have made the commitment to services-based applications and have retooled their key processes to reflect their new direction."

The report adds that "many of the changes are focused on bringing quality to bear on the entire lifecycle of the project by getting business users more involved. Doing this requires the top-down commitment of the enterprise. The enterprise as a whole must be willing to set aside the time for key players to be involved in projects as they move from phase to phase, and to supply them with the appropriate tools necessary."

Posted by joemckendrick in Management | Digg This | Add to del.icio.us

Trackback Pings

TrackBack URL for this entry:
http://www.ebizq.net/mt/mt-tb.cgi/2280

Comments

There is a free copy of this Aberdeen report (signup required of course) here:
http://www.aberdeen.com/link/sponsor.asp?spid=30410567&cid=4117

Good analysis, we've seen that trend too - it is also relevant to the organizational aspects of SOA quality. It is much easier to achieve quality, and Trust that the expected service levels and functionality are delivered by SOA applications vertically along the chain of command. Horizontal Trust across divisions and business partners leveraging shared applications is a more difficult challenge requiring both technical savvy, and a strong impetus for cooperation. But to realize the value of SOA, that is precisely what companies must do.

Posted by: Jason English at September 6, 2007 02:27 PM