SOA in Action

Over at the spicy ITGumbo blog community, Leah Archibald has put together some great in-the-trenches views of what it means to be service-orienting.

Leah's Mumbo Gumbo comic takes us through the first step on the road to SOA, which is: "admitting you have a problem." (Is that Harry Potter in there?)

In the follow-up installment, Leah explains SOA in simple terms "that hungry people can understand."

Good stuff!

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

In anticipation of the upcoming "SOA in Action" virtual conference, October 30-31, ebizQ's own Gian Trotta recently chatted with Randy Heffner of Forrester Research to talk about some of the misconceptions around service-oriented architectures these days. Randy will be keynoting day two of the event (details here).

Since the conference will fall on that certain time of the year when hobgoblins rule, Randy provided Gian some of the "tricks and treats" coming out of SOA efforts these days. (Summary and podcast here.)

Randy's SOA trick or treating advice includes the following:

Keep Web services separate from SOA. Companies still make the mistake of viewing the two concepts as one single endeavor, thus sabotaging their best efforts, Heffner says. “Instead, view SOA as a broad set of design concepts centering on your major business processes and transactions. View Web services as one set of application-to-application communication protocols by which to access your services.”

Think of SOA as more -- far more -- than a technology. “That’s a very small view of SOA,” says Heffner. What’s more important is the idea of SOA as a business design to enable strategic transformation and flexibility to optimize processes, which is the higher level view of SOA.

You don't need to buy anything new to get started with SOA. You don’t necessarily need to buy loads of new products that break the bank in order to do SOA, Heffner says. “While it’s true that you may eventually buy enterprise services buses and SOA management and repositories and appliances and such, you may well be able to get started on SOA and achieve strong business benefits without buying anything new.”

Don't get too caught up with the SOA registry. “The real treat within an SOA registry repository solution is not the registry, but the workflow and the service lifecycle tools around the repository,” Heffner said. That’s what gets you a start on what makes SOA successful — strong governance and organizational maturity to use a repository in the right way.

I will also be leading a session at the event, a panel discussion entitled: “SOA and Web 2.0: Mashups, SaaS, and Collaboration: Putting the Pieces Together.” The premise is that SOA and Web 2.0 share common goals, and there is enormous potential for Web 2.0 to accelerate SOA efforts.

But how do we connect the dots between Web 2.0 and SOA? SOA and SaaS, for example, share common frameworks. Then there’s the enhanced collaboration SOA developers and architects can achieve as a result of Web 2.0 tools. Then there’s the mashup scenario, in which end-users can build their own applications — the ultimate vision of SOA. Finallly, front-end environments such a Ajax and other rich client interfaces may help extend the “last mile” of SOA to the desktop.

So join me at the conference, and along with a panel of distinguished experts (to be announced), we’ll figure out how to put those pieces together.

A full SOA in Action Conference agenda can be found here.

Posted by joemckendrick in SOA Events  |  Permalink  | Comments (0)  | TrackBacks (0)

A couple of years back, at brokerage giant Morgan Stanley, it became obvious that the firm's information technology was a drag on its competitiveness.

As this new article by Penny Crosman relates, the brokerage giant turned to service-oriented architecture to fix its internal mess, with positive results.

In the article, Morgan Stanley executives described their IT architecture up until 2005 as "substandard." The firm had a lot of the usual issues hold back many organizations -- stovepiped systems, branch offices that wouldn't, or couldn't, communicate or share files, and a corporate culture in which developers were accustomed to doing everything themselves. The company needed more and better performance data, made available through dashboards and portals, so it could better track the profitability of individual branches and product lines.

Morgan Stanley's first move was on the hardware side -- upgrading its network infrastructure connecting the branch offices.

The brokerage giant then set out to develop a service-oriented architecture layer of common services that could be used and reused across its various operations. But, as with many things SOA, this was easier said than done. As Lance Braunstein, Morgan's managing director of field, data and application services, put it: "Everyone talks about SOA, but there is so much sunk cost in legacy systems that they're hard to back out of. You start where you can."

Braunstein's team found a good starting point to be in the rewriting of commonly used software components into Web services that can be used by any application. Initial efforts focused on data access calls and identity management functions. As noted above, this was easier said than done, especially in a culture in which developers did everything themselves. "At first, developers felt it was disempowering to have to plug in to the framework and leave things like making an identity call to the standard Web service," Braunstein said. However, he continued, over time, the developers began to see the merit of not having to rewrite those routines.

Next were the legacy apps themselves. Portions of these appls had to be rewritten, which involved retrofitting them into a common, consistent interface. Using IBM WebSphere Server and SOAP XML as a messaging standard, Braunstein's team established a "canonical service bus" that serves as the backbone of its SOA.

The firm is seeing a range of benefits from its SOA move, including more consistent data being delivered to online channels. Morgan Stanley was also able to develop a new workstation for its advisers, calculating that the amount of revenue each financial adviser brings in grew from $479,000 to $819,000 annually, and assets under management per financial adviser grew from $40 million to $89 million. In addition, the brokerage developed a customer portal which leverages the new infrastructure.

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

Since its current inception a few years back, SOA has been mainly about delivering integration and agility within the corporate firewalls. But there's a vast frontier -- business-to-business interactions -- that some SOA implementations have only begin to tap.

In this new article in Supply Chain Executive, Bruce Williams, general manager and vice president of BPM solutions at webMethods, talks about one retailer's initiative to maximize its supply chain, employing new SOA and BPM strategies.

First, as anyone in retail and consumer goods manufacturing can attest, the age of overstock is long over. As Williams notes, "companies simply can no longer afford to invest in huge physical warehouses and carry massive amounts of stock in anticipation of customer demand. Neither can they wait days or weeks for data to be loaded into data warehouses in order to create demand forecasts that may be egregiously out of phase with customer activity by the time those analyses are produced. Market forces are requiring businesses create ever-tighter connections between product delivery and customer needs and wants, and take the waste and latency out of the processes by which they fulfill those needs."

Such was the challenge for Albert Heijn (AH), a big retailer based in the Netherlands, which lacked visibility into its supply chain, and was forced to rely on fixed replenishment schedules that were based on local demand forecasts.
The company needed to minimize the amount of inventory in warehouses and across its 170 stores.

The solution was to change from a once-daily to real-time monitoring replenishment cycles. This required "replacing existing integration points with an infrastructure capable of fully connecting all 720 stores to corporate systems and suppliers," Williams said.

The solution AH pursued included a SOA-based approach to bridge its previously separated technology environments. AH's new platform consists of webMethods' Integration Server (IS) on each store's back-office server, employing a JDBC Adapter to communicate with the server's database for processing outbound and inbound messages. Each store's IS connects to a central broker at AH's headquarters, which in turn connects to back-office applications.

AH rolled out the new system to the first store in November 2005, a process that was completed for all 720 stores by May 2007. AH can now bridge previously separated application-to-application and B2B environments within a single infrastructure that also controls the supply chain. Information on each retail sale streams all the way back to suppliers. All point-of-sale data is automatically aggregated with sales data from across all 720 stores and delivered to a centralized replenishment application. Decision makers now get replenished data every half hour to help with supply chain decisions.

To build the case within the business, AH created cross-functional teams to bring together business and IT people, and focus the strategy on the business needs and projected benefits of the project.

AH is now implementing webMethods Business Activity Monitoring technology to provide end-to-end monitoring of key performance indicators (KPIs), Williams adds. This new capability "will enable the retailer to continuously improve the replenishment process, monitor technical connectivity between stores and headquarters and gain additional visibility into timeframes for transactions in replenishment processes."

Posted by joemckendrick in Business Process Management  |  Permalink  | Comments (0)  | TrackBacks (0)

It may sound like heresy for a blog that is supposed to tout the wonders of SOA in Action, but not all great integration projects have to be strictly about SOA, at least in the strictest sense of the definition.

Lorraine Lawson at IT Business Edge points to an example of a manufacturer that got its supply chain act together employing basic EDI and XML. As Lorraine so aptly puts it: "It's a This is a textbook example of how integration can transform a business." In this case, she explains, Emerson used two 'older' technologies, EDI and XML, to create an integration hub for its 70 business units and sundry partners.

The challenge, as explained by InformationWeek's Andrew Conry-Murray, was that Emerson, a manufacturer, has 70 separate business units that purchase goods from 35,000 suppliers. "Each unit communicates with its own suppliers via a combination of e-mail, spreadsheets, faxes, and phone calls. Asking a logistics provider to step into the middle of this tangled transaction Web simply wasn't feasible."

Emerson wanted to interact with the outside world as a single entity -- not 70. The solution was a communications hub that supported two data formats -- EDI, using the ANSI ASC X12 format, and OAGIS XML from the Open Applications Group. The Emerson Transaction Hub messaging platform that supports these standards was built on Sun Microsystems' SeeBeyond software.

There was one obstacle Emerson had to surmount before all this was set to go, however. The transaction hub concept had to be sold to the 70 business units, each of which typically make their own IT decisions, as well as outside providers and suppliers. Here, Emerson's central IT department had to serve as a partner to the various departments.

Another challenge was describing standard transactions, such as purchase orders and advance-ship notices, in the EDI and XML data formats. Many of these transactions came out of legacy systems. A PO had to be defined, for example. "We spent a lot of time with our primary trading partners trying to understand the business processes and common processes to support, and then defining a standard transaction," Russ Hermes, Emerson's logistics IT program director, is quoted as saying. This process took about a year to complete.

The effort seems to have been worth it, with impressive results. Emerson spent about $500,000 developing the hub, but reports it "has recovered those costs several times over." For example, putting 10 suppliers in the same shipping container cuts costs by 35%, and the company says it has saved millions in transport costs by consolidating shipping. There's also greater visibility into the supply chain, as well as greater leverage to negotiate deals with suppliers, since Emerson now appears as one united company.

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

In a recent post, I talked about how BT, the UK telecom giant, has been in the process of shutting down hundreds of legacy systems as it moves deeper into SOA.

However, Michael Oara of Relativity, reacting to this report, advises that companies hold off a bit on wholesale dispensing of their legacy assets. There's still a lot of value in many systems that many modernized architectures may not be able to replace, he says: "I believe that the riskiest aspect of shutting down the application is the potential loss of some existing functionality that is still needed by the business. As an example, imagine that all screen interfaces were replaced by Web services. In such a case, is it possible that the application still receives some feed from an outside source? Even for the case of the user interface, how are we sure that all data elements and user actions are being replaced and nothing is lost? Furthermore, what about the application boundaries – could it be that we shut down something that does not properly belong to this application, or is shared with another application?"

Those are all great questions Michael raises. I ran across a great example in my own research for a recent piece in Insurance Networking News. Namely, The Hartford Insurance -- which leads the industry in SOA thinking and actions -- has some 30-40-year-old mainframe systems. However, while it is replacing those systems that serve commodity markets such as personal lines, it won't be throwing out the mainframes it uses for its high-value commercial lines anytime soon. Those mainframes are packed full of processes and logic that continue to porvide the company with competitive advantage.

Before thinking about throwing the switch on a legacy asset, Michael Oara advises performing "a careful and exhaustive analysis that would reveal (1) the application boundaries and (2) all application interfaces. Both these could be handled more or less automatically by a modern legacy analysis tool."

"A careful analysis can thus insure that no application input or output is left behind, as the user interface is replaced by Web services. Once the risk is eliminated, applications may be 'shut down' at a higher pace."

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

Everyone knows retail is a tough, dog-eat-dog business. And, in recent years, it's been no fun being a department store. They that have been eaten up by specialty stores and big-box warehousey stores alike.

"It's a street fight everyday," said Jeff Marshall, senior vice president and CIO of Kohl's department store. However, as I heard Marshall discuss in his keynote at BEAWorld in San Francisco this week, it's been a fight he and other retailer types love to engage in.

Kohl's is one of the fast-growing department stores around, with growth quadrupling over the past 10 years. The firm made $13 billion in 2005, and is on a path to grow to $24 billion by 2010.

But this growth is not happening by accident. Nor can it all be attributed to good locations. As Marshall related in his BEAWorld keynote, by effectively leveraging technology in new and innovative ways, Kohl's has been able to beat the relentless competition at every corner. SOA is proving to be an agile force for the large retailer.

Kohls' strategy for the next several years is to increase its profitability by five percent a year, primarily through operating efficiency, Marshall elaborated. The retailers' business is highly diverse, with a wide assortment of merchandise categories, as well as extensive supply chain. "Our processes need to marry up with the processes that our partners use," he said. Over the past year, Marshall's department has been engaged in projects to enhance analytics and decision support, marketing and CRM, multi-channel systems, point of sale systems, and a warehouse management system. Plus, the company forged an agreement with Chase to manage its credit card business, which required closer integration with the bank's systems.

"Our systems need to operate a real-time speed," Marshall said. This includes making the same information available to internal users, as well as on the Website, and, soon, to cellphone and PDA users. "We needed services that are available across the enterprise -- not just to the logistics department."

Marshall acknowledged that SOA is a "long journey," but in order to demonstrate the viability of the effort, the IT department engaged in some "early wins" with service-enabling various aspects of internal IT operations, including requirements management, change management, impact analysis, and governance models. As part of this effort, Kohl's "looked at its entire stack of of software, and evaluated its legacy partners in a holistic fashion," Marshall explained.

A key part of the service-oriented process was the training being delivered to its technology staff, he added. "We offered training programs that skills that emphasized integration with business partners, not the old COBOL/Java stuff." While Marshall did not offer precise metrics or cost saving numbers, he noted that "this IS transformation is helping drive business goals."

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

Andy Baer, CIO of Comcast, helped kick off the BEA World conference in San Francisco this week with a keynote presentation on the communications giant's efforts to employ SOA methodologies to better connect customer service activities, and thus avoid such gaps in analyzing network problems.

In an operation on the scale of Comcast -- with more than 20 million customers using the carrier's cable, video on demand, and nascent phone business -- there are literally thousands of legacy systems and applications scattered across hundreds of business units and acquired companies. It's a challenge, but such scale also presents enormous opportunities for rapid productivity improvements -- and this is where SOA makes its first mark.

Much of Comcast's SOA efforts are focused on integrating and wrappering back-end and legacy systems to provide new front-end services for the business. "We weren't in a position to start over," Baer said, noting that Comcast grew over the decades through acquisitions of local cable operators and media companies. As a result, the company has a large range of siloed applications and siloed product lines.

Employing an enterprise service bus and registry/repository directory (all from BEA Aqualogic), Comcast was able to establish a service layer that forms the basis of its SOA. "The first thing we did was define a mediation layer so we could start consuming applications," Baer said.

Examples of new services launched include portals enabling wireless access for technicians, as well as a customer service portal -- both sharing and leveraging the same services. Baer observed that with tens of thousands of technicians on staff, just saving 15 minutes a day in productivity for each would save Comcast $100 million a year. Here's where SOA did make a difference for Comcast.

Baer said the company’s 20,000 customer service technicians originally had to look through 10 or more applications to attempt to solve problems. Now, a unified call center and problem resolution portal not only helps its staff save time and money, but the same services are also now made available to technicians over their mobile phones as well as to customers. And the portal also includes information about service across the customer’s neighborhood.

Using SOA methodologies involving loosely coupled, coarse-grained services, a team of 25 was able to roll out the entire feature in about five months.

Of course, productivity improvement and cost cutting are just the initial benefits of SOA. As the approach matures in organizations, it has the potential to reshape business agility and improve time to market -- in other words, SOA can play a direct role in growing revenues.

Baer sees this potential as well. Comcast is increasing its pace of new product rollouts, and the company's technology needs to support the pace of these rollouts, rather than stand in the way as an obstacle.

For example, Baer wants to actually cut turnaround times for new offerings from months to 24 hours, using SOA methods. “It sometimes takes six months, or sometimes one year to get a new product out the door,” he said. “The goal for our technology team is 24-hour product introductions.”

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

When is the best time to turn off legacy systems once their functions are captured in the SOA middleware layer? The sooner the better, says the chief architect at BT, the UK-based telecom mega-giant.

A new report from InformationWeek's Charles Babcock notes that BT is shutting down hundreds of applications or systems each year as it moves more deeply into SOA.

The article quotes George Glass, chief architect at BT, who argues that SOA payback won't be seen until these older systems are put out to pasture. "To get cost savings, mining Web services from old applications only pays off if IT follows through and shuts down the legacy applications," the article notes. InformationWeek also ran a survey that reveals that "adding complexity and failing to get cost savings could result from companies adding new Web services without ridding themselves of old applications. Thirty-six percent of respondents didn't replace older technology with SOA."

For BT's part, it has so far shut down 205 legacy systems the first year of its SOA implementation, 710 its second year, and 260 in the first quarter of its third year.

BT's SOA focus has been on extending functionality formerly locked away in its internal systems out to business partners. Those included services such as billing or customer address checking, that could be used for BT's retail customers or for a new third-party broadband provider piggybacking on BT's network. Glass says BT "was able to identify 160 core capabilities that BT provides, each with five to 15 operations." As a result, BT has been able to cut time to market with a new service from 270 days to 90 days, says Glass.

This is a shining example of SOA going straight to the bottom line. Most SOA implementations have been able to demonstrate internal productivity savings, but BT is demonstrating that such a flexible architecture is enabling it to quickly go in new directions with its 400 business partners.

Posted by joemckendrick in SOA  |  Permalink  | Comments (1)  | TrackBacks (0)

Lorraine Lawson at IT Business Edge just surfaced some helpful words on SOA planning from the director of enterprise architecture and strategic planning for U.S. Coast Guard.

If this is what the Coast Guard recommends, you can't go wrong. Not only that, but the author of this advice, Andrew N. Blumenthal, was also the former chief enterprise architect at the U.S. Secret Service.

Blumenthal makes the following recommendations:

Know when to use services. "Define the extent to which we will and will not use services; services are selected with intent and do not randomly spring up," Blumenthal says.

Think big, start small. I couldn't agree more. As Blumenthal puts it: "This allows us to validate the architecture, while giving the organization value, realized as usable services."

Build on what you have Viva la mainframe!

Use SOA to streamline business processes. Many people forget that this is the most immediate and measurable impact of an SOA effort. As Blumenthal puts it: "SOA is an inherently flexible and interoperable model for hosting application functionality; this provides an opportunity to rethink and improve business processes."

Incorporate standards. There's a lot of criticism that there are too many standards. However, there are core standards that will ensure the hot-swappability of components and applications. "Use industry Web service standards for navigation, application logic, integration, data stores, and enterprise infrastructure," according to Blumenthal.

Build around a security model. We need a lot more of this in SOA these days. And, as Blumenthal admonishes: "Functional design needs to be built around security, and not vice versa (security cannot be added as an afterthought)."

Design with quality in mind. "Quality must be designed into the product, and not inspected into it."

Organize development resources. "Group development team around logical business tasks, and not around technologies."

Train developers. Definitely not enough of this going on. "Ensure designers and developers have the skills to implement SOA properly," Blumenthal says.

Posted by joemckendrick in Management  |  Permalink  | Comments (0)  | TrackBacks (0)

There's been quite a bit of debate in the industry as of late as to whether a "bottom-up" or "top-down" approach is the best way to roll out an SOA program. Why not do both?

ebizQ colleague David Kelly has been tracking the development of SOA within a large, multi-billion dollar US technology company which has been employing such a "dual" approach -- fostering pilot projects in business units not ready for SOA, while at the same time formulating an enterprise architecture strategy build around SOA methodologies.

The company's main challenges include managing a wide range of heterogeneous applications, including ERP, CRM and other packaged applications from a variety of vendors, as well as being able to integrate consistently with business partners, customers and other external entities.

"This company decided on a step-by-step approach," David writes. "They've started off by implementing a selected number of services that are core to a number of business processes and deploying them in situations where there's a high degree of confidence that the solution will work. At the same time, however, they are experimenting and piloting SOA solutions in some areas that are not as fully ready for deployment. This dual-track approach allows them to demonstrate short-term success from specific projects while continuing to move the organization towards a strategic, SOA infrastructure."

Read the rest of David's account here.

Posted by joemckendrick in Management  |  Permalink  | Comments (0)  | TrackBacks (0)

New research from Aberdeen Group shows that to effectively test and provide QA to SOA, companies are testing not only vertically (using unit and functional testing as their benchmark for quality) but are also testing horizontally across an entire business process. In addition, in successful testing/QA environments, there is close involvement of the business user in more phases of the development lifecycle.

Aberdeen Group has just issued a report on the challenges around testing and quality assurance for SOA and Web services.

The study of 240 end-users found that so-called "best-in-class" companies are taking a multi-pronged approach to the SOA testing and QA problem by incorporating automation in the testing lab and process change at the organizational level. More importantly, best-in-class companies are increasing the involvement of business users in all phases of the development lifecycle, and view quality not as something that'’s done at the end of the cycle, but as a horizontal attribute that spans multiple business processes.

The survey found that 61% of the best-in-class companies saw a reduction in the number of defects discovered in production. By comparison, only 18% of the "laggard" companies could say they were able to reduce defects. Overall, 94% of the best-in-class companies reported an increase in the quality of deployed software, versus 17% of the laggards.

How can a company evolve into a best-in-class organization when it comes to testing and QA? The leading companies "have attacked the problem from multiple angles," Aberdeen notes. "It isn'’t enough to just deploy automation, and it isn'’t enough to simply rely on functional tests – QA for composite applications needs a horizontal, process-oriented view, not the vertical unit-test methods used in the past."

Aberdeen notes that the typical best-in-class company gets business users involved in all aspects of quality.uses automation to increase test coverage, and sees quality as more than just an end-of-the-lifecycle task.

Best-in-class companies "are almost three times as likely as industry average and laggard organizations to have redesigned their entire testing process," the report continues. "The best-in-class organizations have made the commitment to services-based applications and have retooled their key processes to reflect their new direction."

The report adds that "many of the changes are focused on bringing quality to bear on the entire lifecycle of the project by getting business users more involved. Doing this requires the top-down commitment of the enterprise. The enterprise as a whole must be willing to set aside the time for key players to be involved in projects as they move from phase to phase, and to supply them with the appropriate tools necessary."

Posted by joemckendrick in Management  |  Permalink  | Comments (1)  | TrackBacks (0)