SOA in Action

As has been the case with UDDI and BPEL, the ESB concept has taken its share of knocks over the years. While proponents say ESBs provide a quick on-ramp to SOA, detractors say they are only temporary fixes at best, and create even more incompatible islands at worst. One analyst, Loek Bakker, even equated ESBs to the phenomenon of a "one-day fruit fly," existing for only a brief time on this earth to serve some mysterious biological purpose.

However, the enterprise service bus has been gaining a lot of respectability as of late.

David Clark of Cape Clear went as far as to say that noted that ESBs are the "next generation application server." In a session at last month's SOA in Action conference here at ebizQ, Clark said that ESBs are currently "the best way to implement SOA." In addition, he added, "we consider ESB the principal container for business logic."

MIchael Meehan of SearchWebServices ranked the resurgence of the ESB as one of the top eight SOA news stories of 2006, noting that a lot of acquisition and open-source activity over the past year has been around ESBs. (Progress bought Actional and combined it with Sonic Software ESB; Sun Microsystems offered an ESB as part of its Composite Application Platform; IONA released Celtix.) "Clearly the ESB was evolving far beyond its messaging roots," Meehan wrote. However, in the same article, ZapThink's Ron Schmelzer cautioned that "the idea of the SOA mega-suite is too un-SOA in philosophy," he said.

The only tangible product we have in the SOA market too un-SOAish? Can that be true? In a new post here at ebizQ, Ronan Bradley warns that there's a simple problem to all the happy talk on ESBs: no one can really define what an "ESB" is. "The core problem with the category has not been resolved and in fact have got worse," Ronan says. "Although every vendor seems to have one, nobody can agree on precisely what one is."

At the same SOA in Action session as David Clark, Sonic Software's Dave Chappell agreed that "unfortunately it's the IT departments who are trying understand and evaluate ESBs that suffer the most from the current lack of clarity of a definition of ESB in the marketplace." Chappell did offer up a definition of ESB, calling it a platform "used to connect mediate and control the interactions between a diverse set of applications that are exposed through the bus using service level interfaces." Clark added that ESBs will continue to change as approaches to SOA change, he added. "As the level of capability and sophistication and understanding of how you can implement SOA increases or evolves, the definition of ESB and how it should rightly behave will also evolve."

Todd Biske, however, took issue with Clarke's declaration of an ESB taking on the role of "application server" while also being pushed as a mediation platform. "You can’t have your cake and eat it too, yet this is the confusion currently being created. The IT teams are being thrown under the bus trying to figure out appropriate use of the technology," Bisk said in a recent post.

"The problem is that the ESB products on the market can be used to both connect consumers and providers and build new services," Todd explains, adding that the orchestration/service building capabilities of ESBs are "getting lost in all the debate because it’s being packaged together with the mediation capabilities." The mediation side of ESBs have "struggled to gain mindshare, since they are more operational focused than developer focused. A product sold on those capabilities isn’t as attractive to a developer. Likewise, a product that emphasizes the service construction capabilities may get implemented without regard for how part of it can be used as mediation framework, because the developers aren’t concerned about the externalization of those capabilities."

Ronan says the role of ESBs as part of the middleware stack is unclear -- and "it appears the industry is speaking, or at least mumbling, with different voices on this one." Ronan observes that a vendor calling its product an ESB "can mean anything from reliable messaging with bells-and-whistles added, all the way across to what was formerly known as EAI."

Ronan also warns that there are now four open-source ESB products on the market, creating additional uncertainty about the long-term viability of commercial ESB products. Not only that, Ronan cautions that the open-source projects also face an uncertain future, "as these are mostly single-vendor backed rather than relying on a broad developer community."

So, the vendors appear to be pretty hot on the ESB concept, which is understandable -- ESBs are one of the few tangible SOA products they can latch on to. (SOA is an architectural concept, not a technology, after all.) But the debate over ESB rages on -- is it a one-day fruit fly for tactical, get-it-out-ASAP service deployments, or are there long-term advantages to building SOA on an ESB backbone? Is ESB for developers, or for operational teams, or both?

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

Last week, I provided live examples of a number of companies that have been planning, building, and managing SOAs, as reported on in this blogsite since the September launch.

Over the past year, I also accumulated a list of enterprises in the thick of SOA over the past year over at my ZDNet blogsite. Here are eight more shining examples of SOA in Action:

Abstracting enterprise information from underlying systems. eBay has built a service architecture to manage more than six million lines of code and two petabytes worth of data, employing SOA middleware to enable integration across disparate technology stacks.

Reducing application inventories. IBM has at least 80 shareable and reusable services in production — ranging from authentication to order fulfillment — as part of its own service-oriented architecture. With its own SOA, Big Blue reports that it was able to reduce its inventory of 16,000 applications in 1998 to 4,000 applications today. The secret sauce to streamlining down to a quarter of its applications was SOA governance.

"Rocking the boat," bringing IT closer to the business, and improving business productivity. IT movers and shakers at Wachovia Bank used SOA techniques to "rock the boat" and changed their organization's culture. Wachovia's SOA consists of business services and frameworks available for reuse across the enterprise. Previously, separate business units had been building duplicated capabilities over and over, which included desktop presentation, data management, workflow management, messaging, and customer information management.

Cutting operational costs. Hewlett Packard implemented an SOA that has seen up to $70 million in savings. HP says the initial paybacks from SOA came through consolidation, reduction of redundancy and reuse across services through its e-business center.

Handling growing transaction loads as simply as possible. Amazon moved off its mainframe to SOA-based middleware to achieve a more flexible architecture that could handle what is now a base of 60 million customers and one million partners.

Enabling the "separation of powers" between corporate, divisions, and departments. Citigroup recognized early on that just as it would be impossible and dangerous to manage a nation of 300 million citizens with a single government entity, it would be just as difficult to manage the IT of a company with 300,000-plus employees and more than $1 billion in revenues every 11 days. Thus, the financial services giants put into place a federated SOA governance structure, with a "separation of powers" similar to the way the US federal government is structured.

Moving business rules out of applications. OnStar, the vehicle communications platform, was moving its software business rules, now embedded in applications, to a middleware layer of reusable components. The company said at least seven or eight application platforms will be moved to the SOA middleware layer, starting with Emergency Services, Vehicle Services, Business Objects and Billing. Such applications help handle service calls and provide remote vehicle diagnostics.

Making movies -- or at least, the business systems behind the movies. DreamWorks Animation SKG, producer of the Shrek trilogy (number 3 is due out in 2007), made a transition to SOA to simplify and consolidate key business operations. The company took a smelly green monster of an IT infrastructure — in the formof 12 legacy ERP applications running on Sun servers — and made it a bit more handsome, in the form of Linux servers, Oracle databases, and JBoss middleware. The SOA model also supports company directories, employee bulletin boards, vacation requests, and cafeteria menus. It also supports a new copyright-tracking application with authorization and authentication features for incoming film scripts.

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

One of the goals of this SOA in Action blog is to relay shining examples of, well, SOA in action. Since the launch of this site in September, we have reported on a number of companies that have implemented SOA, along with any successes they've seen, as well as the challenges that may have slowed down their progress toward SOA.

Here are ten good reasons to go to SOA, cited by ten companies that began their journey down the SOA path:

Integrate back-end legacy systems: International Truck had all types of back-end legacy systems that the company wanted to surface into a "Common Vehicle Tracking system" that could track truck production in near real time, and flag any defects or bottlenecks in production. If successfully deployed, such a system could save International Truck up to $3 million a year. The solution consisted of two Java EE-based interfaces to a homegrown order management system and Baan ERP system. Plans are to extend these interfaces to customers and trading partners. Eventually, the company expects to be able to swap out the order management system with little or no disruption to the tracking system.

Better connect with partners. MedicAlert embarked on the SOA path to achieve interoperability not only between its own internal applications, but also with partners -- hospitals, doctors’ offices, EMTs, and other medical professionals and establishments -- to provide up-to-date personal health records. The company's system, called E-HealthKEY, is intended to provide the foundation’s partners more seamless access to pertinent medical information. “The level of interoperability that is provided by implementing an SOA is really what we’re after,” said Jorge Mercado, lead architect of the software architecture group for the MedicAlert Foundation. “Keeping information in our repository and our repository alone is a good thing, but that’s not where we want to be in the future. We want to be able to share information with hospitals, doctor’s offices, labs, and pharmacies.”

Componentize product offerings: Experian leveraged SOA-based processes and technologies to develop a Customer Event Management system (CEMS) to support its base of leading financial institution customers. The system enables financial institutions to rapidly assess and process new accounts using Experian's online services. The credit giant's services are being delivered as standard components through Experian Web services, including Detect (application fraud prevention) and Delphi (credit scoring). The CEMS was built with the .NET Framework, and is currently being tested by a major financial customer. "We are taking our technology investment and re-engineering customer-related services as a set of Web services. We have smashed large monolithic applications into smaller components. Componentizing all our available product sets is a big and ongoing job, and has changed the way we work in terms of IT development and delivery," said John Finch, director of development and delivery for the information solutions division at Experian.

Abstract multiple ERP functions into a single service layer: Washington Group employed SOA-based middleware to move to an SOA that would abstract many of the functions used in various ERP systems across the company into a common service layer. "By integrating these systems, we can -- on the back end then on the front end -- build some business processes that aren't being executed within the legacy application execution systems, and have it done in a layer above that. We can then update the appropriate systems where needed," said Rich Colton, application integration manager with Washington Group International. "We have not found an ERP system that meets the engineering/construction industry's requirements. We ended up having to use the best of breed in different areas. Up until now, it's required a lot of duplication, triplication, and more, of information from one system to another."

Mirror the philosophy of the business: Ameriprise Financial's credo for its customers is "dream, plan, track" and this is reflected in its SOA effort as well. Ameriprise began developing an SOA several years ago, and lately has been moving its SOA effort from the realm of bits and bytes to a full-blown business proposition. "We realized is that in order for SOA to deliver the full business value, it has to become a business strategy," said Tracy Legrand, chief architect for Ameriprise. The SOA effort followed a roadmap which brought SOA functionality to new applications or services as they came online. Services covered within the company's SOA include customer management, asset management, and moving money between products, between funds, or accounts. Legrand estimates that Ameriprise has saved almost $10 million in cost-avoidance over a three-year period just from one of its leading services -- customer management.

Streamline requests to IT: The IT operations group at Siemens AG, which first built services around automating and streamlining the processes for fulfilling internal requests to IT for new equipment and passwords. Thomas Buse, section manager of concepts and processes for Siemens, said that "once users from various departments started using that system for new workers, they asked IT to similarly automate and improve the processes in their departments." He added that in the process, Siemens cut the time required to implement new processes by 83%. The company releases four to eight new business processes to run on its SOA every six to 12 weeks.

Maintain service quality: The Hartford has a very strong SOA governance effort, led by an SOA steering committee consisting of application architects. Committee members assess proposed new services based on such criteria as supportability, reusability and adherence to the company’s SOA reference architecture. Ben Moreland, architectural director at The Hartford, is also aware of the issues around services pushed into the registry without IT support to back the service up and maintain it, and therefore requires that IT sign off on all proposed services. “We don’t want a junk drawer of unsupported services in our SOA registry,” he said. This process keeps business and IT groups from proliferating "junk" services to the group’s master Universal Description, Discovery and Integration (UDDI) registry, the article noted. Moreland hopes to automate much of the approval process through workflow tools that integrate with its UDDI registry.

Keep vendors on their toes: The Federal Bureau of Investigation has been investigating SOA, and seen impressive results from a more outward-facing -- and smaller-scale -- SOA project. The agency launched a Regional Data Exchange, or R-DEx, a series of information sharing pilots with regional databases. Under R-DEx, the FBI has created plug-ins to Justice Department databases for four regional law enforcement data sharing associations, with more to come -- using an SOA registry built with off-the-shelf IT products. Because the project is built using SOA methodologies, the bureau has already been able to pull out and swap some of R-DEx's components, including an underused portal function and search engines. An interesting effect of this swappability is that it has kept "vendors on their toes," said project manager Margie Lonergan. The knowledge that they're easily replaced, "entices them to make sure they stay best of breed."

Expand a global reach: Monster -- the online jobsite -- recently expanded its reach to 24 countries across the globe, and needed a service oriented architecture that would stretch across separate regional units. Prior to the SOA implementation, new orders would be manually routed to a financial system for invoicing, explained Joan Lawson, director of global integration for Monster. Job postings were then distributed to appropriate regional units via a point to point extract, transform, and load process. "Due to a global marketplace and offshoring trends, our customers were demanding real-time integration of jobs and resumes across the various regional platforms," Lawson said. "An employer entering a job in North America may also have a job available in India." Monster developed an SOA-based middleware layer to take away these manual processes. When a customer posts a new opening, Lawson explained, the SOA middleware delivers the order to the applications within the various regional units. "If an employer sends a file feed of a job that's going to be available in North America as well as in Czech Republic, that will flow all the way through."

Get your motor running: Harley-Davidson employed SOA to to surpass its older, more rigid technology to a more flexible model that enabled it to better support its marketing efforts. Harley-Davidson's CIO, Jim Haney, explained that the company has marketing programs for turning showroom fantasies into realities. "We want to put together a good financial package to entice and incite people to get into the sport," he said. Previously, however, Harley’s credit and loan origination process wasn't up to the task. The company's financial services applications were tightly coupled with each other, and making a change in one program meant making changes to countless other applications as well. Harley's answer was to break it all up. "We actually busted apart all of those systems, and put the SOA with WebSphere in the middle of all that," Haney said. "We loosely coupled these things. Our goal is to be able to change any one of those systems. If we see key indicators in the industry, we want to very quickly put different marketing programs in place, and not have to go and touch and test every single system."

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

For many IT managers and professionals, SOA represents their first attempt to take pieces of applications and make them available across the enterprise enterprise. This is no small task for organizations with longstanding heritages of silos and separate business units, all powered by legacy systems.

In fact, nothing screams "silos" louder than a well-entrenched phone company that has been around for decades, and likely has business processes buried deep, deep, within hardwired systems. In an effort to break this mold, one such telecom, Bell Canada, embarked on an SOA journey four years ago -- long before other companies even were aware of the concept. A new report in Computing Canada describes how it all came together: Bob Noseworthy, vice president of infrastructure and enterprise architecture for Bell Canada, is quoted as saying that the telecom company had a lot of independent business units coming together as part of a consolidation effort. "We did a lot of work four years ago putting together a real, broad enterprise architecture strategy."

Bell Canada is a big, bureaucratic company, so its SOA efforts began modestly. Noseworthy's team started by building a handful of services -- starting with the call center -- that would have a better probability of reuse across the business. "We saw that as a way to learn, get it under our belts, and try to gain some support for that approach," he is quoted as saying, adding that "we have lots of bruises to show for it."

Where did most of these bruises come from? Mainly from trying to overcome internal organization resistance, Noseworthy said. To this day, he added, most organizations -- and their vendors -- are not prepared for the organizational and cultural shifts required for SOA, along with territorial ownership issues. "We were doing this at a time when it was still a very new approach, so there was lots of internal selling," he said.

The internal selling eventually paid off, however. The first call center-based services were eventually reused within the company's public Website and its retail outlets. The Bell Canada team is now concentrating on offering up a list of reusable enterprise services available through a service registry and managed through a governance framework.

Along with business buy-in, challenges include effective governance, change management, performance management, service-level management, and capacity planning for services being made available across the enterprise. "When you start to have multiple business units, multiple delivery teams, all trying to reuse existing services, those kinds of challenges come to the fore," Noseworthy said. "We've done it through trial and error, but it's the school of hard knocks."

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

The latest Web services/SOA from Evans Data finds that more than 40% of developers working on SOA can now complete a typical SOA development effort within three months – more than twice as many as a year ago. Additionally, more than 60% of all SOA projects are completed within just six months,the survey also finds.

I analyzed and authored the Evans Data survey report, which is based on interviews with more than 400 developers and managers, conducted in November. The rise in rapid development of SOA-related projects can be attributed to a growing comfort with SOA methodologies, as well as a maturing of products and tools.

The survey also found that more than four out of ten companies either have SOA projects underway or are planning to initiate SOA within the next 12 months. If number of services is any indication, then we are reaching that point of critical mass. Over the last two years, the total number of companies with more than 40 Web services in production has doubled and that number is expected to double once again in the next twelve months.

Obstacles to SOA are plenty, of course -- especially from the business side of the house. Determining return on investment for SOA is considered the number-one challenge to SOA development and deployment efforts, followed by getting organizational buy-in.

The dramatic productivity increases found in the survey come at the same time as developers and IT professionals embrace Microsoft’s .Net and Java for SOA in almost equal proportions, and both the Java and C# languages’ market share continue to grow as SOA implementations mushroom. Managed code, in fact, is becoming the wave of the future -- three years from now, two out of three SOA developers will be running the majority of their applications in managed code.

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

Service oriented architecture -- at least in its current incarnation -- has now been on the scene for a few years, and we're beginning to benefit from a growing base of experience in growing and managing these beasts. David Linthicum -- who has built his share of service organizations -- boils his insights down to the important points in a new article, Ten Things to Think About When Building the Perfect SOA" published over at Sys-Con.

1. Focus holistically, act locally: SOAs are not mere "projects" -- its about the entire enterprise.

2. Define the value: This gets said a lot, but can't be said enough -- build a business case for SOA.

3. Don't neglect service design: Design services for reuse, not tied to any specific applications or platforms.

4. Embrace legacy systems. They may be the majority of services that you leverage within your SOA. (The System i rules.)

5. Learn to deal with semantics: "If you don't understand application semantics, or, simply put, the meaning of data, then you have no hope of creating a proper SOA," David says.

6. Orchestrate: Define how the services work together, including business logic, sequencing, exception handling, and process decomposition, including service and process reuse.

7. Security now -- not after the fact: Web services are not for internal user anymore.

8. Classify the patterns of use: "Determine how the SOA will be leveraged within the enterprise - not only now, but in the future."

9. Persistence is important: Since services may come out of more than a dozen different systems, so "it makes good sense to centralize the persistence for the composites and processes, as well as some supporting services to a central data tier or central data service."

10. Test, test, and test some more: Develop a test plan -- this is particularly important because of the difficulty of testing an SOA solution. "Most source and target systems are business-critical and therefore cannot be taken offline," David writes. "As a result, testing these systems can be a bit tricky."

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

We on the ebizQ blogging team talk a lot about convergence between SOA and business process management. SOA enables rapid, model-driven approaches to BPM. As SOA increasingly supports BPM initiatives, it in turn evolves SOA from being "just another IT project" that may or may not be of use to a group of end-users to one that truly involves the entire enterprise and delivers value in a real way.

For the latest insights and analysis on BPM, I urge you to visit the sites of fellow ebizQ bloggers David Ogren, Sandy Kemsley,and James Taylor, who all frequently have quite a bit to say about the growing synergies between BPM and SOA methodologies.

Sandy, in fact, just wrote a piece in the Savvion newsletter on the BPM-SOA convergence, noting that "a year ago, many CIOs couldn't even spell SOA, much less understand what it could do for them." Now, only 12 months later, she observes, "Service-Oriented Architecture and BPM are seen as two ends of the spectrum of integration technologies that many organizations are using as an essential backbone for business agility."

BPM solutions are adopting the SOA way, Sandy observes. BPM systems are consuming SOA-based services, and adding human interaction to create a complete business process. In addition, the past year has seen ongoing industry consolidation, "particularly with vendors seeking to bring SOA and BPM together in their product portfolios," Sandy adds. Helping to drive BPM capabilities into the SOA space (and visa versa) is the growing acceptance of Business Process Execution Language (BPEL), a service orchestration language now widely accepted as an interchange format, if not a full execution standard.

Analysts are all hot on this growing convergence as well. In last month's SOA in Action virtual conference, Forrester analyst Ken Vollmer talked about BPM-SOA convergence in his keynote, noting that the time is ripe to begin pursuing combined BPM-SOA approaches to increase the flexibility of business processes. Vollmer observed that "SOA provides a more standards-based approach of doing BPM," and that without the foundational standards of SOA (SOAP, WSDL, UDDI, BPEL), "model-driven development in the BPM tools would not be possible." In other words, BPM would be a slow, expensive manual slog.

Gartner analysts have also weighed in on the trend, predicting that, beginning in 2007, BPM will become the driver for SOA implementations. (Reported here in SearchWebServices.) The technology for the convergence of BPM and SOA may not fully mature until 2010, but the analysts urge the adoption of a "process architecture" to make this convergence work. Gartner analyst Jim Sinur defined a "process architecture" as one that would start with the 10-15 most important business processes and drill down from there to avoid process sub-optimization."

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

"In my travels around the world talking with people about SOA and ESB, I have noticed a shift in people's thinking and witnessed a trend of enlightenment over the past two years."

Dave Chappell of Progress Software, who literally wrote the book on ESBs, has contributed a two-part series to th ebizQ site. In Part 1, he discusses why ESBs may be just what the doctor ordered (or make that what the CIO ordered) to broker the emerging event-driven architectures which are the next phase of SOA.

"The real sweet spot where an ESB has shown its power and flexibility is in process-oriented, event driven architectures," Dave writes. In enterprise integration projects, "the key to success is to have an architecture that allows for each application to be decoupled from the rest of the SOA by using the ESB as a form of mediation."

Dave adds that "the event-driven interaction style is a major advantage of keeping applications decoupled from one another. When plugged into an ESB, each application does not need to understand the intimate details of how to interact with all the other applications. The ESB handles all the protocols, data formats and different interaction styles." However, he cautions, its important that the ESB implementation have "reliable asynchronous messaging and high availability capabilities" to avoid errors in the transaction chain. In addition, he adds, the applications themselves often "need to be written or adapted to this event driven style of interaction."

In Part 2, Dave discusses the various jobs ESBs must do, including Web Services Management and SOA governance, Advanced Web Services (WS-*), complex event processing, and semantic data integration.

Interestingly, Dave dove into the role ESBs can play in complex event processing (CEP), sometimes known as event stream processing, which is a relatively new field in the area of Event Driven Architectures. CEP is about capturing and analyzing high-volume data streams coming out of algorithmic trading, fraud detection in financial services and supply chain management with Radio Frequency Identification (RFID) processing and filtering.

"Typically, a CEP engine that is plugged into the ESB as a service will perform these tasks," Dave explains. "The stream of events may come through the ESB or may be another source such as an external stock ticker feed or an RFID reader. The course of action taken when a complex pattern of events is identified may vary, but can range from alerting a business user in a Business Activity Monitoring (BAM) dashboard or to invoke a service or a business process via the ESB."

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

In previous posts here and elsewhere, I have talked about the the misconceptions around the "JBOWS" architecture (Just a Bunch of Web Services), which is often the early, and very incomplete, stage of service oriented architecture.

Nick Malik, a software architect with an integration firm, and a former Microsofter, wants to make it clear that having a plain old Web service (POWS?) is not the same thing as an enterprise-grade, SOA-ready service. Within an enterprise SOA, every service in the enterprise catalog should be automatically verifiable, and basic information can be collected and reported.

Nick lays out the following criteria to define the requirements for having a full-fledged, enterprise-ready, SOA kind of a service:

Keep it simple: "The service must be simple to call," Nick says. Do thye validation in your service, and avoid writing DLLs that create the data and then calls the service. With DLLs, "you customers still have a call-level interface -- iInstead of it being the service, it's the DLL. Nothing gained," he adds. Plus, "EAI components now need an adapter to call your service. Something lost."

Directly callable: "The service must be able to be called directly from EAI components, like Biztalk or your favorite ESB, without engaging an expensive or custom adapter Requiring an adapter means that you don't have a truly sharable service."

Include 'ping' and 'check' tools: "Services running at the enterprise level must have methods that can be called to validate that the service is running, and that downstream connections are valid. Nick suggests that every service should have standard methods for "ping" and "check."

Include infrastructure data: "Services running at the enterprise level should provide data about what servers, data centers, and support teams they are associated with, live and in real time," Nick says, suggesting that each service have a standard method for "GetSupportData" that returns a structure containing all the info.

Keep it asynchronous: "Services running at the enterprise level should normally be asychronous, with a very rapid return of an acknowledgement that indicates that the service itself takes ownership and responsibility for the message."

Correlate messages: "For async message exchange, the receiving side must send back the caller's unique ID, along with any unique ID it has created."

Standardize the call-back structure: This allows "any caller to easily construct the data needed for the receiving system to understand how to call the sender back with the response package."

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

The tried-and-true IT management formulas that worked in the past may no longer work in SOA: In the SOA world, it's not so easy for an IT manager to know what's going on or what's going wrong.

There's plenty of discussion in the industry -- and on this blogsite as well -- that service-oriented architecture differs sharply from normal IT operations because it is an enterprise-wide effort requiring enterprise direction. It's not about technology, but the business right? (Isn't that we hear in all the seminars?)

However, since SOA is founded on technology, ultimately, its day-to-day success rests in the hands of IT managers. Can standard IT management practices be applied to SOA? Or do we need a new approach to managing technology, governance, and funding models to make SOA work the way it's supposed to?

Rich Seeley, writing in SearchWebServices, raises this question, and concludes that current IT management practices need serious revisions before they're ready to function in the SOA realm. "In the SOA world, it's not so easy for an IT manager to know what's going on or in the worst-case scenario what's going wrong," Seeley writes.

Or, as Mary Johnston Turner, VP of Ovum Corp.'s Summit Analyst Firm put it: "How do I know if my end-to-end business process is working? And if it's not working, how do I figure out which one of those interconnected pieces is the culprit?"

As IT increasingly becomes responsible for managing shared services delivered and accessed across and from outside the enterprise, some experts advise adhering to the principals laid out within the IT Infrastructure Library (ITIL) framework. Originally developed in the UK, ITIL has been steadily gaining converts across Europe and North America. ITIL provides a management framework for 10 discrete processes that focus on IT service support and service delivery.

A few months back, I had the opportunity to speak with Troy DuMoulin, executive consultant with Pink Elephant, a consulting company that has been promoting ITIL adoption on this side of the Atlantic. He observed that it's getting more difficult to separate business processes from IT -- in fact, they're now one in the same.

"There’s an evolutionary change in IT’s position with the business," DuMoulin related. For example, recent compliance regulations -- Sarbanes-Oxley and HIPAA, for example -- "made a very interesting statement. They basically say that the financial results of a company and all its protection of its data is related to business processes – that makes sense, business processes produce data. But the problem is those processes were automated as IT services and systems. So what you have is no real ability to separate the business process of accounts payable from the financial system of SAP."

In essence, DuMoulin says, "There is no longer a separation between IT and business processes. For years, we thought of IT here on the left hand, and business on the right hand, and IT lobs services over the wall at the business. But there is no longer a wall."

Which brings us back to ITIL. With processes so dependent on technology, and SOA increasing that dependency, there's going to be more pressure on organizations to show that their IT departments are following standard, accepted management practices.

In fact, some proponents out there say ITIL makes SOA easier to implement. In a post last year, David Tyler did a good job of explaining the ITIL-SOA connection, complete with working examples. He observed that "in talking to companies and helping people with their SOA initiatives, I have found a common thread through out. If you have ITIL/ITSM implemented in your IT, implementing SOA is going to be much easier." Why is that? Because ITIL focuses on monitoring, reporting, and responding, which are key elements to SOA success.

But the lines between ITIL and SOA are not yet clearly understood, said Ovum's Turner, who noted that while ITIL is popular, few organizations are applying it to manage SOA. However, complex SOA implementations will need frameworks such as ITIL or Six Sigma.

"The more you rely on SOA, the more pressure that's put on traditional IT management," Turner said in the SearchWebServices article. "If you don't have a good agreement as to the composite service you want to deliver and you don't know how to measure the end-to-end performance in a way that's going to show everyone that you did your job, it's going to be a nightmare."

Posted by joemckendrick in SOA  |  Permalink  | Comments (1)  | TrackBacks (0)

Lately, there's been an increasing amount of discussion around the delta between SOA and virtualization. Todd Biske (MomentumSI) talks about this growing convergence in one of his latest "Outside the Box" posts, observing that "SOA creates a need for more efficient resource allocation," which is the goal of virtualization as well.

In the good old days, when everything could be plopped on a single systems, such as a mainframe, "a wily mainframe operator can do quite a bit to make sure that everything still manages to get done within the processing window," Todd said. With SOA and its increased interdependencies between systems and real time processing, "if we don’t give the operational staff the tools they need to efficiently manage those resources, we’ll be in an even bigger mess. Virtualization is one tool in that chest."

Speaking of the delta between virtualization and SOA, Delta Airlines appears to be taking the virtualization route to move its customer management systems to SOA. ComputerWorld reports that Delta Air Lines has launched to replace its core IT backbone with a service-oriented architecture (SOA).

Delta's IT subsidiary, Delta Technology Inc., is charged with updating the "Delta Nervous System," which the article describes as "an IT backbone used to route messages among multiple systems," managing everything from tracking passenger check-ins and boarding to the SkyMiles frequent-flier program.

Rawls Whittlesey, Delta Technology’s chief architect and director of enterprise architecture and middleware frameworks, is quoted as saying that Delta will be replacing its Tuxedo middleware from BEA Systems with standards-based SOA technology.

The airline's IT operation will port more than 100 Java, C++ and .NET services written with Tuxedo to TIBCO's ActiveMatrix platform, which the vendor launched last week. According to TIBCO, Active Matrix employs a "service container approach" to essentially function as an SOA virtualization platform.

The service virtualization will help reduce application configuration requirements, Whittelsey said. “You want to try to insulate the applications from having to know what server they are on and what environment they are in,” and by using virtualization, “the application teams don’t have to know where a service lives."

Posted by joemckendrick in SOA  |  Permalink  | Comments (1)  | TrackBacks (1)

I know it's still early in the life of service-oriented architecture, and most companies are just starting to wade into the SOA waters, but it's always helpful to look what is coming in the next wave. Two compelling write-ups, one by Jack van Hoof and the other by Brenda Michelson (a colleague here at ebizQ), provide a very detailed picture on what to expect next in the wake of SOA -- Event Driven Architecture, or EDA.

Some industry mavens, in an attempt to start capitalizing on EDA, gave it the stupid label of "SOA 2.0." Thankfully, the SOA 2.0 moniker was quickly shot down and put out of its misery.

Jack van Hoof, who is enterprise integration architect with Dutch Railways, has mapped out what event-driven architecture will mean to enterprises. van Hoof explains that EDA delivers the loose coupling that is only promised -- but not delivered -- by SOA. "EDA is not a synchronous command-and-control type of pattern, but just the contrary: an asynchronous publish-and-subscribe type of pattern. The publisher is completely unaware of the subscriber and vice versa; components are loosely coupled in the sense that they only share the semantics of the message."

There are roles for both SOA and EDA, van Hoof says. "If you are seeking to support strong cohesion in the business processes, situations where all process steps are under one control, SOA is the way to go," he says. "If you are seeking to support independence between business process steps, EDA is the way to go."

Brenda Michelson, principal at Elemental Links, also identified, at a very early stage, EDA as not just the next evolution for SOA, but an architecture that can effectively work in conjunction with SOA. Brenda noted that as we move forward, "the most viable, agile architectures will be comprised of a blend of architecture strategies, including (but not limited to) service-oriented architecture, event-driven architecture, process-based architecture, federated information, enterprise integration and open source adoption. How you blend, depends on your business."

Brenda provides an outline of how the interaction between EDA and SOA is occurring: First, "the occurrence of an event (a notable thing that happens inside or outside your business) can trigger the invocation of one or many services. Those services may perform simple functions, or entire business processes. This interaction between events and services is commonly referred to as event-driven SOA."

Next, "a service may generate an event. The event may signify a problem or impending problem, an opportunity, a threshold, or a deviation. Upon generation, the event is immediately disseminated to all interested parties (human or automated). The interested parties evaluate the event, and optionally take action. The event-driven action may include the invocation of a service, the triggering of a business process, and/or further information publication/syndication. In this interaction, the service is purely one of many event sources in a broader event-driven architecture. A broader event-driven architecture stretches beyond event-driven SOA, to include real-time information flow and analysis, and complex event processing."

Posted by joemckendrick in SOA  |  Permalink  | Comments (2)  | TrackBacks (1)

Attend any SOA conference or read any SOA article, and you can be forgiven for thinking that 'the technology's a cinch, but redrawing business processes are a bear.'

Eric Knorr and Galen Gruman beg to differ somewhat with this widely held view. True, the technology is ready and available to perform all kinds of magic for the SOA,from routing transactions between partner systems to providing analytics on system events. But, the challenge is being able to sift through an array of often bewildering choices to determine the best technology fit for a budding SOA. Writing in CRM Daily, Knorr and Gruman sum up the challenge this way: "Technologists must make key decisions about the platforms on which to build services, as well as how those services will be exposed, managed, and mediated. Some companies may opt for an ESB (enterprise service bus) to connect services, whereas others may focus on standards-based services designed for maximum reuse."

To ESB or not to ESB: Enterprise service buses are another, sometimes vexing, technology choice that needs to be made. For example, the article states, payroll processing giant ADP went years without an ESB, but recently adopted a distributed ESB because "it's difficult to maintain a bunch of one-to-one messaging," according to Bob Bongiorno, ADP's CIO of employer services. The company's number of services grew from nine to more than 30, but along the way, "the management complexity has far more than tripled," he says.

However, the article also quotes Anne Thomas Manes of the Burton Group, who equated ESBs to warmed-over EAI: "EAI is fundamentally different than SOA," she said. "EAI is about bridging business process silos; SOA is about breaking them down." However, Manes disagrees that an ESB should be a gateway to all services, and that there are other alternatives to ESB for managing point-to-point services, such as XML appliances.

Governance and incentives (or disincentives): The technology underpinning SOA needs to have ironclad governance behind it, the article notes. For example, the authors describe how BT (formerly British Telecom) has developed 14 service platforms, each with its its own sets of services, and each with its own architect. The architect's job is to "ensure that all services -- whether developed in-house, provided by a partner, or bought from a vendor -- adhere to the architecture." And there's an enforcement stick to go with it -- "if a BT project doesn't meet the architecture, the development team loses a quarter of its annual bonus. To enforce that compliance, if a BT project doesn't meet the architecture, the development team loses a quarter of its annual bonus."

The topic of incentives versus disincentives is an interesting one that we'll hear more of as enterprises figure out how to promote SOA adoption. Last month, for example, I described AT&T's carrot and stick approach, in which development teams were informed in a memo from a high-level VP that their jobs would be in jeopardy if they did not adhere to standards when rolling out new apps. It isharsh, and AT&T VP Rich Erickson admitted that punitive measures probably wouldn't have gone that far, but it did put standards foremost in everyone's minds.

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

The mission of this blog is to provide insights for planning, building, and managing SOA. I've talked plenty about building and managing SOA in this space, so it's time to give planning some equal time.

In fact, ebizQ just posted an article by Eric Roch, national practice director of Perficient, which presents a great discussion of the planning that needs to go into SOA.

Roche's first piece of advice is: Don't attempt to apply the tried-and-true formulas that have been used for ERP and other IT projects in the past. "An ERP deployment best practice is unfortunately to force a change to the business process itself to support the package out-of-the-box," he writes. SOA is about adapting technology to the business, not the other way around. (I actually posted a blog with this theme, "Processes Should Shape Software, Not the Other Way Around," in October, which describes the SOA promise not to force-fit processes into software.)

Next, to bring the business into the SOA planning process -- and avoid creating some monster that nobody will use -- Roche advises the creation of an SOA Steering Committee comprised of members from across the organization. The membership should consist of the SOA sponsor, a lead architect from each functional-application system and an enterprise architect. What will this committee do? Their job will be to "establish standards, processes, and the SOA roadmap the organization can manage the dramatic changes necessary for the SOA transformation."

Next, there's the vision thing. The SOA Steering Committee need to devise a strategic services "blueprint" in which proposed services are determined "by decomposing business functions keeping in mind the cross-functional needs (reuse potential) for each candidate," Roche says. "Service decomposition looks at the enterprise as a whole and seeks to define the core services needed to run the company - the future-state services catalog."

The committee needs to determine areas that may benefit from service enablement, such as customer service, business partner integration, supply chain, employee benefits, and product lifecycle management efforts.

The key factor here is that the business needs to be in the driver's seat of the SOA effort -- in too many SOA efforts these days, IT often is going it alone, a move that will only result in rudderless, siloed SOA projects.

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

Without quality assurance, there is no trust in the services that are delivered via SOA. Without trust, there is no reuse.

Hurwitz & Associates, in conjunction with Mindreef Software, recently issued the results of a survey of 99 enterprises in which hey measured the value of reuse. (An ebizQ news summary is available here.)

Reuse of services was seen as the leading benefit of SOA, cited by close to 90% of the respondents. Just under half of the companies surveyed, 47%, reported they were satisfied with the level of reuse they were achieving. This is a much higher level than in any survey I've seen. In fact, a recent survey sponsored by BEA found that for the most part, companies only expect to reuse 20% or 30% of the services they create.

Interestingly, the report's authors, Carol Baroudi and Dr.Fern Halper, talked a lot about service quality as the way to promote better reuse. "As SOA takes center stage, Hurwitz & Associates believes that unless organizations pay attention to software quality and an overall formal SOA governance process, expectations for reuse will not be realized. According to our results, only 22% of those respondents that have implemented a SOA have some sort of quality plan in place."

How much of a difference can quality make? Quality means more trust, as well as reliability. The authors go on to note that "those respondents who had implemented a SOA environment in conjunction with a quality plan were more likely to be satisfied with their deployment than those respondents without a plan.... 42% of respondents who implemented a SOA along with a quality plan were completely satisfied with the quality of their SOA, while only 7% were completely satisfied if they implemented a SOA without a quality plan in place."

That's a big difference, for sure. What would constitute a "quality plan" for SOA? "A registry and repository is not enough to ensure SOA quality. SOA governance is not enough," the authors state. Rather, quality needs to be a rigorously enforced "strategy." Such a strategy involves moving to standards-based solutions, as well as conducting lifecycle testing of services.

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

"Sometimes I lie awake at night, and I ask, 'Where have I gone wrong?' Then a voice says to me, 'This is going to take more than one night.'" -Charlie Brown.

Perhaps Charlie Brown should invest five cents for Lucy's expert counseling. But even Lucy may not be able to relieve the angst that comes from SOA implementations that fall short of their goals.

Ash Parikh and Murty Gurajada, writing in JavaWorld, say just as Charlie Brown has his worries, what keeps many IT managers up at night are the disappointing results they have seen from their SOA investments, which have been spurred by vendor overpromising.

What's gone wrong? Along with confusion in the market, and corporate cultural barriers to cross-enterprise sharing of services and data, the authors note that many implementations have either been non-SOA or halfhearted SOA approaches that "lead to custom integrations, which cost a lot of money and make the systems more tightly coupled, thus exacerbating the problem."

Fear not, say Parikh and Gurajada, who provide a roadmap for the successful ingredients for SOA. The path to SOA excellence includes effective governance tools and practices, along with business process management and workflow solutions.

At the core of any well-designed SOA should be a metadata repository and data orchestration engine, they observe. "It should not only contain all the data and metadata relevant to an organization's functioning, but also provide a comprehensive set of features to manage these artifacts effectively."

In addition, they add, all highly functioning SOAs have another thing in common -- they also include business process management and workflow elements. "The application layer may discover and consume services directly from the service registry/repository layer, or it may go through the workflow/business process management platform for more flexibility," they write.

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)