SOA in Action

Sandra Gittlen, writing in Datamation, explored "three ways to avoid SOA snafus.* Here are the three biggest snafus and how to address them:

1. Operating in a vacuum. SOA is an enterprise project, and should be managed as such. "You have to really get everyone on board -- from the programmers to the shareholders to the end users -- to be successful with SOA."

2. Not creating a governance framework. Governance is a large part of developing an SOA strategy, and organizations must decide ahead of time what their reuse and publication policies will be for the services within their architecture."

3. Treating security as an afterthought. "Security can be one of the trickiest areas of SOA because organizations are reusing services that might have been created externally. "Most services, to be made as reusable as possible, have security stripped out of them. This isn’t good.” IT groups should add service interfaces that address compliance mandates and cater to the highest level of security each application requires. "n addition, IT groups should carefully plot out their authorization and authentication strategies."

I would like to add a fourth bit of advice, which Sandra mentioned in the article but did not elevate to a "Big Mistake:"

4. Pushing SOA through as a massive enterprise initiative. That's sort of like, as they say, trying to "boil the ocean." SOA should be introduced incrementally, starting out with quick wins to demonstrate its value to the rest of the organization. If SOA is launched as this galactic transformative mega-project, constituents will be quickly disappointed when they don't see mega-results to their areas of the business.

Posted by joemckendrick in SOA  |  Permalink  | Comments (1)  | TrackBacks (0)

Does Business Process Execution Language lack the "human touch"? Some industry leaders say that BPEL is too automation-centric, and lacks support for human interaction with the workflow. However, not everyone agrees on the best way to resolve the matter.

Process workflows are like the rivers that dot the planet; each one has its own unique sources and tributary streams, terrains to be travailed, and eventually emptying out somewhere, be it an ocean, bay, larger river, or lake. But there are also plenty of waterfalls, dams and locks on the way. Workflows are as unique as the companies that create them, and all have their own points where humans intercede.

Responding to such concerns, a couple of years back, IBM and SAP put forth an extension to BPEL called BPEL4People, intended to be "layered on top of BPEL as specification" describing a proposed extension to BPEL, intended to cover a "broad range of scenarios that involves people within business processes."

As the IBM-SAP white paper explained it, "during the lifetime of a long-running business process, conditions that require human involvement can occur. An example of this would be if a process is stuck because no one has been assigned to perform a particular task. Another example would be if a process waits for input from human participants or Web services, and the input must be collected within a certain number of hours. If the timeout occurs, a user must be notified to decide how the process should proceed."

As BPEL exists today, getting a process "unstuck" would "require undoing parts of the business process. But if the business process has already run for multiple days, invoking partner operations, collecting results, and so on, compensation may not be desirable, since this wastes resources and efforts already spent." Thus an administrator will have to manually intervene to take the corrective action to get the process moving again.

BPEL needs to incorporate a "special kind of implementation of an activity - a communication step which may be called 'people activity.'" The paper defines people activities, simply, as "tasks performed by users," noting "there are scenarios where it is desirable to define which people are eligible to start a certain business process."

This week, IBM and SAP were joined by Active Endpoints, Adobe, and Oracle in the publication of the BPEL4People specifications, which define an approach for integrating human interactions using Web Services Business Process Execution Language ( WS-BPEL ) 2.0.

ebizQ colleague Michael Dortch has just posted his observations about BPEL4People, noting that the spec "could bridge and significantly narrow the gap between service-oriented architectures (SOAs) and the humans attempting to use the services to do work that makes the business go."

Michael says the prospects for BPEL4People are promising, but cautions that "the vendors supporting BPEL4People have to do three things, and they have to do them quickly, transparently, and well." This includes attracting more vendors to the fold, delivering products that actually use BPEL4People, and third, make it an official OASIS industry standard.

We are only in the early stages of automating business process management, and have only begun linking business processes to SOA. Processes get touched many times, and sometimes are required to be by law or regulation. BPEL promises to speed up much of our workflows, but the points requiring human interaction may negate efficiency and speed gains. The question is whether BPEL4People - or other approaches - can compensate for the human equation.

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

Anyone listening to or reading many of the vendor statements and pundit analysis these days can be forgiven if they think that SOA is pervasive across all organizations large and small. However, most complete SOA efforts are still few and far between. In most cases, organizations are wrestling with a spaghetti architecture of Web services.

Consider these trends:

- Web services runs wide, but not deep: For a number of years, Web services has seen widespread adoption across enterprises, but for only select applications

- Reuse is gaining traction: The value seen in SOA is service reuse across more than one business unit; survey confirm that this is beginning to take place

- The blurry line between JBOWS and SOA: The journey to SOA is a gradual process, most companies are in stages in between.

I’ll be delivering a presentation at SOA World in New York tomorrow (Wednesday, June 27) on the topic of “JBOWS* or SOA — A Reality Check.” (*Just a Bunch of Web Services). I will explore some survey results that explore how far along companies are in moving from JBOWS to SOA.

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

Web services may offer the path of least resistance to SOA, but theoretically, are not essential to SOA.

SOA is now being employed for the monitoring controls for Europe's largest particle accelerator, but employing Java-based technologies versus Web services. And all seems to be humming along nicely.

Particle accelerators at the CERN ((European Organization for Nuclear Research) are monitored by an event-driven service-oriented architecture (SOA) system based on Java Messaging Service (JMS) and Enterprise Java Beans (EJB).

A new TechTarget report describes the center's Technical Infrastructure Monitoring (TIM) system, which is SOA built on Java standards and technology, versus Web services standards and technology. The system monitors vital signs such as temperature and pressure within the accelerators including the new 17-mile Large Hadron Collider (LHC), the world's largest particle accelerator. IN the event of failure, an alarm will sound.

The SOA-based system takes readings from 30,000 gauges and publishes them to an enterprise service bus. Technicians at workstations and PC browsers -- as well as autonomic systems and auditing databases --subscribe to the service readings.

The TIM system is built on Apache servlet engines, Oracle application servers, and uses the SonicMQ JMS messaging technology from Progress. For desktop views, which mostly go to technicians at workstations, but also includes browser-based systems, CERN used JViews from ILOG Inc.

"This is a huge system of very different types of equipment," Peter Sollander, technical infrastructure operations department manager at CERN, is quoted as saying. "There are 30,000 data points coming from 100 different local systems. We process 1.3 million value changes per day. That's the throughput we're dealing with. We plan that it will go up with the introduction of new systems, new data sources."

Interestingly, as the system is built on Java/JMS-based services, no Web services standards are in use. "This is very much event-driven SOA, but there are no Web services in use," said Hub Vandervoort, CTO of the Enterprise Infrastructure Division at Progress Software Corp.

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

One point I and many other analysts in the ebizQ blogosphere proclaim with great persistence is that SOA and business process management are made for each other. (I just said as much last week, here.) A service-oriented architecture provides the systems flexibility and agility to model and design business processes as needed by the business.

However, there are voices that question the legitimacy (or at least the efficacy) of this marriage.

Loraine Lawson points to a new post by Steve Jones, who, for one, is now questioning the notion that "a service is a specific business process step that can be composed and reused in different business processes," calling it "completely and utterly wrong."

Steve is no lightweight in this market, as he is the head of SOA and SaaS for Capgemini’s global outsourcing business, member of the OASIS SOA Reference Model group, and author of a number of technology books, including Enterprise SOA Adoption Strategies.

The problem, Steve says, starts on the BPM side, with efforts that seek to address services as "steps." As a result, he says, "every service looks like a step."

What's wrong with that? Steve says that in the process, legacy function calls merely get swapped out with new protocols, but that does not a service-oriented architecture make. If you are using a BPM approach and saying "step = service," then "you are doing VISUAL Cobol and replacing function calls with service. The fact that you are using WS-* or XML does not make these elements 'services.'"

In essence, he says, "BPM-driven SOA tends to make bad SOA as it is driven from a procedural and process view, has poor separation of concerns and is mostly all about driving things from the technology perspective. SOA makes great BPM, BPM makes crappy SOA."

Strong words, indeed. And, ironically, Steve is arguing that BPM-driven SOA is too technical. That's one of the greatest arguments around SOA, is that it needs to be business-driven, and linking SOA to BPM is seen as a way to introduce business drivers to the effort.

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

SOA approaches and offerings are increasingly incorporating “event-driven architecture,” in which processes are launched or redirected in response to specific cues from applications. One of the most compelling enablers of real-time enterprise is radio frequency identification (RFID), which can provide a broader, real-time picture of where products or transactions are at within the value chain.

Industry experts agree that coordinating new data sources such as RFID presents new challenges and opportunities in the move to more of a real-time enterprise. A little while back, I had the opportunity to chat with Peggy Chen, product director for RFID and sensor-based services at Oracle. She explained that the key is in identifying and drilling down to the data that is of value to the business. Such adroit and targeted data management “does help optimize the data, as well as help alleviate the big scare out there that’s there’s going to be too much data,” “Part of it is being smart about your data, and knowing how to actually store and manage that.”

RFID and other sensors help bridge the gap between the physical world and IT world, Chen related. “In a real-time business, managers need to be able to act on information in a more timely manner – not just the information we have today in data centers, but data about all products and assets. It’s all of this information together that builds the full picture, and turns it into insight and intelligence that’s going to help them make better decisions.”

Daniel Linstedt, chief technology officer for Myers Holum Inc., cautions, however, that enterprises need to tread cautiously and deliberately with RFID-type technologies. “If your RFID is feeding data to your systems, every 20 seconds, or 10 seconds, or every time it moved, what’s the value of knowing that at the high level of the organization? Is there a competitive value to knowing that this component with an RFID tag has mixed from shelf A to shelf B? There may be certain points, data gates, where it’s important to know exactly where something is. But in between those points, it’s extraneous data, it’s information overload, unless it answers a specific business question.”

Chris Clauss, director of Sensor Information Management at the IBM Software Group, was recently interviewed in Dr. Dobb's on the potential of using SOA to build high-performance applications that can capture and read RFID data.

Clauss reports IBM has been working with a standards body called EPC Global, which has defined two Web services -- one around capturing and bringing events into a repository; and the second around query, to selectively draw back out RFID events to solve some problem.

"What this allows us to do is it allows us to separate the data capture from the data usage, and to have a high velocity data repository for allowing me to use RFID events by filtering them," Clauss said.

Clauss explains how such a tool would be applied in a business setting:

"Let's say I'm using RFID in a factory to manufacture products and to ship products, and I have a certain application that's really only focused on shipping. That application can subscribe to only the shipping events and it can ignore and not get notified of any of the manufacturing events. So basically what we're again trying to do is use Web services and Services-Oriented Architecture to separate the RFID event capture from the RFID event usage in order to enable multiple applications to access the events, and we very much do this in a high-velocity manner so we want to treat every incoming event as a mission-critical event. We don't want to drop any events and we want to make sure that events can be gathered and used internally."

The new EPC standard also enables enterprises to exchange sensor events with trading partners, Clauss said.

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

Nick Malik recently published, in one spot, the four fundamental concepts of SOA. Nick observes that these four concepts are tightly interrelated, though they might not seem like it when reviewed one by one:

Enterprise Canonical Data Model - The 20% of data that matters. As Nick puts it: "The data we all agree on.. to do our business."

Canonical Message Schema - What the data means.

Event Driven Architecture - An event with one application triggers an event with another downstream, and so on. "A set of relatively independent actors who communicate events amongst themselves in order to achieve a coordinated goal."

Business Event Ontology -- The list. "A reasonably complete list of business events, usually in a hierarchy, that represents the points in the overall business process where two 'things' need to communicate or share."

Nick sums up the relationship this way: "Business Events occur in a business, causing an application to send a Canonical Message to another application. The Canonical Message Schema is a subset of the Canonical Data Model. Event Driven Architecture is most efficient when you send a Canonical Message Schema message between components. This provides you with more consistent data, which is better for creating a business intelligence data warehouse at the end."

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

If enterprise architects act as the "city planners," who worry about the way neighborhoods are developed, and how people and commerce can flow most effectively, then what is the role of SOA architects? Should they function as the service providers, maintaining telecommunications, police and fire services?

Or, shouldn't enterprise and SOA architects they be one in the same?

These are the questions debated by panelists at the most recent SOA BriefingsDirect podcast, led by Dana Gardner. (Link to podcast here.) Panelists included Steve Nunn, vice president and COO of The Open Group, John Bell, an enterprise architect at Marriott International, Jim Kobelius, and Neil Macehiter.

Marriott’s John Bell describes the role of enterprise architect more as a city planner, one who oversees how the entire landscape comes together. “When they’re looking at the entire city, they’re looking at how the various neighborhoods, how the various business zones, etc., fit into that city.” The SOA architect, on the other hand, is focused on delivery of city services and facilitating communication within the city.

“My view is that the enterprise architect is at the top of the hierarchy, and at some place, working with the enterprise architect is an SOA architect, and their focus is on, “What are the services that are being delivered, how am I delivering them? What’s the infrastructure I am using to deliver it? Do I need – using that town model — a police station? Do I need a fire station? Do I need a school? Do I need a museum? And, if I do, how do I get that service out to the community or to the entire city, not just an individual neighborhood?”

However, Jim Koblieus and Neil Macehiter saw the SOA architect’s role as more expansive than facilitating communications. “The challenge that many SOA architects face is more around understanding what the services are that need to be delivered in a business-meaningful way, not just about communication and plumbing. It’s also about understanding the high-level, business-meaningful services,” Niel observed.

Neil also said he finds that trying to make a distinction between SOA and enterprise architects is difficult:

“There is a business strategy, there are business processes and priorities, and there are the services we need at a business level. Then, there’s a handoff to what’s currently defined as the SOA architect, who will actually define how those services are deployed in technology terms. So, the distinction is quite blurred. A service-oriented approach is one of the methodologies and the approaches that you can utilize to deliver or to support an enterprise architecture initiative.”

Jim said if there is a distinction, he’ll put “enterprise architecture at that very top layer, concerned with the end-to-end set of resources.” that is followed by the SOA architect at the middle layer of development and reuse. At the layer below that, the IT or infrastructure architect handles components such as ESBs.

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

At Wednesday's BI in Action virtual conference, I will be joining the Beth Gold-Bernstein, along with the folks from Business Objects and Savvion, in a panel discussion on the relationship between Business Intelligence, Business Process Management, and SOA. (Link and registration here.)

What is the relationship between SOA and BPM? Sure, the two need each other, but how much? Is it a mutually dependent relationship, or a casual friendship, or somewhere in between?

Recently, Tibco's Emily Burns brought some clarity to this uncertain relationship, observing that SOA and BPM are more like to two sides of the same coin. One needs the other, she explained, because "a coin with only one face is of no value, because it is no longer legal tender. BPM without SOA or vice versa, while not useless, is certainly of less value to the business. BPM and SOA can reciprocally leverage the assets of each other. By unifying them, the incremental value is not the arithmetic increase of 1+1, rather it is an exponential increase in the services and processes made available to each."

She observes that bringing SOA together with BPM "unparalleled visibility into all the workings of a business process-from design through deployment to production. This approach results in greater and more effective engagement of every person involved in managing business processes, from business analysts, to IT, to system administrators."

Perhaps the biggest advantage to building BPM on SOA is in the amount of manual labor needed to make BPM a reality. Traditional BPM efforts have always required extensive application development to integrate everything and make it all work. Thus, businesses would not only assume the costs of BPM software, but also of the associated development that goes with it. SOA, which involves the deployment of reusable services that can be assembled to form new "composite" services and applications, cuts out a lot of development time.

I recall how Forrester's Ken Vollmer described BPM without SOA as akin to "juggling with oine hand tied behind your back." As he put it: "Business process management could be implemented without SOA, because both capabilities have been available for some time. But implementations of BPM without the SOA foundation take longer. There's more ongoing maintenance and support costs are higher, and therefore ROI is delayed. An analogy of doing BPM without an SOA foundation would be like a juggler with one hand tied behind his back. He can still do some juggling, but not as effectively and not as fast as it could be."

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

We hear a lot about SOA governance in the context of policy enforcement and change management. But, ultimately, none of this will mean a thing if there isn't a meeting of the minds. That's what will make or break an SOA effort.

Let me put it into a simple formula:

In other words, various business units, development and architectural teams will need to open up, lay down the weapons, put aside the conflicting agendas, and talk with each other. Easier said than done sometimes, I know. But without negotiation, you end up Service-Averse Architecture instead of SOA, and my colleague Elizabeth Book will not be happy to hear that.

In a new post, MomentumSI's Jeff Schneider discusses the role of negotiation in the SOA governance process:

'SOA will force more occasions where departments and business units will need to find a common ground. I.T. shops have had the need to negotiate for shared infrastructure in the past. If you move forward with SOA, this activity increases significantly. There are no magic answers to SOA Governance. ...Make sure that the tools, processes, roles and committees are in place to make the negotiation process as efficient as possible. Said another way, a competitive advantage for the Service Oriented Enterprise is the ability to efficiently negotiate differences and take action."

Well said, Jeff.

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

Some industry observers have equated service-enabling formerly proprietary legacy applications to "putting lipstick on a pig," and this serves, at best, as a temporary solution until a full-fledged SOA environment is put into place.

I think some legacy systems, such as mainframes and System i, can be full-functioning citizens of an SOA community, but I digress.

United Airlines, which reportedly has been working on service-enabling its back-end systems since 2001, decided that the time had come to take the next step with its previously service-enabled mainframes to move to an even more open-standards-based middleware architecture. "We are in the infancy stage," said Ramnath Cidambi, manager of middleware engineering and support services at United Airlines. "We are taking the next steps to SOA."

Elliot King recently spoke with Cidambi about United's challenge in the latest issue of BPM Strategies. Over the years, United had built an extensive J2EE-based middleware layer to integrate its large assortment of IBM and Unisys mainframes.

Cidambi's team set out to migrate its flight information system from legacy-based SOA to a publish/subscribe model. "Airline operations are completely event-oriented -- as events occur, their impact ripples throughout the entire system and beyond," Cidambi explained.

The main thrust was development of an SOA-enabled system called EasyFIDS, which is a flight information system to track the current status of flights, Elliot reports. EasyFIDs is able to relay information about flight status, in real time, to multiple endpoints, including airport monitors, crews at different airports, the Federal Aviation Administration, and most importantly, passengers themselves, who may or may not already have checked in. This calls for a system capable of communicating with a variety of endpoints.

Currently, United has about 22 SOA-enabled mission-critical applications in production as part of EasyFIDS, Elliot reports. If bad weather blows in after an airplane has already pulled onto the tarmac, for instance, the publish/subscribe application is used to communicate with the new priority for take-off.

While United has been able to put these new applications to work, the challenge continues to be a need for more standardization been applications from all industries, Cidambi said. For example, the airlines need to interact better with partners in the hospitality and travel industries. In addition, he noted, J2EE application servers may have been standardized, but this is not the case with Enterprise Service Buses, where every vendor has its own flavor.

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

"The learning curve is steep. It is not so much any given aspect which is difficult to learn, it is rather the sheer number of technologies and tools one must be familiar with to execute the complete project."

Early last year, Safeco, an insurance company, launched an SOA initiative intended to support new product development and business process improvements. Members of the company's SOA team provided this account of the company's SOA effort.

The goal of the SOA project was to create an enterprise component which can process all matching requests by comparing Motor Vehicle Registry (MVR) records to policy records. There were often manual processes associated with the MVR matching process, the SOA team wrote. "A customer service representative will manually review the record and policy information to decide whether the policy needs to be re-priced. If this is the case, an updated policy is sent to the agent and the customer. About 35% of MVR records match the policy information."

The company chose Microsoft Windows Communication Foundation for the Enterprise Services tier, IBM WebSphere
Process Server for the process tier and ASP.Net for the presentation tier. "WCF is one of the best service containers to-date. The team also deployed Service Component Architecture (which is language agnostic) that provided an 'assembly mechanism' that enabled integration developers to rapidly assemble Web services and heterogeneous components written in Java, C++, amd BPEL.

However, the greatest challenge to this project wasn't technology, but the fact that "new products, solutions and improvements are often specified without consideration for department or system boundaries," the team said.

The SOA team also found that while the success of SOA depends on business involvement, it doesn't pay to try to get businesspeople too wrapped up in the technical aspects of the project. "From our perspective, it seems difficult to have the expectation to involve business users in modeling activities. An experienced business analyst can easily translate business requirements into business process models. This model is generally not deployable in the process engine as-is. An integration developer needs to be involved to make the process executable."

The SOA team advises that initially, SOA projects "should be treated as traditional implementation projects. Over time, there is value in providing an as-deployed model to a business team."

The Safeco SOA team saw at least four major benefits coming out of their efforts:

Legacy Reuse: "We could reuse legacy code that was otherwise impossible to reuse without exposing it as a service using Web services technologies," they observed.

Interoperability: "Web services technologies enable interoperability, which in turn is a key success factor when building composite applications."

Rapid Turnaround: "We were able to deliver a complex solution integrating over five systems in less than eight weeks with a team of four developers, two QAs and two architects."

Very Few Lines of Code Required: "The process implementation was done with less than 20 lines of code which were written for a special mapping capability. The assembly between the process component, the services and the human CSR was achieved with SCA. These capabilities demonstrate that a model driven approach is effective to implement real-world solutions and this approach eliminates over 90 percent of the code at the process level. If we were to code this process implementation using a traditional application model, (e.g. J2EE, or .Net), without an orchestration engine, this implementation would have taken several thousand lines of code."

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

Experts with leading vendors of course, are optimistic about adoption of SOA, but in a recent series of podcasts, they candidly pointed out that many of their customers are only on the first leg of their journey to SOA.

ebizQ colleague Gian Trotta informed me he has posted full transcripts from my recent podcasts with SOA industry leaders for the InfoWorld SOA Executive Forum.

There are many notable and quotable observations about the current state of SOA in these podcasts and transcripts:

"People are still on a learning curve [with SOA]. We do see, in some situations, where customers and organizations have... between 10 and 20 services to see how that would play out within the other architecture that they use within their organization and enterprise. ...It seems like organizations still find it a little bit challenging to take the next step and really roll these services into production. ...it's unknown how these services would function in a production environment as far as security is concerned."

- From a podcast with Software AG's Mighael Botha

"Some companies have been doing [SOA] for a long time using older technology such as CORBA, MQ series and WebSphere MQ. One of our pioneering SOA customers, Credit Suisse has been working on an SOA for about ten years based on CORBA, and last I heard they had about 2,000 services in production and processing about a billion transactions a year. Yet, some other divisions of that same institution are just sort of getting started with their SOA projects...."

- From a podcast with IONA's Eric Newcomer

"We tend to think of SOA as being a factor of structure, scale and speed. ...I would characterized folks who have really gotten into that intersection as only being about 300 major companies worldwide, including some governments. If you look at the rest of the market, they may actually be taking a couple of different things, maybe two out of three but not all three. So, there are companies that are sort of achieving some level of agility but they're not doing so at a very large scale...."

- From a podcast with webMethod's Miko Matsumura

"A couple of years ago, the major emphasis was about education about SOA, and about the value that SOA brings to an Enterprise. Today, we believe that a majority of the companies have crossed that hump and have actually bought into the religion of SOA and they are starting to invest into specific point projects for SOA. So they're starting to take simple implementations where they would have used point-to-point integration techniques to connect two disparate applications. Today they are actually moving towards the SOA-based architecture of connecting these applications together. Or optimizing or automating business processes."

- From a podcast with Oracle's Ashish Mohindaroo

"Businesses are finding that what they intended the services to be used for only scratches the surface of what they actually could be used for. Deep organizational changes are required as a part of the implementation, but these changes present companies with new business opportunities -– in part, as a result of the deconstruction of their former business capabilities."

- From a podcast with BEA's Charles Stack.

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)