SOA in Action

One of the challenges in bringing business process management together with service-oriented architecture is that the two essentially exist in two separate worlds, managed and promoted by different teams with different agendas. However, the two need each other more than ever in order to succeed.

In a new post, Nick Malik describes the fast-evolving relationship brewing between BPM and SOA.

"These two activities are twin brothers," he says. "Their parents are the same, and they have many similarities, but the serve different needs in the enterprise." Nick takes the analogy (a good one) a step further to describe the their common "father" as data, and their common "mother' as the business event.

Who's your Daddy? Nick describes the fatherly role as the common data model, one of the few pieces really needed to "make enterprise architecture work." In addition, he adds, "the common data model should contain ONLY enough information to define the things that must be present to define a business document, to identify the unique key for that document, and to define the relationships with other business documents."

The motherly role, then, is the business event, or "an event that defines the boundaries between business processes." He adds that "you have to know where a process starts and where it ends. Otherwise, you could find yourself comparing a single apple to a grove of apple trees."

Of course, data and business events processing need to be mature themselves before they're ready to raise children. This is not a top-down design, but it is a top-down constraint. "If you build your SOA before you understand the data, you are creating multiple versions of the truth," Nick points out. "If you attempt BPM before you identify your events, you will burn huge efforts to compare apples to apple trees, producing no value."

Posted by joemckendrick in Business Process Management  |  Permalink  | Comments (2)  | TrackBacks (0)

Many analysts predict that event-driven architecture and complex event processing represent the next leap for SOA. This leap is explored in greater detail by David Linthicum, Roy Schulte, and Dr. K. Mani Chandy, who recently posted insights here at ebizQ on event processing, and the role it is increasingly playing in SOA.

Schulte and Chandy write that complex event processing, or CEP, can deliver three capabilities to the enterprise -- the ability to run faster and smarter because it has "situational awareness," can "sense-and-respond" to opportunities and threats, and can "track-and-trace" items as they go through their lifecycles.

They note that "traditional application systems and most forms of BI do not provide these capabilities. The key to enabling near-real-time operational BI is event processing. Event processing has become practical for a much wider range of applications. Companies have streams of business events that did not previously exist. Companies that leverage CEP have a distinct competitive advantage over those that are unaware of the potential information value of the events that are in their midst."

Essentially, CEP is about information, and the ability to move information to where it's needed, at just the right time. As Dave Linthicum points out, CEP has always been needed and desired, but "never really had a name until now." CEP addresses needs that were first seen during the formative years of EAI (Enterprise Application Integration). "We've been considering the notion of the movement of information around complex business events, in that all interested entities should have access to perfect information in real-time and can respond instantaneously as to processing these events in the best interest of the organization. CEP puts some formality around it."

Schulte and Chandy agree that the concept of CEP has been around for some time now, but it's only recently that the concept has been extended to mainstream business applications. This is changing for several reasons, including the fact that "some of the design techniques and algorithms for applying rules to high-volume event streams were only invented in the last 10 years, and were brought to market in commercial software products even more recently." Plus, lower hardware and network costs have made CEP more affordable. Add to that the growth of sensors, including RFID, bar code and GPS devices.

Schulte and Chandry predict that the the "killer application" for event processing is in the form of business activity monitoring, or BAM, which is appearing in the form of business dashboards tied to key performance indicators (KPIs) in near real time.

What's the connection between SOA and CEP? Dave Linthicum connects the dots. CEP is "based upon the core foundations of integration, and is even more relevant in the world SOA. Indeed, as we build a SOA, we need to pay close attention to how all of the systems in the problem domain interact, and thus the paradigm of managing the interaction of these events layers into the core foundation of the architecture. Therefore, you need to consider CEP as you build and deploy your SOA."

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

It's said that SaaS applications are nice, but the most vexing issue is achieving meaningful integration with existing in-house financial and ERP systems. Integration is the next great frontier of SaaS -- and the point where SOA can help.

Robert Schneider, writing in SOA Magazine, talks about how Software as a Service (SaaS) and SOA composite applications interact, and how to best leverage this interaction to provide business value.

He talks about how a deep-discount rent-a-car shop, "Pay-N-Pray Motors," deployed a deep-discount SaaS solution from "CheapCRM4U.com."

Things are running smoothly with the new hosted CRM arrangement, Schneider explains. Then, a wrench is thrown into the works. Pay-N-Pray's finance department decides to institute a retroactive three-percent "courtesy fee" for all rentals that occurred in the past two years.

Ouch. Remind me never to rent from this outfit.

While the additional charge goes unnoticed by most customers, a blogger gets dinged with the fee, and all heck breaks loose. Everyone is demanding the additional charge be taken off, pronto. As Schneider describes it, "The besieged customer service representatives (who spend all of their time in the CheapCRM4U.com application) have no easy way of viewing these transactions, and it's even harder to reverse them."

SOA to the rescue for this public relations and logistical nightmare.. Using SOA methodologies, it is able to quickly throw together a cross-system composite application that can be launched from within CheapCRM4U.com.

The composite application presents a unified view of a given customer's contact detail from the CRM package, billing history from the financial software, and service representatives' notes. The customer service reps can continue to work inside the CheapCRM4U.com user interface, but also use the composite application whenever they need to interact with data from the financial application.

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

Microsoft keeps talking more and more about service-oriented architecture, and even now offers an e-book on the topic, SOA in the Real World.

Thanks to Redmond Magazine's Chris Kanaracus for the pointer

As Microsoft puts it: "SOA is an architectural approach to creating systems built from autonomous services. With SOA, integration becomes forethought rather than afterthought.This book introduces a set of architectural capabilities, and explores them in subsequent chapters."

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

Jon Brodkin, writing in JavaWorld, recently posted some answers to the six "burning" questions that stick in people's minds when they think about SOA.

1. Is anyone saving (or making money) using SOA? Good one. The challenge is that SOA costs up front, but delivers savings/ROI later. How much? Jon quotes Ashok Kumar of Avis Budget Group, who says the company has shaved the cost of bringing aboard a new partner from $50,000 to about $4,000, because the time it takes to put a new partner in the system was cut from a month to a day.

2. Why is it so hard to find employees with SOA expertise? I like this quote from Forrester's Larry Fulton: One client told him "the best method for identifying architects is to put a group of 10 developers to work, monitor them for 10 years and then decide who the architect is."

Training, training, training is the best approach to bringing not only IT people, but also businesspeople up to speed on that SOA can offer. Avis' Kumar cautions that SOA requires a different mind-set than traditional approaches to
building an IT infrastructure -- many people can program in Java and understand how to make a single Web service, but putting it all together in a services-oriented architecture is difficult.

3. Has Microsoft gotten a clue about SOA? Another great question. Forrester's Fulton and Judith Hurwitz say yes, and I agree. Heck, they even call BizTalk an enterprise service bus.

4. How does SOA affect network performance and management? The answer is Big Time. You can't predict how many people will use a service at a particular time. That's why SOA managers need to pay close attention to network and performance metrics. And, as the article notes, "since each application in a SOA is composed of many individual software components, a failure anywhere in the network can bring down an application. Your own performance in monitoring the network and immediately responding to problems is thus even more important after an SOA is deployed."

5. Do security requirements change when an IT department uses SOA? Big Time on this as well. Identity management is a big concern. Plus, there's lots more components to manage.

6. What are SOA's dark sides? Another great question. As the article puts it, "Security is clearly posing a challenge to at least some IT executives deploying SOA, but it?s not the only dark side you'll find when building a service-oriented architecture. One of the 'dark underbellies' of SOA is the challenge of providing a unified view of data and access to data across multiple business services."

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

ebizQ columnist David Kelly observes how we've -- both on a personal level and organizationally -- have become overwhelmed and inundated with data from all sources, on all kinds of devices and platforms.

Can SOA help us through this morass? Unfortunately, as David points out, "problems with data
consistency become significantly worse as organizations move toward implanting SOA."

The time is ripe, he believes, for organizations "to re-evaluate their information management strategies. Over time, it will become simply impossible for most organizations to continue managing data as inconsistently and stovepiped as they have."

As David astutely points out, there's a huge disconnect between SOA and enterprise information management. These are often, in fact, under the domains of separate teams, with separate agendas. And both require fairly robust investment of time and resources -- including enterprise participation and governance -- to get it right. Should such efforts be integrated?

Enterprise information management needs SOA. SOA needs enterprise information management. Simple as that. Enterprise information management is all about getting at the right information at the right time. By being able to sift and sort through data,decision makers can identify and predict new product demand, inventory flows, and spot potential fraud even before it happens. Likewise, SOA is all about getting at the right information at the right time.

Too many enterprises have too many silos, and need ways to cost-effectively publish data from any application, running on any system, regardless of original data format.

Posted by joemckendrick in SOA  |  Permalink  | Comments (1)  | TrackBacks (0)

While SOA, rightly or wrongly, is primarily seen as a strategy for optimizing IT operations, it's always good to see where it directly impacts the business.

In this month's Baseline Magazine, David Carr provides an interesting account of how service-oriented architecture did just that -- having a direct impact on a business's growth.

Carr talks about how Aeroplan, originally the frequent-flier plan for Air Canada, was able to expand its reach to both consumers and business partners in supporting customer loyalty rewards programs. Aeroplan's mainframe-based systems were originally built around supporting frequent-flier miles, until Air Canada spun the company off two years ago. Aeroplan sells mileage credits to 60 partners, including credit card issuers, and last year, issued 69.7 billion mileage credits and redeemed 49.3 billion, which were applied to airline tickets, hotel rooms, and merchandise.

Aeroplan's challenge was to expand its Its range of offerings beyond what was a limited number of flights on Air Canada routes.

Enter Web services, which helped Aeroplan integrate with partners to expand its program beyond airline seats. Aeroplan sought to increase the percentage of miles redeemed for non-flight rewards to 25% a year, up from 11% in 2006 and eight percent in 2005.

To meet this goal, Aeroplan wanted to still leverage the mainframe as a back-end transaction engine, but add service layers for interacting with business partners and consumers. Aeroplan employed XML Web servuces in conjunction with open-source technologies such as Linux, the Apache Web server and MySQL database.

Carr observes that "before creating the ability to tap into the systems of its partners via Web services, Aeroplan provided a way for partners to tap into its own services that way." Aeroplan also was making some use of XML exchange data between its mainframe and other applications, such as the Website and an interactive voice response system.

"Rather than rewrite those XML messages and disrupt the applications that already depended on
them, Aeroplan used the Reactivity appliance to automatically translate them into a standardized format for Web services transactions with partners. The appliance also translated between the IBM MQSeries protocol, used to transmit data between internal applications, and the Web-standard HTTPS protocol used for functions
such as credit card transactions."

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

Can SOA help forstall or even render big ERP upgrades unnecessary? I have heard a number of accounts of how enterprises were able to move various pieces of functionality off their aging ERP systems to a service-oriented middleware, and in the process, modernize their operations with replaceable standardized components.

A pipe dream? Recent reports in ComputerWeekly (here and here in ComputerWeekly describes how one of the largest corporations in the world was able to SOA-enable its SAP R/2 system, making an upgrade to R/3 unnecessary.

Don't feel too bad for SAP -- they're willing partners in this end-run around a new systems upgrade. Dow's "Next Enterprise Architecture (NEA)" is based on SAP's Netweaver and MySAP. The report notes that the SOA implementation will allow Dow to develop new systems while running existing SAP R/2 and in-house systems, which it plans to replace over the next 10 years. Don't bother with upgrading to our next version -- we'll help you do something cheaper and more effective -- how's that for vendor cooperation?

Also interesting is that the SOA effort is designed to do what SOA is ultimately designed to do -- provide the company greater flexibility to respond to market changes quickly. In the chemicals sector, speed to market can mean billions. Here's a big money-maker -- most of the SOA stories I've come across so far talk more about developer productivity and cost savings.

Dow tested the SOA tools by building a system for its product development department to streamline regulatory documentation. Melanie Kalman, Dow's information systems program director, said, "We have not yet quantified the results, but the team saw such a drastic reduction in time that all future products will take this route."

The first five applications Dow is developing as part of NEA will provide a highly integrated view of Dow's
operations worldwide. In addition, managers and staff will gain easier access to the information they need, and systems development staff will be able to respond to their changing needs more quickly.

And here's s statement for the longevity of SOA -- Dow said it expects to use NEA for the next 15 to 20 years.

One more statement about SOA as a journey: Kalman said that "the biggest challenge has been getting key IT suppliers to understand that Dow is on a transformational journey. We are not simply executing a technical upgrade of the capability we have today we expect the IT capabilities to help us deliver the desired business results."

Posted by joemckendrick in SOA  |  Permalink  | Comments (1)  | TrackBacks (0)

Sandy Kemsley, who has graced many of these pages with her insights, says SOA and Business Process Management (BPM) are a difficult mix, and, truth be told, one can be implemented without the other. But doing one without the other makes things twice as hard as they ought to be.

In a recent interview, Sandy points out that "You can do BPM without SOA, but it's much harder since you have to write all the interfaces to the underlying systems as well as other functionality that the services might provide as part of the BPM project, instead of just consuming existing services."

Likewise, she adds, SOA doesn't really need BPM. "You can also do SOA without BPM, but without a 'killer application' to use those services, it's difficult to justify the cost of creating the services layer in the first place," she explains.

"BPM and SOA are fundamentally different from a technology standpoint. SOA is the design philosophy by which you service-enable your enterprise, either by wrapping Web services around existing/legacy applications, or by building new functionality within the services. BPM is a consumer of those services: A service may be called at any point in a process by a BPM system. BPM orchestrates services and people into a complete business process."

Another trend to watch closely is Web 2.0, Sandy points out. While companies are cautious about approaching the social networking aspects of Web 2.0 -- blogs and wikis -- the integration aspects, as embodied in mashups, may prove to be a formidable force in the near future. "Many integration projects -- and I've been involved in a lot over the past 15 years -- involved too much code and too much time, even with today's tools. Web services, by which I mean those built on the WS-* standards, are considered too complex and time-consuming to develop by many people, who are using techniques like REST and JSON to create mashups."

Posted by joemckendrick in SOA  |  Permalink  | Comments (1)  | TrackBacks (0)

What's on the horizon for business intelligence? What's business process management got to do with it? And, for that matter, what's service-oriented architecture got to do with it?

Those are the questions recently taken up at a panel discussion led by ebizQ's Beth Gold-Bernstein to wrap up the BI in Action virtual conference. I had the opportunity to join Guy Weismantel of Business Objects, Rob Risany of Savvion, and Michael Corcoran of Information Builders for a rousing discussion on how the pieces of BI, BPM and SOA all fit together to greatly amplify the an organization's intelligence. (Transcript and link to the podcast is posted here.)

First, the BI-BPM role. As Rob put it, "BPM -- business process management -- basically creates a role for business people that they haven't really had before in enterprise architecture. Business process management gives business people the ability to think about the things that affect them most and turn them into running solutions within the business."

BI, in the context of BPM, "is about providing enough information so that the process solutions that are created by businesspeople used in conjunction with IT have enough information so that the right decisions can be made," Rob added.

Then, finally, there's BI-SOA. To quote Rob again, "SOA is the infrastructure which IT uses to enable the business initiatives.... BI creates a business face for the data, BPM creates a business face for the process while SOA is an underlying approach for building applications across the business."

Examples of the BI-BPM-SOA triangle at work:

Disneyland in Paris employs a number of data warehouses and applications to monitor events around the park in real time. "They can determine if they need to move Disney characters around different parts of the park based on the crowd levels and crowd flows," Guy said. "They can get more food and move inventory around based on the lunch crowd and the dinner crowd in different parts of the theme park."

Another theme park, Universal Studios in Orlando, employs a BPM engine to streamline its planning and budgeting processes. The system enables the park's managers to "reduce the cycle time that it takes to go ahead and do budgeting," Guy explained. "Universal is probably one of the most unique companies in that they actually do planning and budgeting on a daily basis," versus once a year or so.

The reason Universal plans daily is "because conditions at their theme park in Orlando are always changing and the plan for, you know, June 19th of 2007 is going to be different than the plan for June 19th of 2006. The weather could have been different, the day of the week could have been different, different rides could have been functional or not operational so they have a limited ability to go ahead and take past information and make some sense of it now they certainly use that but they're always look at the conditions during that day in the park and what they can forecast."

Universal would not be able to accomplish this "unless they had the planning and budgeting and more performance management environment tied in real closely with the key processes that help them drive the business," Guy added.

Motorola has built and leverages business intelligence on top of ongoing process data which is automatically gathered in audit trails, which is used to track SOA compliance. The reported data is also analyzed "to find anomalies in the specific products that are causing a service level problem," Rob explained, such as order delivery to drop ships to customers. "The role of BI in action in that regard is one of how can I affect the strategic decision making, of how I can improve the process overall."

What's holding back organizations from really moving forward with the new BI-BPM-SOA paradigm? Michael reported that many companies will say that "'we're not doing anything with BI right now because we're very focused on our architecture.' I think you have to broaden that thought process to say 'BI is part of the architecture, it's a service in a service-oriented world.'"

"There is no such thing as a perfect architecture," Michael continued. "I still see organizations who haven't done anything for twelve years just because they're waiting for that wonderful CORBA architecture or whatever the new standard was at the time. I think the perfect architecture is the one that works, the one that you can use to integrate and leverage now.... it's all about business needs, not technology needs."

Posted by joemckendrick in SOA  |  Permalink  | Comments (1)  | TrackBacks (0)

Last week, I had the opportunity to do a presentation at SOA World East in New York, in which I explored some of the issues facing organizations making the crossing from JBOWS (Just a Bunch of Web Services, of course) to fully functioning SOA.

I discussed some of the Evans Data research I have worked on that showed most organizations, if anything, are still stepping out of the JBOWS stage and just starting to get their arms around the various pieces that make up SOA: governance and registries, ESBs business process management, and service sharing/reuse, to name a few.

Though I didn’t have a chance to check out all three days, SOA World had a very compelling program that explored a lot of these issues — as well as newer approaches such as Web 2.0 and Ajax.

To provide a brief recap of my session, here some of the half-baked ideas (HBIs) — or fearless predictions — I made about the future of SOA:

Most organizations are not at the full-functioning SOA stage yet. They are still managing a JBOWS environment, and will be at this stage for a while. (I’ve said plenty about this in this blog.)

The SOA Paradox: The organizations that are adopting SOA are the ones that don’t need SOA. The organizations that could really use SOA the most are not likely to be adopting it anytime soon.

The “loosely coupled” ethic of SOA will influence companies to become “loosely coupled businesses” themselves.

SOA, SaaS, and open source will give rise to application vendors that follow the “Dell” model – assemble components to order; don’t make them yourself. SOA, SaaS, and open source will also provide opportunities for microbusinesses to sell pieces of applications, on demand.

Vendors will lose interest in the term “SOA” – prepare to hear more about next-gen solutions, such as Event Driven Architecture or Enterprise 2.0.

More convergence between SOA and enterprise data management — companies are anxious to turn data into competitive advantage as fast as they can; SOA will enable this. (I also recently joined an ebizQ panel discussion in which talked in some detail about this convergence.)

More SOAs will be built on open source software. Already, there are an impressive array of open-source app servers, ESBs, and business process management solutions out there. This will commoditize SOA solutions, and disrupt what is currently a lucrative market for high-end vendors.

SOA success will be uneven. As mentioned above, some organizations are more ready to take advantage of SOA than others.

SOA will play a greater role for budget-conscious companies looking to control IT costs.

Dave Linthicum, also a participant at SOA World, made this interesting observation while attending one of the case studies given during the general sessions: “The focus was more on the technology and vendors they selected, ESB, governance, etc., and not on how they derived the solution and the method and process for doing so. At the end of the day I felt that what they really had was a JBOWS (just a bunch of Web services) with some fancy software around it. I hope they solved their business issues, but from the presentation it was not clear.”

Dave also said vendors knew their technology well, but really couldn’t explain how their products fit into SOA efforts. “SOA technology vendors need to teach as well as sell,” Dave says.

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

That's a headline that certainly caught my eye when I spotted it in Application Development Trends. This is especially argumentative, in light of Steve Jones' recent pronouncement that SOA and BPM aren't necessarily a good mix, since BPM breaks down services into "steps."

Kurt Mackie over at ADT sees things differently, however. Kurt walks us through the debate over SOA value as an enable of reuse, noting that some observers say few SOA services will actually see reuse in the end.

BPM is the reason for businesses to invest in an SOA, Kurt says, and he backs it up with data, courtesy of IBM's Peter Rhys Jenkins. IBM's own survey of 765 CEOs worldwide found that top-performing companies were twice as likely to stress the need for business model innovation compared with the opinion of their counterparts in underperforming companies. Then, there's an InformationWeek survey that found that 65 percent of IT professionals plan to streamline or optimize their business processes in 2007.

So the world is hungry, starving, for effective BPM. BPM is a way to encapsulate "tribal knowledge" in a company. It's difficult to get information about how workers do their jobs because workers fear the possibility that their jobs may be outsourced. SOA provides a medium for services that can surface that tribal knowledge where and when it is needed.

I would beg to differ about the value of reuse and sharing as a benefit of SOA -- but it's clear that SOA and BPM will evolve closer together, and appear to need each other.

Posted by joemckendrick in SOA  |  Permalink  | Comments (3)  | TrackBacks (0)