SOA in Action

Financial services companies have had a tough ride over the past year, so anything that can be done to better take advantage of their information technology assets to streamline operations and increase the situational awareness and agility of the business is a good thing. That's where SOA and business process management (BPM) come in.

JPMorgan Chase, of course, is right at the epicenter of the financial storm that has been ravaging the industry, and the company's dominance and stability is due in no small part to its ability to smartly leverage SOA and BPM.

At ebizQ's recent Financial Services Live Panel, Ron Ambuter, CTO of the BPM Workstream Group at JPMorgan Chase, described how his company has been able to adapt and adopt technology and methodologies to help reduce costs, improve customer experiences, and maintain a competitive edge. Ronan Bradley, analyst with Lustratus Research and former CEO of PolarLake, led the discussion, and was joined by Keith Swenson, chief architect at Fujitsu Computer Systems, and Hub Vandervoort, CTO of Progress Software. (Archived recordings and transcript available here.)

Ambuter described how JPMorgan Chase was interested in the concept of reusability, which would help the organization "get better leverage out of the efforts that we were making to build and buy applications by reusing components instead of rebuilding and rebuying them over and over again."

With the troubles the industry has been having around subprime mortgages, writedowns, and tighter credit, it's likely

regulation and oversight will only increase, particularly in regards to risk management. Bradley questioned whether the prevalence of regulation holds back or encourages SOA adoption. Ambuter said regulatory mandates have accelerated his company's drive to SOA, noting that "the concept of SOA allows us to react probably quicker, and more expeditiously, more cost effectively."

The panel also discussed the challenges around extending managing services and infrastructure across multiple groups within organizations. Financial services organizations, of course, typically have multiple lines of business, from securities to brokerages to mortgages. Ambuster explained that organizational issues were his greatest challenge as well in implementing a cohesive SOA and BPM strategy at JPMorgan Chase. "We have rules and responsibilities that are compartmentalized by groups of folks, and we realize that as you go into this stuff, education and cross-pollination of information is critically important. It's not just the technology folks that need to understand this stuff, it's the risk folks, and it's the governance folks and it's the finance folks, and it's the business side folks."

Vandervoort agreed that "it's easy to get bogged down in trying to get alignment on a lot of different points across all the groups." He recommended three approaches to the problem, including "getting your transports aligned between business entities so that you can use eventing-oriented mechanisms to communicate across domains"; establishing SLA and security policies that ensure visibility; and establishing a common enterprise data model.

"You have to get your semantics aligned among the members," he said. "And that doesn't have to be a common vocabulary in its entirety, but certainly what we regard as the data in flight, those things that fly between domains and different working groups have to be highly normative."

Aligning SOA and BPM is also important. "While SOA is basically a technology trend, BPM is essentially a management trend," said Swenson. "A lot of people look at BPM and they get confused between the management trend and the technology trend. A lot of technology product companies have looked at BPM and looked only at the technology and come up with a kind of a programming language that they claim to solve the problem, which is great for the IT side of the house, but it leaves the business side out of this. As far as the adoption of BPM goes, one of the biggest barriers to adoption of the real business process is essentially the fear that the manager will lose control of the process."

Complex event processing is also a key initiative financial services companies need to undertake. As Vandervoort observes, "financial services is moving kind of in a logarithmic increase in velocity... things are ten times faster than they were a decade ago. When you go from three days order to settlement or trade to settlement to now under two hours trade to settlement, if I'm doing nightly reporting, I'm way out of alignment. Whereas ten years ago, that was three times faster than the speed of the pipeline. Today, you the ability to run awry in risk, terms, and exposure terms has been seen multiple times in the industry in recent years."

For companies seeking to increase agility and be more responsive to these highly competitive markets -- not just in financial services -- the combination of SOA and BPM can break down the constrictive silos that cut off essential data and processing resources.

____________________________________________________________________

Posted by joemckendrick in Business Process Management • Data Management • Event Processing • Management • SOA • SOA Events  |  Permalink  | Comments (0)  | TrackBacks (0)

Complex events "can be anything from a sequence of temperature readings to something like the First World War, which was a very complex event indeed." -Dr. David Luckham

David Luckham -- considered the "father of complex event processing" -- recently joined Dr. K. Mani Chandy, Rodney Morrison, and Beth Gold-Bernstein for an informative panel discussion on the relationship between EDA (Event Driven Architecture) and SOA. The panel, part of ebizQ's recent Event Processing Virtual Conference, which brought together the leading thinkers and proponents of EDA and Complex Event Processing (CEP).

The panelists were divided, however, over the extent of the role of human involvement in complex events.

Chandy, for example, said that human engagement is essential at some point in the complex event processing cycle. "I’ve seen quite a few applications, but almost none in which there is no human involved" For example, Chandy said, "in military applications, when there is a gun or device is fired to kill somebody. It’s always done with a human being responsible for their final action."

As Chandy pointed out:

"I think its absolutely critical for human beings to be part of this sense and respond system. It's important how the application supports the human being if you're looking at trading or fraud detection. In all these cases, its really important to have a human being involved. Fraud is one case where you might have an application that informs a credit card user that something inappropriate may be going on, without having a human being first check that."

Luckham, however, pointed out that there are many events are processed independent of human intervention. "There are many examples of event driven architectures where there are absolutely no humans whatever," he said. "The CPU on your computer is an event driven architecture, believe me. And its entirely event driven, clocked, without a human in the loop.

EDA is part of SOA on two different levels, Chandy said:

"One way that I’ve seen EDA used in conjunction with SOA is for service management. Many SOA vendors are exposing metrics that can give you information like end to end process time and activity times. Those metrics can be provided to a CEP system to help control and manage those services. I can, for example, do dynamic provisioning for a service that's getting maxed out."

Chandy also connected the dots between EDA and SOA. "If SOA means loosely coupled subsystems with very clean interfaces, so that new systems could be coupled into the substrate. Then EDA events fit within that framework, because EDA is also based on a loose framework, and is extensible." In a request-reply SOA scenario, "then EDA can still be coupled. There will be layering between the push and the pull parts."

_____________________________________________________________

Posted by joemckendrick in Business Process Management • Event Processing • SOA • SOA Events  |  Permalink  | Comments (0)  | TrackBacks (0)

Behold the zebra in the wild savanna -- when a it senses the presence of a lion, it knows to run away, as fast as it can. When it senses food supplies, it knows to act.

Do today's companies have this much situational awareness, and an ability to act quickly to survive and thrive? Not yet, but thanks to new approaches such as complex event processing, they're getting there, according to Gartner VP Roy Schulte. "We can teach computers to do what a zebra does," Roy said. "To collect and process event data to respond quickly and effectively."

In his keynote kicking off ebizQ's recent Event Processing virtual conference, Schulte broke down the essential pieces of complex event processing, and describing how businesses can leverage CEP, or being able to act, in real time, on multiple streams of event data flowing in from different parts of the enterprise. "The value of complex event processing, overall, can be summarized as improving situation awareness. Simply put, that is just knowing what is going on, so you can figure out what to do." The benefits of complex event processing, Schulte said, include better decision quality, faster response times, reduced information glut, and reduced costs.

Schulte defined a business event as a "meaningful change in a state that is something that is relevant to the business. Examples include depositing or withdrawing money from a bank, submitting a purchase order, or hiring an employee." There is also a second term, "event object," that describes how the event is packaged for processing, typically as an XML document these days. "We have to record events using event objects so computers can receive them and do computations on those events," Schulte said.

However, while all companies have always been event driven -- with millions, if not billions, of events in a single day, most events are still handled manually, by people, not computers. "At any one second, a large company has on its network anywhere from 10,000 to 10 billion business events," Schulte explained. "At the low end, that's almost a billion events per day -- at the high end, that’s almost a trillion events per day."

The challenge is that most of the stovepiped and legacy applications that power enterprises are not yet event driven, Schulte observes.

But there's great practicality in automating the ability to capture and make decisions on multiple event streams coming into the core business systems, Schulte says. "For example, you can have a complex event that says, ‘this mornings sales were 30% above our daily average.' That of course is much easier to digest and act on than sending a person 500 detailed sales records, and making the person compute what happened that day manually."

The growing array of sensors, such as RFID tags, combined with front-end systems such as business activity monitoring (BAM) dashboards make complex event processing a reality with today's technology, Schulte points out.

"In many cases, the complex event processing system Is just a front end being used for decision support. The output of the CEP engine is sent to a person through a BAM dashboard, or through an alert such as email or SMA or an Atom or RSS feed. in this case, we have a two-stage computation. In the first stage we’re using a computer to narrow down the data. And in the second stage, we still have a person involved to do the final analysis.

"An application system, or some other device or some other system, detects the event, and generates a message or a notification that is sent to a person. That notification is the event object or event report sent in the form of a message through message-oriented middleware, RSS, a Web service, or an email, or some other communication mechanism. The response to an event may be a manual activity, done by a person or it may be a SOA service or business process or some other application."

However, things could get interesting as CEP systems develop, Schulte added. Namely, the need for human processing could be taken out of the equation all together. "We can bypass that person entirely; we can build enough smarts into the complex event processing engine to determine the specific response that is needed."

Schulte provided a working example of complex event processing in action within the airline industry:

"In large airlines, there is an event oriented middleware that... acts as an enterprise nervous system. Information from hundreds of sources, including sensors on board the aircraft, information coming in from the FAA, and information coming in from standard systems is sent to the enterprise nervous system, and is temporarily stored in event databases. It helps to create the data, the outgoing alerts and notifications that is sent to hundreds of applications on the consuming side to respond to threat and opportunity situations as they emerge. By having information quickly, each of these systems in their respective departments can respond faster. ...Information helps the fueling and maintenance management applications to change their schedules and so forth. By using an event based system, the turnaround time of each plane can be shortened… Fewer airplanes are needed to handle the same schedule."

______________________________________________________________________

Posted by joemckendrick in Business Process Management • Data Management • Event Processing • Management • SOA • SOA Events • SOA Research and Analyst Reports  |  Permalink  | Comments (0)  | TrackBacks (0)

Business-IT alignment... Has a day gone by over the past ten years that we have not heard that phrase?

While it seems to be a constant but elusive dream -- like peace on Earth -- some companies indicate that SOA may be moving them closer to reality.

Fellow ebizQ colleague James Taylor, out at the IBM Impact confab this week, provides an account of an end-user customer panel on the ever-vexing challenge of business-IT alignment. Is service-oriented architecture helping to bring about some of this alignment?

A panel of corporate practitioners talked about their SOA efforts, and the impact SOA was having on alignment -- a positive impact, by the way. James reports that Randy Wallace from Michelin said one of his company's biggest challenges is "B2B with lots of billers to interface directly in their order management systems so, for instance, allow dealers to integrate orders with Michelin. They are currently evolving their order-to-cash system using Process Server and SOA."

Who much alignment has Michelin seen? According to Wallace, the company has come a long way, "from having a very small percentage of IT spend aligned with key business goals (6%) to one that is much more so (81%)."

That's pretty impressive. Wallace cited some examples: "For instance, in the past business units in different regions picked i2 and Manugistics at the same time and both were implemented resulting in separate systems. A stronger governance process and overall architecture are now established, driven by business ambitions and regularly updated. Far fewer and more focused projects as a result. Senior executive user satisfaction has risen steadily."

Austin Waldron from Health Care Services Corporation (HCSC) said his company's "focus is on using SOA in legacy modernization where many disparate systems are being replaced by a unified set of shared services. The governance issues seem to have been key for HCSC."

Waldron also talked about moving closer to business-IT alignment. "They had some years of IT spend focused more on basic IT infrastructure (security, robustness etc) but now investments much more driven by the business strategy."

Another panalist talked about more alignment at the front lines of the business. Jeff Auker from The Hartford "talked about challenge of consumer front-ends. Consumers working directly with The Hartford now expect a much more interactive online experience for sales and service - this is being driven by GEICO and Progressive’s campaigns. SOA is key because they have some front-ends that are tightly integrated with very old back-ends and SOA let’s them decouple them."

Thanks again to James Taylor for this report .

______________________________________________________________________

Posted by joemckendrick in Business Process Management • Case Study • Management • SOA • SOA Events • SOA Vendors  |  Permalink  | Comments (0)  | TrackBacks (0)

By now, we're familiar with the promises of loose coupling and flexibility SOA is intended to bring to corporate systems. Now, the move is on to extend SOA beyond the firewall and enable business partners or outsourcers to access and share services.

In a new article, Jack van Hoof explains that when a company outsources parts of its processes (and who doesn't these days), standard SOA may nt be appropriate. Instead, the way to go may be Event-Driven Architecture, or EDA.

The "synchronous command-and-control nature of SOA is a way of tightly coupling application components which doesn’t allow for this kind of flexibility," he explained. "SOA may be loosely coupled in the technical domain, where common Web services technology is used, but it certainly is not in the functional domain where SOA is associated with ‘calling’ foreign (reusable) services and eliminating data redundancy."

The problem with SOA, he says, is "the availability of services and stored data can be vanished after an act of outsourcing, which may lead to costly consequences and high risks. This has all to do with creating dependencies with SOA. The promise of SOA delivering loose coupling, which typically is asynchronous, could at the functional level be stated as a false promise."

EDA helps "achieve loose coupling and autonomy" in these multi-enterprise settings, van Hoof writes. "In contrast to SOA, EDA provides a way of loose coupling. EDA is not a synchronous command-and-control type of pattern, but just the contrary: an asynchronous publish-and-subscribe type of pattern. The publisher is completely unaware of the subscriber and vice versa; components are loosely coupled in the sense that they only share the semantics of the message."

"If you are seeking to support strong cohesion in the business processes, situations where all process steps are under one control, SOA is the way to go," said van Hoof. "If you are seeking to support independency between business process steps, EDA is the way to go."

Since SOA (as we know it today) first came on the scene five years ago, the emphasis has been on employing it as a methodology for integration of disparate enterprise applications. Now, companies are looking to establish clusters of SOAs that extend beyond their enterprises to those of operational partners. This means SOA should no longer be treated as just another IT network.

Posted by joemckendrick in Business Process Management • SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

For years, the disciplines of enterprise integration -- followed by SOA -- have always been perceived as "hard and techie." This has made discussions with the business process management (BPM) side of the house difficult.

Now, thanks to the introduction of new lightweight approaches via Web 2.0, SOA, BPM, and integration are now becoming more flexible, user-driven methodologies.

David Linthicum recently spoke with Dr. Dale Skeen, founder of Vitria, a well-established integration vendor, about this shift. (The podcast is here, and a transcript of Dave's discussion with Dale Skeen is available for viewing here.)

This is part of SOA moving into its next generation -- "the next great leap going together is really leveraging three very important technologies in the enterprise -- SOA, BPM and Web 2.0," Dale said. While these three areas are seen as separate technologies or methodologies, Dales sees their inevitable convergence into what he calls an "enterprise platform."

First of all, Dale said, SOA and BPM have always been a natural pairing. "SOA is an enabler that allows you to access business functions, and services, and data universally. BPM is a higher level that orchestrates these business services and human interactions in ways that allow you to meet a business objective. So hence, I've always considered these to be the perfect complementary technologies to work together."

Now, Web 2.0 is bringing SOA-BPM closer to the end user, Dale said. "SOA brings this universal access to services and data through the SOA enablement tools. It does in a secure, manageable, and governed fashion. Now, Web 2.0 brings rich internet interfaces, rich user experiences based on technology such as AJAX and Flex, which are universally available in your Web browser."

"Application integration is hard, and very techie. Web 2.0 allows this notion of mashups where you let users sort of integrate in a flexible, lightweight, easy-to-do fashion."

These are all new trends that are shifting IT empowerment to the business end user, Dale added. "Simple things like mashups, we're talking visual layering of information, we're talking about collaboration technologies such as instant messaging. All of these have a role in enterprise software. And actually, the introduction of that can be very exciting for both the IT and the business side."

IT professionals need not fret over this shift, however, Dale said. "It means that IT will be able to do faster technology upgrades because of that, they have more control over the server aspect of it, and the client side they don't have to worry about. They're going to be able to lower the support costs and it also brings fundamentally new deployment models such as Software-as-a-service, which we think, is going to be a fundamental part of business IT going forward."

_____________________________________________________________________

Posted by joemckendrick in Business Process Management • Data Management • Enterprise 2.0 • Management • SOA • SOA Podcasts • SOA Vendors  |  Permalink  | Comments (0)  | TrackBacks (0)

Does the road to SOA lead through BPM [business process management]?

Yes, says Kaushal Mashruwala, writing in Financial Express.

Kaushal relates a recent Forrester Research Survey that found 38% of companies with more than 1,000 employees are not using SOA and have no plans to do so. Of the companies that are using SOA in some form or fashion, 40% haven’t begun or are using SOA with no clear strategy in place. "The 80/20 rules seem to imply that 80% of people don’t and won’t get SOA in the organization," he says.

The bottom line is that business leaders don't really care about SOA all that much, but do care very much about BPM. So, if you're an SOA advocate, why not give them what they want?

BPM, when fused with SOA, give business managers more power to change, through technology, the way their businesses are run, Kaushal observes. “The management philosophy of BPM empowers business people to think about the processes that affect their day-to-day lives and operations. It gives them a new role in defining requirements, on their terms, and creates a common language for business and IT to address real implementation level concerns. This role of BPM as the business face of SOA is not just a possibility. It’s happening now.”

_____________________________________________

Posted by joemckendrick in Business Process Management • SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

SOA makes us more agile, BPM makes us consistent. That's a combination explored in some depth by Gartner analyst Daryl Plummer at this year's DIALOG 08 event.

ebizQ colleague James Taylor is blogging live from DIALOG 08, and provides a thorough report on Daryl's presentation. Daryl spoke of the rise of "Dynamic BPM," which is the intersection between business process management and service-oriented architecture. As James relates, Daryl pointed out that "Dynamic BPM is becoming a requirement, not an option, as the frequency of change is increasing while the amount of change (the amplitude) is also increasing."

Companies that don't start marshaling their BPM and SOA resources in an integrated fashion risk increased costs and diminished opportunities, Daryl said: "If we don’t keep up we get lots of chaos, more expensive governance as we react, business agility becomes harder and decision making becomes increasingly 'seat of the pants.'"

The movement away from distributed systems or silos make business agility more of a possibility. As James quoted Daryl: "The growth of standards and frameworks, middleware and more has made it easier and provided a better platform. Adding event-processing, ESB and SOA meant that BPM could really deliver."

I like Daryl's ultimate definition of a service: "Something which does something for me without me having to do it or know how it was done - I ask, it does, it tells me its done."

Posted by joemckendrick in Business Process Management • SOA Events  |  Permalink  | Comments (0)  | TrackBacks (0)

At Delta Airlines, SOA is cutting the cost of ownership by half for its various applications and systems. That's good news, of course, but the company's experience with proactive governance also provides some valuable pointers for other companies wrestling with the politics of SOA.

Delta's SOA has been under development for more than 10 years and is now its third phase, says Bret Martin, principal enterprise architecture for Delta Technology Inc., a wholly owned subsidiary of Delta Airlines.

In a session (registration required) at a recent online conference put on by Tibco, Martin said that phase one of what the airline calls its "Digital Nervous System," or DNS, commenced with proprietary and home-grown integration hooks in 1996. As SOA and Web services evolved earlier this decade, Delta focused on bringing DNS in line with industry standards such as SOAP over HTTP, delivered through an enterprise service bus at the back end.

Now, in the latest phase of the effort, the goal is to see that "SOA is infused with the enterprise," Martin related. "SOA is a way of life for implementing business applications across the enterprise."

The greatest value Delta is seeing from SOA is reuse of IT assets and data, Martin pointed out. "Reuse is one of the big drivers for our SOA environment," he said. Delta's SOA, for example, is reusing the same customer and operational data across a range of systems, from the Delta.com Website to ticketing kiosks to ticketing counters and gate systems. "It allows check-in to happen in a uniform way," he explained. Another way services are being leveraged are by exposing services to vendors and partners, such as American Express or operations companies.

Delta engaged a cross-enterprise governance team to perform all the tasks that are expected from an SOA governance group: managing registry and repository in making sure that services are registered, defining the policies that are going to go into the deployment of a service, defining security, defining service-level agreements.

However, Delta recognized the value that the governance team was bringing beyond merely approving, registering, and managing services, Martin says. "Its not only registering services, but also trying to promote services that we already have defined, and have already put into production." This is essential, he said, because developers and architects aren't necessarily aware of what services are available out there, or where they can be found.

"What we needed is a governance organization that says they can help, 'here is a SOA business process, and is is how it can be implemented," says Martin. "Having a group that stands in front of those services, and matches business requirements to the services that we have… is a huge benefit."

Posted by joemckendrick in Business Process Management • Case Study • Management • SOA • SOA Events  |  Permalink  | Comments (0)  | TrackBacks (0)

You ever hear about the "rent-a-patient" scheme hatched by a pair of unscrupulous doctors in the LA area? In this incredible-but-true scheme, the doctors performed unnecessary surgery on patients that were recruited by marketers with promises of cash or discounted cosmetic surgery procedures.

Patients were instructed by recruiters to describe false and exaggerated symptoms that were used to create medical charts used to make the surgical procedures appear to be justified. The doctors reportedly racked up claims totaling more than $2 million before being caught. Procedures performed on the otherwise normally healthy patients included colonoscopy, sinus surgeries and thoracic sympathectomy, commonly called "sweaty palm surgery."

Claims fraud gives many insurance carriers sweaty palms, as I documented in this recent article in Insurance Networking News. The good news is that technology -- in the form of analytics -- is providing carriers the tools to rapidly sift through claims data to connect the dots on suspicious activities. Time is of the essence -- claims must be paid within a certain period of time, and once the check is cut and mailed, it's difficult to get the money back.

Here's a role SOA can play as well. Sandy Carter, VP with IBM, just published details in CIO about how SOA is enabling MIB, the industry's largest fraud detection service that is used by nearly every North American life and health insurance carrier, to spot fraudulent activity.

Sandy relates that MIB, which has 500 members, can sift through files from a range of systems to help detect potential fraud among new applicants to insurance policies. MIB's systems can access the applicant's insurance and medical history; the proximity of marriage, establishment of a life insurance policy and a spouse's untimely death, and the frequency of auto accidents resulting in medical claims.

MIB is also employing SOA to "help build an online community among credit reporting agencies, healthcare and insurance providers, and government agencies by extracting the information that’s trapped in various software applications that are located inside and outside a company and bringing it together under one roof in a faster, more secure and economical way. This information helps create an accurate and comprehensive credit history profile while also adhering to industry regulations."

Posted by joemckendrick in Business Process Management • Case Study • Data Management • SOA • SOA Vendors  |  Permalink  | Comments (0)  | TrackBacks (0)

At this week's Open Group Enterprise Architecture conference conference, I had the opportunity to join a vigorous panel discussion in which we performed a "reality check" on the state of SOA. We were joined by moderator Eric Knorr of InfoWorld, along with Tony Baer (onStrategies), David Linthicum, Tom Morgan (AutoDesk) and Chris Harding (Open Group).

Members of the panel examined the matter of SOA efforts getting stuck, or whether a "trough of disillusionment" had set in. Our panel also explored SOA's uncertain relationship with other disciplines. It was agreed, for example, that business process management is a key element of SOA and visa-versa. However, we also agreed that there is still a wide chasm between the two. Tom Morgan observed that while his organization, AutoDesk, has had a far-reaching SOA effort underway for a number of years now, they "punted" on applying BPM to the effort.

The panel also discussed the issues that linger around governance, especially in the areas of registry and repository. Tony Baer observed that there were still too many issues at the metadata level, and vendors keep pushing registry/repository as a panacea for governance.

Mashups and Web 2.0 are another area colliding with SOA. Dave Linthicum stated that mashups are, indeed, a viable part of SOA taking place within organizations. However, Eric Knorr said he has heard many organizations are not keen on letting data in or out of the corporate firewall.

SOA can work hand in hand with data management, and Tom pointed out that AutoDesk has been very effectively employing SOA-based services to pull, rationalize, and cleanse customer support data from across the enterprise.

Prior to the panel, David Linthicum delivered the keynote for the Open Group event, issuing a warning that all too often, we're still engaging in SOA for SOA's sake. SOA proponents need to redouble efforts to emphasize the business case, and de-emphasize the technology aspect, he said. "It all boils down to architecture." Still, he said, the reuse inherent in SOA can be made to work, and projects where CEOs can see demonstrable value -- such as a real-time analytics dashboard -- can achieve quick success.

Posted by joemckendrick in Business Process Management • Data Management • Enterprise 2.0 • Management • SOA • SOA Events  |  Permalink  | Comments (0)  | TrackBacks (0)

How big will complex event processing (CEP) get?

Long seen as a technical backwater, some industry observers say CEP is going to be huge, that it's the next big thing at the intersection of SOA, business process management, and data management. In a new post, Don DeLoach, president and CEO of Aleri, said that "we are on the dawn of this explosion, and I feel it is surely coming."

What's driving the market? DeLoach thinks "the CEP market is going to be huge because it is being driven by a few unmistakable elements and irreversible trends. The volume of data that organizations need to deal with is growing and the time frames in which they need to act are shrinking. CEP is specifically designed to absorb large amounts of data in real-time and analyze that data on the fly."

DeLoach compares the current era of CEP to relational databases and client/server in the 1980s, when the mainstream market wasn't taking these approaches seriously.

Use-cases which cry out for CEP include market data absorption and enrichment, algorithmic trading, smart order routing, market making and pricing engines, market liquidity discovery, market surveillance, transaction cost analysis, regulatory and compliance requirements, and especially real-time risk management -- any situation where decision-makers need to know when things are happening on a real-time basis.

Indeed, complex event processing (CEP) has been getting a lot of attention as of late, and is seen as the next stage of evolution for SOA projects. Now it has the big vendors -- including IBM -- promoting it.

IBM WebSphere CTO Jerry Cuomo recently said that he sees the SOA-EDA-CEP marriage as one of the most pronounced trends to watch in 2008. As Cuomo explained in an interview: “I really believe [event processing is] the next big thing in SOA…. we’re taking it very seriously.”

IBM even wants to take the technical allusions out of the term itself, referring to it as "Business Event Processing." The renaming to BEP makes sense, since I doubt if a concept that starts with the word "complex" is going to win a lot of converts.

Last week, Big Blue put its money where its mouth is, announcing it would be buying AptSoft, a CEP tools and platform vendor. Sandy Carter, senior vice president of IBM's WebSphere division, said that what they call "business event processing" should be at the disposal of, well, the business. "Everybody is talking about complex event processing. We are trying to rename that category, because we believe the real value is in Business Event Processing, the focus on the business. We believe we are elevating for customers something that was a deep technology capability, something that only engineers understand."

With its purchase of AptSoft and plans to include CEP (er, BEP) functionality into its WebSphere platform, IBM obviously sees a robust market opportunity emerging. Carter said IBM already has invested a billion dollars in the events processing space, not counting its AptSoft purchase.

Posted by joemckendrick in Business Process Management • Data Management • SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

Almost half of respondents that have responded so far to a new ebizQ survey report they have SOA solutions already in place somewhere within their enterprises, but these efforts are scattered.

One out of four at this time report having more than five SOA projects going on across their organizations, and most say these are separate, unconnected efforts. For the most part, few touch mission-critical enterprise applications, though a majority of respondents expect to be interfacing with at least a few of such apps within the coming year.

These are preliminary, top-line results from ebizQ's current survey of SOA best practices, which is still open and awaiting your participation. Click here to complete the survey, if you have not already done so. All participants will be entered into a drawing for a $300 American Express Gift Card.

Posted by joemckendrick in Business Process Management • Case Study • Management • SOA • SOA Research and Analyst Reports  |  Permalink  | Comments (0)  | TrackBacks (0)

There's been quite a bit of confusion and controversy over the role Business Process Execution Language (BPEL) plays in gluing business processes together with an SOA context. Some analysts even say that BPEL is not a language, it's not a about the business, and it has nothing to do with process.

In this TechTarget article, Daniel Rubio addresses some of these questions and connects the dots between Business Process Execution Language and business process execution.

Rubio reminds us that business processes are created in the minds of business managers, with technology being a secondary consideration.

When the specifications are handed down to IT folks to make the vision happen, this is where BPEL comes into play. "In the same technical context as SOAP, WSDL and WS-* standards play a role in creating individual Web services," Rubio relates, BPEL "focuses on the technical requirements needed to materialize a business process made up from Web services."

Prior to BPEL deployment, however, "a business process be broken down into manageable units of work, allowing each unit to be owned, developed and tested separately," Rubio observes. This serves not only to increase "manageability through the classical divide-and-conquer development approach, but also simplifies many design issues involved with Web services."

BPEL is then engaged through BPEL engines or business process management tools.

Posted by joemckendrick in Business Process Management • Management • SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

What is the connection between enterprise decision management (EDM) and service-oriented architecture? Is there a connection? Should there be a connection?

ebizQ colleague James Taylor, co-author of "Smart (Enough) Systems,", has connected the dots between "smart-enough systems" and service-oriented architecture in a new article, published in SOA Magazine.

This convergence will be seen in the form of "decision services" that will be available as part of SOA. As James puts it, decision services are "applications in your application portfolio or services in your service inventory that automate and manage highly targeted decisions that are part of your organization's day-to-day operation." A decision service can exist "a component or service that answers a business question for other services."

Not only can decision services help in the process of automating decision making, but they also can serve as a "single point of decision making throughout all your systems and processes," James adds. As a result, improvements and enhancements need only be focused on tweaking the specific service or services -- rather than mucking around with the back-end systems in an effort to improve business intelligence. "Decision services can also eliminate the time, cost, and technical risk of trying to reprogram many systems simultaneously to keep up with changing business requirements."

Many industry watchers have been casting about in an effort to identify the "killer application" for SOA -- the business case that will cause executives' eyes to light up and get the to exclaim, 'Aha! So that's how SOA will help us!" I think James Taylor has identified that killer application.

Posted by joemckendrick in Business Process Management  |  Permalink  | Comments (0)  | TrackBacks (0)

Can we really have it all – from SOA to Event Driven Architecture, to Business Intelligence, Business Process Management? That’s the question put forth by ebizQ colleague Beth Gold-Bernstein at her recent SOA in Action conference panel. (Full transcript here.)

Beth observed, for example, that when it comes to SOA and BPM, “companies are being bombarded with the message that you've got to have it all, and they are confusing messages about how these things relate. …I was rather surprised to hear [IBM vice president] Steve Mills use the terms ‘SOA’ and ‘BPM’ interchangeably, as if they were the same thing…” Plus, Beth said, there’s confusion as to whether EDA is a subset of SOA, or if it should be considered a separate architecture.

Panelists agreed that the relationship between SOA and BPM has always been a difficult one to pin down. And, in many ways, still off in the future. As panelist Derek Miers, CEO of BPM Focus, pointed out, “To be honest, when I put a paper together on the relationship between BPM and SOA a year or so back. I thought I was going to find a lot. And the reality is, I didn't find too many [companies] that were taking that combined BPM-SOA sort of style of approach.”

There are signs that this may be changing, Derek added, noting that there are ways SOA and BPM complement each other. “In a sense, SOA is like a best practice design principle,” he explained. “And if you look inside a lot of the BPM case studies, you'll find people applying business practices and principles that incorporate SOA-type functionality.” However, he adds, “I'm not sure I've seen a SOA case study that talks to the BPM end of the spectrum either.”

Instead, SOA-BPM efforts tend to fall within a “spectrum of activity,” he says. “Certainly, one can find plenty of cases that have services-oriented components within them. But did they set out to do a SOA architecture to begin with, or did they set out really to organize and solve some significant business problem with a business process related issue?”

Panelists agreed there is a developing synergy between the various trends coming together – SOA, EDA, and BPM. “I think that what we're seeing… today in convergence is really the result of an evolution that had taken place within the industry with the types of technologies that are available, systems that are available,” said JT Taylor, CTO of iWay Software. “The solutions that are being built for customers by combining different technologies, different types of underlying Middleware products, BPM suites, BI tools, into an overall solution that's really geared towards the true operational business processes of an organization. And really combining all these tools to support that.”

Lyndsay Wise of Wise Analytics agreed that she has little working examples yet of companies pulling together SOA and BPM within a common effort. For business intelligence applications to operate successfully, however, such convergence is important. “SOA becomes a key piece in terms of BI moving across the organization and being placed on top of different ERP, SCMs or CRM systems, in order to really have these solution interoperate and really integrate and converge with one another.”

Posted by joemckendrick in Business Process Management  |  Permalink  | Comments (1)  | TrackBacks (0)

As part of my recent industry experts podcast series for InfoWorld, I explored the emerging trend toward Event-Driven Architecture, and how it fit into SOA as we know it today.

Below are some of the pressing questions about EDA, and the answers provided by experts in the podcasts:

Why should we care about EDA, and what will EDA do for SOA-based organizations?

Oracle’s Dave Chappell says as an SOA is formed, it creates a collective network capable “of receiving asynchronous communications operating as an autonomous unit of work, and then feeding results back into the collective network. That collective network becomes this live stream of events that are constantly communicating information about your live business as it's happening. And then there are technologies such as complex event-processing engines that can tap into those live streams of events and look for a complex or sophisticated patterns and then report that up to a high level business function to make something meaningful out of it.”

As a result, end users can “gain real-time insight into your business as its happening and make real-time decision and course corrections along the way. The analogy I like to make is not only can you see that you're losing a million dollars an hour, you can also do something about it in a real-time fashion.”

(Download the full interview with Dave Chappell here.)

Is EDA a separate initiative from SOA, or is it an extension of mature SOA environments?

BEA's Theo Beack finds that EDA is an extension of SOA. “What we found is that the interest really comes from those customers that have more the real-time type of environment and are quite mature in their SOA strategies and implementations,” he says. “EDA then becomes a natural extension of this, of the existing work that they've done over the last three, four, five years.” EDA projects will take time, just as SOA projects have, Theo says. “As we saw with SOA over the last three or four years, customers are taking a very methodical approach. They really know that there's risk involved with implementing and doing something brand new that they have to learn this learning curve that they have to go through, they have to really familiarize themselves with concept, technologies and so forth.”

How difficult is EDA to implement?

David Nichols of Accenture says EDA fits very neatly into emerging SOAs infrastructures. “The nice thing and the appealing thing about event driven architectures is you can address this early in the architecture deployment, or sometime in the future with very little disruption to the existing architecture that’s in place.” Nichols says that he’s involved on SOA efforts that do not yet include EDA, but these projects will be quickly adaptable to EDA when the time comes. “The foundation is clearly laid without too much disruption to address and implement event driven architecture sometime in the future. We believe this will eventually be a very large component of most SOA-based solutions.”

How long will the transition to EDA take for most organizations?

As with the transition from JBOWS (Just a Bunch of Web Services) to SOA, EDA is a gradual process that will occur over the course of several years.

Hub Vandevoort of Progress Software says SOA is undergoing the same transition as personal computers underwent from DOS to Windows. “That took about a decade to happen,” Hub says. “It wasn't until 1995, more or less, that programmers were completely comfortable with the idea of the Windows programming model. The tools that evolved, the component models were rich, the libraries were extensive and now, every programmer programs that way. But it was a ten-year transition.”

SOA is undergoing the same transition, Hub points out, which is “exactly the same metaphorical shift from command and control-oriented, interactive SOA to more event-driven SOA. That shift will take place over ten years, but will result in the same kind of quantum leap that the Windows shift did.”

(Download the full podcast interview with Hub Vandervoort here.)

These podcasts were part of this fall's SOA Executive Forum, produced by InfoWorld. ebizQ, in cooperation with InfoWorld, has published a special supplement on SOA: Building a Foundation for Continuous Change. The report features interviews with the industry’s top practitioners to reveal the best practices, customer case studies and industry surveys that you can use to transform you tactical SOA systems into the right strategic mix of governance, and integration with complementary technologies like BPM that will increase the depths and directions of your business agility.

Posted by joemckendrick in Business Process Management  |  Permalink  | Comments (1)  | TrackBacks (0)

Forrester's Ken Vollmer, who has been a guest here for ebizQ events, is a firm believer in SOA-business process management convergence.

He recently told SearchWebServices' Rich Seeley that enterprise architects should be adopting the latest integration-centric business process management suites (IC-BPMS) in their SOA development efforts.

(I know, IC-BPMS sounds more like a long-range missile than a software tool, but that's what they're calling them...)

Such tools can help organizations implement combined BPM and integration capabilities based on an SOA foundation, Vollmer said.

As Vollmer put it at last year's SOA in Action conference: "Business process management could be implemented without SOA, because both capabilities have been available for some time. But implementations of BPM without the SOA foundation take longer. There's more ongoing maintenance and support costs are higher, and therefore ROI is delayed. An analogy of doing BPM without an SOA foundation would be like a juggler with one hand tied behind his back. He can still do some juggling, but not as effectively and not as fast as it could be."

However, many companies are taking a wait-and-see approach regarding the use of these tools.

Vollmer said organizations implementing these tools are already seeing results. For example, one leading international provider of paper products was used to automate its cash receivable business process using BPM over SOA tools.

"The old process required manual intervention to interpret payment reconciliation transactions received from its bank prior to allocating the cash to the correct enterprise resource planning (ERP) system accounts," Vollmer said. "The new process feeds the reconciliation statements to an integration-centric business process management suite (IC-BPMS) that automatically allocates receivables directly into the correct ERP accounts by electronic interpretation of simple business rules stored in the BPM tool. This change completely eliminated manual efforts while shortening processing times by two days."

Posted by joemckendrick in Business Process Management  |  Permalink  | Comments (1)  | TrackBacks (0)

Many products -- especially software -- can sit on shelves for years without raising a stink. Not so in the sour cream business -- if today's product doesn't ship, things can get really, well, sour.

That was the challenge facing Daisy Brand, a leading domestic sour cream manufacturer headquartered in Dallas, TX. Like most companies these days, the company had a variety of systems and formats that tended to throw monkey wrenches into its distribution processes.

Order changes, for example, span manual (fax, email, spreadsheets, etc.) and system functions (EDI, ERP, third party freight, home-grown logistics and shipping applications), causing the process to slow and increasing spoilage costs. There were yawning gaps between systems, business processes and employee roles coupled with inevitable customer-driven changes were causing those processes to breakdown and impact the business.

SOA and new process technologies saved the day, according to Kevin Brown, Daisy Brand's director of IS, who spoke at the recent Gartner BPM Summit. Interestingly, the company went with a Microsoft-intensive platform on which to build its SOA.

Daisy Brand's first priority was to automate as many processes as possible, yet preserve the flexibility of the manual system to adapt to customer demands. The company sought to convert "order change" and many other business functions could become services in process-based applications.

The company went with Ascentn's AgilePoint Business Process Management Suite (BPMS), integrated with Microsoft Office SharePoint Server, Microsoft BizTalk Server and the .NET Framework. The company's order change process now incorporates everything from a faxed order to customer EDI shipping acknowledgements to RFID product information flowing across an Enterprise Service Bus.

Brown states that Daisy Brand generated a positive return on investment within six months and provided shared visibility into the order change process across groups and roles. This enabled employees to examine the actual state of an order to proactively address issues. Now "order change" is an easily used business service within an Microsoft SharePoint Server-based employee portal as a part of Daisy Brands' growing SOA library of process-based services.

Posted by joemckendrick in Business Process Management  |  Permalink  | Comments (0)  | TrackBacks (0)

Since its current inception a few years back, SOA has been mainly about delivering integration and agility within the corporate firewalls. But there's a vast frontier -- business-to-business interactions -- that some SOA implementations have only begin to tap.

In this new article in Supply Chain Executive, Bruce Williams, general manager and vice president of BPM solutions at webMethods, talks about one retailer's initiative to maximize its supply chain, employing new SOA and BPM strategies.

First, as anyone in retail and consumer goods manufacturing can attest, the age of overstock is long over. As Williams notes, "companies simply can no longer afford to invest in huge physical warehouses and carry massive amounts of stock in anticipation of customer demand. Neither can they wait days or weeks for data to be loaded into data warehouses in order to create demand forecasts that may be egregiously out of phase with customer activity by the time those analyses are produced. Market forces are requiring businesses create ever-tighter connections between product delivery and customer needs and wants, and take the waste and latency out of the processes by which they fulfill those needs."

Such was the challenge for Albert Heijn (AH), a big retailer based in the Netherlands, which lacked visibility into its supply chain, and was forced to rely on fixed replenishment schedules that were based on local demand forecasts.
The company needed to minimize the amount of inventory in warehouses and across its 170 stores.

The solution was to change from a once-daily to real-time monitoring replenishment cycles. This required "replacing existing integration points with an infrastructure capable of fully connecting all 720 stores to corporate systems and suppliers," Williams said.

The solution AH pursued included a SOA-based approach to bridge its previously separated technology environments. AH's new platform consists of webMethods' Integration Server (IS) on each store's back-office server, employing a JDBC Adapter to communicate with the server's database for processing outbound and inbound messages. Each store's IS connects to a central broker at AH's headquarters, which in turn connects to back-office applications.

AH rolled out the new system to the first store in November 2005, a process that was completed for all 720 stores by May 2007. AH can now bridge previously separated application-to-application and B2B environments within a single infrastructure that also controls the supply chain. Information on each retail sale streams all the way back to suppliers. All point-of-sale data is automatically aggregated with sales data from across all 720 stores and delivered to a centralized replenishment application. Decision makers now get replenished data every half hour to help with supply chain decisions.

To build the case within the business, AH created cross-functional teams to bring together business and IT people, and focus the strategy on the business needs and projected benefits of the project.

AH is now implementing webMethods Business Activity Monitoring technology to provide end-to-end monitoring of key performance indicators (KPIs), Williams adds. This new capability "will enable the retailer to continuously improve the replenishment process, monitor technical connectivity between stores and headquarters and gain additional visibility into timeframes for transactions in replenishment processes."

Posted by joemckendrick in Business Process Management  |  Permalink  | Comments (0)  | TrackBacks (0)

At least one big vendor says that the best route to business process management is through SOA. Or, conversely, the path to SOA goes through BPM.

IBM, it is reported, is throwing a ton of resources at bringing together business process management (BPM) and service-oriented architecture (SOA), and, hopefully, in the process, getting IT to think more like business users in order to use this technology more effectively. (PDF available to registered ebizQ members here.)

That's the latest word from Dennis Callaghan, analyst with the 451 Group (and a colleague of mine going back to our Midrange Systems days).

IBM, in its IBM Way of doing things, has launched a head-spinning assortment of initiatives around fusing BPM and SOA, including products, partnership programs, and university-based certification programs.

But in the big picture of things, Dennis observes that "such a confluence of technologies means that IT will have to collaborate more with business users – and so will need greater knowledge of the business side."

There are other vendors tacking in this direction as well, including SAP, Ramco Systems, BEA, and Sun Microsystems, Dennis observes.

Posted by joemckendrick in Business Process Management  |  Permalink  | Comments (1)  | TrackBacks (0)

One of the challenges in bringing business process management together with service-oriented architecture is that the two essentially exist in two separate worlds, managed and promoted by different teams with different agendas. However, the two need each other more than ever in order to succeed.

In a new post, Nick Malik describes the fast-evolving relationship brewing between BPM and SOA.

"These two activities are twin brothers," he says. "Their parents are the same, and they have many similarities, but the serve different needs in the enterprise." Nick takes the analogy (a good one) a step further to describe the their common "father" as data, and their common "mother' as the business event.

Who's your Daddy? Nick describes the fatherly role as the common data model, one of the few pieces really needed to "make enterprise architecture work." In addition, he adds, "the common data model should contain ONLY enough information to define the things that must be present to define a business document, to identify the unique key for that document, and to define the relationships with other business documents."

The motherly role, then, is the business event, or "an event that defines the boundaries between business processes." He adds that "you have to know where a process starts and where it ends. Otherwise, you could find yourself comparing a single apple to a grove of apple trees."

Of course, data and business events processing need to be mature themselves before they're ready to raise children. This is not a top-down design, but it is a top-down constraint. "If you build your SOA before you understand the data, you are creating multiple versions of the truth," Nick points out. "If you attempt BPM before you identify your events, you will burn huge efforts to compare apples to apple trees, producing no value."

Posted by joemckendrick in Business Process Management  |  Permalink  | Comments (2)  | TrackBacks (0)