SOA in Action

There are two types of events that fit into enterprise event processing scenarios -- those that matter to IT, and those that matter to the business. While many organizations seem to have a good grasp on managing IT events -- such as a server crashing -- few are ready to handle business events. But this is changing.

In his keynote presentation at ebizQ's recent Event Processing conference, Forrester's Charles Brett described how business event processing is the next great horizon for business, but is fraught with many challenges. (Replay and transcript of Brett's talk here.) The important challenge for organizations is to "understand which events matter," he said, adding that "some people think that all events matter." Those businesses that can successfully leverage event processing are those that can identify the events that have the greatest impact.

This ability rests with the business, not IT, he adds:

"IT doesn't necessarily know which events exist in the business and which ones could be used, or which are more relevant, or which are less relevant. Indeed, one the big dangers in event processing is one could have too many events many of which may not actually have a great deal of relevance."

Brett outlined some scenarios where business event processing can make a difference. For example, if a customer didn't buy something its stops halfway through a transaction, it pays to understand why this happened.

Or, if "a financial exchange came to a halt because they didn't know that something wasn't happening. They thought they knew what was happening; all systems showed green but an event screen wasn't coming through and the exchange grind to halt." Employing event processing to walk them through the chain of events can help prevent this disruption from happening again.

Predictive analysis run against an event engine can help schedule field service calls to improve the uptime of critical services to customers.

Such "non-IT events" businesses need to process and digest may come from sensors, signaling, production lines, and other sources inside the company, as well as outside sources such as radio, television, news channels, weather channels, Websites, and GPS, Brett says. Such events are "the ones that haven't been processed in the past but will be in the future."

The best way to capture business events and direct them to business managers is through Business Activity Monitoring (BAM), Brett says. "BAM is really about taking events and raising them to a level that decision-makers or people who have responsibility for taking actions can do something."

BAM is used for real-time analytics, "not only for processing and analyzing of event data, but also to feed visual dashboards and the like in order that people can see what is going on within the business -- very much intended for business users," he continued. BAM is still relatively new on the scene, and will take time to fully integrate into the business. The challenge with BAM is to avoid overwhelming business users with information, or alternately, reducing information about events to "such simplistic levels that it's just ignored."

While some analysts say event processing is a natural extension of SOA, Brett feels that Integrating event processing capabilities into some earlier SOA implementations may prove difficult. SOA-based services "can be picked up by event processing and similarly when an event processing engine emits events out of on the downstream side, there's no reason why services shouldn't pick it up. So there's no architectural reason why they shouldn't fit together. The question really is going to be how the services were originally designed, architected, and delivered."

A replay and transcript of Brett's talk is available here.

______________________________________________________________________

Posted by joemckendrick in Event Processing • SOA Events • SOA Research and Analyst Reports  |  Permalink  | Comments (1)  | TrackBacks (0)

Financial services companies have had a tough ride over the past year, so anything that can be done to better take advantage of their information technology assets to streamline operations and increase the situational awareness and agility of the business is a good thing. That's where SOA and business process management (BPM) come in.

JPMorgan Chase, of course, is right at the epicenter of the financial storm that has been ravaging the industry, and the company's dominance and stability is due in no small part to its ability to smartly leverage SOA and BPM.

At ebizQ's recent Financial Services Live Panel, Ron Ambuter, CTO of the BPM Workstream Group at JPMorgan Chase, described how his company has been able to adapt and adopt technology and methodologies to help reduce costs, improve customer experiences, and maintain a competitive edge. Ronan Bradley, analyst with Lustratus Research and former CEO of PolarLake, led the discussion, and was joined by Keith Swenson, chief architect at Fujitsu Computer Systems, and Hub Vandervoort, CTO of Progress Software. (Archived recordings and transcript available here.)

Ambuter described how JPMorgan Chase was interested in the concept of reusability, which would help the organization "get better leverage out of the efforts that we were making to build and buy applications by reusing components instead of rebuilding and rebuying them over and over again."

With the troubles the industry has been having around subprime mortgages, writedowns, and tighter credit, it's likely

regulation and oversight will only increase, particularly in regards to risk management. Bradley questioned whether the prevalence of regulation holds back or encourages SOA adoption. Ambuter said regulatory mandates have accelerated his company's drive to SOA, noting that "the concept of SOA allows us to react probably quicker, and more expeditiously, more cost effectively."

The panel also discussed the challenges around extending managing services and infrastructure across multiple groups within organizations. Financial services organizations, of course, typically have multiple lines of business, from securities to brokerages to mortgages. Ambuster explained that organizational issues were his greatest challenge as well in implementing a cohesive SOA and BPM strategy at JPMorgan Chase. "We have rules and responsibilities that are compartmentalized by groups of folks, and we realize that as you go into this stuff, education and cross-pollination of information is critically important. It's not just the technology folks that need to understand this stuff, it's the risk folks, and it's the governance folks and it's the finance folks, and it's the business side folks."

Vandervoort agreed that "it's easy to get bogged down in trying to get alignment on a lot of different points across all the groups." He recommended three approaches to the problem, including "getting your transports aligned between business entities so that you can use eventing-oriented mechanisms to communicate across domains"; establishing SLA and security policies that ensure visibility; and establishing a common enterprise data model.

"You have to get your semantics aligned among the members," he said. "And that doesn't have to be a common vocabulary in its entirety, but certainly what we regard as the data in flight, those things that fly between domains and different working groups have to be highly normative."

Aligning SOA and BPM is also important. "While SOA is basically a technology trend, BPM is essentially a management trend," said Swenson. "A lot of people look at BPM and they get confused between the management trend and the technology trend. A lot of technology product companies have looked at BPM and looked only at the technology and come up with a kind of a programming language that they claim to solve the problem, which is great for the IT side of the house, but it leaves the business side out of this. As far as the adoption of BPM goes, one of the biggest barriers to adoption of the real business process is essentially the fear that the manager will lose control of the process."

Complex event processing is also a key initiative financial services companies need to undertake. As Vandervoort observes, "financial services is moving kind of in a logarithmic increase in velocity... things are ten times faster than they were a decade ago. When you go from three days order to settlement or trade to settlement to now under two hours trade to settlement, if I'm doing nightly reporting, I'm way out of alignment. Whereas ten years ago, that was three times faster than the speed of the pipeline. Today, you the ability to run awry in risk, terms, and exposure terms has been seen multiple times in the industry in recent years."

For companies seeking to increase agility and be more responsive to these highly competitive markets -- not just in financial services -- the combination of SOA and BPM can break down the constrictive silos that cut off essential data and processing resources.

____________________________________________________________________

Posted by joemckendrick in Business Process Management • Data Management • Event Processing • Management • SOA • SOA Events  |  Permalink  | Comments (0)  | TrackBacks (0)

Complex events "can be anything from a sequence of temperature readings to something like the First World War, which was a very complex event indeed." -Dr. David Luckham

David Luckham -- considered the "father of complex event processing" -- recently joined Dr. K. Mani Chandy, Rodney Morrison, and Beth Gold-Bernstein for an informative panel discussion on the relationship between EDA (Event Driven Architecture) and SOA. The panel, part of ebizQ's recent Event Processing Virtual Conference, which brought together the leading thinkers and proponents of EDA and Complex Event Processing (CEP).

The panelists were divided, however, over the extent of the role of human involvement in complex events.

Chandy, for example, said that human engagement is essential at some point in the complex event processing cycle. "I’ve seen quite a few applications, but almost none in which there is no human involved" For example, Chandy said, "in military applications, when there is a gun or device is fired to kill somebody. It’s always done with a human being responsible for their final action."

As Chandy pointed out:

"I think its absolutely critical for human beings to be part of this sense and respond system. It's important how the application supports the human being if you're looking at trading or fraud detection. In all these cases, its really important to have a human being involved. Fraud is one case where you might have an application that informs a credit card user that something inappropriate may be going on, without having a human being first check that."

Luckham, however, pointed out that there are many events are processed independent of human intervention. "There are many examples of event driven architectures where there are absolutely no humans whatever," he said. "The CPU on your computer is an event driven architecture, believe me. And its entirely event driven, clocked, without a human in the loop.

EDA is part of SOA on two different levels, Chandy said:

"One way that I’ve seen EDA used in conjunction with SOA is for service management. Many SOA vendors are exposing metrics that can give you information like end to end process time and activity times. Those metrics can be provided to a CEP system to help control and manage those services. I can, for example, do dynamic provisioning for a service that's getting maxed out."

Chandy also connected the dots between EDA and SOA. "If SOA means loosely coupled subsystems with very clean interfaces, so that new systems could be coupled into the substrate. Then EDA events fit within that framework, because EDA is also based on a loose framework, and is extensible." In a request-reply SOA scenario, "then EDA can still be coupled. There will be layering between the push and the pull parts."

_____________________________________________________________

Posted by joemckendrick in Business Process Management • Event Processing • SOA • SOA Events  |  Permalink  | Comments (0)  | TrackBacks (0)

Behold the zebra in the wild savanna -- when a it senses the presence of a lion, it knows to run away, as fast as it can. When it senses food supplies, it knows to act.

Do today's companies have this much situational awareness, and an ability to act quickly to survive and thrive? Not yet, but thanks to new approaches such as complex event processing, they're getting there, according to Gartner VP Roy Schulte. "We can teach computers to do what a zebra does," Roy said. "To collect and process event data to respond quickly and effectively."

In his keynote kicking off ebizQ's recent Event Processing virtual conference, Schulte broke down the essential pieces of complex event processing, and describing how businesses can leverage CEP, or being able to act, in real time, on multiple streams of event data flowing in from different parts of the enterprise. "The value of complex event processing, overall, can be summarized as improving situation awareness. Simply put, that is just knowing what is going on, so you can figure out what to do." The benefits of complex event processing, Schulte said, include better decision quality, faster response times, reduced information glut, and reduced costs.

Schulte defined a business event as a "meaningful change in a state that is something that is relevant to the business. Examples include depositing or withdrawing money from a bank, submitting a purchase order, or hiring an employee." There is also a second term, "event object," that describes how the event is packaged for processing, typically as an XML document these days. "We have to record events using event objects so computers can receive them and do computations on those events," Schulte said.

However, while all companies have always been event driven -- with millions, if not billions, of events in a single day, most events are still handled manually, by people, not computers. "At any one second, a large company has on its network anywhere from 10,000 to 10 billion business events," Schulte explained. "At the low end, that's almost a billion events per day -- at the high end, that’s almost a trillion events per day."

The challenge is that most of the stovepiped and legacy applications that power enterprises are not yet event driven, Schulte observes.

But there's great practicality in automating the ability to capture and make decisions on multiple event streams coming into the core business systems, Schulte says. "For example, you can have a complex event that says, ‘this mornings sales were 30% above our daily average.' That of course is much easier to digest and act on than sending a person 500 detailed sales records, and making the person compute what happened that day manually."

The growing array of sensors, such as RFID tags, combined with front-end systems such as business activity monitoring (BAM) dashboards make complex event processing a reality with today's technology, Schulte points out.

"In many cases, the complex event processing system Is just a front end being used for decision support. The output of the CEP engine is sent to a person through a BAM dashboard, or through an alert such as email or SMA or an Atom or RSS feed. in this case, we have a two-stage computation. In the first stage we’re using a computer to narrow down the data. And in the second stage, we still have a person involved to do the final analysis.

"An application system, or some other device or some other system, detects the event, and generates a message or a notification that is sent to a person. That notification is the event object or event report sent in the form of a message through message-oriented middleware, RSS, a Web service, or an email, or some other communication mechanism. The response to an event may be a manual activity, done by a person or it may be a SOA service or business process or some other application."

However, things could get interesting as CEP systems develop, Schulte added. Namely, the need for human processing could be taken out of the equation all together. "We can bypass that person entirely; we can build enough smarts into the complex event processing engine to determine the specific response that is needed."

Schulte provided a working example of complex event processing in action within the airline industry:

"In large airlines, there is an event oriented middleware that... acts as an enterprise nervous system. Information from hundreds of sources, including sensors on board the aircraft, information coming in from the FAA, and information coming in from standard systems is sent to the enterprise nervous system, and is temporarily stored in event databases. It helps to create the data, the outgoing alerts and notifications that is sent to hundreds of applications on the consuming side to respond to threat and opportunity situations as they emerge. By having information quickly, each of these systems in their respective departments can respond faster. ...Information helps the fueling and maintenance management applications to change their schedules and so forth. By using an event based system, the turnaround time of each plane can be shortened… Fewer airplanes are needed to handle the same schedule."

______________________________________________________________________

Posted by joemckendrick in Business Process Management • Data Management • Event Processing • Management • SOA • SOA Events • SOA Research and Analyst Reports  |  Permalink  | Comments (0)  | TrackBacks (0)