SOA in Action

We've talked quite a bit here and throughout the ebizQ-sphere about the convergence between SOA and enterprise data management, and how services can be employed to source and leverage data for various analytics.

In a new interview with Michael Meehan, Kirstan Vandersluis, founder and chief scientist for XAware Inc., directs the discussion to a new type of service emerging -- the "data service mashup." As the name suggests, he is talking about employing lightweight Web 2.0 methodologies to build front-end apps that pull needed data into a dynamic presentation for the business end user.

But it's a lot more than the on-the-fly Google Maps-style mashups we've been seeing in recent times. As Vandersluis describes it, a data services mashup is "a mashup in terms of pulling data from multiple sources into a logical unit. So it's a mashup in terms of data. It's not a visual mashup. That's a whole different ballgame. We're pulling data from virtually anywhere into a logical unit. We're trying to take a bunch of back-end data systems, which are typically very complex, and make it look like something much more rationalized in terms of an XML Schema that somebody put some thought into designing."

The model Vandersluis is proposing is built around an XML schema. He notes that "the applications are relying on the contract, which is defined by the XML schema."

Data services mashups may work in conjunction with, or even serve as an alternative to data warehouses and data marts in some situations, Vandersluis also said. "It's just a different way to do things and there are trade offs.... The one benefit to our approach is it's all real-time. We're sending the sub-queries out in real-time to whatever systems you want."

So if initially you pointed all your data services at your data warehouse, and you find out later that's not real-time enough, you can change the data sets that are being mashed up, so one of them hits an operational data source.

Vandersluis and other data service mashup proponents have their work cut out for them. Understanding and adoption at the business level has been "spotty" to date, Vandersluis observes. "I think it's more a matter of how well they understand service-oriented architecture, which is the typical use case of a data services layer."

Organizations that are embracing SOA are more likely to be early adopters of data services mashups, he says. In addition, there is impetus at sites developing rich Internet applications. Such Web 2.0-ish are more likely to see a large volume of services, versus the more centralized and business-focused services that will be part of SOA.

____________________________________________________________________

Posted by joemckendrick in Data Management • Enterprise 2.0 • SOA  |  Permalink  | Comments (2)  | TrackBacks (0)

SOA comes in many shapes and sizes, but most times when the discussion comes up, it's about the challenge of service-orienting some aspect of core mission-critical systems. To date, of course, most SOA and Web services deployments have been on the periphery. Sometimes, these peripheral projects have been referred to as "lunchroom Web services" -- only a prelude to the grander SOA yet to come.

That's why it was interesting to run across this article, published in Thomas Erl's SOA Magazine, that brings up the topic of service-orienting the periphery as part of the overall enterprise SOA strategy.

At the periphery are "edge" applications, databases, and systems are probably more numerous and pervasive than the systems found at the enterprise core. They may consist of repositories, embedded systems, open source databases, and distributed or departmental servers. Often, they are low or no-budget efforts that address a specific point solution.

The challenge is bringing these systems and their processes into the main enterprise SOA effort. The article's authors, Paulo Rosado and Rodrigo Castelo, say there's a good business case to be made for service-orienting these edgy (or "shadow IT") systems, because while edge apps may be cheaper and faster to implement, they end up costing enterprises a bundle in maintenance and overhead.

These siloed projects cost more than more centralized projects, and eventually mushroom into more headaches for the centralized IT department. What seem like small, off-the-radar projects eventually end up with "ever-rising maintenance and support expenditures," the authors say. Ultimately, they estimate, the total cost of ownership could run up to be five times as much as centralized apps and systems. Plus, they explain, "ignoring shadow IT incurs the risk of hindering your SOA evolution, overlooking potential application innovations that could streamline business process and increase revenue, and increasing expenditures in the long run due to retrofitting, and also potential security and governance breaches."

Rosado and Castelo provide four key questions that need to be asked as part of any attempt to bring edge systems into an SOA effort:

• How fast can you deploy new services in your SOA environment, and at what cost? This is an important metric to discover, since time-to-market of new services will determine how well SOA is adopted at the edges. Issues around scalability, SLAs, access, and authorization (detailed below) will also affect speed of service deployment.


• Is your SOA environment ready to deliver new services with no SLA impacts on your core systems? Perform tests on your infrastructure to see how well it scales, Rosado and Castelo advise. "Because once you provide shadow IT the requested services, you need to assure their quality of service and the SLAs of your core systems, which are now serving requests to the outside. Shadow IT systems may initially be used by only a couple of end-users but can then be rapidly adopted by an entire department. An onslaught of requests will decrease your core systems' responsiveness, making them unusable by your shadow IT and any other systems dependent on them. In the worst case, this can overload your core systems and cause them to collapse."


• Can you control which systems access which services? "Once you expose services to your shadow IT, you must somehow ensure that only granted systems can access them."


• Can you transitively assure authentication and authorization once you lose control over the information? Rosado and Castelo advise adding a layer to the SOA to manage fine-grained governance issues. "More important than just controlling which systems can access which services, is controlling which entities in your organization can access and change the information that is now accessible through your edge applications."

____________________________________________________________________

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

There are two types of events that fit into enterprise event processing scenarios -- those that matter to IT, and those that matter to the business. While many organizations seem to have a good grasp on managing IT events -- such as a server crashing -- few are ready to handle business events. But this is changing.

In his keynote presentation at ebizQ's recent Event Processing conference, Forrester's Charles Brett described how business event processing is the next great horizon for business, but is fraught with many challenges. (Replay and transcript of Brett's talk here.) The important challenge for organizations is to "understand which events matter," he said, adding that "some people think that all events matter." Those businesses that can successfully leverage event processing are those that can identify the events that have the greatest impact.

This ability rests with the business, not IT, he adds:

"IT doesn't necessarily know which events exist in the business and which ones could be used, or which are more relevant, or which are less relevant. Indeed, one the big dangers in event processing is one could have too many events many of which may not actually have a great deal of relevance."

Brett outlined some scenarios where business event processing can make a difference. For example, if a customer didn't buy something its stops halfway through a transaction, it pays to understand why this happened.

Or, if "a financial exchange came to a halt because they didn't know that something wasn't happening. They thought they knew what was happening; all systems showed green but an event screen wasn't coming through and the exchange grind to halt." Employing event processing to walk them through the chain of events can help prevent this disruption from happening again.

Predictive analysis run against an event engine can help schedule field service calls to improve the uptime of critical services to customers.

Such "non-IT events" businesses need to process and digest may come from sensors, signaling, production lines, and other sources inside the company, as well as outside sources such as radio, television, news channels, weather channels, Websites, and GPS, Brett says. Such events are "the ones that haven't been processed in the past but will be in the future."

The best way to capture business events and direct them to business managers is through Business Activity Monitoring (BAM), Brett says. "BAM is really about taking events and raising them to a level that decision-makers or people who have responsibility for taking actions can do something."

BAM is used for real-time analytics, "not only for processing and analyzing of event data, but also to feed visual dashboards and the like in order that people can see what is going on within the business -- very much intended for business users," he continued. BAM is still relatively new on the scene, and will take time to fully integrate into the business. The challenge with BAM is to avoid overwhelming business users with information, or alternately, reducing information about events to "such simplistic levels that it's just ignored."

While some analysts say event processing is a natural extension of SOA, Brett feels that Integrating event processing capabilities into some earlier SOA implementations may prove difficult. SOA-based services "can be picked up by event processing and similarly when an event processing engine emits events out of on the downstream side, there's no reason why services shouldn't pick it up. So there's no architectural reason why they shouldn't fit together. The question really is going to be how the services were originally designed, architected, and delivered."

A replay and transcript of Brett's talk is available here.

______________________________________________________________________

Posted by joemckendrick in Event Processing • SOA Events • SOA Research and Analyst Reports  |  Permalink  | Comments (1)  | TrackBacks (0)

In physics, they say the simplest solution to a problem is usually the right solution. Applying the laws of physics to SOA, then, it stands to reason that enterprise service buses (ESBs) are the simplest path to SOA within organizations -- and perhaps in many cases, the right path for the organization.

There's been plenty of concern that an economic downturn would skim already bare-bones IT and integration budgets; therefore making it more difficult to launch SOA initiatives that cut across the enterprise. ESBs are a vehicle that can help break the "complex links directly between applications, which are reducing agility, and consuming chunks of the IT budget just in maintenance," says IBM's Lief Davidson.

Lief recently sat down with ebizQ's own Peter Schooff to discuss how SOA managers can employ ESBs to get around any budget crunches incurred from today's tough economy. (Link to transcript and audio file here.)

"The business needs to become simpler and faster to address the various business problems and, hopefully, with a reduction of the complexity that IT is being bound into today, more of the budget can actually be spent in delivering innovative new value rather than keeping existing systems running." This is where ESBs come in, Lief said.

There's been a lot of discussion across the industry as to how far organizations that don't necessarily have the resources or executive political can pursue SOA. Add budget crunches, and SOA becomes an impossible sell -- even though it does provide cost savings and containment in the long run. ESBs may provide a way to kick-start SOA in any climate.

_____________________________________________________________________

Posted by joemckendrick in SOA • SOA Podcasts • SOA Vendors  |  Permalink  | Comments (0)  | TrackBacks (0)

In a new post, Alex Cameron, an EDS Fellow, notes that most well-established products and services have what is called a "dominant design" methodology behind them. For example, in the vehicle manufacturing industry, "the enclosed steel body car is a classic dominant design." The innovative growth and experimentation stage for the industry has long passed, and most, if not all, auto manufacturers agree on a common design. This can also be applied to aircraft, motor cars, lamps, typewriters, and integrated circuits -- you see little, if any, deviation by maverick manufacturers.

This same evolution from fast-paced innovation to dominant design has been taking place within the software industry -- much of the design has coalesced around Unified Modeling Language (UML).

In the next phase of software evolution, Cameron suggests that the industry is coalescing around service oriented architecture as the dominant design for integrating systems together.

"The concept of services and loose coupling is the product of a high level of innovation in the area of software architectures. The past saw mainframe centric architectures, Client Server, N-Tier, Layered Architectures, EAI, BPM and many others; all innovations towards solving a common problem, that of developing and integrating software components to form an information system architecture. However, none of them could be called a dominant design, simply because none had a follow-on and distinct phase of process innovation. In the case of a Software Architecture we could see this as a rise in "standards" that are adopted to control its implementation."

Cameron adds that SOA has taken software design well beyond what previous models have achieved. "We are now seeing the emergence of SOA Governance, SOA Maturity models and the like - I can guarantee no one has heard of a Client-Server Governance Model or Maturity model!"

It makes sense to conclude SOA has reached the stage in which it is the dominant design model. Of course, there are also plenty of analysts and pundits declaring SOA to be somewhat of a failure. So everybody and nobody is implementing SOA. But think about it -- if vendors and practitioners want to standardize and enable interoperability between disparate applications, interfaces, and services, what's the alternative to SOA? How else would they be doing it?

______________________________________________________________________

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

For some time now, I've heard consultants and analysts propose launching "centers of excellence" or "competency centers" to better manage newer IT initiatives such as SOA is one that has been proposed by consultants for some time now. Not only does such a body help kick-start SOA efforts in the business, but it's also a way to keep such efforts above organizational politics and fiefdoms. The SOA effort can proceed without being encumbered by individual department or management agendas.

Excellent idea. However, such efforts have been slow to evolve, and may be difficult for smaller to medium-size businesses to get in place. ebizQ's recent survey on SOA governance, conducted in partnership with SAP, found that only nine percent of respondents had established some form of competency center/center of excellence to promote and keep SOA efforts on the right track. Almost all of these were larger companies with revenues exceeding $1 billion or more annually. (Our Webinar on the survey results can be viewed here.)

While getting management to commit time and resources to an SOA center in and of itself may be an uphill battle, there may be value in extending the reach of such centers to enterprise data integration as well, according to John Schmidt, VP of Informatica and Chair of the Integration Consortium. This center may be described as an "Integration Competency Center," or ICC. In a chat with Beth Gold-Bernstein, Schmidt provides some guidelines to help businesses of all sizes implement an Integration Competency Center (ICC). He describes the elements that go into an ICC:

Financial management: "The ICC operates as a shared service. This is a set of best practices around charge back for shared infrastructure and individual services."

Architecture: "The ICC does not do enterprise architecture, but is responsible for the information architecture. They work with the enterprise architecture group, and 'connects the dots,' by mapping schemas to physical data sources to enable the translation, transformation, and integration. This ICC is the central federated repository."

Business Process Management. "This is not BPM per se, but this includes service flow modeling, information flows, business event modeling, and common definition of business events."

Integration methodology: "The process of running an ICC, defining it, organizing it, all the things you need to run an integration group, and how it will interact with other IT groups."

Metadata management: "The ICC group is responsible for data assets. Metadata ends up being a federated model."

Modeling management: "This includes techniques around canonical data modeling, what are the best practices, how do you build them."

Integration Systems. "This is about running integration systems as a specific class of applications – all the discipline of how your manage, plan and operate the system."

_____________________________________________________________________

Posted by joemckendrick in Data Management • Management • SOA • SOA Vendors  |  Permalink  | Comments (0)  | TrackBacks (0)

In a new post here at ebizQ, Tony Baer has weighed in on the mashup phenomenon that has been gaining ground within the enterprise world.

Tony and I are in agreement that mashups -- the whole idea of lightweight front-end apps that can be assembled by business users -- add a new, more palatable dimension to SOA.

Some observers have been saying that mashups are too consumerish and funtime for more serious SOA efforts. However, there's also an opposing stream of thinking -- some Web 2.0 proponents don't think SOA should be brought into the mashup world -- it will ruin all the fun. David Linthicum says that he even has heard from people "who did not want the term 'mashups' sullied with the term 'SOA.'" As he noted, "the core message is that they view SOA as something that's "enterprisy," and mashups as much more innovative and not really enterprise related."

Here's what Dave had to say about that:

"Not sure I agree with that. While indeed mashups are an innovative way of building very cool applications from many available resources, visual and non-visual, they are still composite applications. While I'm seeing mashups that are completely Web-hosted, I'm seeing more and more that are a mix of Web and enterprise resources, as well as mashups that are true "'enterprise mashups.'"

Indeed, mashups are gaining ground. Dion Hinchcliffe, for one reports fast-breaking progress in mashup adoption across the industry. He noted that there were at least nine different announcements around Web-based mashups coming out of the recent Web 2.0 conference. Dion also cites the latest market overview from Forrester, which estimates that this space is expected to grow into a $700 million a year industry sector by 2013, or about 1% of the entire software industry.

Mashups do not present an alternative or competition to the composite apps that have been part of the SOA world. As Tony puts it, "the approach is not a black and white SOA vs. mashups choice for enterprise integration, but rather, use of mashups for the last mile of integration that may, in many cases, utilize data services, feeds, or other sources that more often than not are exposed as Web or RESTful Services."

And, may I add, the ease and lightweightness of mashups make it easier to sell the concept of SOA to the business. Because now they can see and feel and touch service orientation. They can (gasp) actually create services on their own. (Here's a case where good governance has to come in -- can't you just see business users, having had a taste of their own service creation, going wild?)

However, as Dion notes, we're not quite there yet with easy-for-business-users-to-use tools. Dion observed that "the tools that empower users to weave together existing Web parts and open APIs into the exact solutions they need are just now becoming easy enough and robust enough to readily enable these scenarios."

This is in line with research I have been involved with (Evans Data), which, in a recent survey of 380 enterprises, found that the greatest obstacle to user application creation, cited by 22% of respondents from a list, is the lack of availability of easy-to-use assembly tools.

Tony agrees, and sees the progression of tools and methodologies as such:

"I believe that this represents stage 2 of 3 -- the first was emergence of primitive Ajax tooling, the second is more formal delineations that in many ways reflect existing silos within enterprise software architecture. That is, you have database tools, coding tools (also known as IDEs), and then you have your Web design. Ultimately, in stage 3, these approaches themselves will mash up as simply multiple technology-driven paths to the same summit. As long as IT vets it, you shouldn't need different tooling to combine a structured data service with an unstructured content feed, a piece of business logic, with some rich expressive user interface design artifacts."

If mashups can be brought into the same governance space as SOA services -- that is, automated and non-intrusive vetting of services deployed, and accessible directories of what is already out there -- we will be entering an era when business professionals take responsibility for their own domains and applications. That will free up IT to spend more time with the strategic aspects of the business.

IT will better understand the business and the business will better understand IT -- that sounds like the best mashup of all.

____________________________________________________________________

Posted by joemckendrick in Enterprise 2.0 • SOA Vendors  |  Permalink  | Comments (0)  | TrackBacks (0)

When most people think of the US Peace Corps, images of volunteers drilling wells and building schools in impoverished corners of the globe come to mind. Service oriented to the hilt, in the old-fashion sense of the phrase. This view is accurate, but these efforts still require a information technology infrastructure to best allocate and deliver its resources.

Until recently, that was provided by a mainframe system sitting in a silo in Washington, DC. A new report in Government Computing News describes how the Peace Corps moved to a service-oriented architecture methodologies to help keep track of and provide information services to its varied overseas activities.

Ram Murthy, director of application systems at the Peace Corps, recognized that a highly distributed organization -- spread across 70 countries -- needed a far more distributed IT infrastructure than the mainframe system it formerly relied on. Overseas users were unable to access the mainframe.

As is the case within many organizations public and private, Murthy encountered plenty of headwinds. Government managers were skeptical of the value of SOA to an organization such as the Peace Corps, because they thought SOA required 24x7 connectivity to Web services, and many Peace Corps officials were in areas that didn't even have electricity. Murphy successfully countered that the emphasis of the SOA wouldn't be on providing constant connectivity, but providing services when they are called. A more asynchronous view, if you will.

“We use the same principles of SOA, breaking it up into multilayer data services, business services and so on. But what we’ve also done is make sure that the framework supports disconnected mode,” Murthy said.

The Peace Corps' SOA effort began in 2006, when Murthy’s team was authorized $1 million in funding to start moving applications off the mainframe into a more distributed computing environment based on the .NET framework. The result was three applications that support posts and volunteers in the field: Crime and Incident Reporting, Site Enhancement and Development, and the Volunteer Information Database Application (VIDA).

_____________________________________________________________________

Posted by joemckendrick in Case Study • SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

Financial services companies have had a tough ride over the past year, so anything that can be done to better take advantage of their information technology assets to streamline operations and increase the situational awareness and agility of the business is a good thing. That's where SOA and business process management (BPM) come in.

JPMorgan Chase, of course, is right at the epicenter of the financial storm that has been ravaging the industry, and the company's dominance and stability is due in no small part to its ability to smartly leverage SOA and BPM.

At ebizQ's recent Financial Services Live Panel, Ron Ambuter, CTO of the BPM Workstream Group at JPMorgan Chase, described how his company has been able to adapt and adopt technology and methodologies to help reduce costs, improve customer experiences, and maintain a competitive edge. Ronan Bradley, analyst with Lustratus Research and former CEO of PolarLake, led the discussion, and was joined by Keith Swenson, chief architect at Fujitsu Computer Systems, and Hub Vandervoort, CTO of Progress Software. (Archived recordings and transcript available here.)

Ambuter described how JPMorgan Chase was interested in the concept of reusability, which would help the organization "get better leverage out of the efforts that we were making to build and buy applications by reusing components instead of rebuilding and rebuying them over and over again."

With the troubles the industry has been having around subprime mortgages, writedowns, and tighter credit, it's likely

regulation and oversight will only increase, particularly in regards to risk management. Bradley questioned whether the prevalence of regulation holds back or encourages SOA adoption. Ambuter said regulatory mandates have accelerated his company's drive to SOA, noting that "the concept of SOA allows us to react probably quicker, and more expeditiously, more cost effectively."

The panel also discussed the challenges around extending managing services and infrastructure across multiple groups within organizations. Financial services organizations, of course, typically have multiple lines of business, from securities to brokerages to mortgages. Ambuster explained that organizational issues were his greatest challenge as well in implementing a cohesive SOA and BPM strategy at JPMorgan Chase. "We have rules and responsibilities that are compartmentalized by groups of folks, and we realize that as you go into this stuff, education and cross-pollination of information is critically important. It's not just the technology folks that need to understand this stuff, it's the risk folks, and it's the governance folks and it's the finance folks, and it's the business side folks."

Vandervoort agreed that "it's easy to get bogged down in trying to get alignment on a lot of different points across all the groups." He recommended three approaches to the problem, including "getting your transports aligned between business entities so that you can use eventing-oriented mechanisms to communicate across domains"; establishing SLA and security policies that ensure visibility; and establishing a common enterprise data model.

"You have to get your semantics aligned among the members," he said. "And that doesn't have to be a common vocabulary in its entirety, but certainly what we regard as the data in flight, those things that fly between domains and different working groups have to be highly normative."

Aligning SOA and BPM is also important. "While SOA is basically a technology trend, BPM is essentially a management trend," said Swenson. "A lot of people look at BPM and they get confused between the management trend and the technology trend. A lot of technology product companies have looked at BPM and looked only at the technology and come up with a kind of a programming language that they claim to solve the problem, which is great for the IT side of the house, but it leaves the business side out of this. As far as the adoption of BPM goes, one of the biggest barriers to adoption of the real business process is essentially the fear that the manager will lose control of the process."

Complex event processing is also a key initiative financial services companies need to undertake. As Vandervoort observes, "financial services is moving kind of in a logarithmic increase in velocity... things are ten times faster than they were a decade ago. When you go from three days order to settlement or trade to settlement to now under two hours trade to settlement, if I'm doing nightly reporting, I'm way out of alignment. Whereas ten years ago, that was three times faster than the speed of the pipeline. Today, you the ability to run awry in risk, terms, and exposure terms has been seen multiple times in the industry in recent years."

For companies seeking to increase agility and be more responsive to these highly competitive markets -- not just in financial services -- the combination of SOA and BPM can break down the constrictive silos that cut off essential data and processing resources.

____________________________________________________________________

Posted by joemckendrick in Business Process Management • Data Management • Event Processing • Management • SOA • SOA Events  |  Permalink  | Comments (0)  | TrackBacks (0)

Complex events "can be anything from a sequence of temperature readings to something like the First World War, which was a very complex event indeed." -Dr. David Luckham

David Luckham -- considered the "father of complex event processing" -- recently joined Dr. K. Mani Chandy, Rodney Morrison, and Beth Gold-Bernstein for an informative panel discussion on the relationship between EDA (Event Driven Architecture) and SOA. The panel, part of ebizQ's recent Event Processing Virtual Conference, which brought together the leading thinkers and proponents of EDA and Complex Event Processing (CEP).

The panelists were divided, however, over the extent of the role of human involvement in complex events.

Chandy, for example, said that human engagement is essential at some point in the complex event processing cycle. "I’ve seen quite a few applications, but almost none in which there is no human involved" For example, Chandy said, "in military applications, when there is a gun or device is fired to kill somebody. It’s always done with a human being responsible for their final action."

As Chandy pointed out:

"I think its absolutely critical for human beings to be part of this sense and respond system. It's important how the application supports the human being if you're looking at trading or fraud detection. In all these cases, its really important to have a human being involved. Fraud is one case where you might have an application that informs a credit card user that something inappropriate may be going on, without having a human being first check that."

Luckham, however, pointed out that there are many events are processed independent of human intervention. "There are many examples of event driven architectures where there are absolutely no humans whatever," he said. "The CPU on your computer is an event driven architecture, believe me. And its entirely event driven, clocked, without a human in the loop.

EDA is part of SOA on two different levels, Chandy said:

"One way that I’ve seen EDA used in conjunction with SOA is for service management. Many SOA vendors are exposing metrics that can give you information like end to end process time and activity times. Those metrics can be provided to a CEP system to help control and manage those services. I can, for example, do dynamic provisioning for a service that's getting maxed out."

Chandy also connected the dots between EDA and SOA. "If SOA means loosely coupled subsystems with very clean interfaces, so that new systems could be coupled into the substrate. Then EDA events fit within that framework, because EDA is also based on a loose framework, and is extensible." In a request-reply SOA scenario, "then EDA can still be coupled. There will be layering between the push and the pull parts."

_____________________________________________________________

Posted by joemckendrick in Business Process Management • Event Processing • SOA • SOA Events  |  Permalink  | Comments (0)  | TrackBacks (0)

Behold the zebra in the wild savanna -- when a it senses the presence of a lion, it knows to run away, as fast as it can. When it senses food supplies, it knows to act.

Do today's companies have this much situational awareness, and an ability to act quickly to survive and thrive? Not yet, but thanks to new approaches such as complex event processing, they're getting there, according to Gartner VP Roy Schulte. "We can teach computers to do what a zebra does," Roy said. "To collect and process event data to respond quickly and effectively."

In his keynote kicking off ebizQ's recent Event Processing virtual conference, Schulte broke down the essential pieces of complex event processing, and describing how businesses can leverage CEP, or being able to act, in real time, on multiple streams of event data flowing in from different parts of the enterprise. "The value of complex event processing, overall, can be summarized as improving situation awareness. Simply put, that is just knowing what is going on, so you can figure out what to do." The benefits of complex event processing, Schulte said, include better decision quality, faster response times, reduced information glut, and reduced costs.

Schulte defined a business event as a "meaningful change in a state that is something that is relevant to the business. Examples include depositing or withdrawing money from a bank, submitting a purchase order, or hiring an employee." There is also a second term, "event object," that describes how the event is packaged for processing, typically as an XML document these days. "We have to record events using event objects so computers can receive them and do computations on those events," Schulte said.

However, while all companies have always been event driven -- with millions, if not billions, of events in a single day, most events are still handled manually, by people, not computers. "At any one second, a large company has on its network anywhere from 10,000 to 10 billion business events," Schulte explained. "At the low end, that's almost a billion events per day -- at the high end, that’s almost a trillion events per day."

The challenge is that most of the stovepiped and legacy applications that power enterprises are not yet event driven, Schulte observes.

But there's great practicality in automating the ability to capture and make decisions on multiple event streams coming into the core business systems, Schulte says. "For example, you can have a complex event that says, ‘this mornings sales were 30% above our daily average.' That of course is much easier to digest and act on than sending a person 500 detailed sales records, and making the person compute what happened that day manually."

The growing array of sensors, such as RFID tags, combined with front-end systems such as business activity monitoring (BAM) dashboards make complex event processing a reality with today's technology, Schulte points out.

"In many cases, the complex event processing system Is just a front end being used for decision support. The output of the CEP engine is sent to a person through a BAM dashboard, or through an alert such as email or SMA or an Atom or RSS feed. in this case, we have a two-stage computation. In the first stage we’re using a computer to narrow down the data. And in the second stage, we still have a person involved to do the final analysis.

"An application system, or some other device or some other system, detects the event, and generates a message or a notification that is sent to a person. That notification is the event object or event report sent in the form of a message through message-oriented middleware, RSS, a Web service, or an email, or some other communication mechanism. The response to an event may be a manual activity, done by a person or it may be a SOA service or business process or some other application."

However, things could get interesting as CEP systems develop, Schulte added. Namely, the need for human processing could be taken out of the equation all together. "We can bypass that person entirely; we can build enough smarts into the complex event processing engine to determine the specific response that is needed."

Schulte provided a working example of complex event processing in action within the airline industry:

"In large airlines, there is an event oriented middleware that... acts as an enterprise nervous system. Information from hundreds of sources, including sensors on board the aircraft, information coming in from the FAA, and information coming in from standard systems is sent to the enterprise nervous system, and is temporarily stored in event databases. It helps to create the data, the outgoing alerts and notifications that is sent to hundreds of applications on the consuming side to respond to threat and opportunity situations as they emerge. By having information quickly, each of these systems in their respective departments can respond faster. ...Information helps the fueling and maintenance management applications to change their schedules and so forth. By using an event based system, the turnaround time of each plane can be shortened… Fewer airplanes are needed to handle the same schedule."

______________________________________________________________________

Posted by joemckendrick in Business Process Management • Data Management • Event Processing • Management • SOA • SOA Events • SOA Research and Analyst Reports  |  Permalink  | Comments (0)  | TrackBacks (0)

Oracle's Dave Chappell has always done a great job of linking SOA efforts to measurable business results, and he recently posted an account of a busy online service that was able to integrate its widespread base of customer systems into a well-integrated service layer.

Move Inc., a growing online real estate provider, had what Oracle's Dave deemed a "tall order:" consolidating disparate applications across two business units into an enterprise CRM solution that could provide a single, accurate view of customer data to improve efficiencies and automate auditing, billing and fulfillment processes.

Move Inc. (formerly Homestore, Inc.) provides information on real estate property listings (homes and apartments), moving, home and garden and home finance through its network of online sites, which includes www.realtor.com, www.welcomewagon.com, and www.moving.com.

Tall order indeed. As Dave describes it, Move had systems and touchpoints all over the map. CRM applications were built on .NET, and there were numerous disparate systems and manual touchpoints built to support various business functions for CRM.

The company developed a SOA that focused on re-usability of services, with business processes were at least as efficient if not better than what the current systems offered. Using standard WSDL interfaces, Oracle BPEL PM and Oracle ESB were used to extend the reach and normalize the integration across Siebel CRM, PeopleSoft Enterprise Financial Management and other systems both internal and external

The consolidated customer database is intended to provide support for existing sales activities, order capture, integration with fulfillment systems, integration of customer usage from Web sites into Siebel, and integration to PeopleSoft Billing. The solution involves the use of BPEL, Web services, ESB Mediation, application adapters, and canonical data models.

The project took six months to implement and deploy, Dave reports -- actually, pretty good timing for such an effort. Move was able to achieve both quicker and real-time integration across these systems, and saw a reduction in cost of ownership on existing applications.

"It's always great to hear about successful adoption of SOA, and its particularly exciting when its being used for what you'd think," Dave remarked.

______________________________________________________________________

Posted by joemckendrick in Case Study • Management • SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

Everyone considers their systems to be "special" in many ways. They represent years of planning, installations, and modifications. However, sometimes "specialness" can get in the way of the ability to deliver agility and flexibility to business clients.

At the recent Tibco TUCON users' conference in San Francisco, Allstate's vice president of technical solutions, Anthony Abbatista, described how his company managed to automate most of its systems and processes in the 1960s and 1970s with what was in it's time a new generation of "special applications." Everything from policy administration to underwriting was captured on the company's array of mainframe computers.

Of course, application designers in these first waves of automation could never have imagined the uses that their systems would be getting nowadays. They never could have envisioned parts of these systems being opened up to the world via the Web.

These systems, Abbatista explained, were all hand-crafted and hand-coded. Commercial off-the-shelf applications just didn't exist in the 1960s and 1970s. However, the Allstate commitment to rolling its own applications continued right up to recent years. At Allstate everything was custom-built, "and we were proud of that," he said. "everything was very 'special,' with limited reuse."

Adding to the complexity and inflexibility, the company had a plethora of systems and vendors for the commercial solutions it did start buying. "Up until five years ago, we never met a software vendor we didn't like," Abbatista said.

The company, which now has $156 billion in assets and 17 million customers, always had plenty of resources for information technology. However, the company needed more responsive and agile IT solutions to succeed in an increasingly competitive environment for the insurance industry.

"When I first arrived at Allstate, we had three to five different integration platforms," Abbatista related. "We had an executive IT board that mirrored the military model, run by 'generals.' We had an abundance of point-to-point integration."

Allstate knew that it needed to embark on a more consolidated, service oriented architecture if its systems were to continue to deliver value.

Areas being modernized include claims processing, which was spread across nine different mainframe-based silos tied together by custom-built middleware. The company employed a standard enterprise service bus to handle claims processing through a single interface across the entire enterprise.

Another initiative was to establish a common integration layer for the company's fast-growing data environment. "Our data hubs are growing 30%-40% a year," Abbastista said. The company also pared down its vendor list to a "handful of strategic partners," numbering 12 to 15 vendors, he said.

Changing the organizational culture was another challenge, Abbatista remarked. "We had to make a substantial commitment to technology, skills, and also selling integrated teams that we had a better approach," he said. SOA governance was also tied to the operations of the procurement organization. This helped provide better visibility of available services, as well as increase reuse, he said. "How many error-logging services did we really need?" he asked.

______________________________________________________________________

Posted by joemckendrick in Case Study • Management • SOA • SOA Events  |  Permalink  | Comments (0)  | TrackBacks (0)