SOA in Action

We've talked quite a bit here and throughout the ebizQ-sphere about the convergence between SOA and enterprise data management, and how services can be employed to source and leverage data for various analytics.

In a new interview with Michael Meehan, Kirstan Vandersluis, founder and chief scientist for XAware Inc., directs the discussion to a new type of service emerging -- the "data service mashup." As the name suggests, he is talking about employing lightweight Web 2.0 methodologies to build front-end apps that pull needed data into a dynamic presentation for the business end user.

But it's a lot more than the on-the-fly Google Maps-style mashups we've been seeing in recent times. As Vandersluis describes it, a data services mashup is "a mashup in terms of pulling data from multiple sources into a logical unit. So it's a mashup in terms of data. It's not a visual mashup. That's a whole different ballgame. We're pulling data from virtually anywhere into a logical unit. We're trying to take a bunch of back-end data systems, which are typically very complex, and make it look like something much more rationalized in terms of an XML Schema that somebody put some thought into designing."

The model Vandersluis is proposing is built around an XML schema. He notes that "the applications are relying on the contract, which is defined by the XML schema."

Data services mashups may work in conjunction with, or even serve as an alternative to data warehouses and data marts in some situations, Vandersluis also said. "It's just a different way to do things and there are trade offs.... The one benefit to our approach is it's all real-time. We're sending the sub-queries out in real-time to whatever systems you want."

So if initially you pointed all your data services at your data warehouse, and you find out later that's not real-time enough, you can change the data sets that are being mashed up, so one of them hits an operational data source.

Vandersluis and other data service mashup proponents have their work cut out for them. Understanding and adoption at the business level has been "spotty" to date, Vandersluis observes. "I think it's more a matter of how well they understand service-oriented architecture, which is the typical use case of a data services layer."

Organizations that are embracing SOA are more likely to be early adopters of data services mashups, he says. In addition, there is impetus at sites developing rich Internet applications. Such Web 2.0-ish are more likely to see a large volume of services, versus the more centralized and business-focused services that will be part of SOA.

____________________________________________________________________

Posted by joemckendrick in Data Management • Enterprise 2.0 • SOA  |  Permalink  | Comments (2)  | TrackBacks (0)

For some time now, I've heard consultants and analysts propose launching "centers of excellence" or "competency centers" to better manage newer IT initiatives such as SOA is one that has been proposed by consultants for some time now. Not only does such a body help kick-start SOA efforts in the business, but it's also a way to keep such efforts above organizational politics and fiefdoms. The SOA effort can proceed without being encumbered by individual department or management agendas.

Excellent idea. However, such efforts have been slow to evolve, and may be difficult for smaller to medium-size businesses to get in place. ebizQ's recent survey on SOA governance, conducted in partnership with SAP, found that only nine percent of respondents had established some form of competency center/center of excellence to promote and keep SOA efforts on the right track. Almost all of these were larger companies with revenues exceeding $1 billion or more annually. (Our Webinar on the survey results can be viewed here.)

While getting management to commit time and resources to an SOA center in and of itself may be an uphill battle, there may be value in extending the reach of such centers to enterprise data integration as well, according to John Schmidt, VP of Informatica and Chair of the Integration Consortium. This center may be described as an "Integration Competency Center," or ICC. In a chat with Beth Gold-Bernstein, Schmidt provides some guidelines to help businesses of all sizes implement an Integration Competency Center (ICC). He describes the elements that go into an ICC:

Financial management: "The ICC operates as a shared service. This is a set of best practices around charge back for shared infrastructure and individual services."

Architecture: "The ICC does not do enterprise architecture, but is responsible for the information architecture. They work with the enterprise architecture group, and 'connects the dots,' by mapping schemas to physical data sources to enable the translation, transformation, and integration. This ICC is the central federated repository."

Business Process Management. "This is not BPM per se, but this includes service flow modeling, information flows, business event modeling, and common definition of business events."

Integration methodology: "The process of running an ICC, defining it, organizing it, all the things you need to run an integration group, and how it will interact with other IT groups."

Metadata management: "The ICC group is responsible for data assets. Metadata ends up being a federated model."

Modeling management: "This includes techniques around canonical data modeling, what are the best practices, how do you build them."

Integration Systems. "This is about running integration systems as a specific class of applications – all the discipline of how your manage, plan and operate the system."

_____________________________________________________________________

Posted by joemckendrick in Data Management • Management • SOA • SOA Vendors  |  Permalink  | Comments (0)  | TrackBacks (0)

Financial services companies have had a tough ride over the past year, so anything that can be done to better take advantage of their information technology assets to streamline operations and increase the situational awareness and agility of the business is a good thing. That's where SOA and business process management (BPM) come in.

JPMorgan Chase, of course, is right at the epicenter of the financial storm that has been ravaging the industry, and the company's dominance and stability is due in no small part to its ability to smartly leverage SOA and BPM.

At ebizQ's recent Financial Services Live Panel, Ron Ambuter, CTO of the BPM Workstream Group at JPMorgan Chase, described how his company has been able to adapt and adopt technology and methodologies to help reduce costs, improve customer experiences, and maintain a competitive edge. Ronan Bradley, analyst with Lustratus Research and former CEO of PolarLake, led the discussion, and was joined by Keith Swenson, chief architect at Fujitsu Computer Systems, and Hub Vandervoort, CTO of Progress Software. (Archived recordings and transcript available here.)

Ambuter described how JPMorgan Chase was interested in the concept of reusability, which would help the organization "get better leverage out of the efforts that we were making to build and buy applications by reusing components instead of rebuilding and rebuying them over and over again."

With the troubles the industry has been having around subprime mortgages, writedowns, and tighter credit, it's likely

regulation and oversight will only increase, particularly in regards to risk management. Bradley questioned whether the prevalence of regulation holds back or encourages SOA adoption. Ambuter said regulatory mandates have accelerated his company's drive to SOA, noting that "the concept of SOA allows us to react probably quicker, and more expeditiously, more cost effectively."

The panel also discussed the challenges around extending managing services and infrastructure across multiple groups within organizations. Financial services organizations, of course, typically have multiple lines of business, from securities to brokerages to mortgages. Ambuster explained that organizational issues were his greatest challenge as well in implementing a cohesive SOA and BPM strategy at JPMorgan Chase. "We have rules and responsibilities that are compartmentalized by groups of folks, and we realize that as you go into this stuff, education and cross-pollination of information is critically important. It's not just the technology folks that need to understand this stuff, it's the risk folks, and it's the governance folks and it's the finance folks, and it's the business side folks."

Vandervoort agreed that "it's easy to get bogged down in trying to get alignment on a lot of different points across all the groups." He recommended three approaches to the problem, including "getting your transports aligned between business entities so that you can use eventing-oriented mechanisms to communicate across domains"; establishing SLA and security policies that ensure visibility; and establishing a common enterprise data model.

"You have to get your semantics aligned among the members," he said. "And that doesn't have to be a common vocabulary in its entirety, but certainly what we regard as the data in flight, those things that fly between domains and different working groups have to be highly normative."

Aligning SOA and BPM is also important. "While SOA is basically a technology trend, BPM is essentially a management trend," said Swenson. "A lot of people look at BPM and they get confused between the management trend and the technology trend. A lot of technology product companies have looked at BPM and looked only at the technology and come up with a kind of a programming language that they claim to solve the problem, which is great for the IT side of the house, but it leaves the business side out of this. As far as the adoption of BPM goes, one of the biggest barriers to adoption of the real business process is essentially the fear that the manager will lose control of the process."

Complex event processing is also a key initiative financial services companies need to undertake. As Vandervoort observes, "financial services is moving kind of in a logarithmic increase in velocity... things are ten times faster than they were a decade ago. When you go from three days order to settlement or trade to settlement to now under two hours trade to settlement, if I'm doing nightly reporting, I'm way out of alignment. Whereas ten years ago, that was three times faster than the speed of the pipeline. Today, you the ability to run awry in risk, terms, and exposure terms has been seen multiple times in the industry in recent years."

For companies seeking to increase agility and be more responsive to these highly competitive markets -- not just in financial services -- the combination of SOA and BPM can break down the constrictive silos that cut off essential data and processing resources.

____________________________________________________________________

Posted by joemckendrick in Business Process Management • Data Management • Event Processing • Management • SOA • SOA Events  |  Permalink  | Comments (0)  | TrackBacks (0)

Behold the zebra in the wild savanna -- when a it senses the presence of a lion, it knows to run away, as fast as it can. When it senses food supplies, it knows to act.

Do today's companies have this much situational awareness, and an ability to act quickly to survive and thrive? Not yet, but thanks to new approaches such as complex event processing, they're getting there, according to Gartner VP Roy Schulte. "We can teach computers to do what a zebra does," Roy said. "To collect and process event data to respond quickly and effectively."

In his keynote kicking off ebizQ's recent Event Processing virtual conference, Schulte broke down the essential pieces of complex event processing, and describing how businesses can leverage CEP, or being able to act, in real time, on multiple streams of event data flowing in from different parts of the enterprise. "The value of complex event processing, overall, can be summarized as improving situation awareness. Simply put, that is just knowing what is going on, so you can figure out what to do." The benefits of complex event processing, Schulte said, include better decision quality, faster response times, reduced information glut, and reduced costs.

Schulte defined a business event as a "meaningful change in a state that is something that is relevant to the business. Examples include depositing or withdrawing money from a bank, submitting a purchase order, or hiring an employee." There is also a second term, "event object," that describes how the event is packaged for processing, typically as an XML document these days. "We have to record events using event objects so computers can receive them and do computations on those events," Schulte said.

However, while all companies have always been event driven -- with millions, if not billions, of events in a single day, most events are still handled manually, by people, not computers. "At any one second, a large company has on its network anywhere from 10,000 to 10 billion business events," Schulte explained. "At the low end, that's almost a billion events per day -- at the high end, that’s almost a trillion events per day."

The challenge is that most of the stovepiped and legacy applications that power enterprises are not yet event driven, Schulte observes.

But there's great practicality in automating the ability to capture and make decisions on multiple event streams coming into the core business systems, Schulte says. "For example, you can have a complex event that says, ‘this mornings sales were 30% above our daily average.' That of course is much easier to digest and act on than sending a person 500 detailed sales records, and making the person compute what happened that day manually."

The growing array of sensors, such as RFID tags, combined with front-end systems such as business activity monitoring (BAM) dashboards make complex event processing a reality with today's technology, Schulte points out.

"In many cases, the complex event processing system Is just a front end being used for decision support. The output of the CEP engine is sent to a person through a BAM dashboard, or through an alert such as email or SMA or an Atom or RSS feed. in this case, we have a two-stage computation. In the first stage we’re using a computer to narrow down the data. And in the second stage, we still have a person involved to do the final analysis.

"An application system, or some other device or some other system, detects the event, and generates a message or a notification that is sent to a person. That notification is the event object or event report sent in the form of a message through message-oriented middleware, RSS, a Web service, or an email, or some other communication mechanism. The response to an event may be a manual activity, done by a person or it may be a SOA service or business process or some other application."

However, things could get interesting as CEP systems develop, Schulte added. Namely, the need for human processing could be taken out of the equation all together. "We can bypass that person entirely; we can build enough smarts into the complex event processing engine to determine the specific response that is needed."

Schulte provided a working example of complex event processing in action within the airline industry:

"In large airlines, there is an event oriented middleware that... acts as an enterprise nervous system. Information from hundreds of sources, including sensors on board the aircraft, information coming in from the FAA, and information coming in from standard systems is sent to the enterprise nervous system, and is temporarily stored in event databases. It helps to create the data, the outgoing alerts and notifications that is sent to hundreds of applications on the consuming side to respond to threat and opportunity situations as they emerge. By having information quickly, each of these systems in their respective departments can respond faster. ...Information helps the fueling and maintenance management applications to change their schedules and so forth. By using an event based system, the turnaround time of each plane can be shortened… Fewer airplanes are needed to handle the same schedule."

______________________________________________________________________

Posted by joemckendrick in Business Process Management • Data Management • Event Processing • Management • SOA • SOA Events • SOA Research and Analyst Reports  |  Permalink  | Comments (0)  | TrackBacks (0)

"The simultaneous growth of investment in BI and SOA technologies is no coincidence," says Gilman Tobin, senior director of BI product marketing for Oracle. Both are top strategic focus areas, and along with BPM, form the foundation for a new application platform comprised of the “B3” categories (BI, BPM, and business rules).

Tobin, in a piece just posted here at ebizQ, effectively connects the dots between business intelligence and SOA.

For example, he points out, "being able to analyze the performance impact of business processes executing in a business process platform is a key objective for organizations developing SOAs." Data is captured and aggregated in dashboards made available to decision makers, made possible by SOA.

As Tobin puts it:

"Orchestrating the delivery of a cell phone to a customer may generate data such as time spent provisioning the cell phone, updating the appropriate accounts and delivering the product to the customer. The BI tool can combine this data with data stored from other systems, such as usage data stored in another relational database by a billing system and business intelligence may be used to present an aggregate view. By combining call usage data with process cycle time data captured from business intelligence tools, organizations can gain insight into the impact of process delays, such as the amount of lost revenue that can be attributed to delays in the process."

The combination of SOA and BI provides greater insight and visibility across business processes. And visibility paves the way to new innovation.

______________________________________________________________________

Posted by joemckendrick in Data Management • SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

For years, the disciplines of enterprise integration -- followed by SOA -- have always been perceived as "hard and techie." This has made discussions with the business process management (BPM) side of the house difficult.

Now, thanks to the introduction of new lightweight approaches via Web 2.0, SOA, BPM, and integration are now becoming more flexible, user-driven methodologies.

David Linthicum recently spoke with Dr. Dale Skeen, founder of Vitria, a well-established integration vendor, about this shift. (The podcast is here, and a transcript of Dave's discussion with Dale Skeen is available for viewing here.)

This is part of SOA moving into its next generation -- "the next great leap going together is really leveraging three very important technologies in the enterprise -- SOA, BPM and Web 2.0," Dale said. While these three areas are seen as separate technologies or methodologies, Dales sees their inevitable convergence into what he calls an "enterprise platform."

First of all, Dale said, SOA and BPM have always been a natural pairing. "SOA is an enabler that allows you to access business functions, and services, and data universally. BPM is a higher level that orchestrates these business services and human interactions in ways that allow you to meet a business objective. So hence, I've always considered these to be the perfect complementary technologies to work together."

Now, Web 2.0 is bringing SOA-BPM closer to the end user, Dale said. "SOA brings this universal access to services and data through the SOA enablement tools. It does in a secure, manageable, and governed fashion. Now, Web 2.0 brings rich internet interfaces, rich user experiences based on technology such as AJAX and Flex, which are universally available in your Web browser."

"Application integration is hard, and very techie. Web 2.0 allows this notion of mashups where you let users sort of integrate in a flexible, lightweight, easy-to-do fashion."

These are all new trends that are shifting IT empowerment to the business end user, Dale added. "Simple things like mashups, we're talking visual layering of information, we're talking about collaboration technologies such as instant messaging. All of these have a role in enterprise software. And actually, the introduction of that can be very exciting for both the IT and the business side."

IT professionals need not fret over this shift, however, Dale said. "It means that IT will be able to do faster technology upgrades because of that, they have more control over the server aspect of it, and the client side they don't have to worry about. They're going to be able to lower the support costs and it also brings fundamentally new deployment models such as Software-as-a-service, which we think, is going to be a fundamental part of business IT going forward."

_____________________________________________________________________

Posted by joemckendrick in Business Process Management • Data Management • Enterprise 2.0 • Management • SOA • SOA Podcasts • SOA Vendors  |  Permalink  | Comments (0)  | TrackBacks (0)

Data, you may not realize it sometimes, but we do it all for you. Honest.

In a new posting here at ebizQ, Oracle's Jeff Pollack says the time has come to bring business intelligence, data warehousing, and master data management (MDM) into the SOA house. Or bring SOA into their houses.

Jeff, an executive on the Fusion side of Oracle, and author of the enterprise software book "Adaptive Information“ (John Wiley & Sons), points out that vendors, analysts and enterprises alike often get caught up in the software side of things for too long. "In the past 15 years, with the rise of Java, the hype surrounding EII, EAI and SOA, and the rise of XML, and quietly, the billions spent in ETL projects -- it's too easy to forget why we build and buy all that infrastructure. We do it for the data."

The problem is, many SOA projects may not be ready to handle all the data corporations are churning out these days. Most, if not all, operate within a Java Virtual Machine container and can't effectively handle large data loads. Plus, XML can be a cumbersome markup language for data.

Thus, data managers keep relying on their relational databases to do what needs to be done. As Jeff puts it: "Most new XML-centric solutions for data can't scale to the mult-terabyte sized problem that is typical of a Global 2000 business. Thus, a knowledgeable architect will revert to the proven patterns of RDBMS as the backbone of a data architecture using MQ, TPS, and ETL interfaces as the pipes for pushing all that data around."

That's the challenge going forward, Jeff says. SOA still suffers from "poor bulk data transformation performance and inefficiency."

To address this problem, Jeff recommends the creation of "SOA Data Services," which are decoupled endpoints that expose components for working on all types of data. Interestingly, he points out, such services can be created without SOA, and such services have been around for years now, in EDI and Java applications.

Start simple, Jeff advises. A Data Service could simply be a "boring Web Service with simple data actions, or thin SOA façades for wrapping conventional data technology." And remember, it isn;t just about XML. Data Services created these days should encompass different data delivery patterns and formats, Jeff says, noting that "SOA pundits often assume that XML is the only desirable delivery format, but for a data solution to be truly useful for the enterprise, it must support several different delivery styles."

Posted by joemckendrick in Data Management • SOA  |  Permalink  | Comments (2)  | TrackBacks (0)

More evidence that SOA implementations are moving closer to an enterprise data management role: The Stanwell Corporation recently began a project intended to integrate and provide quick access to data currently stored in siloed IT systems.

In a new ComputerWorld interview, Stanwell enterprise technology architect Greg Behrendt said the new platform will synchronize data from numerous sources throughout the enterprise and transform it into real-time information. "We will have access to information we can act upon and update straight away; this will save us time and money by streamlining business processes and accessing information from one location," he is quoted as saying. Stanwell turned to BEA's AquaLogic Enterprise Service Bus and data services platform to undertake the project.

Perhaps the Stanwell and BEA folks read ebizQ's own David Kelly a few months back, when he so succinctly made the SOA-data connection a few months back, observing that "any organization considering or pursuing SOA should also be creating a data services strategy....Simply put, putting together the right data services are an important strategic investment for organizations pursuing SOA."

While data warehouses and data marts are intended to accomplish this de-silozation of information, they need middleware support to effectively disperse the data to the right parts of the business through the right applications. "There's an ever-growing need for delivering data, especially from multiple, disjointed sources, in real-time," David explained. Thus, data needs to be moved closer to the applications, and applications need to be moved closer to the data, and somewhere in the middle they meet. What a killer role for SOA.

Posted by joemckendrick in Case Study • Data Management • SOA  |  Permalink  | Comments (2)  | TrackBacks (0)

Oracle's Dave Chappell writes about a new paradigm sweeping the financial services world, extreme transaction processing (XTP). The good news that used in conjunction with SOA, XTP offers power to applications such as fraud detection, trade resolution, and risk management calculations, running off of low-cost commodity hardware, versus the premium servers required for these applications in days gone by.

"We're seeing that SOA coupled with XTP (eXtreme Transaction Processing) is the future for financial services infrastructure," Dave says. XTP enables certain applications to process large amounts of data/

What is the connection between XTP and Complex Event Processing (CEP)? They are both "positioned to solve some of the same thorny problems of tracking business exceptions such as what is required for fraud detection," Dave says. XTP is tied to CEP "in that they are both about consuming and correlating large amounts of event data and doing something meaningful with it." However, traditional systems relying on CEP may not have the storage management capacity needed to handle large amounts of data.

"Tie-ins to SOA are many, but the most obvious one is that once the exceptions are detected, there are actions to be taken," he explains. "Typical actions to be taken are to send alerts to a BAM dashboard and to invoke a business process or human workflow, such as what can be defined and executed using the Oracle BPEL process manager."

Posted by joemckendrick in Data Management • SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

You ever hear about the "rent-a-patient" scheme hatched by a pair of unscrupulous doctors in the LA area? In this incredible-but-true scheme, the doctors performed unnecessary surgery on patients that were recruited by marketers with promises of cash or discounted cosmetic surgery procedures.

Patients were instructed by recruiters to describe false and exaggerated symptoms that were used to create medical charts used to make the surgical procedures appear to be justified. The doctors reportedly racked up claims totaling more than $2 million before being caught. Procedures performed on the otherwise normally healthy patients included colonoscopy, sinus surgeries and thoracic sympathectomy, commonly called "sweaty palm surgery."

Claims fraud gives many insurance carriers sweaty palms, as I documented in this recent article in Insurance Networking News. The good news is that technology -- in the form of analytics -- is providing carriers the tools to rapidly sift through claims data to connect the dots on suspicious activities. Time is of the essence -- claims must be paid within a certain period of time, and once the check is cut and mailed, it's difficult to get the money back.

Here's a role SOA can play as well. Sandy Carter, VP with IBM, just published details in CIO about how SOA is enabling MIB, the industry's largest fraud detection service that is used by nearly every North American life and health insurance carrier, to spot fraudulent activity.

Sandy relates that MIB, which has 500 members, can sift through files from a range of systems to help detect potential fraud among new applicants to insurance policies. MIB's systems can access the applicant's insurance and medical history; the proximity of marriage, establishment of a life insurance policy and a spouse's untimely death, and the frequency of auto accidents resulting in medical claims.

MIB is also employing SOA to "help build an online community among credit reporting agencies, healthcare and insurance providers, and government agencies by extracting the information that’s trapped in various software applications that are located inside and outside a company and bringing it together under one roof in a faster, more secure and economical way. This information helps create an accurate and comprehensive credit history profile while also adhering to industry regulations."

Posted by joemckendrick in Business Process Management • Case Study • Data Management • SOA • SOA Vendors  |  Permalink  | Comments (0)  | TrackBacks (0)

At this week's Open Group Enterprise Architecture conference conference, I had the opportunity to join a vigorous panel discussion in which we performed a "reality check" on the state of SOA. We were joined by moderator Eric Knorr of InfoWorld, along with Tony Baer (onStrategies), David Linthicum, Tom Morgan (AutoDesk) and Chris Harding (Open Group).

Members of the panel examined the matter of SOA efforts getting stuck, or whether a "trough of disillusionment" had set in. Our panel also explored SOA's uncertain relationship with other disciplines. It was agreed, for example, that business process management is a key element of SOA and visa-versa. However, we also agreed that there is still a wide chasm between the two. Tom Morgan observed that while his organization, AutoDesk, has had a far-reaching SOA effort underway for a number of years now, they "punted" on applying BPM to the effort.

The panel also discussed the issues that linger around governance, especially in the areas of registry and repository. Tony Baer observed that there were still too many issues at the metadata level, and vendors keep pushing registry/repository as a panacea for governance.

Mashups and Web 2.0 are another area colliding with SOA. Dave Linthicum stated that mashups are, indeed, a viable part of SOA taking place within organizations. However, Eric Knorr said he has heard many organizations are not keen on letting data in or out of the corporate firewall.

SOA can work hand in hand with data management, and Tom pointed out that AutoDesk has been very effectively employing SOA-based services to pull, rationalize, and cleanse customer support data from across the enterprise.

Prior to the panel, David Linthicum delivered the keynote for the Open Group event, issuing a warning that all too often, we're still engaging in SOA for SOA's sake. SOA proponents need to redouble efforts to emphasize the business case, and de-emphasize the technology aspect, he said. "It all boils down to architecture." Still, he said, the reuse inherent in SOA can be made to work, and projects where CEOs can see demonstrable value -- such as a real-time analytics dashboard -- can achieve quick success.

Posted by joemckendrick in Business Process Management • Data Management • Enterprise 2.0 • Management • SOA • SOA Events  |  Permalink  | Comments (0)  | TrackBacks (0)

How big will complex event processing (CEP) get?

Long seen as a technical backwater, some industry observers say CEP is going to be huge, that it's the next big thing at the intersection of SOA, business process management, and data management. In a new post, Don DeLoach, president and CEO of Aleri, said that "we are on the dawn of this explosion, and I feel it is surely coming."

What's driving the market? DeLoach thinks "the CEP market is going to be huge because it is being driven by a few unmistakable elements and irreversible trends. The volume of data that organizations need to deal with is growing and the time frames in which they need to act are shrinking. CEP is specifically designed to absorb large amounts of data in real-time and analyze that data on the fly."

DeLoach compares the current era of CEP to relational databases and client/server in the 1980s, when the mainstream market wasn't taking these approaches seriously.

Use-cases which cry out for CEP include market data absorption and enrichment, algorithmic trading, smart order routing, market making and pricing engines, market liquidity discovery, market surveillance, transaction cost analysis, regulatory and compliance requirements, and especially real-time risk management -- any situation where decision-makers need to know when things are happening on a real-time basis.

Indeed, complex event processing (CEP) has been getting a lot of attention as of late, and is seen as the next stage of evolution for SOA projects. Now it has the big vendors -- including IBM -- promoting it.

IBM WebSphere CTO Jerry Cuomo recently said that he sees the SOA-EDA-CEP marriage as one of the most pronounced trends to watch in 2008. As Cuomo explained in an interview: “I really believe [event processing is] the next big thing in SOA…. we’re taking it very seriously.”

IBM even wants to take the technical allusions out of the term itself, referring to it as "Business Event Processing." The renaming to BEP makes sense, since I doubt if a concept that starts with the word "complex" is going to win a lot of converts.

Last week, Big Blue put its money where its mouth is, announcing it would be buying AptSoft, a CEP tools and platform vendor. Sandy Carter, senior vice president of IBM's WebSphere division, said that what they call "business event processing" should be at the disposal of, well, the business. "Everybody is talking about complex event processing. We are trying to rename that category, because we believe the real value is in Business Event Processing, the focus on the business. We believe we are elevating for customers something that was a deep technology capability, something that only engineers understand."

With its purchase of AptSoft and plans to include CEP (er, BEP) functionality into its WebSphere platform, IBM obviously sees a robust market opportunity emerging. Carter said IBM already has invested a billion dollars in the events processing space, not counting its AptSoft purchase.

Posted by joemckendrick in Business Process Management • Data Management • SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

Increasingly, industry analysts and practitioners are connecting the dots between three emerging megatrends -- SOA, Web 2.0, and Business Intelligence. It's becoming increasingly clear that the combination of SOA and Web 2.0 technologies is changing the face of business intelligence.

These were the topics raised at a recent ebizQ Webinar -- which I had the opportunity to moderate -- with noted author Don Tapscott, Katrina Coyle, BI manager with Molson Canada, and SAP's Lothar Schubert. (Audio replay available here.)

Don Tapscott, who broke new ground in 1996 with his book, The Digital Economy: The Promise and Peril of Network Intelligence, and recently co-authored Wikinomics: How Mass Collaboration Changes Everything, said BI is on the verge of a revolutionary transition, thanks to SOA and Web 2.0 approaches to technology innovation.

"Web 2.0 and service oriented architecture are really becoming a new mode of production," he says. "They’re changing the ways that we innovate, the ways theat we make decisions, the ways that we collaborate, and the ways that companies engage with the rest of the world."

This has very profound implications for business intelligence, which is evolving into "collaborative intelligence." Don continued. "Prior to the introduction of Web 2.0 methodologies, he explained, internal data had "been accessible in various limited ways through traditional ERP reporting systems, MIS and business intelligence."

"The marriage of this new accessible data with the firm’s traditional internal data creates an unprecedented challenge, as well as an opportunity to gain insight into the behavior of the company’s most important stakeholders, and to translate that knowledge into success in the marketplace."

The speed of Web 2.0 processes is also changing what end-users expect from BI approaches as well. "Think about if you do a Google search, you get the results back instantly. If the results took half a minute, or five minutes, or 10 minutes, you’d probably stop using Google so much. Traditional BI was kind of like that -- which is part of why we didn’t use it so much Because you’re calling out to a disk, basically." In-memory technologies are also making new generation BI technologies lightning fast as well.

Molson's Katrina Coyle also credits SOA with reshaping her company's ability to compete in a fast-changing and often fickle market. "One of the terrific things that we’ve had in the last year is service oriented architecture," she explained. "We can now deliver information to our business in any way they want…. we can drive information through emails, text, BlackBerries, and widgets. If we have issues anywhere in the supply chain, we can get that information out in real time to supply chain managers."

As with many SOA endeavors, corporate culture presents some barriers and challenges. "We’ve been brewing beer for a very long time," Katrina noted, stating that "when you’ve been doing something as long as we have, you get a lot of habits that are pretty well ingrained. Trying to shaking the business out of those habits is a challenge."

Molson's strategy to transform its organization includes reaching out to a new generation of younger adults through Web 2.0-based marketing strategies, and leveraging service-oriented architecture and data warehouse approaches to build its brands across the globe.

An important emphasis is real-time analytics, Katrina said. "We are constantly having to shift and change and shift and adjust very quickly to changes in the marketplace, " she explained. "We all have to be extremely agile. You can’t spend a week trying to figure out whether the promotion is successful. You have to be able to react within hours."

It used to be that companies didn't know if a promotion was successful until then end of a quarter, if then. Real-time analytics can look at patterns and trends and provide insights if something is working or not, enabling a quick change in direction.

For example, one trend that Molson was able to jump on fairly rapidly was a sudden craze for cold beer in UK pubs -- long bastions of warm beer. Katrina explained that each of its global sites have their own go-to-market models, but all this information needs to be rapidly assimilated. "We have a data warehouse, with lots of information coming in different ways," she said. "It's not necessarily all coming from a centralized ERP system. We also have data coming in from AC Nielsen, for example. "We’ve got to bring that data in, and make sure that it has a harmonized look, so our business can actually make tactical decisions on it."

By adopting in-memory technology available through SAP's BI Accelerator, Katrina reports that Molson has been able to move data quickly through its data warehouse. "We’re able to process data now in real time in our warehouse -- we’re not tied to a load once a day or once a week."

Audio replay of the ebizQ Webcast, "The New Paradigm for Business Intelligence - Collaborative, User Centric, Process Embedded," is available here.

Posted by joemckendrick in Data Management • Enterprise 2.0 • SOA • SOA Podcasts  |  Permalink  | Comments (0)  | TrackBacks (0)

Let's face it -- say what you want about the pharmaceutical industry -- researching and creating new drugs costs a bundle of money, sometimes billions of dollars.

If you're running a company that has to put up megadollars before a new product can be rolled out, there's a lot of pressure to make the right decisions up front. You need all the information you can get, fast.

Such is the challenge described in a new article by Daniel Eng, manager at Pfizer Global Research and Development. "At Pfizer Global R&D where the company's drug discovery takes place, research scientists and managers require vast amounts of up-to-the-minute information on lab results, submission status, and project schedules to move new research forward quickly. Management must constantly analyze the entire portfolio of new medicines in discovery to look for opportunities, trends, and areas where attention is needed. Researchers and managers strive to bring together the best in ideas, practices, policies as well as the use of information."

Eng says the company turned to a SOA-based strategy to bring data together for analysis in the drug discovery process. The end result was faster delivery of data through "new real-time portals and composite applications that rely heavily on existing data sourced from multiple systems from across the enterprise."

Pfizer appears to be well ahead of the curve in bringing the two disciplines of SOA and data management together. In November, I posted some details about Pfizer's leveraging of master data management (MDM) in combination with SOA to bring together data assets from across its global enterprise into a single, centralized data definition. (Explained by Martin Brodbeck, executive director for strategic architecture at Pfizer at the InfoWorld Executive Forum.)

Eng says that prior to service-orienting this process, delivering such data to researchers and managers "has been one of our biggest bottlenecks, adding months and cost to our project timelines." As part of the new drug discovery process, Eng relates, "managers must pull data from sources such as packaged applications, historical data from data warehouses, document repositories, and custom systems. Each source has its own access mechanisms, syntax, and security. Few are structured properly for consumption, let alone reuse. These combined factors slow down new application development projects."

Eng's team was challenged with helping senior management, project team leaders, business analysts, and research scientists across Pfizer's R&D and commercial business units to be able to continuously evaluate the company's portfolio of discovery projects and drugs in development.

The solution was to deploy a portal that can be used across the enterprise that provides access to all pertinent data. Pfizer employs SOA approaches behind the portal environment "to increase reuse of existing components, save development time, and cut costs. So, we strive to use SOA methods and technologies whenever possible," Eng explains.

"We've found that SOA helps break down silo-type data gathering and integration processes by standardizing how data is promoted and reused," Eng relates. The ability to virtualize and abstract via data services helps groups to easily understand and consume data confidently, reliably, and quickly without having to hunt for these sources or rely on manual processes for gathering and integrating them."

Posted by joemckendrick in Data Management  |  Permalink  | Comments (0)  | TrackBacks (0)