SOA in Action

At this week's Open Group Enterprise Architecture conference conference, I had the opportunity to join a vigorous panel discussion in which we performed a "reality check" on the state of SOA. We were joined by moderator Eric Knorr of InfoWorld, along with Tony Baer (onStrategies), David Linthicum, Tom Morgan (AutoDesk) and Chris Harding (Open Group).

Members of the panel examined the matter of SOA efforts getting stuck, or whether a "trough of disillusionment" had set in. Our panel also explored SOA's uncertain relationship with other disciplines. It was agreed, for example, that business process management is a key element of SOA and visa-versa. However, we also agreed that there is still a wide chasm between the two. Tom Morgan observed that while his organization, AutoDesk, has had a far-reaching SOA effort underway for a number of years now, they "punted" on applying BPM to the effort.

The panel also discussed the issues that linger around governance, especially in the areas of registry and repository. Tony Baer observed that there were still too many issues at the metadata level, and vendors keep pushing registry/repository as a panacea for governance.

Mashups and Web 2.0 are another area colliding with SOA. Dave Linthicum stated that mashups are, indeed, a viable part of SOA taking place within organizations. However, Eric Knorr said he has heard many organizations are not keen on letting data in or out of the corporate firewall.

SOA can work hand in hand with data management, and Tom pointed out that AutoDesk has been very effectively employing SOA-based services to pull, rationalize, and cleanse customer support data from across the enterprise.

Prior to the panel, David Linthicum delivered the keynote for the Open Group event, issuing a warning that all too often, we're still engaging in SOA for SOA's sake. SOA proponents need to redouble efforts to emphasize the business case, and de-emphasize the technology aspect, he said. "It all boils down to architecture." Still, he said, the reuse inherent in SOA can be made to work, and projects where CEOs can see demonstrable value -- such as a real-time analytics dashboard -- can achieve quick success.

Posted by joemckendrick in Business Process Management • Data Management • Enterprise 2.0 • Management • SOA • SOA Events  |  Permalink  | Comments (0)  | TrackBacks (0)

How big will complex event processing (CEP) get?

Long seen as a technical backwater, some industry observers say CEP is going to be huge, that it's the next big thing at the intersection of SOA, business process management, and data management. In a new post, Don DeLoach, president and CEO of Aleri, said that "we are on the dawn of this explosion, and I feel it is surely coming."

What's driving the market? DeLoach thinks "the CEP market is going to be huge because it is being driven by a few unmistakable elements and irreversible trends. The volume of data that organizations need to deal with is growing and the time frames in which they need to act are shrinking. CEP is specifically designed to absorb large amounts of data in real-time and analyze that data on the fly."

DeLoach compares the current era of CEP to relational databases and client/server in the 1980s, when the mainstream market wasn't taking these approaches seriously.

Use-cases which cry out for CEP include market data absorption and enrichment, algorithmic trading, smart order routing, market making and pricing engines, market liquidity discovery, market surveillance, transaction cost analysis, regulatory and compliance requirements, and especially real-time risk management -- any situation where decision-makers need to know when things are happening on a real-time basis.

Indeed, complex event processing (CEP) has been getting a lot of attention as of late, and is seen as the next stage of evolution for SOA projects. Now it has the big vendors -- including IBM -- promoting it.

IBM WebSphere CTO Jerry Cuomo recently said that he sees the SOA-EDA-CEP marriage as one of the most pronounced trends to watch in 2008. As Cuomo explained in an interview: “I really believe [event processing is] the next big thing in SOA…. we’re taking it very seriously.”

IBM even wants to take the technical allusions out of the term itself, referring to it as "Business Event Processing." The renaming to BEP makes sense, since I doubt if a concept that starts with the word "complex" is going to win a lot of converts.

Last week, Big Blue put its money where its mouth is, announcing it would be buying AptSoft, a CEP tools and platform vendor. Sandy Carter, senior vice president of IBM's WebSphere division, said that what they call "business event processing" should be at the disposal of, well, the business. "Everybody is talking about complex event processing. We are trying to rename that category, because we believe the real value is in Business Event Processing, the focus on the business. We believe we are elevating for customers something that was a deep technology capability, something that only engineers understand."

With its purchase of AptSoft and plans to include CEP (er, BEP) functionality into its WebSphere platform, IBM obviously sees a robust market opportunity emerging. Carter said IBM already has invested a billion dollars in the events processing space, not counting its AptSoft purchase.

Posted by joemckendrick in Business Process Management • Data Management • SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

In this blogsite and many others, we constantly talk about business systems "agility" as the holy grail of service-oriented architecture, business analytics, and everything else.

But what exactly is agility, and how do we know when we have it?

That's the intriguing question put forth by Cliff Longman, chief technology officer for Kalido: "The industry talks about 'agile' systems, and 'flexible IT'. Could someone propose a way to BENCHMARK agility?"

Cliff suggests that perhaps agility should be measured in the amount of time it takes to change an application may be a yardstick for agility. "Some people would say that adding a new hierarchy to an existing BI application in less than a month is 'agile'. Some would say that only less than a week is 'agile'. Some say that only less than a day is agile."

Is there a way of putting a quantitative value on agility of business systems? Readers, we'd love to get your thoughts on what, exactly, is meant by 'agility,' and how that can be measured, if at all.

Posted by joemckendrick in Management  |  Permalink  | Comments (1)  | TrackBacks (0)

What lessons have we learned from SOA that can be applied to Web 2.0? Plenty. But perhaps the most important is the role of governance, so organizations can get the most out of the services that are created under their roofs.

ebizQ colleague Gian Trotta recently spoke with Kelly Emo, SOA product marketing manager with HP Software, who worries that end users may be running away with Web 2.0-based services as end-runs around busy IT departments. (Listen to the complete podcast here, or see the transcript here.)

She notes that IT departments are dealing with "meaty back office problems" but end users are sometimes too impatient to wait for IT departments and their planning processes. So they take their needs into their own hands with new approaches like mash-ups and Web collaboration.

IT is caught up in issues such as "'how do I leverage the legacy infrastructure structure,' 'how to do I change my point-to-point integration into more flexible, loose coupling, more dynamic.' 'how do I break up my application silos'? They’re using serious architectural disciplines such as identifying their key reusable components and exposing those as standard space services."

What is happening on the user end is that "creative end users don’t have the time or the patience to wait for this plan to meticulous processes," Emo explains. "So they’re taking their needs into their own hands and using approaches, kind of Wild West approaches like mash-ups and Web collaboration."

But don't worry -- it's all good, Emo says. "These mash-ups in many cases are resulting in big productivity gains," she says. And these productivity gains "are getting the attention of the VPs of the business domains -- the folks with the money," she points out. "And they’re coming back to IT and saying 'make it so' -- support this application."

So the ball eventually ends up in IT's court anyway. The key is that IT will embrace the new "Wild West of Web 2," but pay attention to governance, Emo says. "They can embrace this capability. And they can make it work using the same level of robustness, the same level of service, quality of service. Or they can put up roadblocks and say, 'no, I am not going to let this rogue capability into IT.'

The best bet for managing Web 2.0 approaches, Emo says, is to "combine it with the productivity and architectural best practices of SOA. Effectively, what IT is doing is combining innovation and discipline. And the concept behind this is what HP is calling 'implicit governance.'"

The good part about implicit governance is that Web 2 consumers are "not even aware that they’re participating in an IT governance process, but in essence they are," Emo relates. "They’re assured of getting the service that they just basically have grown up to expect -- the always-on capability."

Emo will be speaking on this issue in a presentation called "Enterprise Mash-Ups for Wall Street: Leveraging SOA Web 2.0" at the Web Services/SOA on Wall Street show set for February 11.

Posted by joemckendrick in Enterprise 2.0 • Management • SOA • SOA Events • SOA Podcasts • SOA Vendors  |  Permalink  | Comments (0)  | TrackBacks (0)

Increasingly, industry analysts and practitioners are connecting the dots between three emerging megatrends -- SOA, Web 2.0, and Business Intelligence. It's becoming increasingly clear that the combination of SOA and Web 2.0 technologies is changing the face of business intelligence.

These were the topics raised at a recent ebizQ Webinar -- which I had the opportunity to moderate -- with noted author Don Tapscott, Katrina Coyle, BI manager with Molson Canada, and SAP's Lothar Schubert. (Audio replay available here.)

Don Tapscott, who broke new ground in 1996 with his book, The Digital Economy: The Promise and Peril of Network Intelligence, and recently co-authored Wikinomics: How Mass Collaboration Changes Everything, said BI is on the verge of a revolutionary transition, thanks to SOA and Web 2.0 approaches to technology innovation.

"Web 2.0 and service oriented architecture are really becoming a new mode of production," he says. "They’re changing the ways that we innovate, the ways theat we make decisions, the ways that we collaborate, and the ways that companies engage with the rest of the world."

This has very profound implications for business intelligence, which is evolving into "collaborative intelligence." Don continued. "Prior to the introduction of Web 2.0 methodologies, he explained, internal data had "been accessible in various limited ways through traditional ERP reporting systems, MIS and business intelligence."

"The marriage of this new accessible data with the firm’s traditional internal data creates an unprecedented challenge, as well as an opportunity to gain insight into the behavior of the company’s most important stakeholders, and to translate that knowledge into success in the marketplace."

The speed of Web 2.0 processes is also changing what end-users expect from BI approaches as well. "Think about if you do a Google search, you get the results back instantly. If the results took half a minute, or five minutes, or 10 minutes, you’d probably stop using Google so much. Traditional BI was kind of like that -- which is part of why we didn’t use it so much Because you’re calling out to a disk, basically." In-memory technologies are also making new generation BI technologies lightning fast as well.

Molson's Katrina Coyle also credits SOA with reshaping her company's ability to compete in a fast-changing and often fickle market. "One of the terrific things that we’ve had in the last year is service oriented architecture," she explained. "We can now deliver information to our business in any way they want…. we can drive information through emails, text, BlackBerries, and widgets. If we have issues anywhere in the supply chain, we can get that information out in real time to supply chain managers."

As with many SOA endeavors, corporate culture presents some barriers and challenges. "We’ve been brewing beer for a very long time," Katrina noted, stating that "when you’ve been doing something as long as we have, you get a lot of habits that are pretty well ingrained. Trying to shaking the business out of those habits is a challenge."

Molson's strategy to transform its organization includes reaching out to a new generation of younger adults through Web 2.0-based marketing strategies, and leveraging service-oriented architecture and data warehouse approaches to build its brands across the globe.

An important emphasis is real-time analytics, Katrina said. "We are constantly having to shift and change and shift and adjust very quickly to changes in the marketplace, " she explained. "We all have to be extremely agile. You can’t spend a week trying to figure out whether the promotion is successful. You have to be able to react within hours."

It used to be that companies didn't know if a promotion was successful until then end of a quarter, if then. Real-time analytics can look at patterns and trends and provide insights if something is working or not, enabling a quick change in direction.

For example, one trend that Molson was able to jump on fairly rapidly was a sudden craze for cold beer in UK pubs -- long bastions of warm beer. Katrina explained that each of its global sites have their own go-to-market models, but all this information needs to be rapidly assimilated. "We have a data warehouse, with lots of information coming in different ways," she said. "It's not necessarily all coming from a centralized ERP system. We also have data coming in from AC Nielsen, for example. "We’ve got to bring that data in, and make sure that it has a harmonized look, so our business can actually make tactical decisions on it."

By adopting in-memory technology available through SAP's BI Accelerator, Katrina reports that Molson has been able to move data quickly through its data warehouse. "We’re able to process data now in real time in our warehouse -- we’re not tied to a load once a day or once a week."

Audio replay of the ebizQ Webcast, "The New Paradigm for Business Intelligence - Collaborative, User Centric, Process Embedded," is available here.

Posted by joemckendrick in Data Management • Enterprise 2.0 • SOA • SOA Podcasts  |  Permalink  | Comments (0)  | TrackBacks (0)

Almost half of respondents that have responded so far to a new ebizQ survey report they have SOA solutions already in place somewhere within their enterprises, but these efforts are scattered.

One out of four at this time report having more than five SOA projects going on across their organizations, and most say these are separate, unconnected efforts. For the most part, few touch mission-critical enterprise applications, though a majority of respondents expect to be interfacing with at least a few of such apps within the coming year.

These are preliminary, top-line results from ebizQ's current survey of SOA best practices, which is still open and awaiting your participation. Click here to complete the survey, if you have not already done so. All participants will be entered into a drawing for a $300 American Express Gift Card.

Posted by joemckendrick in Business Process Management • Case Study • Management • SOA • SOA Research and Analyst Reports  |  Permalink  | Comments (0)  | TrackBacks (0)

"I hear there is going to be a recession. I decided not to participate." - Sam Walton

I've been engaged in an across-the-blogosphere discussion of what the implications of an economic downturn would mean for service-oriented architecture.

Tony Baer discusses our posts and some of the impiciations for SOA here at ebizQ, and Brenda Michelsen raised the issue a few weeks here at ebizQ as well.

My thoughts? We have to first cut through the absolute media hysteria that has been raging out there for the past month or so. Economists really can't agree if this will be a recessionary year or not. And, no, the sky isn't falling, just like it didn't fall in 1982, 1991, or 2001. In fact, in each case, we eventually came roaring out of those downturns.

However, let's face it, when it comes to budget crunches, the pinch was never lifted off IT since the post-dot-bomb and post-Y2K hangovers of 2001. IT has continued to operate under tight constraints, constantly being asked to do a lot more with less. Even if the economy were roaring with 25-percent growth this year, budgets would still be tight. Companies would still outsource.

Hence, the interest in SOA, Web 2.0, and open source approaches to problems. SOA offers a way to trim development costs by reusing and sharing services and eliminate duplication. Web 2.0 offers ways to secure computing and data resources on an incremental or low-cost fashion. Open source software, of course, offers unlimited licenses at virtually or little cost.

Again, enterprises would push for the SOA-Web 2.0-open source troika no matter what the economy looks like outside.

Plus, I think the greatest issue will continue to be a shortage of talent to conceive, design, and maintain SOA-enabled architectures. As I discussed a few days back, the amount of SOA project work out there far exceeds the available talent to fulfill it.

Having said that, it is fair to conclude, as Tony points out, that the forward-looking companies that "get" SOA understand its value and will keep pushing for it. The companies that don't quite get SOA -- and see it as a "special project" to be funded without understanding where the return will be -- are likely to cut off such efforts. Then, of course, there's those really crummily managed organizations that will use what they call "SOA" as a hammer to justify layoffs, but let's not talk about them. (Remember "re-engineering"?)

Tony also notes that previous recessions did not slow down major IT transformations, anyway. As he observes, "the recession of 1990s occurred just as IT was on the verge of another great architectural migration, in that case to client/server. While investments in client/server were obviously slow back at that time, when dollars flowed back several years later, they did so with a vengeance. And that was before IT organizations got caught up in the Y2K and Internet booms."

Lorraine Lawson also does not see a huge impact from a potential economic downturn on SOA efforts and IT shops, echoing Sam Walton's sentiment that they will likely decide not to participate.

In the event SOA proponents are asked to cut back, she offers these pieces of advice. And, I will add, good advice even when the economy starts roaring again:

1. Focus on short-term ROI.
2. Think tactical, not strategic (related to #1)
3. Tie your SOA to Web 2.0 and social networking projects.
4. Tie your SOA to emerging business changes.
5. Focus on the dead or dying. (Use SOA to rejuvenate older legacy systems)
6. Reroute funds from other projects.
7. Try Guerrilla SOA. (My favorite)

Posted by joemckendrick in Management • SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

How do you get executives as excited about service oriented architecture as you are? Here's some advice I pulled together in a new article published in this month's Insurance Networking News:

Focus on the business benefit, not technical benefit: Business people don’t seek “SOA,” they seek cost-effective solutions to their problems. I

Focus on the solution without the technology: Look at a process that needs service-enabling and determine where the bottlenecks are, and how things would flow without technology.

Think long term: SOA, at least initially, costs money. Over the long run, as the SOA gets up and running and the stable of services grows, economies of scale kick in. But until then, there’s some up-front investments required that will be reflected in per-project costs.

Measure success: Key performance metrics, tied to specific areas of the business, will help calculate the return SOA is delivering in specific areas. Calculating return on investment need not be a complex undertaking. The key to measuring future SOA value delivered is to first establish a baseline of the company’s current costs and status. “Where am I today?”

Posted by joemckendrick in Management • SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

There's been quite a bit of confusion and controversy over the role Business Process Execution Language (BPEL) plays in gluing business processes together with an SOA context. Some analysts even say that BPEL is not a language, it's not a about the business, and it has nothing to do with process.

In this TechTarget article, Daniel Rubio addresses some of these questions and connects the dots between Business Process Execution Language and business process execution.

Rubio reminds us that business processes are created in the minds of business managers, with technology being a secondary consideration.

When the specifications are handed down to IT folks to make the vision happen, this is where BPEL comes into play. "In the same technical context as SOAP, WSDL and WS-* standards play a role in creating individual Web services," Rubio relates, BPEL "focuses on the technical requirements needed to materialize a business process made up from Web services."

Prior to BPEL deployment, however, "a business process be broken down into manageable units of work, allowing each unit to be owned, developed and tested separately," Rubio observes. This serves not only to increase "manageability through the classical divide-and-conquer development approach, but also simplifies many design issues involved with Web services."

BPEL is then engaged through BPEL engines or business process management tools.

Posted by joemckendrick in Business Process Management • Management • SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

Looking for a quick way to show your business what SOA can do for it?

Neil Macehiter and Neil Ward-Dutton of Macehiter Ward-Dutton offer a series of five visuals here at ebizQ which say it all. Check it out here.

The Big Five:

1) Flexibility -- Ease of change isolation
2) Reuse -- Improved quality, lower costs
3) Transparency -- Operational management
4) Openness -- Ease of extensibility
5) Comprehensibility -- Ease of business alignment

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

Service-oriented architecture is a very popular concept, but with that popularity comes a price. Namely, finding people with the skills to help put SOA in place is a growing challenge.

Last year, ZapThink's Ron Schmelzer sounded warning bells about a looming enterprise architect "drought" which could derail many SOA efforts. He noted that there "is a significant demand in the marketplace for experienced SOA talent," and "a burgeoning of SOA consulting companies that offer kick-start approaches to SOA in which they supply the experienced architects and their customers supply the heavy-lift labor to implement the services."

Miko Matsumura, vice president of WebMethods, also has expressed concern that SOA movers and shakers in organizations may be too few and far between. That role is falling to what Miko calls "visionary architects," who can provide a "scalability of understanding" to drive the scalability of the system.

AMR Research has just issued a report that also warned that demand for SOA-skilled professionals is exceeding supply. As a result, consultants are jumping into the gap -- As reported in SearchSOA TechTarget, SOA given rise to a massive opportunity for consultants, the consultancy observes. This is much like the demand the way "Web consulting was at the end of the 1990s," writes Ian Finley, AMR research director.

However, not all service providers are comfortable with SOA quite yet. Interestingly, AMR noted that SOA consulting was not the most profitable route for software vendors, with the exception of IBM, which has a large SOA practice. Once a company begins to move toward SOA, this dynamic changes. "Once a company is comfortable with SOA as its standard approach, it often seeks to shift all project work over to SOA," Finley wrote in the report. "Given the shortage of SOA skills in a typical company and the difficulty training some traditional IT staff in the SOA approach, consulting firms are well positioned to reap a windfall of SOA projects the coming years."

The AMR report notes that this is "a sea change from what systems integrators and service providers has done in the past, such as integration between applications, implementation of ERP applications, along with IT planning and strategy work. All those things now are being converted over to SOA at customer request. In other words the customers are telling them we don't want to do things in the old way. We want to do this with the knowledge of SOA."

Posted by joemckendrick in Management • SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

SOA is ultimately a big initiative, and we lean on our enterprise architects and IT staffs to make it happen.

How do these overworked folks do it? As Michael Hugos so eloquently put it in a recent ComputerWorld piece, architects and their IT colleagues are charged with balancing "multi­year infrastructure development work with short, 30-to-90-day projects that deliver business application systems as business needs evolve. And do it so that even as you’re implementing the IT infrastructure, you’re delivering new applications that use this infrastructure..."

"You might think that you might as well try to change the tires on a race car while it’s still moving. How can you build systems that use a new IT infrastructure until that infrastructure is in place?"

This is where maintaining common sets of standardized enterprise components come into place, Michael writes. He recommends using pre-defined, iterative system-development techniques under the guidance of the enterprise architecture standards. "Combine selected infrastructure components as needed with small chunks of custom code to create new systems. In this way, systems can be delivered and enhanced quickly. What keeps them from becoming an unmaintainable mess is that the systems are all created from the same set of components. That means that the IT groups trained in using this enterprise architecture can maintain and enhance any system built from those components."

Chunking. I like that concept. I've heard management guru Tom Peters talk about chunking, in which seemingly complex and overwhelming projects are broken down into manageable... well, chunks. Tom credits chunking with helping successful companies push through even the most complex projects.

Michael applies this thinking to enterprise architecture: "A good systems architect also understands that, when done right, these short application-development projects actually drive much of the longer-term rollout of the new IT infrastructure. Most infrastructure components — whether servers, operating systems, databases, middleware, Web portals, SOA tools, packaged software or programming languages — can be rolled out in phases that build upon one another."

This is a good strategy to keep in mind for 2008. EbizQ colleague Brenda Michelson has been exploring scenarios for SOA in the event of a rocky economy, and I believe one such effect will be IT professionals and architects being asked to do more with less -- meaning few additional staff or consultants will be brought aboard to help with new projects.

Heck, even if the economy keeps growing through 2008, IT folks will still get asked to do more and more with less and less. Enterprise standards and components hold the key to being able to do much more with less.

Posted by joemckendrick in SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

SOA, by its very nature, encourages a highly decentralized, loosely coupled approach to management. That's why it often is a difficult fit within organizations that attempt to shoe-horn "SOA" approaches into rigid, hidebound corporate cultures.

Progress Software's Hub Vandervoort recently published an ebook, Socially Oriented Architecture, that talks about the role of management and corporate culture in SOA success.

ebizQ colleague Beth Gold-Bernstein recently spoke with Hub about his observations, in which he points out that "the worst failures in SOA that I’ve seen have been those where they try to employ a very federated environment and capitalize on reusing services in other domains and so forth, but then they try to manage it with absolute hierarchy. Because what you find is that the domain members tend to want to secede from the union, so to speak."

The bottom line is trust and commitment have to replace command and control for SOA to be successful in organizations, Hub explains. "In the experience we’ve had in rolling out now over 400 ESBs and very large SOA infrastructures, that attempt to approach multiparty interactions at a very large scale, what we’re realizing is there is as much a need for figuring out how to get the people in those various communities and different organization structures to work together in a harmonized way," he says.

"We use the word 'governance' a lot but your SOA needs a way of interacting socially so that all the people who own their various bits of the SOA can work together."

Posted by joemckendrick in Management • SOA  |  Permalink  | Comments (0)  | TrackBacks (0)

Two colleagues here at ebizQ have weighed in on the most vexing issue in the SOA world these days: how does SOA mesh with mashups, SaaS, and all the other aspects of the Web 2.0 and Enterprise 2.0 wave.

Krissi Danielsson talks about the disconnect between SOA and SaaS, noting that many commentators get the two concepts confused.

Keith Harrison-Broninski talks about the intersection between SOA and Web 2.0, which can called to as "Enterprise Web 2.0." In the midst of this intersection is enterprise mashups, which will eventually point the way to business process-driven architectures.

The growing convergence between Enterprise 2.0 (which includes SaaS) is the SOA story of 2007, and likely the story of 2008 as well. SaaS presents an interesting opportunity for SOA proponents as well. Business decision makers who may be flummoxed about the meaning of SOA typically get "SaaS" pretty quickly. SaaS providers that build their services on SOA-based standards will find their solutions best interlock with customers. Plus, a good analogy for SOA is that it's SaaS, delivered internally.

Posted by joemckendrick in Enterprise 2.0 • SOA • Software as a Service  |  Permalink  | Comments (0)  | TrackBacks (0)