SOA - Service Orientation Applied

Posted by Animesh Ghosh, Technical Architect

SOA is a set of Policies, Practices and Architectural patterns; it is not Technology, Product or Standard. These set of Policies, Practices and Architectural patterns must be a part of Governance which is a decision and accountability framework. Governance does not prescribe how to manage an organization on a daily basis. However, it provides a collection of solutions and policies coupled with a method that encourages desirable strategic behavior.
 

It has never been too early or too late for an organization to start SOA Governance, as soon an organization starts its SOA journey, it must be accompanied by a proper Governance mechanism. This journey could be well supported if there is a proper SOA Management and Monitoring infrastructure in place.

The key aspect of SOA is to make business functionality available through a set of well governed, standards based, loosely coupled services and processes, defined in a flexible and agile manner. To successfully achieve this, SOA must be done under proper SOA governance with associated tools in place

Why SOA Management and Monitoring Tool?
 

When an organization progresses with its SOA initiatives it recognizes the challenges that come with the benefits that a successful SOA implementation offers. The Organization realizes that it will need a special set of IT Governance which is SOA Governance. Conventional governance and management does not work because loosely coupled services and their interactions require enforcement of metadata and policy that current tools do not provide. Some of the service-based applications are distributed across different organisations which work in co-operation to perform a certain task.
 
In SOA, an organization is tasked with understanding and controlling a live, dynamic network of interdependent services, one that evolves quickly and includes application components beyond the control of the IT staff of a particular organisation. Governing this de-centralized production system is critical to success with SOA and requires a specialised tool. A SOA Management and Monitoring Tool can provide an insight view of the SOA service infrastructure to the Business and IT team, giving more relevant information for decision making and accordingly the team can manipulate the services.

Figure 1 below shows how a tool becomes an integral part of an organisation’s SOA journey.

 

Figure 1

How to select a SOA Management and Monitoring Tool?

One of the main aspects of a successful SOA is the careful implementation of a SOA Management and Monitoring infrastructure. As SOA Governance is about decision and accountability framework, while selecting a tool we need to understand how well a particular tool might fit into the existing framework and help to manage and monitor SOA infrastructure. SOA is an incremental process. ‘Big Bang’ approach really does not work here. An organization needs to select a tool that can support it at any level of SOA maturity and during the transition to the next level.

The ultimate aim here is to satisfy customer’s needs and business is the main customers over here. A tool must provide a meaningful business view from where an non-IT (i.e. business) will be able to get what he needs to make business decision which is directly linked to organizational Goal and Strategy.
 

Tools from mega vendors like IBM, Microsoft, HP, Oracle etc. each of them tried to address different aspects of SOA Governance and Management challenges in their own way using their own set of technologies. However, the most crucial aspects to consider when choosing a SOA Management and Monitoring tools are:

• Policy-based approach (Business)
  
• Service network monitoring (Technology)
  
• Service and infrastructure discovery (Technology)
  
• Service level management (Business)
  
• Exception management (Business)
  
• Policy enforcement (Business)
  

Organizational aspects are more important than Technology aspects while selecting a SOA Management and Monitoring tool which must facilitate the comprehensive SOA Governance of the organization.

Figure 2 below summarises what to consider while selection a SOA Management and Monitoring tool.

 

Figure 2

Tools from each of the vendor are good in certain angle but deciding the right tool for an organization requires careful consideration of the needs and the relative strengths of each product. There are several tools from the major vendors and here is what my personal view about them.

o    HP OpenView can provide end-to-end monitoring capability for virtually every kind of application in this world. HP Systinet focuses on comprehensive SOA Governance and Management. You got to use them together.
o    IBM Tivoli Composite Application Management for SOA emphasizes core web services management and integration with the Tivoli Monitoring framework. This tool is matured enough to count on.
o    The only tool focusing extensively on organizational governance aspects. AmberPoint focuses on managing the runtime aspects of SOA Applications and helps to manage performance with service-level objectives.
o    Actional focuses more on service monitoring and it provides a comprehensive end-to-end view of activity and performance of executing processes.
o    Oracle is a leader in Forrester's SOA Lifecycle Management Wave, via acquisition of BEA, providing comprehensive support for governing the entire SOA ecosystem and out-of-the-box support for capturing and measuring your SOA investment.

Posted by Guest Author at 08:34 PM | Permalink | Comments (0) | TrackBacks (0)

Gartner predicts that by 2009, a service-oriented eco-system will mature and provide the foundation for a massive wave of business process innovation. Most organizations today have some level if interest in SOA as a 'future' strategy. However, an early barrier to the adoption of SOA is the appreciation of the difference that SOA can make to the organization. Maturity of SOA adoption  across the globe has not yet reached the level that can provide measurable benchmark for predictable ROI commitment. Hence, the qualitative appreciation of the SOA values can probably play a critical role in the accelerating the momentum of the SOA initiatives. Range of opportunities that SOA brings to table allow organizations to exploit multi-dimensional benefits  from SOA. SOA can make lot of difference to the organization if it is adopted as a mainstream Business-IT framework.

Consistent and high-quality Service experience
SOA enforces standardization of service behavior which will mean that consumers of the services will experience high degree of consistency and quality throughout the life-cycle of the service delivery. This is particularly important because service experience is a key component of the ‘service orientation’. When services are being designed and developed, service behavior becomes an integration part of the considerations so that when services are deployed, the experience it creates with service consumer in terms of service engagement, service usage and service support, that makes lot of difference to consumer community.
More meaningful Business - IT mapping
Traditionally, I always felt that transition from business requirements to technical architecture has been a challenge in the industry due to lack of practical frameworks that can be used as transition model. With SOA, now we have service architecture sitting in between business functional design and technical design. This allows business functional maps to be easily configured and mapped to IT capabilities (typically using BPM wireframe) without creating hard-wiring between the two. This way in my view, whole service modeling and service architecture exercise brings Business and IT together where each has a critical role to play to make the SOA happen. This becomes partnering model between business and IT as opposed to traditional ‘handover’ model where business disengages itself after certain stage expecting IT to take over.
Shared view between Business and IT
Further to previous point, Service architecture acts as a common language / platform / view between business and IT and thus enables more meaningful and collaborative engagement between the two. Typically the gap between business and IT in terms of delivery model as well as common vocabulary has been a concern for most of the enterprises. Now Business and IT can talk the language of services that will make sense to both communities.
Service Level driven focus
Service level focus, in my view is a very essential part of the ‘service-orientation’ that also plays important role in the ‘service experience’ that we talked of before. In an SOA driven environment, service levels are governed to cater to stringent business performance metrics in direct fashion and hence entire monitoring, reporting, alignment, support and continuous improvement efforts are focused around the service level performance of the services. This changes the tradition model of ‘managed by exception’ to ‘managed for excellence’ which hopefully will change the level of accountability and ownership IT teams specifically take today for the business outcomes.
Separation of logical capabilities to provide simpler and cleaner architecture
Service architecture provides ability to consolidation and rationalize the business / IT capabilities in optimized fashion that have similar behavior, anatomy and operations. This allows more cohesive and efficient interactions/collaborations across capabilities in various logical layers (by design rather than by accident) to deliver final business outcomes. In principles, software architecture had the ‘logically’ separated layers always but with service architecture, now it is much easier to separate out the logical layers at execution level as well.
Easier Change Management
Changes ( of all sorts, technology change, business change, regulatory changes, IT vendor change, IT in-house staff change) are here to stay in the industry, at least for good number of years in my view. Through various core principles like decoupling, virtualization, meta modeling etc in SOA, it will make the life easier to change different elements of the solution without too much pain or expense. Change management is adopted as a mainstream focus for service management and hence we can expect more discipline in maintaining the software solutions.
Hide technical complexities of service implementation
Service architecture allows diversity in service implementation (through mash-ups, service virtualization etc.) to make use of new technologies while enforce standardized service interface to shield the consumers from complexity of service integration and implementation. As I see, industry is far from standardizing on a single technology platform and as we move further in future, more and more technology advancements will put the challenge to technology standardization. If we accept that fact, service architecture paradigm will allow enterprises to leverage new technologies in more flexible, open and easy manner.
Quicker and simpler solution composition
Making the software solution composition (highly productive, cost effective by reuse and ability to innovate solution patterns etc.) and development life-cycle (allows develop, testing and deployment done at service level where service being the modules of development and not the components) quicker and simpler is another difference SOA adoption can bring into the organizational DNA. This has long lasting implications, much wider than what we might think. Service architecture based development practice will bring innovation to transition typical ‘build’ intensive solutioning methodology to ‘composition’ intensive methodologies. This will not only allow the need of having large ‘coding’ teams with intense technology knowledge to be shifted to more ‘solution composition’ teams that may not so much technology intensive knowledge requirements. That way, today software development talent profiles are already upgrading themselves into new roles which I think is a positive change.
Business Scalability
Ability to support (and even ignite) the business growth (more services, more users, more geographies, more partners) through accelerated scalability, global integration capability and easy service on-boarding infrastructure is definitely a great win for business community. Business scale is not just in terms of doing more of the same but actually lot more of ‘doing different things’ for which time to market is a key differentiator. Service fabric of the business-IT architecture allows easier composition of the solution as we learnt in previous points and further to that, it ensure that when business goes out for scaling in terms of functionality, geographies, value chain expansion etc, the SOA enabled business-IT architecture acts as a enabler to support the change.

In general, I could say that SOA has lot in store to transform the organization but probably it is the seriousness that industry has to get to make it happen. I'm very confident that it will happen. What do you think?

Posted by Rakesh Mishra at 09:59 AM | Permalink | Comments (0) | TrackBacks (0)

In the last 2 blog posts, I talked about use of XML Appliances to turbocharge your SOA infrastructure. In this blog, I intend to provide some numbers provided by Intel. Most other vendors do not seem to have these numbers in a public document.  The features described here are available from many different vendors, and the other products in the space are equally good, with comparable performance numbers.

Intel's appliance is not actually an appliance but a product called SOAExpressWay. It runs on standard Linux servers, providing the "benefits of having an appliance, with the flexibility of a server, while using commodity hardware".   

 The benchmark numbers are impressive, and are for a dual CPU Intel Box. I wish the benchmarks used Java 1.5, which is approved in many enterprises rather than Java 1.6, which is not. But I do not think the numbers will be dramatically different.  

 Want to protect your SOA infrastructure from being flooded by badly written XML that does not conform to your schema? This could be an external denial of service attack, or more likely an error at the XML producer end. With SOA and agile development, there is the advantage of incremental development, but it is coupled with the risk of error prone code getting into your production or performance/staging environment.  The Intel SOAExpressway can validate XML to a schema at the rate of 507 MB/second, on a dual CPU box.

Want to validate the data that has come in using simple XSLT based rules? The Intel SOAExpressway provides XSLT processing at very high speeds. For example, specific fields could have valid ranges, or be constrained based on values of other fields.

An interesting feature of the SOAExpressway architecture, that it is not a pure appliance- non-XSLT based validations written in Java or C++  can also therefore be done on the XML data. As pointed out in this  excellent presentation on parleys.com(see slide #20,21) ,  there is always something that is very difficult, if not impossible to do in XML.

Want to convert about 1 GB of flat file into XML? It should take about 30 minutes or less with the SOAExpressway running on a dual CPU Intel Linux server. A past engagement showed a data volume of 50 MB per hour exchanged with an external application service provider. It would have been very convenient to use an appliance to validate, transform and import/export data using a product like the Intel SOA Expressway. The flat file to XML conversion is comparatively slower.

XML to XML conversions run a lot faster. The benchmark paper mentioned earlier, shows XML to XML conversions running at between: 500 KB per second to 41 MB per second. This is on a dual CPU box. This is not "wirespeed" as most appliance vendors hype, but it is enough to meet the needs of  many enterprise or even consumer facing enterprise portal needs. For example, consider a WML page of 4 KB on a cellphone, which is refreshed every 60 seconds. Assuming a transformation rate of 20 MB per second, a dual CPU server could support around 30,000 users.

XPath execution speeds in the  benchmark paper ranged from 400,000 to 1.5 million expressions per second. Using XPath to perform rule based validation, is thus very scalable using the Intel SOA Expressway. Incoming XML document can be validated for consistency, such as "To-date" being greater than from date.

Why not use XALAN instead? The manageability features of the SOAExpressway including basics such as SNMP Trap creation, Email notification, as well as the more advanced features such as  application level end-point support  and  Scriptable management using CLI are critical in an enterprise environment. 

A common problem in many SOA Projects is creating a "sub-set" XML document, from a given XML document. For example: The XML provided to you has 250 fields, but you need only 10 of them, in your application. A standard pattern is to read the large XML document, parse it to discover the 10 needed elements. The memory consumed in this parsing operation, can be large, causing frequent garbage collections.  Performing this operation on Intel SOAExpressway should be a lot faster than in the application server.  Moreover, you are  performing the transformation on a server that is comparatively cheaper. A fully loaded dual J2EE  Portal Server can easily cost  upwards of $100,000 per fully loaded server. In one portal project, about 95% of the portlets, called webservices on the backend, and transformed the XML so obtained, to generate the HTML markup. The  XML transformation load could be a large fraction of the CPU use on your expensive portal server.

In a given project, thinking about using a XML appliance may be difficult, but if XML Transformation and validation are provided as standard services, then this technology can percolate into the mainstream of your enterprise.

Posted by Kevin Apte at 06:39 PM | Permalink | Comments (5) | TrackBacks (0)

Well as with any other conference, the excitement slowly goes down as number of days pass by.  There were fewer participants for Day 3 and Day 4, with hardly authors seen around for presenting their work.  Keynotes in my view should throw some light on to new research directions, trends and innovation opportunities.  Keynote themes at SCC 2008 on three days also were pointing a direction.

Adaptive service based for developing future software, game changing cloud computing and its complementaries with SOA were the themes that were discussed during the keynotes.   This has shown that research community and academia is excited about the cloud computing.  Industry has already seen success of cloud computing with most notable one being Amazon’s EC2 and S3.  The debate was how research community would need to fuel innovation for cloud computing. The explosive growth in social networking, mobile platforms and collaboration platforms will see more and more innovations in near future.

Clearly as it could be noticed based on all the papers presented, the service computing research community is heading towards developing platforms for large scale development.  By large scale development what I meant was development that not localized but distributed across many locations.

Also, I feel worth mentioning here is an intresting poster stuck near the registration desk, it talked about a mashup application.  An application which could help in monitoring and possibly predicting the effect of wild fire.  It took the information provided by google earth API's and weather information service. It is so interesting to see so many valuable innovation and possiblities through mashups.

Posted by Naveen Kulkarni at 10:40 AM | Permalink | Comments (0) | TrackBacks (0)

It was the day of presenting research and industry papers.  There were many interesting topics that caught my attention; but I had the choice of attending only a few as they were mostly running in parallel breakout sessions.  I certainly did not want to miss the sessions that were discussing - management of distributed collaborative centers for executing work requests, change request scheduling, using domain knowledge for service integration, establishing service model from business model and variation oriented engineering... 

As I looked at the program schedule and started marking the topics to attend, it became evident that I was still loomed with yesterday's discussion.  My inquisitiveness - to find answers for those questions had made we choose the above mentioned topics.  Interestingly most of these topics were presented by researchers from IBM.  Most of the discussion was around the capabilities of execution centers, managing resources, an important conclusion that I could draw was that - It is necessary to model work allocations based on execution centers rather than on task and resources.  The resources should however be considered as the capability of execution center.  Further from other session researchers explained the importance change management, knowledge, service modeling and variation based approaches for Service Engineering.  There were few approaches proposed for which I would need more detailed reading.

After the initial excitement about Service Engineering, post lunch it was time for the topic which I have been looking for - Architecture for service based computing.  This tutorial proposed an 'Enterprise Service Architecture' (ESA) and gave the background and stressed on the fact of why this was necessary.  This explored the use of competence theory for answering few important questions that researchers perceived very essential. However, most interesting part of the tutorial was when the authors reviewed various frameworks that could be the basis of ESA.  Few of these frameworks that I remember were Porter's Value chain model, Business motivation model, Strategy maps, i*, and few more.  The tutorial was concluded by highlighting few gaps and setting few more motivations.  This being my area of interest, I feel there is lot research that needs to be put in both from formalizing it and finding ways to apply to real life situations.

The day ended with a simple and quite banquet, a regular tradition at any conference....

Posted by Naveen Kulkarni at 11:10 AM | Permalink | Comments (1) | TrackBacks (0)

I had to travel around the world to attend a conference.  Well it was at Hawaii, so travel really didn't matter much.  It was Day 1 of IEEE SCC 2008 conference, this day was dedicated to some interesting workshops like Electronic service marketing, Web X.0, WS composition and adaptation, Scientific workflows.  I had to present three papers at Service and Process Oriented Engineering Workshop (SOPOSE) (you can find the program here). 

This workshop followed a unique model of each paper having a discussant.  This, in fact, kicked off some interesting discussions.  One which interested me was around the topic of goal mapping to existing services.  The author had presented an approach (BGSC Graph) to match services closely to requirement while discovering them.  However, my understanding was that author had made an assumption of having a service portfolio but hadn't considered the origin of service.  It has been our experience that service created within organizations today still lack the rigor of having gone through the top down approach (starting with what the business wants).  This would mean it would be difficult for breaking up the business goals and matching with objectives with which services were created.  However, on another note, I feel this would be useful if the services are being sourced from partners. Today we have Amazons and SAPs offering out of the box services and this method (BGSC Graph) can help in evaluating them.
 
There were a few other discussions around verification of correctness of contracts, an architectural model for SOA and modeling virtual enterprises.  Interestingly though not in earlier program, there was a key note from Dr. Banavar, Director of IBM India research lab.  Following the key note discussion loomed around what would the future and according to Dr. Banavar, it was Services Engineering.  "Disciplined Service Engineering can help the world economies to systematize service development, and focus on service innovation" to quote from the presentation was the compelling factor and the necessity for the services community to investigate.  He brought out two immediate areas to make service engineering possible.  Firstly, was similar to industrialization ie., large scale development using distributed collaboration across the world and secondly the technology aspect - a new way called variation-oriented engineering.  The discussion ended with the note that we the research community in services computing have to raise to address the challenges of the future, which will come as Services Engineering.

Will continute to post some interesting discussion at conference...

Posted by Naveen Kulkarni at 12:47 PM | Permalink | Comments (0) | TrackBacks (0)

As IT is always full of jargons, SOA is no exception.  One of the most commonly used terminology in SOA world is the term ‘Reference Architecture’. However, there are several places where people seem to mix-up the term ‘Reference Architecture’ with ‘Reference Model’ which puzzled me initially to a large extent. Since then I have been trying to understand the fundamental difference between these two and trying to address the following questions that kept coming up in my mind –
-          Are they same?
-          What is the relationship between the two
-          Do  they co-exist
-          How does one define them and what are the different characteristics

 

So what’s the ‘Reference Model’

•          SOA Reference Model (SOA-RM) is a framework which captures the various capabilities/patterns (e.g. Multi Channel, Process Automation) required to effectively implement a service based information technology platform for achieving a specific business function or a set of business functions
•          These capabilities are grouped in 2 main categories
•          Functional – The capabilities in this category help the services meet a specific business function or a set of business functions
•          Non-functional – The capabilities in this category help the services meet the business functionality within the defined contracts
•          SOA-RM is not directly tied to any technology, standards or concrete implementations. It provides a common understanding that can be used across different implementations
•          Is superset of Reference Architecture: Reference Architecture is derived from Reference Model
•          Highest level of abstraction

 

What’s the ‘Reference Architecture’
-          It defines a consistent vision of SOA throughout the enterprise/business Unit and  supplies the context (for identified patterns) for imposing best practices on development and deployment of SOA
-          It offers an architectural framework for planning SOA-based projects that maximize interoperability and reuse across the enterprise or business unit
-          It represents IT’s architectural alignment (through selection of technology stack) with the business objectives for SOA. In the process, this helps the business case associated with the SOA initiative, including infrastructure requirements (technology, process, etc) and investment priorities.
-          Guided by Reference Model
-          Drives towards concrete Technology Architecture. Considers Protocol, Standards, Specifications etc. Accounts for Business and Technical Requirements, IT/Business Goals

 

Lets take a real life example, to ensure the difference is clear enough. Reference model for a vehicle would be:
·         Need for a vehicle which would help to move from one place to another in a reasonably faster way
·         Must carry people in sitting position
·         Should have a mechanism to control the speed/direction
·         Either automatic or manual manoeuvre
·         May have cruise control

Now taking this to next level i.e. A Reference Architecture. Eventually you might end up having different Reference Architecture depending on the need of the final Vehicle. In this case, the Reference Architecture for a car would address the following
·         Must have seat to ‘carry people’
·         Must have engine ‘for speed’
·         Must have steering wheel to ‘control the direction’
·         Must have wheels ‘for movement’
·         Must be automatic
·         No need to have boots
·         No cruise control

Now concrete implementation would have
·         How many seat car should have – 2 seats or 4 seats or more
·         Engine capacity 1.2ltr or 2.0ltr or more
·         Size of the steering wheel
·         Number of Wheels – 4 or 6 or more

In terms of decreasing level of abstraction: Reference Model --> Reference Architecture (can be many RA for a RM) --> Concrete Implementations

Posted by Shubhankar Sumar at 07:42 PM | Permalink | Comments (0) | TrackBacks (0)

One of the key benefits of BPM solution is that it provides insights into the performance of business processes through Business Activity Monitoring (BAM), and enables process analysts to identify impediments & bottlenecks in performance of the business processes. While that sounds very exciting, getting pointers to the real bottlenecks is not easy. ......

For any meaningful process performance analysis, it’s imperative that the statics gathered through BAM have business significance. For this it is very important that business relevant high impact KPIs are defined during process design. ...

 

To define business relevant KPIs following methodology can be adopted

-         Define key objectives for BPM initiative, such as enhance customer responsiveness, improve quality, expand market share, reduce product design cost etc.

-         Translate the objectives into quantifiable goals, e.g. reduce turnaround time for customer on-boarding from 5 days to 2 days, increase new customer contacts in southern region to 45 per week, …

-         Having defined quantifiable goals for BPM initiative next step would be to identify candidate processes for implementation and categorize these processes into critical process, value add process, support process etc. Categorizing processes into these broad categories helps define goals for the process categories (and thus for each process), e.g. value add process have maximum potential for enhancing market penetration, increase revenue etc, while support processes greatly impact customer satisfaction.

-         Once processes are categorized and process goals are defined, business analysts would analyze each process and from the process goals derive matrices and KPIs for all critical activities

-         Next step is to set milestones and targets for improvements in these KPI measures

 

 

Defining KPIs derived from business goals helps analyze performance of the business processes and facilitates identifying real impediments in achieving the performance goals, which helps redesign the business processes and improve business performance

 

Posted by Dhiren Mehta at 09:33 PM | Permalink | Comments (0) | TrackBacks (0)