Infosys Microsoft Alliance and Solutions blog

Main | October 2006 »

September 29, 2006

BizTalk as an ESB

Mohan commented on my earlier blog and asked my views on BizTalk as an Enterprise Service Bus (ESB). My take is that if we look at the functionality that ESBs are meant to provide, BizTalk does fit in pretty well, however if we look at some of the deployment considerations (like feature specific deployment), then BizTalk doesn't fit in, since we have to install the entire package. We can however, selectively enable certain funtionality by appropriate configuration.

Considering the better processing power we can get these days, I would tend to put more weightage to the features than deployment aspects. The deployment requirements might result in higher costs, but then there could be various other factors forcing organizations to anyway go in for high end servers.

Getting back to feature set, there are a few definitions of ESB out there. At a high level ESB fits in the middleware category and is primarily about intergration applications using standards based bus (messaging engine). Below, I will try and list the key features that we expect from an ESB and then see where BizTalk fits into it. This list is from wikipedia.

  • It is not an implementation of Service Oriented Architecture --> This is more of concept behind ESB and hence we need not map BizTalk to this.
  • It is usually operating system and programming language agnostic; it should enable interoperability between Java and .Net applications, for example --> BizTalk isn't OS or programming language agnostic by itself, but can help interoperate/integrate Java and .net applications via say the SOAP adapter.
  • It uses XML (eXtensible Markup Language) as the standard communication language --> Though BizTalk can work with any document type, most of features are best used when working with XML. In fact, a common myth is that it is necessary to convert all input documents to XML before sending them for processing within BizTalk.
  • It supports Web services standards --> BizTalk has SOAP adapter that helps you work with web services. It also has adapter for working with Web Services Enhancements (WSE) 2.0.
  • It supports messaging (synchronous, asynchronous, point-to-point, publish-subscribe) --> BizTalk inherantly works on pub-sub model. Messages are published into the BizTalk Message box database and orchestrations and send ports subscribe to them. BizTalk also support message routing, message translation, short and long running transactions and various other message delivery related features. Via use of specific adapters like MSMQ, asynchronous message delivery can be easily achieved.
  • It includes standards-based adapters (such as J2C/JCA) for supporting integration with legacy systems --> BizTalk has a plethora of out-of-box and third party adapters that can be used to integrate with various legacy and LOB systems. A list of such adapters is available here. BizTalk also can work closely with Host Integration Server to connect to mainframes.
  • It includes support for service orchestration & choreography --> Orchestration is one of the key features of BizTalk. BizTalk conforms to Business Process Execution Language (BPEL) standards. Not only does BizTalk provides orchestration execution features, it also has a rich designer support well integrated with Visual Studio 2005. Additionally, it has Orchesration Designer for Business Analysts (ODBA) that integrates with Visio and business users can use it to designing orchestrations.
  • It includes intelligent, content-based routing services (itinerary routing) --> BizTalk supports content based routing via promoted properties concept.
  • It includes a standardized security model to authorize, authenticate, and audit use of the ESB --> BizTalk has mechanisms in place to provide security. It can even work with Certificates and supports ability to resolve incoming requests via these certificates.
  • It includes transformation services (often via XSLT) between the format of the sending application and the receiving application, to facilitate the transformation of data formats and values --> BizTalk has extensive support for data transformation via the Mapper functionality. It provides host of funtoids out-of-box to manage the transformations and also includes ability to custom script additionaly functionality that isn't available via the functoids. For this transformation and schema creation itself, BizTalk again has great tool that integrates with VS. We don't have to worry about writing XSLT manually.
  • It includes validation against schemas for sending and receiving messages --> The pipelines in BizTalk provide mechanisms to validate messages based on published schemas.
  • It can uniformly apply business rules, enrichment of the message from other sources, splitting and combining of multiple messages, and the handling of exceptions --> BizTalk supports interchange handling (splitting messages) and also allows aggregation of messages. Via pipelines and mappers we can work with the messages and modify them. There is good business rules engine (BRE) support via which we can define rules and policies and use them in orchestrations. The policies are managed external to the orchestration and hence can be changed at any time without having to touch the orchestartions. Since BizTalk supports long running transactions, it also has mechanisms for compensating transactions.
  • It can conditionally route or transform messages based on a non-centralized policy - meaning that no central rules engine needs to be present --> Already discussed this above in terms of content based routing and mapping functionality.
  • It is monitored for various SLA (Service-Level Agreement) thresholds message latency and other characteristics described in a Service Level Agreement --> BizTalk has Business Activity Monitoring (BAM) features that provide a portal based approach to track the messages and orchestration progress. Additionally, extensive performance monitor counters are available that can be used to get additional information about message processing speeds etc.
  • It (often) facilitates "service classes," responding appropriately to higher and lower priority users --> This is managed more by appropriate deployment and configuring specific features on specific servers. 
  • It supports queuing, holding messages if applications are temporarily unavailable --> BizTalk can work with adapters like MSMQ that provide such features. It also has capabilities of setting retry count and retry intervals.
  • It is comprised of selectively deployed application adapters in a (geographically) distributed environment --> BizTalk has to be deployed in full, thought post that selective features can be enabled on the machines.

So in a nutshell, as stated earlier, from feature standpoint BTS does offers features that make it a worthy ESB.

September 26, 2006

Asynchronous Programming Model in .NET Framework 2.0 - Part III,,,

In my previous blogs “Asynchronous Programming Model in .NET Framework 2.0 Part I and Part II, I had explained the basics of EBAPM and about various approaches to call an operation in a separate thread. In this final sequel to those blogs, I am going to explain how an operation should be designed so that it would be invoked asynchronously based on EBAPM. It is better to recollect that EBAPM provides more flexibility to an asynchronous operation. EBAPM provides flexibility in canceling an ongoing asynchronous operation. It also supports retrieving incremental result and status from an asynchronous operation. Apart from those, it is possible to invoke the same operation more than once asynchronously if application logic requires that.

 

To explain how to code an operation to provide such flexibilities, I have created a simple application that retrieves the private data such as address, credit card history, employment details, medical history and travel details of a person based on the unique identification of person such as SSN.

However, as this SpyHard application is just to show the capabilities of EBAPM and I am not aware of the central repository wherein I can retrieve all such private data, I have used Thread.Sleep() so liberally in my coding as a substitute to implementing that logicSmile. And it also does not have logic to do the name resolution from supplied SSN so you will get to see the same hard coded data of mine for any SSN even if there is no SSN (So, don’t try to get the private details of your friends and foes through this applicationSmile). So, This SpyHard application is as dumb and clumsy as the hero (my favorite comedian – Leslie Nielsen) of the “SpyHard” movie which is a great spoof on many other adrenalin pumping action movies. But what we are interested in this code, is to know, if our application needs to implement an operation that is time consuming one, how to implement that based on EBAPM, so that client can invoke it asynchronously and get the incremental details about the operation easily. For that purpose, I have developed a bit contrived operation that retrieves private details of a person, in a time consuming fashion. This operation is also contrived to provide increment result such as name, address, credit card details etc and increment status such as the percentage of completed work.

We need following constructs in order to create an asynchronous operation for BackgroundCheck information retrieval.

BackgroundCheckAsync method

This method implements the logic to help invoking it asynchronously. Two things to be noted in this method are the second parameter – taskId and AsyncOperation class. While this method takes SSN as the first parameter, some unique id needs to be passed as another parameter that helps EBAPM to invoke this method multiple times asynchronously from client side.  AsyncOperation class is one that represents and keeping track of life time of each asynchronous operation that is distinguished by the unique id that I mentioned above.  AsyncOperation object for each asynchronous operation should be stored in the HybridDictionary to ensure that this asynchronous operation should not be invoked more than once with the same unique id. AsyncOperation provides two methods Post and PostOperationCompleted methods that are key methods to inform details such as increment result as well as progress and completion of execution of asynchronous operation, to client applications. These two methods will be invoked within an asynchronous operation. The Post method is to inform the client about the intermediate result and progress of the operation. Basically, this Post method triggers an event, to which, clients will be subscribing to, to get the notification. The PostOperationCompleted method is to inform the client about the completion of execution of asynchronous operation, by triggering an event from within asynchronous operation. The events to be triggered by both Post and PostOperationCompleted methods of AsyncOperation are wrapped inside a delegate of type SendOrPostCallback in System.Threading namespace. You should also notice that, the business logic of this method, has been wrapped up in another private method – RetrieveBackgroundCheckInfo that invokes Post and PostOperationCompleted methods appropriately within it. This private method will be invoked asynchronously in BackgroundCheckAsync method through BeginInvoke of a delegate that represents RetrieveBackgroundCheckInfo method.

BackgroundCheckAsyncCancel method

This method helps to cancel the execution of BackgroundCheckAsync method. It does nothing but removing the AsyncOperation object that keeps track of the life time of the asynchronous operation from HybridDictionary collection.

BackgroundCheckCompleted event

Client application will subscribe to this event to get notification on the completion of asynchronous operation. This is an event, which gets triggered by PostOperationCompleted method of AsyncOperation object within the asynchronous method implementation.

BackgroundCheckProgressChanged event

Client application will subscribe to this event to get increments result value and status of ongoing asynchronous operation. This is an event, which gets triggered by Post method of AsyncOperation object within the asynchronous method implementation.

BackgroundCheckCompletedEventArgs class

This class is derived from “AsyncCompletedEventArgs” class of System.ComponentModel namespace. This class represents the result of the BackgroundCheckAsync method.

BackgroundCheckProgressChangedEventArgs class

This class is derived from “”ProgressChangedEventArgs” class of System.ComponentModel namespace. This class represents the percentage of completion of an asynchronous operation.

BackgroundCheckCompletedEventHandler delegate

It provides the method signature of a delegate that needs to be bound with BackgroundCheckCompleted event at client application.

BackgroundCheckProgressChangedEventHandler delegate

It provides the method signature of a delegate that needs to be bound with PrivateDetailProgressChanged event at client application.

I know that it may seem to be bit of too much to be grasped at one take, as these have been explained in bits and pieces. Seeing the complete code that is given below will provide better context and understanding of all above items.

using System;
using System.Collections.Generic;
using System.Text;
using System.ComponentModel;
using System.Threading;
using System.Collections.Specialized;
namespace SpyHardLibrary
{
    public class SpyHard
    {
        private HybridDictionary userStateToLifetime = new HybridDictionary();
        private delegate void WorkerHandler(string ssn, AsyncOperation asyncOp);
        public delegate void BackgroundCheckCompletedEventHandler(object sender, BackgroundCheckCompletedEventArgs e);
        public delegate void BackgroundCheckProgressChangedEventHandler(BackgroundCheckProgressChangedEventArgs e);
        public event BackgroundCheckCompletedEventHandler BackgroundCheckCompleted;
        public event BackgroundCheckProgressChangedEventHandler BackgroundCheckProgressChanged;
        private SendOrPostCallback onProgressReportDelegate;
        private SendOrPostCallback onCompletedDelegate;
        public SpyHard()
        {
            onCompletedDelegate = new SendOrPostCallback(BackgroundCheckFunctionCompleted);
            onProgressReportDelegate = new SendOrPostCallback(BackgroundCheckReportProgress);
        }
        private void BackgroundCheckFunctionCompleted(object state)
        {
            BackgroundCheckCompletedEventArgs e = state as BackgroundCheckCompletedEventArgs;
            OnBackgroundCheckCompleted(e);
        }
        private void BackgroundCheckReportProgress(object state)
        {
            BackgroundCheckProgressChangedEventArgs e = state as BackgroundCheckProgressChangedEventArgs;
            OnBackgroundCheckProgressChanged(e);
        }
        protected void OnBackgroundCheckCompleted(BackgroundCheckCompletedEventArgs e)
        {
            if (BackgroundCheckCompleted != null)
            {
                BackgroundCheckCompleted(this, e);
            }
        }
        protected void OnBackgroundCheckProgressChanged(BackgroundCheckProgressChangedEventArgs e)
        {
            if (BackgroundCheckProgressChanged != null)
            {
                BackgroundCheckProgressChanged(e);
            }
        }
     
        public void BackgroundCheck(string ssn, ref string nameDetails, ref string addressDetails, ref string ccDetails, ref string employmentDetails, ref string healthDetails, ref string travelDetails)
        {
            string strName = string.Empty;
            string strAddress = string.Empty;
            string strCCHistory = string.Empty;
            string strEmploymentHistory = string.Empty;
            string strHealthHistory = string.Empty;
            string strTravelHistory = string.Empty;
            //logic to retrieve Name
            Thread.Sleep(10000);
            strName = "Ganesan Krishnamurthy";
           
            //logic to retrieve Address
            Thread.Sleep(10000);
            strAddress = "Infosys Technologies Limited, Bangalore, India";
          
            //logic to retrieve Credit Card Details
            Thread.Sleep(10000);
            strCCHistory = "Not So Bad!!!";
            //logic to retrieve Employment Details
            Thread.Sleep(10000);
            strEmploymentHistory = "12+ years of IT experience";
            //logic to retrieve Health Details
            Thread.Sleep(10000);
            strHealthHistory = "Robust";
            //logic to retrieve Travel History
            Thread.Sleep(10000);
            strTravelHistory = "Had been to Melbourne, Australia in 2005";
            nameDetails = strName;
            addressDetails = strAddress;
            ccDetails = strCCHistory;
            employmentDetails = strEmploymentHistory;
            healthDetails = strHealthHistory;
            travelDetails = strTravelHistory;
        }
        public void BackgroundCheckAsync(string ssn, object taskId)
        {
            AsyncOperation asyncOp = AsyncOperationManager.CreateOperation(taskId);
            lock (userStateToLifetime.SyncRoot)
            {
                if (userStateToLifetime.Contains(taskId))
                {
                    throw new Exception("taskId is already existing!!!");
                }
                else
                {
                    userStateToLifetime[taskId] = asyncOp;
                }
            }
            WorkerHandler workerHandler = new WorkerHandler(RetrieveBackgroundCheckInfo);
            workerHandler.BeginInvoke(ssn, asyncOp, null, null);
        }
        public void BackgroundCheckAsyncCancel(object taskId)
        {
            AsyncOperation asyncOp = userStateToLifetime[taskId] as AsyncOperation;
            if (asyncOp != null)
            {
                lock (userStateToLifetime)
                {
                    userStateToLifetime.Remove(taskId);
                }
            }
           
        }
        private void RetrieveBackgroundCheckInfo(string ssn, AsyncOperation asyncOp)
        {
            ProgressChangedEventArgs pceArgs = null;
            string strName = string.Empty;
            string strAddress = string.Empty;
            string strCCHistory = string.Empty;
            string strEmploymentHistory = string.Empty;
            string strHealthHistory = string.Empty;
            string strTravelHistory = string.Empty;
            pceArgs = new BackgroundCheckProgressChangedEventArgs("", 5, asyncOp.UserSuppliedState);
            asyncOp.Post(onProgressReportDelegate, pceArgs);
            //logic to retrieve Name
            Thread.Sleep(10000);
            strName = "Ganesan Krishnamurthy";
            pceArgs = new BackgroundCheckProgressChangedEventArgs(strName, 15, asyncOp.UserSuppliedState);
            asyncOp.Post(onProgressReportDelegate, pceArgs);
            //logic to retrieve Address
            Thread.Sleep(10000);
            strAddress = "Infosys Technologies Limited, Bangalore, India";
            pceArgs = new BackgroundCheckProgressChangedEventArgs(strAddress, 30, asyncOp.UserSuppliedState);
            asyncOp.Post(onProgressReportDelegate, pceArgs);
            //logic to retrieve Credit Card Details
            Thread.Sleep(10000);
            strCCHistory = "Not So Bad!!!";
            pceArgs = new BackgroundCheckProgressChangedEventArgs(strCCHistory, 45, asyncOp.UserSuppliedState);
            asyncOp.Post(onProgressReportDelegate, pceArgs);
            //logic to retrieve Employment Details
            Thread.Sleep(10000);
            strEmploymentHistory = "12+ years of IT experience";
            pceArgs = new BackgroundCheckProgressChangedEventArgs(strEmploymentHistory, 60, asyncOp.UserSuppliedState);
            asyncOp.Post(onProgressReportDelegate, pceArgs);
            //logic to retrieve Health Details
            Thread.Sleep(10000);
            strHealthHistory = "Robust";
            pceArgs = new BackgroundCheckProgressChangedEventArgs(strHealthHistory, 75, asyncOp.UserSuppliedState);
            asyncOp.Post(onProgressReportDelegate, pceArgs);
            //logic to retrieve Travel Details
            Thread.Sleep(10000);
            strTravelHistory = "Had been to Melbourne, Australia in 2005";
            pceArgs = new BackgroundCheckProgressChangedEventArgs(strTravelHistory, 100, asyncOp.UserSuppliedState);
            asyncOp.Post(onProgressReportDelegate, pceArgs);
            Exception error = null;
            bool cancelled = false;
            object userState = asyncOp.UserSuppliedState;
            BackgroundCheckCompletedEventArgs pdceArgs = new BackgroundCheckCompletedEventArgs(strName, strAddress, strCCHistory, strEmploymentHistory, strHealthHistory, strTravelHistory, error, cancelled, userState);
            asyncOp.PostOperationCompleted(onCompletedDelegate, pdceArgs);
        }
    }
    public class BackgroundCheckCompletedEventArgs : AsyncCompletedEventArgs
    {
        private string strNameValue = string.Empty;
        private string strAddressValue = string.Empty;
        private string strCCHistoryValue = string.Empty;
        private string strEmploymentHistoryValue = string.Empty;
        private string strHealthHistoryValue = string.Empty;
        private string strTravelHistoryValue = string.Empty;
        public BackgroundCheckCompletedEventArgs(string strName, string strAddress, string strCCHistory, string strEmploymentHistory, string strHealthHistory, string strTravelHistory, Exception error, bool cancelled, object userState)
            : base(error, cancelled, userState)
        {
            this.strNameValue = strName;
            this.strAddressValue = strAddress;
            this.strCCHistoryValue = strCCHistory;
            this.strEmploymentHistoryValue = strEmploymentHistory;
            this.strHealthHistoryValue = strHealthHistory;
            this.strTravelHistoryValue = strTravelHistory;
        }
        public string NameDetails
        {
            get
            {
                return strNameValue;
            }
        }
        public string AddressDetails
        {
            get
            {
                return strAddressValue;
            }
        }
        public string CCHistoryDetails
        {
            get
            {
                return strCCHistoryValue;
            }
        }
        public string EmploymentHistoryDetails
        {
            get
            {
                return strEmploymentHistoryValue;
            }
        }
        public string HealthHistoryDetails
        {
            get
            {
                return strHealthHistoryValue;
            }
        }
        public string TravelHistoryDetails
        {
            get
            {
                return strTravelHistoryValue;
            }
        }
    }
    public class BackgroundCheckProgressChangedEventArgs : ProgressChangedEventArgs
    {
        private string strDetailsValue = string.Empty;
        public BackgroundCheckProgressChangedEventArgs(string strDetails, int progressPercentage, object userState)
            : base(progressPercentage, userState)
        {
            this.strDetailsValue += strDetails;
        }
        public string DetailsValue
        {
            get
            {
                return strDetailsValue;
            }
        }
    }
   
}

Though it is bit tedious to implement such asynchronous operation supporting classes, outcome of high flexibility because of this model, really outweighs the effort taken to implement this class. To understand this point better, I have implemented the following windows form application that consumes the above class – its asynchronous operation.

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Text;
using System.Windows.Forms;
using SpyHardLibrary;
namespace SpyHardWinApp
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }
        private void button1_Click(object sender, EventArgs e)
        {
            SpyHard spyHard = new SpyHard();
            spyHard.BackgroundCheckProgressChanged += delegate(BackgroundCheckProgressChangedEventArgs pdpceArgs)
            {
                progressBar1.Value = pdpceArgs.ProgressPercentage;
                if(string.IsNullOrEmpty(textBoxName.Text)){
                    textBoxName.Text = pdpceArgs.DetailsValue;
                }
                else if(string.IsNullOrEmpty(textBoxAddress.Text)){
                    textBoxAddress.Text = pdpceArgs.DetailsValue;
                }
                else if (string.IsNullOrEmpty(textBoxCCHistory.Text))
                {
                    textBoxCCHistory.Text = pdpceArgs.DetailsValue;
                }
                else if (string.IsNullOrEmpty(textBoxEmploymentHistory.Text))
                {
                     textBoxEmploymentHistory.Text = pdpceArgs.DetailsValue;
                }
                else if (string.IsNullOrEmpty(textBoxHealthHistory.Text))
                {
                    textBoxHealthHistory.Text = pdpceArgs.DetailsValue;
                }
                else
                {
                    textBoxTravelHistory.Text = pdpceArgs.DetailsValue;
                }
            };
            spyHard.BackgroundCheckCompleted += delegate(object sender1, BackgroundCheckCompletedEventArgs pdceArgs)
            {
                textBoxStatus.Text = "Operation Completed!!!";
            };
            spyHard.BackgroundCheckAsync(textBoxSSN.Text, "1");
        }
       
    }
}

The UI of the application will look like as given below after the execution of the asynchronous operation BackgroundCheckAsync.

SpyHard Application UI image

I hope this sample code in this and in my previous blogs on Asynchronous Programming model throw enough light on the concepts behind event based asynchronous programming model and various approaches in .NET Framework 2.0 to invoke an operation in a separate thread, for understanding the asynchronous model that is very essential to develop better performance as well as better UI experience applications.

Vista Performance Booster

This is one really cool feature I came across in Vista. If you wonder, how you would upgrade the memory in your Laptop to serve up the hunger of Vista, you have better things to do than buying up a 1GB memory for $300 and panting with it.

Windows Vista introduces a new concept in adding memory to a system. Windows ReadyBoost lets you use a conventaional USB-based flash memory device, such as a USB thumb drive, to improve system performance without actually upgrading the memory. Called as Windows ReadyBoost, this technology improves performance because it can retrieve data kept on the flash memory more quickly than it can retrieve data kept on the hard disk, decreasing the time needed to wait for PC to respond.

I have tried extending a laptop with 1GB of RAM using a 2GB fast USB thumb drive and the performance really took off. This is certainly something to think about for all you laptop users out there that are considering running Windows Vista.

Troubleshooting BizTalk Applications

Microsoft recently published a pretty exhaustive guide for developers working with BizTalk 2006. It has information on best practices to build a BizTalk application and contains lot of troubleshooting tips. A must have for every BizTalk developer. Get the guide here.

September 21, 2006

Asynchronous Programming Model in .NET Framework 2.0 - Part II,,,

Let us continue from where we left off in “Asynchronous Programming Model in .NET Framework 2.0 – Part I”. But, in this blog, instead of delving deep into EBAPM implementation details, I will step into various ways to achieve asynchronous invocation of method calls, in .NET Framework 2.0. That will give us better perspective of Asynchronous programming model. Asynchronous programming model can be simplistically defined as invoking methods in separate threads from main UI or primary threads.

 

.NET, being a rich framework, it provides more than one option to achieve it. In fact, a few options that we may be familiar with, even without knowing that they invoke methods in different threads than the main ones.

 

Asynchronous invocation is possible through the following approaches, apart from the ones that we have seen in my previous blog,

  1. through QueueUserWorkItem method of ThreadPool class
  2. through an explicitly created thread
  3. through Timer class
  4. through BackgroundWorker class

Out of these, BackgroundWorker class is new to .NET Framework and it has been introduced in .NET Framework 2.0. While the approaches are more or less the same in invocation part, new asynchronous features in .NET Framework 2.0 such as EBAPM, BackgroundWorker class provide more flexibility in terms of canceling the ongoing asynchronous operations and getting the incremental status of background running operations so that progress bar updates is possible at UI side.

I have created code snippets for all these approaches in the following windows form application that will have the UI as given below.

Asynchronous Invocations in .NET Fwk 2.0 

And I have invoked the following very primitive Web Service in that windows Form application.

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Text;
using System.Windows.Forms;
using System.Threading;
namespace BackgroundWorkerWinApp
{
    public partial class Form1 : Form
    {
        BackgroundWorker bgw;
        BackgroundWorkerWinAppWS.BackgroundWorkerWinAppWS proxy;
       
        public Form1()
        {
            InitializeComponent();
            proxy = new BackgroundWorkerWinAppWS.BackgroundWorkerWinAppWS();
            #region Asynchrouns invocation through WS Proxy - event binding
            proxy.HelloWorldCompleted += delegate(object sender, BackgroundWorkerWinAppWS.HelloWorldCompletedEventArgs e)
            {
                textBoxStatus.Text = e.Result.ToString();
                textBoxStatus.BackColor = Color.Gray;
            };
            #endregion
            #region Asynchronous invocation through BackgroundWorker - event binding
            bgw = new BackgroundWorker();
            bgw.DoWork += delegate(object sender, DoWorkEventArgs e)
            {
                textBoxStatus.Text = proxy.HelloWorld();
                textBoxStatus.BackColor = Color.Green;
            };
            #endregion
        }
        #region Synchronous invocation of Web Service
        private void btnInvokeWebService_Click(object sender, EventArgs e)
        {
            textBoxStatus.Text = proxy.HelloWorld();
            textBoxStatus.BackColor = Color.Blue;
        }
        #endregion
        #region Asynchronous invocation through BackgroundWorker class
        private void btnInvokeWebServiceThroughBGW_Click(object sender, EventArgs e)
        {
           bgw.RunWorkerAsync();
        }
        #endregion
        #region Asynchronous invocation through Web Service proxy
        private void btnInvokeWSAsyncThroughWSProxy_Click(object sender, EventArgs e)
        {
            proxy.HelloWorldAsync();
        }
        #endregion
       
        #region Asynchronous Invocation through QueueUserWorkItem
        private void QueueUserWorkItemWSInvocation(object state)
        {
            textBoxStatus.Text = proxy.HelloWorld();
            textBoxStatus.BackColor = Color.GreenYellow;
        }
        private void btnInvokeWSAsyncThroughQueueUserWorkitem_Click(object sender, EventArgs e)
        {
            ThreadPool.QueueUserWorkItem(new WaitCallback(QueueUserWorkItemWSInvocation));
        }
        #endregion
        #region Asynchronous invocation through custom thread
        private void ThreadStartWSInvocation()
        {
            textBoxStatus.Text = proxy.HelloWorld();
            textBoxStatus.BackColor = Color.Honeydew;
        }
       
        private void btnInvokeWSAsyncThroughCustomThread_Click(object sender, EventArgs e)
        {
            ThreadStart threadStart = new ThreadStart(ThreadStartWSInvocation);
            Thread thread = new Thread(threadStart);
            //Thread, by default, behaves like a foreground thread
            //thread.IsBackground = true;
            thread.Start();
        }
        #endregion
        #region Asynchronous invocation through Timer
        private void TimerBasedWSInvocation(object state)
        {
            textBoxStatus.Text = proxy.HelloWorld();
            textBoxStatus.BackColor = Color.Indigo;
        }
        private void btnInvokeWSAsyncThroughTimer_Click(object sender, EventArgs e)
        {
            System.Threading.Timer timer = new System.Threading.Timer(new TimerCallback(TimerBasedWSInvocation));
            timer.Change(0, Timeout.Infinite);
        }
        #endregion
    }
}
 

Now, Let us go through the details of each approach.

Asynchronous invocation through QueueUserWorkItem.

The primary reason to use QueueUserWorkItem method of ThreadPool class is to invoke a method asynchronously using an available thread from CLR thread pool. In .NET Fwk 1.x, it is the recommended one for invoking an operation asynchronously. The following code snippet shows how to use QueueUserWorkItem method.

ThreadPool.QueueUserWorkItem(new WaitCallback(QueueUserWorkItemWSInvocation));

And WaitCallback delegate in our code, given below, invokes the web service synchronously.

private void QueueUserWorkItemWSInvocation(object state)
{
      textBoxStatus.Text = proxy.HelloWorld();
}
 

Asynchronous invocation through an explicit thread.

Though the usage of QueueUserWorkItem is highly preferred, we still need to invoke operations by a thread that is created explicitly by us, when we need to have some control on the thread that executes the operation. For instance, when we use a thread pool thread, we have no control over the type of thread (whether it is a foreground or background thread), priority of thread and apartment state of the thread (by default, CLR threads are set to MTA). And another important reason to go for explicit thread creation is that we may not want to tie up a thread from common thread pool for a very long running operation, because threadpool will be shared across applications. When we need to exert control on any of the above mentiones ones, we have to resort to creating an explicit thread.

The following is the delegate that is required to be passed as a parameter into ThreadStart class which is required to create a thread.

private void ThreadStartWSInvocation()
{
        textBoxStatus.Text = proxy.HelloWorld();
}
    
  
private void btnInvokeWSAsyncThroughCustomThread_Click(object sender, EventArgs e)
{
         ThreadStart threadStart = new ThreadStart(ThreadStartWSInvocation);
         Thread thread = new Thread(threadStart);           
         thread.Start();
}

Asynchronous invocation through Timer.

There is another way to invoke an operation in a separate thread, is by using a timer class. You might know that there are timer classes in three different namespaces.    

  1. Timer class in System.Threading namespace
  2. Timer class in System.Windows.Forms namespace
  3. Timer class in System.Timers namespace

Essentially, a timer is to execute an operation periodically in background using a separate thread. The purpose of existence of each one of them is distinct. The best explanation that I have read for each of them is from “CLR via C# - Applied Microsoft .NET Framework Programming” by Jeffrey Richter. According to that book, System.Windows.Forms.Timer associates the timer with the calling thread, and thus ensures that an operation is invoked through that timer in the same thread (main UI thread) that created the timer. System.Timers.Timer is a wrapper over System.Threading.Timer and this timer is specially designed to be used in design surface of windows form applications. And out of these three, System.Threading.Timer is the recommended one to invoke an operation in background periodically. So, I have used System.Threading.Timer in my code snippet as given below.

The following delegate invokes web service in synchronous manner.

private void TimerBasedWSInvocation(object state)
{
        textBoxStatus.Text = proxy.HelloWorld();
}

The following method shows how to use the timer to invoke the above delegate.

It is to be noted that first parameter of change() method is to denote how soon, from the time of invocation of timer, the method needs to be executed (here, 0 means immediately) and second parameter is to denote, how frequently the operation needs to be executed repeatedly (Timeout.Infinite means the operation is to be executed only once).

private void btnInvokeWSAsyncThroughTimer_Click(object sender, EventArgs e)
{
        System.Threading.Timer timer = new System.Threading.Timer(new TimerCallback(TimerBasedWSInvocation));
        timer.Change(0, Timeout.Infinite);
}

Asynchronous invocation through BackgroundWorker class.

BackgroundWorker class is introduced in .NET Framework 2.0, mainly, to invoke long running operations in Windows form applications, without worrying about using a delegate in callback method (to switch over to main UI thread from worker thread, as happened in Begin/End<Method> approach) in order to update the UI controls. As I had mentioned earlier, it is more flexible and powerful than .NET Fwk 1.x approaches, as it supports canceling as well as retrieving the incremental status about the ongoing asynchronous operations. To make the examples simple, I have not coded for those features here.

The BackgroundWorker class is instantiated and a delegate that is going to call web service synchronously is linked to a predefined event “DoWork” of BackgroundWorker object. The code snippet is given below.

BackgroundWorker bgw = new BackgroundWorker();
bgw.DoWork += delegate(object sender, DoWorkEventArgs e)
{
      textBoxStatus.Text = proxy.HelloWorld();              
};

private void btnInvokeWebServiceThroughBGW_Click(object sender, EventArgs e)
{
      bgw.RunWorkerAsync();
}

Having seen those different approaches to call an operation asynchronously, let us move to the implementation details of EBAPM in my next blog. But, an interesting thing that we have to understand with related to the type of thread (foreground or background) is the process will be terminated only after the completion of execution of all foreground threads in it. For example, when we create an explicit thread, it will be created as a foreground thread by default. It means, process will not be torn down till the operation executed by this thread is completed. But an explicitly created thread can be set to be a background thread, by setting that thread’s property - IsBackground to true. I tested this code to know how threads in other approaches behave with respect to the termination of application. What I did was, I executed this windows form application along with Windows Task Manager. I clicked on either one of the buttons in the UI and immediately close down the application. Then I checked how long it takes for the application instance to disappear in the windows task manager. What I have noticed is only explicitly created thread is a foreground thread and all other approaches utilize background threads. Then I set the explicitly created thread to be background thread by setting IsBackground to true and repeated the experiment. Yes, now process got terminated immediately once I shut down the application.

 

 

September 19, 2006

Asynchronous Programming Model in .NET Framework 2.0 - Part I,,,

One of my favorite features in my book “101 reasons to go for .NET Framework 2.0” is Event based asynchronous mechanism. I had mentioned that asynchronous mechanism in .NET Framework 2.0 is more based on events rather than relying on delegates. We all know that event and delegate are distinct ones but differences between them are very subtle as one depends on another.

 

Events are basically to trigger a special occurrence in an application, and these special occurrences should be notified to subscribers who are interested in them. In subscribers’ side, invariably the delegates (if loosely defined, managed functional pointers) are being used to bind callback methods to these events. Thus, implementation of event based programming depends on both events and delegates. Asynchronous programming model is one that requires this event/delegate mechanism.

.NET Framework 1.x makes it easy to invoke any synchronous method in asynchronous manner, through internal creation the asynchronous method invocation mechanisms for them, by default. The widely used asynchronous pattern, through Begin<MethodName> and End<MethodName>, heavily depends on delegates. Once you used to the intricacies of this asynchronous model such as IAsyncResult, AsyncCallback and how to retrieve the result from within callback methods through End<MethodName> by passing IAsyncResult object (that is returned by Begin<MethodName> ) as a parameter into that method, you might start liking its simplicity. However, you might also have felt that this simple mechanism can be made simpler by letting the developers not to deal with certain threading related aspects such as waiting for a signal from the worker thread to proceed further in main thread or not to deal with bit of tedious coding practice to retrieve the result by invoking the mandatory End<MethodName>. (You might also aware of dreadful consequences of not invoking End<MethodName> ). New event based asynchronous programming model (if you are an acronymphil, start using catchy EBAPM now onwardsSmile) in .NET Framework 2.0, makes asynchronous programming simpler by abstracting out those above mentioned nagging points. And it also provides better support to invoke same asynchronous calls multiple times (looking back at DBAPM – delegates based asynchronous programming mode – another tidbit to acronymphils – it is tedious to do that as you have to pass appropriate and specific IAsyncResult objects of Begin<MethodName> into End<MethodName> methods), to cancel as well as to get the incremental status on the ongoing async operation.

In .NET Framework 2.0, to invoke methods of components are based on this EBAPM model, asynchronously, you no longer require invoking the famous pair of Begin and EndInvoke methods. Instead, you will invoke <MethodName>Async().No more dealing with that nice interface “IAsyncResult”. Retrieving the result from the asynchronously invoked method is also not through the invocation of End<MethodName> .  To be brief, developer need not bother too much about the threading aspects involved in asynchronous programming model. Though asynchronous invocation is so easy and clutter-free, implementing asynchronous mechanism into your component or class requires thorough understanding of new EBAPM concepts and newly introduced classes such as AsyncOperation, AsyncOperationManager, etc. I will provide details on them in my subsequent blogs.

However, before we delve deep into those concepts, we have to see how easy it would be to invoke the methods of components that are based on new EBAPM. And ASP.NET Web Service in .NET Framework 2.0 is one of such components. What it means is, don’t get surprised not to see those Begin and End methods syntax in the web service proxy. Instead you would see a method “<MethodName>Async” and an event “<MethodName>Completed”.

So, let us build a simple web service in .NET Framework. The following is the code snippet of that simple web service.

using System;
using System.Web;
using System.Web.Services;
using System.Web.Services.Protocols;
using System.Threading;
[WebService(Namespace = "http://tempuri.org/")]
[WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]
public class OrderProcessingWebService : System.Web.Services.WebService
{
    public OrderProcessingWebService () {
       
    }
    [WebMethod]
    public string ProcessAnOrder() {
        Thread.Sleep(10000);
        return "Order Processed!!!";
    }
   
}

Now, let us build a windows form application that consumes this web service. You have to use “Add Web References” to create the proxy for the web service.

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Text;
using System.Windows.Forms;
using System.Threading;
namespace MyAsyncWebServiceWindowApp
{
    public partial class Form1 : Form
    {
        OrderProcessingWS.OrderProcessingWebService proxy = null;
        public Form1()
        {
            InitializeComponent();
            proxy = new OrderProcessingWS.OrderProcessingWebService();
            proxy.ProcessAnOrderCompleted += delegate(object sender,  OrderProcessingWS.ProcessAnOrderCompletedEventArgs e)
            {
                  callbackThreadTextBox.Text = Thread.CurrentThread.ManagedThreadId.ToString();
                  statusTextBox.Text = e.Result.ToString();
            };          
        }
       
        private void button1_Click(object sender, EventArgs e)
        {
            uiThreadTextBox.Text = Thread.CurrentThread.ManagedThreadId.ToString();
            statusTextBox.Text = "Order is to be processed, asynchronously!!!";
            proxy.ProcessAnOrderAsync();
        }
    }
}

Now, you can compare this approach with previous one in .NET Framework 1.x. If you liked the previous asynchronous model very much, you would miss things such as Begin and End Methods, IAsyncResult interface and dealing with WaitHandle, but you would not deny that this approach is more intuitive and clutter-free. And you would also have noticed in the above code, that one tiny but a neat feature in .NET Framework 2.0, anonymous method, eliminates the necessity of creating a separate method to be linked with a delegate in event binding.

And another important thing that you would have noticed is that you need not worry about using delegate to switch over from worker thread to main UI thread, when you try to update the UI controls from within callback method, in this approach, as you have to do in DBAPM in .NET Framework 1.x in Win Form applications.

In my next blog, let us see more conceptual and technical details of EBAPM. Hope we would say good bye to DBAPM soon Smile.

Extending the reasons to go for .NET Framework 2.0,,,

While I could definitely say that I really enjoyed writing the book “101 reasons to go for .NET Framework 2.0”, I would also like to say that more technical details should have been provided at each feature level. For example, when I mentioned that GC has been enhanced to handle unmanaged memory, in better way, I really wanted to provide more information on internals of GC such as its generation based collection mechanisms, finalization list, freachable queue; issues associated with finalize method and weak reference types etc.

 

This is true not only for GC but also for many of my favorite features such as new code behind model, Threading, Asynchronous programming, Transaction model and various other security related features. However, the aim of the book is not to make it in-depth one (and who can successfully compete with the ilk of Jeffrey Richter and Don Box in explaining the .NET Framework so lucidly!!!) but to provide compilation of interesting features with some background information to architects, development and business users community so that they can decide whether to go for .NET Framework 2.0 or not. Because of that, some of features may seem bit sketchy in their explanation, in that book. I would like to take on such features in a series of blogs on .NET Framework 2.0 along with .NET Framework 3.0 related blogs.  So, be tuned to those blogs and let me know your feedbacks.

September 18, 2006

BizTalk 2006 Documentation

Download the latest BizTalk 2006 documentation (published on 13 Sept 2006) from http://www.microsoft.com/downloads/details.aspx?FamilyID=3294ddaf-9f67-409f-a92d-2f6070dc0d1a&displaylang=en

September 16, 2006

Database Migration

Database Migration Solutions  team of SI consists of folks with rich experience in database industry working geographically across  various locations providing end-to-end Migration, Upgrade and Consolidation solutions for various databases . This is done with reduced downtime, well-managed change of environment using industry best practices, knowledge base, Predictability and Global Delivery Model to our Clients.

We have been working on heterogeneous/homogeneous database migrations across different platforms and have certified professionals having specialized skillset on SQL Server migrations .

Some of our key-value differentiators in providing solutions are:

In-house tools and methodologies developed for database migration
Database Migration Centre Of Excellence
Certified Database Administrators with migration expertise
Support from Alliance partners like Microsoft, Oracle, IBM etc


Going forward we would be sharing our learning’s which we gathered while executing migration projects.

 

 

September 15, 2006

Document your BizTalk Installation

I recently downloaded the UK SDC BizTalk 2006 Documenter and ran it on my local BizTalk installation. The output CHM is amazing and captures quite a bit of information.

The good part is that it is able to provide details like maps, schemas, variables, messages, ports etc within an orchestration. This helps when you have multiple orchestrations within a single application, since the BizTalk Admin console groups only at application level and not at individual orchestration level inside it.

A few minor issues however were

  1. It didn't document all the policies. I ran it a couple of times and it picked only 1 policy. It did list all the vocabularies though.
  2. Path for output file set in Output options screen isn't saved and restarting the exe resets this back to the default.
  3. It worked well with local BizTalk installation. For remote machines I got "object reference not set to an instance of an object" error. This could be due to the fact that the tool tries to loads the assemblies listed in the bts_assembly table in the BizTalkMgmtDb. It will not document in detail if it fails to load these (via GAC).

Overall, this is a great tool to have handy

Undocumented Code Access Security change in .NET 2.0

I was attempting to deploy a simple Windows Application by publishing it over the Web. While it worked in its plain-vanilla form, it started behaving awkwardly after I had a number of Enterprise Library components with its related config stuff going into the App.Config file. After much toil, I finally discovered the reason behind it, which actually seems to be a security-enhancing mechanism whose workaround seems to have gone undocumented in MSDN.

Now, its something like this. During the days of .NET 1.1, an application could load the configuration nodes in App.Config or Web.Config (the section providers) under Medium Trust. Come .NET 2.0, and the Code Access level permissions required to load the various configuration settings of the config file are extended.

In the new way, Medium Trust checks a requirePermission attribute on the configuration section declaration - the section at the top of your config file.  For example,

<section name="exceptionManagement" requirePermission="false" type="MyNamespace.MyExceptionHandler, ExceptionManager"  />
<section name="logger" requirePermission="true" type="MyNamespace.MyLogger, Logger"/>

If they are declared as above then access to the "exceptionManagement" provider section is allowed but access to the "logger" section is not (under Medium Trust). By default, requirePermission is set to true, and we need to explicitly add the requirePermission="false" attribute to the element. And in the case of click-once deployment that made me stumble on this, I had to add this additional piece in App.Config to get it all going.

<system.web>                                                                                                                                                  

<trust level="Medium" originUrl="