Use of generic types in BizTalk and XLANG/s

This is just a quick post to note down something that I have been explaining to people lately. The project I am working on at the moment has several teams of developers working together and most of the code is being cut in Visual Studio 2008 / .Net 3.5. As such, we in the BizTalk feature team are taking delivery of components coded in other teams and integrating them. We are using BizTalk 2006 R2 and developing the BizTalk orchestrations in Visual Studio 2005, although there are no compatability issues between the binaries developed in 2008 and 2005, and anyway generic types were introduced in 2005 / .Net 2.0.

However, where I have come across an issue is in the use of generic types, and specifically assigning generic types to orchestration variables. In some cases orchestrations and expression shapes support generics and in some cases they do not.

All I am going to do is to illustrate some cases where generics are OK and some where they are not, with a bit of explanation as to why.

Assigning to variables

Imagine that you have a class:

public class MyClass {}

And you want to use a collection of this class in your code, these days you would usually use a generic collection in our code:

Collection collection = new Collection<MyClass>();

However, if you try to assign this type to a variable in an orchestration you will find that you can't. This is because you can only pick from a type that is compiled, and generic types aren't in there. If you want to use a collection like this in an orchestration you will have to create a type for it. You can define a new class:

public class MyClassCollection : Collection<MyClass> {}

If you do this, you can reference "MyClassCollection" as a variable in your orchestration and everything will be fine. Note that the usual rules about classes being marked as serializable apply.

The same applies if you create a class that has a generic interface such as this:

namespace AndrewGenerics.Components
{
[Serializable]
public class OrchestrationInstanceHelper<T>
{
public void UpdateInstance(T objectInstance)
{
// Some code in here
}
}
}

If you see this class in the type picker and try to assign it to a variable you will be able to pick it:

However you will not be able to assign the generic type:

As a result if you try to compile you will get a whole heap of errors like these:

Error 1 '` (0x60)': character cannot appear in an identifier C:\Projects\AndrewGenerics\AndrewGenerics.BizTalk\TestOrchestration.odx 62 63

Error 2 '` (0x60)': character cannot appear in an identifier C:\Projects\AndrewGenerics\AndrewGenerics.BizTalk\TestOrchestration.odx 67 80

Error 3 identifier 'OrchestrationInstanceHelper' does not exist in 'AndrewGenerics.Components'; are you missing an assembly reference? C:\Projects\AndrewGenerics\AndrewGenerics.BizTalk\TestOrchestration.odx 62 36

Error 4 cannot find symbol 'AndrewGenerics.Components.OrchestrationInstanceHelper' C:\Projects\AndrewGenerics\AndrewGenerics.BizTalk\TestOrchestration.odx 62 36

Error 5 expected 'identifier' C:\Projects\AndrewGenerics\AndrewGenerics.BizTalk\TestOrchestration.odx 62 64

Error 6 unexpected token: 'numeric-literal' C:\Projects\AndrewGenerics\AndrewGenerics.BizTalk\TestOrchestration.odx 62 64

Error 7 identifier 'helper' does not exist in 'TestOrchestration'; are you missing an assembly reference? C:\Projects\AndrewGenerics\AndrewGenerics.BizTalk\TestOrchestration.odx 67 14

Error 8 cannot find symbol 'helper' C:\Projects\AndrewGenerics\AndrewGenerics.BizTalk\TestOrchestration.odx 67 14

Error 9 identifier 'OrchestrationInstanceHelper' does not exist in 'AndrewGenerics.Components'; are you missing an assembly reference? C:\Projects\AndrewGenerics\AndrewGenerics.BizTalk\TestOrchestration.odx 67 53

Error 10 cannot find symbol 'AndrewGenerics.Components.OrchestrationInstanceHelper' C:\Projects\AndrewGenerics\AndrewGenerics.BizTalk\TestOrchestration.odx 67 53

Error 11 'new OrchestrationInstanceHelper': a new expression requires () after type C:\Projects\AndrewGenerics\AndrewGenerics.BizTalk\TestOrchestration.odx 67 23

Error 12 expected 'identifier' C:\Projects\AndrewGenerics\AndrewGenerics.BizTalk\TestOrchestration.odx 67 82

Error 13 unexpected token: '(' C:\Projects\AndrewGenerics\AndrewGenerics.BizTalk\TestOrchestration.odx 67 82

Error 14 illegal statement '1' C:\Projects\AndrewGenerics\AndrewGenerics.BizTalk\TestOrchestration.odx 67 81

All those errors are because of a couple of things, but mainly because when you add a variable to an orchestration, or add code in an expression shape, the designer writes some C# code for you and then compiles it. Because the way that the designer handles generics is incorrect the C# doesn't compile and you get the build errors.

Again though, if you created a class that inherited from the above class and assigned the generic type you'd be OK:

public class MyClassInstanceHelper : OrchestrationInstanceHelper<MyClass> {}

Passing parameters to methods

OK. Let's now create a helper component that we will call from an orchestration. [Disclaimer: The code below if for illustration purposes only!]

namespace AndrewGenerics.Components

{
public static class OrchestrationHelper
{

// Note that this uses the collection type
public static void UpdateItemsInCollection(MyClassCollection collection)
{
// Some code in here
}

// Note that this uses the generic type
public static void UpdateItemsInCollection2(IEnumerable<MyClass> collection)
{
// Some code in here }
}
}

The first of these calls uses the inherited type and so will go through no problem:

AndrewGenerics.Components.OrchestrationHelper.UpdateItemsInCollection(myCollection);

But even if we use the second of the method calls with the IEnumerable<MyClass> parameter, that still works OK:

AndrewGenerics.Components.OrchestrationHelper.UpdateItemsInCollection2(myCollection);

Therefore, even though the method in a helper component has a generic type in the code, when the class is compiled the generic type gets "baked" into the interface and so the variable assignment works.

Assigning output from methods to variables

Now let's add another method onto the helper class that we can use to receive a collection of objects, but again through a generic type.

public static IEnumerable<MyClass> GetCollection()
{
Collection<MyClass> coll = new Collection<MyClass>();

coll.Add(new MyClass());

return coll;
}

In order to call this I might use a line of code like this in an expression shape:

myCollection = (AndrewGenerics.Components.MyClassCollection)AndrewGenerics.Components.OrchestrationHelper.GetCollection();

Again, this is handled OK by the BizTalk compiler when we are assigning to our collection and the we are casting the type of the result. For the same reasons as above, we can't create a variable of IEnumerable<MyClass;> so we can't receive the output of this method without casting it, but we can at least handle it. Obviously, if at runtime we were presented with a different object that conformed to IEnumerable<MyClass;> (such as an array) we would get a runtime error because of an invalid cast.

However, we have been able to declare a method that uses a generic type, and for the same reason we can use it, i.e. when the helper component is compiled the method signature becomes fixed and the orchestration compiler can handle the output.

Conclusions

Here are some key points from this blog post:

1. You cannot declare generic type in an orchestration because you can only select types in the type picker. If you select a generic interface that does not have the type assigned you get some crazy errors.
2. In order to use a generic type you can create a class that implements the generic type. This will then be usable by BizTalk.
3. When a helper component uses generic types in the interface the types get baked into the interface and can be used by BizTalk.

Am I missing something?

I was shopping on Amazon for my son's Christmas present this afternoon. He wants a laptop and I was looking for an entry-level laptop that is running Windows 7. I found this nice looking Compaq Presario:

However, when I scrolled down on the description I found the following:

That's it. The only thing that other people have bought. I was expecting to see antivirus and MS Office! Just shows that you can't assume what choices the public will make!

BizTalk Orchestration Designer Crashes Visual Studio

This is just a quick post to note in passing an issue that I encountered recently. In my current project I am again working as a BizTalk architect, and we are using BizTalk Server 2006 R2, and hence the designer is hosted in Visual Studio 2005. After taking over some enormous orchestrations from another member of the team recently I started to find that Visual Studio started crashing often and unpredictably. So much so, in fact, that I had to press ctrl+S after just about every shape change, as I was losing work so often.

Now, I had worked on a project a couple of years ago where we suffered this a lot. I think that one was BizTalk Server 2006 R1. After having done some stints with BizTalk 209 / VS 2008 it came as a nasty surprise to be getting this issue again.

As you might expect, I decided to search to see who else had had the same issue. I got a couple of relevant hits. This one, http://continuouslyintegrating.blogspot.com/2008/01/orchestration-designer-crashes-visual.html, was interesting as it implied that there was something in your profile that was causing the issue. I didn't really want to have to rebuild a profile in the middle of a critical phase in the project so I kept searching.

I then came across this one, http://www.sabratech.co.uk/blogs/yossidahan/labels/visual%20studio.html, and more things to try. The thing that seemed to be intuitively right was that the size of the orchestration might be an issue. I had only started experiencing issues once I started to work on huge orchestrations. Before that there had been no problem. When working on modest orchestrations there was no problem.

I therefore took the suggestion that had the least amount of pain, to decrease the colour depth in my display settings from 32-bit down to 16-bit. I was hoping that this was going to do the trick. I then started working on the offending orchestrations and I could at least get started, but I did experience further crashes.

I then thought of a further step I might take, and it seemed to follow on from reducing the graphics load. I zoomed out. Never had another crash after that!

I think part of the problem is that the orchestration designer renders the orchestration as an image, or rather as a series of overlapping images. Then, depending on where you scroll to, a certain portion of the image is displayed. Therefore, no matter how small your viewing area in the designer the orchestration designer is still rendering a pretty big image. However, if you scroll out you reduce the overall size f the image that needs to be rendered. And, as mentioned, if you decrease the colour depth you reduce the size of it still further.

Conclusion

Huge orchestrations, from a design point of view, are bad. Huge orchestrations, from the Visual Studio Orchestration Designer point of view, are bad. If you must have them (as in my case where I was handed them and had to make them work), reducing the graphics load on your machine is a quick way to prevent Visual Studio from crashing under the load.

Use of interfaces within BizTalk Orchestrations and XLANG/s

Abstract

This blog post discusses the way in which interfaces are handled in BizTalk. In particular, the reason why variables defined by interfaces cannot be saved to orchestration state. This causes a compiler error [a non-serializable object type '{type name} {variable name}' can only be declared within an atomic scope or service ].

Introduction

I have recently been working back on BizTalk, and I have come across a strange issue with the way that interfaces are handled by the BizTalk compiler. As usual, this is of interest not only because there are people who may encounter this issue, but also because of the implications it has for understanding how the BizTalk orchestration engine works.

Let's consider a scenario: You want to encapsulate a common sub-process within a single orchestration. You need to invoke some business logic within this sub-process and this may change depending on which orchestration has invoked the sub-process.

In my case I decided to use a factory pattern, so I created an interface for the business logic and then the orchestration invoked the factory to receive the correct business logic instance. I have created a simplified example project to demonstrate the issue.

Example Solution

First, I created an interface that defines how the business logic is to be called:

public interface IOrchestrationHelper

{

void DoStuff();

}

I then created a base class (more on this later):

[Serializable]

public class OrchestrationHelperBase : IOrchestrationHelper

{

#region IOrchestrationHelper Members

public virtual void DoStuff()

{

throw new Exception("The method or operation is not implemented.");

}

#endregion

}

And then I created 2 implementations of the class:

public class OrchestrationHelperA : OrchestrationHelperBase

{

public override void DoStuff()

{

Debug.WriteLine("This is OrchestrationHelperA");

}

}

[Serializable]

public class OrchestrationHelperB : OrchestrationHelperBase

{

public override void DoStuff()

{

Debug.WriteLine("This is OrchestrationHelperB");

}

}

And the factory to instantiate the classes:

public static class OrchestrationHelperFactory

{

public static IOrchestrationHelper CreateHelper(string helperName)

{

switch (helperName)

{

case "A":

return new OrchestrationHelperA();

case "B":

return new OrchestrationHelperB();

default:

throw new Exception("Could not match a helper to the input specification.");

}

}

}

OK, so far so good. Simple stuff, we do this sort of thing every day don't we? This needed to be hooked into the BizTalk processes, so I incorporated the calls to the factory and the business logic into an orchestration, as follows:

If you look at the orchestration, I have a parameter called helperSpecification of type System.String that is passed in by the caller, which defines the piece of business logic to invoke (in practice this would possibly be an enum, but this is just to demonstrate). There is also an orchestration parameter called orchestrationHelper of type IOrchestrationHelper that contains the instance of the business logic component.

In the first expression shape I create the orchestration helper:

orchestrationHelper = Andrew.InterfacesXLANGs.Components.OrchestrationHelperFactory.CreateHelper(helperSpecification);

And in the next expression shape I call the business logic:

orchestrationHelper.DoStuff();

Again, this is almost as simple an orchestration as it is possible to get. However, when I try to compile it I get the following error:

Error 1 a non-serializable object type 'Andrew.InterfacesXLANGs.Components.IOrchestrationHelper orchestrationHelper' can only be declared within an atomic scope or service C:\Documents and Settings\v-anriv\My Documents\Visual Studio 2005\Projects\InterfacesXLANGs\InterfacesXLANGs\SubProcess.odx 46 66

Now, if you look into the cause of this error it is quite simple. BizTalk is a reliable messaging and orchestration server; the mechanism for achieving this reliability is that the state of messages and orchestrations is persisted to the Message Box database at runtime, either at persistence points (send ports, timeouts, exiting atomic scopes) or when the service decides to save the state to manage load. This is where the issue lies. In order to save the state of a variable it must be marked as serializable. When an orchestration hits a persistence point it serializes all of its variables and saves the data into the database. When the orchestration is "rehydrated", the state is deserialized and the processing can continue.

I mentioned scopes just above. Atomic scopes are a special case in BizTalk. These are the scopes in which an atomic (MSDTC) transaction is running. Obviously, in order to marshal the resources for such a transaction the orchestration must remain in memory during the processing of an atomic scope. This means that the scope must complete, or if it fails half way through BizTalk will assume that the work in the atomic scope has not been done, and will attempt to re-run it when the orchestration is started.

A side-effect of atomic scopes is that variables that are defined in an atomic scope will never be persisted to the database as they will always be in memory until the scope is complete. Because of this, it is possible to define a variable that is a non-serializable class.

As you can imagine, when a BizTalk host is running an orchestration t is just like any process executing code. However, when a persistence point is reached there is a double-hit on the performance as the state is serialized and is then saved into the message box database as a recoverable state point. This increased the latency of the orchestration considerably, and if throughput performance is an issue then you should minimise the number of persistence points. If you are more concerned with reliability then it's not such a bad thing.

If you look at the classes I defined they were all marked as serializable, so they could all happily exist in BizTalk orchestrations. However, because the variable was defined to use the interface, the compiler did not know if the class that would implement the interface would also be marked as serializable, therefore it generated an error.

The Solution, and some words of warning

In order to allow the BizTalk compiler to be fooled into accepting the interface, you need to declare the variable as the base class, and mark the base class as serializable. But be careful, if you do anything in your subclasses to make them non-serializable then there will be some unfortunate unintended consequences when the object is instantiated by the factory and loaded into the orchestration state.

Summary

I have used an example where an interface is used in a declaration in BizTalk to illustrate how BizTalk manages orchestration state through serialization of the orchestration variables. If you think about why and where orchestrations serialize and persist / dehydrate / rehydrate you will also start to get a grip on the factors that affect orchestration performance, and you can alter your designs to suit.

Just read this somewhere....

"A computer without COBOL or FORTRAN is like a chocolate cake without ketchup or mustard"

Reminder : Fusion Log Viewer

Just a quick note - been having some errors loading assemblies lately and have called on the little-known tool in the SDK called Fusion Log Viewer. You can load it from the Visual Studio Command Prompt using the command fuslogvw or load it from here:

C:\Program Files\Microsoft SDKs\Windows\v6.0A\bin\ FUSLOGVW.exe

You'll get a view of all assembly load failures, which is really useful with systems that dynamically load stuff from the GAC at runtime such as BizTalk.