Friday, May 15, 2009

BizTalk Pipeline Component Wizard

I have been really busy lately!  Not that it's much interest to the general internet public out there but I am getting up to speed on a new ESB project on BizTalk 2009 and I haven't had much time to author any posts.

My new project, being a service-bus design that makes maximum use of messaging and minimum use of orchestration, requires me to design a lot of custom pipelines and pipeline components.  I have noticed that on Codeplex there is a new version of the pipeline component wizard that is updated for BizTalk 2009 (presumably this is just the versions of the referenced assemblies as the interfaces look pretty much the same).  You can find it here:

If you're interested in BizTalk but have not created your own pipelines yet, then please do go through the SDK to see how they're built.  There is a huge amount of power in there that is just waiting to be unleashed!

Also, because the messaging features of BizTalk (i.e. send + reveive ports) use the application bindings to link the components together they are far more flexible for installing updated versions than orchestrations, which have subscriptions that need to be managed in an upgrade scenario.

Happy Pipelining!

Wednesday, May 06, 2009

Problems with recursion in WCF serialization (hierarchical structures)

I know the title isn't too hot, but I wanted to write a post about a problem I've encountered lately with WCF serialization in hierarchical structures.  These come up a lot in the real world, and in the normal world of .Net, when relationships between classes are object references, there is no problem.  However, when you serialize these objects into XML using the WCF data contract serializer you find that the object references are replaced by child XML nodes.  When your references are circular you get into an in infinite loop.

Let me explain by example.  Imagine that we have a database table containing a list of employees.  It might look something like this:

This encapsulates a simple hierarchy.  An employee can have a line manager and so the table is keyed back onto itself to form the hierarchy.  The data might be this:

As you can see, you have 3 records.  One manager and two direct reports.  If we were to represent this in classes we might have the following (note that I have implemented them as data contracts):

    public class Employee
        [DataMember(Order = 0)]
        public string Name { get; set; }

        [DataMember(Order = 1)]
        public string JobTitle { get; set; }

        [DataMember(Order = 2)]
        public Employee LineManager {get; set;}

        [DataMember(Order = 3)]
        public EmployeeCollection DirectReports { get; set; }

    public class EmployeeCollection : List {}

Now, we can see that for an employee we have a collection of their direct reports, and for each employee we can create a reference back to their line manager.  So, what I will do now is use a quick bit of LINQ-to-SQL to retrieve the employees from the database and an extension method to perform the conversion into the data contracts.  My code, if I were to create a simple service, might like this:

namespace EmployeeService
    public class EmployeeServiceInstance : IEmployeeService

        #region IEmployeeService Members

        public Employee GetEmployeeByName(string name)
            using (DataAccess.EmployeesDataContext ctx = new DataAccess.EmployeesDataContext())
                var q = from a in ctx.Employees
                        where a.Name == name
                        select a.Translate();

                return (Employee)q.SingleOrDefault();


    public static class Translator
        public static Employee Translate(this DataAccess.Employee entity)
            Employee employee = new Employee()
                Name = entity.Name,
                JobTitle = entity.JobTitle,
                DirectReports = new EmployeeCollection()

            foreach (DataAccess.Employee e in entity.Employees)

            //This is where you get the problem, by adding the reference back to the parent entity
            employee.DirectReports.ForEach(p => p.LineManager = employee);

            return employee;

[Disclaimer:  This code is for illustration only.  My production code and production designs would not look like this!]

So, what happens when we run this and search for "Bob" using the service?  We get the following error:

Test method EmployeeService.Tests.UnitTest1.TestMethod1 threw exception:  System.ServiceModel.CommunicationException: The underlying connection was closed: The connection was closed unexpectedly. --->  System.Net.WebException: The underlying connection was closed: The connection was closed unexpectedly..

If you look this error up on the net it will tell you that it has occurred when the WCF message size limits have been breached.  This usually happens when you need to bring back lots of rows or when you are handling binary data.  In reality, in this case the error has occurred because the data contract serializer is stuck in an infinite loop passing backwards and forwards between the line manager and then employee records.  Because it is serializing as XML it loses the object references!

So all we have to do is comment out the line where we add the parent reference in...

            //employee.DirectReports.ForEach(p => p.LineManager = employee);

...and it all works!  It's a shame because I still like to use lambda expressions whereever possible because they're still new and cool (to me anyway).

In conclusion, be careful with hierarchies in WCF, and note that the error message that you get does not neccessarily reflect the underlying cause.