BizTalk Multi-Part Messages and Serialization

I've been back working on BizTalk lately so the next few posts are going to be about BizTalk issues that have arisen from some things that have come up lately.

This post is going to cover a defect that I had to resolve lately, and the insight into using BizTalk that comes from the cause / solution.

I am currently working on a banking system that needs to reliably flow payment transactions to people's accounts. The functionality of the system I was working on can be described as follows, without any danger of giving away anything commercially sensitive:

• The system batch-processes BACS payments onto a ledger. Once per day input files come from a mainframe system and the payments need to be loaded into the ledger.
• An SSIS process decodes the files into a staging database location.
• The payments are extracted from the database for transmission to the ledger. This is done using BizTalk.
• The payments are split into batches and transmitted to the ledger.
• The ledger responds asynchronously with a message that describes the success / failure of the payment transaction.
• Payments that ultimately fail get written into a SQL table.
• Once per day, when processing of the payments is complete, the failed payments are extracted into a BACS file using an SSIS process.

Now, given this, there was a bug raised which stated that the amount value for the payments that are in the failed payment output file were being written as zero. Here is a description of the basic fault finding process:

• Bug was raised against the SSIS developer who wrote the output package. He unit tested the extraction process and verified that the value was being written correctly.
• The bug was then assigned to the developer who wrote the data access layer that writes the failed payments into the output staging table. He then verified that the amount values get written correctly into the database.
• The bug was then assigned to the BizTalk team and that meant that me, being one of the BizTalk architects with an overview of the entire process, was called in to look at the issue.
• The first thing I did was to attach a debugger onto the BizTalk hosts, so that I could look at the actual values being passed through the system. First, I debugged the orchestration that writes out the failed payments. I verified that the amount being written by BizTalk was zero - thus confirming that there was no issue with the data access code.
• I then debugged the orchestration that receives the failed payments and verified that the payment amount was non-zero. This meant that somewhere between receiving the payment and writing the payment out the value was being set to zero - but how?

The answer to this lay in the way that BizTalk handles messages. Using good practice, my payment information was held in a multi-part message type (see previous post). Because of the required throughput of the system and the need to access the payment object model, the payment data is held in BizTalk as a .Net object rather than an XML message. Now, this is OK - I can assign a .Net class to a message part as well as an XML schema - as long as the classes are XML serializable. This is because the multi-part message, when sent to the message box, gets XML serialized.

Now, we're getting somewhere. In the process I was looking at the payments are received (as mentioned) and then context information is promoted on the multi-message and it is written into the message box. Different orchestrations then subscribe to the multi-part message by filtering on the context and performing payment-specific business processing. Through further debugging I narrowed the fault down - before we write to the message the value is non-zero and after the message box, in the subscribing orchestration, the value is zero. Baffling.

Now, the answer to this lay in the serialization, as you could probably guess from the title of this post. The payments were originally defined using an XML schema and then the .Net classes were generated using XSD.exe.

OK, so let's strip this back to the essence of the problem. Let's say that I have a payment schema:

And I then create a serializable class for this using XSD.exe:

//------------------------------------------------------------------------------

//

// This code was generated by a tool.

// Runtime Version:2.0.50727.3074

//

// Changes to this file may cause incorrect behavior and will be lost if

// the code is regenerated.

//

//------------------------------------------------------------------------------

//

// This source code was auto-generated by xsd, Version=2.0.50727.3038.

//

namespace Andrew.Blog.MultiPart.Entities {

using System.Xml.Serialization;

///

[System.CodeDom.Compiler.GeneratedCodeAttribute("xsd", "2.0.50727.3038")]

[System.SerializableAttribute()]

[System.Diagnostics.DebuggerStepThroughAttribute()]

[System.ComponentModel.DesignerCategoryAttribute("code")]

[System.Xml.Serialization.XmlTypeAttribute(AnonymousType=true, Namespace="http://Andrew.Blog.MultiPart.Schemas.Payment")]

[System.Xml.Serialization.XmlRootAttribute(Namespace="http://Andrew.Blog.MultiPart.Schemas.Payment", IsNullable=false)]

public partial class Payment : object, System.ComponentModel.INotifyPropertyChanged {

private decimal paymentValueField;

private bool paymentValueFieldSpecified;

///

[System.Xml.Serialization.XmlElementAttribute(Form=System.Xml.Schema.XmlSchemaForm.Unqualified)]

public decimal PaymentValue {

get {

return this.paymentValueField;

}

set {

this.paymentValueField = value;

this.RaisePropertyChanged("PaymentValue");

}

}

///

[System.Xml.Serialization.XmlIgnoreAttribute()]

public bool PaymentValueSpecified {

get {

return this.paymentValueFieldSpecified;

}

set {

this.paymentValueFieldSpecified = value;

this.RaisePropertyChanged("PaymentValueSpecified");

}

}

public event System.ComponentModel.PropertyChangedEventHandler PropertyChanged;

protected void RaisePropertyChanged(string propertyName) {

System.ComponentModel.PropertyChangedEventHandler propertyChanged = this.PropertyChanged;

if ((propertyChanged != null)) {

propertyChanged(this, new System.ComponentModel.PropertyChangedEventArgs(propertyName));

}

}

}

}

And you will see that the serialized entity not only has a field for the decimal, but it expects the "PaymentValueSpecified" property to be set to indicate that the decimal has a value. This is because the field is marked as an optional field in the XML schema, to handle the cases where the field is nullable or unassigned. Unfortunately, in the code the flag to indicate that the payment value was set had not been changed and was still false. [It would be a good idea to set this field in the property set.] Therefore, the XML serializer was still thinking that there was no value and hence the payment value element was not present in the XML when the object was serialized (as it is written to the message box). When the message is deserialized the XML element for payment value is not present and so the decimal field defaults in value to zero. Hence the bug.

In Summary

• .Net objects can be used as part of multi-part message types; this improves the maintainability of your orchestrations and also allows context information to be written onto .Net objects.
• .Net objects, when assigned to a message part, are XML serialized when they are written to the message box. For this purpose, they must be XML serializable. It is easiest to generate these classes from an XML schema.
• Be careful that your .Net object serializes as you expect, because if it doesn't you can get unexpected and unexplained issues in your solution.

I would point out that I inherited the schemas and entities here but somehow I was the one who had to sort out this issue. as usual ;)