Collection Shortcuts in Everest 1.0

The goal of the Everest framework data types is to provide functionality that allows developers to easily construct and interact with the HL7v3 data types. In previous versions of the Everest Framework, creating collections could be difficult. Consider the AD data type which is nothing more than a collection of ADXP components. In previous versions of Everest, creating this structure would look something like this:

AD homeAddress = new AD(
  new SET<CS<PostalAddressUse>>()
  {
     PostalAddressUse.Alphabetic,
     PostalAddressUse.Direct
  },
  new ADXP[] {
     new ADXP("123 Main Street", AddressPartType.StreetAddressLine),
     new ADXP("West", AddressPartType.Direction),
     new ADXP("Hamilton", AddressPartType.City),
     new ADXP("Ontario", AddressPartType.State),
     new ADXP("Canada", AddressPartType.Country)
  }
);

This code can be quite large and is difficult to track if not styled properly (indentation is the key here). So, to make the construction of sets a little easier, we've added static "creator" methods on each of the collection data types. They are used as follows:

AD homeAddress = AD.CreateAD(
   SET<PostalAddressUse>.CreateSET(
      PostalAddressUse.Alphabetic,
      PostalAddressUse.Direct
   ),
   new ADXP("123 Main Street", AddressPartType.City),
   new ADXP("West", AddressPartType.Direction),
   new ADXP("Hamilton", AddressPartType.City),
   new ADXP("Ontario", AddressPartType.State),
   new ADXP("Canada", AddressPartType.Country)
);

The benefit of this shortcut is illustrated better with more complex sets such as SXPR and QSET. The following snippet represents the construction of an SXPR that represents numbers {1..10}, intersected with the result of a union of numbers {3..5} and {7..9}.

SXPR<INT> result = new SXPR<INT>()
{
   new IVL<INT>(1, 10),
   new SXPR<INT>() {
       Operator = SetOperator.Intersect,
       Terms = new LIST<SXCM<INT>>() {
          new IVL<INT>(3, 5),
          new IVL<INT>(7, 9)
          {
              Operator = SetOperator.Inclusive
          }
       }
   }
};

Using the new constructor which is a shortcut, the following expression can be used:

SXPR<INT> result = new SXPR<INT>(
   new IVL<INT>(1, 10),
   new SXPR<INT>(
       new IVL<INT>(3, 5),
       new IVL<INT>(7, 9)
       {
           Operator = SetOperator.Inclusive
       }
   )
   {
      Operator = SetOperator.Intersect
   }
);

These new shortcut methods are intended to assist developers even more than previous attempts at the Data Types implementation and are one of the many improvements in the Everest 1.0 data types library.

SNOMED Expressions in Everest 1.0

Wow, it has been a long time since I have posted a new article on this blog and it has mainly been because of the crazy-busy schedule at work. But, I just thought I had to share this with everyone in the HL7 integration world looking at implementing Data Types R2 and R1 (or those migrating R1 to R2), and some of the benefits that Everest 1.0 will have when released (it is getting close).

First, I will provide a little bit of an overview. In HL7v3 clinical concepts within messages are represented using one of four different data types (two in the Data Types R2 specification). These are:

R1	R2	Summary
CS	CS	Coded Simple - A simple code where only the code mnemonic is unknown
CV	CD.CV	Coded Value – A more complex code structure whereby the code system from which the mnemonic is taken is unknown at design time.
CE	CD.CE	Code with Equivalents – A CV instance where translations (or equivalents) can optionally be specified.
CD	CD	Concept Descriptor – A code mnemonic taken from a code system, optionally with one or more concept roles which qualify the primary code. For example, the code LEFT qualifies FOOT to mean LEFT FOOT.
CR	N/A	Concept Role - A name/value pair where the value concept qualifies the semantic meaning of the primary mnemonic by way of the named concept.

Wait a minute! Notice some differences? Well, for starters CV and CE are no longer "proper" types according to the data types, they are flavors of CD. This is an appropriate change as they remain structurally identical to the R1 structures.

The big change comes in the concept descriptor. Notice how the CR data type is not present in data types R2. When I first saw this I thought nothing of it, however when looking at how each are represented on the wire the difference is very pronounced.

I'm a code kind of guy, so I thought I would explain this using code. First off, Everest uses a hybrid of DT R1 and R2, so the codified data types in Everest resemble those found in R1 (and are mapped to appropriate R2 flavors on formatting). With that in mind, let's represent the following example: "severe burn on the skin between the fourth and fifth toes on the left side", in Everest.

First, we create the primary code of "burn":

var burnCode = new CD<string>("284196006", "2.16.840.1.113883.6.96") {

DisplayName = "Burn of Skin",

CodeSystemName = "SNOMED-CT",

CodeSystemVersion = "2009"

};

Next, the we want to qualify "Burn of Skin" with a severity of "Severe". This is accomplished by creating a CR instance:

// Severity
var severityQualifier = new CR<string>(
   new CV<String>("246112005", "2.16.840.1.113883.6.96")
            { DisplayName = "Severity" },
   new CD<String>("24484000", "2.16.840.1.113883.6.96")
            { DisplayName = "Severe" }
);

Next, our code has a finding site. The burn was located on the skin between the fourth and fifth toes, so once again it is another CR instance:

// Finding Site
var findingSiteQualifier = new CR<String>(
    new CV<String>("363698007", "2.16.840.1.113883.6.96")
             { DisplayName = "Finding Site" },
    new CD<String>("113185004", "2.16.840.1.113883.6.96")
             { DisplayName = "Skin Between fourth and fifth toes" }
);

Next, we want to describe the fact that the the burn was found on the skin between the fourth and fifth toes "on the left hand side". Obviously we want to create another qualifier for this:

// Laterality
var lateralityQualifier = new CR<String>(
    new CV<String>("272741003", "2.16.840.1.113883.6.96")
            { DisplayName = "Laterality" },
    new CD<string>("7771000", "2.16.840.1.113883.6.96")
            { DisplayName = "Left Side" }
);

But how would we structure these concept roles to describe the situation? First we have to look at each term and ask the question, "What does this qualify". So, for example, does "Laterality of Left Side" qualify the burn? Technically no, the laterality qualifies the finding site (i.e.: We found the burn on the toes on the left hand side). So we want to add the lateralityQualifier to the findingSiteQualifier's value:

// Laterality applies to the finding site
findingSiteQualifier.Value.Qualifier = new LIST<CR<string>>() { lateralityQualifier };

What does finding site qualify? Technically finding doesn't qualify the severity it qualifies the primary code (i.e.: The burn was "found on" the skin…), and the same with applies to the severity (i.e.: We didn't find a severe skin between toes, we found a severe burn). So we add these two qualifiers to the primary code:

// Finding site and severity apply to primary code
burnCode.Qualifier = new LIST<CR<string>>() {
     severityQualifier,
     findingSiteQualifier
};

Now comes the easy part, when we format the data type using data types R1 formatter:

var formatter = new MARC.Everest.Formatters.XML.ITS1.Formatter();
formatter.ValidateConformance = false;
formatter.GraphAides.Add(
     typeof(MARC.Everest.Formatters.XML.Datatypes.R1.DatatypeFormatter)
);

// Setup the writer
StreamWriter sw = new StreamWriter("C:\\temp\\temp.xml");
XmlWriter xw = XmlWriter.Create(sw, new XmlWriterSettings() { Indent = true });
XmlStateWriter xsw = new XmlStateWriter(xw);

// Format and produce the XML file
try
{
     xsw.WriteStartElement("code", "urn:hl7-org:v3");   xsw.WriteAttributeString("xmlns", "xsi", null, http://www.w3.org/2001/XMLSchema-instance);
   var p = formatter.Graph(xsw, burnCode);   sw.WriteEndElement();
}
finally
{    xw.Close();    sw.Flush();
    formatter.Dispose();
}

The output of this is the following XML:

<code code="284196006" codeSystem="2.16.840.1.113883.6.96" codeSystemName="SNOMED-CT" codeSystemVersion="2009" displayName="Burn of Skin">
   <qualifier inverted="false">
      <name code="246112005" codeSystem="2.16.840.1.113883.6.96"
            displayName="Severity" />     
      <value code="24484000" codeSystem="2.16.840.1.113883.6.96"
            displayName="Severe" />
   </qualifier>
   <qualifier inverted="false">
      <name code="363698007" codeSystem="2.16.840.1.113883.6.96"
            displayName="Finding Site" />
      <value code="113185004" codeSystem="2.16.840.1.113883.6.96"
             displayName="Skin Between fourth and fifth toes">
          <qualifier inverted="false">             
              <name code="272741003" codeSystem="2.16.840.1.113883.6.96"
                    displayName="Laterality" />
              <value code="7771000" codeSystem="2.16.840.1.113883.6.96"
                     displayName="Left Side" />
          </qualifier>     
      </value>
   </qualifier>
</code>

But as I mentioned previously, CR is not supported in DT R2. So the question arises, "How do I qualify a code in HL7v3 DT R2?". Well, the answer is not so simple. In DT R2, the concepts for SNOMED terms are described using an expression language defined by IHTSDO. The SNOMED expression for our scenario is:

284196006|Burn of Skin|:{246112005|Severity|=24484000|Severe|,363698007|Finding Site|=(113185004|Skin Between fourth and fifth toes|:272741003|Laterality|=7771000|Left|)}

Intuitive right? Not really. So how do I represent this in a CD instance? Well, the answer is really ugly, and in my opinion violates first normal form (I will post an opinion post later about my thoughts of using 1NF in XML and how I think standards bodies seem to have forgotten it).

Anyways, so what is this supposed to look like in DT R2? The answer is below:

<code code="284196006:{246112005=24484000,363698007=(113185004:272741003=7771000)}"
      codeSystem="2.16.840.1.113883.6.96"
      codeSystemName="SNOMED-CT"
      codeSystemVersion="2009">
    <displayName value="Burn of Skin"/>
</code>

I warned you it wasn't pretty. So how do you get Everest to format a concept descriptor like this? Well, the answer is simple, change this line of code:

// Old Line: formatter.GraphAides.Add(typeof(MARC.Everest.Formatters.XML.Datatypes.R1.DatatypeFormatter));

formatter.GraphAides.Add(typeof(MARC.Everest.Formatters.XML.Datatypes.R2.DatatypeR2Formatter));

And Everest will automatically handle the generation of these expressions for SNOMED concepts. Parsing? It is the same. Everest 1.0's data types R2 formatter has been developed so that you are shielded from having to understand the complexities of SNOMED expressions.

As a matter of fact, when parsing a SNOMED concept with an SNOMED expression, Everest will construct the appropriate hierarchy of concept roles for you.

What do I think of this change in HL7? I think it was pointless, and simply over-complicates processing of XML instances. In my opinion, there is no need to mix the hierarchical language of SNOMED expressions as an attribute within the hierarcal container of XML. At least a framework like Everest has enough logic in the formatting of codes to shield the developer from changes like this.

Next time, I'm going to blog about a good change in R2, changes in the continuous set expression data types (SXPR).

Formatting to a String in Everest

I've never really liked strings for transporting data as they're memory pigs and introduce a huge performance penalty, however formatting to a string can be useful if you're debugging an application or just want to see the results of formatting. So I figured I'd do a short-ish post today and illustrate how this is done.

First, we'll need an instance, I've created a small sample message from the UV NE2008 assembly but it can be any instance that is IGraphable.

MCCI_IN000000UV01 instance = new MCCI_IN000000UV01(
    Guid.NewGuid(),
    DateTime.Now,
    MCCI_IN000000UV01.GetInteractionId(),
    ProcessingID.Training,
    "I",
    AcknowledgementCondition.Never);

Next, we'll define the formatter instance that is going to take this RMIM structure and render it using an ITS. I'm using XML ITS 1 with data types R1.

var formatter = new MARC.Everest.Formatters.XML.ITS1.Formatter();
formatter.ValidateConformance = false;
formatter.GraphAides.Add(typeof(MARC.Everest.Formatters.XML.ITS1.Formatter));


I've disabled conformance checking as I assume since we're formatting to a string, we're just fiddling with Everest. I'd recommend setting ValidateConformance to true if you're seriously trying to create a conformant message. Next, we create a StringWriter class and attach an XmlWriter with indentation turned on (makes it easier to read):

StringWriter sw = new StringWriter();
XmlWriter xw = XmlWriter.Create(sw, new XmlWriterSettings()
      { Indent = true }
);


We could just format the instance as is, however it is always recommended you use an XmlStateWriter when dealing with Everest, so that what this next line does:

XmlStateWriter xsw = new XmlStateWriter(xw);

Next, we just format our instance and flush (or close) the XmlWriter.

try
{
   formatter.Graph(xsw, instance);
}
finally
{
   xw.Close();
   sw.Flush();
}

Notice I'm using the Graph() method, this is a new construct introduced in Everest RC3. If you're using RC2 or prior you can call GraphObject() instead. Finally, you can get your string by calling the ToString() method on the string writer. I've decided to print out to the console for demonstrative purposes:

Console.WriteLine(sw.ToString());
Console.ReadKey();

And there you have it! An XML Instance as a string. I wouldn't recommend using strings too heavily in your production code, like I said they're horrible from a memory consumption and performance POV. Streams and Writers/Readers are much faster and flexible.

Cheers

-Justin

Emitting Unit Tests from Assemblies Part 2

In my previous post I've illustrated how to setup the necessary utility functions for emitting C# code from assemblies. In this post I'll tie it all together and finish up the application so that we can generate equality unit tests from an assembly.

The first step is of course to define a program entry point.

static void Main(string[] args)
{

In my application this will load the assembly and generate unit tests for each of the types which inherit from ANY (the purpose of this application is to generate equality tests for Everest, you can modify this to suit your needs). I'm using a lambda expression to perform this operation:

var types = typeof(II).Assembly.GetTypes();
var anyTypes = Array.FindAll(types, o => !o.IsAbstract &&
    o.GetCustomAttributes(typeof(ObsoleteAttribute), true).Length == 0 &&
    typeof(ANY).IsAssignableFrom(o) &&
    o.GetMethod("Equals", new Type[] { o }) != null);

The lambda expression that searches each type looks confusing but can be easily explained. The variable types is an array of all types in the assembly which defines the II type (ie: the assembly we want to search), the lambda clauses find all types in that assembly that:

!o.IsAbstract	Where the type is not abstract (ie: we can instantiate an instance of the type)
o.GetCustomAttributes(typeof(ObsoleteAttribute), true).Length == 0	The type is not obsolete (ie: there are no ObsoleteAttribute defined on the type)
typeof(ANY).IsAssignableFrom(o)	The type can be assigned to a variable of type ANY (ie: is a ANY)
o.GetMethod("Equals", new Type[] { o }) != null	The type has a method called Equals that accepts a parameter of itself (ie: equality can be determined between two instances of the type)

Next, we'll iterate through each of the types that we've found and will generate a unit test file (if I were a Java developer I'd call this an Iterator pattern to make it seem more impressive, but I like to keep things simple)

foreach (Type t in anyTypes)
     GenerateTestFile(t);

Before I go into the contents of the GenerateTestFile method, I'll explain a little bit about the structure of a Visual Studio unit test. All unit tests are represented as methods within a test class. The test class defines the test context as well as any setup and teardown code. Since I didn't want to store this as a string (very un-maintainable), I created a template file and added it as a resource called Templates.

GenerateTestFile does exactly what you'd expect, it generates the test file. First, it checks if T is a generic type definition and if it is, creates a generic type using the INT class (in .NET each generic class is a new type as it uses reified generics).

if (t.IsGenericTypeDefinition)
{
   // Create a generic type of INT
   List<Type> genericArguments = new List<Type>();
   foreach (Type genParm in t.GetGenericArguments())
       genericArguments.Add(typeof(INT));
   t = t.MakeGenericType(genericArguments.ToArray());
}

This little snippet of code will create a list (or array) of generic arguments of INT for each generic argument available in the type definition. For example, if it encounters QTY then QTY<INT> is created, for RTO the type RTO<INT,INT> is created.

Next we do some pretty standard .NET stuff. I always use this pattern for generating files. I'm not going to outline the code here are it is pretty easy to follow in the attached file

TextWriter tw = null;
try
{
   // Code Here
}
finally
{
   if(tw != null)
      tw.Close();
}

The complete source for this program is available here: http://pastebin.com/P06dRAnu

Remember, a piece of software is only as good as the testing and QA processes it passes through. Emitting unit tests from a target test assembly is a great start for mundane tests like Equality or Comparisons but you will eventually have to manually write unit tests for more complex logic in your application. As an example, Everest's test battery consists of 9,500 unit tests of which approximately 1,000 are hand written and the rest are generated.

Happy testing!

Emitting Unit Tests from Assemblies Part 1

In the pursuit of correct software, the team and I have written several tools to "discover" the features of various Everest components and create unit tests for them. Here are just a few of the tools we developed:

• A nifty little gem that reflects the generated assemblies created by GPMR and populates as many properties as it can find, serializes and compares to XSDs
  
• The same gem also generates a round-trip test, that is, populate as many properties as possible, serialize then de-serialize and then compare (to make sure they're the same)
  
• A program that reflects the DataTypes library, populates all the values in two instances, then changes one and compares them (to make sure they're not equal)
  
• A program that serializes just datatypes and compares equality (datatype round-tripping)
  

Now, that seems pretty boring, and you might be asking why the heck is he posting this? I'm bored. Well, I'll show you how I wrote the equality generator so you can do the same for your projects.

In order to generate a program to test equality we need to think of an algorithm for such a program. If we have two classes and we want to test the .Equals method, we'll need to generate a unit test class with (n + 1) tests where n is the number of properties in the type. For example, if our type looks like this:

MyClass

+ Name : String - Age : Int + DOB : DateTime

We'll need to generate 3 unit tests, two testing inequality and one testing equality. The tests generated should look like this:

Test #	Property	A Instance	B Instance	Assertion
1	Name	"Bob"	"Bill"	A Not Equal B
	DOB	01-01-1990	01-01-1990
2	Name	"Bob"	"Bob"	A Not Equal B
	DOB	01-01-1990	01-01-1991
3	Name	"Bob"	"Bob"	A Equal B
	DOB	01-01-1990	01-01-1990

In our final output, the generated code should look like this:

[TestMethod]
public void TestNameInquality()
{
    MyClass aInstance = new MyClass(), bInstance = new MyClass();
    aInstance.Name = "Bob";
    aInstance.DOB = DateTime.Parse("01-01-1990");
    bInstance.Name = "Bill";
    bInstance.DOB = DateTime.Parse("01-01-1990");
    Assert.AreNotEqual(aInstance, bInstance);
}

First, let's define a structure for holding our test data, since I couldn't think of a better name, we'll call it TestData. I'm feeling a little lazy right now, so we'll just make it a simple structure:

struct TestData
{
   public String TestName { get; set; }
   public List<KeyValuePair<String, String>> AValues { get; set; }
   public List<KeyValuePair<String, String>> BValues { get; set; }
   public bool IsTrue { get; set; }
}

In this structure, we'll store the name of the test, as well as the expected outcome (ie: are they equal is what I meant by IsTrue). The structure also defines the A Instance and B Instance values are key/value pairs of string. Since the program we're writing is going to emit code, and code is ... well, for the most part, a text file the array of key/value pairs will hold the name of the property and the necessary code to set the value.

Next, we'll create a utility function that can take those nice System.Type classes and format them as a C# type reference. This is a relatively simple process, we start with the method signature:

private static string CreateClassRef(Type t)
{

This utility function will expect a System.Type parameter and will generate the C# datatype reference (as appropriate for code). Next, we'll determine the actual name of the class. In .NET, generic types are represented with a back-tick, so CE<T> is actually CE`1[[.... when you call System.Type.FullName. Since the emitted code won't work with the back-tick, we need to turn CE`1[[ back into CE< ...

String className = t.FullName;
if (className.Contains("`"))
   className = className.Substring(0, className.IndexOf("`"));

Next, the code needs to determine if the type is indeed a generic, and if so, it needs to generate those funky angle brackets. In .NET each generic type definition (ie: CE<T>) can be used to generate one or more generic types (ie: CE<String>). Since .NET doesn't do nasty type erasure like some languages who shall remain nameless (*cough* Java) it is possible to generate a code signature from a System.Type:

if (t.IsGenericType)
{
   className += "<";
   foreach (var genParm in t.GetGenericArguments())
     className += String.Format("{0},", CreateClassRef(genParm));
   className = className.Remove(className.Length - 1);
   className += ">";
}

This little snippet of code will turn CE`1[System.String, mscorlib... into CE<System.String>. Notice that it is recursively calling itself. This is to handle nested generic type definitions, such as LIST<CE<String>>. Finally, we'll finish the method by returning the generated refernece:

   return className;
}

Our next utility function will be used to populate those nice (albeit scary looking) AValue/BValue properties on the TestData class. If we think about our outputs, this method will have to generate code that populates all the data in the AValue and BValue. The approach I've taken is to write a utility function that returns one Key/Value pair class for a single property. In order for the function to know what it is generating, it needs to know if you want option 1 or option 2 (ie: which initializer data) to be assigned to the property.

/// <summary>
/// Create property value setter
/// </summary>
private static KeyValuePair<string, string> CreatePropertyValue(PropertyInfo info, int p)
{
   string initializer = GetInitializer(info.PropertyType, p);

   return new KeyValuePair<string, string>(info.Name, initializer);
}

I use yet another utility function to actually generate the initializer code. Why? So I can emit code for constructors with parameters, or any .Add methods the program might find. Let's take a look, first my method signature:

private static string GetInitializer(Type type, int p)
{

As you might expect, in order to generate code to initialize something, the function will need to know the type of the something. Also, I'm propogating the initial value choice (parameter p). Next, I start to determine if the type is something I can directly or easily just assign:

string initializer = "";
if (type == typeof(byte) || type == typeof(byte?))
    initializer = p.ToString();

This little snippet will generate initialization code for either a byte or nullable byte. Since p is an integer, my code can simply use the value passed to it (ie: in code it is ok to write: byte myByte = 3;). Notice I don't append a ";" to the end of the initializer string, remember this method can be called to set constructor parameters as well. For all of the simple datatypes I follow the same pattern:

else if (type == typeof(int) || type == typeof(int?))
  initializer = p.ToString();
else if (type == typeof(double) || type == typeof(double?))
  initializer = p.ToString("0.0f");
else if (type == typeof(decimal) || type == typeof(decimal?))
  initializer = String.Format("(decimal){0}", p);
else if (type == typeof(String))
  initializer = String.Format("\"{0}\"", p);
else if (type == typeof(bool) || type == typeof(bool?))
  initializer = Convert.ToBoolean(p).ToString().ToLower();

Next, my function may encounter something that is a DateTime, since I can't just assign my P value to a dateTime I have to do something a little more complex. I've chosen to create a date on January 1, 201x.

else if (type == typeof(DateTime) || type == typeof(DateTime?))
   initializer = String.Format("DateTime.Parse(\"201{1}-1-10\")", p + 1);

If my type is an array, I can emit code that generates an array. So, for example, if my type is byte[] I can emit the code new byte[] { 1 } (again it is perfectly legal to emit: byte[] myByte = new byte[] { 1 };). However, because my class might be a more complex object, I recursively call the GetInitializer method (example emitting: DateTime[] myDates = new DateTime[] { DateTime.Parse("2012-01-01") };)

else if (type.IsArray)
{
   initializer = String.Format("new {0} {{ {1} }}", CreateClassRef(type), GetInitializer(type.GetMethod("Get").ReturnParameter.ParameterType, p));
}

If my type is an enumeration, well that is easy, we can just pick a random enumeration value to assign:

else if (type.IsEnum)
{
    initializer = String.Format("{0}.{1}", type.FullName, type.GetFields()[p == 0 ? 1 : type.GetFields().Count() - 1].Name);
}

Finally, we come to complex classes. Let's say I call the GetInitializer method and the type parameter is MyClass, well I'd have to emit code to construct a MyClass object (example: MyClass aInstance = new MyClass();). But, in order to make sure I generate good code, I first have to ensure I'm not constructing an abstract class:

else if (!type.IsAbstract)
{

Then, I have to look for a parameterized constructor, where I can create an initializer for all of the parameters in the particular constructor overload. I've wanted to be adventerous and always call a parameterized constructor. If you're more cautious you can remove the o.GetParameters().Length > 0 part and call parameterless constructors as well.

var ctor = Array.Find<ConstructorInfo>(type.GetConstructors(), o => o.GetParameters().Length > 0 && !Array.Exists<ParameterInfo>(o.GetParameters(), pa => String.IsNullOrEmpty(GetInitializer(pa.ParameterType, p))));

Then, it's easy, if we didn't find any constructors, we can't create the type so we just return nothing

if (ctor == null)
   return string.Empty;

Otherwise, we start constructing our constructor call. First, we have to emit the appropriate constructor call, ie: new typeX( where typeX is our appropriate C# code reference to the type (for which I can call CreateClassRef):

else
{
   StringBuilder initBuilder = new
   StringBuilder(String.Format("new {0}(", CreateClassRef(type)));

Then, the code iterates over the parameters, and appends the parameter data to the function.

   foreach (ParameterInfo pi in ctor.GetParameters())
     initBuilder.AppendFormat("{0},", GetInitializer(pi.ParameterType, p));
   if(!initBuilder.EndsWith("(")) initBuilder.Remove(initBuilder.Length - 1, 1);
   initBuilder.Append(")");
   initializer = initBuilder.ToString();
}
}

Finally, if my type has an Add method, I can append a type initializer. So, if my type reference is List<Int32> I can emit the code: List<Int32> d = new List<Int32>() { 0 }). That is what this snippet does:

if (type.GetMethod("Add") != null && type.IsGenericType)
{
   initializer += String.Format(" {{ {0} }}", GetInitializer(type.GetGenericArguments()[0], p));
}
return initializer;
}

So, where does that leave the program? Well, we have a structure that describes our unit tests, a function that creates fancy C# code declarations for classes (CreateClassRef), a function that populates the AValues/BValues' KeyValuePair, and a function that creates the necessary code to initialize an object.

In my next post, I'll show how it all comes together to generate unit tests.

Everest Framework 1.0 Formatter Changes

Well, it's been awhile since I've posted anything on this blog and for good reason. I've been a busy little bee working on several projects and one of these is the all important rush to get Everest 1.0 ready for production. Seeing as I have to wait for 5,200 unit tests to finish, I thought I'd take some time to blog about Everest 1.0.

Everest 1.0 introduces many new features, the biggest being jEverest (a Java version of Everest), which I might blog about in the future (I might just make you wait, depends on the mood) but not today. Today's post is about the exciting world of Formatters in Everest 1.0 ... yay! I've never been happy with the way that formatting messages has been handled in Everest and when it came time to implement them in Java, I decided an overhaul was needed (don't worry, I made sure it was backwards compatible). First, let's create a simple message to illustrate the formatting process:

static
IGraphable CreateASimpleMessage()
{
   return new MCCI_IN000002CA(
      Guid.NewGuid(),
      DateTime.Now,
      ResponseMode.Immediate,
      MCCI_IN000002CA.GetInteractionId(),
      MCCI_IN000002CA.GetProfileId(),
      ProcessingID.Production,
      AcknowledgementCondition.Always);
}

I've chosen a general acknowledgement for the sample as you can tell :) Anyways, in Everest, we'd traditionally do the following to format the message and get the formatter details:

static void FormatAMessage()
{
    var fmtr = new MARC.Everest.Formatters.XML.ITS1.Formatter()
                  { ValidateConformance = false };
    fmtr.GraphAides.Add(typeof(MARC.Everest.Formatters.XML.Datatypes.R1.Formatter));
    fmtr.GraphObject(Console.OpenStandardOutput(), CreateASimpleMessage());

    // Output details
    foreach (var dtl in fmtr.Details)
      if (dtl.Type == MARC.Everest.Connectors.ResultDetailType.Error)
         Console.WriteLine("Error: {0}", dtl.Message);
}

The process should look familiar if you've used Everest before. We setup the formatter, turn off validation (I know that the message I create is invalid), and graph to the console. Then I can iterate over the Details array in the Formatter object and get the problems encountered while formatting.

While this works, it does present some problems:

1. Details[] is attached to the instance of Formatter, which means that the formatter cannot be shared across threads or concurrent functions as they will attempt to write to the same array
   
2. Anytime we want to reuse the formatter we have to call .Clone() to copy our settings over (or create one of those *gag* factories and call NewInstance())
   

So, I fixed the design issue in Everest 1.0. I liked the manner that we send/receive messages using the ISendResult and IReceiveResult interfaces in Connectors so I decided to model the formatters in the same way. We now have IFormatterGraphResult and IFormatterParseResult which can be used to share a formatter and call Graph() or Parse() on different threads. Here is an example of how this used:

static void FormatAMessageInACleanWay()
{
    var fmtr = new MARC.Everest.Formatters.XML.ITS1.Formatter()
         { ValidateConformance = false };
    fmtr.GraphAides.Add(typeof(MARC.Everest.Formatters.XML.Datatypes.R1.Formatter));

    // Now we get an IFormatterGraphResult from Graph()
    IFormatterGraphResult result = fmtr.Graph(Console.OpenStandardOutput(), CreateASimpleMessage());

    // Output details
    foreach(var dtl in result.Details)
       if (dtl.Type == MARC.Everest.Connectors.ResultDetailType.Error)
           Console.WriteLine("Error: {0}", dtl.Message);
}

Much better. I recommend using the new method of Formatting in Everest whenever you can as it is cleaner and we'll probably remove the GraphObject/ParseObject methods in future versions of Everest (not immediately though).

Also, you don't have to worry about all your Everest code not working because of this change. GraphObject and ParseObject are wrappers for this new pattern and are completely backwards compatible (at least that's what about 3,000 unit tests are telling me).

Translating Everest rc1 Details to Acknowledgement Error Codes

Everest has always included the IResultDetail interface for reporting errors back to developers that serialize or parse HL7v3 structures. More recently in Everest rc1, more implementations of this class have been added to make error reporting more granular. One of the things we can do with this data is create the transport acknowledgement details back to other systems so they can understand the errors as well.

First, let's take a look at the IResultDetail classes that come with Everest:

As you can see, the built in result detail classes all inherit from the ResultDetail. A quick review of each of the classes (from: the Tech Exchange Library)

Class	Description
MandatoryElementMissingResultDetail	Identifies that an element marked as mandatory is missing from the structure
VocabularyIssueResultDetail	Identifies that an issue has been detected with the vocabulary values specified in a structure
RequiredElementsMissingResultDetail	Identifies that an element marked as required is missing from the structure and no null flavor was specified
NotImplementedResultDetail	Identifies that an element is not recognized by the formatter and its data is lost.
InsufficientRepetitionsResultDetail	Identifies that an element does not meet the minimum number of repetitions
FixedValueMisMatchResultDetail	Identifies that the value of an element does not match its fixed value. The fixed value is taken in place of the original value.

So, how can we use these to create acknowledgement details? Well, simple, we just have to iterate through each result and create the matching code.

Let's write a simple program that reads in a file and creates a response. First, create a new console project with an appropriate CA message assembly and the R1 Formatter, and ITS1 Formatter.

Next, create we start to fill out Program.cs, let's create a function called ValidateMessage that reads a message and returns the validation errors:

static IResultDetail[] ValidateMessage(string file) {

First, we have to setup the formatter, so do that. I'm using an XML ITS1 formatter with DataTypes R1:

MARC.Everest.Formatters.XML.ITS1.Formatter fmtr = new MARC.Everest.Formatters.XML.ITS1.Formatter();
fmtr.GraphAides.Add(typeof(MARC.Everest.Formatters.XML.Datatypes.R1.Formatter));

Next, we'll open the file for read (standard .net stuff)

// Open the file
FileStream fs = null;
try
{
    fs = File.OpenRead(file);

Then we need to get the formatter to parse the object from the file, this will force a validation and will create a basic sanity check. Whenever you call ParseObject from a formatter, if the object returned is null, then the stream content was not HL7v3 content.

IGraphable obj = fmtr.ParseObject(fs);
if (obj == null)
    return new IResultDetail[] {
          new  ResultDetail(
                 ResultDetailType.Error,
                 "Bad Message",
                 (string)null
           )
    };
Next, we return the validation or Details array of the object:

return fmtr.Details;

Finish up our function:

   }
   finally
   {
 
      if (fs != null) fs.Close();
    }
}And the ValidateMessage function is complete. The next function we'll create will turn the array of IResultDetail instances to HL7v3 acknowledgement details.

static IEnumerable<AcknowledgementDetail> CreateAcks(IResultDetail[] dtls)
{


The fact that we can create this function is due to Everest's new combining functionality whereby all AcknowledgementDetail classes are combined into one class type (instead of 10). To create the Acks, we need an IEnumerable structure that we can populate. We'll iterate over the dtls parameter and create a new AcknowledgementDetail for each IResultDetail we're passed:

List<AcknowledgementDetail> retVal = new List<AcknowledgementDetail>();
 foreach(var dtl in dtls)
{
     AcknowledgementDetail ackDtl = new
     AcknowledgementDetail();
 

The first thing we need to do is populate the type code, since the IResultDetail class isn't used by the generated assembly, we need to do a manual translate:

switch(dtl.Type)
{
    case ResultDetailType.Error:
        ackDtl.TypeCode = AcknowledgementDetailType.Error;
        break;
    case ResultDetailType.Warning:
        ackDtl.TypeCode = AcknowledgementDetailType.Warning;
        break;
    case ResultDetailType.Information:
        ackDtl.TypeCode = AcknowledgementDetailType.Information;
        break;
}

Next, we want to translate the type of IResultDetail to a code, here are some examples of how I would do this:

if (dtl is InsufficientRepetionsResultDetail)
    ackDtl.Code = AcknowledgementDetailCode.InsufficientRepetitions;
else if (dtl is MandatoryElementMissingResultDetail)
    ackDtl.Code = AcknowledgementDetailCode.MandatoryElementWithNullValue;
else if (dtl is NotImplementedElementResultDetail)
    ackDtl.Code = AcknowledgementDetailCode.SyntaxError;

 There would be an if-statement for each IResultDetail you'd want to translate. Next, I just fill in the location and text:

ackDtl.Text = dtl.Message;
ackDtl.Location = new
SET<ST>((ST)dtl.Location);

Then we add it to our return value and finish up the function:

     retVal.Add(ackDtl);
    }

  return retVal;
}


Now all we have to do is fill out our Main function to load the message and create a response. I'm going to use MCCI_IN000002CA just to illustrate.


static void Main(string[] args)
{



MCCI_IN000002CA ret = new MCCI_IN000002CA(
    Guid.NewGuid(),
    DateTime.Now,
    ResponseMode.Immediate,
    MCCI_IN000002CA.GetInteractionId(),
    MCCI_IN000002CA.GetProfileId(),
    ProcessingID.Production,
    AcknowledgementCondition.Never
);ret.Acknowledgement = new Acknowledgement(
    AcknowledgementType.ApplicationAcknowledgementAccept,    new TargetMessage());
ret.Acknowledgement.AcknowledgementDetail.AddRange(
    CreateAcks(ValidateMessage("C:\\test.xml"))
);


// Format to the screen
MARC.Everest.Formatters.XML.ITS1.Formatter fmtr = new MARC.Everest.Formatters.XML.ITS1.Formatter();
fmtr.GraphAides.Add(typeof(MARC.Everest.Formatters.XML.Datatypes.R1.Formatter));
fmtr.ValidateConformance = false; // Just a sample, I know this message isn't correct
fmtr.GraphObject(Console.OpenStandardOutput(), ret);

 }


And voila, you should get a proper AcknowledgementDetail message.