Emitting XML Comments from Everest

A great question came to me the other day. How does one pretty-up the XML output generated by Everest so that humans can read the XML? Of course there are the old tricks of indentation and formatting the output however that only gets us so far.
Wouldn’t it be great if Everest had the capacity to emit comments in the XML instances. Sadly this isn’t a use case for vanilla Everest however there is a way to easily do this in the upcoming 1.2 release of Everest (being released on June 5th BTW).
Lets say I want to emit a comment that annotates the <acceptAckCode> element, something like this:
<?xml version="1.0" encoding="utf-8"?>
<PRPA_IN101301UV02 ITSVersion="XML_1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:hl7-org:v3">
  <id root="F043D3BF-02C4-48BF-8C7E-6FFEE7D75B52" />
  <creationTime value="20130531171954.939-0400" />
  <interactionId root="2.16.840.1.113883.1.18" extension="PRPA_IN101301UV02" />
  <processingCode code="T" />
  <processingModeCode code="A" />
  <!--The acknowledgement code—>
  <acceptAckCode code="AL" />
</PRPA_IN101301UV02> The way I would do this is to add an extension method to IGraphable to allow users to add commnents: public static class CommentExtension
{
    private static List<KeyValuePair<IGraphable, String>> s_comments = new List<KeyValuePair<IGraphable, string>>();
    private static Object s_syncLock = new object();
    public static void AddComment(this IGraphable me, string comment)
    {
        lock (s_syncLock)
                s_comments.Add(new KeyValuePair<IGraphable, String>(me, comment));
    }
    public static string GetComment(this IGraphable me)
    {
        return s_comments.Find(o=>o.Key == me).Value;
    }
}
We can then extend the XmlIts1Formatter and override the WriteElementUtil method to emit the comment added prior to serializing the element:
public class XmlIts1FormatterWithComments : XmlIts1Formatter
{
    public override void WriteElementUtil(System.Xml.XmlWriter s,
               string elementName,
               MARC.Everest.Interfaces.IGraphable g,
               Type propType,
               MARC.Everest.Interfaces.IGraphable context,
               XmlIts1FormatterGraphResult resultContext)
    {
        String comment = g.GetComment();
        if (comment != null)
            s.WriteComment(comment);
        base.WriteElementUtil(s, elementName, g, propType, context, resultContext);
    }
}
Then, using this new formatter we can simply add the comment and format!


PRPA_IN101301UV02 test = new PRPA_IN101301UV02(
    Guid.NewGuid(),
    DateTime.Now,
    PRPA_IN101301UV02.GetInteractionId(),
    ProcessingID.Training,
    ProcessingMode.Archive,
    AcknowledgementCondition.Always);
test.AcceptAckCode.AddComment("The acknowledgement code");
var formatter = new XmlIts1FormatterWithComments();
formatter.GraphAides.Add(new DatatypeFormatter());
formatter.Graph(Console.OpenStandardOutput(), test);
Console.ReadKey();
Hope that helps anyone else with the same problem!

Everest for Windows Phone 7–WCF Connector

Well, as I mentioned in my previous post, Everest is here for WP7. Currently we’re still sorting out some bugs but the first bit of code is available in the 1.1 branch on our public SVN.

So, what can you do with Everest for WP7? Well, pretty much anything you’d like to do with HL7v3. I made a sample application that will hit the MARC-HI client registry and shared health record to demonstrate how the WCF Connector works in the mobile version of Everest (it is slightly different than the full fledged desktop version).

We’re going to be making this app:

Create a simple Windows Phone 7 application in Visual Studio 2010 (like you normally would) and add these references to your project:

• MARC.Everest.Phone.dll
• MARC.Everest.Phone.Connectors.WCF.dll
• MARC.Everest.Phone.Formatters.XML.ITS1.dll
• MARC.Everest.Phone.Formatters.Datatypes.R1.dll
• MARC.Everest.RMIM.CA.R020402.Phone.dll

First, we’ll need to create a ServiceReferences.ClientConfig file and set its build action to Content. In that file, we’ll configure the WCF client configuration (similar to the Everest desktop version):

<configuration>
    <system.serviceModel>
        <bindings>
            <basicHttpBinding>
                <binding name="hl7v3Binding" maxBufferSize="2147483647"
                    maxReceivedMessageSize="2147483647">
                    <security mode="None" />
                </binding>
            </basicHttpBinding>
        </bindings>
        <client>
            <endpoint address="http://cr.marc-hi.ca:8080/cr" binding="basicHttpBinding"
                bindingConfiguration="hl7v3Binding"
                contract="MARC.Everest.Connectors.WCF.Core.IConnectorContract" name="ClientRegistry" />
        <endpoint address="http://shr.marc-hi.ca:8080/shr" binding="basicHttpBinding"
            bindingConfiguration="hl7v3Binding"
            contract="MARC.Everest.Connectors.WCF.Core.IConnectorContract" name="SHR" />
      </client>
    </system.serviceModel>
</configuration>

Note that WCF for the phone platform only supports basicHttpBinding and customBinding (no wsHttpBinding so SOAP 1.2 will need adjustments).

Next, we’ll initialize two connectors: one for the client registry and one for the shared health record. Because of the way that the WCF connectors work on Silverlight mobile (async only) you’re going to be using BeginSend and EndSend methods so these methods are fields for the Main.xaml.cs class:

private WcfClientConnector m_clientRegistryConnector;
private WcfClientConnector m_sharedHealthRecordConnector;

Next, we’re going to need a shared reference to the resolved patient identity received from the client registry, this is done via another field:

private MARC.Everest.RMIM.CA.R020402.PRPA_MT101104CA.IdentifiedEntity m_patientIdentification;

Since the WCF client connector will be sending the message asynchronously, meaning the actual sending is done on a separate thread. When the result callback is executed, it will be in the context of the thread that sent the message and not the thread that the UI is running on, we need to use the dispatcher to update the UI. Let’s declare a delegate to perform the UI update:

delegate void MessageSentDelegate(IGraphable connector);

Now, we can initialize the WCF connectors on the main page. I’ve done this in the constructor after InitializeComponent() is executed (which is hackish, but this is a tutorial after all).

// Constructor
public MainPage()
{
     InitializeComponent();
    m_clientRegistryConnector = new WcfClientConnector("endpointname=ClientRegistry");
    m_clientRegistryConnector.Formatter = new XmlIts1Formatter() { ValidateConformance = false };
    m_clientRegistryConnector.Formatter.GraphAides.Add(new DatatypeFormatter());
    m_clientRegistryConnector.Open();
    m_sharedHealthRecordConnector = new WcfClientConnector("endpointname=SHR");
    m_sharedHealthRecordConnector.Formatter = new XmlIts1Formatter() { ValidateConformance = false };
    m_sharedHealthRecordConnector.Formatter.GraphAides.Add(new DatatypeFormatter());
    m_sharedHealthRecordConnector.Open();
    ResolveClientIdentifiers();
}

Our resolve client identifiers code will execute a client registry lookup to translate our local identifier (I’m using 000-123-456) and get additional information about the client (their name, address, etc.). Here is the code:

private void ResolveClientIdentifiers()
{
    PRPA_IN101103CA instance = new PRPA_IN101103CA(
        Guid.NewGuid(),
        DateTime.Now,
        ResponseMode.Immediate,
        PRPA_IN101103CA.GetInteractionId(),
        PRPA_IN101103CA.GetProfileId(),
        ProcessingID.Production,
        AcknowledgementCondition.Always,
        new MARC.Everest.RMIM.CA.R020402.MCCI_MT002200CA.Receiver(
            null,
            new MARC.Everest.RMIM.CA.R020402.MCCI_MT002200CA.Device2(
                new II("1.3.6.1.4.1.33349.3.1.1.2", "CR")
            )
        ),
        new MARC.Everest.RMIM.CA.R020402.MCCI_MT002200CA.Sender(
            new MARC.Everest.RMIM.CA.R020402.MCCI_MT002200CA.Device1(
                new II("1.3.6.1.4.1.33349.3.1.1.20.4", "MARC-W1-1")
            )
        ),
        new MARC.Everest.RMIM.CA.R020402.MFMI_MT700751CA.ControlActEvent<MARC.Everest.RMIM.CA.R020402.PRPA_MT101103CA.ParameterList>(
            Guid.NewGuid(),
            PRPA_IN101103CA.GetTriggerEvent(),
            new MARC.Everest.RMIM.CA.R020402.MFMI_MT700711CA.Author(
                DateTime.Now
            ),
            new MARC.Everest.RMIM.CA.R020402.QUQI_MT120008CA.QueryByParameter<MARC.Everest.RMIM.CA.R020402.PRPA_MT101103CA.ParameterList>(
                Guid.NewGuid(),
                new MARC.Everest.RMIM.CA.R020402.PRPA_MT101103CA.ParameterList()
                {
                    ClientId = new List<MARC.Everest.RMIM.CA.R020402.PRPA_MT101103CA.ClientId>() {
                        new MARC.Everest.RMIM.CA.R020402.PRPA_MT101103CA.ClientId(
                            new II("1.3.6.1.4.1.33349.3.1.3.12", "000-123-456")
                        )
                    }
                }
            )
        )
        {
            EffectiveTime = new IVL<TS>(DateTime.Now)
        }
    );
    instance.ProfileId[0].Extension = "R02.04.02";

    instance.controlActEvent.Author.SetAuthorPerson(new MARC.Everest.RMIM.CA.R020402.COCT_MT090502CA.AssignedEntity(
        new MARC.Everest.RMIM.CA.R020402.COCT_MT090102CA.Organization(
            new II("1.2.840.114350.1.13.99998.8734"),
            "Good Health Hospital"
        )
    ));
     var retVal = m_clientRegistryConnector.BeginSend(instance, this.CrConnectorCallback, null);
            

}

You will notice that the callback for the BeginSend method is a callback delegate CrConnectorCallback. This method will be executed on the thread that the message was processed on, and will use the Dispatcher object to update the UI:

private void CrConnectorCallback(IAsyncResult result)
{
            
    var sendResult = this.m_clientRegistryConnector.EndSend(result);
    var res = this.m_clientRegistryConnector.Receive(sendResult);
    this.Dispatcher.BeginInvoke(
            new MessageSentDelegate(this.MessageSentResult), 
            new object[] { res.Structure });
            
}

This callback executes the MessageSendResult delegate (of type MessageSentDelegate that we created earlier). This updates the UI to include the patient’s name in the Application Text area.

private void MessageSentResult(IGraphable message)
{
     if(message is PRPA_IN101104CA)
    {
        var msg = message as PRPA_IN101104CA;
        if(msg.Acknowledgement.TypeCode != AcknowledgementType.ApplicationAcknowledgementAccept)
            return;
        var rreg = msg.controlActEvent.Subject[0].RegistrationEvent.Subject.registeredRole;
        this.m_patientIdentification = rreg;
        this.ApplicationTitle.Text = rreg.IdentifiedPerson.Name[0].ToString("{FAM}, {GIV}");
        ExecuteGetSummary();
    }
}

The result of this will be an app that, a few seconds after loading, will display the name of patient 000-123-456 in the application text bar.

In my next post, I’ll show you how to get the clinical summary using COMT_IN100000CA messages with query continuation (a very useful construct on the phone).

Everest for Windows Phone 7.1

Here at the Everest team we’ve been so busy building the Java version of Everest (jEverest) that I haven’t had a chance to discuss another exciting Everest feature, Windows Phone 7 Silverlight libraries.

Basically we’ve done some cross-compiling in Visual Studio to remove the features of Everest that don’t work on the mobile platform, affected features are:

1. Code generator formatter is disabled since the mobile platform doesn’t support code-dom or runtime code generation.
2. The only connector supported is the WCF client connector
3. XML ITS1 and Data Types R1 is currently only supported in the phone assemblies.

The current 1.1 unstable branch of Everest contains the Everest Phone code in the api.phone.sln code. As with all code in the unstable branch it is untested and your mileage may vary.

Ciao

jEverest Bubbling Away

Well, it has been a busy week and the Everest team (mostly yours truly) has been busy churning the jEverest code. I’m pleased to report that there has been lots of progress made as far as core functionality goes.

We have a basic core library! That’s right there is a working copy of ca.marc.everest.jar containing about 85% of the HL7v3 data-types to Java. Not all the methods are fully fleshed out but structurally everything is there (no jUnit tests yet, volunteers?). The core library is at a point where it can be used with the latest unstable GPMR to generate Java classes for RMIMs.

Wait, did you say RMIMS? Why yes I did! Right now the latest unstable version of GPMR will generate RMIM jarchives for Canadian release 2.04.01, 2.04.02 and 2.04.03 (just trying out the NE2008 one right now). You can’t really do much with these until the other infrastructure components come online, namely the formatters and formatter utilities.

There is still a long way to go but for those wishing to see what Everest RMIM JARs will look like can download the latest unstable at: https://fisheye.marc-hi.ca/svn/Everest/branches/1.1/ (un: Guest). The copy of GPMR will render Java (after being compiled) and executing:

gpmr -c -r RIMBA_JA -o C:\java\r02.04.01  --quirks --rimbapi-gen-vocab=true --rimbapi-compile=true --optimize=true --combine=true --rimbapi-profileid=R02.04.01 --rimbapi-target-ns=ca.marc.everest.rmim.ca.r020401 --rimbapi-license=apache --rimbapi-org="Health Level Seven" --rimbapi-jdk="C:\Program Files (x86)\Java\jdk1.6.0_26" --rimbapi-jdoc=true  .\CA\R02.04.01\*.*mif

After a few minutes GPMR will spit out a complete Eclipse project (it will also attempt to compile the resulting files).

Source code for the core jEverest file can be found in the jEverest folder.

Java Everest (jEverest) Update

I was looking through some old Everest code on the trunk and 1.0 branch about a month ago and realized that the jEverest project had stalled. I was trying to think of a killer new feature for Everest 1.2 and I hadn’t realized the answer was staring me right in the face this whole time.

jEverest is a Java version of Everest which was intended to be completed by January of this year, however because of more pressing issues it was dropped in favour of other work (such as the Client Registry reference implementation and Shared Health Record implementations). But not anymore. We’ve made some really good progress on the datatypes and supporting classes and with our public SVN server you can take a look!

jEverest is in the 1.1 branch (1.2 development) of the Everest project on the server. So far we have completed a port of the majority of data types and some enhancements to the GPMR java renderer. Hopefully we’ll have something to demonstrate by HL7 WGM in January but I wouldn’t expect a release until later in 2013.

Anyone who has talked to me (in person) knows that I’m not the biggest fan of the Java platform (IMO C# and .NET is much better) however it is not nearly as terrible as I had remembered it. Java 1.7 does have some nice features (like strings in switch cases, finally!) however I do have some legitimate complaints about Java development (just observations, don’t want to start a war)

1. Eclipse : Nice IDE, lots of features and bells and whistles however I’d much rather prefer an IDE which has less features but does them properly. For example:
   • Subclipse isn’t the best when you’re in the “Java perspective”, it simply wraps the SVN command line and places conflict markers directly in your code which I then have to resolve by “Right Click -> Open in Conflict Editor”. Why? Maybe I’m just used to VisualSVN.
   • Why are there two places to get Eclipse extensions? Some extensions I have to visit “Help -> Install New Software” and others “Help -> Eclipse Marketplace”. There should be one place to get extensions.
   • On the topic of extensions, my major complaint with Eclipse is the plethora of options. Everywhere I look, options options options! The context menus are huge and cluttered with options.
2. Type Erasure : By far, this is my number one pet-peeve with Java. The fact that LIST<REAL> at runtime is indistinguishable from LIST<INT>. This causes lots of problems especially since the majority of data types use generics. Every time I implement a conversion method or validation method that really does need to know the type of a generic I can’t (or I have to use some hack)
3. Closures : Believe it or not, this actually does matter in jEverest, especially when searching a collection. In Everest all collections have a Find() method (for example, to find all numbers in a list larger than a given number : for(int i = 0; i < 10; i++) numbers.FindAll(o=>o > i);). This can be a problem jEverest and to get around it I have to use a custom IPredicate / Predicate implementation. (contrast the previous Everest example with jEverest : for(int i=0; i<10; i++) numbers.findAll(new Predicate<INT>(i) { public bool match(INT c) { return c > this.getScopeValue(); } }); )

Overall these aren’t showstoppers, just require some clever trickery to make compatible solutions. The idea behind jEverest isn’t to be 100% code compatible with the .NET version, but to be a meaningful port of the functionality.

I’ll end on a positive note: Java enumerations are really cool and much better than C#/.NET. Being a class means that enumerations in Java can implement interfaces and I’ve taken full advantage of this for representing enumerated vocabularies (via IEnumeratedVocabulary) and hierarchical vocabularies (via IHierarchicEnumeratedVocabulary).

Keep your eyes open, jEverest is coming!

Custom SOAP Headers

Wow, it has been awhile since I've been able to post any new content on this blog. It has been a very busy summer and I have to say that there are a few great projects that I've been working on and hope to release to the wild in a few weeks/months time.

In the meantime, I thought I'd post some code samples illustrating some of the new features of the WCF connector in the upcoming Everest 1.0.2 release (a minor update and bug-fix release). If you've ever had to use the WcfClientConnector or WcfServerConnector, you'll know that dealing with SOAP headers was … well… cumbersome. It required setting some odd properties and wasn't really thread safe. This has been addressed in the 1.0.1 release.

For this example, I'll show you how to read/write the WS-Addressing headers in a received message from the WCF connectors. First, to access the SOAP headers from a message received from a WcfServerConnector, you can simply access the Headers property on the WcfReceiveResult. The following code is written in the MessageAvailable event handler for a WcfServerConnector:

static void conn_MessageAvailable(object sender, MARC.Everest.Connectors.UnsolicitedDataEventArgs e)
{
     // Get the sending connector that raised the event
     var connector = sender as  WcfServerConnector;
     if (connector == null)
          throw new ArgumentException("Must be called from a WcfServerConnector", "sender");
     // Receive the message
     var receiveResult = connector.Receive() as WcfReceiveResult;

Pretty standard Everest stuff, next we'll emit the value of the WS-Addressing headers:

if (receiveResult.Headers != null){

     Console.WriteLine(receiveResult.Headers.To);
     Console.WriteLine(receiveResult.Headers.Action);
}

We can access the Headers array just like any other WCF Header, this applies to constructing the response as well. To construct the response, populate the ResponseHeaders on the receiveResult prior to call "Send()" on the server connector.

receiveResult.ResponseHeaders = new System.ServiceModel.Channels.MessageHeaders
(receiveResult.Headers.MessageVersion);
receiveResult.ResponseHeaders.Add(MessageHeader.CreateHeader("myHeader", "urn:my-ns:com", "Value"));
connector.Send(new MCCI_IN000002CA(), receiveResult);

This code will return the following soap header:

<tns:myHeader xmlns:tns="urn:my-ns:com">Value</tns:myHeader>

More complex headers can be added the same way you would add standard System.ServiceModel.Channel.MessageHeader objects. It is also possible to send message headers using the overridden Send() method on the WcfClientConnector:

var conn = new WcfClientConnector();
// trimmed
MessageHeaders messageHeaders = new System.ServiceModel.Channels.MessageHeaders(MessageVersion.Soap12);messageHeaders.Add(MessageHeader.CreateHeader("myHeader", "urn:my-ns:com", "Value"));
conn.Send(instance, messageHeaders);

As I said, this enhancement will be available in the 1.0.2 release of Everest, or is available in the 1.0 branch of the Everest SVN server (or the trunk if you're feeling adventurous and want to try Everest 1.2 features out as well).

Everest Clean-Up

It's been awhile since I've posted anything new on this blog and I must apologize, we've been very busy with various projects at the MARC-HI (now called MEDIC) and I haven't been able to post any updates.
One of the tasks that we've performed is a clean-up of the Everest source code, a migration to SVN, and a new roadmap for Everest features.

Clean-up of Source Code

The Everest source code has undergone some clean-up to facilitate the migration to SVN. This includes better documentation on the source code itself, clearer versioning (see roadmap) and some general enhancements to the way things are done in the Everest Framework.

Public SVN Access

The Everest source code is now publicly available via SVN at https://fisheye.marc-hi.ca/svn/Everest. Read only guest access is provided via the "Guest" account, patches can be submitted via the forums or e-mailed to an Everest team member.
The structure of the SVN repository is as follows:
/branches/1.0 – The branch from which the Everest 1.0 release was built
/branches/1.1 – The unstable "feature" branch for the 1.2 release of Everest
/trunk – Stable development trunk.
This branching structure may seem odd but we do have some rationale for structuring the repository like this. The development branches (odd numbers) are not guaranteed to compile or pass unit tests and are where we do the majority of our work on the main roadmap, whereas the trunk is always guaranteed to compile and pass regression tests and only includes new features that are stable.

New Versioning Scheme

The new Everest versioning scheme is as follows:

Major.Minor.Patch

Where Major is the major version (currently the 1.0 series) which are guaranteed to be backwards compatible with previous versions. Minor version numbers are revision or releases which introduce new functionality to the framework. Odd minor version numbers denote a development version whereas even numbers are releases (for example 1.1 is the development branch for 1.2 and 1.3 for 1.4, etc.).
Patches are normally used for tags and represent a collection of bug fixes for the framework components.
The team hopes that these changes will make Everest clearer and will open the gateway to encourage new developers to work on the framework. We're working on a methodology for allowing public commits to our repositories however this will most likely not be completed for quite some time.

Everest in WCF vs. WCF in Everest

The connector architecture in the Everest framework is designed to be consistent regardless of the underlying transport. The four patterns provided by the connectors in Everest (Listen/Wait, Listen/Wait/Respond, Send and Send/Receive) provide common patterns for communicating on a variety of transports including queues, http, files and others.

When it comes to communicating or hosting services that respond using the WCF framework, the current Everest connector pattern is implemented in the form of a WcfServerConnector and WcfClientConnector. While this is great for consistency with the rest of the Everest framework it is awkward for developers who are familiar with WCF. In this manner of speaking WCF is used within Everest. In order to be truly easy to use, we needed to make Everest in WCF.

The current development branch of Everest (1.1) includes this functionality. Consider the process of receiving a message on the WCF connector, the function looks something like this:

/// <summary>

/// Message is available
/// </summary>
static void connector_MessageAvailable(object sender, MARC.Everest.Connectors.UnsolicitedDataEventArgs e)
{

    var connector = sender as IListenWaitRespondConnector;

    var receiveResult = connector.Receive();

    // Determine the type of interaction

    if (receiveResult.Structure is PRPA_IN201305UV02)
        ; // Do something here
    if (receiveResult.Structure is PRPA_IN201307UV02)
        ; // Do something here

    connector.Send(transactionResult, receiveResult);

}

Which makes sense from an Everest connectors point of view, however it is quite awkward to a typical WCF developer. In the new Everest framework version we can use the familiar WCF pattern as illustrated below:

[EverestSerializerFormat(Formatter = typeof(XmlIts1Formatter), GraphAide = typeof(DatatypeFormatter), ValidateConformance = false)]
[ServiceContract(Namespace="urn:hl7-org:v3")]
public interface IServiceContract
{
    /// <summary>
    /// Do something
    /// </summary>
    [OperationContract(Action="*")]
    IGraphable Anything(IGraphable request);

}
public class ServiceBehavior : IServiceContract
{

    #region IServiceContract Members

    /// <summary>

    /// Default message
    /// </summary>
    /// <returns></returns>
    public IGraphable Anything(IGraphable request)
    {
     
         // Determine the type of interaction
       if (request is PRPA_IN201305UV02)
          ; // Do something here
         if (request is PRPA_IN201307UV02)
          ; // Do something here
                    
        return transactionResult;        
    }

    #endregion

}

The EverestSerializerFormat attribute is very similar in functionality to the XmlSerializerFormat attribute and instructs the WCF layer to use Everest as the serialization engine rather than the built-in XML or data contract serializer.

Alternatively, developers can now piggy-back on the other WCF services to classify operations that are to be executed:

[EverestSerializerFormat(Formatter = typeof(XmlIts1Formatter), GraphAide = typeof(DatatypeFormatter), ValidateConformance = false)]
[ServiceContract(Namespace="urn:hl7-org:v3")]
public interface IServiceContract
{
    /// <summary>
    /// Handle PRPA_IN201305UV02
    /// </summary>
    [OperationContract(Action="PRPA_IN201305UV02")]
    PRPA_IN201306UV02 DoSomething(PRPA_IN201305UV02 request);

    /// <summary>

    /// Handle PRPA_IN201307UV02
    /// </summary>
    [OperationContract(Action = "PRPA_IN201307UV02")]
    PRPA_IN201308UV02 DoSomethingB(PRPA_IN201307UV02 request);

}

One limitation of using the EverestSerializerFormat attribute is the lack of access to validation errors and the result details.

Everest Developer’s Handbook Available

One of the big projects that I've been working on over the past year is the creation of an in-depth guide to the Everest framework intended to be used by developers. I'm happy to announce "The Official Advanced Everest Developer's Handbook" is now available for purchase at lulu.com in both e-Book and Hardcover editions. As part of the launch, I am providing the e-Book free of charge and a 25% discount on the beautiful hardcover version for the month of May 2012.
This handbook is an in-depth look at the Everest Framework components and is intended to document how the Everest Framework implements various HL7v3 constructs and includes:

• Over 100 examples of using various Everest Framework components
• In-depth documentation about the Everest Data Types library
• Documentation about the GPMR tooling and generating new RMIM assemblies
• Documentation about advanced concepts of the framework such as attributes, utility classes and writing custom formatters and connectors.

Proceeds from the sales of the books are used to fund further enhancements to the Everest Framework at Mohawk College.
"The Official Advanced Everest Developer's Handbook (Hardcover)"

"The Official Advanced Everest Developer's Handbook (e-Book)"

Finally, a shout goes out to Ken Wong, a Mohawk graduate who donated some of his free time in designing the Cover and first page artwork. Obviously the art pays homage to the "Advanced Dungeons and Dragons" books released in the early 1980's.

Everest 1.0 Is Here!

It has been a long journey, but the Everest Framework 1.0 release is now available at the MARC-HI Technology Exchange (http://everest.marc-hi.ca). There are a multitude of new features and usability enhancements include:

• Support for NE2010, NE2011 and CDAr2 (Though only base versions of NE2010 and CDAr2 are included)
  
• Formatters available for Data Types R2 (ISO21090:2009)
  
• A new GPMR Wizard which makes turning MIFs into DLLs much easier
  
• Changes in the way that GraphAides are added to formatters
  
• Improvements in formatter error detail reporting
  

The next version on our roadmap is 1.5 slated for a 2013 release. Features so far include:

• A JSON ITS Formatter
  
• A REST based connector
  
• Improvements to way that IListenWaitConnector and IListenWaitRespondConnector implementations raise notifications of messages received
  
• JavaScript renderer for GPMR (output JS files that can be used to construct JSON objects using the JSON ITS)
  

The whole Everest team hopes that you enjoy the new version of Everest. Please feel free to leave comments on our forums (http://te.marc-hi.ca/forums) or submit any issues you encounter.

Collection Shortcuts in Everest 1.0

The goal of the Everest framework data types is to provide functionality that allows developers to easily construct and interact with the HL7v3 data types. In previous versions of the Everest Framework, creating collections could be difficult. Consider the AD data type which is nothing more than a collection of ADXP components. In previous versions of Everest, creating this structure would look something like this:

AD homeAddress = new AD(
  new SET<CS<PostalAddressUse>>()
  {
     PostalAddressUse.Alphabetic,
     PostalAddressUse.Direct
  },
  new ADXP[] {
     new ADXP("123 Main Street", AddressPartType.StreetAddressLine),
     new ADXP("West", AddressPartType.Direction),
     new ADXP("Hamilton", AddressPartType.City),
     new ADXP("Ontario", AddressPartType.State),
     new ADXP("Canada", AddressPartType.Country)
  }
);

This code can be quite large and is difficult to track if not styled properly (indentation is the key here). So, to make the construction of sets a little easier, we've added static "creator" methods on each of the collection data types. They are used as follows:

AD homeAddress = AD.CreateAD(
   SET<PostalAddressUse>.CreateSET(
      PostalAddressUse.Alphabetic,
      PostalAddressUse.Direct
   ),
   new ADXP("123 Main Street", AddressPartType.City),
   new ADXP("West", AddressPartType.Direction),
   new ADXP("Hamilton", AddressPartType.City),
   new ADXP("Ontario", AddressPartType.State),
   new ADXP("Canada", AddressPartType.Country)
);

The benefit of this shortcut is illustrated better with more complex sets such as SXPR and QSET. The following snippet represents the construction of an SXPR that represents numbers {1..10}, intersected with the result of a union of numbers {3..5} and {7..9}.

SXPR<INT> result = new SXPR<INT>()
{
   new IVL<INT>(1, 10),
   new SXPR<INT>() {
       Operator = SetOperator.Intersect,
       Terms = new LIST<SXCM<INT>>() {
          new IVL<INT>(3, 5),
          new IVL<INT>(7, 9)
          {
              Operator = SetOperator.Inclusive
          }
       }
   }
};

Using the new constructor which is a shortcut, the following expression can be used:

SXPR<INT> result = new SXPR<INT>(
   new IVL<INT>(1, 10),
   new SXPR<INT>(
       new IVL<INT>(3, 5),
       new IVL<INT>(7, 9)
       {
           Operator = SetOperator.Inclusive
       }
   )
   {
      Operator = SetOperator.Intersect
   }
);

These new shortcut methods are intended to assist developers even more than previous attempts at the Data Types implementation and are one of the many improvements in the Everest 1.0 data types library.

SNOMED Expressions in Everest 1.0

Wow, it has been a long time since I have posted a new article on this blog and it has mainly been because of the crazy-busy schedule at work. But, I just thought I had to share this with everyone in the HL7 integration world looking at implementing Data Types R2 and R1 (or those migrating R1 to R2), and some of the benefits that Everest 1.0 will have when released (it is getting close).

First, I will provide a little bit of an overview. In HL7v3 clinical concepts within messages are represented using one of four different data types (two in the Data Types R2 specification). These are:

R1	R2	Summary
CS	CS	Coded Simple - A simple code where only the code mnemonic is unknown
CV	CD.CV	Coded Value – A more complex code structure whereby the code system from which the mnemonic is taken is unknown at design time.
CE	CD.CE	Code with Equivalents – A CV instance where translations (or equivalents) can optionally be specified.
CD	CD	Concept Descriptor – A code mnemonic taken from a code system, optionally with one or more concept roles which qualify the primary code. For example, the code LEFT qualifies FOOT to mean LEFT FOOT.
CR	N/A	Concept Role - A name/value pair where the value concept qualifies the semantic meaning of the primary mnemonic by way of the named concept.

Wait a minute! Notice some differences? Well, for starters CV and CE are no longer "proper" types according to the data types, they are flavors of CD. This is an appropriate change as they remain structurally identical to the R1 structures.

The big change comes in the concept descriptor. Notice how the CR data type is not present in data types R2. When I first saw this I thought nothing of it, however when looking at how each are represented on the wire the difference is very pronounced.

I'm a code kind of guy, so I thought I would explain this using code. First off, Everest uses a hybrid of DT R1 and R2, so the codified data types in Everest resemble those found in R1 (and are mapped to appropriate R2 flavors on formatting). With that in mind, let's represent the following example: "severe burn on the skin between the fourth and fifth toes on the left side", in Everest.

First, we create the primary code of "burn":

var burnCode = new CD<string>("284196006", "2.16.840.1.113883.6.96") {

DisplayName = "Burn of Skin",

CodeSystemName = "SNOMED-CT",

CodeSystemVersion = "2009"

};

Next, the we want to qualify "Burn of Skin" with a severity of "Severe". This is accomplished by creating a CR instance:

// Severity
var severityQualifier = new CR<string>(
   new CV<String>("246112005", "2.16.840.1.113883.6.96")
            { DisplayName = "Severity" },
   new CD<String>("24484000", "2.16.840.1.113883.6.96")
            { DisplayName = "Severe" }
);

Next, our code has a finding site. The burn was located on the skin between the fourth and fifth toes, so once again it is another CR instance:

// Finding Site
var findingSiteQualifier = new CR<String>(
    new CV<String>("363698007", "2.16.840.1.113883.6.96")
             { DisplayName = "Finding Site" },
    new CD<String>("113185004", "2.16.840.1.113883.6.96")
             { DisplayName = "Skin Between fourth and fifth toes" }
);

Next, we want to describe the fact that the the burn was found on the skin between the fourth and fifth toes "on the left hand side". Obviously we want to create another qualifier for this:

// Laterality
var lateralityQualifier = new CR<String>(
    new CV<String>("272741003", "2.16.840.1.113883.6.96")
            { DisplayName = "Laterality" },
    new CD<string>("7771000", "2.16.840.1.113883.6.96")
            { DisplayName = "Left Side" }
);

But how would we structure these concept roles to describe the situation? First we have to look at each term and ask the question, "What does this qualify". So, for example, does "Laterality of Left Side" qualify the burn? Technically no, the laterality qualifies the finding site (i.e.: We found the burn on the toes on the left hand side). So we want to add the lateralityQualifier to the findingSiteQualifier's value:

// Laterality applies to the finding site
findingSiteQualifier.Value.Qualifier = new LIST<CR<string>>() { lateralityQualifier };

What does finding site qualify? Technically finding doesn't qualify the severity it qualifies the primary code (i.e.: The burn was "found on" the skin…), and the same with applies to the severity (i.e.: We didn't find a severe skin between toes, we found a severe burn). So we add these two qualifiers to the primary code:

// Finding site and severity apply to primary code
burnCode.Qualifier = new LIST<CR<string>>() {
     severityQualifier,
     findingSiteQualifier
};

Now comes the easy part, when we format the data type using data types R1 formatter:

var formatter = new MARC.Everest.Formatters.XML.ITS1.Formatter();
formatter.ValidateConformance = false;
formatter.GraphAides.Add(
     typeof(MARC.Everest.Formatters.XML.Datatypes.R1.DatatypeFormatter)
);

// Setup the writer
StreamWriter sw = new StreamWriter("C:\\temp\\temp.xml");
XmlWriter xw = XmlWriter.Create(sw, new XmlWriterSettings() { Indent = true });
XmlStateWriter xsw = new XmlStateWriter(xw);

// Format and produce the XML file
try
{
     xsw.WriteStartElement("code", "urn:hl7-org:v3");   xsw.WriteAttributeString("xmlns", "xsi", null, http://www.w3.org/2001/XMLSchema-instance);
   var p = formatter.Graph(xsw, burnCode);   sw.WriteEndElement();
}
finally
{    xw.Close();    sw.Flush();
    formatter.Dispose();
}

The output of this is the following XML:

<code code="284196006" codeSystem="2.16.840.1.113883.6.96" codeSystemName="SNOMED-CT" codeSystemVersion="2009" displayName="Burn of Skin">
   <qualifier inverted="false">
      <name code="246112005" codeSystem="2.16.840.1.113883.6.96"
            displayName="Severity" />     
      <value code="24484000" codeSystem="2.16.840.1.113883.6.96"
            displayName="Severe" />
   </qualifier>
   <qualifier inverted="false">
      <name code="363698007" codeSystem="2.16.840.1.113883.6.96"
            displayName="Finding Site" />
      <value code="113185004" codeSystem="2.16.840.1.113883.6.96"
             displayName="Skin Between fourth and fifth toes">
          <qualifier inverted="false">             
              <name code="272741003" codeSystem="2.16.840.1.113883.6.96"
                    displayName="Laterality" />
              <value code="7771000" codeSystem="2.16.840.1.113883.6.96"
                     displayName="Left Side" />
          </qualifier>     
      </value>
   </qualifier>
</code>

But as I mentioned previously, CR is not supported in DT R2. So the question arises, "How do I qualify a code in HL7v3 DT R2?". Well, the answer is not so simple. In DT R2, the concepts for SNOMED terms are described using an expression language defined by IHTSDO. The SNOMED expression for our scenario is:

284196006|Burn of Skin|:{246112005|Severity|=24484000|Severe|,363698007|Finding Site|=(113185004|Skin Between fourth and fifth toes|:272741003|Laterality|=7771000|Left|)}

Intuitive right? Not really. So how do I represent this in a CD instance? Well, the answer is really ugly, and in my opinion violates first normal form (I will post an opinion post later about my thoughts of using 1NF in XML and how I think standards bodies seem to have forgotten it).

Anyways, so what is this supposed to look like in DT R2? The answer is below:

<code code="284196006:{246112005=24484000,363698007=(113185004:272741003=7771000)}"
      codeSystem="2.16.840.1.113883.6.96"
      codeSystemName="SNOMED-CT"
      codeSystemVersion="2009">
    <displayName value="Burn of Skin"/>
</code>

I warned you it wasn't pretty. So how do you get Everest to format a concept descriptor like this? Well, the answer is simple, change this line of code:

// Old Line: formatter.GraphAides.Add(typeof(MARC.Everest.Formatters.XML.Datatypes.R1.DatatypeFormatter));

formatter.GraphAides.Add(typeof(MARC.Everest.Formatters.XML.Datatypes.R2.DatatypeR2Formatter));

And Everest will automatically handle the generation of these expressions for SNOMED concepts. Parsing? It is the same. Everest 1.0's data types R2 formatter has been developed so that you are shielded from having to understand the complexities of SNOMED expressions.

As a matter of fact, when parsing a SNOMED concept with an SNOMED expression, Everest will construct the appropriate hierarchy of concept roles for you.

What do I think of this change in HL7? I think it was pointless, and simply over-complicates processing of XML instances. In my opinion, there is no need to mix the hierarchical language of SNOMED expressions as an attribute within the hierarcal container of XML. At least a framework like Everest has enough logic in the formatting of codes to shield the developer from changes like this.

Next time, I'm going to blog about a good change in R2, changes in the continuous set expression data types (SXPR).

Formatting to a String in Everest

I've never really liked strings for transporting data as they're memory pigs and introduce a huge performance penalty, however formatting to a string can be useful if you're debugging an application or just want to see the results of formatting. So I figured I'd do a short-ish post today and illustrate how this is done.

First, we'll need an instance, I've created a small sample message from the UV NE2008 assembly but it can be any instance that is IGraphable.

MCCI_IN000000UV01 instance = new MCCI_IN000000UV01(
    Guid.NewGuid(),
    DateTime.Now,
    MCCI_IN000000UV01.GetInteractionId(),
    ProcessingID.Training,
    "I",
    AcknowledgementCondition.Never);

Next, we'll define the formatter instance that is going to take this RMIM structure and render it using an ITS. I'm using XML ITS 1 with data types R1.

var formatter = new MARC.Everest.Formatters.XML.ITS1.Formatter();
formatter.ValidateConformance = false;
formatter.GraphAides.Add(typeof(MARC.Everest.Formatters.XML.ITS1.Formatter));


I've disabled conformance checking as I assume since we're formatting to a string, we're just fiddling with Everest. I'd recommend setting ValidateConformance to true if you're seriously trying to create a conformant message. Next, we create a StringWriter class and attach an XmlWriter with indentation turned on (makes it easier to read):

StringWriter sw = new StringWriter();
XmlWriter xw = XmlWriter.Create(sw, new XmlWriterSettings()
      { Indent = true }
);


We could just format the instance as is, however it is always recommended you use an XmlStateWriter when dealing with Everest, so that what this next line does:

XmlStateWriter xsw = new XmlStateWriter(xw);

Next, we just format our instance and flush (or close) the XmlWriter.

try
{
   formatter.Graph(xsw, instance);
}
finally
{
   xw.Close();
   sw.Flush();
}

Notice I'm using the Graph() method, this is a new construct introduced in Everest RC3. If you're using RC2 or prior you can call GraphObject() instead. Finally, you can get your string by calling the ToString() method on the string writer. I've decided to print out to the console for demonstrative purposes:

Console.WriteLine(sw.ToString());
Console.ReadKey();

And there you have it! An XML Instance as a string. I wouldn't recommend using strings too heavily in your production code, like I said they're horrible from a memory consumption and performance POV. Streams and Writers/Readers are much faster and flexible.

Cheers

-Justin