The protocol profile describes the data items, their attributes and where they 
fit
in the overall structure.
The DSL is there to encode the representation from a human readable format
It's not per se specific to the medical field, the same trick could be used 
for other standard business protocol. It has no notion of what is a medical
record, only the significant attributes like the field relative position, its 
data type,
a human readable description, its name and its position in
the structure, 

The multiple representations serve different needs:

A) string representation is to provide an HL7/DICOM message acceptable to end 
points
     that only support this exchange mechanism. Only HL7 fields acceptable
     to the end device are retained. If necessary, rules are applied to marshall
    the ouput from the profile and the data at hand. Reason: several 
implementations
    of HL7. In fact we apply an endpoint specific profile to make the data 
backward
    compatible. Then we may apply rules (allowed field sizes may vary, sometimes
    fields have to be swapped, ...)

B) the full representation is a merge of profile information and data,
    it allows some processing that require navigation in the message structure.
    Having it merged is easier (you can use zippers, compare differences, ...)

C) the light weight one is a serialization format, we use to this to pass 
messages
    internaly (this runs on a small cluster to provide full redondancy)
    It's merelly imbricated vectors with minimal information so the
    receiving end can decode this back to B using of course the same profile.
    Empty fields are represented as empty vectors.

D) the api to create a message is based on collection imbrications.
     The profile is then used to place data in it's rightful spot in the 
structure.

We choose between a,b,c at runtime using Clojure protocols to extend the String 
class
and some of Clojure's persistent structures and yes we pass the profile
either by name or as a structure when calling these protocols.

D) is a convenience to shrink the code when creating messages, it's also
easier to write generic code with this approach. It's pretty dumb,
 [name value] pairs imbricated (can be any collection type) and no
 predefined order has to respected by the caller. The profile knows
 where to insert this stuff.

To summarize, the profile drives the data in and out with the help of a small
amount of code.

Some end points are actually
using maps to associate data entities in the messages from the profile
to database equivalents.
We need to make some improvement here but we should be able to drive this
with generic code and push back in this associative map the mapping of
relations and fields as data, not as code.

We could add any other attributes in the profile, load sensitive stuff 
are better handled near the end points. Some of these are not message based
(database queries or worse) and some are load sensitive.

Luc

> On Jun 18, 2012, at 5:35 PM, Softaddicts <lprefonta...@softaddicts.ca> wrote:
> 
> > Lets talk a bit about my world here.
> >
> > We created a product to link medical equipments
> > using a variety of protocols, some talk HL7, DICOM, others have proprietary 
> > means to
> > access data from various places.
> >
> > From the start we chose the richest data representation, some of it came 
> > from
> > a few medical standards and another part is our own creation to carry stuff 
> > that is outside the scope of the business field (non medical data).
> >
> > We did not define specific records, instead we factored out the protocol
> > definition (the contract) from the representation(s) and created its 
> > definition using
> > persistent data structures and Clojure protocols
> >
> > The profile is being augmented often, new fields are added to keep up with
> > support of new devices, new standards,...
> >
> > We never remove stuff from protocol definitions and we always aim to 
> > generate
> > fully populated messages from the data available at a specific end point.
> >
> > It's all data (above 10k lines for as profile using our custom DSL) and 
> > less than
> > 500 lines of code to define and handle.
> >
> > We have three message representations more or less equivalent, one is 
> > string based
> > (HL7-enabled devices accept this representation), another one is
> > for serialization and the last one is a "full" representation with each data
> > component populated with all its attributes from the profile (name, relative
> > position, ...)
> >
> > All the above representation allow you to switch from one to the other.
> > We even have a pretty print so we can decipher message when investigating
> > bugs.
> >
> > We also have a DSL to create messages from code to promoto some
> > brevity when creating messages and lax us fro having to precisely order
> > fields when writing code.
> 
> Couldn't one say that a dsl is yet another more domain specific
> representation of the medical record?  Why did you code against the
> medical record rather than the dsl? My assumption is that the dsl is
> just a subset but just curious.
> 
> >
> > None of the above uses defrecord or deftype. We use
> > Clojure's persistent structures and protocols to implement how to handle
> > these various formats. Roughly we do this in less than 1000 lines splitted
> > in 4 namespaces. This code never refers to a specific piece of data by its
> > name.
> 
> A bit confused.  So you have profile that describes how to convert
> from the medical record to the preferred internal representation which
> allows you to support multiple medical record representations?
> 
> How might you handle the situation where a new medical record contains
> an abbreviation or medical code but you need the full value and to get
> the complete value you need to perform a web service query, so you
> want to avoid this unless the value is required... Ideally, the
> internal representation would know that the value is expensive and to
> request it only if necessary.  So This would probably change the
> contract, but for your boss needs this to process this non-standard
> record format tomorrow if the client likes what he sees youll have
> time to support it properly, if not then you can just rip the code
> put.  How would you go about this change 1 if you needed it functional
> tomorrow and 2 the ideal/best way?
> 
> 
> 
> >
> > It's all data driven, the contract is data in our internal protocol profile 
> > definition.
> > Validation, searching the metadata of the profile, ... is done within the 
> > profile definition.
> >
> 
> So do you have one function that takes the profile, record and desired 
> property?
> 
> > The name spaces allowing us to  support the different representations all 
> > rely on the
> > profile definition. They never refer to specific data items.
> >
> > Some devices do not support all the fields or are using older versions
> > of protocols compared to what we implemented internally.
> >
> > We have a rule engine and rules attached to specific device profiles to
> > alter the common format message so we can spit out a valid message that
> > can be understood by the device. We implemented a Clojure protocol that lets
> > us manipulate a message using Clojure generic fns to strip items,
> > swap items from one spot to the other, ....
> >
> > These rules are applied on top of a generic message just before sending or 
> > just
> > after receiving to/fro  the device. We do not have to recode messages for 
> > each device
> > type. This is the only place aside from message generation that will ever 
> > refer
> > to specific data items.
> >
> >
> > What about maintenance ? Why remove a data item from the profile ?
> > If ever necessary, we can flag it as unusable and the rest will carry on.
> >
> > Any attempt to refer to it will trigger an exception.
> > Searching the code base to find references to it is no more difficult than
> > finding getters. And we know it's a required change. Either it disappeared 
> > or
> > the data item changed. We can remove it somewhere in the future, no need
> > rushing.
> >
> > It eases the maintenance of the device specific rules, if they do not
> > refer to the disabled data item, they are still working.
> >
> > We get a uniform way of dealing with messages, choosing the most efficient
> > representation when needed, having uniform naming and a firewall preventing
> > us from specifying invalid data items without writing a single internal 
> > line of
> > code referring to specific data elements.
> >
> > It's mostly done with data definitions...the key is using proper data 
> > abstractions.
> > I agree, if none exist, you have to create them but this is where the 
> > secret of the
> > sauce is. Refrain from writing tons of code lines if your data abstractions 
> > are not
> > well defined. Experiment and tune them before hand.
> >
> > Before this era, we used to have classes wrapping data items and so forth.
> > It was a pain in the ass to navigate and modify this stuff each time
> > we needed to enrich our data model  with all these wrappers around us
> > not withstanding dependency management.
> >
> > Now, the contract is the data. End of story.
> >
> > Luc P.
> >
> >>
> >>
> >> On Monday, June 18, 2012 8:01:47 AM UTC-6, tbc++ wrote:
> >>>
> >>>> Isnt that just creating an api? Everywhere the old model exists you need
> >>> to
> >>>> call a function to create the desired data structure and this adds
> >>> another
> >>>> layer of complexity that needs maintained.  Not all conversions are
> >>> straight
> >>>> forward, may require additional context of whatever introducing the need
> >>> for
> >>>> deferred computation.
> >>>
> >>> Can you provide a concrete example (with code perhaps)? I guess I'm
> >>> not seeing where the problems arise. In this situation I normally
> >>> either, 1) create a new "fetch" function to create a new map of data
> >>> in the new format 2) provide duplicate data in the map that's computed
> >>> differently, 3) update the old code to the new map.
> >>>
> >>> Here at work we have a 20 year old application that is almost 100%
> >>> backwards compatible (the server can talk to much older clients). And
> >>> it does so by using pure data. The server has setup contracts saying
> >>> "this is what Project data looks like, this is what Company Data looks
> >>> like" and those contracts never change. The 3D Software Blender
> >>> (blender.org) operates on this same principle. After almost a decade,
> >>> their latest software can open 1.0 .blend files, and vice versa. All
> >>> these applications work off of unchanging data contracts.
> >>>
> >>> So I'm interested in seeing concrete examples of how your application
> >>> differs.
> >>>
> >>
> >> Its not the external contracts that are changing but the internal contracts
> >> between namespaces... as I add features I add new requirements clarify my
> >> understanding and decide to replace them with new ones.  I wish I could say
> >> just don't change them, but the internal contracts are "implementation
> >> details" no one uses them but me.  If I decide to change them stuff breaks
> >> and that would be true in OO too, but in I know how to go about these
> >> refactorings incrementally in OO until the old abstractions fall away and
> >> become dead code, in clojure new abstractions mean lots of breaking changes
> >> and nothing works until everything works.
> >>
> >> I wouldn't mind sitting down with someone in the Denver area and getting
> >> some feedback on my code, eventually I would like to open source the code
> >> but I don't expect that to happen anytime in the near future.
> >>
> >>
> >>>
> >>> Timothy
> >>>
> >>
> >> --
> >> You received this message because you are subscribed to the Google
> >> Groups "Clojure" group.
> >> To post to this group, send email to clojure@googlegroups.com
> >> Note that posts from new members are moderated - please be patient with 
> >> your first post.
> >> To unsubscribe from this group, send email to
> >> clojure+unsubscr...@googlegroups.com
> >> For more options, visit this group at
> >> http://groups.google.com/group/clojure?hl=en
> > --
> > Softaddicts<lprefonta...@softaddicts.ca> sent by ibisMail from my ipad!
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "Clojure" group.
> > To post to this group, send email to clojure@googlegroups.com
> > Note that posts from new members are moderated - please be patient with 
> > your first post.
> > To unsubscribe from this group, send email to
> > clojure+unsubscr...@googlegroups.com
> > For more options, visit this group at
> > http://groups.google.com/group/clojure?hl=en
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with your 
> first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> 
--
Softaddicts<lprefonta...@softaddicts.ca> sent by ibisMail from my ipad!

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Reply via email to