NetKernel News Volume 3 Issue 3

November 11th 2011

What's new this week?

• Repository Updates
• Mapper - Representation Type
• xml:exi library
• Tom's Book: Draft 9 with DPML and XRL
• On Scope - Intermission

Catch up on last week's news here

Repository Updates

The following NetKernel 5.1.1 updates are available in the NKSE and NKEE repositories...

• lang-math 1.2.1
  • Removed <dynamic/> tag as is best practice for a library.
• soap-ws 1.7.1
  • Update to metadata of the SOAP Bridge zone representation
• wink 1.19.1
  • Update to ensure text/html transreption (see below).

Mapper - Representation Type

To all intents and purposes, the application-level, comprising: libraries, standard-module and layer0, is the same between NK4 and NK5. Therefore existing NK4 systems will be forward compatible with NK5 (and, for the moment, vice-versa).

However, we did introduce a small but subtle change, in order to pin down a loose corner of the abstraction. The Standard-Module's mapper implementation has been changed in NK5 so that (if not overridden by a specific representation declaration) it now constructs its mapped-request to demand the same representation type as was received on the incoming request. Previously, in NK4, its default was to issue a request with no type preference (java.lang.Object).

On face value this seems like a minor change but it has an important consequence.

Previously an application using the mapper to define its logical service/app interface would issue its sub-request and return as its response whatever had been generated by the mapped-to endpoint in the application. So, usually the application would be requested from a fulcrum and therefore it was necessary that the fulcrum had within its scope any necessary transreptors, should the returned representation require serialization for the transport response.

In general this worked fine for most scenarios since the standard fulcrums have access to many of NK's standard resource models (XML, layer1 etc). But you can see that looking forward it was a poor default pattern since it cross-coupled the representation model of the application into the fulcrum layer.

By changing the mapper to relay the desired representation through on its mapped sub-request we have eliminated this cross-coupling.

It is now guaranteed that transreption must occur before leaving the mapped application layer. So typing and transreption is bounded within the context of the application, which therefore can provide any necessary transreptors as may be required. This is a good thing and, as a consequence, will eliminate those annoying "transreptor not found" development errors that used to sometimes come up in NK4 when the application context had been descoped.

XML is not HTML

In general this change should cause no forward-compatibility effects for a solution on NK4 running on NK5. However there is a corner-case which you should be aware of. It occurs if you have an HTML web application, which uses the mapper and in which you are doing the resource composition in the XML domain and for which that XML composite is to be serialized to HTML (not XHTML).

In this scenario, if you do not explicitly perform your own serialization then the default XML serializer will be used. The default serializer is aware that HTML is not XML and it uses the mimetype of the resource to switch to an HTML mode if the resource to be transrepted has mimetype text/html.

With the change to the mapper, the XML serialization will now occur in the context of the mapper overlay. If the requested endpoint returns a mimetype of text/html all is good. However if the requested endpoint does not specify a mimetype, the transreptor has no clue that this XML resource should be treated as HTML - it will therefore produce an XML serialization.

Now here's the specifics of the corner case. You may have previously set the mimetype as a header in the mapper endpoint declaration.

Previously, when transreption was delayed until the fulcrum layer, an XML resource might leave the mapper with an XML representation and a mimetype of text/html, triggering the fulcrum default serializer to treat it as HTML.

However, mapper headers are set once the response has been received. But, since the mapper is now triggering transreption in the application context, the mimetype header is assigned to the response after transreption has taken place! So you can see that for applications using "XML treated as HTML" this would be too late.

Options

So what are you to do? Well if you have a solution that already works for you in NK4 then you can do a very simple zero-order change. Simply add a representation type of java.lang.Object on your mapper endpoints so that when using NK5's mapper it will behave identically as with NK4's...

This interface will now be exactly the same as in NK4 and will defer transreption until the response gets to the fulcrum layer. This solution works on NK5 and remains perfectly valid on NK4 too (its just being explicit about no type preference) and so your application will work as before across heterogeneous NK4/NK5 clusters.

But really this is a short-term solution, and as we outlined above, its not a good long-term answer.

A better solution is to ensure that whatever you request in the mapped request sets its mimetype to "text/html". This will work for all time and is really, strictly speaking, what the application should have been doing already.

You'll find that some of the XML library's tools provide the option to make "text/html" their response mimetype already. For example active:xrl2 has a mimetype argument you can specify. active:xslt will use text/html if the <xsl:output> is set to HTML and the requested response is a binary stream (serialized domain). And, of course, if you're mapping to a language runtime you can set the mimetype programmatically with INKFResponse.setMimeType().

If you're generating HTML representations then making the highest tier of your solution report a mimetype of text/html is generally a good thing.

Finally, for absolute control, you can always add an explicit serialization stage to your application. For example xml:saxon has a serializer that supports a doctype declaration and will produce very strictly conformant HTML serializations.

Discussion

Take note: This entire discussion is only relevant to existing applications that are generating HTML from within the XML domain.

The change to the mapper type relaying was necessary for the long-term and is now formally correct. You can very easily add a <representation>java.lang.Object</representation> declaration to your existing NK4 mapper endpoints to have them run exactly as before on NK5. However we think it would pay-off if your were to modify your first-line endpoints to set the mimetype to text/html as a better long-term path. (Incidentally such a change will also still be backwards compatible on NK4).

This world of HTML and XML is an interesting one. The reason that it surfaces is because the domain specific typing models of the XML world are being transrepted down to a general universal representation type (the binary stream).

In an ideal world we would avoid this entirely by having internal XML processing and XHTML serialization (XML!). But alas, browsers still don't universally support XHTML and, perhaps worse, XHTML introduces certain constraints in the web-application that make it hard and unfamiliar for existing javascript/AJAX developers.

So we are left in a pragmatic world of compromise. As recently discussed, its powerful to work in the XML domain on the server-side, and yet, for the foreseeable future, its very likely we will want to carry on using the highly tolerant and flexible mode of "text/html" on the browser-side.

As a rule of thumb, when generating HTML using XML tools, take care to set text/html mimetype at the point of generation and not as an after-thought in a mapper header and you'll be alright.

If you're not using the XML domain to generate HTML - just ignore everything above!

xml:exi library

During my visit to Brussels, I had the opportunity for some excellent conversations with Rene Luyckx, CEO of Steria Benelux. At some point he mentioned the recent ratified W3C EXI standard and suggested that this would be a valuable addition as a library for NK.

EXI is a new binary encoding for XML which has been developed to provide a highly efficient interchange format for XML-based messaging. It will always beat gzip compressed XML in size, but will also parse and serialize more efficiently too. This image (linked from the Exificient performance pages) shows how it performs across a range of tests...

So this week I took a few hours to learn about and create a library. Here it is with some unit tests...

• urn.org.netkernel.xml.exi-0.1.1.jar
• urn.test.org.netkernel.xml.exi-0.1.1.jar

These are modules so add an entry for the them in modules.xml. After install the docs are here...

http://localhost:1060/book/view/book:urn:org:netkernel:xml:exi/

Initially I used the exificient library - which is relatively mature and has a clean API for integration. But unfortunately the Seimens team that developed this library are unable to offer it under anything but a full GPL v2 license. This means we are unable to distribute it with the module. On the ROC community's behalf I requested if we could use an alternate license (LGPL for example) but was told many people had requested the same thing but that this would not be possible.

With a license stalement, I therefore looked around for an alternate and discovered the Fujitsu-based OpenEXI library. Half an hour later and I had a fully equivalent solution using the new library. OpenEXI only provides a SAX interface for integration, so it was a little trickier to hook-in, but now its in an accessor nobody needs to care - they're just black box tools.

What do they do? Well its as simple as

active:XML2EXI+operand@doc....

active:EXI2XML+operand@doc....

For completeness I've included the exificient-based tools as reference code too. But you will have to take your own decision about licensing and download a copy of the excificient.jar to make these functional.

Let us know if this is useful and following feedback we'll make it available in the repos.

Tom's Book: Draft 9 with DPML and XRL

Here's the latest news from Tom...

---

The 'Practical NetKernel Book', also known as 'Hello NetKernel' and 'NetKernel in Action' is in its 9th iteration.

Chapter 6 on DPML is now completed, Chapter 7 on XRL is in progress. There is also news of another nature (about the book), but let me wait until the ink on the paper is dry and I have everything double- and tripple (this is not about Belgian beers) confirmed. Watch this space though !

You can download iteration 9 - NK5 from http://www.netkernelbook.org/serving/pdf/hello_netkernel_nk5-0.9.pdf or if you want to follow along and have the latest 'build' http://www.netkernelbook.org/serving/pdf/hello_netkernel_nk5.pdf

As always, your feedback and input is highly appreciated. You can send it to tom(dot)geudens(at)hush(dot)ai

On Scope - Intermission

Last week I began a discussion of scope. I sort of left you hanging with the suggestion to play with the demo-scope module.

I was in London for a couple of days this week, but I promise I will return to this topic next week. When we'll discuss how resolution of ROC identity and the contextual ROC spaces combine to provide an unambiguous scope which has the properties of behaving like lexical scope (for developer sanity) but which is actually dynamic and extrinsic and so provides the rich contextual identity necessary to normalize computational state.