NetKernel News Volume 1 Issue 49

October 8th 2010

What's new this week?

• Repository Updates
• *NEW* active:forEach
• *NEW* active:XMLSplitter / active:XMLMerger
• Nobel Prize

Catch up on last week's news here

Repository Updates

The following packages are available in both NKEE and NKSE repositories...

• coremeta 1.6.1
  • update to docs to reflect hds change (see layer0)
• demo-foreach 1.1.1
  • A new demo of the active:forEach tool (see next item)
• layer0 1.43.1
  • HDSFactory - support for transreption options for XML comments and CDATA. RequestBuilder - fix of method="from-string" to take values from supplied literal arguments.
• layer1 1.20.1
  • Added new active:forEach tool (see below)
• module-standard 1.29.1
  • Indicative metadata on StandardTransport base class to support static structure diagrams.
• xml-core 1.8.1
  • Added new XMLSplitter/Merger tools (see below)

*NEW* active:forEach

active:forEach is a new tool in layer1. It provides a simple high-level means of developing batch sequential processes or parallel asynchronous processes. If required, it also provides extensible merging of the results.

The active URI syntax is:

active:forEach
 +operand@set
 +do@request
 +merge@declarative request
 +...varargs....

active:forEach takes a java.lang.Iterable set of resources as its operand. For-each item in the iterable operand set it issues the item to a do-request specified with a declarative request.

By default, the response is an ordered set of java.lang.Iterable representations returned by each do-request.

If specified, an optional merge request will be called with the list of responses. The called merge service should return a single composite resource, which is then returned as the response of the active:forEach request.

The tool offers several modes including synchronous or asynchronous processing, strict or tolerant exception handling and asynchronous timeouts.

If required, any additional arguments (varargs) provided to the tool may be relayed to the do-request and/or the merge-request. In addition, both the total number of items and the index of the current item can be relayed to the do-request.

(Full reference details are available here).

Install

To get hold of this tool just accept today's update to layer1. After installing the update the local docs for the tool are here.

demo-foreach

A new demo module demo-foreach is in the repositories to show various usage examples of active:forEach.

Here's an example of a parallel XML pipeline process which splits an input document into an iterable list of Document fragments and then parallelizes the XSLT transform before merging the resulting documents back into a single document. The example is shown implemented in both DPML and Groovy versions...

DPML: Parallel XML Pipeline

Groovy: Parallel XML Pipeline

req=context.createRequest("active:XMLSplitter")
req.addArgument("operand", "res:/resources/demo4/input.xml")
req.addArgumentByValue("xpath", "/resultset/row")
rep=context.issueRequest(req)

req=context.createRequest("active:forEachAsync")
req.addArgumentByValue("operand", rep)
req.addArgumentByValue("do",
"""
<request>
    <identifier>active:xslt</identifier>
    <argument name="operator">res:/resources/demo4/styleRow.xsl</argument>
    <argument name="operand">arg:item</argument>
</request>
"""
)
req.addArgumentByValue("merge",
"""
<request>
    <identifier>active:XMLMerger</identifier>
    <argument name="operand">arg:representations</argument>
</request>
"""
)
resp=context.issueRequestForResponse(req)

context.createResponseFrom(resp)

Install

The demo is in the repositories as package demo-foreach. Just install with apposite. After installation the demo docs are located here

Fire and Forget

As well as being able to provide "fork-merge" style asynchronous processes, it can also do fire-and-forget sub-process fork where the initiating request thread returns immediately to continue its own tasks. Details are in the reference documentation.

BackStory

If you dig inside the source to layer1 you'll find that active:forEach is actually just a utility built on top of the standard capabilities of the NKF API. The reason we've developed it is so that this core capability can be accessed in declarative technologies such as the DPML example above, but also to allow the introduction of enhanced architectural patterns around existing services.

For example, one anticipated use case is for active:forEach to be used with the mapper so that any given service (the target of the do request) can be enhanced with a parallelized alternate interface presented through the mapper. When the mapper interface is requested it would invoke active:forEach as a wrapper to parallelize the service targeted by the do-request.

Let us know how you get on or if you have suggestions for complementary technologies that could offer similar high-level decoupling.

*NEW* active:XMLSplitter / active:XMLMerger

The xml-core library has a couple of new companion tools active:XMLSplitter and active:XMLMerger.

active:XMLSplitter offers the ability to split an XML document into an iterable list of org.w3c.dom.Document based on an XPath selector of document fragments.

The active:XMLMerger tool takes an iterable list of org.w3c.dom.Document and merges them back into a single document. The new document will have a root element (and child tree structure) corresponding to the supplied XPath argument.

An example of their use is shown above and you can play with it in the demo-foreach package.

Nobel Prize

I couldn't miss the opportunity to congratulate Andrei Geim on receiving this year's Nobel Prize in Physics.

Andrei was a post-doc at Nottingham, in the same research group, at the same time as I was doing my PhD (92-95). The research centered on the Physics of solid-state low-dimensional quantum systems and required low-temperatures and high magnetic fields.

Having low-temperature facilities enables a lot of important experiments to be performed. I personally investigated freeze drying sandwiches, exploding Blu-Tack spheres with hammers and making ultra-bright 'witches-brew' LED disco lighting. But whilst I was quite happy to dunk my hand in a bucket of liquid Nitrogen, I remember Andrei was a real specialist and could gargle the stuff.

He also proved open to the esoteric opportunities of very high magnetic fields; prior to the Nobel, he received an Ig Nobel for his pioneering work levitating frogs. He therefore now has the unique distinction of being the first person to win both prizes.

The scotch-tape technique he and Konstantin Novoselov devised for producing Graphene proves you should never dismiss something as a "crazy idea". Always keep an open mind, you just never know until you try it (wait for it... you know there's a plug coming...), a bit like ROC.

Have a great weekend - I'm off for a pint of Nitrogen.

[If you've got kids, or even if you don't, and have 2 minutes this weekend why not make the world's simplest electric motor, 4 components, 10k-rpm: (big) kids love it.]