NetKernel News Volume 5 Issue 9

May 16th 2014

• Repository Updates
• HDS Generation II - Release
• One Way Trapdoor Overlay
• NetKernel - Microservices with Enhanced Resource Binding
• On the Manufacture of Intellectual Property, Part 2: Diversity of Labour

Repository Updates

We have released the new HDS module. If you're new to NetKernel, Hierarchical Data Structure, is an object model that we were led to develop in order to embody the typical characteristics of resource representations: immutable, fast, small, invariant under change, easily transreptable.

We consider HDS to be the "house default". Internal NetKernel tools invariably choose to use HDS for their representations. When we create applications the motto: "if in doubt use HDS", rarely fails.

Details of the original layer0 version are documented here.

The new second generation version is available now via Apposite as the package hds-g2. As previously discussed, it provides a backwards compatible evolution of the core layer0 HDS resource model. Notable features include...

• Full XPath support for querying (including functions and selectors)
• Full XPath support for updating and combining documents
• Builder pattern support for creating documents
• Full support for XPath keys which can be baked into representations and provide constant time lookup into document
• Optimised implementation without internal "parent reference" which reduces memory footprint when making changes
• Clear representation vs endpoint internal interfaces

To preserve compatiblity the original layer0 HDS remains unchanged. This new HDS-G2 is provided as a module (as opposed to the original which is a core Java library outside of the ROC domain) therefore to use it you just import it to your application.

The module includes very efficient bi-directional HDS2<->HDS1 transpreptors - so existing tools that produce HDS1 representations can be used seamlessly. Over time its likely that we will increasingly use HDS2 in the core system tools and so we will eventually ship it as an out-of-the-box module with the distributions.

Full details of usage and API are documented in the new module.

Today's update to the nkee-architecture package provides a new OneWayTrapdoor Overlay.

The OneWayTrapdoor overlay may be used to wrap a target space. It allows requests to be resolved down into the target space, however it prevents any sub-requests that may be issued inside from accessing the request scope that lies outside the trapdoor overlay.

Pass-by-value state and state in durable scopes are allowed across the trapdoor layer and so regular pass-by-value arguments are resolvable inside the trapdoor.

Usage Patterns

There are several scenarios where a one-way trapdoor architecture can be valuable...

Sandbox

Use the one way trapdoor overlay to create a security sandbox allowing requests to have limited access to a sandboxed request space but no access to the main application architecture in the superstack.

This use-case is particularly useful when using NKP, which supports symmetrical client-server cloud architectures, to provide a limited scope callback address space for requests originating from the remote server to callback into the requesting client.

Server-Client Boundary Layer

In symmetrical client-server architectures it can be helpful to use the OneWayTrapdoor to create a clean boundary-layer separation between the server-side architecture, receiving requests, and the client side, issuing outbound requests.

With the One-Way-Trapdoor, any state (such as HTTP request state) must remain on its side of the boundary layer and so cannot be coupled into client-side requests. If necessary state can be passed across the boundary as pass by value arguments.

By enforcing a clean isolation layer between the two sides, any client-side requests can be cached without concern that transient server-side scopes may interfere and prevent caching.

Documentation

After update the documentation can be found here.

I recently had the pleasure of introducing Mike Norman, of the Virtualization Practice, to NetKernel and ROC. I'm delighted that as follow-up he has written an excellent and thoughtful article...

http://www.virtualizationpractice.com/netkernel-microservices-enhanced-resource-binding-26864/

For those without an understanding of ROC as context, it offers an insight as to how NetKernel enhances and goes beyond the current micro-services perspective. His conclusion: "It is worth spending some time investigating [NetKernel] — at the very least, you will be exposed to these [new] ideas."

Thanks Mike!

In the first part I courted controversy by suggesting that most software projects are intrinsically less complicated than most physical products. I compounded this by also stating that software production is also significantly less complex than most other intellectual property products.

Of course I have no empirical proof, but I think you'll agree there is some fairly strong evidence...

Would you credit it

Imagine if, as with movies or TV shows, software projects had end-credits; they might look something like this...

                                    Cast and Crew:
                                    -------------- 
                         First Software Developer -  Oscar Coder
                        Second Software Developer - Amy Breakpoint
                      Third Software Developer - Codie Johnson
                       Fourth Software Developer -  Sarah Coderson
                                         ...
                         Nth Software Developer - Deb Ugger 
                                    -------------- 
                           Project Manager - Alice Spring
                                    -------------- 
                                    IDE  -  Eclipse
                                 Source Control - Git
                       Language - (Insert cool language of the day)

... now, compare that to the end credits of a movie or TV show...

Even if you don't know a Best Boy from a Grip, you know from the tedious ten minutes at the end, just how many damned people go into making a movie.

Now for sure, I've neglected business analyst, DBA and architect from the software credits, but I've missed literally hundreds of roles from the movie credits list.

I think it is uncontroversial to say, the diversity of roles in software production is very low.

As we established previously, this should not be surprising since software development is organised in the guild model of production.

By definition, the guild model enforces and perpetuates the uniform skill set of the guild member. Diversity is not something a guild requires. Skill in the material is the overwhelming concern of the guild. In the software guild, this simply equates to coding proficiency in the language specialism of the guild.

reductio ad absurdum

Imagine if movies were made using the guild model?

Question: What is the material of a movie? Answer: Film.

Therefore, proficiency with film is the single quality that would admit us to the movie guild.

The skills we would exhibit would include loading cameras with film, exposing film to light in a camera, processing exposed film, splicing and dicing processed film. Being a guild, we would feel compelled to specialize in a single specific format of film: super-8, 35mm, VHS, digital or 3D.

We don't know what movies a guild might make, but we know because of the innate limits on complexity imposed by the guild model, they would likely be short, single scene, and probably pretty dull, since technical proficiency over film offers no guarantee of creativity of imagination.

A similar exercise of applying the guild model in any other media leads to similar absurdities. It follows that making intellectual property products must involve something beyond the mere material of production?

Category Error: Step away from the API

When all you have experienced is APIs its forgiveable to consider code as the primary material for production of software.

The guild model of software development is not the result of wilful protectionism. It is simply the default organisational model whenever a new material appears.

Copper-age, Bronze-age, Iron-age, Code-age...

We always begin an "age" by forming guilds that can make things with the new commodity.

However, I hope at this point, it is clear that software isn't especially complex when compared to other intellectual property products. It is also clear that, somehow, like industrial manufacture, other intellectual property industries have grown and diversified beyond the guild model.

Here's the thing: It is a category error to mistake the material embodiment of a movie, a book, software or a tv-show with the intellectual property, from which the product is truly comprised.

A movie is not the film, a book is not the pages, software is not the code...

Intellectual property is abstract. It is, purely and simply, information.

In order to move to the next level, to organise ourselves to achieve a higher order of complexity, we must challenge our guild-think; that production of software equates to mastery of code.

It is surely clear that sophisticated intellectual property products are produced by an organisation and division of labour that has mastery of information.

Mastery of information, with all the diversity of roles that this engenders...

---

Next time we'll discuss the roles and skills that sophisticated software production will demand.

In the mean time, maybe, just maybe you are getting a better sense of where we've been coming from with ROC: software is not about mastery of code, its about mastery of information; abstract resources. This is the point at which code ends and resource oriented computing begins...

Finally, to close this section, here's a thought provoking, complex intellectual property product: a movie with the soundtrack of David Foster Wallace's commencement address at Kenyon College...

We can choose to think differently...

If you want to work on the default setting. You can.

If you want to see a different way, a way that leads from the tedium of turning code, to greater complexity, to real freedom. All you have to do is choose...

This is water