NetKernel News Volume 3 Issue 45

November 9th 2012

What's new this week?

• Repository Updates
• On Modules
• What is a module?
• Module Varieties
• Standard Module
• Module Manager
• Init Endpoint
• Deployment Strategies

Catch up on last week's news here

Repository Updates

The following updates are available in the NKEE and NKSE repository...

• image-util 1.3.1
  • Enhancement to image serializer to prevent ImageIO using local disk cache - which on virtual servers like AWS can be a serious performance inhibitor. Thanks to Donnie Armstrong on the BestBuy Pisces Image team for discovering this.
• nkse-visualizer 1.22.1
  • Fix to the formatting of the CPU time.

On Modules

To NetKernel there's no such thing as a module. Yet NetKernel is a modular architecture.

How can we reconcile this apparent paradox...

The NetKernel ROC abstraction is extremely concise. The kernel only deals with three concepts.

1. Spaces
2. Endpoints
3. Requests

Endpoints construct and issue requests. Spaces resolve requests to Endpoints. Endpoints evaluate requests and construct more requests... And so it goes.

There is no concept of a Module. Module's are a purely user-side concept.

What is a module?

So what is a module? A module is a physical unit of deployment. Its a container for stuff.

The stuff can be regular Java level bytecode compiled classes - so it can look exactly like a jar file (or an expanded classpath directory). The stuff can be data such as images or HTML or XML templates or CSV or... whatever. The stuff can be scripts, ie data which when transferred to a language runtime becomes executable state.

In short, from a physical perspective, a module is an encapsulated filesystem for stuff.

But the thing that makes a module more than just a physical box of stuff is that it also defines an ROC address space. That is, it defines and provides the description for a logical ROC resource set and the relationships to endpoints that will be resolved by requests for those logical resources.

Module Varieties

As you might expect if you're immersed in ROC. Fundamentally, the representational form of a module is of absolutely no interest or concern to the NetKernel system. Which means if you really wanted to you could design and implement your own module structures and implementations.

From the ROC system's perspective there's only really one requirement for a module - it needs to provide some way to instantiate an ROC address space. A space that the kernel can use for ROC resolvable requests.

I won't go into the details - but for the curious, you might be interested to know that you can even implement a module as a Java class. There's even a factory in layer0 org.netkernel.layer0.module.java.JavaModuleFactory.

So that's the general picture. Anything goes and if you wanted to you could create a new module ecosystem. But we really don't recommend it. The reason being is that any Space your module provides needs to be a good ROC citizen. It needs to fundamentally understand and embody the ROC abstraction including coherently enabling all that is involved in both the resolution and execution phases.

Standard Module

Since the two-phase model of ROC can be extremely rich and varied, the NetKernel system provides the Standard Module.

The Standard Module provides an encapsulated filesystem and enforces a convention for ordering those resources. For example one of the things it does is to provide an extensible physical Java classloader that takes the package root as being the base of the module filesystem.

The most important capability of the Standard Module is to implement a language for declaratively defining ROC address spaces. By convention this is physically located in the file module.xml in the module's filesystem.

When a module is instantiated the module.xml is evaluated to construct the ROC spaces. Note the plural - a Standard module offers the ability to simultaneously embody any number of ROC address spaces.

So how does the ROC system find out about these Spaces? How does a module become animated and an active participant in the ROC world?

Module Manager

The NetKernel system has a Module Manager. The module manager is very simple. Its job is to maintain a list of modules and when required to present a module to an extensible set of Module Factories for construction and instantiation of its Space(s). When the spaces have been instantiated it takes care of informing the kernel about them - or in fact it's also the other way round - it takes care of telling the Spaces about the kernel.

The Module manager doesn't know or care what physical form a module takes. The only thing it cares about is the resource identifier of the module. Which is of course a URI.

The Module Manager is a singleton in layer0 org.netkernel.layer0.boot.ModuleManager. It can be used to add and remove modules at runtime...

To Add a Module

To add a module you call addModule() with the URI of the location of the module...

public void addModule(URI aSource)

You can add as many modules as you like but calling this interface is only a preparatory step. Due to the interactions and the necessity to ensure that currently executing requests are gracefully managed, the change to the active modules occurs when the sync() method is called...

sync(false);

To Remove a Module

To remove a module is just as simple. This time you need to know the IModule object that is active - this can be obtained and selected from the list provided by the getModules() interface and then removed like this...

public void removeModule(IModule aSource)

Again this is only preparatory. To make the change you then need to call sync(false).

Init Endpoint

As anyone who knows the Unix abstraction will know - the first process that a Unix kernel executes after boot is the init process. This process is the bootstrap that starts all the other processes and which on a Mac or Linux desktop eventually gives you a fancy UI.

NetKernel is the same. The bootloader instantiates the kernel, registers some module factories and instantiates the "stem-system". This consists of layer0, layer1 and the ext-system module. The stem-system is the minimal functional ROC operating system. It is very very small.

When the ext-system module is commissioned it instantiates the Init Endpoint.

The Init endpoint is a regular ROC endpoint but it has a special job to do. When it is itself commissioned it makes an ROC SOURCE request for the file etc/modules.xml located in the installation path of the NK system.

It iterates through the list of module URIs and informs the ModuleManager about the modules (by using the methods described above).

When it has told the module manager about all the modules it calls sync() to commission the modules.

Finally the Init endpoint starts the ModuleManager's monitor thread - which periodically polls for changes to etc/modules.xml and any files in the etc/modules.d/ directory.

In the event of a change to the etc/modules.xml or /etc/modules.d/xxxx files, the module manager automatically reconciles changes to the registered set of modules and reconfigures the system.

Deployment Strategies

If you want to you can add and remove modules programatically from your own ROC domain application. But a simpler and more elegant pattern is to use the Unix philosophy of keeping things decoupled.

Apposite follows this philosophy. It does not actively manage what is deployed. It contains a resource that describes the expected current set of loaded modules for the system but it is really just a glorified tool whose principle job (aside from downloading, verifying and unpacking packages) is to write the managed list of modules to the modules.xml file.

My recommendation is that if you want to deploy and manage the deployment of your modules to NetKernel you follow the same pattern. You just have a tool that writes a module list for your application set to a file in etc/modules.d/ - something like etc/modules.d/my-modules.xml. It has the same structure as the core etc/modules.xml file...

When you want to reconfigure your application set - which might involve replacing an existing version with a new version or removing a no-longer required module - all you do is write the changed list to etc/modules.d/my-modules.xml

NetKernel is modular but the paradox is that it uses the ROC abstraction and logical ROC spaces to be able to treat the modules as resources. Chicken meets egg.