NetKernel News Volume 6 Issue 9

September 25th 2015

• Repository Updates
• How many microservices have you got? There's a microresource for that...
• Locking Down the Admin Tools
• REST API Tooling - Swagger Support

Repository Updates

Last week I was speaking at the Goto London conference. I'll share a video of the talk next time, once it's been posted.

I tried to contextualize the current excitement around microservices.

My basic theme was: yeah microservices are great and all, but really we're sort of missing the point by calling them services. The Web is a resource oriented architecture, so what we should be excited about is 'microresources'.

What makes the trend to microservices "a good thing" is that by increasing the granularity of systems we increase the opportunity to combine the pieces into new composite resources.

This is good because there are always unanticipated requirements, or the world changes. To deal with this we can often quickly provide new resources by "taking a bit of that and combining it with a bit of that".

It is well understood in other engineering disciplines that it is very often that case that:

The value of a composite is greater than the sum of the parts.

Just think about reinforced concrete, carbon fibre, integrated circuits, Unix tools...

It follows that the trend for finer granularity increases the opportunity to move to composite resource oriented systems.

The Paradox of Statelessness

For those of us who've lived in the Resource Oriented paradigm for some time it is, fortunately, now an increasingly smooth path to explain how we go from microservices, to microresources, to Resource Oriented Computing.

Although, even though we now have a starting point for the journey, I think it will still take time for the idea of removing the Web's constraint of a single global address space to be properly understood. However it is absolutely necessary.

The Web's global flat addressing model is a triumph but also a severe problem for the progression to microresources.

Why is it a problem? Well firstly every microservice is a peer in the address space and so is responsible for its security, availability and scalability.

We also know that to be RESTful a microservice should be stateless.

Another way of saying this is that every request is context free (all state must be transfered if the endpoint is stateless). But context is what allows us to introduce important properties such as trust (aka security) etc.

So we are left with a paradox: stateless = good, but stateless = context free && context free = bad.

Context, Scope and Scale

Moving from a flat address space, to ROC's unlimited spaces enables us to keep the best of REST, but introduce the critical contextual spacial scope that makes solving real problems much simpler, safer and (ironically) more scalable.

One way to understand the different perspectives is to compare scales.

In a Web-centric solution its usual to think of one Web, several services, many microservices.

So what scale are we working at in ROC?.

There's a resource for that...

If you install today's update to ext-system, you'll find we've added a new resource active:moduleStats. If you got to the scripting playpen you can find out with this groovy code...

stats=context.source("active:moduleStats");
context.createResponseFrom(stats.toString());

Here's what my day-to-day development system reports...

null : null
	statistics : null
		spaces : 905
		modules : 228
		physicalEndpoints : 3411
		logicalEndpoints : 3773

So I have 905 spaces in 228 modules (Eclipse projects!) and close to 4000 endpoints.

Clearly in ROC there is something new happening.

I don't have one resource oriented address space like the Web. I have close to a thousand. So in relative scale terms, we are literally talking about microwebs.

I have several thousand microservices. So, in relative-scale terms, we are talking nanoservices.

But, lets be careful with our language, talking about microweb's and nanoservices is thinking that will take us backwards on the path that we thought we'd just journeyed down.

ROC's only concern is spaces and resources and these are scale invariant.

This came up recently as a support question, so I thought it would be worth sharing.

When you deploy a production instance of NK, you want to be able to get to the backend system, but you want to limit access. The recommended best way is to ensure your firewall blocks port 1060 (check this but any sensible system should default to block all ports and only have openings for your public services).

By default the backend binds to host 0.0.0.0 - this means its available from the public NIC as well as the local loop network (localhost, 127.0.0.1). If the NIC network is firewalled you're ok but to be ultra safe you can change the host to bind only to the local loop. You can do this by setting the following system property in bin/jvm-settings.cnf

-Pnetkernel.http.backend.host=127.0.0.1

You should now have port 1060 on the local loop with no external access. To get to it you can now set up an ssh tunnel...

ssh -L1061:localhost:1060 myhostname

Now with your browser you can go to http://localhost:1061 and you'll have access to all the remote backend tools. (notice we offset the ssh tunnel to use port 1061 locally, this means you can still have your local NK running on the default port 1060). If you need to you can also add user accounts and role based access models to the tools using the control panel''s access control layer (the little person icon in the top panel of the control bar).

Certain people, namely Mr B Sletten and Mr R Hitchens, have made it known to me that we need to provide a smoother path for developing modules that provide REST APIs. Brian actually has some very good ideas for an automated tool set that will facilitate some very cool auto-configured API management capabilities and we've both committed to collaborate on this (community assistance welcome).

To put a stake in the ground and signal that there is a real intent here. This week, I took the first step to smooth the API development path by creating a prototype active:swaggerParser service.

This tool will parse a Swagger 2.0 API description and produces a mapper configuration containing the corresponding endpoints, request mappings and critically the <rest> metadata for use when wrapped by the RESTOverlay.

It takes care of the content negotation denormalisation, endpoint verb distinction etc. It also provides simple grammars with correctly matching argument slots and relays those arguments all the way through to the requested internal ROC endpoint.

By default every auto-generated ROC endpoint request will make a call to active:groovy and return the "TBD" resource. You can edit the generated file to point the mapped endpoint request to the appropriate internal implementation. You can also add appropriate RESTOverlay features such as compression, etag format, Error- and 404-Handlers, Response wrapping etc etc.

Download

Here's two modules, the Swagger library and a unit test module that both tests the tool and also has a PetStore REST API auto-generated from a swagger interface description...

• urn.org.netkernel.rest.api.swagger-20150925.jar
• urn.test.org.netkernel.rest.api.swagger-20150925.jar

Example - Pet Store Demo

If you unzip the test module you can see that I generated a mapper config from the example Petstore API.

All I then had to do was wrap the <mapper> inside the RESTOverlay and I had an instant Pet Store API.

Here's what the space looks like...

In the space explorer the RESTOverlay is showing these REST interfaces...

The petstore unit tests make real REST client calls to these API microservices on port 8080 to prove that this really is providing a full REST API.

If you look at the swagger parser's unit tests you can see how to call it. Basically it's

active:swaggerParser+operand@the-swagger-definition

This tool was knocked up in a couple of hours - so it will almost certainly evolve. But even so, it already provides something that will take away most of the boilerplate work to quickly get a REST API set up as a module.

If you cut and paste that example <rootspace> I reckon it should take about 2 minutes to go from a Swagger description to full stubbed out live REST interface.

Expect a similar tool for RAML API definitions - or if you're inspired grab the RAML Java parser and adapt my code for an active:ramlParser.