NetKernel News Volume 5 Issue 8

May 16th 2014

• Repository Updates
• Asynchronous HTTP Client - Release
• On the Manufacture of Intellectual Property, Part 1: The Guild

Repository Updates

After extensive testing we're happy to announce the new HTTP client module is available as a general release available as an update in Apposite.

The key feature of this update is to introduce asynchronous client capability - meaning that the engineering throughput of a system can be decoupled from upstream REST servers. The details of what this means and how it is implemented were covered in this previous news article.

We've also added support for a new HttpClientResponseRepresentation.

If you express this as the preferred representation class then, instead of the default, returning the HTTP response entity, the HTTP endpoints will return an HttpClientResponseRepresentation - which collects together the entity, headers and response code in a single value object.

In essence all of the same information is available as with the default mode, but it is now collected together in a convenient way if, for example, you are making use of a REST-API and have a need to write code that interprets the details of response.

Details of this, and discussion of the asynchronous usage patterns, are documented in the new module.

It is not difficult to fall into the assumption that developing an advanced technology that exhibits radically improved economics, would be sufficient to transform the appalling inefficiencies and waste of today's tragic software landscape.

However it is apparent that it is necessary but not sufficient. It has taken me a while to understand that we have to simultaneously transform the technology and the intellectual property process.

If we look to history, we see parallels. Imagine that we had invented the steam engine. Imagine if the steam engine had simply been dropped into the guild model of production of the late 18th and early 19th century?

The steam engine changes the economics and scale of production and so drives a reorganisation of labour. A reorganisation that leads to differentiated specialist skills that enables the industrial manufacture of ever more complex products...

The Guild Model

In the guild model, highly skilled artisans create a product by meticulously applying skilled crafts from start to finish. Each member is responsible for one product and takes it through all the necessary steps. Every member of the guild is (approximately) equally skilled.

In the guild culture there is no conception of differentiation of labour. All craftsmen in a guild can use all the tools applicable to the guild. To become a full member of the guild takes time and dedication and requires that an apprenticeship be served.

We have forgotten the origin, but the English language has a very concise way to indicate membership of this organisational model. We take the name of the material that the guild specialises in and attach the ending "smith". So we have Goldsmiths, Silversmiths, Blacksmiths.

It's so engrained in us culturally that we even started to use these guild affiliations as our own names: Mr Smith.

If software had existed in the late 18th century, then we would be labelled this way too. We wouldn't say we were a software developer, you might be called a Javasmith or Groovymith or Rubysmith or...

The Economics of the Guild Model

We can see that the guild model is a natural and good solution to the organisation of labour in agrarian societies. Its yield of products is consistent and its economics are simple:

cost of materials + cost of labour = price

You might not be wholly convinced that we organise software manufacture in the guild model - but to see that we do, all we need to do is follow the money...

How many projects have you bid-for, worked-on or negotiated budget for, using a time-and-materials pricing model? Time and materials is the economic model of a guild.

Irrespective of whether you align yourself with waterfall, scrum or agile variants of the development cycle, we are culturally organised in the guild model.

Limitations of the Guild Model

There is a very significant problem with the guild model. The reason why it has almost entirely been abandoned in the production of physical goods.

The guild model limits the complexity of the goods which it can create.

It does this in a number of ways:

Firstly, guilds are oriented around a skill in a specific material. Gold or silver or iron or copper. In software the material-skill of a guild is the programming language it has adopted: Java or Ruby or Groovy or...

In a guild every craftsman must be able to do any task, so it only makes sense to use one language in the production of a given software project (product).

Secondly, guilds are most efficient when they repeatedly produce similar products.

Producing similar products allows a guild to invest in creating specialist tools that are aligned to the tasks required by that type of product. Look at a blacksmith's forge: the anvil, the files, the hammers.

You may have noticed that developers love meta-development: writing code that helps them to write code.

Over dinner recently, I was told by Neal Ford, the "CTO" at Thoughtworks that, unconstrained, the development teams they work with will naturally put 80% of their effort into meta-development.

Meta-development is another way to express the tool investment behaviour of a guild.

Frameworks

Somewhere half way between the material constraint and the tooling constraint lies the framework.

A framework is half-tool and half-material.

In software we gravitate to frameworks because they are a tool that precisely fits a class of product, while the process of adoption of the framework makes our guild more specialised in the material of the framework (one word: Spring).

This investment and adoption of tools within the guild means that the available tools are on a march to a closer and closer match with the product.

This is a double edged sword since it is equally true that our toolset constrains the scope of the potential complexity of a product.

Or put another way, a framework delivers certain products. When our product requirement is more complex than the solution set of a given framework our inclination is to increase the potential complexity of the product by introducing another framework.

This is the path to hell. Whilst the scope of complexity of possible products increases, the effort to combine the frameworks consumes increasingly large amounts of the guilds labour. Progress slows, the quality goes down and the total cost can easily exceed the budget.

We let this happen, and the guild model goes unchallenged, since for some reason it is accepted and acceptable that, due to its nature as an intellectual property product, "software is special".

It is not...

"But, Software is Special"

We tend to think that, since it is abstract and intangible, software is of a different order of complexity than the production of something physical.

Certainly it's very much harder for someone without knowledge of software to determine the complexity of software when compared to a physical product. What you can't see or touch must be more complex?

Software has the opportunity to allow this default assumption to go unchallenged since deep knowledge of its materials is expensive to acquire. Languages are their own self-serving world - even a skilled Codesmith, but from outside the guild's coding material, finds it hard to determine by inspection if the product is complex or simply complicated.

The truth is that software is not materially special. Nor are software projects particularly complex when measured against the production of physical products - for example, a car or a house or an aircraft, or even something as ubiquitous as woven cloth and clothing.

Even when measured against the production of other intellectual property products, software is not especially complex.

I would argue that a movie, a TV series or a children's book are of equivalent or much much greater complexity than the typical software project.

Maybe the organisational models for dealing with the "extra complexity" of software have already been solved by other intellectual property industries...

Next time we'll consider how established models of intellectual property production are missing from software production and why, uniquely, ROC allows us to do something about it...