NetKernel News Volume 4 Issue 2

January 18th 2013

What's new this week?

• Repository Updates
• On The Economics of ROC
• Interview - Frank Boddeke: CTO

Catch up on last week's news here

Repository Updates

No updates this week.

On The Economics of ROC

Back in HP Labs in the late 90's I was an outsider to the software technology world (my background was in quantum physics). I started building large scale internet systems and realised that the state of the art technologies didn't make economic sense - it seemed like while you could always build something, you ended up on a saw-tooth lifecycle - building stuff, but at some critical level of complexity, being forced to throw away and start over. This seemed pretty crazy (and at a fundamental level nothing has changed today).

Luckily, I was inside a research organisation and had the freedom to think about the fundamentals. I realised (back in '97) that the Web seemed to be growing continuously, to be adapting to change and evolving - but it wasn't displaying the classical saw-tooth software life-cycle. We've never had to throw the Web away and start again.

The Web was obeying a different set of economic rules.

So we set out to fundamentally understand how and why the Web worked - to extract the essence of it, to generalise it, to bottle it and to apply this right down inside the core of how software is done. ''To take a resource oriented approach to computing.''

This wasn't motivated by technology or programming languages - one of the powers of the Web is that absolutely no-one who uses it gives a damn about the technology its built with - this was motivated by being able to tap into the Web's economic properties and to exploit these in any software system (not just Web apps).

The result is something we call ROC and our platform is NetKernel. Our product is a carrier class infrastructure tool that powers national telecoms systems, online retail and is deployed in sectors including medical, legal, insurance, defence etc. NetKernel is even used to power core parts of the emerging semantic-Web infrastructure (purlz.org).

It has some very powerful technological properties - it scales linearly, it caches and minimizes computational cost. Its damned fast and very cool technology...

But its never been about the technology! Its only a means to an end - and the end is the economics.

When you really embrace emergent, decoupled, resource oriented composition - the economic rules of the game are rewritten.

But here's my problem - in a fundamental way - its impossible for me to show to you that ROC is tapping into a different set of economic rules. Because, any rational impartial person would listen to me and say "yes, you would say that wouldn't you".

The paradox is that you get to experience the change in economics when you've adopted and used NetKernel for a while. Not just on a one-off project - but on a day to day basis over the long term.

I realized that a way out of this paradox is for me to shut up and let others speak.

Here's the first in an occasional series of interviews with hard-core ROC adopters. With a focus, not on technology, but the economics of ROC...

Interview - Frank Boddeke: CTO

I intend to interview a range of ROC adopters across a range of business positions - to get a broad perspective of what ROC adoption means. First up is Frank Boddeke the CTO of a company that makes and sells software products. A business where the economics of the choice of the technology are not arbitrary but which directly impact the bottom-line...

What sort of business do you work for?

I work for a small company Edge Technologies BV that sells a range of software products to the telecommunication industry. Our products do network monitoring, diagnostics and reporting for network service providers.

What is your position and what are your day to day responsibilities?

One part of being a small company is trying to look bigger. That is why I call myself the chief technology officer: responsible for the technology... and making coffee (which is not to be taken lightly!)

What do you use NetKernel for?

Everything! All our products are built on NetKernel.

When did you first start using NetKernel?

We opened our eyes in 2006. Just before NetKernel 3.1 was released. I had to look it up in my e-mail, it is that long ago. and guess what: one of the first things I developed was a soap interface and we still use that interface on several production instances. It was such a relief to use NetKernel for this instead of one of those java-soap frameworks we used before.

Can you remember what convinced you to adopt NetKernel?

No I cannot remember. But I just read it in my e-mail: "platform with minimal overhead and linear scaling" (although we did not really believe it at first).

Can you remember how you learned how to use it? Any tips for newbies?

Take a small problem you are working on, chunk it up in simple questions and responses and start to assemble and/or build those (by copying/modifying examples). Make good use of all the nice tools NetKernel has to offer (my favourite is the Visualizer power trace tool).

Which of the NetKernel tools/libraries/languages do you use most often?

Almost all is Java.

To a traditional software engineer NK's extreme loose coupling can be unnerving. How do you deal with this and do you have any strategies for how you create new address spaces?

I am a physicist, not a software engineer. So for me it was no problem at all. (now make me use one of the java frameworks and I feel like programming with my nose with my hands tied behind my back).

Address spaces? Hey, I am old-school hard-core nk3... our address spaces are very simple: external is ffcpl:/<-- module-name -->/<-- request --> and internal is active:<-- some descriptive name -->. we did make up one extra space for dealing with events in our event-transport: event:/<-- listener -->

We talk about the 3C's of construct, compose, constrain. How much of your time do you typically spend on each C?

Most of our products are re-shuffles of modules we already have. So obviously that is a big "c" for compose. (and make no mistake: reshuffling can make complete new products!) Some other products, typically that have to do with external systems/interfaces or new concepts will have a bigger "c" for construct. We always start unconstrained and put in throttles and checks along the way to protect the system and because some users are scared of stack traces.

We claim that ROC leads to real re-usability in software - do you agree? Do you have any examples?

Yes, absolutely. The only reason we are able to do what we do with limited resources is because of reusability. Over the years we build up a pretty huge toolbox and now a lot of work is just like playing with lego bricks.

Our products deal with network monitoring, diagnostics and reporting. And although generating a pdf of last week's traffic statistics of your ISP's network domains and checking the copper cable to your home sound very different, for us it is 75% the same. One of the first components we made was this concept of a central engine with probehosts in each network domain. (We have customers dealing with 20 walled off network domains). We still use this in all our products.

Recently we've been saying ROC eliminates (or at least smooths out) the saw-tooth development cycle. The classical build-it-throw it away-start again software life-cycle. With ROC you can evolve solutions and continuously adapt them. Do you agree? If so, what does this mean to your business?

Yes, again, absolutely. This is basically how we have grown our products = our business. All our products are built up out of a limited number of modules. Each module groups together similar/complementary resources and functionality. These modules evolve over time. Evolve as in, most of it stays the same, some functionality is added and some is rewritten. A new product is not starting from scratch but putting together the right functionality we already have and add what is missing.

Do you have a favourite ROC war story to share?

I like extreme numbers (I am a physicist remember). Here are two of our more extreme accomplishments using netkernel:

- our largest system uses 44 instances of netkernel (one loadbalancer, two engines and 40 probehosts in 20 different domains) - one of our network monitors (a single netkernel instance) can query/store and act on 50 parameters on 1000 network routers every 5 minutes

Anything else you'd like to tell us?

The one thing I always tell people when I introduce them to NetKernel is how amazed I was when I first started developing on top of NetKernel: the more you get it, the less lines of code you need!