NetKernel News Volume 2 Issue 48

October 7th 2011

What's new this week?

• Repository Updates
• WARNING: Java 7 *still* has critical bugs
• Status: NetKernel Standard Edition 5.1.1 Preview 1
• Tony Gets Absurd

Catch up on last week's news here

Repository Updates

The following update is available in the NKSE and NKEE in version 4.1.1

• http-client-2.7.1.nkp.jar
  • Added compatibility update to support NTLM proxy servers that attempt to negotiate the authentication model. Big thanks to Kevin Ansberry for helping diagnose this scenario.

WARNING: Java 7 *still* has critical bugs

I'm sad to report that we are still unable to recommend using any version of NetKernel on Java 7. You may recall that in August we warned of difficulties with the Jetty http library but managed to find and ship a workaround.

Unfortunately we've also discovered that a SIGSEGV crash will occur when doing a full system index (which NK does soon after first boot, and also whenever a new package is installed). It turns out that this crash is the widely reported Lucene PorterStemmer Java 7 JVM crash...

https://issues.apache.org/jira/browse/LUCENE-3335

...for which there is no known workaround and for which, if we're lucky, Oracle have said they may fix the JVM at the second round of bug fixes. We have no idea when that may be.

I don't know about you, but this is a critical bug in an "optimization routine" in the JIT compiler. This was a known bug in Java 6 - but it was not critical since it was only triggered by setting an optional flag to enable the "optimization". In Java 7 it is now on by default and cannot be disabled!

If I were responsible for this product then it seems like my first critical action would be to turn off the "optimization" (instability!) and ship a version that didn't/couldn't crash by default. As it is, the recommended Java download on the Oracle site is still Java 7 and it remains completely broken until further notice.

This is no way to manage an infrastructure product.

Thanks to Tom Geudens for sharing this.

Status: NetKernel Standard Edition 5.1.1 Preview 1

You may have noticed that my newsletters have been somewhat more brief of late. This is a good sign - it shows that rather than spinning my gears writing my usual claptrap - I've been focused on pushing forward to get the NK5 release to the finish line.

Last week's release of NK5-preview-1 unleashed a torrent of extremely useful feedback. Its interesting that although substantial parts of the system are already out there in the NK4 repository, just the act of calling the collection something new has stimulated fresh insight and helped with refining the whole.

When's a float not a float?

For example, did you know that Float.parseFloat() is not consistent with respect to International locale? No, neither did we. We discovered this in one line of code of the new kernel that was using parseFloat() to read a floating point configuration parameter - a new parameter we've included in the cost accumulation algorithm of the kernel to weight for CPU core performance.

It turns out that in Belgium and Holland (and other countries no doubt) the default locale for floating point numbers is to use a comma (",") rather than the British/American convention of a full-stop (".", period, decimal-point). For example "1,5" as opposed to "1.5".

Naively one might expect that the JVM would use its locale setting to decide the appropriate parsing model for parseFloat() - during boot, when the parameter was written to its configuration property it had used the locale to write it with the comma when in Belgium. But alas it doesn't, it is hard coded to the culturally imperious Anglo-American hegemony. (You see what happens when I don't release the claptrap - it spews out all at once).

We are fortunate that rather than taking umbridge at this offensive presumption, Tom Geudens reported the parse exception, and we were able to fix it with a one liner switch to NumberFormat.getInstance().parse(). We slipped this fix into the preview-1 download and also posted a patched kernel in the NK5.1.1 repo.

NTLM Proxy

If there's one thing that has caused more pain than any other over the years it is getting Java to talk through a Windows IIS NTLM proxy server. Or rather, that should be getting an NTLM proxy to allow Java to talk through it.

Yet again I'm indebted to another "friend of ROC", Kevin Ansberry, who reported that he was unable to get out of his corporate firewall. Kevin has in the past helped us work through this and, we both thought that we'd put this to rest a long time ago. We were wrong.

It seems Kevin's corporate infrastructure has changed, including his NTLM proxy. For some reason the new proxy wasn't doing what the earlier one was. The diagnosis of what was going on was compounded by a red-herring in that he's also switched to Windows 7 from XP.

You do not need to know the tragic details of how we used the IT equivalent of neutrino-scattering to work out what was happening. The upshot is that the proxy had decided to start negotiating authentication protocols and to start with had decided to use Kerberos rather than NTLM. We found a way to make the apache HTTP client library specify its preferred authentication order. Net result we're back where we thought we always had been. NKEE will use an NTLM proxy correctly again (at least until it stops the next time!).

Plan

As you can see, these minor niggles are well in hand and we're on track to release the final versions of both NKSE and NKEE 5.1.1 next week. Please take the time to kick the tyres and let us know if there's anything else out of order. Otherwise our plans are on track. Watch this space...

Tony Gets Absurd

Steve Jobs said "Think Different". Daniel Schechtmann, this weeks winner of the Nobel prize for chemistry was told his discovery of quasi-crystals was "impossible" and he was asked to leave the NIST research team for being an embarrassment.

Its not easy defying convention.

We have had our fair share of scepticism over the years. So it is entirely appropriate that Tony has been contemplating what "thinking different" really entails...

http://durablescope.blogspot.com/2011/10/ending-absurdity.html

NetKernel Europe Bootcamp - Brussels, Belgium, Thursday 27th October 2011

Sponsored by Steria Benelux

http://www.1060research.com/conference/NKEU2011/

The event takes place on Thursday 27th October. If you've registered already, thank you! We will soon be getting in touch with the details so you can plan your travel.

If you want to take advantage of face-to-face opportunity we will also be around on both Wednesday 26th and Friday 28th for meetings etc.

We're looking forward to seeing you in Brussels.