I stumbled across Miles Metcalfe’s blog post about grammar and I started following links around the ‘net looking for confirmation that you can indeed now start a sentence with a conjunction. But I hear you scream. That’s crap English. Of course it is. But, well, yer but no but yer but… You can’t end a sentence with a conjunction yet, but.

Back in school I was taught to make use of joined up script and to think likewise. Not in starts and stops. I gave up reading National Geographic as it was too disjointed. The pictures were stunning but try reading the text. No thanks. I’m not a goldfish you know, you can hold my attention for more than one paragraph. I like sentences that flow into each other using elegant language constructs that an educated person can follow, allowing them to build up a mental picture of the writer of such prose. I can’t stand the crap you get in National Geographic.

Conjunction commencement is just a sympton of a lazy fat chav-o-verse that’s growing, literally, in front of the nation’s tv sets. Staccato conversation between bursts of gratuitous screen violence. Syllabically challenged gruntings between goals. Can’t miss the action on the box. Conversation becomes protocol noises on the wire. Just enough information conveyed to procure another lager tinny.

Then there’s the Americanisation of English. I always make a point of asking vendors of ez software, “My good fellow, what does the Z stand for?”. I always pronounce it as it’s meant to be pronounced by any civilised individual, “ee-zed”. They always think I’m a crank, which I probably am but a grammatically correct crank, thank you very much.

Truncation and numerical substition I can just about understand. You know, you’re on the train from Mallaig to Glasgow and you’re about to go into the 50,000th tunnel and you’ve got to text ahead for a taxi. “u got 1″. Phew, made it before the tunnel. Of course, an educated person would have a coverage map of the route and would prepare a grammatically correct text, ready for sending at the next available reasonable signal strength.

But back to conjunctions. But I’ve just done it again but like. Oh dear. I had a great respect for the late Donald Dewar. He was a well read and educated person but he couldn’t speak on the telly. There was a comedy show on the box a long time ago that did a hilarious sketch where the newsnight chap was interviewing our Donald over a video link and the they were lampooning his speech impediment. He used to make excessive use of “ah” and “um” when on the telly and they portrayed this ad nauseam in the sketch. In the end, the interviewer got up and started banging on the top of the telly, thinking the picture was stuck, while Donald kept going “um ah um um um ah”. But Donald never ever wrote “ah and “um” in his speeches or his books.

Conjunctions are the cogs and wheels of our brain. They are the temporary scaffolding upon which we build conversation but now more and more of the populace are not bothering to put the finishing touches on the ediface of erudite speech and are instead just knocking up those kit built sentences and sending them out rough and ready.

How long before we read in the New Chav edition of Gone With the Wind:

“but frankly my dear, um, I don’t give a damn”

I’ve now finalised the SAMUEL2 package structure. I think this will work and I’ve got it working with AttributeQuery and Attribute interfaces. The test app is coding against the SAMUEL interfaces while in the background the XMLBeans provider is doing all the work.

Having all the XMBeans impl classes in the same package makes it easier to combine SAML tokens as they can get access to each other’s protected XMLBeans objects, rather than have them in assertion and protocol packages and do method level copying.

Following on from the previous post, I’ve decided to get into the spirit of the web services and SOA revolution and rename my blog. It’s been the cakeBlog from the start but the name is getting a bit long in the tooth and I don’t eat a lot of cakes now anyway. Why did I? Read this post.

So what’s the new name? Well, if you read the previous post you’ll see I’ve found mashups to be very useful indeed, especially the ones favoured by rollnecks, i.e. the ones devoid of XML. It was the word recombination that did it for me. My eyes were opened and I discovered a rich seam of analogies with atomic physics that could be mined to describe mashups and web services.

Basically what one must do to join the mashup/SOA revolution is analyse your information output, how it’s displayed, how it’s generated, where it’s content is stored. Are you reinventing the wheel? Then use mashups.

Ionise your information.

Join the SOA revolution by putting your information in a particle accelerator and smashing it against the wall. See what the fragments are. Gather those fragments and turn them into web services. Go shopping for better versions of the ones you’ve identified, then recombine the final set of services with your original, streamlined information.

I’ve done this with my blog, using webcam services from other sites and weather information parser services I’ve written myself. The dashboard of my blog uses information services from wordpress.org.

My blog has been ionised.

Welcome to the ionisationBlog.

I was reading this interesting article from OCLC on mashups, where they’ve included the content of external “web services” into their library web pages. The term they use is “recombination” and as you can see from Wikipedia the more familar term, for me as a Physics graduate anyway, is:

“recombination refers to an electron becoming bonded with an ionized atom”.

In the case of mashups, the web page is the ionized atom and each web service whose output the page consumes is an electron. Now I understand mashups! In fact, I’ve even been using them on this blog, with the webcam images and weather information. All those “electrons” are recombining with my ionized blog to make it more feature rich. And all without a hint of SOAP, XML, Axis or SAML. Wonderful! I’m seriously thinking of renaming my blog along the lines of this epiphany. So watch this space.

Another example of a mashup is over on my eBothy blog. I’ve “recombined” one of my videos from youtube using their embeddable player. You just copy/paste the HTML they give you when you upload a video and you can play the video from any web page. Again without any sign of XML.

The atomic physics analogy is great for understanding web service granularity. Electrons are fundamental particles. They have no structure and so are standalone entities in the web services world. However, other fundamental particles are Quarks and they can be used to build bigger entities. So you can see where I’m heading with web services granularity.

A web service that just reverses a word sent to it would be an electron type service. It just does it’s work within it’s own structure. It doesn’t need to use the services of other web services.

A web service that verifies a credit card would have lots more structure though. It would take the credit card information and use the services of other financial type web services to make a decision.

There are lots of analogies I could start drawing with the particle physics world and that’s just what I’m off to do. And to think of a new name for my blog.

So I’ve started on SAMUEL 2 in my spare time. SAMUEL 1 was getting a bit old in the tooth and was a custom partial implementation of the SAML1.1 spec. It did it’s work using SAX and DOM depending on whether you were parsing or creating XML representations of SAML tokens. It became obsolete in Guanxi when I discovered XMLBeans which gave me a much more powerful toolkit for working with SAML without the overhead of redesinging SAMUEL. However, this means that Guanxi is tightly coupled to the SAML1.1 schema. Not a real problem, just not quite aesthetic enough for me.

So now I’ve started on the next generation of SAML support for Guanxi, the new SAMUEL 2 toolkit. It’s designed to insulate Guanxi from the SAML schemata. It does this by exposing an API that covers the assertion and protocol areas of the SAML specs and a SAMLFactory that an application such as Guanxi can use to create and parse SAML tokens. The SAMLFactory will be responsible for creating the top level tokens that are standalone and do not depend on other tokens. Those top level tokens will provide methods to add or create their respective sub tokens.

The SAMLFactory loads up a properties file and instantiates a SAMLProvider, to which it delegates all calls so the provider does all the work of implementing the api. The application codes against the api.

At the moment I’m coding an XMLBeansSAMLProvider.

Since upgrading to XMLBeans 2.2.0 I’ve been getting this error frequently. It occurs every 2 or 3 times an InputStream is parsed:

org.apache.xmlbeans.XmlException: error: Unexpected end of file after null

from the code:

soapEnvelopeDoc = EnvelopeDocument.Factory.parse(aaConnection.getInputStream());

Seems XMLBeans 2.2.0 has problems parsing an InputStream. I got some advice from the user list to convert to a String and parse that instead. So I knocked up a quick fix:

InputStream in = aaConnection.getInputStream();
BufferedReader buffer = new BufferedReader(new InputStreamReader(in));
StringBuffer stringBuffer = new StringBuffer();
String line = null;
while ((line = buffer.readLine()) != null) {
stringBuffer.append(line);
}
in.close();
soapEnvelopeDoc = EnvelopeDocument.Factory.parse(stringBuffer.toString());

and it seems to sort the error. So I’ll have to update the Guanxi IdP and SP releases.

Following on from the success of the GSK proof of concept, I’ve sorted out the classloader issues and cleaned out /shared/lib so all the Guanxi SP specific libraries now reside in /webapps/shibb/WEB-INF/lib. Here’s what the layout looks like now. All paths are relative to TOMCAT_HOME:

/components/sakai-guanxi-pod-manager-pack – provides GSKPod services for the shibb portal to use
/components/sakai-guanxi-user-pack – the Guanxi UserDirectoryProvider and GroupProvider implementations

/shared/lib/sakai-guanxi-gskpod-api-1.0.jar – the GSKPod api. Implementations of GSKPod (Guanxi Shibb Kit Pod) offer SAML attribute policy enforcement etc.
/shared/lib/sakai-guanxi-pod-manager-api-1.0.jar – the Guanxi PodManager api. Implementations of this allow the shibb portal to register GSKPods with Sakai

/webapps/shibb – the Guanxi Shibboleth portal. This is where it all happens. Users get here after they’ve been through the Shibboleth process.

It’s certainly alpha at the moment as it only works with the Guanxi IdP on my machine as I’ve set up the attribute mapping rules to support the crude profile I’ve created to get the shibb portal working. To get in to Sakai via Shibboleth you need these attributes. You’ll notice their tightly bound to Sakai’s UserEdit:

• sakaiUserID – currently mapped from our LDAP’s cn attribute. This is the Sakai eid.
• sakaiFirstName – currently mapped from our LDAP’s givenName attribute
• sakaiLastName – currently mapped from our LDAP’s sn attribute
• sakaiMail – currently mapped from our LDAP’s mail attribute

I haven’t bothered with URNs on the attribute names as this is alpha. What should probably happen is a Sakai attribute naming space be defined, urn:org:sakai, or something similar and IdPs can map accordingly. Well the Guanxi IdP can map. Not sure about other IdP implementations. The 4 attributes listed above can be handled by eduPerson though. What’s not clear yet is the group memberships coming in from Shibboleth. In which attribute should they be held? If we define our own we risk excluding IdPs that can’t map. Is that a good thing? It’s not good or bad I suppose. Certainly however, the technical limitations of a Shibboleth implementation should not affect how Sakai defines attribute sets. This is policy. IdPs must satisfy the policy or their users won’t get in.
So there are a few steps to take to get to a beta release. I’d say I’m in alpha at the moment as the GSKPod only works with raw HTTP headers containing attribute names and values. I’ve just noticed that the Guanxi Pod from the Guard doesn’t cache the raw SAML assertions. Instead it just passes them to a Bag which streams them through a SAX parser into their constituent attribute names and values. I’ve never really used the Bag in anger, so now it’s time to refactor it to cache the raw SAML assertions. The shibb portal can then add them to the GSKPod it registers when the user gets past Shibboleth. So what do I have to do in the short term to get to beta?

• Refactor the Guanxi Guard Pod and Bag subsystem to cache raw SAML assertions from an IdP. This will result in a point release of the Guanxi SP.
• Add SAML assertion support in GSKPod. Currently it’s just mirroring Sakai’s UserEdit structure. The fields are set according to attributes in the HTTP headers.
• Add a GSKPod factory so GSKPod implementations can be changed.
• Sort out some sort of error reporting to the user if they have a problem with their Pod or they can’t get in to Sakai via /shibb/gx
• Implement logout for a Shibboleth user
• Implement all the methods in GuanxiUserDirectoryProvider and the corresponding helper methods in PodManager
• Add the Guanxi Engine to the distribution. Currently /shibb/gx is using the Engine on my machine.
• Come up with a first pass attribute profile, possibly using eduPerson, possibly not.

That should bring it up to beta and ready for testing. Once I’ve got my head round subversion I’ll commit the whole lot to Sakai’s contrib.

Then the work will begin on SAML assertion policy enforcement. Attribute TTLs and Audiences etc. That should be interesting.

So, after a few late nights and some head scratching, not to mention serious classloader issues, I’ve now shibbed Sakai 2.2.2. I used the newly released Guanxi SP 1.3.0. At the moment, the Engine is separate with just a Guard in the /shibb portal but now I know the shibboleth portal concept works I’ll bring in the Engine to make it a standalone system.

The major problem I had was the “closed” nature of the Shibboleth subsystem, i.e. the Guard and the Pod. The Pod was causing classloading problems, so get round them I just bunged the whole lot in /shared/lib and it worked, as I knew it would.

But now that the proof of concept is out of the way, I’m going to do a GSKPod which will conform to Sakai’s api/impl architecture, with the api going in /shared/lib and all the Guanxi SP stuff staying put in /webapps/shibb/WEB-INF/lib. It’ll be the /shibb portal’s job to do the subsystem bridging, from a Guanxi Pod to a GSKPod. That should get round the classloading problems and make the GSK a proper Sakai player. I think I’ll implement all the SAML Assertion policy management in the GSKPod. What’s now emerging is an architecture whereby the low level Guanxi SP (Engine + Guard) do the Shibboleth work and the GSK takes over and adds application specific bells and whistles, in as generic a way as possible. So raw SAML comes out of the Guard and is refined in the GSK. It’s the GSK that applications will work with.

The shibboleth functionality requires CARET’s FilterUserDirectoryProvider though as I don’t replace anything in Sakai. The /shibb portal is complementary and does not affect normal Sakai login. As Sakai can’t chain UDPs, the FilterUserDirectoryProvider is used to do the chaining. It just has one UDP in it’s chain, a GuanxiUserDirectoryProvider, which delegates to a GuanxiPodManager, implementing the PodManager interface. It’s this GuanxiPodManager that I’ll refactor to work with GSKPods.

But the /shibb portal concept works. I’ve proved that today.

It’s a long and lonely road is the old shibboleth highway. Full of tumbleweed and parched bones. No wonder. So the latest on the GSK for Sakai is this. I’ve got everything installed and running. Done a UDP and a PodManager to build a bridge between SAML attributes and Sakai’s subsystem and I’m 90% there. But now I’ve hit Sakai’s classloading nightmare. Here’s the problem.

The PodManager, that provides the bridge between SAML and Sakai is a Sakai component that lives in components/sakai-pod-manager-pack. As it works with Pod classes, Pod.class is in:

components/sakai-pod-manager-pack/WEB-INF/lib/guanxi-common-1.2.9.jar

so Pod.class is loaded by the components classloader.

The /shibb portal is contained in /webapps so it has it’s own classloader. So when the Shibboleth session is complete, i.e. the user has authenticated at their IdP and the Guard at Sakai has their attributes, the Guard creates a Pod and bungs their attributes into it. That Pod class was loaded by the webapps classloader. Pod.class is in:

webapps/shibb/WEB-INF/lib/guanxi-common-1.2.9.jar

All well and good. /shibb/gx traps this Pod and asks the PodManager to register it.

Now the trouble begins.

The Pod that webapps/shibb creates, when it’s passed to PodManager.register(Pod pod), causes this error:

java.lang.LinkageError: loader constraints violated when linking org/guanxi/common/Pod class

The Pod class from /webapps/shibb has crossed the classloader boundary to /components. Now this is dark ages stuff. Linkage errors? In the name of the wee man.

Anyway, I’m now trying to resolve this issue and it might take a wee while. I could probably dump the whole lot in /shared/lib but that’s not a nice solution. I think perhaps guanxi-common-1.2.9.jar is not granular enough. Well, what I mean is it’s not designed for usage in Sakai.

What’s really the problem is shibbing an application. One size certainly does not fit all. In an ideal world, the Guard would use a Sakai service to produce a Pod but Sakai services don’t exist outside Sakai. So how would a Guard ever know about Sakai, or every other application on the planet for that matter.

As ever in Shibboleth, it’s always that last little bit…

Following on from the previous post, I sat down and rewrote the weather.php script for parsing the Met Office’s severe weather warning page for Highland and Eilean Siar. It took a bit of effort to figure out what was going on in their javascript but it was worth it as the warnings are back and I now have a much better record of the storms, as I can now get access to the messages that accompany the warnings. So I’ve retired the v1 record of storms and moved on to the v2 one, which contains a lot more information. At some point I’ll publish these storm information records on the wiki.