coffee break shibboleth idp install with custom certificate login page and ldap

Thu, Feb 7, 2013

In this tutorial I show how to install the Internet2 Shibboleth IdP, link its authentication to LDAP, customise its login page and use custom certificates. I’ll be using SAML2 rather than Shibboleth as that’s the main use case these days and installing on unix (OS X in my case). For an overview of the relationship between Shibboleth and SAML you could do worse than read my contribution to the SCURL Walk In Access Report which tries to clear up the confusion that can arise when people talk about Shibboleth and SAML. It’s on page 30, Section 3.1 Security Assertion Markup Language of the PDF report. You’ll also register and test your new IdP with the TestShibb2 service and hopefully along the way keep your sense of humour while dealing with middleware configuration. What’s that about coffee break though? Surely it takes more than 15 minutes to install a Shibboleth IdP? Well, it took me about an hour the first time, what with having to search around for how to do the various things but after blowing it away and doing it again, it did take about 15-20 minutes. Enough time for a refreshing brew. So, let us begin…

What exactly is an IdP? It’s more of an Identity Broker than an Identity Provider. Something ‘out there’ wants information about one of your users before they can get access to that something’s content. Your user already has ‘identity’, they just need some way of transporting aspects of that identity to that something. That something is a Service Provider (SP) hungry for information.

If we consider the content as the magnificent interior furnishings of Blandings Manor, not to mention the sumptuous tea trolley contained therein and Harry McDesperate, the worlds most violent house visitor(!), then Harry must first negotiate with Jives the doorman to gain entry. The conversation would go something like this.

Harry pulls on bell cord. Distant sound of ringing followed by footsteps. Door opens to reveal Jives looking none too pleased at the sight of this most aggressive of house visitors.

“I wish to see the fine furnishings contained in your master’s house”, says Harry.

“Who shall I say is calling?”, replies Jives

“Why Harry McDesperate of course!”.

“May I see your identification sir?”.

Harry hands Jives a letter from his keeper. Jives scans the content of the letter, frowns and moves aside.

“Very well sir but please keep your feet off the furniture”.

What happens next is left as an exercise for the reader’s imagination but the scenario illustrates what goes on when someone wants access to something but needs something else to vouch for them. In this case, Harry is the user, Jives the SP, the furniture and tea trolley are the supplier’s goods (perhaps ebooks for instance) and the thing that provides the letter is the IdP. The mysterious letter is a SAML statement. The statement contains information on why the IdP believes it is dealing with Harry. It will say how Harry ‘proved’ he was Harry along with enough information about Harry to please most doormen. I say ‘proved’ as what the statement really says is ‘the person presenting this letter is in possession of the password for Harry McDesperate’s institutional account’. That’s good enough for most doormen but there are others that require more confident statements of identity but we won’t go there. We want to keep it lighthearted. One must always keep one’s sense of humour when dealing with middleware. For the curious, below is a minified mysterious letter.

In the SAML Response there is an Assertion about a Subject, who’s identity was ‘proved’ using the urn:oasis:names:tc:SAML:2.0:cm:bearer method, i.e. the bearer of a password. This is the part that corresponds to ’the person presenting this letter is in possession of the password for Harry McDesperate’s institutional account’. Then there is an authentication statement with some more details on how the authentication was performed and finally, an Attribute about the Subject. In this case we can see that Harry is a bona-fide member of There are lots more things in a real SAML Response but the above gives you the gist of what that mysterious letter contains. Gist! geddit? ho ho!

So now that we’ve covered the ideas of IdP, SP and their interaction, how do you actually set up an IdP? Well dear reader, read on.

There are a few essentials you will need in order to follow along. I won’t cover them here as I want to concentrate on the IdP:

  • Java JDK. 6 is fine.
  • A functioning LDAP server with some users in it.
  • OpenSSL installed (for the custom certificate). yast2, yum, apt-get are all fine for this.
  • Tomcat 6.
  • A valid Tomcat certificate in a Java Keystore (JKS). I’ll show how to generate a self signed one though.

You might have heard about Apache too but the recommendations from the Shibb developers is not to use it to front Tomcat. I’ve never really understood why anyone would want to do that anyway. Most of what the IdP produces is dynamic, especially so with SAML2, so there really is no need to complicate things even further by stuffing Apache in front of it all.

First download Tomcat 6, unzip it somewhere which we’ll call TOMCAT_HOME. The create a directory to store all the IdP files. I put mine in /home/idp. We’ll call that IDP_HOME.

If you don’t have a valid Tomcat certificate, here’s how to create a self signed one:

Let’s rename whatever JKS you have (self signed or valid) to keep it simple. Call it custom.jks and put it in IDP_HOME/cert/custom.jks. Now change TOMCAT_HOME/conf/server.xml to contain these connectors:

Download tomcat6-dta-ssl-1.0.0.jar from here and put it in TOMCAT_HOME/lib. Now create a new file TOMCAT_HOME/conf/Catalina/localhost/idp.xml and put this in it:

then create the directory IDP_HOME/war.

Now download the latest Shibboleth IdP software from here. I used Move to IDP_HOME/src and go into that directory. unzip to get IDP_HOME/src/shibboleth-identityprovider-2.3.8-bin. Go into IDP_HOME/src/shibboleth-identityprovider-2.3.8-bin.

Now you’re ready to install the IdP. From the directory IDP_HOME/src/shibboleth-identityprovider-2.3.8-bin run the command:


choose /home/idp as the directory and use sensible values for the rest. The host name you choose will determine the entityID of the IdP. So if you choose then the entityID will be and the certificates will be for

When the install is finished and before you start up Tomcat, copy all the file in IDP_HOME/src/shibboleth-identityprovider-2.3.8-bin/endorsed to TOMCAT_HOME/endorsed. You can now start Tomcat and check the IdP is OK:

it should say ‘OK’.

So that’s the IdP installed and running. Now shut it down and we’ll customise it by using the custom certificate, a custom login page and connecting to an LDAP server for authentication.

Let’s start with the easy bit, LDAP. Open the file IDP_HOME/conf/handler.xml, comment out LoginHandler RemoteUser and uncomment LoginHandler UsernamePassword. Open the file IDP_HOME/conf/login.config and uncomment Example LDAP authentication. This is what I used:

Let’s do another easy one, customise the login page. Go into the directory IDP_HOME/src/shibboleth-identityprovider-2.3.8-bin and edit the file src/main/webapp/login.jsp. Customise it to your heart’s content but paying attention to the Required Parts otherwise your new page won’t work. Then run ./ again but this time choose not to overwrite the IdP config! Running again will generate a new idp.war in IDP_HOME/war with the new login.jsp in it.

The last part is using a custom certificate but first let’s talk a bit about what the certificates are used for. There are three types of certificate in use in the IdP. They can all be the same physical certificate or they can be separate. The main one is for Tomcat to protect port 443, i.e. HTTPS (SSL). This is the certificate the user’s browser sees when they go to login.jsp. This is normally not a self signed certificate as the signing authority’s root certificate must be present in the browser otherwise, for example, Internet Explorer will panic and recommend the user head for the hills. You can get this certificate from the Janet Certificate Service as a JKS. This will be the JKS you configure in Tomcat’s port 443 connector. The other two certificates are used directly by the IdP to sign and encrypt the SAML Response (the mysterious letter). The user never sees these and they can be self signed and very long lived. Usually you can only get a non self signed certificate for a maximum of 3 years but the signing/encryption certs the IdP installer creates are valid for 20 years! What does this mean for you, the IdP administrator? Well, every three years you must renew your Tomcat certificate but you never need to renew your signing/encryption certificates. Well only every 20 years. These are the ones in IDP_HOME/metadata/idp-metadata.xml and this metadata is the metadata you will send to the UK Federation to register your IdP. So in effect you never need to renew your metadata with the Federation. So this step of customising the signing/encryption certificates isn’t necessary and will create more work for you as an IdP administrator as every 3 years you will have to update your metadata with the new ones. But it’s handy to know how to customise them in case you need to.

First backup your original ones created by the installer. They are in IDP_HOME/credentials. idp.crt, idp.jks and idp.key. I put mine in IDP_HOME/credentials/original. Now go into IDP_HOME/cert to find your valid JKS and do this:

then edit and get rid of everything apart from the top private key entry and include the BEGIN/END gubbins and save as You now need to export your certificate in X509 format in order to put it in the metadata so do this:

Now open the file IDP_HOME/conf/relying-party.xml and in the Security Configurations section change the PrivateKey password=“…” and Certificate to point to IDP_HOME/cert/ and IDP_HOME/cert/ respectively. Then open IDP_HOME/credentials/idp-metadata.xml and replace the signing and encryption certificates with your file but without the BEGIN/END gubbins.

That’s it all done now, phew! The final step is to register with TestShibb2 and see if everything is working. Before you do that you need to copy idp-metadata.xml to something you can use with TestShibb2. The reason is, whatever filename you choose will be associated with this metadata so you need to use the same filename if you want to update the metadata at TestShibb2 later. e.g.

cp idp-metadata.xml

Go to the TestShibb2 registration site and click on the register with TestShib link. Upload your file and follow the configuration instructions it gives you. This lets the IdP trust the TestShib SP. Once all that’s done you can test by going to the TestShibb SP and copy/paste your entityID from idp-metadata.xml into the box and you should then see your customised login page where you can login using your LDAP credentials and the IdP will then sign/encrypt the SAML Response with your custom certificates and lo and behold the TestShibb2 doorman says, ‘Welcome Mr. McDesperate. Please do come in but keep your feet off the furniture!”


comments powered by Disqus