Skip to content

Yet another cool OpenSolaris feature: Crossbow flows

December 2, 2009

While backing up my macbook to my OpenSolaris/ZFS server (what else would you trust your backup to?) I wanted to figure out if rsync was taking full advantage of my network card.

Add the opensolaris nic to a flow:

# flowadm add-flow -l rtls0 -a transport=tcp flow1

Then show the flow statistics:

# flowadm show-flow -S

flowadm shows approx ~90,000 Kb/sec flowing across the interface (not bad for a 100 Mpbs NIC card).

Advertisements

Adding OpenId support to a Scala / Lift application

November 23, 2009

The Lift web framework includes basic support for OpenId using the the openid4java library. I wanted to learn a little more about Scala, Lift and OpenId – so what follows are my notes on how to get this running, along with some minor improvements that I have made.

You will want to start by familiarizing yourself with the OpenId chapter in the (rather excellent) Lift book.

Out of the box, the lift-openid support gets  you a vanilla login form where the user can enter their OpenId URL and submit the request.  Something that looks like this:

While this is nice for identity geeks, your grandma isn’t going to be using this anytime soon. The OpenId UI issues are being hotly debated as you read this, and I won’t open that can of worms. Suffice to say that mere mortals will prefer to use a “Nascar” branded interface that let’s them login using their favourite email provider such as Google or Yahoo (The Nascar UI is a reference to the logos that are plastered over the race cars).  Most users won’t have a clue what their OpenId provider (OP) URL is , but they can spot the logo of their email provider a mile away.

Too add a slicker login interface, I grabbed a copy Jarret Vance’s OpenId JQuery Plugin.   Lift uses JQuery by default, so integration of Jarret’s plugin is quite easy.  The plugin consists of provider logos (put these in your /webapp/images directory), a Javascript JQuery extension (jquery-openid.js – put this in your webapp/scripts directory), and some html templates for the login page.

I created a Lift template for the login page and placed it under my webapp/templates-hidden directory. The template is pretty much a copy of the default template, but adds some javascript for the openid integration. In the head tag of the template include the following:


<script id="jquery" src="/classpath/jquery.js" type="text/javascript"/>
 <script id="json" src="/classpath/json.js" type="text/javascript" />
 <script id="openid" src="/scripts/openid-jquery.js"  type="text/javascript"/>
 <script type="text/javascript">

 $(document).ready(function() {
 openid.init('openIdUrl');
 });
 </script>

Note that Lift performs “head” merging – which is to say that any content withing <head> tags in a snippet is merged in with the templates head tag. So the above approach is not strictly necessary.

Now that we have a login template, we can define the login page. Most of the content is copied verbatim from Jaret’s plugin

<lift:surround with="openid" at="content">
 <head>
 <title>
 <lift:loc>Login</lift:loc>
 </title>
 </head>
 <h2>
 <lift:loc locid="Welcome">Login Page</lift:loc>
 </h2>
<p>This site uses <a href="openid.net">OpenId</a> so you don't have to remember yet-another password.
 <br/>Enter your OpenId URL, or click one one of the icons below to sign on with an existing account.
 </p>
 <!-- Simple OpenID Selector -->
 <form action="/openid/login" method="post" id="openid_form">
 <div>
 <ul>
 <li title="OpenID"><img src="/images/openidW.png" alt="icon"/>
 <span><strong>http://{your-openid-url}</strong></span></li>
 <li title="Google">
 <img src="/images/googleW.png" alt="icon"/><span>https://www.google.com/accounts/o8/id</span></li>
 <li title="Yahoo">
 <img src="/images/yahooW.png" alt="icon"/><span>http://yahoo.com/</span></li>
 <li title="AOL screen name">
 <img src="/images/aolW.png" alt="icon"/><span>http://openid.aol.com/<strong>username</strong></span>
 </li>
 </ul>
 </div>
 <fieldset>
 <label for="openid_username">Enter your <span>Provider user name</span></label>
 <div><span></span><input type="text" name="openid_username"/><span></span>
 <input type="submit" value="Login"/></div>
 </fieldset>
 <fieldset>
 <label for="openid_identifier">Enter your <a href="http://openid.net">OpenID</a></label>
 <div><input type="text" name="openid_identifier" id="openid_identifier"/>
 <input type="submit" value="Login"/></div>
 </fieldset>
 </form>
</lift:surround>

For brevity I have removed some of the provider logos from the example. The nice thing about this plugin is that it is very easy to add or remove providers.

This gives us a nice “Nascar” login page that looks like this:

Now this will make both the geeks and grandma happy.

In the next blog entry (when I get around to it…) I will describe how to add support for the OpenId attribute exchange protocol (AX). This enables our application to retrieve additional user attributes from the provider such as the user’s email address, screen name, preferred language, etc..  Stay tuned.

Poster Girl for Anti-Vaccine movement makes miraculous recovery…

November 9, 2009

 

It seems as if Desiree Jennings has been “cured” and is on the road to recovery.  Jennings is the young cheerleader who purportedly got Dystonia after receiving a flu shot.

Meanwhile, according to figures released by the ECDC (November 9,2009),   6,483 people have died from H1N1.

None are expected to recover.

 

 

A minor update to the Grails OpenSSO plugin

October 5, 2009

A very small update to the Grails Open SSO Plugin (0.2).

Previously the OpenSSO  loginUrl was hard coded into the plugin. This is now configured in OpenssoConfig.groovy. This configuration file will get installed in your Grails application when you install the plugin.

Here is an example:

opensso {
// default: opensso is active
active = true
// error page url to redirect to when the policy eval fails
errorPage = '/auth/denied.gsp'
// Urls to be treated as anon (access policy is ignored)
anonymousUrls = ['/index.gsp' , '/auth/denied.gsp', '/images/*']

// Url to redirect the user to if they do not have a valid SSO Token
// This should be the url of the OpenSSO login service
loginUrl = "http://localhost:8080/opensso/UI/Login"
}

Using the Jersey Client with Scala (Revisited)

August 5, 2009
tags: ,

In my last post I noted that there is currently a Scala compiler bug that prohibits using the Jersey Client with Scala.

I wanted to learn Scala and the Jersey Client API, so I took a crack at creating a Scala replacement for the offending classes. This isn’t fully  plug compatible with the Java API – but assuming you just want to call the Jersey client from Scala (and not try and call these classes from other Java classs) – you should be OK.

The code is posted below. This is my first foray into Scala – so I am probably doing some bone headed things. Improvements are welcome.

First, let’s start with the Resource class (the name is intentionally different from the replacement WebResource – so we don’t collide on import).

package com.my2do.jersey.client

import  com.sun.jersey.api.client.filter.Filterable
import com.sun.jersey.api.client.config.ClientConfig
import com.sun.jersey.api.client._
import com.sun.jersey.client.impl.ClientRequestImpl
import _root_.java.net.URI
import javax.ws.rs.core.{Cookie, MediaType,MultivaluedMap,UriBuilder} 

/**
 * A Simplified Scala replacement for the Jersey Client WebResource class.
 *
 * This is needed because there is a bug in the scala compiler that
 * prevents the Jersey WebResource class from being used.
 *
 * see <a href="http://lampsvn.epfl.ch/trac/scala/ticket/1539">the bug report</a>
 *
 * This class does not implement UniformInterface. UniformInterface has many
 * methods that reference the GenericType class. The GenericType class
 * looks like it was added to Jersey to support a more conscise way of
 * referencing really nasty parameterized java types. Scala as the type keyword -
 * which serves the same purpose - so the GenericType class does not appear to be needed.
 *
 */

class Resource(clientHandler:ClientHandler, uri:URI) extends Filterable(clientHandler) {
 override def toString() = uri.toString()
 override def hashCode() = uri.hashCode()

 override def equals(other:Any):Boolean = other match {
 case that : Resource =>  isComparable(that) && this.uri == that.getURI()
 case _ => false
 }

 def isComparable(that:Any) = that.isInstanceOf[Resource]
 def getUriBuilder() = UriBuilder.fromUri(uri);
 def getURI() = uri

 /**
 * Create a Resource from this web resource with an additional
 * query parameter added to the URI of this web resource.
 *
 * @param key the query parameter name
 * @param value the query parameter value
 * @return the new web resource.
 */
 def queryParam(key:String , value:String ) = {
 val b = getUriBuilder();
 b.queryParam(key, value);
 new Resource(clientHandler, b.build());
 }

 /**
 * Create a new Resource from this web resource with additional
 * query parameters added to the URI of this web resource.
 *
 * todo: Creat a scala implicit that converts from MultivaluedMap to a scalafied Map?
 * @param params the query parameters.
 * @return the new web resource.
 */
 def queryParams(params: MultivaluedMap[String,String]) = {
 val b = getUriBuilder();
 val i = params.entrySet().iterator();
 while( i.hasNext() ) {
 val entry = i.next();
 val vals = entry.getValue().iterator();
 while( vals.hasNext() ) {
 b.queryParam(entry.getKey(),vals.next())
 }
 }
 new Resource(clientHandler, b.build());
 }

 /**
 * scalifed version of above
 */
 def queryParams(params: Map[String,List[String]] ) = {
 val b = getUriBuilder();
 params foreach { case(key,valueList) => valueList foreach { b.queryParam(key,_)}}
 new Resource(clientHandler, b.build());
 }

 def path(path:String) = new Resource(clientHandler, getUriBuilder().path(path).build());

 // uniform interface methods according to Roy
 def head(r:Request) =  method("HEAD",r);
 def options(r:Request) = method("OPTIONS",r)
 def get(r:Request) = method("GET",r)
 def put(r:Request) = method("PUT",r)
 def post(r:Request) = method("POST",r)
 def delete(r:Request) = method("DELETE",r)

 def method(method:String, r:Request) = {
 getHeadHandler().handle(new ClientRequestImpl(uri,method,r.entity, r.metaData))
 }
}

This is much simpler than the WebResource class it replaces (but admittedly not as functional).  All of the REST methods take the Request class, and all return a Jersey ClientResponse.   The Request class encapsulates the entity being sent (if there is one), and any request headers, cookies, etc:


package com.my2do.jersey.client

import com.sun.jersey.client.impl.ClientRequestImpl
import com.sun.jersey.core.header.OutBoundHeaders
import _root_.java.util.Locale
import _root_.java.net.URI

import javax.ws.rs.core.Cookie;
import javax.ws.rs.core.MediaType;

/**
 * The Request singleton has some syntactic sugar for creating Request objects
 * For example:
 *    resource.get(Request().acceptLanguageString("en")) // creates a request with a null entity
 *    resource.post(Request(myFile).mediaType("multipart/mime"))
 */

object Request {
 def apply() = new Request(null)
 def apply(entity:Object) = new Request(entity)
 def noEntity = new Request(null)
}

/**
 * Encapsulates a  web request
 * including the entity to be sent (optional - for example this will be null for a get)
 * and all of request cookies, accept headers, etc.
 */

class Request(val entity:Object) {

 var metaData = new OutBoundHeaders()

 def mediaType(t:MediaType) = {
 metaData.putSingle("Content-Type", t);
 this
 }

 def mediaType(t:String):Request = mediaType(MediaType.valueOf(t))

 def accept(mediaTypes:MediaType* ) = {
 for (val m <- mediaTypes)
 add("Accept", m)
 this
 }

 def acceptString(mediaTypes:String* ) = {
 for (val m <- mediaTypes)
 add("Accept", m)
 this
 }

 def acceptLanguage(locales:Locale*) = {
 for (val l <- locales)
 add("Accept-Language", l)
 this
 }

 def acceptLanguageString(locales:String*) = {
 for (val l <- locales)
 add("Accept-Language", l)
 this
 }

 def cookie(cookie:Cookie ) = add("Cookie", cookie)      

 def header(name:String, value:Object ) = add(name, value)

 private def add(name:String, value:Object ) = {
 metaData.add(name,value)
 this
 }
}
&#91;/sourcecode&#93;

One of the things that Jersey client API does for you is optionally converts the ClientResponse entity to a class of your choosing (assuming there exists a MessageBodyReader that can perform the conversion). For example <span style="color:#339966;"> resource.get(String.class)</span> will convert the ClientResponse entity to a String (or throw a UniformInterfaceException if the GET failed).

To accomplish the same thing in Scala, I experimented with implicit conversions. This is a neat (but potentially dangerous!) way of having the Scala compiler automagically convert from one type to another.   Here is an example snippet where we call the POST method and auto convert the returned entity to a String:


val multiPart = new MultiPart().
     bodyPart(new BodyPart(someTestBytes, MediaType.APPLICATION_OCTET_STREAM_TYPE));
val result:String = resource.path("/upload").post(Request(multiPart).mediaType("multipart/mixed"))

Note in above example we explicitly set the type of the result object. Normally in Scala we can let the compiler figure things out – but in thise case we don’t want a ClientResponse object -we just care about the entity being returned. The type hint triggers the implicit conversion from a ClientResponse to a String.  We could get the same result if we passed a ClientResult to some other Scala method that required a String (assuming the implicit conversions are in scope).

The conversions are declared like so:

/**
 * Implicit Conversions to convert from a ClientResponse to another type
 */

package com.my2do.jersey.client

import com.sun.jersey.api.client.{ClientResponse,UniformInterfaceException}
import java.io.InputStream

/**
 * Implicit Conversions from a ClientResponse to another type
 * Use with caution!
 * To use this, import com.my2do.jersey.client.Conversions._

 */

object Conversions {
     implicit def response2String(r:ClientResponse):String =  getClientResponseEntityAs(r,classOf[String])
     implicit def response2InputStream(r:ClientResponse):InputStream = r.getEntityInputStream()
     implicit def response2ByteArray(r:ClientResponse):Array[Byte] =  getClientResponseEntityAs(r,classOf[Array[Byte]])
     // todo: What others do we need?

    /**
     * unwrap the entity - thrown an exception if the entity is not valid
     */
     def getClientResponseEntityAs[T](r:ClientResponse, c:Class[T]):T = {
         if( r.getStatus() >= 300 )
            throw new UniformInterfaceException(r)
         r.getEntity(c)
     }
}

After using Scala for a few days, I have to say it is a lot of fun 🙂

Using Scala with the Jersey client

July 30, 2009

Turns out you can’t (or at least not yet).

See  http://lampsvn.epfl.ch/trac/scala/ticket/1539

Maven and the one true repository

July 14, 2009
tags:

One of the things that first attracted me to the notion of maven was the idea that almost all of what you want in terms of dependencies were available from a few repositories. Or at least that was the general idea.

Fast forward to today, and we seem to have a proliferation of repositories. In fact, we have so many repositories that we even have tools to help us manage all of these repositories. I’m using one of these tools (Nexus – a very nice tool BTW ) and I have noticed my Nexus configuration is getting quite complex! I’m wondering if we have traded jar dependency hell for repository management hell?

To my mind, tools such as Nexus are treating the symptoms, and not the disease.

One of the symptoms: maven pom files that contain repostory configuration. This seems like a big fat anti-pattern (violates sepration of concerns, and all that lot). Surely the pom should tell us what artifacts to use, not where to go find them.

What if we had fewer repositories? In fact, what if we had only one big giant global repository in the sky? Let’s call it the BARGE (Big Ass Repository for Global Enlightenment). Would that not vastly simplify the whole isssue of dependency management (and not just for Maven, but for many tools that want to leverage a similiar mechanism)?

Now this sounds like crazy talk; and perhaps it is. But it seems to be at least possible – in a nice hand waving kind of way.

Here’s how it ought to work:

  • There is one global name space for publishing artifacts
  • Artifacts have controlled scope of visibility (i.e. who can see them). For example, a corporation may choose to restrict its artifacts to a private group
  • Artifacts have a TTL. Example: snapshot  releases live for 30 days, and then are purged from the repository.
  • Artifacts have a replication scope. They might be global (of interest to everyone – for example the Apache tomcat jar files), group wide (e.g. just the Acme engineering organization),  or they might be local to a developers machine. While the mechanism for publishing would be the same in all cases- the degree to which artifacts get replicated to the global name space would depend on the replication scope.
  • Anyone can publish artifacts to the global repository . Anyone.  Whether or not those artifacts are replicated is another question.
  • Trust and identity are key to making this all work. The degree to which artifacts are replicated would depend on the level of trust that people place on the publisher.

Conceptually this should work in a fashion similar to dns. You point to an upstream provider that sends you updates and accepts your published artifacts (assuming your provider trusts you enough to accept updates to your chunk of the name space).

Above all, this system should be dead simple to use – especially for the 99% of developers who just want to consume artifacts. For most developers, this should be a one time configuration pointing their build management tool at the closest repository cache.

The implementation of the BARGE is trivial and is left as an exercise for the reader 🙂

Only the true repository would deny its divinity!

Only the true repository would deny it's divinity!