Ring Benchmark - my first concurrent Erlang

Following the advice of Dave Thomas who said: “any developer should learn a new language every year” on RailsConf 2007 in Berlin, I decided that Erlang was going to be my challenge for 2008. However, the first months of 2008 passed without any Erlang until I listened to Joe Armstrong himself at Qcon in London. His talk was inspiring to me so right afterwards I bought his book and start reading it on my flight home.

After working myself through the first chapters, I finally got to the part that explained concurrent programming. In his book, he invites the reader to create an Erlang program that implements the Ring Benchmark. So I did and here is my version.

Problem:

Create N processes in a ring. Send a message round the ring M times so that a total of N * M messages get sent. Time how long this takes for different values of N and M.

Solution:

Download source

If you are interested in numbers, N=10.000, M=10.000 on my Mac Book Pro took 101.290 milliseconds (runtime). I might update this post later after doing some more tests with different parameter sets.

SelfDiagnose for Ruby on Rails

SelfDianogse is a library of tasks to diagnose a running system with respect to its dependent external resources. Recently, I did some Rails development and was thinking about how to implement it for the Ruby on Rails framework ? Obvious choice is to make a plugin that on installation adds the SelfDiagnoseController.rb to the application. Because of the scripting nature of Rails framework, putting the configuration in XML (as it is done for Java) is not the Ruby-way. So either use YAML or put the configuration directly into the controller. Let’s investigate the latter.

class SelfdiagnoseController < ActionController
  include SelfDiagnose

  tasks {
      check_database_connection 'mysql'
      check_directory_readable '/path/to/dir'
      check_log_writeable 'production.log'
      check_url_accessable 'http://s3browse.com'
  }
end

And the module would be defined as something similar to:

module SelfDiagnose
  def tasks
    yield if block_given?
  end
  def check_database_connection(db_name)
    # create a new CheckDatabaseConnection task and register it for running
  end
  def check_log_writeable(log_name)
    # create a new CheckLogWriteable task and register it for running
  end
end # module

Looking back at this approach, I don’t see the need for having a task registration in the Rails version of SelfDiagnose. I might as well implement the controller like this:

class SelfdiagnoseController < ActionController
  include SelfDiagnose

  def index
      check_database_connection 'mysql'
      check_directory_readable '/path/to/dir'
      check_log_writeable 'production.log'
      check_url_accessable 'http://s3browse.com'
  end
end

Then, the index.html.erb will put together a nice report of the results.

Sorting XMLListCollection by attribute

Just want to share the result of a small puzzle I had about sorting elements of an XMLListCollection by one of the attributes of such an element. At first, I tried sorting the elements of the XMLList I had, but the API does not provide any methods to do that. Bruce Phillips wrote a snippet that pointed me to right direction. So, here it is:

var doc:XML = <root>
<item name=”C”/>
<item name=”B”/>
<item name=”A”/>
</root>
var collection:XMLListCollection = doc.item
var sort:Sort = new Sort()
sort.fields = [ new SortField("@name", true) ]
collection.sort = sort
collection.refresh()

As expected, the collection will show:

<item name=”A”/>
<item name=”B”/>
<item name=”C”/>

The XMLHelper class in the Dunelox library has been extended with a sort function to encapsulate this.

External configuration for SelfDiagnose

Initialization of the SelfDiagnose component is done by reading the resource selfdiagnose.xml which must be located on the classpath of the Web application. This configuration declares the diagnostic tasks that are executed in the context of the Web application. The upcoming release (1.1) will have a new feature that allows you to override the local configuration and dynamically load the configuration from a provided URL:

http://your-app/selfdiagnose?config=http://some-other-server/selfdiagnose.xml

or even a local file can be passed:

http://your-app/selfdiagnose?config=file:///some-other-path/selfdiagnose.xml

You can restore to the application-defined tasks by:

http://your-app/selfdiagnose?reload=true

This feature allows for the following scenario in developing the configuration:

Create a local empty configuration (in e.g. /src/main/resources) such that the SelfDiagnose component is initialized upon starting the application. Now, create a copy of that configuration and store it on some local path. Next, open your favorite editor (TextMate will do just fine ) and start declaring diagnostic tasks. For each task entered, you ask SelfDiagnose to load the local configuration and run all its tasks. Immediately, you can verify whether the tasks are properly configured and you can see the results. Repeat this until you have check all relevant (=environment dependent) resources. Finally, you replace the empty configuration with your local, completed one.

With the “external configuration” feature you can request any active SelfDiagnose component to run tasks defined elsewhere. Although the standard diagnostic tasks will only inspect the state of your running application, make sure that any custom task will do the same.

Foreign compile-time dependencies in SelfDiagnose

In the upcoming release 1.1 of the SelfDiagnose task library, a new task has been added called CheckEndecaService. Endeca is commercial software that provides excellent Search services to e.g. e-commerce web applications. This particular task checks the availability of that service and can perform a simple query. To implement this, the following snippet has been used:

HttpENEConnection connection = new HttpENEConnection();
connection.setHostname(host);
connection.setPort(port);
UrlENEQuery eneQuery = new UrlENEQuery(”N=0″, “UTF-8″);
ENEQueryResults eneResults = connection.query(eneQuery);
Navigation navigation = eneResults.getNavigation();
long numERecs = navigation.getTotalNumERecs();
double time = eneResults.getTotalNetworkAndComputeTime();

Without a full understanding, you can see that this snippet is using classes which belong to the Endeca API package. SelfDiagnose is a open-source library and therefore can and will not have any dependencies on commerical software. So how can we still use this code without the compile time dependencies?. Of course, in order to execute this code, the runtime dependencies must be available. It does not make sense to use the CheckEndecaService if your application is not using it.

One way is to use some scripting language that dynamically binds the objects. But then the SelfDiagnose package will have a dependency to that scripting language. Therefore, I choose a more lightweight solution which is completely based on the standard Java Reflection API. To make the code a bit more fluent, I created the class Instance which hides all the exception handling stuff.


Instance connection = Instance.create(”com.endeca.navigation.HttpENEConnection”);
connection.invoke(”setHostname”, eneHost);
connection.invoke(”setPort”, enePort);
Instance eneQuery = Instance.create(”com.endeca.navigation.UrlENEQuery”,new Object[]{query,”UTF-8″});
Instance eneResults= connection.invoke(”N=0″,eneQuery);
Instance navigation = eneResults.invoke(”getNavigation”);
long numERecs = navigation.invoke(”getTotalNumERecs”).longValue();
double time = eneResults.invoke(”getTotalNetworkAndComputeTime”).doubleValue();

Although the compilation no longer depends on the client jar of Endeca, it still does depend on the API. Future changes in the API may invalidate this solution. Putting a version identification in the class name of task will accomodate this.

Amazon Web Services Test Drive Needed

Designing and implementing applications that make use of Amazon Web Services such as S3, SQS and SimpleDB, require extensive testing to ensure a robust integration. In addition to scenarios for the “happy flows”, test scenarios should be created to handle the critical exceptional situations. How does my application respond to failures in communication, checksum errors, service unavailability, never ending transactions and extremely frequent or heavy traffic?

In the spirit of test driven development, I see a need for writing up such scenarios and simulate the unexpected behavior of AWS just to verify the correct handling of each exceptional event. Such a Test Drive service would implement the API of services available at Amazon Web Services. It should provide extra control services to put a Web Service in some “testing mode” such as “simulate disk failure”, “simulate terminated http connection” or “simulate invalid access key”or a combination thereof.

To implement the persistent feature available in SQS, S3 and SimpleDB, a basic filesystem or relation database could be used to support test scenarios that involve more then one request. And of course all actions are properly logged and event information is made available to verify that a scenario is completed with success, i.e. the exceptions are handled correctly.

Finally, if I had such a Test Drive Service, I could startup multiple instances on different machines and run scenarios that involve combinations of AWS services SQS, S3 and SimpleDB. How much effort is needed to build such a Test Drive Service in e.g. Ruby on Rails? Isn’t this a common testing practice for all public service APIs out there on the Web?

Flex Project version and buildnumber

I wanted to associate version and build information with every compilation of a Flex project. Such information is useful when deploying to different staging environments (e.g. are you sure you uploaded the correct SWF ?) and helps tracking feedback (e.g. bugs) from users using a particular build.

First, I added a simple resource version.xml to store version and build info:

<?xml version='1.0' encoding='UTF-8'?>
<version name=’0.1′ build=’124′/>

Then I defined a component Version.mxml to show that information in plain text:

<?xml version="1.0" encoding="utf-8"?>
<mx:HBox xmlns:mx=”http://www.adobe.com/2006/mxml” horizontalGap=”0″>
<mx:XML id=”version” source=”/version.xml” />
<mx:Label text=”Version:”/>
<mx:Label text=”{version.@name}”/>
<mx:Label text=”Build:”/>
<mx:Label text=”{version.@build}”/>
</mx:HBox>

So far nothing to get exited about. What is left to do is get this information updated automatically for every Flex compilation (no I am not using Ant or Maven for Flex at this time).

I am using Eclipse for all my Java, Ruby and Flex projects. Eclipse uses Builders to fire all kinds of project tasks. So I added a custom builder (see Project Properties) that runs a simple Ruby script (as an External Program) that reads, changes and writes the version.xml resource. Make sure this builder is the first one to run.

#!/usr/bin/ruby

# This script updates the version.xml file by incrementing the build number

require 'rexml/document'
file = File.new(”version.xml”)
root = REXML::Document.new(file).root
root.attributes['build'] = root.attributes['build'].to_i + 1
File.open(”version.xml”,’w') do |f| f << root.parent end

Deploying Flex using Capistrano

Capistrano is a deployment tool initially created to support the remote installation of Rails applications. One of the assumptions Capistrano makes is that the application (source) can be pulled out of a source code management system such as Subversion.

In case of Adobe Flex applications, compiled sources (SWF,HTML,…) need to be deployed to the Web server. So instead of pulling sources from a repository, I needed to push compiled code to the server.

The script below overrides the update_code task of the standard recipe and defines the steps to secure copy the compiled sources to the release location. To make this also work on the Windows platform, you need to install pscp

Now, just as any other Rails application, you can say:

cap deploy

and the application will be deployed to a new release location on the Web server.

Get the source of this Capfile from my public S3Browse.com bucket.

Note: the version shown here will not work for deploying on multiple web servers. This is on my big TODO list…

QuickSilver and Current Application

On a Mac, accessing the current application from QuickSilver provides interesting new possibilities as can been on an episode of the Merlin Show. If you agree that this is something you wanted all along, then follow these instructions to set it up. (Thanks to BOONE).

• In QS prefs, make sure “Enable advanced features” is checked
• In QS Plug-ins, enable the User Interface plugin
• In MacOSX System Preferences–>Universal Access, check “Enable access for assitive devices”
• In QS prefs–>Catalog–>Quicksilver, check and reload “Proxy Objects”

Flex Bindable Hash

For the Dunelox library, I needed a simplified Flex version of the Spring ApplicationContext which is a generic Hash object containing model objects accessed by a key (String). Visual components that are wired to these models objects need to be updated automatically using the standard event mechanism in Flex.

So, I had a look at the standard ArrayCollection to find out how it notifies components that are wired to elements by index. This gave me the idea to implement the HashCollection as a extension to this class.

If you need to wire a value by key in a component such as:

<mx:TextInput text=”{hash.get(’somekey’)}” />

And you declare the variable hash as a HashCollection like this:

[Bindable]
public var hash:HashCollection = new HashCollection()

Then your component is updated if you change the value associated to that key:

hash.put(’somekey’, ‘changed the value of somekey’ )