Matt Payne Consulting RSS feed

My New Bike

Mon, 30 Aug 2010 20:32:33 +0000

I've actually had it for a couple of months now, but this is my new Devinci Caribou 2.

Been riding it since early summer, and it's an amazing bike.

Just need to add some accessories to it.

Jambase4Net Released!

Thu, 04 Jun 2009 16:42:02 +0000

I've recently been working an libraries for various languages to access and make use of the Jambase API. Why? Well, for one thing, I love live music. For another, I believe that being able to create a mashup or something similar that incorporates live music is pretty useful. Jambase is a pretty unique service and the amount a data that it provides is staggering.

To date, I've released Jambase4R, a framework-agnostic Ruby library and Jambase4J, a Java library. Both confirm to similar interfaces and make accessing the Jambase API simple.

Today, I released Jambase4Net, which allows .NET developers to access the API. It's quite simple to use, as this Gist shows:

Hopefully that pretty straightforward. Have a look at the source for more details and feel free to fork it.

JamBase4R Released

Sat, 25 Apr 2009 20:01:30 +0000

I've just recently committed an update to an older plugin of mine - JamBase4Rails. It's now called JamBase4R and is not tied to any specific framework. It can be used with Rails or Sinatra and will include specific portions automatically, however.

What is JamBase4R?

It's a Ruby library designed for simple interaction with the JamBase online live music service API. It makes it ridiculously simple to integrate a search for live music into your Ruby application.

Configuring JamBase4R

Using JamBase4R In Your App

That's really all there is to it! Feel free to try it out or fork it and let me know what you think of it. As a side note, JamBase4Rails will be deprecated soon.

Embedding Code Snippets in Your Blog Using Gists

Sun, 19 Apr 2009 20:42:23 +0000

I'm not sure why, but I've never paid much attention to Gists. Gists are pasted code snippets hosted by GitHub. You can see them here. It's a service similar to PasteBin or one of the others, but it offers more, including versioning, branching, etc. To be honest, though, what I really like is the ability to embed a Gist into my blog, like so:

I really like this approach because it removes the requirement for syntax highlighting within the blog code. Additionally, when I update a Gist, that change is reflected on the blog.

Recently I've commited a new Ruby library called Gistr that allowed developers using Rails or Sinatra to embed Gists into their views. It's a really simple library and probably not of any great use to anyone, but I thought it was fun. It could easily be extended to add functionality to other Ruby web frameworks. Feel free if you find it useful.

The main point of this post is that maybe things like Google syntax highlighter are unecessary when you can just embed a Gist.

New Look

Sun, 01 Mar 2009 00:00:00 +0000

A few days ago, I posted about hating the colors and look of my site. Well, I finally sat down and made some changes. I think it looks cleaner and simpler now.

Creating Domain Specific Objects From JSON In Android

Sat, 14 Feb 2009 16:57:09 +0000

In my last post, I discussed issuing a GET request to a web API, downloading and converting the response to a string and creating JSONObjects from it. The next step is to create your domain specific objects from the JSON.

Here's an example:

The process is pretty simple. Loop over the JSONObjects and build up you domain specific objects. Then they can be freely used within your Android app.

Accessing RESTful APIs From An Android Application

Sat, 14 Feb 2009 16:23:02 +0000

I'm currently in the midst of writing my first Android application. Android is Google's new open source OS for mobile devices.

Just for fun, I'm writing an application called Complainatron. Complainatron is an application that allows mobile users to bitch randomly about the things that bother them. Users are able to view and vote on other peoples' complaints. Additionally, within the mobile application, users can see where various complaints originated via Google maps.

The application consists of 2 components. Firstly there is the Android application that runs on a user's mobile device. Second, there is the Complainatron API, a RESTful API build using Sinatra, a lightweight Ruby web framework. The web API returns all responses as JSON. This keeps the payload reasonably small.

The Complainatron client interacts with the web API to get lists of complaints and to submit new complaints.

How does the mobile client do this? Android is based on Java, so for the most part is should be a fairly famililar process: make a web request to the API and parse the results. Here's an example of a GET request:

Fairly straightforward, right?

Converting the response to a string for processing is nothing complicated. Just loop over the stream, reading line by line and build up a string.

The next step is to create objects from the response. I achieve this using the classes from the org.json package. These classes include JSONArray, JSONObject and others. Essentially you just need to create a new JSONArray with the response from the web API.

From there, it's simply a matter of looping over the JSONObjects within the JSONArray and creating your domain specific objects from each.

Ultimately, what I end up with is a collection of domain specific objects. In this case, Complaints. These can then be used within the Android Actions to display data to the user, be it in a list, table or map.

I'll write more on this as I go.

I Hate The Colors Of MySite

Sat, 14 Feb 2009 01:41:45 +0000

This is just a rant. I'm not a web designer by any stretch of the imagination. When I first built this site I thought that the orange-y/red looked ok. In truth, it's that color because that's the color of the tab images! I no longer like it, though. It must (and will) be changed!

Using CruiseControl.Net To Automate Project Builds, Testing and Deployment

Sat, 14 Feb 2009 01:32:59 +0000

It's been a while since I've posted. Things have been busy, but that's no excuse.

Today I wanted to discuss CruiseControl.Net and how useful it can be. What is CruiseControl.Net? It's a port of the Java based Continuous Integration framework CruiseControl. From now on, I'll refer to CruiseControl and leave it at that, since most of my points can be taken in the context of either. Lots of people use and have written about CruiseControl. This is just an account of my experiences. If you are looking for a technical article with lots of instructions, you should look elsewhere as I don't want to rehash what's already been extensively covered.

My scenario:

Recently I was working on a large, pre-existing .NET web application. It was reasonably well written, but was severely lacking in both unit and functional tests. Determining whether it would build properly or not was a manual exercise.

There were a couple of issues we wanted to solve:

Running unit tests
Running functional tests
Building and deploying of the application
Notification when things

We decided that unit tests and functional tests (using WatiN) were necessary. It would be foolish to try to write tests to cover the existing codebase, but going forward, we would write unit tests for new code. Additionally, we would write tests whenever we refactored the existing codebase. Functional testing would be handled slightly differently since we had a person who would be soley responsible for UI and functional testing. We knew that we wanted unit tests run whenever code was commited to source control. The functional tests would run nightly.

Another problem was that of build and deployment verification. Did the codebase build? Could you build and run the installer project?

Finally we wanted to be notified when there were problems.

CruiseControl helped us deal with these things and so far has been running very smoothly.

Setting up CruiseControl was fairly straightforward. After downloading and running the installer, I configured the Dasboard website that CruiseControl installs to allow developers easy remote access to the status of the build. There was one minor glitch in that the web.config file needed to be modified to run on Windows Server 2008 and IIS7. The details can be found here.

Next, I configured the CruiseControl server using the ccnet.config file. This consisted of defining the projects that CruiseControl would manage. Within each project you define:

The project's source control repository
A trigger (which tells CruiseControl when to rebuild the project)
Any number of tasks
1. Build the project
2. Run unit tests
3. Build the installer
4. Uninstall the previous version
5. Install the new version
6. Run WatiN tests against the newly installed application
Publishers - Email failure results, merge build and test files, etc.
There are other things as well. See here.

The last thing to do was to install the CCNet client, a desktop application that sits in the system tray and monitors the server for changes. It notifies you when a build fails, suceeds, etc. You can also use it to force a build or to navigate the the web dashboard.

On the whole, CruiseControl seems to really have helped the development process for that project. The development team has much more confidence in their product. They know that they are beginning to get a good amount of code coverage. They know that their product builds and installs correctly. When any of these are not true, the developers are quickly made aware and can fix the problem.

DRYing Up My Controllers

Wed, 31 Dec 2008 07:00:18 +0000

I'm working on a Rails project right now. It's been a while, since I've been doing most Ruby work with Sinatra lately.

Millions of posts have talked about DRYing up controllers. I'm not talking about the skinny controller/fat model concept, necessarily, although that should be addressed as well. What I'm refering to is the constant repetition in all controllers. A create action, for example, always has essentially the same code.

I'm not writing this post to claim that I've solved the problem in some unique, world changing way. I'm really just writing to acknowledge that I see it and have decided to deal with it. Repetition bugs me. Repeatedly spec'ing the same controller code bugs me.

The bottom line: I'm writing a module that can be included in my controllers that will remove the need to constantly add index, show, new, create, edit, update, destroy and the code that goes along with them. I will test it well. I will override in the controllers to handle special circumstances. My simple controllers should NOT need to be more that 50 lines long (if that).

I guess this is actually a New Year's resolution. So there!

Representing Data With Gruff Bar Charts

Thu, 06 Nov 2008 22:39:39 +0000

I'm currently working on Complainatron on the side. It's a simple but fun application that allows anonymous bitching and complaining. People like to do that, right? It also attempts to capture geographical information from users and does various things with it - including using the geographical data to create bar charts. For this task, I use Gruff (created by Geoffrey Grosenbach), a nice ruby wrapper around RMagick/ImageMagick.

Gruff's homepage explains how to install it, so I won't go into much detail. Suffice it to say that you need ImageMagick, RMagick, and, obviously, Gruff.

Creating a bar chart

I've got data that is essentially a collection of ComplaintCategory objects. Each complaint category has a category name and the number of submissions against that category. What I want to do in the case of this example, is to create a bar chart like that shown below.

Below is an example of how I could go about doing this (in the application, it's actually a lot more abstracted, but this works as an example).

Naturally, it's necessary to pass in options to let Gruff know how you want it to create the chart. As well, the data to chart needs to be present. Because I have a collection of ComplaintCategory objects, I can set the chart labels to an array the contains the category names. I can set the actual data for the chart to an array that contains the counts for each category.

On the whole, Gruff is easy to use and does a nice job. I wanted to post this information because although there are various examples using Gruff out there, there weren't too many discussing creating bar charts.

Hope someone finds it useful!

RDefensio A Framework Agnostic Ruby Library For Defensio

Wed, 29 Oct 2008 21:15:25 +0000

Yesterday, I posted about switching this blog over to Defensio. Today I'd like to look briefly at how I went about integrating it.

Defensio provides an API against which you can make HTTP posts. This is a pretty straightforward thing. I simply wrote a library that makes it a little less low-level.

The code is on GitHub, here.

The library is designed to be framework agnostic and to have as few dependencies as possible. The specs rely on RSpec, but that's about all that it requires.

Configuration

For any app, the first thing that needs to be done to use RDefensio is to configure it. You must have an API key from Defensio. This site is built with Sinatra, so I configure RDefensio within a Sinatra configure block:

The owner url is simply the url you registered with Defensio when you signed up. In my case, it's http://mattpayne.ca/blog

If you're using Rails, this could be done in an initializer.

That's all it takes to configure it.

Usage

Well, Defensio exposes several API methods that can be called. RDefensio mirrors them.

All of these methods return an OpenStruct object that represents the result of the post to Defensio.

The get_stats method is particularly useful if you want to display information about your Defensio interactions. It will return values for # of spam comments, # of false positives, # of false negatives and several other nice informational items

That's it for my post about RDefensio. It seems to be working well for me over the past few days.

Keep Your Post Comments Spam Free With Defensio

Wed, 29 Oct 2008 04:15:36 +0000

A few days ago, I talked about adding Akismet functionality to this blog on top of an existing captcha implementation. Ironically, there was a bug in the new code that prevented anyone from commenting! I guess that's one way to reduce comment spam.

The person that let me know about the problem was Carl Mercier, of Karabunga. Karabunga is a Canadian development shop near Montreal that has developed a spam filtering service called Defensio.

First and foremost, let me say that I don't get a lot of spam. I also don't get much traffic, so I guess that makes sense ;) Spam is still a concern to me though. I think it looks ugly and takes away from the real content.

I wasn't overly happy with my Akismet/captcha combination in the first place. The captcha is a bit of a hassle to read and many people dislike entering them. Plus, according to Carl, captchas are not terribly secure anymore as people are beginning to hire humans to solve them. I don't have any real issues with Akismet, but I wasn't entirely happy with my implementation of the two together. I decided that perhaps the entire commenting component of my blog should be rewritten - this time using Defensio.

Why Defensio? I'm not going to do a comparison with Akismet here; several folks have already done that here and here and, I'm sure, in many other places. Suffice it to say that I liked what I saw:

Defensio has the ability to learn. The more you work with it, the smarter it gets about you and what you believe to constitute spam.
It provides "spaminess" ratings for submitted comments.
It promotes transparency in terms of making people aware of how well you, as a blogger, are doing in combination with the Defensio service. I like that.
Defensio has a nice API.
Defensio is Canadian-made (have be a little patriotic after all).

There are a couple of things to keep in mind:

Defensio learns through your interactions with it, so initially it may not be too "smart".
There is a bit of comment maintenance to deal with since someone has to handle the ones that may or may not be spam.

Personally, I'd rather have more control than not enough.

It was a slow weekend, so I removed all of the captcha and Akismet functionality and replaced it with the new Defensio functionality.

As a result of this effort, I also created a framework agnostic Ruby library for accessing the Defensio API. I call it RDefensio. Why framework agnostic? I like framework agnostic code and I use Sinatra for this site. For the folks looking to incorporate Defensio into a big established framework like Rails or .NET, I suggest taking a look at Defensio's download page, here.

Anyway, that's it for now. Maybe a few comments would allow me to test the system ... ;)

Incorporating Akismet

Thu, 23 Oct 2008 15:25:48 +0000

I decided today that I wanted to incorporate Akismet functionality into this site. I do currently use Captchas as a way to ensure that only real comments get submitted, but Akismet seems to provide an extra layer of security. Plus, I was bored ;)

What is Akismet? According to the site:

"When a new comment, trackback, or pingback comes to your blog it is submitted to the Akismet web service which runs hundreds of tests on the comment and returns a thumbs up or thumbs down."

Pretty cool.

There are plenty of plugins and libraries for Akismet, but not one specifically for Sinatra, which is what this site is built on. I decided to use Ruby Akismet API, by David Czarnecki. It's a nice, simple, small library that does what I need. Basically, it posts data to the Akismet API and returns true if the comment has been evaluated as spam.

So my controller action looks more or less like this now:

  post "/blog/create/comment/:slug" do
    @post = Post.find_by_slug(params["slug"])
    raise_post_not_found(params["slug"]) unless @post
    @comment = Comment.new(params[:post])
    errors = @comment.validation_errors || []
    unless Captcha.valid?
      errors << "Invalid captcha. Please try again." 
    end
    unless AkismetWrapper.valid?(@comment, {
        :ip => user_ip, :referrer => user_referrer, :user_agent => user_agent}
      )
      errors << "Your comment appears to be spam. Please try again."
    end
    if errors.empty?
      @comment.save
      redirect "/blog"
    else
      @errors = errors.join("<br />")
      erb :new_comment
  end

I created a module called AkismetWrapper simply because I didn't want to use the Akismet API class directly from my controller code. Additionally, it'll help me with testing since I can just stub the method on the simple wrapper module.

So now I've got a dual layer of protection: Captchas and Akismet. Is it necessary? I don't know. Did I learn something from it? You bet! That's all I really set out to do anyway.

Multiple Unobstrusive Onload Events Using Prototype in Ruby on Rails

Mon, 06 Oct 2008 14:44:29 +0000

I'm currently working on a web application in Ruby on Rails that has a high degree of client-side scripting requirements. In almost every view, many things need to take place just after the page loads. I'm using Prototype, but any good javascript framework will do.

Since the application has a large amount of forms to be filled in by users, almost every view needs to set focus on a textbox in the form. On top of this requirement, almost every view has other javascript-related requirements when a particular view loads. For example, several of the views need to be able to set the form to its previous state when reloaded.

The typical way of registering an OnLoad event looks something like:

//application.js
  window.onload = function() {
    setFocus();
  }

  function setFocus()
  {
     focusElement = getFocusElement();
     if(focusElement && focusElement.focus)
     {
        focusElement.focus();
     }
  }

  function getFocusElement()
  {
     $$('input.focus').first();
  }

This sets up a callback that, when the window is loaded, locates an input element with a class of 'focus' and calls focus on it. This works fine. I put this in my application.js file and load it with every view.

Now what about running additional OnLoad actions for specific views? I don't want to code everything in application.js. That approach has several problems. Firstly, it means that application.js gets really bloated and if it loads with every view, there is additional overhead. Secondly, it means that you may be loading a bunch of code, required for other views, that is not needed for the current view. Thirdly, some of the OnLoad code may expect certain elements to be on the page, and they won't always be there. The bottom line: this is not a good approach. So what is the solution?

For me (and you may have other, better solutions), the approach that works best is a combination of application.js and content_for in my views.

Firstly, I ensure that in my layout I include an addtional named yield in the head section of the HTML. For example:

<%= yield :head %>

This gives me a place to include additional javascript files. In my view, I can do

  <% content_for :head do %>
     <%= javascript_include_tag "user" %>
  <% end %>

In this, case, I would be adding a javascript file called user.js (located in public/javascripts) to the head section of this view.

That solves one problem: that of only loading javascript code appropriate to the view currently being rendered.

There is only one more problem to get past: In my application.js, I am registering an OnLoad handler (as show above). In my user.js file, I also need to register an OnLoad event handler. As it sits right now, whichever file gets loaded last will clobber all the previous OnLoad handler registrations. How can this be addressed?

My solution (and, I'm sure, others') is as follows:

  //application.js
  var onLoadHandlers = $A();

  window.onload = function() {
    onLoadHandlers.reverse();
    onLoadHandlers.each(function(method) {
       method();
    })
  }
  
  function registerOnLoadHandler(method) {
     onLoadHandlers.push(method)
  }

  registerOnLoadHandler(focusHandler);

  function focusHandler() {
    //as shown previously
  }

If application.js is loaded with every view, we can effectively ensure that it's possible to call registerOnLoadHandler from any other javascript file loaded after it.

As a result, in my user.js file, I can do the following:

  //user.js
  registerPageLoadHandler(setViewState);
  registerPageLoadHandler(roleChangedHandler);

  function roleChangedHandler() {
    //code
  }

  function setViewState() {
    //code
  }

These functions will also be run when the page is loaded.

I like this technique and it seems to work well. It minimizes the amount of javascript code that I need to load on a per page basis and ensures that only javascript required for a particular view is loaded for that view. I also think that it makes it very easy to find javascript code related to a particular view.

Using Shoes To Write A Platform Independent Desktop Client

Sat, 27 Sep 2008 04:42:54 +0000

Lately I've been looking for ways to track my billable time. I stumbled upon Slim Timer a while back and the more I play with it, the more I like using it. Thanks to Richard White for making it available.

One small problem I have with it, though, is that I don't enjoy using a web interface for tracking my time. I don't know why, it just seems to break my workflow. According to the site, there is a Windows client called Bubbles that can be used; however, I run Linux, not Windows, so that option is out.

I had some free time, so I got to thinking about how I could easily write a desktop application that would run on a variety of platforms to interact with the Slim Timer API. My first thought was a Java Swing client, but that seemed inelegant.

Then I remembered Shoes. I had heard about Shoes a while back on Ruby Inside, but had not given it much thought.

On further investigation, though, it seemed like Shoes might be a good solution. A Shoes application is written using Ruby and can be compiled to run in Linux, Mac OS, and Windows. This is great in my opinion since I like Ruby and you never know when someone else on a different platform might be able to use the application

Another really cool feature of Shoes it that although you create desktop applications with it, in many ways it's a lot like creating a web application. Shoes has the concept of links that point to within (or without) the application. Shoes also has the concept of styles. Using styles is similar to styling HTML.

So far I've build a prototype of my Slim Timer On Shoes that works on Windows, Mac and Linux. It will likely take some time since I need to learn more about Shoes as I go!

Here's a very simple example of a Shoes application:

#SlimTimerOnShoes.rb
Shoes.setup do
  gem 'slimtimer4r'
end
$:.push(File.join(File.dirname(__FILE__), "lib"))

#loads a variety of additional helper classes - not shown
require 'loader'

class SlimTimerOnShoes < Shoes
 
  #'urls' within the application
  url '/', :index
  url '/task/new', :create_task
  url '/task/stop/(\\d+)', :stop_task
  url '/task/complete/(\\d+)', :complete_task
  url '/task/show/(\\d+)', :show_task
  
  @@actions = {
    :main => MainAction.new,
    :show => ShowAction.new,
    :configure => ConfigureAction.new,
    :complete => CompleteAction.new,
    :start => StartAction.new,
    :stop => StopAction.new,
    :create => CreateAction.new
  }
  
  def index
    Settings.configured? ? show_index : configure
  end
  
  def show_index
    run_action(:main)
  end
  
  def configure
    LoggerFacade.log("Application not configured. Bringing up configuration screen.", LoggerFacade::INFO)
    run_action(:configure)
  end
  
  def complete_task(task_id)
    run_action(:complete, task_id)
  end
  
  def start_task(task)
    run_action(:start, task)
  end
  
  def stop_task(task_id)
    run_action(:stop, task_id)
  end
  
  def create_task
    run_action(:create)
  end
  
  def show_task(task_id)
    run_action(:show, task_id)
  end
  
  private
  
  def run_action(action, *args)
    a = @@actions[action]
    a.app = self
    a.execute(*args)
  end
  
end

Shoes.app :title => "SlimTimer On Shoes", :width => 500, :height => 500, :resizable => true

named_scope

Sun, 21 Sep 2008 03:45:12 +0000

Lately everyone has been going mad over a relatively new addition to rails: named_scope. Up until now, I hadn't had the chance to try it out. I always just assumed that it was more syntactic fluff that seems to accumulate. Wow, was I wrong! It's an unbelievably cool and useful idea. Here's a really simple example. Say I have 2 models: Country and Region. They are as you would expect. A Region belongs to a Country and a Country has many Regions, etc. I use them pretty much for dropdown lists and things like that. Here's some code:

class Region < ActiveRecord::Base
  belongs_to :country
end

class Country < ActiveRecord::Base
  has_many :regions
end

Prior to named_scope, if you wanted to get all Regions in alphabetical order (as an example), you'd have to do something like:

class Region < ActiveRecord::Base
  #other code
  def self.ordered
    find(:all, :order => "name ASC")
  end
end

Which was ok, but this is nicer:

class Region < ActiveRecord::Base
  named_scope :ordered, :order => "name ASC"
  #other code
end

This creates a method on the Region class called ordered that can simply be called.

OK, so that's cool and all, but really not that big a deal, right? Well, no - until you consider that you can chain calls to named_scope, thus adding more and more constraints. For example:

class Region < ActiveRecord::Base
  belongs_to :country
  named_scope :ordered, :order => "name ASC"
  named_scope :by_country, lambda { |c| { :conditions => ["country_id = ?", c.id] } }
  named_scope :containing, lambda { |s| { :conditions => ["name like ?", "%#{s}%"] } }
end

class Runner
 c = Country.find_by_name("Canada")
 regions = Region.by_country(c).containing("A").ordered
 puts "Provinces in Canada containing the letter A in ascending order: #{regions.inspect}"
end

This generates SQL that looks like this:

SELECT * FROM `regions` WHERE ((name like '%A%') AND (country_id = 1)) ORDER BY name ASC

Another reason that I like name_scope is that because models such as these are frequently eused in dropdowns, etc, you can use named scope to return a very lightweight version of the class, while excluding attributes that are unnecessary for a simple dropdown. For example:

class Region < ActiveRecord::Base
  belongs_to :country
  named_scope :ordered, :order => "name ASC"
  named_scope :by_country, lambda { |c| { :conditions => ["country_id = ?", c.id] } }
  named_scope :containing, lambda { |s| { :conditions => ["name like ?", "%#{s}%"] } }
  named_scope :simple, :select => "id, name"
end

This allows me to only bring back the attributes that I need. And again, it could be chained with other named scopes

I think that this is a very powerful concept that allows developers to write more DRY and readable code.

Interesting Time Tracking site

Tue, 16 Sep 2008 03:50:53 +0000

SlimTimer looks like an interesting web application. Build using Ruby on Rails, it allows you to create tasks and time them. Then you can report on them, etc. Could be quite useful for the freelancer.

Searching Enabled!

Sat, 13 Sep 2008 21:14:12 +0000

As per my last post, I've decided to turn on fulltext indexing on post columns within MySQL. This seems like the lowest overhead solution and appears to work well.

Full Text Search

Fri, 12 Sep 2008 16:12:41 +0000

I've come up with my next chunk of functionality that I'd like to implement for this site - full text searching on blog posts. There appear to be several options:

UPDATED: I could also use MySQL fulltext indexes and just search those if I wanted to make it really easy ;)

I don't want anything complicated and I'd rather not add any big additional overhead to my already low on resources slice, so I'm leaning towards acts_as_indexed.

I realize that there'll be some work involved regardless since I'm not using ActiveRecord here and all of these plugins assume Rails.

Additionally, I'd like to be able to use/port the search solution within Django so that my Django mirror site works the same way.

Anyone out there have an opinion?

Importing Data From Sinatra App To Django App

Thu, 11 Sep 2008 14:39:35 +0000

Since I have recently ported this site to Django (see here), I realized that it would be nice to have my existing blog entries and comments ported over as well. I decided to basically just copy them from one database to another. While this would get the existing posts and comments, it would not move new ones over. That's fine. I just want some data in the Django version.

So my approach is relatively simple. Since both data models match (with the exception of table names), why not just use ActiveRecord? There are other ways, obviously, but I thought that this might be a nice exercise.

Here's the basic layout of my DataConversion project

DataConversion
  lib
    Source
      source_base.rb
      post.rb
      comment.rb
    Destination
      destination_base.rb
      post.rb
      comment.rb
    main.rb

Essentially, the concept here is to model each database seperately, but to allow them to interact through active record.

Here are the base classes:

#The destination base class
module Destination
  class DestinationBase < ActiveRecord::Base
    establish_connection(
      :adapter => "mysql",
      :username => "username",
      :password => "password",
      :database => "the_dest_db")
    self.abstract_class = true
  end
end

#The source base class
module Source
  class SourceBase < ActiveRecord::Base
    establish_connection(
      :adapter => "mysql",
      :username => "username",
      :password => "password",
      :database => "the_source_db")
    self.abstract_class = true
  end
end

Really all each does is set up it's own connection and declare itself abstract.

Here are the models:

#The destination models
module Destination
  class Post < DestinationBase
    self.table_name = "blog_post"
    has_many :comments
  end
end

module Destination
  class Comment < DestinationBase
    self.table_name = "blog_comment"
    belongs_to :post
  end
end

#The source models
module Source
  class Post < SourceBase
    has_many :comments
  end
end

module Source
  class Comment < SourceBase
    belongs_to :post
  end
end

Pretty straighforward, yes? The thing to note is that the Destination models specify a table name, since it is non-standard for ActiveRecord.

Now, here's the main.rb file

require 'rubygems'
require 'activerecord'
['source_base', 'post', 'comment'].each {|f| require File.join(File.dirname(__FILE__), "source", f)}
['destination_base', 'post', 'comment'].each {|f| require File.join(File.dirname(__FILE__), "destination", f)}

puts "Reading source posts ... "
puts "\n"

src = Source::Post.all

puts "Creating destination post from source post ... "
puts "\n"

src.each do |p|
  dest = Destination::Post.new
  p.attributes.select {|attr, value| attr != "id"}.each {|attr, value| eval("dest.#{attr}=p.#{attr}") }
  dest.save!
  p.comments.each do |c|
    dest_comment = Destination::Comment.new
    c.attributes.select {|attr, value| attr != "id" || attr != "post_id"}.each {|attr, value| eval("dest_comment.#{attr}=c.#{attr}")}
    dest_comment.post = dest
    dest_comment.save!
  end
end

puts "Done!"

So, in the main app, we load all of the source posts and iterate through them while creating new destination posts by copying their attribute values (and ignoring ids). We do the same with the comments for each, while also ignoring the source post id and making sure to assign it to the newly created destination post.

Django Version Up And Running!

Tue, 09 Sep 2008 21:11:25 +0000

I was interested in comparing the speed of Sinatra/Ruby against Python/Django. Not in any scientific way, just as side-by-side apps.

As a result, I've created a Django version of this site.

So far, they both feel about equally responsive.

Creating a tag cloud

Mon, 08 Sep 2008 19:37:25 +0000

I'm working with Python and Django at the moment (essentially just replicating this site) and I wanted to produce the same tag cloud that can be seen to the right of this blog. Basically, I want it to display all my tags, randomly, with a count of each tag and with the font size of the link controlled by the count. In other words, the more entries for a particular tag, the bigger that tag's font. Why? I don't know - I guess I just like the concept.

So the first thing to do is make sure that we can get a randomized list of all tags.

class Post(models.Model):
    #other code
    @classmethod
    def all_tags(self):
        container = {}
        for post in Post.objects.all():
            tag_array = post.tags.split(",")
            for tag in tag_array:
                tag = tag.strip()
                if container.has_key(tag):
                    container[tag] += 1
                else:
                    container[tag] = 1
         result = []
         for tag, count in container.iteritems():
             result.append(Tag(tag, count))
         random.shuffle(result)
         return result

So, that's great. Now I have a randomized list of Tag objects. Now, how do I display them?

Obviously, I need to iterate over the tags in my template and output them as links. That's pretty straight forward: create a template called tags.html. But how do I control the size of the link?

The answer is a custom filter. Here's the code:

from django import template
from django.utils.safestring import mark_safe
register = template.Library()

@register.filter(name='randomize_tag_size')
def randomize_tag_size(tag):
    map = {1:  "8px", 2: "12px", 3: "14px", 4:  "18px", 5: "20px", 6: "24px"}
    maximum = max(map.keys())
    minimum = min(map.keys())
    font = ""
    if tag.count >= maximum:
        font = map[maximum]
    elif tag.count <= minimum:
        font = map[minimum]
    else:
        font = map[tag.count]
    return mark_safe("style='font-size: %s'" % font)

Then, in your tags.html template, simply call {{ tag|randomize_tag_size }} within your link tag and you'll get a style

Rails Django Comparison Part 1

Mon, 08 Sep 2008 05:23:34 +0000

I've recently spent some time playing with Django. In a previous post, I mentioned that I'd post some thoughts on how Django compares with Ruby on Rails.

Both frameworks alllow for RAD style development. Each include tools that allow the developer to focus on writing code that is directly related to the domain and not to underlying infrastructure. Without a doubt, both frameworks provide a leg up.

Both frameworks follow a MVC approach. With Rails, this is very obvious - there are models, views and controllers. Django does not use exactly the same terminology, but ultimately also uses an MVC approach. Django has the concept of models. The slight deviation, however, comes with views and controllers. What Django terms views are actually (at least in my opinion) controllers (or methods of a controller). The Django equivalent to a Rails view is called a template. This is the place where the html goes.

In terms of models, both frameworks make use of the the Active Record pattern. Django models are slightly more verbose than Rails models, but also contain far less 'magic' due to the nature of the Python language. In some respects, this is good. It's certainly easier to understand a Django model than it is an Rails model - at least in cases where the model is complex.

There are definitely some differences, though. While Rails is an opinionated framework that highlights convention over configuration, Django is much more about choice. Many components of the Django stack are modular. Don't want to use Django models? Build your own (although you'll have to write your own data access layer as well). Don't want to use the built in Django template system? Use a different one. In some ways, I like the Rails approach. It's sort of nice to not have the choice. At the same time, in some cases it's nice to have alternatives.

The templating 'views' systems of each framework are also different, but not in the way you might think. Ultimately, both get translated into pure html. The main difference is that in ERB (in Rails), Ruby code is actually executed and any arbitrary Ruby code can be placed within the template. With Django, no Python code is present within a template. There are control structures called tags and transformers called filters that serve similar purposes. Not a big deal, but something to get used to.

Lastly, I noted that there is a big difference between the two frameworks where things like view helpers are considered. Rails has more than enough helpers (link_to, link_to_remote, etc), while Django lets the developer or designer use pure html. I'm not sure yet how I feel about not having helpers. In a way I like it. I feel like I have more control. I also think it makes it easier for designers if they are the folks writing the views. On the other hand, Rails' helpers sure are handy.

Anyway, that wraps it up for now. This is a pretty high-level comparison, I realize. I'll likely expand on it in the future.

Deciding to Work

Thu, 04 Sep 2008 05:38:01 +0000

It's been a little over 4 months since I began my 'sabbatical', and it's time to get back to work. During my time off, I did some fun stuff. I developed several Ruby on Rails plugins (see the projects page), I took part in a failed startup attempt and I got married. The getting married bit was definitely the best of the bunch.

Now, though, I find myself wondering what to do. There are plenty of traditional software development jobs here in Waterloo, but I wonder if I'm cut out for the traditional 9-5 thing. I've always wanted to do freelance work and be my own boss. This site is an indication of that. I'm just not sure yet that I can make a full time go of it.

Additionally, I've lately come to realize that I really like to work in languages like Ruby and Python. Unfortunately, most of the work out there is set in Java or .NET land. Not that I dislike either - they're fine and are sufficient for the work at hand. I just prefer to work in something a little more elegant.

So I guess the questions that I need to ask myself are:

Do I really want to be an employee again?
Am I motivated and driven enough to try freelancing?
Is there some happy medium in which I could do both?

Anyway, food for thought over the next few days.

Playing with Django

Tue, 02 Sep 2008 14:09:16 +0000

Yesterday I had a brief opportunity to play around with Django for the first time in a while. I'm proficient in Python, and I've done work previously with Django, but it's been a while. Lately I've been enamoured with Sinatra, a Ruby web framework that's much more lightweight than Rails. Django feels odd at the moment, maybe because I've been away for a while. In my next post, I'm planning to write up a bit of a comparison between Django and Ruby frameworks. I know - it's been done to death, but I feel like writing and am looking for an outlet.

Handling IE Differently

Sun, 31 Aug 2008 13:27:19 +0000

I noticed when viewing this site in IE 7 that the tabs at the top of the page were not properly aligned. They look good in Firefox (on Ubuntu) and Konqueror, but not IE 7. Big suprise, right? Anyway, the easiest thing to do is create a stylesheet specifically for IE that overrides the existing styles and is only loaded when the requesting browser is IE. The conditional code can be placed in the header section of the layout file. For example:

This seems simplest. If the browser is IE, the additional stylesheet will be loaded and the styles will be properly overriden. Is there an easier way to accomplish this?

Sluggable Posts

Mon, 11 Aug 2008 06:44:52 +0000

I was putting it off, but I decided tonight to implement slug urls in the blog. It's still in beta mode, so I feel its ok to go ahead and make the change.

So, now instead of seeing urls like: http://mattpayne.ca/blog/post/3, the urls will look like: http://mattpayne.ca/blog/post/this-is-a-post

I think it looks much nicer and also somewhat hides the data implementation. For all anyone knows, the posts could ve stored in an xml file on the server.

Counting Post Tags

Mon, 11 Aug 2008 05:16:05 +0000

I decided to adopt the simplest possible approach to recording blog entry tags. Each post has a string property called tags, which simply corresponds to a text field in the MySQL database. Each tag in the string is simply separated by a space. I'm the only one creating posts, so why bother with something more complicated? At some point I may need to breakdown and create a normalized data model for it, but for the time being I'm happy. In creating this site, I wanted to be able to create a tag cloud with each tag having a count of the number of posts it was applied to. This proved a fun challenge. Here's the code:

class Tag

  attr_reader :tag, :count

  def initialize(tag, count)
    @tag, @count = tag, count
  end

end

class Post < Base  

 #Other code ...

  def self.all_tags
    hash = self.all.inject({}) do |h, post|
      unless post.tags.blank?
        tags = post.tags.split(" ")
        tags.each {|t| h.key?(t) ? h[t] += 1 : h[t] = 1}
      end
      h
    end
    #shuffle just randomizes the array of tags
    hash.inject([]){|arr, (k,v)| arr << Tag.new(k, v); arr}.shuffle
  end

  #Other code ...

end

Overall, I'm happy with this approach. We'll see how well it scales.

Letting Google Know About The Site

Sun, 10 Aug 2008 16:56:29 +0000

I just went through most of the tools in Google's webmaster tools. Interesting stuff. I created at robots.txt file and placed it in the root of the site. I also created a sitemap.xml file and placed it in the site's root. According to Google, these are two of the most important pieces to have in place if you want your site indexed. Interesting stuff up there for webmasters - lots of analytical tools. Check it out.

Google Analytics

Sat, 09 Aug 2008 17:12:17 +0000

Originally on this site, I had implemented a kind of crappy hit tracking mechanism. Today after thinking about it, I thought - why am I bothering to code this when I can get a much better service from Google and for free as well. Check out Google Analytics if you haven't.

Google Syntax Highlighter - Nice!

Fri, 08 Aug 2008 02:19:21 +0000

I was looking for a way to highlight code in this blog. There appear to be a few ways to go about it. What I like most so far is Google's SyntaxHighlighter. Using nothing but CSS and Javascript, I can make code snippets look good.

For example

C#:

public class SomeClass : SomeBaseClass
{
     public string FirstName {get; set;}
     public string LastName {get; set;}

     public SomeClass(string firstname, string lastname)
     {
         this.FirstName = firstname;
         this.LastName = lastname;
     }

     public string GetFullName()
     {
         return String.Format("{0} {1}", this.FirstName, this.LastName);
     }
}

Python:

class Spam:
    def __init__(self):
        self._state = []

    def eggs(self, arg):
        self._state += arg
end

Ruby:

class SomeRandomClass

  def initialize(hash={})
    #do some initialization with the hash
  end

end

I think that this is pretty slick.

The other cool thing is that I can conditionally control when to include the css and js files. Say I have a collection of posts to display on this page. If none of the posts contains code, why include the additional files?

  #The /posts action
  get '/posts' do
   @posts = Post.all
   #not_empty? is a method I've added to Array/Hash
   @requires_highlighting = @posts.select {|p| p.contains_code?}.not_empty?
   erb :posts
  end

An then in the layout.erb file, I just check for the presence of the @requires_highlighting variable. Simple, but effective

  <% if @requires_highlighting %>
    <%= render_syntax %>
  <% end %>

Getting a Sinatra App Running with Passenger

Thu, 07 Aug 2008 05:35:12 +0000

So, I finally managed to get this site running properly. It's now using Apache and Passenger to serve. It's quite easy to get Apache and Passenger setup. Just google it and follow the instructions. More difficult, however, is the setting up of the rackup file since Sinatra runs under Rack. For posterity, here is my working config.ru for this site (with sensitve stuff changed):

 #I keep Sinatra in a vendor folder under the app root - like Rails
  vendor_path = File.expand_path(File.join(File.dirname(__FILE__), "vendor", "sinatra", "lib"))
local_path = File.expand_path(File.dirname(__FILE__))
$:.unshift(local_path) unless $:.include?(local_path)
$:.unshift(vendor_path) unless $:.include?(vendor_path)

require 'rubygems'
require 'sinatra'

Sinatra::Application.default_options.merge!(
  :run => false,
  :env => :production,
  :raise_errors => true,
  :sessions => true,
  :app_file => '/home/public_html/app/the_app.rb',
  :root => "/home/public_html/app",
  :views => "/home/public_html/app//views"
)

log = File.new("sinatra.log", "a")
STDOUT.reopen(log)
STDERR.reopen(log)

require '/home/public_html/app/the_app'
run Sinatra.application

This also allows for logging and enables sessions. The important thing to note is the setting of absolute paths for the application root and the view directory. This took me some time Googling, so maybe it will help someone in the future.

Specs

Wed, 06 Aug 2008 21:24:30 +0000

I know that I should be practicing BDD, and normally I do, but for this site I'm writing specs after the fact. BDD/TDD are awesome concepts and can be useful design tools, but they are somewhat useless when you are creating something without any upfront specs. Frankly, I started this site just by playing - to see what I liked and what I didn't. So now that it's at the barebones stage and I feel comfortable that what is here won't change much, I'm going to start writing specs. Going forward, as I think of new features that I'd like to add, I'll likely go at their creation from a BSS perspective.

Added Integration To GitHub

Wed, 06 Aug 2008 21:20:07 +0000

I use GitHub to manage my source code. The guys at GitHub have even provided an API so that various bits of data can be read programmatically. I used this API to read my list of repositories and provide links to them. See the right side of the main blog page.

Added A Tumblr Feed

Wed, 06 Aug 2008 04:00:41 +0000

I've written some code that allows me to pull my most recent Tumblr posts. These posts can be seen on the right hand side of the page.

Added An Rss Feed

Wed, 06 Aug 2008 03:56:38 +0000

I just finished adding an RSS feed for this blog. To add this blog to your feed reader, click on the RSS icon at the bottom of the page (in the footer).

mattpayne.ca minimally up and running!

Tue, 05 Aug 2008 16:24:50 +0000

So, I finally decided that it was time to get a domain and some hosting. This site is now hosted on a Slicehost slice (Ubuntu Hardy Heron).

Still have some things to do, however.

Setup Postfix and Dovecot so that I can have emails, etc.
Finished the site setup - I still need to get the site running via Apache.
Do some administrative work on the server.