I've come up with my next chunk of functionality that I'd like to implement for this site - full text searching on blog posts. There appear to be several options:

• Xapian and acts_as_xapian
• Solr and acts_as_solr
• Sphinx and UltraSphinx
• Ferret and acts_as_ferret
• acts_as_indexed

UPDATED: I could also use MySQL fulltext indexes and just search those if I wanted to make it really easy ;)

I don't want anything complicated and I'd rather not add any big additional overhead to my already low on resources slice, so I'm leaning towards acts_as_indexed.

I realize that there'll be some work involved regardless since I'm not using ActiveRecord here and all of these plugins assume Rails.

Additionally, I'd like to be able to use/port the search solution within Django so that my Django mirror site works the same way.

Anyone out there have an opinion?

Since I have recently ported this site to Django (see here), I realized that it would be nice to have my existing blog entries and comments ported over as well. I decided to basically just copy them from one database to another. While this would get the existing posts and comments, it would not move new ones over. That's fine. I just want some data in the Django version.

So my approach is relatively simple. Since both data models match (with the exception of table names), why not just use ActiveRecord? There are other ways, obviously, but I thought that this might be a nice exercise.

Here's the basic layout of my DataConversion project

DataConversion
  lib
    Source
      source_base.rb
      post.rb
      comment.rb
    Destination
      destination_base.rb
      post.rb
      comment.rb
    main.rb

Essentially, the concept here is to model each database seperately, but to allow them to interact through active record.

Here are the base classes:

#The destination base class
module Destination
  class DestinationBase < ActiveRecord::Base
    establish_connection(
      :adapter => "mysql",
      :username => "username",
      :password => "password",
      :database => "the_dest_db")
    self.abstract_class = true
  end
end

#The source base class
module Source
  class SourceBase < ActiveRecord::Base
    establish_connection(
      :adapter => "mysql",
      :username => "username",
      :password => "password",
      :database => "the_source_db")
    self.abstract_class = true
  end
end

Really all each does is set up it's own connection and declare itself abstract.

Here are the models:

#The destination models
module Destination
  class Post < DestinationBase
    self.table_name = "blog_post"
    has_many :comments
  end
end

module Destination
  class Comment < DestinationBase
    self.table_name = "blog_comment"
    belongs_to :post
  end
end

#The source models
module Source
  class Post < SourceBase
    has_many :comments
  end
end

module Source
  class Comment < SourceBase
    belongs_to :post
  end
end

Pretty straighforward, yes? The thing to note is that the Destination models specify a table name, since it is non-standard for ActiveRecord.

Now, here's the main.rb file

require 'rubygems'
require 'activerecord'
['source_base', 'post', 'comment'].each {|f| require File.join(File.dirname(__FILE__), "source", f)}
['destination_base', 'post', 'comment'].each {|f| require File.join(File.dirname(__FILE__), "destination", f)}

puts "Reading source posts ... "
puts "\n"

src = Source::Post.all

puts "Creating destination post from source post ... "
puts "\n"

src.each do |p|
  dest = Destination::Post.new
  p.attributes.select {|attr, value| attr != "id"}.each {|attr, value| eval("dest.#{attr}=p.#{attr}") }
  dest.save!
  p.comments.each do |c|
    dest_comment = Destination::Comment.new
    c.attributes.select {|attr, value| attr != "id" || attr != "post_id"}.each {|attr, value| eval("dest_comment.#{attr}=c.#{attr}")}
    dest_comment.post = dest
    dest_comment.save!
  end
end

puts "Done!"

So, in the main app, we load all of the source posts and iterate through them while creating new destination posts by copying their attribute values (and ignoring ids). We do the same with the comments for each, while also ignoring the source post id and making sure to assign it to the newly created destination post.

I was interested in comparing the speed of Sinatra/Ruby against Python/Django. Not in any scientific way, just as side-by-side apps.

As a result, I've created a Django version of this site.

So far, they both feel about equally responsive.

So, I finally managed to get this site running properly. It's now using Apache and Passenger to serve. It's quite easy to get Apache and Passenger setup. Just google it and follow the instructions. More difficult, however, is the setting up of the rackup file since Sinatra runs under Rack. For posterity, here is my working config.ru for this site (with sensitve stuff changed):

 #I keep Sinatra in a vendor folder under the app root - like Rails
  vendor_path = File.expand_path(File.join(File.dirname(__FILE__), "vendor", "sinatra", "lib"))
local_path = File.expand_path(File.dirname(__FILE__))
$:.unshift(local_path) unless $:.include?(local_path)
$:.unshift(vendor_path) unless $:.include?(vendor_path)

require 'rubygems'
require 'sinatra'

Sinatra::Application.default_options.merge!(
  :run => false,
  :env => :production,
  :raise_errors => true,
  :sessions => true,
  :app_file => '/home/public_html/app/the_app.rb',
  :root => "/home/public_html/app",
  :views => "/home/public_html/app//views"
)

log = File.new("sinatra.log", "a")
STDOUT.reopen(log)
STDERR.reopen(log)

require '/home/public_html/app/the_app'
run Sinatra.application

This also allows for logging and enables sessions. The important thing to note is the setting of absolute paths for the application root and the view directory. This took me some time Googling, so maybe it will help someone in the future.