Subscribe to the MPC blog rss feed feed-icon-14x14
 

Full Text Search

Posted on 04:12PM on 09/12/2008
Tags: search, rails, python, ruby, django, sinatra

I've come up with my next chunk of functionality that I'd like to implement for this site - full text searching on blog posts. There appear to be several options:

UPDATED: I could also use MySQL fulltext indexes and just search those if I wanted to make it really easy ;)

I don't want anything complicated and I'd rather not add any big additional overhead to my already low on resources slice, so I'm leaning towards acts_as_indexed.

I realize that there'll be some work involved regardless since I'm not using ActiveRecord here and all of these plugins assume Rails.

Additionally, I'd like to be able to use/port the search solution within Django so that my Django mirror site works the same way.

Anyone out there have an opinion?

0 Comments (Show) (Comments are closed for this post)

Importing Data From Sinatra App To Django App

Posted on 02:39PM on 09/11/2008
Tags: migration, django, sinatra, activerecord

Since I have recently ported this site to Django (see here), I realized that it would be nice to have my existing blog entries and comments ported over as well. I decided to basically just copy them from one database to another. While this would get the existing posts and comments, it would not move new ones over. That's fine. I just want some data in the Django version.

So my approach is relatively simple. Since both data models match (with the exception of table names), why not just use ActiveRecord? There are other ways, obviously, but I thought that this might be a nice exercise.

Here's the basic layout of my DataConversion project

DataConversion
  lib
    Source
      source_base.rb
      post.rb
      comment.rb
    Destination
      destination_base.rb
      post.rb
      comment.rb
    main.rb

Essentially, the concept here is to model each database seperately, but to allow them to interact through active record.

Here are the base classes:

#The destination base class
module Destination
  class DestinationBase < ActiveRecord::Base
    establish_connection(
      :adapter => "mysql",
      :username => "username",
      :password => "password",
      :database => "the_dest_db")
    self.abstract_class = true
  end
end

#The source base class
module Source
  class SourceBase < ActiveRecord::Base
    establish_connection(
      :adapter => "mysql",
      :username => "username",
      :password => "password",
      :database => "the_source_db")
    self.abstract_class = true
  end
end

Really all each does is set up it's own connection and declare itself abstract.

Here are the models:

#The destination models
module Destination
  class Post < DestinationBase
    self.table_name = "blog_post"
    has_many :comments
  end
end

module Destination
  class Comment < DestinationBase
    self.table_name = "blog_comment"
    belongs_to :post
  end
end

#The source models
module Source
  class Post < SourceBase
    has_many :comments
  end
end

module Source
  class Comment < SourceBase
    belongs_to :post
  end
end

Pretty straighforward, yes? The thing to note is that the Destination models specify a table name, since it is non-standard for ActiveRecord.

Now, here's the main.rb file

require 'rubygems'
require 'activerecord'
['source_base', 'post', 'comment'].each {|f| require File.join(File.dirname(__FILE__), "source", f)}
['destination_base', 'post', 'comment'].each {|f| require File.join(File.dirname(__FILE__), "destination", f)}

puts "Reading source posts ... "
puts "\n"

src = Source::Post.all

puts "Creating destination post from source post ... "
puts "\n"

src.each do |p|
  dest = Destination::Post.new
  p.attributes.select {|attr, value| attr != "id"}.each {|attr, value| eval("dest.#{attr}=p.#{attr}") }
  dest.save!
  p.comments.each do |c|
    dest_comment = Destination::Comment.new
    c.attributes.select {|attr, value| attr != "id" || attr != "post_id"}.each {|attr, value| eval("dest_comment.#{attr}=c.#{attr}")}
    dest_comment.post = dest
    dest_comment.save!
  end
end

puts "Done!"

So, in the main app, we load all of the source posts and iterate through them while creating new destination posts by copying their attribute values (and ignoring ids). We do the same with the comments for each, while also ignoring the source post id and making sure to assign it to the newly created destination post.

0 Comments (Show) (Comments are closed for this post)

Django Version Up And Running!

Posted on 09:11PM on 09/09/2008
Tags: sinatra, django, ruby, python

I was interested in comparing the speed of Sinatra/Ruby against Python/Django. Not in any scientific way, just as side-by-side apps.

As a result, I've created a Django version of this site.

So far, they both feel about equally responsive.

0 Comments (Show) (Comments are closed for this post)

Creating a tag cloud

Posted on 07:37PM on 09/08/2008
Tags: django, python

I'm working with Python and Django at the moment (essentially just replicating this site) and I wanted to produce the same tag cloud that can be seen to the right of this blog. Basically, I want it to display all my tags, randomly, with a count of each tag and with the font size of the link controlled by the count. In other words, the more entries for a particular tag, the bigger that tag's font. Why? I don't know - I guess I just like the concept.

So the first thing to do is make sure that we can get a randomized list of all tags.

class Post(models.Model):
    #other code
    @classmethod
    def all_tags(self):
        container = {}
        for post in Post.objects.all():
            tag_array = post.tags.split(",")
            for tag in tag_array:
                tag = tag.strip()
                if container.has_key(tag):
                    container[tag] += 1
                else:
                    container[tag] = 1
         result = []
         for tag, count in container.iteritems():
             result.append(Tag(tag, count))
         random.shuffle(result)
         return result

So, that's great. Now I have a randomized list of Tag objects. Now, how do I display them?

Obviously, I need to iterate over the tags in my template and output them as links. That's pretty straight forward: create a template called tags.html. But how do I control the size of the link?

The answer is a custom filter. Here's the code:

from django import template
from django.utils.safestring import mark_safe
register = template.Library()

@register.filter(name='randomize_tag_size')
def randomize_tag_size(tag):
    map = {1:  "8px", 2: "12px", 3: "14px", 4:  "18px", 5: "20px", 6: "24px"}
    maximum = max(map.keys())
    minimum = min(map.keys())
    font = ""
    if tag.count >= maximum:
        font = map[maximum]
    elif tag.count <= minimum:
        font = map[minimum]
    else:
        font = map[tag.count]
    return mark_safe("style='font-size: %s'" % font)

Then, in your tags.html template, simply call {{ tag|randomize_tag_size }} within your link tag and you'll get a style

0 Comments (Show) (Comments are closed for this post)

Rails Django Comparison Part 1

Posted on 05:23AM on 09/08/2008
Tags: rails, django

I've recently spent some time playing with Django. In a previous post, I mentioned that I'd post some thoughts on how Django compares with Ruby on Rails.

Both frameworks alllow for RAD style development. Each include tools that allow the developer to focus on writing code that is directly related to the domain and not to underlying infrastructure. Without a doubt, both frameworks provide a leg up.

Both frameworks follow a MVC approach. With Rails, this is very obvious - there are models, views and controllers. Django does not use exactly the same terminology, but ultimately also uses an MVC approach. Django has the concept of models. The slight deviation, however, comes with views and controllers. What Django terms views are actually (at least in my opinion) controllers (or methods of a controller). The Django equivalent to a Rails view is called a template. This is the place where the html goes.

In terms of models, both frameworks make use of the the Active Record pattern. Django models are slightly more verbose than Rails models, but also contain far less 'magic' due to the nature of the Python language. In some respects, this is good. It's certainly easier to understand a Django model than it is an Rails model - at least in cases where the model is complex.

There are definitely some differences, though. While Rails is an opinionated framework that highlights convention over configuration, Django is much more about choice. Many components of the Django stack are modular. Don't want to use Django models? Build your own (although you'll have to write your own data access layer as well). Don't want to use the built in Django template system? Use a different one. In some ways, I like the Rails approach. It's sort of nice to not have the choice. At the same time, in some cases it's nice to have alternatives. 

The templating 'views' systems of each framework are also different, but not in the way you might think. Ultimately, both get translated into pure html. The main difference is that in ERB (in Rails), Ruby code is actually executed and any arbitrary Ruby code can be placed within the template. With Django, no Python code is present within a template. There are control structures called tags and transformers called filters that serve similar purposes. Not a big deal, but something to get used to.

Lastly, I noted that there is a big difference between the two frameworks where things like view helpers are considered. Rails has more than enough helpers (link_to, link_to_remote, etc), while Django lets the developer or designer use pure html. I'm not sure yet how I feel about not having helpers. In a way I like it. I feel like I have more control. I also think it makes it easier for designers if they are the folks writing the views. On the other hand, Rails' helpers sure are handy.

Anyway, that wraps it up for now. This is a pretty high-level comparison, I realize. I'll likely expand on it in the future.

0 Comments (Show) (Comments are closed for this post)

More

1 2
Please note that I am currently unavailable for any large, long term work.