Tomash Corner

Integrating with Dropbox API for fun and profit

2012-09-04T00:00:00+02:00

Integrating with Dropbox API for fun and profit

04 Sep 2012 – Warsaw

TL;DR

Integrating with Dropbox API and using its one, relatively new feature, resulted in both improved user experience and reduced operation costs for my store with music. Includes technical details and code.

A Shameless Plug

If you follow me on Twitter (or any other social service), you probably know that in may I’ve launched my first startup, MusicRage.org which is about, basically, selling time-limited packages of independent music with strict time limit (a package is available to buy for two weeks), priced with pay-what-you-want model. Yes, it is heavily inspired by HumbleBundle , but for music.

You can see for yourself, as at the time of publishing this blog post we’re running fourth round of packages, with four different genres (rock, electronic, indie/alternative and comedy).

The problem with buying music on the internet…

… is not only the actual buying, but downloading and managing bought files. One usually needs to either download and decompress an archive file (usually zip) containing the whole album or, in worst-case-scenario, download every. Single. File. Separately. This is one of the things that Apple did properly with iTunes: combining buying, managing and listening of music in one, convenient application. But this is also something that a store nowhere close to iTunes market share is able to pull off.

When MusicRage launched, we’ve offered both “classical” ways of downloading bought files: either single files or a zip archive with whole album in desired format (be it WAV or MP3). But, for a whole package of four albums, that still means downloading (click, select destination, ‘save’) and unzipping (click, select destination, ‘extract’) four zipfiles. I’m not even touching the idea of downloading all files separately.

There should certainly be a more convenient way of getting bought music, maybe even more convenient than what every BitTorrent client does (download a whole folder of ready-to-play files). I agree with Gabe Newell that customers want to buy content and support creators, it’s just that the service usually sucks .

Enter Dropbox

I love Dropbox . They have solved the problem of backing up, synchronizing and collaborating on files in a way that’s painless for user and doesn’t require technical knowledge.

What’s even better, less than a year ago, Dropbox has launched their API that allows third-party applications to interact with user’s Dropbox, in a way that’s both secure (by default applications can operate only within a specified directory in user’s Dropbox account) and convenient (authorizing applications via OAuth).

Dropbox integration could do for MusicRage what iTunes does: have all the music files of a given album downloaded, backed up and ready-to-play with whole operation stripped down to single click of “add this album to my Dropbox”. Use a cool API that could also reduce our operations costs (see below)? There was simply no reason for me to refuse such idea.

How Dropbox could reduce our operations costs? We host our downloadable files on S3, which has many upsides (bandwidth, scalability, security) and one downside (it costs). If we could find a way to put the files on Dropbox once and then make them copy to other users’ accounts within Dropbox infrastructure, that could remove the need to upload file for every single user. Which saves time and bandwidth for everyone.

A short story of copy_ref

There was once a functionality of Dropbox that prevented multiple uploading of same file by different users: when the application has detected that a file of this size and hash is already on Dropbox, it would just copy it internally (probably make a hardlink, I have no idea about Dropbox’ internals) and therefore finish “upload” of even largest file in a split second.

This feature been disabled after concerns have been raised and some applications that exploit this feature have appeared . Other parts of it (uploading only changed parts, similar to how RSync works) are still there, but if we were to upload MusicRage files onto users’ Dropbox, for every file we’d still have to do at least one full upload for every single user. Not the effectiveness I’ve been hoping for.

I don’t want to initiate a Dropbox upload for every single file added-to-dropbox by every single MusicRage customer. It takes a noticeable time, so would require background job workers and, since uploaded by our servers, puts us at risk of hitting the limit of our upload bandwidth (it’s exactly the reason why I’ve decided to host downloadables on S3 in the first place). Might make the dropboxy experience actually worse than manual file download. No way.

But then, a few months ago, copy_ref has been introduced which is basically the same functionality, but implemented in a secure and non-exploitable way. It’s available only via API and application needs to know the exact reference hash of the file already stored on Dropbox to make a quick copy. Which is exactly what I was looking for!

Choose your poison gem

MusicRage is a Ruby On Rails application, so I began looking for decent gem that’d wrap Dropbox API in some nice Ruby objects.

Copy_ref has been introduced 6 months ago (as of writing this blog post, obviously), but official Dropbox gem still doesn’t have support for it. I’ve of course found that out after some experiments, registering MusicRage as a Dropbox application and basically producing some working code.

There’s an another Ruby gem for interacting with Dropbox API, but I’ve closed my browser tab after reading about authorizing by (de)serializing OAuth session object. That’s not how you do OAuth and I’m not going that way.

Finally I’ve settled for dropbox-api gem that’s not only supporting copy_ref part of Dropbox API, but also has more ruby-esque library design than the official one. I was positively surprised after finding out that it’s actually written and maintained by Marcin Bunsch , a guy I know (and admire).

Getting (finally) dirty

Enough fluff. I like sharing code and ideas, and I’m all out of ideas. Let’s get down to the actual Ruby code that lets a webapp like MusicRage use Dropbox API and its copy_ref part for great justice!

First, a standard setup of application key-secret pair in an initializer:

# config/initializers/dropbox.rb
# you might want to put the key and secret in a configuration file
# that's not versioned with application code
Dropbox::API::Config.app_key    = 'xxxyyyzzz'
Dropbox::API::Config.app_secret = 'aaabbbccc'
# 'sandbox' mode because designated app-exclusive directory is fine for us
Dropbox::API::Config.mode       = "sandbox"

Before user can request application to add files to his Dropbox, he needs authorize the application with Dropbox first:

# app/controllers/dropbox_controller.rb
def authorize
  consumer = Dropbox::API::OAuth.consumer(:authorize)
  request_token = consumer.get_request_token
  session[:request_token] = request_token.token
  session[:request_token_secret] = request_token.secret
  redirect_to request_token.authorize_url(:oauth_callback => order_authorized_callback_dropbox_url(@order.access_token))
end

def authorized_callback
  consumer = Dropbox::API::OAuth.consumer(:authorize)
  request_token = OAuth::RequestToken.new(consumer, session[:request_token], session[:request_token_secret])
  access_token = request_token.get_access_token(:oauth_verifier => params[:oauth_token])
  # MusicRage doesn't have 'Users', just Orders accessed with secure URLs
  # that's why we save key-secret pair on Order, not User basis
  @order.dropbox_access_key = access_token.token
  @order.dropbox_access_secret = access_token.secret
  @order.save
  redirect_to order_download_path(@order.access_token)
end

Of course due to copy_ref constraints, every single file should store its copy_ref after being uploaded to Dropbox. Here’s a part of “Dropbox seed” script that goes through newly-added albums, asks the Dropbox for given file and saves its copy_ref in the database:

# script/dropbox_seed.rb
# album_asset is an instance of ActiveRecord model
album.tracks.each do |album_asset|
  original_filename = find_original_file_name(album_asset.asset)
  df = $client.find(relative_file_path(album, original_filename))
  cr = df.copy_ref.copy_ref
  album_asset.update_attribute(:dropbox_copy_ref, cr)
end

Now that we have copy_refs and a user willing to get these files into his Dropbox, let’s try the simplest, most-straightforward way to use them (synchronously, therefore with an optimistic assumption, for now, that the time for all those copy_ref API calls will be before the request times out):

# app/controllers/dropbox_controller.rb
# that's some ugly code, a proof-of-concept
def add_album
  # Check if user has no dropbox session, redirect to authorize if so
  return redirect_to(:action => 'authorize') unless(@order.dropbox_access_key.present? && @order.dropbox_access_secret.present?)

  client = Dropbox::API::Client.new(:token => @order.dropbox_access_key, :secret => @order.dropbox_access_secret)

  album = @order.pack.albums.find(params[:album_id])
  count = 0
  album.tracks(params[:format]).each do |aa|
    if(aa.dropbox_copy_ref.present?)
      client.copy_from_copy_ref aa.dropbox_copy_ref, "#{album.dropbox_directory_name}/#{aa.asset_spaced_filename}"
      count += 1
    end
  end
  flash[:notice] = "Added #{count} files to your Dropbox"
  redirect_to order_download_path(@order.access_token)
end

And… here it is. The code presented here is ugly and from the feature-spike time, but proven to work exactly as expected. It’s the least-working code, so do yourself a favor and refactor it!

Conclusion

Adding this single feature results in everyone winning:

User: it improves user experience (getting album with a single click)
MusicRage: decreasing operation costs
Dropbox: engaging users to use Dropbox more (maybe upgrade to a paid account)

This might sound mighty fluffy, but I’m all in for situations where some work (in this case, a few lines of code) makes everyone involved win.

If you ever do an application that provides user with large or multitude of files, think about how many clicks it will require to use these files. Because maybe there’s a service your application could leverage to improve the experience.

Oh, and if you haven’t yet, check out MusicRage.org , there’s some really awesome music there.

A new laptop

2012-02-25T00:00:00+01:00

A new laptop

25 Feb 2012 – Warsaw

Requirements

My previous, worn but reliable (and still kicking) everyday laptop, Asus K80S, got over 3 years old this winter. It was getting old, not only in terms of performance (Core2Duo and 2GBs of RAM was enough of a spec for everyday programming work under Ubuntu), but also in terms of comfort: below 2 hours battery time started to be an issue recently, during long train or plane trips, or when doing some coding at the cafes.

I’ve decided to upgrade. I wanted my laptop light and thin: below 2 kg, without optical drive. I wanted a decent performance: at least i3, preferred i5. And, last but very important, I wanted at least 4-5 hours of battery time when working (text editor, rails server, occasional test run) in Linux.

After some research I’ve settled for Asus U36SD. Asus, because this brand has never failed me (despite around dozen Asus laptops being in the family since quite a few years) and laptop reliability studies confirm that. U36SD seemed to have exactly the configuration I needed, packing i5 CPU and 8 hours of battery time in a small, stylish, portable and light (1.5kg) chassis with 13.3" screen. As an added bonus, it came with dynamically-switched NVidia GeForce card and OEM Windows 7, for occasional gaming.

All under $1000 (for us, Europeans, the prices are a bit higher due to VAT), which is a steal and therefore a laptop I’m not afraid of losing (or having stolen etc.).

Experiment

I’ve heard mixed opinions about recent Ubuntu versions since switching to Unity, some people recommending going with Mint and others being happy with this interface overhaul.

Because I had some time for fun and experiments, I’ve decided to install Ubuntu 64bit (!) 12.04 alpha1 (!!) on a new (released in october 2011) laptop (!!!), thus introducing at least three reasons for everything to fail with a bang. But the nerd and hacker inside me wanted to have some fun, even if it meant blowing things up and starting from scratch.

To everyone’s surprise, Ubuntu 12.04 64bit LiveCD booted up and had everything working out of the box, including wifi, sound and webcam. Installed like a charm on some freshly-fred disk space and… that’s the whole experiment.

The only tinkering left for me was, based on official Ubuntu wiki for U36SD (albeit written for older versions), adding a script to control USB buses on suspend and installing bbswitch to completely disable Nvidia graphics card in Linux (to use only Intel HD3000 and extend battery life to almost 6 hours). Everything else Just Worked™, often even better and smoother than on my friends’ Macbooks. Ubuntu has come a long way since the early releases, in a direction I’m really happy with.

Note: 6 hours of actual work time in Linux, with either WiFi or 3G modem (Android phone).

Step Three

There’s no step three. I have an everyday laptop for work that I’m really happy with after almost a month of usage, not much to add here. Whenever somebody’s looking for a new laptop with Linux for programming work, I wholeheartedly recommend Asus U36SD + Ubuntu 12.04 combo.

Disclaimer: This post was in no way endorsed by Asus or its business partners. Although it’d be cool if it were.

Bitspudlo.com now powered by Spree

2011-07-01T00:00:00+02:00

Bitspudlo.com now powered by Spree

01 Jul 2011 – Warsaw

Originally at Spree-User google group

This is a recap and extension of post I’ve written on Spree group on 28th may:

https://groups.google.com/d/topic/spree-user/FWQxXPMyLG0/discussion

Migrating to Spree, part 1: The Code

2011-05-01T00:00:00+02:00

Migrating to Spree, part 1: The Code

01 May 2011 – Warsaw

(continued from previous post)

The Plan

… was pretty simple. First, export products and categories to JSON files, using some very simple rake tasks:

namespace :export do
  desc 'export categories'
  task :categories => [:environment] do
    puts Category.all(:conditions => {:available => true}).to_json(:methods => [:name_pl, :name_en, :description_pl, :description_en])
  end
  
  desc 'export products'
  task :products => [:environment] do
    puts Product.all(:conditions => {:available => true}).to_json(:methods => [:title_pl, :title_en, :description_pl, :description_en, :full_image_path])
  end
end

Second, check the differences in schema (column names etc.) and, while trying to preserve IDs (pretty crucial for seamless import), import these JSON files into Spree store.

The final working import code is (I’ve decided that english will be the basic language for database content, while Polish Zloty will remain as the base currency) ended up as another rake tasks, this time within the codebase of new, Spree-powered engine lib/tasks/import.rake

Execution

It worked. Of course the import code required some work to achieve the final working state it’s now in, but basically it was one of the smoothest migrations I ever worked on.

Outstanding Issues

I still have this itching urge to import orders as well, for all our customers to have a nice order history, but this is quite a task and I’ve left that in the backlog, “to be done when I’ll have both the time and will”.

What’s Next?

Of course migrating data isn’t the only thing that had to be done in order to have a fully working new Spree-powered Bitspudlo.com.

But all the other steps are basically documented in a post on Spree group I wrote after completing the successful migration .

Migrating to Spree, part 0: Introduction

2011-04-24T00:00:00+02:00

Migrating to Spree, part 0: Introduction

24 Apr 2011 – Warsaw

Nostalgia

When I’ve decided, back in 2007, to learn Rails seriously, I was looking for some “real” project to code while learning Rails. Just like DHH, I believe that trying some technology only makes sense if you do a real project with it.

And so a real project happened, when Krzysztof approached me with an actual plan to implement our common idea and start an e-commerce focused on selling bits from Games Workshop plastic sets. Long story short, one “Agile Web Development With Rails” (first edition — I do not recommend this book to anyone) and a few weeks laters, Bitspudlo.com was born.

There’re a lot of SVN (remember, it was the first half of 2007!) and Git commits documenting the development of this store engine. But I’ve refused to go with a ready-to-go e-commerce solution. First, I wanted to learn Rails on a real project. Second, back then all open-source e-commerce engines were written in PHP. In a very bad PHP (and some of them remain such today).

Hard Decisions

Flash forward four years and it’s april 2011. Bitspudlo, started on Rails 1.2.6, updated regularly and now powered by Rails 2.3.8, is the only application I’ve been working with during the last 7 months that’s still on Rails 2.×. Moving it to 3.0 would require quite a bit of tiresome work (especially with some of the gems and plugins used) and there are still some features in my backlog that either Krzysztof, I, or both of us would like to have implemented (especially now that our store has grown to a truly international one). In the meantime I’ve set up a few e-stores based on an awesome Rails-based e-commerce engine called Spree, most notably Topastic and Pulp City Store.

The decision has been made to migrate to Spree. I was ready to move to Spree. The thing is, Spree and its ecosystem of extensions wasn’t ready yet for us. Two key business requirements, namely multi-linguality (of both the views and database-stored content) and multi-currency, couldn’t be met unless I wrote all the code by myself. Which is kind of hard, considering the fact I have a full-time job and Bitspudlo.com is just a hobby project.

One Step At A Time

Being quite fluent with Globalize3 (actually being one of the maintainers of this awesome gem), back in november 2010 I’ve hacked the simplest working extension for globalizing names of products and categories (in Spree they’re called “taxons”, with a good reason behind that), basically gluing Spree and Globalize3 together. The simplicity of it made me love Ruby, Rails, Globalize and Spree even more. It was more like proof-of-concept, but good enought to consider Spree integrated with Globalize.

Around the same time (november 2010) when I was playing with making Spree globalizable, two talented programmers, namely pronix and parallel588 have started a Spree extension for multi-currency support, spree-multi-currency .

It was more than enough. There were no more excuses to avoid the migration.

TO BE CONTINUED

'Globalize is slow' meme must die

2010-11-08T00:00:00+01:00

‘Globalize is slow’ meme must die

08 Nov 2010 – Warsaw

Instead of introduction

It all started here, on Spree google group after I’ve released skeleton of extension allowing to integrate edge Spree with Globalize3

Some history of ‘Globalize is slow’

Globalize1 kept all the translations in one table and therefore used polymorphic associations (and, I don’t remember exactly, but possibly every single translated field had its own row). Which of course was slow as hell after the number of translated stuff grew considerably.

Additionally the problem of N+1 queries (for each product on page there was a query for getting its translated fields, each one slow already by itself) was hard to circumvent due to mixed philosophy of catch-all (single polymorphic table) and atomicity (each field in separate row).

There were of course attempts to hack it but the damage has been done and thus Globalize has become known as a serious performance hog.

Globalize done right

Globalize2 — and, by extension, Globalize3 which is basically “2” but rewritten for Rails3 compatibility allowing for cleaner code and API — used a different approach. Every translated model has it’s own “something_translations” table, consisting of columns with translated values (and, of course, locale for which it’s used). Which is not a very surprising design decision, provided both were developed mostly by Sven Fuchs, the author of the blog post linked above.

Now, theoretically the problem of N+1 queries is still here (every single item needs to query for a single row from product_translations), but it’s hell of a lot easier to prevent it now:

class Product 
  translates :name, :description 
  named_scope :with_translations, :include => :translations 
  # or even, for maximum comfort 
  default_scope :include => :translations 
end

I’ve been using Globalize2 for some time now in my store
and using the above solution doesn’t impair performance at all, the
app is super-fast.

Conclusion

Simple and short, as shown above — given the current state of Globalize, its “slowness” is just a thing of the past.

Diasappointment

2010-09-16T00:00:00+02:00

Diasappointment

16 Sep 2010 – Warsaw

The anticipation

I was having high hopes since the first day the Diaspora Project has been announced. I don’t like Facebook’s attitude towards privacy and I have soft spot for distributed networks in Jabber spirit. That’s why I was keeping track of their official blog, promoting the name and idea amongst all of my friends and waiting for the first code release. The fact it was going to be built in Ruby made me even more excited — I wanted to hack on the thing!

And so, a few hours ago, the developer preview release has been announced and the code put on GitHub.

WTFs per minute

So I’ve cloned the repo (mind you, it was a few hours and a lot of commits after the announcement) and started going through it.

And, fuck, I am disappointed.

Licensing

AGPL. Affero GPL. To those who haven’t heard of that before: it’s the GPL of GPL-based licenses. Basically viral exactly like GPL, but worse: any code using AGPL library or tool (not only linked, also merely using!) has to be AGPL licensed too. And also: web application has to have its full source code easily downloadable if the code is AGPL licensed.

Seriously, what the fuck?

This way you, dear Diaspora developers, have just cut off any commercial potential of Diaspora. No company is going to fork it and offer customized (or in any other way “premium”) version. Nobody is going to extract any solution (think: protocol implementation) from the code and release it, because basically AGPL makes a library or tool unusable for anyone.

As a homework you can try to think where Ruby On Rails would be today, had it been released under AGPL.

This is even more surprising considering the fact it’s been written in Ruby on Rails — an ecosystem where MIT license is the standard thing. You’re using a toolset and ecosystem grown on and thanks to stuff released on MIT license and suddenly shit in this environment with your ridiculous uber-viral and unusable AGPL?

Code quality

But hey, it’s Ruby on Rails project, and considering how good these programmers boasted to be, they for sure have a decent test suite that could be used in writing one’s own implementation of at least the protocol. A test suite covering all the code that’s so awesome that in fear of having it forked and not released they had to AGPL the shit out of it.

Libraries, gems and “gems”

I’m not going to rant about specifying Bundler version in the Gemfile — it’s dumb and redundant, but only since recently and maybe they just play safe.

But let’s go further into Gemfile.

Devise for authentication? Cool! Unfortunately not the original one from Jose Valim but some BadMinus fork that’s already three months behind the upstream.

And these two entries from Gemfile kind of speak for themselves:

gem 'mongo_mapper', '0.8.4', :git => 'http://github.com/jnunemaker/mongomapper.git'
gem 'jnunemaker-validatable', '1.8.4', :git => 'http://github.com/jnunemaker/validatable.git'

Using a released version of given gem while, at the same time, manually giving path to git repo (the official repo from which gem is released on rubygems.org)? Could someone explain that to me? I don’t consider “so that you can clone the whole github repo instead of using a gem already installed in your system gems” a rational answer.

Test coverage

There are some tests (rspec, to be precise). I haven’t read them thoroughly, but the test suite seems kind of sparse. A few tests are better than none, anyway. Still kind of weak for a statement “all commits must be tested” being placed on the first page of README file — how about you, dear Diaspora team, adhere to these rules first?

Focusing on what’s important

This commit says it all.

Data storage

Diaspora uses MongoDB as its primary data storage. Now I’m all hyped about MongoDB and absolutely love it’s approach to interfacing and storing data, but let’s get it straight: MongoDB doesn’t have any implementation of single-server durability as for now and countless flamewars have been burning recently on the internet about that. It’s been scheduled to be implemented though (I guess as a result of these flamewars: that was really some bad press which slowed MongoDB adoption), but still not there.

MongoDB documentation basically recommends using replication as a mean for durability. And this is a sound of thousand DBAs facepalming when hearing about such reliability model.

Now, I know social network isn’t a bank, but not having a single-server durability is a show-stopper for me. I’m not going to deploy Diaspora with two MongoDB servers — Diaspora was supposed to be distributed as in “you can run it for yourself and few friends on a inexpensive VPS”.

I’d prefer to have a SQL db as a primary data storage (tested, reliable, bulletproof and elastic technology) and MongoDB (if having to be used) as some sort of frontend-data/caching/denormalization layer (think wall post with all the comments and authors embedded etc.). Currently MongoDB just isn’t reliable enough for being a primary data storage for a serious service running with single DB server, period.

Protocol

There is no specification of the Diaspora Protocol — a basic thing required for any other diaspora-compliant social networks. While, as far as I remember, there were a few — including Facebook — wording their interest in being a diaspora-compliant application.

Now, I understand being Agile and shit, but not specifying the protocol that is supposed to be the killer-feature of Diaspora up-front?

Oh yes, I’ve tried to guess the specs from the code. Which made me stumble upon such “gems” like XML built with string concatenation:

def to_diaspora_xml
  xml = "<XML>"
  xml += "<post>#{self.to_xml.to_s}</post>"
  xml += "</XML>"
end

Seriously, it’s 2010. Ruby XML::Builder (or, at least, XML ERB templates) being a de-facto standard in the Ruby world since at least 2006.

This XML is also not XMPP. Which is even more disappointing.

Apologies

I now oficially apologise all the people I’ve been advertising Diaspora to. I guess I got my lesson on being excited with unreleased software.

Soothing balm

It’s a rant, but don’t get me wrong and don’t throw Diaspora away. I am not, as it’s still a very promising software.

It’s not that the Diaspora team hasn’t released anything: the guys have released a basically working proof-of-concept. All the technical issues mentioned above are solvable and the codebase has the potential for being great. Hell, I’ll gladly hack on Diaspora and help improve the codebase (I’m still excited by the idea).

Under one condition.

Change the goddamn license. Throw the brainfucked AGPL in favor of something usable like MIT.

Edit

The disenchantment is pretty wide on the internet, mostly about the batshit insane license choice — here’s one from Jarin Udom,

Rails 3 user notes

2010-08-06T00:00:00+02:00

Rails 3 user notes

06 Aug 2010 – Warsaw

The future is now

After some hacking, experimenting and micro-apps with Ruby 1.9 and Rails 3 betas I’ve decided to use this still-considered-bleeding-edge stack for a new project here at Aenima. There’s a lot of new stuff both on the surface and under the hood when switching from Ruby 1.8 + Rails 2.x, so I’ve obliged myself to take notes about new, suprising or otherwise worth mentioning (and memorizing) stuff. It’s not going to be a comprehensive tutorial, HOWTO or whatever-you-call it, but should have just enough information for competent Rails 2.x programmer who wants to give Rails 3 a try without nasty surprises.

Forget system Ruby, It’s the last call to install RVM

Trying to install Ruby 1.9 system-wide, especially on Debian-based systems (Ubuntu) and to live side-by-side with existing Ruby 1.8 is asking for trouble. If you do it, bad dark unicorns will step from oblivion and rip the guts out of your system with their horns.

So, RVM it is. It’s an awesome tool by Wayne E. Seguin, in case you lived under a stone for the last half year and haven’t tried it yet. As for the pitfalls, there’s only one for Ubuntu users: the piece warning about “no ‘&& return’ in .bashrc” is crucial because it’s basically about Ubuntu’s default .bashrc.

After installing rvm (and restarting terminal, making sure rvm is working etc.) all that’s left is

rvm install ruby-1.9.2-head
rvm use ruby 1.9.2-head
gem install rails --pre

I’m using Ruby 1.9.2-head here simply out of habit: I know there’s some rc out with stable on the horizon, but I’ve never had problems with the trunk version (since at least march) and it will give me all the “rc” and “stable” versions goodness on updating.

RVM, while solving most serious problems about developing a Rails 3 app under Ruby 1.9, introduces a few new issues (or pitfalls) on its own, so keep reading.

There’s no escape from Bundler

Bundler is the default and only way for Rails 3 to work with gems. It might be possible to force Rails 3 into working with gems without Bundler in the middle, but it’s a dumb (borderline brainfuck) idea and I’ve never considered it.

If you’ve never used Bundler before with some Rails 2.3 application, it’s better to learn this tool on the old stack that to make a full leap. Bundler introduces quite a shift in both philosophy and methodology (workflow), so it deserves some time dedicated just to learning its fundamentals. The good news is that it’s working like a charm now and I haven’t ran into single Bundler bug for a few months now (and thanks to ~~Drogomir~~ Strzałek I’m using Bundler on a daily basis since january).

So, port your existing Rails 2.3 app to use Bundler. Switching BitsPudlo to Bundler took me 30 minutes and I’m a pretty slow guy with new tools and with git branches there’s no risk involved.

I’ve been using older versions of Bundler (since 0.6 actually) and switching to 1.0 RC also required some learning: the commands and workflow changed a bit, mostly ones related to locking the gemset.

RVM + Bundler + Capistrano needs some love to work together

For starters, Capistrano should run bundle install during deploy:

namespace :deploy do
  desc "Run bundler command for installing gems"
  task :bundler, :roles => :app do
    run "cd #{current_path}; bundle install"
  end
end

after("deploy:update_code", "deploy:bundler")

Also, when there is RVM on the server, rvm use ruby-1.9.2-head should be ran on every shell login (otherwise someone can incidentally screw things). I prefer the former, as every application has its own shell account anyway so there’s no need for to use system ruby by default.

But that doesn’t solve all the Capistrano-RVM cooperation issues. RVM has a Capistrano plugin to make it work smoothly, so this should go into deployment recipe as well:

# taken from http://rvm.beginrescueend.com/integration/capistrano/
set :rvm_type, :user                      # we have RVM in home dir, not system-wide install
$:.unshift("#{ENV["HOME"]}/.rvm/lib")     # Add RVM's lib directory to the load path.
require "rvm/capistrano"                  # Load RVM's capistrano plugin.
set :rvm_ruby_string, 'ruby-1.9.2-head'   # Or whatever env you want it to run in.

I found myself in a situation when Bundler complained and threw an error while running bundle install during deploy. Putting this in deployment recipe fixed the problem

default_run_options[:pty] = true

Mongrel

gem install mongrel --pre

will install a version of (1.2 pre) Mongrel that compiles and works nicely with Ruby 1.9.2 and Rails 3.

Databases

For MySQL, mysql2 is the way to go — it’s Ruby 1.9 compatible and we might see some async performance improvements (thanks to Ilya) in that gem.

For PostgreSQL, last time I tried gem pg had some issues, but that was a long time ago.

Drogomir has been running Mongomapper with Ruby 1.9 and Rails 3 with lot of success, so MongoDB is also a viable choice.

A rule of thumb that applies to every gem, not only db-related ones: check it on IsItRuby1.9

Testing stack

All the sane testing tools have versions/forks that work with Ruby 1.9.2 and Rails 3 already. Because of this adding my favourite test stack only required googling which versions and/or forks should be chosen for compatibility.

So, to get a popular set of RSpec + Cucumber + Factory_girl + Capybara, here’s what should land in Gemfile:

gem 'rspec', '>= 2.0.0.beta.11'
gem 'rspec-rails', '>= 2.0.0.beta.11'
gem 'factory_girl'
gem 'factory_girl_rails'
gem 'capybara'
gem 'database_cleaner'
gem 'cucumber-rails', '>= 0.3.2'
gem 'cucumber', '>= 0.8'
gem 'launchy'    # So you can do Then show me the page

After usual bundle install, running

rails generate rspec:install
    rails generate cucumber:install

will put all the required stuff to work.

Moar

…will be here after further adventures with Rails 3 on Ruby 1.9.2

Headless Selenium+Firefox+Xvfb stack (on Gentoo CI server)

2010-06-12T00:00:00+02:00

Headless Selenium+Firefox+Xvfb stack (on Gentoo CI server)

12 Jun 2010 – Warsaw

The background

There’s this app we’re doing at Aenima that has a pretty large test suite. It’s nothing to brag about in the Ruby community, but here’s the problem: there are a few things covered with Selenium tests (that is, Cucumber tests ran in Selenium environment). Only the Javascript stuff that we cannot otherwise tests, as we’re not big fans of C{u|e}lerity. These Selenium-driven scenarios run great on our development boxes, but recently we have set up Integrity (actually Strzałek did it) on our internal development server for some Continuous Integration goodness.

Having Continuous Integration was a must: the test suite grew so large that a complete run (RSpec + Cucumber) takes from 20 to 40 minutes on our laptops, so it was natural that the frequency of running full test suite dropped considerably within the team, which of course increases the risk of regressions.

Anyway, the dev/CI server is of course headless, no display nor X, so for some time the Cucumber tests were ran with -selenium parameter, but we all knew it was a temporary workaround (I can’t even call that a “solution”). The CI server had to get a running Firefox-Selenium stack.

It was a natural choice to go with Xvfb, the only problem remaining was setting it up and building the rest of the stack on top of it. The man responsible for administration of the dev server, Imanel, has installed Gentoo on it (despite being a Gentoo-fanboy he’s a great guy). Something I can sympathize with, considering my (and not only mine) experience with Ubuntu Server. But still if something can be done in a simple way on most of the distros, it probably cannot be done simply in Gentoo.

Installing Xvfb on Gentoo

Xvfb is basically a part of Xorg system and installing it on a display-less server is something like “installing X without X”. For that I needed some ebuild and USE-flag magic. I’ve copied (with a slight version bump to avoid collision) the latest Xorg ebuild from portage directory that was considered stable on x86, xorg-server-1.7.6.ebuild

mkdir -p /usr/local/portage/x11-base/xorg-server/
cp /usr/portage/x11-base/xorg-server/xorg-server-1.7.6.ebuild /usr/local/portage/x11-base/xorg-server/xorg-server-1.7.6.1.ebuild

And here are two lines the ebuild I had to change (coming from this thread Gentoo Forums) in unified diff format:

--- /usr/portage/x11-base/xorg-server/xorg-server-1.7.6.ebuild  2010-06-04 16:07:04.000000000 +0200
+++ /usr/local/portage/x11-base/xorg-server/xorg-server-1.7.6.ebuild    2010-06-10 14:13:29.000000000 +0200
@@ -16,7 +16,8 @@
 DESCRIPTION="X.Org X servers"
 KEYWORDS="~alpha amd64 arm hppa ~ia64 ~mips ppc ppc64 ~sh ~sparc x86 ~x86-fbsd"

-IUSE_SERVERS="dmx kdrive xorg"
+# IUSE_SERVERS="dmx kdrive xorg"
+IUSE_SERVERS="dmx kdrive xorg xvfb"
 IUSE="${IUSE_SERVERS} tslib hal ipv6 minimal nptl sdl"
 RDEPEND="hal? ( sys-apps/hal )
        tslib? ( >=x11-libs/tslib-1.0 x11-proto/xcalibrateproto )
@@ -152,7 +153,7 @@
                $(use_enable kdrive)
                $(use_enable tslib)
                $(use_enable tslib xcalibrate)
-               $(use_enable !minimal xvfb)
+               $(use_enable xvfb)
                $(use_enable !minimal xnest)
                $(use_enable !minimal record)
                $(use_enable !minimal xfree86-utils)

And now installing Xvfb-exclusive X was as simple as running

USE="minimal xvfb -xorg" emerge xorg-server

Big thanks to Imanel for reviewing the ebuild, building the digest and making it click.

Running Cucumber-Selenium-Firefox trio under Xvfb

To run Xvfb and have all X-demanding applications run within it, issue these two commands:

nohup Xvfb -ac -screen scrn 1280x2000x24 :2.0 &
export DISPLAY=:2.0

And now everything that needs X — including Firefox — will run seamlessly. There are some scrolling/visibility issues with Selenium, that’s why I’ve allocated a larger vertical size for the Xvfb display.

And… that’s it! Withing the same screen session you can just run your Cucumber tests and see the selenium ones running and passing (preferably!). Xvfb can even let you make some screenshots of Firefox as the Selenium tests are running, but I haven’t set that up yet. I’ll probably be lazy and wait for Hubert to crack this nut.

To make it roll with Integrity, wrap the two lines above in a bash script (not ruby script, as the “&” operand won’t work) and have it ran right before the Cucumber suite itself.

Credit for the idea, inspiration and relevant bash lines goes to Hubert Łępicki, to be more specific — a blog post about setting great Rails test stack

Integrating it with multi-project Integrity

Every project added to Integrity should have its own instance of Xvfb, running on separate display, to avoid collision.

…rest is coming soon.

The Awesomeness of Unicorn

2010-01-30T00:00:00+01:00

The Awesomeness of Unicorn

30 Jan 2010 – Warsaw

This article is a stub. You can’t help, I’m going to make it a full-fledged blog post in the future. As for now — Unicorn ;) is Awesome

Unicorn’s killer features

…or simply advantages over other Rails application servers (Passenger, Mongrel etc.). This is a quite personal list of things that really made me go “wow” while and after tinkering with Unicorn and migrating three deployments (development, beta, production) of one application from Passenger to Unicorn.

So, here’re the things that make Unicorn awesome for me:

no matter the amount of instances, application code is loaded from disk only once, by “master” Unicorn process (the one that just passes requests and manages workers), worker processes get the memory copy using fork()
restarts of the application with zero downtime; simplifying things a bit, Unicorn basically serves old version of the application while the new one is loading and jumps to serving new one the moment it becomes available (it’s a bit more complex under the hood, but that’s what it looks like for the client)
master process manages all the workers by himself, taking care of killing them, spawning new ones or changing the number of workers caused differenly (see below)

whole management of Unicorn done via Unix signals

  kill -HUP2 `cat /tmp/unicorn.pid` # to restart the application
  kill -QUIT `cat /tmp/unicorn.pid` # graceful termination (workers can finish processing)

and these are just two examples from the top of my head (ok, from Capistrano scripts I wrote today).

incrementing and decrementing amount of workers is also done by simply sending signals (respectively TTIN and TTOU)
…and other stuff that Unicorn has and does because basically Unicorn is very Unix

More to come soon!

Rails Tricks, part 3a: Distance-based search (introduction)

2009-10-18T00:00:00+02:00

Rails Tricks, part 3a: Distance-based search (introduction)

18 Oct 2009 – Warsaw

They know where you are

Location data and maps are currently all over the internet (I don’t dare say “everywhere”). From the obvious Google Maps and it’s uses (“club I recommend to get hammered in”) to classified ad and auction sites (where you don’t necessarily see the map per se, but can use location-based search). And there’s going to be more of it, considering the growth of smartphones with built-in GPS and the following location-using apps.

It means, simply, that if you haven’t encountered the need to have location-based search in your webapp, chances are you’re going to, very soon.

I want to become “them”!

The whole location-based search stuff is non-trivial due to one reason: Earth is not flat, it’s a sphere (a spheroid to be precise), so we don’t have simple (x,y) coordinates that we could use to calculate distance using trivial pitagorean formula. Instead, location on Earth is determined using a pair of two numbers: latitude and longitude, which are basically a pair of angles.

It means that in order to calculate distance between two points, given their latitude, longitude pairs, we need to use quite fancy formula in order to at least pretend to be accurate.

But we want to do a search based on the results of this formula. And this one’s pretty complicated. And it’s definitely not something you’d like to repeat in every single SQL statement when searching for stuff in a-few-kilometer proximity of given (center) point.

Stored procedures are the shit

Basically if you don’t want to repeat something in every SQL statement, you put it in DB’s stored procedures to be called at any time desired using some simple syntax. This of course is a bit against the idea of “dumb” databases that DHH promotes, but we have to make a choice. I choose to look at the horrendous “great circle distance formula” only once, to write a function that’s going to calculate distances using it between any two given points.

We’re going to use stored procedures also because of the fact that every complete SQL RDBMS supports them — yes, even MySQL, got a procedures language called MySPL.

Teasers are wrong, I know

This post is actually a meta-post, because after this lenghty and scary introduction I’m going to show how to implement distance-based search in Rails (well, it’s actually going to be pretty useful for developers using any language and framework) using these database engines in posts about to follow (in the upcoming week):

PostgreSQL
MySQL
MongoDB

The list entries above will consequently change into links leading to article about implementing distance-based search using the given DB. Stay tuned!

And yes, you read it properly: while first two posts about to follow are going to be about well-known, popular and estabilished players in world of database servers for web application, the third one is going to be about implementing this non-trivial functionality using new kid on the block, a document-oriented database MongoDB gaining popularity amongst Ruby/Rails devs. I already fell in love with it after reading a lot and using a bit. And that’s not only me — the fact that on the upcoming RuPy conference in Poland there’re going to be two talks about MongoDB (with one from Obie Fernandez himself) speaks for itself.

Rails Tricks, part 2: Mislav's Will_Paginate

2009-10-09T00:00:00+02:00

Rails Tricks, part 2: Mislav’s Will_Paginate

09 Oct 2009 – Warsaw

The industry standard

If you ever had a chance to meet Mislav Marohnic (or see him on a video), for the first few minutes you were probably thinking how this crazy and hyperactive guy could have written one of the most popular Rails library. But after some time with Mislav it becomes obvious that he’s an insanely smart guy able to create some of the best ruby code I’ve ever seen.

All the pages, linked everywhere

The Problem (AKA “design”)

In one project (that’s not live yet) a designer has shown a pretty original approach to displaying pagination links and client was immediately convinced that’s the way to go.

The idea was: we present first and last pages as standard links, but between them — instead of the standard three-dot — we put a select with all the pages “between”, of course doing a javascript-powered redirect to the chosen page. Like this:

In terms of business requirements, if the page count is equal to or below 10, we show just the links to the pages. If there are more than 10 pages, we put 5 to the left, 5 to the right and all the excess pages are accessed via a select in the middle. The redirect occurs automatically on select. The current page, if it’s amongst the options in select, is displayed in bold.

The Approach

From the design it seems we need a constant amount of page links on both “sides” of the pagination helper. And (this is an important part of the approach) we need to generate the select in place of gap.

Dive In

That’s where mislav-will_paginate really shines. It can use custom renderers — classes (I put them in app/helpers directory) extending WillPaginate::LinkRenderer with a few methods to (re)implement: in this case we’re going to implement prepare (for assigning our custom gap) and visible_page_numbers (so we always have links on both sides to first and last pages and no links around the gap).

Let’s make it flexible enough so the number “5” (amount of page links to the left and right of select) isn’t hardcoded, just use well-known outer_window option.

In our application we had to reimplement also the to_html method to have some additional fancy stuff on the edges, but that’s not relevant.

Now I know it’s not a very preferred way of learning, but let’s see the code that does the trick:

class FancyRenderer < WillPaginate::LinkRenderer

  def prepare(collection, options, template)
    @gap_marker = options.delete(:gap)
    super
  end
  
  def visible_page_numbers
    inner_window, outer_window = @options[:inner_window].to_i, @options[:outer_window].to_i
    window_from = current_page - inner_window
    window_to = current_page + inner_window
    
    # adjust lower or upper limit if other is out of bounds
    if window_to > total_pages
      window_from -= window_to - total_pages
      window_to = total_pages
    end
    if window_from < 1
      window_to += 1 - window_from
      window_from = 1
      window_to = total_pages if window_to > total_pages
    end
    
    visible = (1..total_pages).to_a #start with all of them...
    
    left_gap = right_gap = (2 + outer_window)...(total_pages - outer_window)
    visible -= left_gap.to_a if left_gap.last - left_gap.first > 1
    visible -= right_gap.to_a if right_gap.last - right_gap.first > 1

    visible
  end
end

It’s pretty obvious from the code (don’t worry, it took me some time to write it) that mislav-will_paginate rendering uses @gap_marker variable and we need to call will_paginate helper with :gap option. If we don’t give it

will_paginate @items, 
    :inner_window => 1, 
    :outer_window => 4, 
    :renderer => 'FancyRenderer'

it looks like this:

So let’s see if it really displays our custom stuff in place of gap:

will_paginate @items, 
    :inner_window => 1, 
    :outer_window => 4, 
    :gap => "hello!", 
    :renderer => 'FancyRenderer'

result:

It’s time to write the code that’ll create the select for remaining “between” pages!

We’re not there yet

You know what I like most about Rails? That it’s written in Ruby. And in Ruby you can really do some fancy shit that’s readable and usable at the same time.

We’re going to write a “gap select” generator that’s going to create a required amount of relevant select options with paginated paths as values. But we want to make the code as elasic and reusable as possible (DRY, motherfucker! Do you follow it?), so calling the given resource path generator within our gap-generating helper is not an option.

So… how about something like this, sexy enough?

will_paginate @items, 
    :inner_window => 1, 
    :outer_window => 4, 
    :gap => page_selector_generator(@items, stripped_params) {|p| items_path(p)}, 
    :renderer => 'FancyRenderer'

The “stripped_params” are here of course for the sake of including all the other GET parameters (see “links for extra parameters” section in my post about Searchlogic), made by cloning params and removing key/value pairs that could interfere with path generator.

For instance:

stripped_params = params.clone 
  stripped_params.delete(:controller)
  stripped_params.delete(:action)

(You might want to put it in helper.)

So, we have sexy, DRY and cool code ready to reuse. Let’s write the page_selector_generator helper (finally!) to feed it with these stripped_params and path helper block.

module ApplicationHelper
  #accepts a block (one argument: page) with a path-generating helper
  def page_selector_generator(collection, extra={}, first = 'jump to', &block)
    arr = Array.new
    arr << [first,'']

    offset = collection.total_pages - 6

    (5..offset).each do |page|
      page += 1
      arr << [page, block.call(extra.merge({:page => page}))]
    end
    
    content_tag :select, :id => 'page_selector' do
      #the second argument is here for giving current page a "selected" attribute
      options_for_select(arr, block.call(extra.merge({:page => collection.current_page})))
    end
  end
end

And it looks this way:

So… voila!

(Yeah, I know, the “5” left-and-right offset is hardcoded, so I’ll probably refactor this code soon)

Almost there, i.e. proofing the awesome

The above code is great with one exception: if you supply some extra parameters, the path in select option’s value is not generated properly — the ampersands (&) get escaped and thus the paths from value don’t yield what would be expected of them.

We have to write our own variant of options_for_select:

module ApplicationHelper
  def not_escaped_options_for_select(container, selected = nil)
    options_for_select = container.inject([]) do |options, element|
      text, value = option_text_and_value(element)
      selected_attribute = ' selected="selected"' if option_value_selected?(value, selected)
      options << %(<option value="#{value.to_s}"#{selected_attribute}>#{html_escape(text.to_s)}</option>)
    end
    
    options_for_select.join("\n")
  end
end

The above code may be ugly (I never really digged html-generating helpers), but it does the trick: the paths are not entity-escaped and we can finally move onto javascript redirects.

I wanted to end it with “this remains as an exercise for the reader”, but I just got reminded how pissed I always was when encountering such crap: it’s a tech blog article, goddamit. People come here for answers, not for fucking riddles.

Of course we’ll write the javascript the unobtrusive way. Here’s the snippet if you’re using Prototype:

$('page_selector').observe('change', function(e) {
  location = 'http://' + window.location.host + $('page_selector').value;
});

I’ll leave the jQuery version as an exercise for the reader.

Searchlogic + Mislav_Will-Paginate = WIN

Old School

If you’re using Searchlogic 1.6.x, you can easily prepare instance variable in a way digestible for will_paginate helper:

search = Model.new_search(...) # searchlogic scope preparation 
# remember to also do the pagination, i.e. make use of :page and :per_page parameters
models_arr = search.find(:all) # array of elements for current page
@models = WillPaginate::Collection.new(search.page, search.per_page)
@models.replace(models_arr)
@models.total_entries = search.count

And you’re ready to use will_paginage(@models) in your views!

Searchlogic 2.x

New searchlogic exposes “all” method’s result as named scope (and thus works great in named scope chains), so it’s even easier to make it work with model methods added by mislav-will_paginate:

named_scopes = Model.whatever.named.scopes.you.use #named_scope
search = named_scopes.search(params)
@models = search.all.paginate(:page => params[:page], :per_page => 10)

(thanks to folks, especially gRuby, from polish RubyOnRails forum)

Conclusion

That’s all folks, at least for the Part 2 of Rails Tricks series.

I hope it succeeded in showing that you can do some really fancy stuff with mislav-will_paginate and exploiting the fact we’re writing Ruby.

Of course suggestions and comments are welcome (damn I learned a lot from previous’ part comments!).

Rails Tricks, part 1: Searchlogic

2009-10-04T00:00:00+02:00

Rails Tricks, part 1: Searchlogic

04 Oct 2009 – Warsaw

This is the first post on a series I plan to write about some clever usage of Rails-related tools (gems, plugins or other) that can really enhance programmer’s and app user’s experience, make code cleaner, and application neater.

Disclaimer In this article’s code examples I use Searchlogic 1.6 branch. I haven’t switched to 2.x with its syntax yet, so the code samples would require some re-writing, albeit the general ideas presented in this article remain valid.

Why Searchlogic is awesome

If you ever had the need to implement some combined searching / filtering / sorting capabilities in your Rails application, chances are you’ve stumbled upon Ben Johnson’s gem searchlogic.

What Searchlogic does great is not only its neat syntax for easy declaration of things that used to be pain in the ass with ActiveRecord.find conditions (think “less than X” and parameter escaping). Searchlogic promotes abstraction of AR.find parameters set as separate entity, a Search Scope — and enables easy conversions to-fro simple Hash. This way a given Scope can be easily saved (serialized), loaded, copied, modified etc.

But before we get into that awesomeness, let’s start by how searchlogic should and should not be used.

Search form: you’re doing it wrong!

Searchlogic’s readme gives an example of simple search/filtering form, submitting parameters that (supposedly) could be fed directly into scope constructor.

This could work and be a proper usage of searchlogic under one condition: it’s in admin panel (trusted user with all priviledges) and admin knows exactly how it works and how should the parameters be fed into it.

But generally it doesn’t work like users of your site would expect. Empty fields aren’t omitted in criteria, they evaluate to “is blank”-kind of conditions (and that’s counter-intuitive)*. If prices are stored as integers instead of decimals (a common design decision, see comments), they’d have to be input that way.
If there are any associations to be joined, there’s some tweaking to do. Plus a clever cracker with a tool as simple as firebug could forge extra fields into this form and get records he shouldn’t be allowed to.

—I seems to be working as expected in Searchlogic 2.x branch. Well, yet another reason to switch.

The proper way is to manually build a Scope, based on parameters with proper sanitization and conversion before they get passed to the Scope. I usually abstract Scope building to static method of related model and call it from controller’s private method:

class ItemsController < ApplicationController
  
  def index
    prepare_search
    @items = @scope.all
  end
  
  private
  def prepare_search
    #searchlogic scopes play well with named scopes!
    @scope = Item.active.prepare_search_scope(params)
  end
end

And the Model.prepare_search_scope(params) manually builds Scope by checking on a

class Item < ActiveRecord::Base
  def self.prepare_search_scopes(params = {})
    scope = self.new_search
    
    scope.conditions.title_like = params[:keyword].strip if params[:keyword]
    
    unless(params[:category_id].blank?)
      scope.conditions.category_id = params[:category_id]
    end
    
    if(params[:price_at_least].to_f>0)
      scope.conditions.price_at_least = params[:price_at_least].to_f*100
    end
    
    if(params[:price_at_most].to_f>0)
      scope.conditions.price_at_most = params[:price_at_most].to_f*100 
    end
    
    scope.order_as = "DESC"
    scope.order_by = :created_at
    
    return scope
  end
end

Take a look at this method’s body: it doesn’t depend on class it’s used in. It means that if you have a few models where scopes are built in similar way, it’s a no-brainer to extract this abstraction into module or superclass.

Links for extra parameters

Usually websites don’t expose all their searching/filtering capabilities as form. Usually there are some links, especially for changing category, ordering and other hard-to-input stuff. Creating links to the same page (action, controller, etc.) with the same extra parameters with just one (few) added or changed is extremely simple… just merge these new parameters!

link_to("category 2", params.merge({:category_id => 2}))

It exploits the fact that url_for (used with link_to’s second parameter) builds path from given hash and params hash is a magic one, containing also the controller, action and other extra stuff that’s been already input in the address bar.

Exploiting Scopes for nice browsing info

Many auction sites — like eBay and polish Allegro — provide users searching for some stuff with a nice side menu listing categories relevant to search (i.e. containing items that fulfill the search criteria) and amount of items in each of these categories.

This can be easily achieved using searchlogic by cloning our Scope (preservation of all the manually-built conditions!) and using some SQL-fu (adding grouping and overwriting SELECT part of our query to get just the count).

categories_scope = scope.clone
categories_scope.group = "category_id"
categories_scope.select= "category_id, count(category_id) AS category_count"
categories_scope.order = nil

Of course now we have two Scopes to return, so I personally prefer returning a hash like {:items => scope, :item_categories => categories_scope}.

And yes, of course the above code could use some joins for pulling the relevant categories and thus avoiding N+1 queries problem. Let’s leave that as an exercise for the reader (as for now, this article needs some tweaking anyway).

What more can I do with Scope?

Serialize it (like Hash) and store it in database for later use. For instance, notifying user when there’s some new stuff (just add a created_at condition) fulfilling given criteria defining his interests.

That’s not all, folks

Searchlogic (at least 1.6.x branch I’m using on a daily basis) does also pagination and can play very well with pagination standard in the Rails ecosystem, i.e. mislav’s will_paginate. But mislav-will_paginate tricks are going to be a subject of next post in “Rails Tricks” series.

Migrating to Rails 2.3 and Globalize2 from Rails 2.1 and Globalize1

2009-06-01T00:00:00+02:00

Migrating to Rails 2.3 and Globalize2 from Rails 2.1 and Globalize1

01 Jun 2009 – Warsaw

The great migration Exodus

Here’s the deal: my good old pet-project, Bitspudlo.com e-store engine (accidentally I also run this store, but that’s offtopic) is still running on Rails 2.1 due to Globalize1 being compatible with rails up to this version — with Globalize for Rails 2.1 being extremely hackish and half-official, if you consider Globalize website.

I didn’t feel well with that. So, since Globalize2 project started, I’ve been observing it almost since it was announced. For some time it still didn’t do the trick for me (despite being written really cleanly and without brutal hacks, thanks to Rails i18n API since 2.2), especially in terms of migrating data from Globalize1.

Last weekend was the day I said “enough” after seeing that Globalize2 matured pretty well and being fed up with my pet Rails project not being “edge” enough. Seriously, I don’t feel very well with hackish solutions and Globalize1 sometimes didn’t work as it should have.

There’s actually a pretty good guide on migrating to Rails 2.3 by Peter Marklund, so that was my starting point. The upside of my migration was that I was starting from 2.1 (alas with code written in the dark ages of 1.2.3 – 1.2.6), the downside was the huge migration step ahead of me in terms of migrating from Globalize1 to Globalize2.

So I started with a bang: change Rails version in config/environment.rb, remove vendor/plugins/globalize, install Globalize2 from Github and get dirty trying to get the app to work (thank git for branching!).

Model translations

The important things come first: model translations. Globalize1 was designed here in a way that texts in “default” language were kept in given model table itself and only translations (deviations) to non-default languages were kept in huge GlobalizeTranslation model with STI onto ModelTranslation. Now, Globalize2, on the other hand, didn’t use the original table at all — all the translatable text fields were kept in model_translations table, regardless of default language. And of course Globalize2 overrided the field and field= methods for translated fields.

My migrations looked exactly like in Globalize2 readme:

class CreateProductTranslationTable < ActiveRecord::Migration
  def self.up
    Product.create_translation_table! :title => :string, :description => :text
  end

  def self.down
    Product.drop_translation_table!
  end
end

Fortunately all the data for default language was still available through attributes hash, so this is the migration script I came up with after some fiddling: available as gist on Github

Feel free to play with this script (hey, it worked for me!), but remember to backup your data beforehand.

Then I had to sweep through controllers (actually only ApplicationController filters and one action responsible for setting up a different locale) to change from Globalize1 Locale to I18n.locale (and of course using now symbols instead of old string)

class ApplicationController < ActionController::Base
  before_filter :set_locale
  def set_locale
    default_locale = 'pl-PL'
    locale = params[:locale] || default_locale
    session[:locale] = locale
    I18n.locale = locale.to_sym
  end
end

(I’ve ommited the code for autodetecting locale based on browser settings / request header etc.)

Then I had to run through helpers and logic responsible for displaying prices in different currencies. I’m not going to paste code here as I feel it’s pretty hackish and still looking for a way to elegantly refactor it. Fortunately despite its hackishness it was still pretty quickly updated and started to work.

Small translations

Not much to say here: I got a chance for some pretty nice refactoring of code that (for instance) was sending e-mails with ActionMailer that had titles depending on language. Previously it was extremely fugly, checking for locale in if statement and setting e-mail’s @title depending on the locale. I made it use new and shiny Rails I18n API with some interpolation and that pretty much did the trick. I would fall in love with Rails I18n API if I hadn’t already while working with it at job for client projects.

Localizing templates / views

Now this got funny.

Globalize1 had a facility that let developer give the template a .en.html.erb extension and have it rendered for en-language locales with falling back to .html.erb for other locales or when .en. version was not found. Globalize2 focuses on extending Rails I18n API and providing nice and clean model translations, but ommited the issue of translating views.

Theoretically speaking it’s because Rails actually does exactly that, but with a twist: I was using both en-US and en-GB locales with Globalize1 to switch currencies based on region. And Rails localized-templates solution doesn’t support fallbacks to generalized locale, so it’s been looking for .en-US.html.erb template (for instance) and, when not found, falling back to .html.erb completely ignoring .en.html.erb one.

That’s partially because locale fallbacks are actually something introduced by Globalize2 and Rails doesn’t know anything about it.

That’s when I found Jose Valim’s Localized Templates plugin which proposed a pretty different solution (cleaner one: subdirectories for locales), but still didn’t have locale fallbacks. Jose turned out to be a very friendly guy and suggested I take a look at his plugin’s code to modify it, so I did and 15 minutes later I had a Localized Templates fork using Globalize2 locale fallbacks that did the trick for me.

The only thing left was moving the actual localized templates from views/path/template.en.html.erb to views/en/path/template.html.erb, but I wanted to do it anyway to have a good separated file structure.

If you want to have both fallbacks and the template.locale.html.erb naming scheme, have a look at this plugin: the code is really clean, nice and easy to hack with.

Is that all?

Some refactoring I did in the meantime (e.g. passing locale in path and relying on session only as fallback) fortunately made all the transition pretty easy anyway. I still have to throw out ActiveScaffold (it doesn’t play well with Haml as far as I see from backtrace) and replace it with normal, decent CRUD, but that was something I’ve been planning nonetheless.

Euruko 2009 reviewed

2009-05-11T00:00:00+02:00

Euruko 2009 reviewed

11 May 2009 – Barcelona / Plane home

Euruko 2009 in Barcelona has just ended and I’m going home. It’s been only a few days, and Euruko 2009 in Barcelona were probably some of my best days this year, but I’m already starting to miss home. Guess it’s a good time to summarize and review the last days on the best Ruby conference under the Sun.

To start with: the initial doubts about spanish Ruby community and how they’ll cope with organising Euruko turned out to be worthless. The Euruko 2009 team did a great job, hands down. If I had to be on one Ruby-related event this whole year, I wouldn’t need even 3 seconds to decide.

Arrival

Together with my girlfriend and my mother (they both always wanted to visit Barcelona) we arrived at the airport on friday evening, having the time and strength only to reach our hotel and go to sleep. It turned out I’ve booked a room in Campanile Hotel that’s on the other side of Barcelona to where the Euruko was held — I heard about Campanile hotel being the hotel of choice for many attendees (and organizers aswell, as most of them were from Madrid), but haven’t taken into account the possibility of having two Campanile hotels in Barcelona (and of course choosing the “wrong” one). Well, unlucky (or: dumb) me. I still had a single fast train from near the hotel to Cornella (near the Euruko venue), so that wasn’t that of a problem.

From now on I’m going to focus on the talks and conference itself. Really, I could probably write a book about all the people I met and all the talks we had during this Euruko, but that’s not the point.

Saturday — Day One

The venue was great, sandwiches in catering were good, weather was wonderful and everyone was in good moods — it was time to start the Euruko!

1) Matz’ Keynote

This time Matz gave a review of evolution of computers and their power with software and its complexity, what it meant for programmers and how it related to dynamic languaged like Ruby, Python or Javascript. There were a few funny moments, but I definitely liked Matz’ keynote from the last year more. Fortunately the following Q&A session made attending the keyone really worth it (and Matz is a uber-great guy to talk with, while being our god at the same time).

2) Ruby In Systems Testing

Great topic, well-thought and prepared presentation, but Stephen wasn’t at the peak of his performance. Probably that was just an early-speaker stress, as he was cool guy to talk with a few hours later. “Weight” of the topic (serious QA and enterprise stuff) and Stephen’s not-the-best performance marked two talks in a row that were a bit below expectations and some people were actually worried about the following ones.

That lasted for about 10 minutes, until Javier Ramirez started his

3) Having fun with Ruby using Gosu

And that was definitely the best talk of the whole Euruko (oops, spoiled the review!). Really, why say anything else? Great from a technical standpoint and Javier is a simply perfect speaker. Just wait for the videos from Euruko to pop up. Half the tweets following this talk were like “OMG I’m so going to hack some game in Gosu ASAP” and that speaks for itself.

4) Creative Image Manipulation using Ruby

I had to take a break, so will have to see this one on the videos.

5) Cooking with Chef

Good speaker + good topic + good presentation (movie FTW!) == really good and inspiring talk. Joshua (from 37signals) showed how to use Chef for offloading some chores related to project bootstrapping, deployment and managing installation in general. Will try Chef soon!

Realizing the possibility of getting audio + video on the talk made me change my talk a bit, to incorporate some new media.

6) Software Craftsmanship

Didn’t attent this one — I was a bit tired and that combined with “fuck fluff, give me crunch” attitude made me spend this talk on grass outside, having some great talks. I’ll gladly watch it on video though.

7) Lightning Talks

There were a few really worth seeing ones (actually some guys claimed that Lightnings were the best part of Euruko). I really enjoyed Bragi’s about parsing and evaluation (and Treetop), Dira’s about vimmish and creating grammars. I actually decided to give one myself about Jekyll, so that my mate Peter could switch to a decent blogging engine, but he didn’t come to the Lightning Talks finally (you owe me one, man).

Amongst the things worth highlighting this are definitely lunch at the shopping mall and discussions on the grass outside the building, including Mislav showing some parkour on the venue after being asked about it.

I missed the saturday party, as the whole excitement tired me (plus it’d be quite hard to return to my hotel at nigh other way than taxi) and evening with my significant other was a much more tempting perspective.

Sunday — Day Two

I missed “Ruby DataObjects” talk because of some train confusion (stepped into wrong train, realized that about two stations after forking, rebase took some time and I’m an hour late) and I miss it. Waiting for the videos!

2) Adhearsion

That one was funny for me. is a very good speaker and delivered a definitely encouraging talk, but I haven’t joined the crowd tweeting “I’m going to do some Adhearsion”. Despite project’s and talk’s high level, I’m still put off trying due to my hatred towards calling and answering machines. Paul Klipp wouldn’t have closed his deals on second call max if he had to do them with some phone-bot ;)

3) Crossplatform Mobile Apps in Ruby using Rhodes

Wow, this one was so great everyone felt sorry for Adam not having the time to show a live demo of Rhohub, a service his company created to aid developers in development and building of Rhodes applications. Tweeter was getting hot from tweets like “I have to do some Rhodes app” during and after this talk. Now if we could only see a live demo… read further.

4) Archaeopteryx

I’m a musical geek, there were some girls in bikini on one slide… what more can I say? This is a must-hack for me! Oh yeah, and Pablo is a very good speaker. I’ve been thinking about hacking some midi with Ruby and that was something just for me.

5) Quality code with Cucumber by Aslan

I really wanted to see Cucumber well-presented, as a fellow developer in Aenima has started using it a few weeks ago and is all hyped (I’m oldschool, staying with Test::Unit and Shoulda), while I didn’t have time nor motivation (RuDy!) to investigate. And Aslan delivered. He stretched his talk (to more than 1 hour) and there wasn’t a simple complaint, it was that good. I’m definitely going to write Cucumber tests in one of projects to follow.

6) my talk on RuDy

I think it was better than my last year’s, from both technical (I think RuDy is a cool project, but hey — author’s bias!) and “speaker” standpoint (people were laughing sometimes, right?). I’m not the one to judge, though, so you should check other reviews. If you’re looking for materials (slides, links, movies even) they’re here .

7) Rhodes / Rhohub demo

Because there was 1h15m window for my talk (+break) and I’ve already told the organizers that I’m going to fit in a 30-minute window (as I was asked to when my talk was accepted), the organizers made a (great!) decision to squeeze in between the Rhodes/Rhohub demo that didn’t fit with Adam’s talk on Rhodes. It turned out it was a perfect idea!

Now, I was so hyped after my RuDy talk that I spent first few minutes in the first row chatting with a programmer from Poland and that interfered with the talk. Accept my apologies, Adam — I really wasn’t aware that we were screwing your efforts. After I’ve realized that we’re messing up with ‘s talk we both shut up and watched the demo — and damn it was worth it! "I’m so going to hack some mobile app using Rhodes" was the preminent theme of attendees’ tweets once again.

8) Lightning Talks 2

Two moment that got my attention: Mislav had a cool topic (Rspactor for automated test creation) but didn’t fit in the 5-minute window, Paul Klipp showed no slides but managed to give an inspiring talk in less than 3 minutes about selling software and Agile philosophy (wow, he really deserves the AgileActivist title!).

Discussion about next year’s venue.

“There were rumors about Poland/Krakow already being decided as the place for Euruko 2010. These are not true, we are going to decide as the Euruko community. So, are there any other countries/cities besides Poland/Krakow? [silence] Guess not. It’s Poland then”.

Fuck yeah!.

I just hope we’re going to live up to the expectations (Euruko 2008 and 2009 raised the bar really, really high) and maybe even exceed them. See you people next year in Krakow!

Congratulations once again to Spanish RUG for giving the european Ruby community such a great Euruko. You rock, fellows!

That’s all for now, folks. I’m waiting for the videos.

For people interested in Ruby beyond what’s on Github, here’s how it started, here are my thoughts about RuDy the day before Euruko and, of course, here are the resources from the talk.

RuDy on Euruko: resources for talk

2009-05-09T00:00:00+02:00

RuDy on Euruko: resources for talk

09 May 2009 – Warsaw

Perform like a Rock Star, be Metal!

The talk itself: RuDy.odp, RuDy.pdf

RuDy@github — clone and see examples/ directory. The README has lots and lots of useful info aswell.

Images of bands courtesy of Google Image Search ;)

Video fragments come from Fat Ed’s Guide to Metal, a promo video for Fur TV.

RuDy on Euruko: last thoughts before the takeoff

2009-05-08T00:00:00+02:00

RuDy on Euruko: last thoughts before the takeoff

08 May 2009 – Warsaw

A bit of history

Over two months ago I’ve put on GitHub repository with first commits for RuDy, exactly on the day when my first very basic Ruby extension written in D worked. That was a culmination of few previous months gathering some data, emailing with Kirk McDonald (author of PyD) about challenges to come and features to implement, converting .h files to .d and generally pushing towards this first successful run.

This first successful (compiling and running) extension was the first milestone of RuDy and a sign that this indeed might work; The day after setting up a github project I’ve submitted a talk topic on this year’s EuRuKo to tell about and show the RuDy project, and to and advertise it amongst fellow rubyists — a library without an application using it is a dead library.

Shortly after the blog post explaining the reasons of starting RuDy appeared.

Head-first

Submitting a talk about a pretty innovative and technology-glue project before it was ready to be shown was a kind of stunt I pulled out (partially) for the sake of motivating myself to work on RuDy.

There were some dangers and possibilites of failing, but Kirk helped me overcome all the obstacles in two ways. First, he did in PyD basically everything (and more) I wanted to do in RuDy, so I knew what was possible anyway and had a lot of working code to look at (and to shamelessly copypaste from). Second, and more important actually, Kirk turned out to be very friendly and helpful guy, ready to help me with problems that were blocking me (mostly due to my small knowledge of D and it’s nuances) whenever I sent him a desperate e-mail.

Another obstacle was basically around my studies: this year I’m supposed to graduate with a Msc. degree from physics and thus my thesis needed (still needs anyway) a lot of work, not to mention my everyday bill-paying job as Ruby developer. Last two months were pretty busy and what I was afraid of occured: I had only two last weeks when I could really focus on RuDy and prepare it for prime-time and announcement at EuRuKo.

Current state

But it worked.

Every single hour of writing D code for RuDy was actually a pleasure and motivator for spending another hours. Everything just worked as expected (there were really a few bugs that made it past compiler — and they were caught by unit tests) and that was a delightful experience. That was a deja vu — last time I’ve been seriously programming in D (simple yet funny physical simulations) made me feel the same. I’ve been coding furiously for the last two weeks (and the last one was completely crazy in that regard) and even took some days off at work to have more time for RuDy.

Now I’m 12 hours before plane from Warsaw takes off and RuDy is nowhere near PyD in terms of features, but complete just enough to show at EuRuKo and inspire fellow developers. Everything I’ve been working on is fully covered with unit tests, so shall any bug arise, a failing test case will be everything that’s needed. Test coverage also eases (and lets me postpone) writing a sensible documentation for the whole library, a task left for the times RuDy matures and stabilizes a bit.

Future shock

I’m still nervous about showing it on Euruko. I have a lot of confidence in it, but due to time constraints I won’t be showing “live” code compilation and running (OK, maybe I will show Dexter, the test application) and limit myself to code snippets in the talk. Not that I have problems with self esteem, but stumbling upon a runtime failure — when everything’s been working and developing so smoothly during the last two weeks — would probably break me down.

I also hope it’s going to attract some enthusiastic developers and I won’t be the sole developer, tester and user of RuDy. This project is too promising and inspiring (yeah, in my opinion, but a few programmer friends confirmed that) to be left unused and I definitely need someone that would be the end-user of the lib for suggestions about shape and convenience of the API. I’ve been shaping RuDy onto library I’d like to (and I surely will) use myself, but “parent’s bias” is something I want to avoid.

For sure there’re going to be questions I can imagine asking after such talk. “Why bother when there’s FFI and inline gem” — well, I don’t personally like how inline and FFI separate me from the generated code and I haven’t seen anything bigger than example-apps written using it (I think it may also be pretty hard to do it). I still like how they ease writing small code snippets in C and do the variable to-and-fro conversions. “What about JRuby, you can’t link D with that” — yes, but D is closer to Java than C and thus it’d be a lot easier to port code from D to Java (all the class-relation design stays the same). “What about garbage collectors of D and Ruby, aren’t they going to conflict with themselves?” — that’s the smartest question I could think about and the honest answer is “I don’t know, it needs to be thoroughly tested. Python’s GC do conflict with D’s GC and that’s why PyD has some code to prevent that — I have some of that code copypasted in RuDy, but commented out until the need arises”.

RuDy is a fun project that hopes to become used by Ruby developers feeling frustrated by C (malloc, free, segfault) every time they have to write a native extension. It’s also my first innovative, full-featured and started Free Software / Open Source project, so enthusiasm won’t wear off soon. I hope it takes off!

I’m taking off in a few hours.

Dear fellow Rubyists: You're a failure to communicate

2009-05-04T00:00:00+02:00

Dear fellow Rubyists: You’re a failure to communicate

04 May 2009 – Warsaw

Preamble

Because of the whole outcome of the situation, here’s the short abstract of what you can expect: it’s going to be about recent shitstorm that followed Matt’s ‘pr0n star’ talk on GoGaRuCo about CouchDB. If you’re tired or offended or otherwise not willing to read about the topic and my bitching about communication standards in Ruby community that’s to follow, it’d be best to read something else.

The Maturity

This is how mature people talk about their differences and discords:

“Hi, do you have a while to talk about your CouchDB presentation?”
“Wow, a girl. Sure I do, did you like it?”
“Actually not quite, I think you pushed the joke too far and too long and I felt disturbed and uncomfortable.”
“My apologies, I didn’t quite expect women to show on the conference and my talk with “pr0n” in title. I’ll be more careful next time, maybe it was indeed carried too far. Can I do something for you to make up for the unpleasant talk?"
“Yeah, I have this app on my laptop here trying to use CouchDB for storage. Could you show me how to do it, the PG-rated way?”

Etc. etc. Well, apparently that could be real only in the times before Teh Interwebz. Spinning the Internet Flame Machine instead of discussing how grown-ups do is a part of bigger phenomenon on how Internet affects the way we communicate, summarized wonderfully by this quote from Californication :

Just the fact that people seem to be getting dumber and dumber. You know, I mean we have all this amazing technology and yet computers have turned into basically four figure wank machines. (…) People… they don’t write anymore, they blog. Instead of talking, they text (…)

Apparently geek community should have “instead of discussing, they flame” added at the end with a special version for Ruby community s/flame/ghetto-flame. So when two women (1 2) present at GoGaRuCo have worded their discomfort while attending this talk, it was immediately picked up by everyone interested and used to fire one of the biggest flamewars in Ruby/Rails world since Zed’s infamous Ghetto rant. Where “everyone interested” means actually all the attention whores (yeah, I’m one aswell, as this post proves) willing to jump into any heated ruby/rails related discussion, from folks at reddit to authorities like DHH himself.

You see, personally I did like the presentation — or at least the version of it that made it onto SlideShare (said to be “lighter” than the originally shown). I’m OK with porn and being a white heterosexual male (and atheist on top of that) there’re actually not many jokes that could be offending to me. But that’s just me. I like dirty jokes and making presentation as un-professional as it could get, as long as it has all the necessary crunchy code and technical stuff (I actually don’t give a fuck about fluff as long as there’s crunch). That being said, I understand that being a woman surrounded by 200+ men and having a 30+ talk filled with even softcore imagery (great ass in G-string on the first slide, Matt!) and sex references can be disturbing and unnerving.

You can read great summary of the whole shitstorm, together with really insightful conclusion on Martin Fowler’s blog and that actually sums it all pretty much. But there’s one unsolved thing that no-one pulled out yet.

Failure To Communicate

I know that Teh Interwebz has its set of rules and actually being a creative part of the Web means being more-or-less an attention whore. That’s good or at least acceptable. I’m also aware of the fact that geek community loves flamewars and shit-throwing, reasons of which are perfectly summarized by century-old term, be them about “the right” editor, linux distro or single talk about one Ruby conference somewhere in San Francisco most of us wouldn’t hear about if the flamewar hadn’t taken place.

But I hate pulling out the whole “Ghetto” metaphor. Actually Sarah started on this one and instead of being politely ignored like a quiet fart amongst polite people (how naive am I to hope for “politely ignored” on the Internet?), it gave a bucketful of fuel for the whole flamewar. Even my mate Peter fell for this and that’s when I realized the things have gone too far. It’s not a way for a good and healthy discussion, it’s an invitation and encouragement to throw shit at each other.

And that’s a failure to communicate.

One more thing.

I am a fucking professional, you cocksucker!

What next Zed-coined term are you going to use next? “Rails community is still bunch of scrawny cocksuckers with carpal tunnel syndrome”? Yeah, that’d definitely make your blog post sound like a weighted and professional one. If you want to talk about professionalism, don’t “talk ranty Zed” about it because it’s simply ridiculous. You don’t get Borat while giving an etiquette lesson, you don’t talk South Park to your client, you don’t dress like Pedobear for serious discussion about child abuse. But you use Zed’s rant language when talking about being professional. How fucked is that?

The saddest thing about that is that Zed’s rants were supposed to be an act. Instead of being a good laugh at Zed’s issues with some members of US Rails community, it became a new standard of communication amongst rubyists. It reached the point where Zed himself put off the rant and probably regrets making it public in the first place. Sure, go on with that convention — as I stated already, I like dirty language and brutal honesty. But don’t talk then about professionalism. It’s unprofessional all the way.

I know that our heroes, for instance DHH and Linus Torvalds, like profanity (stating it openly or making fun of ubuntu names). But there’s a difference between “fuck, that’s awesome” or calling the-other-geeks-group “wanking walruses” and using Zed-talk when encouraging qualities like professionalism and politeness inside one’s own community.

It’s a difference between “innocently funny” and “ridiculously paradoxal”.

I can haz construktiv and polite diskushyn?

Pretty fuckin’ please with sugar on top.

Sadly, PhpBB3 Still Rules

2009-03-29T00:00:00+01:00

Sadly, PhpBB3 Still Rules

29 Mar 2009 – Warsaw

The Setting

I’ve been looking for some decent forum package for Border Princes, the largest Warhammer Fantasy Battle online forum in Poland. It’s been running on a fork of PhpBB2 called Categories Hierarchy for some time now (over two years). For these two, almost three years, it’s been pretty stable and well-performing. So why bother and not just leave it?

The Dilemma

In today’s times of Web2.0, webservices and APIs, “the cloud” and other buzzwords, a stand-alone internet forum that cannot communicate with other webapplications is simply not enough and so 2001. I want to build some site(s) that could use PhpBB’s registration, authorization (the forum has grown a pretty large community) and — most importantly — dead-simple information exchange capabilities. While the first two would be pretty easy, the third one bites without any decent webservice.

Generally speaking, phpBB2 — even with Ptirhiik’s decent on work with Categories Hierarchy that have structurized and objectified lots of its spaghetti-code — sucks balls from a technical standpoint. I’m not even talking about language (phpbb4, yuck!), it started to embrace such modern and extravagant programming techniques like OOP only recently. I mean, what the fuck? My own code has seen OOP since I was 17 (and whenever the language supported it) and that was back in 2001. Authors of phpBB have created the largest and most successful piece of forum software, used by millions of people on a daily basis, and the best they could achieve was structural programming.

The Search

So I started to look for a successor to PhpBB2+CH, to migrate Border Princes onto a decent and modern forum engine.

Generally speaking, I was disappointed.

PunBB — too small and simple, I need subforums and pretty big authorization capabilities (at least ACL+groups). Invision PB — paid, not an option (besides being pretty bloated from what I remember from Reaper forums). SMF — just not enough (and its usability made a few people actually leave the board what was on phpBB previously). And don’t even get me started on migration scripts from PhpBB2, let aside PhpBB2+CH (which introduced a lot of changes to database schema).

It ended up on PhpBB3. Feature-rich, modern (or that’s what I thought when making the decision), fast and well-performing. Having a great migration script from not only PhpBB2, but also from PhpBB2+CH, which made the whole migration process a breeze. Running on my small VPS (512 Slice) on SliceHost (I still haven’t fully switched yet), it manages to serve 3000+ unique visitors daily (Google Analytics data) without even breaking a sweat (this VPS runs also 3 Rails-powered sites, so it’s pretty short on RAM) and visibly snappier than PhpBB2+CH, although it took some MySQL optimization and options tweaking beforehand.

The Horror

And then, after the forum got up and working, I decided to look at PhpBB3 engine codebase to check it out and maybe do a few modifications and… yeah, the title pretty much says it all. To put it lightly, the phpbb team managed to reach pretty high in standards of structural programming. Too bad that ~10MB codebase written in structural php with global variables (yeah — not even a Configuration nor Singleton object) means a crapload of interconnected code. On the good side, the code is pretty readable (as far as structural php code can be), well commented and pretty easy to hack. View part is separared into Smarty-like template files. Still, there are a few things that made me realize I’ve been living in a beautiful world for the last two years, filled with Ruby, RESTful design, design patterns (not even mentioning OOP), well-thought APIs, easy-to-add plugins etc. Meanwhile, in the world of forumboards, the best one out there is still PhpBB with lots of flaws:

(still) no pretty URLs
no plugin API or hooks, still MODs that are all about hacking
MODs aren’t even in .diff (nor any similar) format, just a pack of instructions for human
no gravatar support, not even through mod
no OpenID support
no webservice or any other means of accessing forum data for external apps

The Excuse

I know it’s easy to bash without doing own. I’m not going to bash PhpBB3 team, not much than what I wrote above. Writing a forum engine is a pretty big task — and if it was easy to do such a feature-rich engine with keeping up to the current programming standards, it’d probably be done already. I know that PhpBB3 team are aware of their code flaws and roadmap for 3.2 includes things like proper MVC separation (in place of current MC+V), fully OO-code, plugin API/hooks, dropping support for php4 and other hope-giving points. So I’m waiting eagerly. It’s 2009 already, goddamit! But until you do I have to do something…

The Hacking

I need a proper webservice-based access to my forum data, period. I don’t have time to wait for 3.2, so I did what I was afraid of for the last two years: went back to not only coding php, but actually coding based on someone else’s php code. I guess getting dirty from time to time won’t hurt, so what the hell. Here it is: PhpBB3 Webservices, my attempt to give some modern touch to PhpBB3. It’s on github. It doesn’t require hacking existing PhpBB3 codebase (drop-down in a subdir, has own modified copy of needed phpbb3 files), uses it’s whole authentication and authorization stack and is generally based on the dead-simple idea of XML-producing templates.

Programmer is NOT a Poet

2009-03-28T00:00:00+01:00

Programmer is NOT a Poet

28 Mar 2009 – Warsaw

Poetry is Art. Wat?

I see a growing tendency in the programmer community to call programming “an art”, compare it to poetry (and subsequently programmers to poets and code to poem) etc.. It’s been even supported by many authorities in programming community, like Dave Thomas, which really makes this notion one big WTF. Stop that. Despite its obvious masturbatory value (OMG I’m a poet), it’s simply not true. There is one big similarity, but the one just-as-big difference makes this comparison pretty much worthless.

Let’s start with defining two qualities of art. First, the definition of art itself — or, actually, my own definition (that got me into many arguments actually) for the purpose of this post. Short version: art is “a craft with a personal twist”. The long version: art is a craft used in a irreproducible way to express its author by creating something. The “irreproducible” part is simply tied with the creative aspect that no computer or other, whatever-skilled human can reproduce, given the same conditions — because computers are totally non-creative, and humans are actually tied to their own expression and creativity.

Or because, simply, there’s no process in art. You can dismantle what we all call “creating”, but at some level you finally reach the “think out something”, maybe in a loop of “think out, check, correct or move on”.

And yes, that definition — or understanding — of art got me into many arguments, because simply defining art as creative expression (with craft thrown is) basically rules out instrumentalists that are called “artist” because of simply replaying classical compositions. So yes, using this definition, for an instrumentalist to become an artist would mean writing something new (and pushing the music forward) instead of competing with a MIDI sequencer.

The second quality of art is that while there’s no defined process (as stated above), there are also no objective measures of judging a quality of given piece of art. It’s so subjective it’s actually pointless to judge (maybe only for the sake of recommending finely-crafted art to people with similar taste). You can judge the “craft” part, or how well the artist performed from a technical standpoint (that’s the part computers are capable of doing, like with MIDI sequencers), but the “creative” part will be taken differently by different people. Some people say music is resistant to that, but that’s not true (given the music satisfies some elementary senses of harmony and rhythm) — my father wants to leave the car when I put Bury Your Dead latest album and I don’t want to drive if he’s going to put his country stuff (and we find a common ground in 70’s hard rock). And Dali is waaaay better than Picasso.

Programming is Engineering (yeah, shocking)

Here’s the surprise. The no process part applies to many fields of science and engineering, where any process — no matter how well defined — ends wherever human creativity can (or has to) start on the design level. Architecture? Maths and physics? Any machinery design actually? So, provided you’re really doing any creative stuff when coding (and you do, unless you’re a code monkey), there’s really no difference between your creative programming lack of process and other engineer’s creative/design phase.

So why other engineers don’t call themselves poets?

The second quality is where we, engineers — and, specifically, programmers — are different from artists. Writing software is similar in the first quality — whatever methodology you apply and put a programmer into, he still at some point reaches the “think out code” stage and is on his own. But the second quality, the judge-ability of a given piece of software is actually pretty well defined and measureable, at least to some extent: does the code work as expected and is without bugs. This is engineering — you create something that actually works in a deterministic, repeatable and measurable way (hey, it’s even science!). This is not poetry nor art, as their results (i.e. impression) aren’t repeatable across population and depend on personal taste. Do you like Bury Your Dead? Because I sure do, but I doubt 100% of people reading this post do as well.

That’s why you, fellow programmer, are an engineer — not a poet. If you want to be (what for, to pick up jailbait?), go write some actual poetry. And then claim no-one understands you etc. etc.

One Last Myth To Crush

I know what you think: Dave Thomas, while comparing programming to writing poetry, used also the example of starting with a blank piece of paper or blank page in a text editor. And then creating something. Like, wow. Tabula rasa, creation from nothing and all the other shit.

My girlfriend’s an architect. Guess what she’s starting with? That’s right, a blank white sheet of paper. Or blank black workspace in AutoCAD. And she’s also creating something. But how come she’s not calling herself a poet?

Oh, I know. Us, poet-programmers, create some working thing from nothing, while architects create just a design, a set of data for the actual builders to use for building the house.

BS, yet again. We all just create some sets of instructions in a given domain-specific language. Architects use lines and symbols for their language, we use sets of statemens from programming languages. Our whole creation is non-existent and totally abstract without something that makes a real, working thing out of what we write: bricks and builders for architect’s project, compiler and computer for programmer’s code.

The End

Don’t worry. You can live without being a poet. You still can pick up chicks at bar, just use some less-cheezy lines than calling yourself a poet (or “code poet” — unless you write in Haiku).

Just throw away that old dirty scarf.

Ruby Feed Mix

2009-03-13T00:00:00+01:00

Ruby Feed Mix

13 Mar 2009 – Warsaw

Warning: this post is under development at the moment. Keep coming back until this paragraph disappears.

Disclaimer These are, of course, my personal choices and opinions based on personal taste and other subjective criteria. It’s not supposed to be any sort of attack on any of the blogs.

RubyLearning has just published an impressive list of 50+ ruby-related blogs (it’s 70+ at the moment actually, since people rushed to suggest “add my blog as well, it has rubies!”).

Holy shit. If I was to keep seventy blogs in my RSS Reader, I would spend my whole day switching between the feeds, let alone reading all the new stuff these people post. And since I’m using iGoogle as my RSS Reader (helps accessing all the stuff while I’m not using my own laptop), the page with all this stuff would be about 5-pagedown-presses long. Just come on, 70+ blogs / RSS feeds to subscribe? No fraggin’ way.

I want two things:

filter out these 70+ blogs: it takes some effort to list them all, it takes a lot of time to filter them out
combine all of RSS feeds of these blogs into one fat “feed-fisting” RSS-mixture (because some of the blogs are updated once a month)

The first one will come with time and experience and will actually be a no-brainer to execute when I get such a RSS-mix. And for the second I found a nifty little site RSS Mix that I’m going to try out. There’s no reason to not share the results of such undertaking, right?

And the results will be shared in the form of feeds links, ready to be pasted straight into a new RSS Mix.

Before I do it, there’re definitely some blogs that (for me) don’t need to be subject to this test, as I read them since a few months now and they do deliver without further proving.

The blogs I can recommend whole-heartedly after a few months of regular reading (i.e. I knew them before RubyLearning post):


http://feeds2.feedburner.com/codefluency
http://khamsouk.souvanlasy.com/feed/atom.xml
http://feeds.igvita.com/igvita
http://feeds2.feedburner.com/obie
http://feeds2.feedburner.com/petermarklund
http://feeds2.feedburner.com/railscasts
http://rails.co.za/feed/atom.xml
http://feeds2.feedburner.com/Railsonwave-Home
http://feeds2.feedburner.com/RubyInside
http://www.rorsecurity.info/journal/atom.xml
http://feeds2.feedburner.com/RyansScraps
http://zedshaw.com/feed.xml
http://feeds2.feedburner.com/ZenAndTheArtOfRubyProgramming

So basically below there are two lists comprising of feeds of all the blogs announced in aforementioned RubyLearning. I’m going to put them all in my rssmix and remove systematically the ones that don’t deliver – be it because of too much fluff posts or simply crappy content.

In order of appearance on RubyLearning blog post — ladies and gentlemen, feeds I like and am subscribed to:


http://z.about.com/6/g/ruby/b/rss2.xml
http://buddingrubyist.com/feed/
http://feeds2.feedburner.com/SoftiesOnRails
http://www.exampler.com/blog/feed/
http://blog.headius.com/feeds/posts/default
http://www.thechrisoshow.com/feed/atom.xml
http://feeds2.feedburner.com/LoudThinking
http://feeds2.feedburner.com/DrNic
http://feeds2.feedburner.com/Segment7
http://blog.fallingsnow.net/feed/
http://brainspl.at/xml/rss20/feed.xml
http://austinentrepreneur.wordpress.com/feed/
http://feeds2.feedburner.com/rails-envy
http://feedproxy.google.com/buckblog
http://feeds2.feedburner.com/SoylentFoo
http://feeds2.feedburner.com/hasmanythrough

And here below I’m going to drop the ones that were above, but didn’t like them (and why I did not). This is a very subjective list – you may find these blogs great, they just didn’t fit my taste and appetite for a technical, ruby-oriented blog. By the way, if you think I’m narrow-minded enough to have only ruby-related blogs in my RSS, go fuck yourself. This blog post is about doing one uber-rss-mix of ruby-related blogs, not about everything I have in my reader.

In order of appearance on RubyLearning blog post: feeds that my reader isn’t fed with (and why):

http://feeds.therealadam.com/TheRealAdam (Chomsky is cool, but it was supposed to be Ruby-oriented)
http://feeds2.feedburner.com/slash7/rss (ctrl-f “ruby” — WTF, only one occurence beside sidebar?)
http://feeds2.feedburner.com/Chadfowlercom (I’m reading this one thoroughly now, as I can’t bear the thought of trashing blog of someone like Chad Fowler — it just doesn’t feel right)
http://pragdave.blogs.pragprog.com/pragdave/atom.xml (because… I don’t know, try it yourself)
http://derekneighbors.com/feed/ (Agile is cool, but not ruby enough)
http://simplifi.es/home/atom.xml (later, dude – write some content here)
AkitaOnRails (I loved that blog when author wrote in english, I’ll give it another shot after I learn Portugese)
http://feeds2.feedburner.com/nubyonrails (emacs and commercials — not my favourite)
http://feeds2.feedburner.com/wossname (not yet, but I’ll be reading it from time to time — has potential)
http://feeds2.feedburner.com/iamruinous (fluff)
http://feeds2.feedburner.com/JamesOnSoftware (cool blog, empty feed — dammit!)
http://www.fearoffish.co.uk/ (firefox failed to connect)
http://feeds2.feedburner.com/jayfields/mjKQ (I have this blog in my feedreader, but not in “ruby” tab)
http://jicksta.com/feed (cool topics but not my interest; more announcements and fluff than real code)

I got only to number 32. on the RubyLearning list — stay tuned for updates.

Notes:

I’m not even trying company blogs, I prefer people when it comes to writing (honesty and standing behind your statements). Try them on your own risks. Some are pretty cool anyway, so maybe I’ll put a blog post about them the other day.
Zed’s on the list — I wasn’t going to include his blog on the list since he redone his page to a cojones-less version, removed the rants and declared not to write them anymore. But it took a week or two for him to finally write this little gem that brings back my faith in Zed and his style. I’m still waiting for the stripper-silhouettes though.
Austin of FiveRuns fame stayed on the “good” list even despite using Wordpress ;)
RailsEnvy stayed on the “good” list even despite my dislike for videos and screencasts when it comes to technical content (I have a soft spot for their old Ferret and Rake tutorials)
Jamis/Buckblog — I wonder what he’s going to write about now, given he left his most notable projects? On the “good” list as for now.
Peter Szinek’s Blog wasn’t even mentioned on the RubyLearning list — WTF? It’s a great blog and fully deserves being noted. Oh, and Peter’s a great guy too (that alone doesn’t guarantee landing on the “good” list though ;)).

Free the DMD Compiler

2009-03-06T00:00:00+01:00

Free the DMD Compiler

06 Mar 2009 – Warsaw

If you haven’t heard this one already, Digital Mars D Compiler (DMD) by Walter Bright has just been released with a full source code. That’s the good news, as the closed-source backend of DMD was pretty much repelling for some of the D programmers out there.

The bad news is that backend, while being Open Source, is not Free Software. As Walter explained in other thread on Reddit, some of its bits licensed from Symantec cannot be GPLed as is stated in the license.

Two things, and their consequences, seriously impair D’s widespread adoption and attracting a critical mass of developers. They are, namely, compiler hell and stdlib hell, both pretty unique to the language (and thus more dramatic) at the moment.

Compiler Hell. There are three D compilers out there and there’re issues with each one of them. DMD is not Free Software (thus is not included in official repositories of Debian and Ubuntu) and sucks at creating Linux shared libraries. GDC, being free from both aforementioned flaws, is lagging behind (version 4.2.4 20080705 (prerelease gdc 0.25 20080312, using dmd 1.024) (Ubuntu 0.25-4.2.4-3.1) on my ubuntu here). And there is the new kid, LLVM D Compiler, furiously developed by its authors and even recently announced to be in pretty workable state, but it’s still very fresh, untested and requires some development.

Standard Library Hell boils down to Phobos vs. Tango, an issue that’s still (despite some efforts to bridge both libs) a pain in the ass for many D programmers. I’m not going to elaborate on this one, even despite having a lot to say about the topic (I’m not in a mood for rants).

Generally speaking, if you’re here reading this, you probably already know and dwelled into both hells. So, what’s the point of this post?

The point is that, I like DMD and I’m full of respect for Walter Bright — the man knows and has insane amounts of experience about writing compilers, designed this great language he called “D” (probably should have called it after a beverage, animal, rare stone or just add another strange character to C) and still has enough modesty to admit that DMD has flaws and that not his, but the other stdlib’s (Tango) developers were right when it comes to designing standard library for D. I don’t want DMD, probably together with the language the most notable project of man’s life, to be forgotten or abandoned only because some pieces of code were licensed in a way that impaired its adoption (most ambitious and language-curious programmers are using linux and/or Mac OS X) and prevented code-exchange between D compilers. I think it would even be easier to make DMD produce working .so files (I need that for RuDy) than to develop the two other compilers to implement (and do it right) all the features of D 2.0 that DMD already has.

So, what to do? Simple.

Free the DMD Compiler

Let us, D programmers and enthusiasts, do it! How? “Buy” the code from Symantec, that is to pay for licensing it in a GPL-compatible (or, generally, Free Software compatible) way. Generally speaking, it’s about doing the same thing that community did for Blender.

Who’s in?

And who knows how much would Symantec charge for relicensing these pieces of backend code?

Goodbye, SliceHost!

2009-03-04T00:00:00+01:00

Goodbye, SliceHost!

04 Mar 2009 – Warsaw

Disclaimer: this post has march date, because that was the time I made a decision to move from Slicehost VPS to a dedicated server. But I’ve finally decided only recently (mid-june), with a decision switch from Kimsufi to OVH.pl as my new hosting provider. At the time of writing this disclaimer all my services are already moved to the new host. Slicehost is still ubercool though.

Dear Slicehost,

I’ve been your client for almost a year now. Throughout these 12 months you’ve provided me with great hosting service (actually best I’ve known up to date), with great documentation and tutorials, helpful and smart support and – what’s the most important – steady and reliable hosting I’ve put a few websites on. I’ve been recommending you to friends and companies, so I hope your being great to me turned out to be profitable (at least two of them bought a slice after my recommendation).

You’ve been rock-stable and thanks to that users of all my sites — be it clients of bitspudlo.com store, fellow hobbyists from Border Princes forum or fellow rubyists visiting Warsaw Ruby Users Group site — were always delivered. That worked for my business (bitspudlo.com) and personal reputation (other sites), so you probably understand my warm feelings.

But times change, people change and sites — it’s userbases especially — grow and a small VPS costing 38 bucks a month simply turns out to be not enough, performance-wise. I know I can upgrade to a bigger slice, but that doubles the cost on every step and it’s not something I’m ready to dwell into, especially with recent madness on the currency market when it comes to exchange rate of polish zloty and US dollar. There’s also the matter of transfer limit and disk quota, both pretty low and sometimes even forcing me to pay extra for my sites, and while you tried your best, ping times across half of the globe (Poland, i.e. middle of Europe) cannot go below certain threshold.

So I’m moving on. You’ve been a great hosting, but I’ve got an option of buying a dedicated server from reliable and reputable company (not AS reputable like you, Slicehost, but close), with machines placed physically in Europe (ping times!) and with all the parameters one could expect from a dedicated, for not much more than 38$/month you cost me.

I’ll keep on recommending you to others needing a small and reliable VPS though (especially fellow Rails developers looking for a good hosting for their first Rails/Merb/Sinatra/whatevere project). Goodbye. It’s been a great time. Good luck, you deserve it.

Blogging like a hacker using Jekyll

2009-03-04T00:00:00+01:00

Blogging like a hacker using Jekyll

04 Mar 2009 – Warsaw

It’s been a few days since I started this website using Jekyll since the beginning (engine like this was one of the deciding points) and I still cannot get over its awesomeness combined with (or maybe related to) its simplicity. While the idea of generating static html from half-baked partial files using maybe some sort of templating isn’t new nor original, Jekyll makes it convenient for the site owner while remaining powerful and feature-complete.

Some of this is related to Jekyll’s impressive network of forks on GitHub, with which mojombo merges regularly when asked to. I can’t summarize all these features here, as I don’t know about all of them yet, but seeing a simple commit log should give the clue.

There’s one thing that “mainstream” (i.e. mojombo’s) Jekyll still lacks and that is handling well posts’ tags and categories defined in YAML header instead of post path. I believe somewhere on github there’s a fork with that fixed and working properly (tagging in static blogging engine – wow), but I can’t find it and I’d also like to stay with vanilla Jekyll.

But hey, Jekyll is written in Ruby anyway, so maybe I could fix it myself? As if there weren’t enough forks of this project on github already. Or maybe I could play with Jekyll a bit, adding some completely new features. Or maybe implement some part of it as native extension (“related posts” functionality seems to be pretty slow). Or maybe even do it as native extension, but in D, using RuDy?

Damn, I love opensource and github. Goodnight.

RuDy: Ruby native extensions in D programming language

2009-03-03T00:00:00+01:00

RuDy: Ruby native extensions in D programming language

03 Mar 2009 – Warsaw

There are two facts about Ruby.

Well, at least two I want to focus on.

First: the “official” Matz Ruby Interpreter (MRI) is here to stay and going to remain the mainstream Ruby interpreter. Despite its technical flaws, despite JRuby gaining popularity in some very specific applications (Mike Lee used it pretty creatively with GreaseSpoon in his “Test By Proxy” method) and with Rubinius development practically stalled because of recent EngineYard layoffs.

Second: Ruby 1.9 / 2.0, with all the performance improvements it brings, is (will be) still “slow” in CPU-intensive operations when compared to compiled languages and when performance really matters.

Arbitrary selection of uncheckable hypotheses (almost axioms) leads to a simple conclusion: native extensions are here to stay, whenever and wherever Ruby’s performance is simply not enough.

That would be digestable, except for extensions have to be written in plain old raw C. But I don’t want to write in C.

Don’t get me wrong. I love C for its simplicity (without feature bloat) and being close-to-the-metal, while having some pretty big possibilites. And for all the programs written in it (when you try to say “Linux” with a smile and wide “i” like in “Leenux” it always ends up in a menacing grin). It’s just that I hate C for its simplicity (feature-poor) and being close-to-the-metal, combined with possiblities of shooting yourself in the foot.

It’s not as bad as C++. But it’s still a language I’d gladly leave behind.

I’d love to write native extensions in a compiled (to machine code) and fast, yet modern and pro-programmer language. C-family syntax would be a plus because of familiarity and ability to copypaste library-agnostic code (mostly number crunching, pretty common in numerical simulations) with minimal modifications.

And there is one like that, which I fell in love with after reading some and writing more (whole semester of “Computer Methods of Simulation” programming lab). It’s called D Programming Language and it really deserves more attention that it gets.

There are many aspects of D I love (and some I don’t like), most of them outlined in my talk about D on january WRUG meeting. Basically speaking (for needs of this blog post) D, with all its awesomeness and pragmatism, can interface with C code/applications/libraries both ways. So yes, you can – at least theoretically – write Ruby native extensions using Ruby’s C API, but in D.

Although there are a few gotchas.

Many D and C datatypes are different. Differences can be “trivial”, i.e. ones that shoot in the foot only after moving to a different platform, like constant-size numeric types (int in D is always 32 bit, no matter what insane hardware you’re targetting). They can be because of “pragmatic” syntactic sugar (arrays in D are actually structs with integer size and pointer to data. They can be non-trivial, like struct padding (or structs in general), and these ones are fortunately (for the most part) solved by the compiler that passes stuff differently to functions declared as extern (C) int some_func(...).

These differences pair up with second problem: for interfacing with C functions, D needs to have their declarations. They are usually within .h C header files that D compilers don’t read – they want these declarations in .d source files. For small APIs or small subset of API functions that you want to use it’s cool to copypaste these functions into .d file and modify their accepted datatypes slightly (if necessary – rarely). For bigger APIs this daunting task can be performed using htod (Windows-only, bleargh!) or bcd.gen.

And remember about differences in datatypes. Where C function accepts “a string”, which in C world means zero-terminated array/pointer of chars, in D you have to pass (d_string ~ \0).ptr. That’s fortunately the only really crappy part about C vs D incompatibilities – rest of them can be circumvented or boiled down to lowest common denominator.

All these difficulties are still nothing compared to awesomeness (productivity, features, design) of D programming language. Believe me not, try it yourself. Hell, it’s one position below Ruby in Tiobe Index (and the language is damn young!).

Let’s get back to Ruby. So I want to write my native extensions in D. Ruby has a pretty big C API which doesn’t interface with D code out-of-the-box after converting .h to .d with bcd.gen. And getting the Ruby API to work in D is just the beginning, as I would also like nicer, D-like (object-oriented, using D’s metaprogramming and reflection capabilities) API for Ruby native extensions. Like Ruby object wrapping (into convenient D classes) or automagic class and method definition, i.e. selected class with its method would be accessible from Ruby after writing single line of code, without rb_define_class followed by tens of rb_define_method. And I want my extconf to recognize D extension and prepare proper makefiles, so I can distribute it in a gem to other people who would only see “building native extensions” on gem install and not even getting a clue about the thing being written in D.

It is all possible: D bindings to Ruby API, nice D API for Ruby extensions, D-aware extconf etc. etc. Hell, pythonists already have that, so even the hard D part for the API has been already figured out by someone smart.

I’d love to have something like PyD, but for Ruby of course.

So I started it. And called it RuDy, name inspired by a talk between Ray C. Horn and Kirk McDonald (author of PyD – the man knows damn lot about D and interfacing with interpreted language). And want to make it something really usable, to let fellow rubyists code even their native extensions in a modern language, so they won’t have to dwell into cold and unforgiving depths of raw C world, threatened by memory leaks and segfaults.

As for now it consists only of the “D bindings to C API” – and even that of incomplete, but a pretty usable already, set. On the plus side, by “set” I mean functions with full test coverage in dexter application (part of RuDy project – cover the whole API bindings with unit tests), not all the shit bcd.gen threw onto me. So there’s a pretty big chance that many other functions (I say most should) in this binding work without explicit statements, but it’s not good enough for me if they’re not covered with unittests and passing them.

So, all the fellow Ruby enthusiasts willing to get their hands dirty on some open source code: RuDy needs you! I’ve developed it into a state of having some (the most important in my opinion) working bindigs to Ruby API, and will continue doing that (so you can basically write your extensions in D already), but there’s a lot of work ahead to make it a great project

There are developers needed in all three of the fields, all of them pretty much orthogonal (i.e. you can work on your field without looking at others’ development):

D bindings to Ruby C API — this is basically what I’ve already started and completed some part of; requires knowledge of some C, willingness to learn D (including differences and similarities of both languages) and basic Ruby (unit tests and basically checking “if it works”)
D-aware mkmf (or other build system) — requires some Ruby skill (to read and copy/modify mkmf code) and at least basic makefile-fu (to understand extconf code being modified)
D object-oriented Ruby API — requires willingness to learn (a lot of D, some Python for reading how it’s done in PyD)

Just think about it: an open-source project requiring not very much sophisticated programming skills or knowledge (just the ability and willingness to learn – but hey, we’re programmers!), just some work and looking awesome on your resume (Ruby, D, connecting two languages, open source project). In?

If you want to contact me, do it via one of the contact methods listed on about page.