Tuesday, December 16, 2008

Updating to RubyGems 1.3.1 on OS X

As much as I enjoy ruby and rails, I don't keep up with the community much.

This has its price... sometimes I'm blindsided by things that I imagine are "common knowledge" on the mailing-lists, but aren't documented anywhere that we lesser mortals actually notice.

The problem I had this morning was mentioned, in obfuscated form, on one blog, at least.  I thought I would state it more explicitly:

I updated to Rails 2.2.2, but it complained that I needed to update to RubyGems 1.3.1.  When I tried (sudo gem update --system), I got this message:

Updating RubyGems
Nothing to update

...It turns out you need to use the "alternate install" for RubyGems.  ...despite the fact that the documentation says explicitly that you won't need to if you have a more recent version.  Do this:

> sudo gem install rubygems-update
> sudo update_rubygems

...Now you're running 1.3.1, and Rails will be supplicated.

Wednesday, October 22, 2008

Bugs Me:

script/console test
mongrel -e test
rake db:migrate RAILS_ENV=test


Consistency is a good thing.

Tuesday, October 14, 2008


I generally don't pay attention to server stats, but recently I was getitng frustrated with IE6, so I asked those in the know how prevalent IE6 was.  The reply:

"...as a total, IE 6 is about 22% of all users"

My response: holy shit. That's obscene! IE6 is broken, and to think that over 1 in 5 people are still using that crap-tastic browser is an offense to the internet.  Sure, sure, I can understand needing to keep IE6 on some fragile server somewhere that, if you update it, your entire site breaks.  Fine.  But stop browsing with the damn machine!  There's no good reason to keep using IE6.

One in five!!!

People: switch!

Wednesday, September 17, 2008

Composite Primary Keys (everyone's got an opinion)

Tonight I was playing around with some of the "cooler" things that Rails can do, and I thought I would try them out on our code.

Namely, there are two features that I'd like to use: nested_has_many_through, and named_scope.

This would allow us to write code like "taxon_concept.data_objects.images.visible" without having to go through a hairy find_by_sql, and it would allow us to more easily make changes to the DB (like, say, how we define "visible" for data_objects).  It would have made one of today's changes painless, too (long story).

It would also reduce the amount of code we have by a significant amount.  ...I like less code.

...The problem, though, is that DHH doesn't understand composite primary keys (CPKs) *at all*, and his framework reflects that.  Even with the CPK plugin, things don't work right.  ...That is, when you declare a CPK on a model, it seems that it expects every reference to it to contain all of the PKs.  I tried doing something like (pseudo-code): "DataObject.has_many :data_objects_taxa, :foreign_key => :data_object_id ; DataObjectsTaxon.has_many :data_objects, :foreign_key => :id" (DataObjectsTaxon has a CPK)... but this doesn't work: it STILL tries to pass in two key values.  The resulting queries are actually rather asinine (notice the 1,4 at the end), and fails miserably:

    SELECT * FROM `data_objects` WHERE (data_objects.id = 1,4)

...I could delve into the CPK plugin code and try to resolve this, but I might be gone for days.  ; )

You can kind of get around this by using a habtm relationship.  ...But that only works for that one hop.  So I can get data_object.taxa, but I can't get data_object.resources, using this code:

    has_and_belongs_to_many :taxa
    has_many :resources, :through => :taxa

This throws "Unknown column 'taxa.data_object_id'"... and the docs even warn about this ("You can only use a :through query through a belongs_to or has_many association on the join model").


What I would like to do--knowing full well that this is the wrong thing to do, but bending to the will of Rails--is change our CPKs to Unique Indexes, and add autoincrements to those tables.  I could write the migrations to do such things without too much trouble, but I worry about the affect it would have on the related PHP code (I hope none, since the autoinc should just merrily do its thang).

Again: from a database level of understanding, this irks me.  Where we use CPKs is where they are uniquely identifying entries, and that's how its sputta be.  ...but Rails thinks otherwise.

...Unless someone knows a way to keep things as-is and get the has_many relationships working in both directions.  There my be some trick to it that I'm unaware of. That would be cool, too.

If we could get this to work, I think our code would be much, *much* more elegant (and thus, maintainable/extensible).  I'd be happy, assuming there isn't a noticeable performance hit (but from what people are saying, it's actually quite efficient).


And: ...apologies to California Pizza Kitchen (CPK).

Murphy's Law of Variable Names

There are at least six appropriate names for any given variable, method, column, or table.

You will not choose one of them.

Wednesday, September 10, 2008

You might be a geek if

..."too much rails" sounds grammatical to you.

Friday, August 1, 2008

Composite Primary Keys (CPK) and RSpec Fixtures

I was banging my head against the proverbial wall all week.  This morning's travail was an error like this in my specs:

You have a nil object when you didn't expect it!
You might have expected an instance of Array.
The error occurred while evaluating nil.first
/path/to/rails/app/vendor/gems/composite_primary_keys-1.0.5/lib/composite_primary_keys/base.rb:239:in `find_from_ids'

...After some digging, I found a page that explained the problem, and revealed that you need this in your spec/spec_helper.rb file:

load 'composite_primary_keys/fixtures.rb'

I feel much better now.

Tuesday, July 29, 2008

Repetitive Stress

So, I've been programming pretty much non-stop for the past few months.  My wrists ad neck were suffering.

Rather than start wearing braces again, I opted to try this program (Mac-only) that interrupts you every so often.  ...I have it set for 12 seconds every few minutes, and 10 minutes every hour.  I started using it about a month aogo

It seems to have done wonders for my hands and wrists, so I highly recommend it.

Ain't done squat for my neck, though.  ...And some days, it gets really bad. : \

Wednesday, July 23, 2008

Wednesday, July 2, 2008

Using Vim and MySQL to Create Fixtures (for Rails)

...I've had to build a lot of fixtures using extant data from a MySQL database.  I don't want the whole database in my fixtures... just a subset, based on some examples I use in my specs.

So I run a query to get the subset I want, being sure to grab all the contents from a given table (and ONLY one table).  Something like this:

  FROM a_fancy_table ft
    JOIN some_table st ON (ft.id = st.a_fancy_table_id)
  WHERE ft.id IN (1, 2, 3)\G

The \G is essential, here.  I then copy the output of that to the some_table.yml file, created with vim.

  1. <ap multiple times, until everything is left-justified, then
  2. >ap once, so that it's all indented 1 level, then
  3. :%s/  \*\+ \(\d\+\). row \*\+/^Msome_prefix_\1:
  4. finally, rename any of those that are really significant.
...Of course, the ^M is really ^v[ENTER], and the some_prefix is whatever you want to name your objects.

...If that makes sense, you'll find it's a huge time-saver.  Unless you're copying entire tables, in which case you can use Dmitry's script.

Thursday, June 12, 2008

ZenTest and autotest in Windows with RSpec and Rails

I've been trying to monkey-patch ZenTest's autotest for over a month now.  Originally, I was trying to get it to work (well) with snarl, but that just seems frivolous at this point.  My focus now is just to get the damn tests working as they should.

First of all, Windows doesn't let you break out of the loop.  ...At least, not that I've ever seen.  I haven't heard a SINGLE person on the web talk about this, which is frustrating.  Maybe I'm missing something stupid, but I know other people that this is happening to.

Anyway, the fix for that is to change the "Kernel.sleep 1.5" on line 291 of autotest.rb... it works if you change it to this:

    30.times { Kernel.sleep 0.05 }

I don't know why, but I'm assuming it has something to do with how sleep is implemented on Windows that causes it to lose the scope of self.interrupted.  Or something.  Whatever: this works.

Next, I'll try to find a regex that will actaully find the right files to run, when there has been a change!  Grr.

Wednesday, May 28, 2008


So, I was just thinking about microscopic images, and how they're always so much blurrier than illustrations.  I was thinking about how, in videos, you can see various parts of the organism sharply, depending on the focus of the optics... it's just a razor-thin depth of field.

Then I thought: hey, wait, we have HDR photography.  ...Why can't we use this at a microscopic level to increase clarity?

I could even imagine a machine-controlled microscope that produces one image from a composite of multiple images shot at different focal lengths, helping to bring the entire image into maximal focus.

...Of course, I'm sure someone else has already thought of this.

Wednesday, May 21, 2008

Posting Ruby

So, I broke down and did some research on how to post code to blogger so that it doesn't look hideous. It wasn't nearly as simple as I wanted it to be (most notably, it was a serious pain to use cocoa to use the pasteboard for converting text)... so I hacked together a little rails project that handles things for me using web forms. It would be a long post to explain all the details, but I'll cover the basics.

First, sudo gem install syntax. Then create a new rails project and get it up and running. When you've got that, generate a controller called "Convert". The code for the controller should look like this:
class ConvertController < ApplicationController
def ruby
render :action => :ruby
def to_html
require 'syntax/convertors/html'
convertor = Syntax::Convertors::HTML.for_syntax params[:type]
@code = convertor.convert(params[:text]).gsub(/<span class="punct">([^<]*)<([^<]*)</, "<span class=\"punct\">\\1&lt\\2<")
render :action => :to_html

And then you'll need a view for 'ruby.html.erb'.:
<h1>Convert What?</h1>
<% form_tag('/convert/to_html') do %>
<table><tr><td><%= text_area_tag 'text', '', {:cols => 80, :rows => 30} %></td>
<td><%= radio_button_tag 'type', 'ruby', true%>Ruby<br/>
<%= radio_button_tag 'type', 'xml'%>HTML/XML<br/>
<%= submit_tag 'convert to HTML' ><td>tr><table>
<% end %>

...And a basic response form in 'to_html.html.erb':
<h1>The raw HTML:</h1>
<pre><%= text_area_tag('whatever', @code, {:cols => 80, :rows => 30}) %></pre>
<h1>The results:</h1>
<%= @code.gsub(/&amp;/, '&') %>

...You'll also want to set up the CSS for your site, using the CSS from the site where I stole this idea from. Polishing things is an exercise left to the reader... but that should get you started.


So, I was writing a few RSpec tests today, and required Hpricot to look at a few views. This was the code I wrote:
     links.each do |link|
this_taxa_name = nil # scope
img = link.at("img")
if img.nil?
this_taxa_name = link.inner_text # This was the text link
has_image = true # This is the image.
this_taxa_name = img['alt']
taxa_name ||= this_taxa_name
assert_equal taxa_name, this_taxa_name, 'The names for the featured species were inconsistent (image alt/text)'
taxa_link ||= link['href']
assert_equal taxa_link, link['href'], 'The links for the featured species were inconsistent (image/text)'
assert has_image, 'There was no image for the featured species'

...This was all because I was trying NOT to assume that the image always precedes the text for the image. The test ended up being a bit longer than one page, so I asked a co-worker to look at it. His simple suggestion was "it's okay to assume the image comes before the text". By making that assumption, this block of code above becomes:
      assert !(links.nil? or links.length == 0), 'The featured species had no links'
assert_not_nil links[0].at('img'), 'There was no image for the featured species'
assert_equal links[0]['href'], links[1]['href'], 'The links for the featured species were inconsistent (image/text)'
assert_equal links[0].at('img')['alt'], links[1].inner_text, 'The names for the featured species were inconsistent (image
It's very easy to see why assumptions are the #1 problem that developers have. They make life much, MUCH easier.

I suppose that's true of life as well.

Sunday, May 18, 2008

Oh, and when adding methods to core classes...

There's this neat pattern of checking to see if it's there first. I'm not sure if this really saves any cycles (I would imagine a few), but may aid in avoiding collisions, at the very least. For example thus:

unless String.instance_methods.include? "_sp_clean"
class String
def _sp_clean
str = self.clone
str = CGI.escape(str) # Turns spaces into '+' rather than '%20'... we expect that, though
# URI.escape allows [=&], etc... which we don't want.

...Now I hope that formats nicely when this message posts. : \

Built my first gem

...And it was rather easy.  After using the "newgem" gem to create the skeleton, I just followed some bloke's instructions for editing the Rakefile.  Piece of cake.

Just thought I'd mention it.

Thursday, May 15, 2008

Text and HTML in Thunderbird

So, I'm using Thunderbird now.

One of the first pains that I've encountered is that, when replying to email, I get a warning that HTML may be unavailable to the recipient.

To turn this warning off (and select whatever you want as a default), look under Tools > Options > Composition > Send Options.  The rest should be self-explanatory.


Wow, what a blast these past three days have been!  We had a consultant in from MySQL, and I learned hoards of new tricks, tips, and techniques.  Here are a few:
  • You can store images on the filesystem based on their PK, to save having a field with the path.  For example, image.id = 15579, image.path = images/1/5/5/7/9.png (or whatever level of separation you find appropriate).
  • EXT3--though it can be tuned--is not an ideal filesystem for databases.  XFS is much better.  (He didn't give specific reasons, and I don't particularly care to look into it.)
  • QCache_hits should be much higher than QCache_inserts.  (show with "show global status like 'qc%'")  If it's not, you need to hard-code which queries use the cache and which don't.  Global status can also tell you about the number of locks that are occurring, which you may also wish to tweak.
  • query_cache_type values are: 0 = off, 1 = on, 2 = on demand
  • QCache hits are never logged.  That would take more time.  So if you want to really see how often queries are running, you have to turn of caches.
  • 400M is a reasonable limit for query cache.
  • Stop apache from forking: this is what brings so many sites down.  Instead, figure out how many concurrent processes your box can really handle, and then set the number of connections to this limit. If this is exceeded, the OS will queue additional requests, and keep the system from dying.  100 is a reasonable place to start.
  • MyISAM tables are fine if you're read-only (or nearly so): they will be faster.  But InnoDB is much, much faster, since they store the table completely in memory.  Of course, this means you have to have enough memory to fit the damned thing.  ; )  ...And, of course, there are caveats about very large fields (like blobs, texts, and varchars with large average size).  Shrinking your data as much as possible is important with large datasets... even one unused byte becomes a Meg with a million rows!
  • "show innodb status\g" is a tool for checking InnoDB performance.
  • strace shows all the system calls used for a command (linux).  Very cool!  Not sure how I hadn't seen this one before...
  • Sharding is what massive scaling is all about... but that's a topic you'll have to look up on the web, since there's so much to it.  ; )

Tuesday, May 6, 2008

One more Annoyance with Tiger Terminal

...So, when I moved .bashrc to .bash_profile, I lost my PATH environment variable.  (Rather, it was stripped down to next to nothing.)

To fix this, I moved .bash_profile aside, re-ran terminal, ran env to get the old PATH, and pasted that into my .bash_profile where I used to say "$PATH".  Something like this:

export PATH=/sw/bin:/sw/sbin:$GEM_HOME/bin:/opt/local/bin:/usr/local/bin:/opt/local/sbin:/usr/local/mysql/bin:/Users/cpr4cpu/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/X11R6/bin

...Pain in the arse, this.

Bash Profile, not BashRC

Another lesson learned:

In Tiger, the Terminal doesn't used .bashrc.  Why?  Couldn't say.

Just move your .bashrc file to .bash_profile

Problem solved.  Lame.  There's more discussion about this on the Mac site, but I didn't really care to dig past the workaround.


I've spent the past couple of days setting up my Mac for Rails development. I'm still running Tiger, which is problematic (to say the least)... I hear that Leopard is all set up correctly, but I'm stuck for the time being.

I finally gave up on Aptana, which was just screwing up left and right.  ...This means that I need to fall back to my old friend, Vim. 

There's a script for calling gvim from a terminal, but I also wanted to change the color scheme on load (because my terminal is dark-on-light and my .vimrc colors are customized for that).  To accomplish this, add to your .gvimrc:

colorscheme desert

(Or whatever other scheme you care for.)

This was surprisingly difficult to find (why?!?), so... I hope you stumbled on this page before too much extra work.  ; )

Tuesday, April 29, 2008

Vim and Windows

This is just a note for those who are searching the web...

So, today I was on someone else's machine, and noticed he used "File -> Open" from gVim... and I never do that!  I just right-click and say "open with vim".

"Well," he explained, "when the file I'm opening is on the network, I don't get my _vimrc loaded".

This has never happened to me on my machine, so we looked into it.  Turns out that gVim loads _vimrc from your %HOME% environment variable, but it does so from whatever drive your file is in.  My %HOME% was specific about C:\, and his--actually just a reference to %HOMEPATH%--had no drive designation.  When we added it, all was well.

The error message we were getting only came up when you quit gVim, and was something along the lines of "can't write _vimrc" or some-such.

Web Developers, PLEASE USE DATES

If I could teach the world of web development one thing, it would be have a date on every relevant page.

Whether you like to admit it or not, someday your page is going to go stale, and the visitor needs some indication of this.  I can't tell you how many times a month I'm at a site, find some interesting information, and later find out that it's totally useless because the page it came from is three years old.

Date everything.

...If you could teach webdevs one thing, what would it be?

Thursday, April 17, 2008

Seeing Red (or Not)

It's always interesting to have one's beliefs paddled over the head by science.

Today I learned that our blood is not blue, even when deoxygenated [via, via].  Who'd'a thunk?!?

In short, it turns out that our venous blood is actually deep maroon... ie, not much different from arterial blood.  When vessels are close to the surface of our skin, we see the red.  ...Which is why pale folk like me look vaguely pink.  But as the vessels get deeper (to about .5 mm), some measure of absorbing and reflecting of light takes place, and you end up with a purple color.  Thus, when compared to the pink of your skin, our eyes interpret that purple as blue.

Whacky!  ...And humbling.  There are still too many urban myths that I believe, even if I have become increasingly skeptical over the years.

Monday, April 14, 2008

Design Ideas from Nature

Tonight I watched a series of videos on TED. One in particular got me thinking about how code should have more responsibility on a smaller level (the idea of the sticky foot doing everything it needs to on its own), and how failover might work (the idea of the leg acting as "foot" as needed).

Janine Benyus is not the most engaging public speaker, but it might be worth watching her speech as well, since there are some really huge ideas in there that could be applied to software. Lots of ideas of self-assembly... could data "flow" over code that captured the bits that belong there? Is this kind of what mapreduce is doing?

And this one is still one of my favorites... but that's more eye-candy than biomimicry:

Getting Ruby to Run on Tiger (as if you haven't heard that before)

Today I had the [ahem] pleasure of getting ruby to run in Tiger.

I had been using Locomotive... but that appears to be terribly out-of-date (May 2007).

I had hopes that sakuzaku's script would do the trick... but it failed in several places. Afterwards, nothing worked.  Honestly, I wasn't paying very close attention to it as it ran, and didn't crack it open to see what was going on... but that was my starting point.

I think the basic problem was that I had two other installations of ruby: one in /sw/bin/ruby, and the second in /usr/bin/ruby.  ...So, after the script failed, I went and nuked those two executables and made them symlinks to /usr/local/bin/ruby.

But my gems were still miffed (I was trying to write a very, very simple script to go grab some XML from a web-app, and I was parsing it with hpricot).  That turned out to be an easy fix:
export PATH=$GEM_HOME/bin:$PATH
export RUBYOPT=rubygems
export RUBYLIB=$GEM_HOME/lib/ruby/site_ruby/1.8/
(clearly, this is in bash) ...And that did the trick.

Yes, yes, I know: I need to upgrade to Leopard.  But I'm running on a G5, and I've heard the performance hit is considerable... so I'm dragging my feet.