The Diminishing Returns of Long Hours

A lesson I learned during my years in university and had to re-learn once or twice while working in Silicon Valley is that working long hours is beneficial, but only for a very short period of time. One or two 60 hour weeks is enough, as is even a single all-nighter. When you’re sleep deprived, you don’t make good decisions, and software development is all about good decisions. There is never one solution to a problem. A good engineer picks one of the best solutions through creativity and experience. Sleep deprivation dulls the ability to do either well.

In Don’t be a Hero, Alex Payne makes the additional argument that an individual working these long hours is a sign that your culture has a problem.

Work/life balance is important. These days, I only work long hours if it’s because I made a mistake. Everything I do is, in part, an effort to avoid this: fair and realistic estimates, regular communication with my customers, and not slacking off early in a project.

Bash Substitutes for “seq” and Simple “sed” Expressions

There are two things I find myself doing from the shell just infrequently enough that I can’t remember the exact syntax. The first is generating a sequence and the second is repeated string transformation.

“seq” is a little GNU utility that generates sequences of numbers. I used to find this installed automatically by many Linux distributions, but lately (on Debian and Ubuntu, at least) it’s missing and I haven’t been sufficiently motivated to find it. No need: Bash has something good enough built in.

To generate the numbers from 1 to 10, inclusive:

$ echo {1..10}
1 2 3 4 5 6 7 8 9 10

The second thing is repeated string transformations, for example renaming a set of files to use a different file extension. This used to mean re-reading the sed(1) man page yet again and doing the transformation in a backtick expression. Again, no need: Bash can do this itself:

$ for f in a.foo b.foo c.foo; do echo $f ${f/%foo/bar}; done
a.foo a.bar
b.foo b.bar
c.foo c.bar

The syntax is ${parameter/pattern/replacement}. pattern only replaces the first match, unless it starts with /, in which case it replaces all matches. It can also start with # to force a match at the beginning of the string or % to match at the end.

Take a look at “Parameter Expansion” in the Bash manual for several other useful operations.

On the Pain of Developing for Facebook

I’m in the last stages of building a site that integrates with Facebook. It includes a Facebook Connect component and a separate Facebook iframe application. In the hopes that some of my experience will help someone else out there, I will share some of my suffering and how I worked through it.

This is a Rails 2.3 application, using the Facebooker gem, version 1.0.54.

Safari, iframes, and cookies.

Chances are high you’ve found this page via search, and that you are desperate to find a solution to allow your Rails sessions to carry over from page #1 of your iframe application to page #2 and beyond. Will Henderson has a solution when your Rails application uses the ActiveRecord session store, but new Rails applications use the cookie store. There is no session to retrieve from the database; passing a row ID with links doesn’t help.

I am fortunate that my iframe application has a splash screen, with only one useful link on it, to start the “real” application. I append every fb_sig parameter from the current request to the link. Once the user clicks that link, the iframe no longer falls under Safari’s 3rd party cookie policy, and so the next page can set cookies. Because all of the fb_sig parameters are there, Facebooker gives precedence to them, recreating the Facebook session without consulting cookies (which either don’t exist or are stale).

<%= link_to 'Start the app', params.merge(:action => :app) %>

Now, your app doesn’t always get these. When the application is first installed, you won’t get the fb_sig parameters. In this case, I force a redirect:

unless params[:fb_sig_ss]
  redirect_to "http://#{Facebooker.canvas_server_base}/#{Facebooker.facebooker_config['canvas_page_name']}/"
  return
end

However…

Facebooker’s Rack middleware trashes your params.

Facebooker includes a piece of Rack middleware that automatically validates the signature on fb_sig parameters, and aborts the request if the signature isn’t valid. This works great on the initial request, but fails on the second request, with exactly the same parameters.

The culprit is near the bottom of Facebook#call, in rack/facebook.rb, where it calls convert_parameters!. This method is trying to be helpful, turning things like Unix time_t into Ruby Time instances. If you are trying to pass these parameters along in a link, Time.to_s is not a Unix time_t, and so the signature validation will fail, and you’re screwed.

Pre-Rails 2.3, there is no Rack middleware, and signature validation happens in another part of the Facebooker gem, which does not trash the original params. It does the same conversion, but on a copy. As near as I can tell, calling convert_parameters! is unnecessary, and commenting it out solves my problem. When Safari loads page #2, Facebooker creates its session, which can be stored in the Rails session, using the cookie store.

Reconstituting a Facebook session in Flash.

The bulk of this iframe application is actually written in Flash. Every Facebook application has an App ID (essentially a public key) and a secret key. The secret key must never be allowed outside your organization or people can fake API requests from your application. Embedding your secret key in a Flash SWF is therefore not an option, because a SWF can be decompiled. Facebook’s solution to this is the session secret, passed as fb_sig_ss.

According to Adobe’s ActionScript 3.0 Client Library for Facebook API reference, one can use the FacebookSessionUtil class to recreate a session using only the public key and the session secret. This is not true. I have to pass in all of the fb_sig parameters to Flash in order to make it work. However you start your Flash content, FlashVars must include:

<%= params.collect { |k,v| "#{k}=#{v}" if k.starts_with?("fb_sig")}.compact.join("&") %>

A refreshed page lags a “logged-in” state change until refreshed a second time.

When a user logs out of Facebook and refreshes your Facebook Connect web site, it doesn’t reflect the user’s “logged-in” state change until another refresh. You can automate this and have the page refresh itself when necessary:

<% init_fb_connect 'XFBML', :app_settings => '{"reloadIfSessionStateChanged":true}' do %>
  FB.ensureInit(function() {
    FB.Connect.get_status().waitUntilReady(function(status) {
      switch (status) {
        case FB.ConnectState.connected:
          $$('.fbConnectLoggedIn').invoke('show');
          $$('.fbConnectLoggedOut').invoke('hide');
          break;
        case FB.ConnectState.appNotAuthorized:
        case FB.ConnectState.userNotLoggedIn:
          $$('.fbConnectLoggedIn').invoke('hide');
          $$('.fbConnectLoggedOut').invoke('show');
          break;
      }
    });
  });
<% end %>

There are actually two parts to this trick. First, :app_settings => '{"reloadIfSessionStateChanged":true}' triggers the automatic refresh.

Second, if you have some common part of your layout that switches on a user’s Facebook status, this precludes your use of page caching. For an otherwise static page, this is a real sacrifice. To work around this, I include both sets of mark-up, classed with either “fbConnectLoggedIn” or “fbConnectLoggedOut”, and invoke Prototype’s show() or hide() on them as part of the Facebook Connect initialization. As long as you are careful to use XFBML tags that don’t require actual values from a Facebook session (e.g. <fb:name uid="loggedinuser" ...>), this works very well.

New conditionally_cache Method for Fragment Caching

I just pushed a couple of updates to Banker. The boring one adds the missing “should_not” variants of the existing Shoulda macros.

The more interesting one is a conditionally_cache method for fragment caching. If you use pagination to split content across pages, this might be very useful for you. Pages 2 and later are rarely loaded, falling very quickly to the tiniest sliver of people that see page 1.

conditionally_cache adds a new first argument to the regular cache method. This argument is evaluated in a boolean context. When true, control is passed to the cache method as usual, performing fragment caching. When false, though, the block is captured and output as if the caching code were not there at all.

So instead of having no caching on your paginated results or wrapping your cache call in an ugly if statement, instead you can do this:

<% conditionally_cache params[:page].blank? || params[:page] == '1', 'page1' do %>
  ... usual HTML markup goes here ...
  ... this will be cached on page 1, but not elsewhere ...
<% end %>

Since pages 2+ are not cached, you need only expire the ‘page1’ fragment, and nothing else.

Rails Bug with render :text => proc in Tests

There is a bug in the current stable release of Rails, 2.3.4, when using render :text => proc { ... }. The Proc object is never called by the test framework, which limits what you can test. For example, the following two techniques will both fail:

  • Using assert_select or similar methods to check the content of the response.
  • Setting expectations with a mocking framework on methods that are called within the Proc body.

Lighthouse bug #2423 includes my reworked patch to fix this issue. A couple more people verifying the patch will help get it committed.

Xcode: iPhone Project Dependency on Mac OS X Project

This week, while building an iPhone application for a client, I wanted to run a custom Mac OS X command-line utility during the build phase of the iPhone project. I set up the command-line utility as a dependent project of the iPhone project, but it won’t build, telling me:

target specifies product type ‘com.apple.product-type.tool’, but there’s no such product type for the ‘iphonesimulator’ platform

There is no perfect solution to this problem, but here’s my workaround: rather than let Xcode build the dependent project for me, I simply added a call to xcodebuild as part of my custom script phase. As long as the input and output files for this phase are set up properly, the entire phase is skipped when the output file is up-to-date, and xcodebuild is pretty fast when the target binary is already built, too.

My final script phase looks like this:

(cd SubProject; xcodebuild -configuration Release)
SubProject/build/Release/SubProjectCommand
The Details of Parallel Processing Are the Hardest Part

Mike Ash has an absolutely excellent post up today discussing the importance of the details when designing an application for parallel processing.

His post focuses on using Grand Central Dispatch (part of Mac OS X 10.6 “Snow Leopard”), but the core point is valid for any approach to multiprocessing: it’s not just a simple matter of queueing jobs. You have to consider the impact of those jobs on the system as a whole.

Running Sweepers from a Model

Oh, the pain. Over the last 24 hours I have fought an exhausting battle with Rails and the testing environment to do a couple of seemingly simple things:

  1. Expire cached pages and fragments from outside the context of a normal HTTP request
  2. Test it.

Testing, in particular, is particularly difficult because Rails does not offer any built-in mechanism to test an application’s caching, and I’ve had problems in the past with the only plug-in that does it (cache_test) it on Rails 2.x. I’ve released a new plug-in, called Banker, that provides assertions and the necessary support to test caching, including Shoulda macros.

The first item has come up for me more than once. Complex applications often have scheduled jobs that make changes to the database. If the application does any caching, there is a good chance that these jobs will affect content that is cached. The problem is that expiring caches from outside the context of a controller + request is a pain. Here is the solution:

Create your sweepers as you normally would. Then, either within test code or your script/runner code:

def setup_cache_sweepers(*sweepers)
  sweepers = sweepers.flatten
 
  ActiveRecord::Base.observers = sweepers
  ActiveRecord::Base.instantiate_observers
 
  returning ActionController::Base.new do |controller|
    controller.request = ActionController::TestRequest.new
    controller.request.host = URL_HOST
    controller.instance_eval do
      @url = ActionController::UrlRewriter.new(request, {})
    end
 
    sweepers.each do |sweeper|
      sweeper.instance.controller = controller
    end
  end
end

URL_HOST is a constant, defined in each environment file, with the host:port part of a URL. It is needed in order to generate URLs, and more importantly, fragment cache keys, outside the context of an HTTP request.

The method returns a controller. For unit test code, hang on to it, because it gives you access to named routes:

@controller.send(:users_path)

For code run from script/runner, nothing else is needed. You’ve already instantiated your sweepers and given them the controller instance, so anything you do to a model instance that generates a callback to the sweeper will use that controller, including calling cache expiration methods.

Manage vendor/rails with Git on a Subversion Project

Here is something I’ve been experimenting with over the last week or so, and it’s working out very nicely so far:

Use Git to manage vendor/rails when your project is using Subversion.

Here’s how:

First time freezing? Easy:

  1. cd vendor
  2. git clone git://github.com/rails/rails.git
  3. svn add -N rails; svn ps svn:ignore .git rails
  4. cd rails; git checkout v2.3.3.1 (or whatever version you want)
  5. svn add *

Switching to a different version of Rails is now as simple as:

  1. cd vendor/rails
  2. git checkout master
  3. git pull origin master
  4. git checkout whatever
  5. svn add `svn st | grep ^\? | cut -f7 -d" "`
  6. svn rm `svn st | grep ^\! | cut -f7 -d" "`
  7. svn commit

If you already have a vendor/rails and you want to try this technique out, you should first clone the Git repository to a temporary directory, git checkout the version that matches what is already in your vendor/rails, then copy (or move) the .git directory into vendor/rails. Add the svn:ignore property as above and you’re all set.

Here’s why I’m doing this: when a new Rails release comes out, some files are changed, some added and others deleted. Because Subversion litters a checked-out project with its .svn directories, you can’t just delete the entire thing and re-freeze without losing local patches (yes, I’ve had to do this) and history (which isn’t terribly important, but can be nice). Even ignoring those two reasons, completely deleting and adding vendor/rails will cause your Subversion repository to grow more than necessary (Rails 2.3.3 checks in at 35 MB).

Git, by putting everything in a top-level .git directory, makes itself easy for Subversion to ignore. Checking out a tag to switch to a different release is simple, and Git deletes files, unlike applying a patch, which truncates deleted files to 0 bytes, requiring a find to actually remove them.