Working with large (thus slow) assets

You have a website with fantastic images that are big, plentiful, and full of color.  Like a fashion website, for example.  Or maybe you’ve decided that all the text on your website needs to be in a non-web font–as images–and now you have hundreds or thousands of image files to be deployed.

What to do so as to avoid heinous deploy times robbing your project of momentum and preventing following the credo of good development, “release early, release often?”

Make a separate repo just for your assets.

To accomplish this, you need the repo, you need a cap recipe and callback-hook for doing these deployments on demand (you wouldn’t want to do them every time, or you’re defeating the whole point), and you need a couple symbolic links.

STEP ONE: MAKE A REPO

Just follow Github’s directions for starting a new repo.  Once you’re all done, you’ll need to put your assets into the directory and make your first push.  Since there’s lots of these assets (that’s the whole reason you’re doing this project), it might take a while.

STEP TWO: CAP RECIPE AND CALLBACK-HOOK

For my project, and probably for yours, its important to keep separate asset repos for each environment, in the same way that you’d keep separate code repos for each environment. Are you already using capistrano multistage? I am, so these directions assume that piece. But you should be able to figure it out if you’re not.

config/deploy/staging.rb:

set :statics_deploy_to, Proc.new { "#{deploy_to}/statics-staging" }

config/deploy/production.rb:

set :statics_deploy_to, Proc.new { "#{deploy_to}/statics-production" }

Some thought went into the placement of the directory. Should it mimic the releases structure of a normal rails deployment? I decided that for my purposes it made the most sense to have a directory which didn’t change with releases such that updates could literally be git pulls instead of whole new clones. Since the file sizes are so large, I do want to avoid doing a complete deployment as much as possible.

And now your special recipe … of note here, the command variable inspiration was from the update_repository_cache method in capistrano’s recipes/deploy/strategy/remote_cache.rb and the run block was stolen from scm_run in capistrano’s recipes/deploy/strategy/remote.rb. I tried to use a slightly more abstracted API, Strategy, instead of using the Git scm API directly, but couldn’t get things to work (and since this already did, I didn’t try too hard).

lib/recipes/statics.rb:

set :statics_symlink_target, Proc.new { "#{current_path}/public/images" }
set :statics_repository, Proc.new { "git@github.com:USERNAMEHERE/PROJECTNAMEHERE-statics.git" }
set :statics_branch, 'master'

after 'deploy:symlink', 'statics:symlink'

namespace :statics do
  desc "Deploy the statics"
  task :deploy do
    gitter =
      Capistrano::Deploy::SCM::Git.new(:repository => statics_repository,
                                       :branch => statics_branch)

    # figure out the latest version of the code
    revision =
      gitter.query_revision(statics_branch) { |cmd| run_locally(cmd) }

    # clone or fetch depending on what we're up to
    command = "if [ -d #{statics_deploy_to} ]; then " +
      "#{gitter.sync(revision, statics_deploy_to)}; " +
      "else #{gitter.checkout(revision, statics_deploy_to)}; fi"

    run(command) do |ch,stream,text|
      ch[:state] ||= { :channel => ch }
      output = gitter.handle_data(ch[:state], stream, text)
      ch.send_data(output) if output
    end
  end

  desc "Dynamically link the statics into place"
  task :symlink do
    run "ln -nfs #{statics_deploy_to} #{statics_symlink_target}"
  end
end

STEP THREE: SYMBOLIC LINKS

There’s already a symbolic link that automatically gets made on the server when you deploy to connect your static assets to your code. But you have to do one locally, too. I’ll leave that as an exercise for the reader (hint, its pretty much the exact same ln -nfs line that’s in the recipe).

AND FINALLY

future-book:~/Sites/clientservice matt$ cap production statics:deploy deploy

GOTCHA: Github only allows one deploy key per-repo and now the same account on your hosting server needs to access two repos.  My approach was to setup the deploy key connected to the account of the repo owner, not the actual repo itself.  Multiple Github Accounts is another approach.