119 Labs

More and more sites are using a variety of architectures, and corresponding backend services, that are putting a lot of pressure on local development setup. Fortunately there’s a lot of work going on in the devops space to help this out.  I basically wanted to be able to created a local dev environment, develop, and then push up live and know that my environment would be the same without futzing about.

I started going down the path of using Vagrant for a dev environment and configuring with Chef.  I was really hoping it would all just work, unfortunately I wasn’t so lucky.  Before we get to that though, lets develop a simple test app that we can test out deploying:

I’m not going to add any functionality to this app right now, but it will show as the basis for our deployments.

Chef

Chef is where I ran into the most trouble. Opscode has extensive documentation, but it is definitely geared towards their hosted environment. While their hosted environment seems great, its more well suited to a team trying to automate their infrastructure not a lone developer trying to get things going off the ground. I decided to go with Chef Solo. Fortunately I also found the great gem librarian. Librarian is bundler for your chef recipes and it works as advertised. To get librarian setup in your project run the following:

Vagrant

To get started, Ryan Bates has an excellent screencast on Vagrant. I’m going to present a short version, but if you’re interested in a little more detail I highly suggest you check it out.  Vagrant uses virtual box for virtualization.  In order to get going first go visit virtual boxand download the latest version. As of this writing I have version 4.1.10 installed. Next from a cmd line:

This takes care of all the background work required to have vagrant up and running.

Pulling it together

So we have a rails project with chef dependancies managed by librarian & vagrant. Now we want to be able to actually deploy a minimal rails stack to a virtual box. First thing linux (our deploy target) does not come with a javascript environment so make sure uncomment “gem ‘therubyracer’” in your Gemfile. Also, since we’re using unicorn uncomment “gem ‘unicorn’” while you’re there.

Now lets setup our chef dependancies. These are managed through the Cheffile. Edit your Cheffile so it looks like this:

Now just run librarian-chef install to download those deps to your cookbook directory

You might notice the last dependency, rails-lastmile. I created this cookbook to pull together the other cookbooks: unicorn, ngnix, etc. Over time I plan on added a lot more to this. It will also be the way I keep a consistent deployment structure between Vagrant, EC2, and others. More on this in a future post.

Next lets configure Vagrant to use our dependencies. Open up Vagrantfile in your favorite editor and put in the following:

That should be all you need to do to get ready to deploy. Just run vagrant up and watch the magic happen. The rails application is being hosted in the virtual environment using your code base. Click here to see it. Since the code base is actually on your system, as you edit files you’ll be able to see the changes live in the hosted environment. The only exception being dependancies. If you need to rely on a new dependency run vagrant reload to have the vagrant restart and pull in new bundled dependancies. Next up I’ll be looking at both adding some more infrastructure (like MongoDb) and deploying to EC2.

I’ve gotten a lot of great feedback from everyone on my previous post. Unfortunately the utilities aren’t currently working properly. First on the path to tracking down the issue was to fix up the source repository. The source repository on source forge has been abandoned for years. I’ve applied my patches to the code base and uploaded them on github (forks welcome).

If you checkout the source from github you should be able to type make and have a fresh copy of the class files & libs. After some digging around I’ve been able to reproduce the issues by running the following command:

LD_LIBRARY_PATH=. java $THREADS -classpath jusb.jar usb.view.Foo

This (attempts) to run the jusb test app, usb viewer. Currently it fails for me with a control read error. I’m pretty sure that the issue is that between versions of OSX the OS APIs have changed. I’m currently digging in to that particular code and I welcome any feedback.

Along with OSX 10.7 Lion Apple dropped support for PowerPC applications. My one last holdout was the multitouch utilities I use to change settings on my fingerworks touch stream lp. The mtu (multitouch utility) is actually a java app with a native wrapper. So first step was to run the java app directly. After a little poking in the config files I came up with the following script:

Save the script to your /Applications/Fingerworks directory and remember to chmod it. Now I could run the app, but it still couldn’t see my touch stream.  The issue was the jusb that came with the finger works installation. Jusb is an open source library that hasn’t been updated in years.  This thread put me in the right direction. In order to upgrade jusb first you need to check out the source. Run the following from terminal:

After that you should be able to run the utilities again

I was looking to DRY up my page titles but was having some trouble with Rails 3 view contexts. Here’s what I finally ended on:

Just place this method in your application_helper.rb. This automatically looks for an entry in your translation store based on the path of the view template which you call the helper from. So if you you have a template under user_sessions/new.html.erb like this:

And an entry in en.yml like this:

It will automatically generate an H1 containing your translated text.

I’m working on an app that makes heavy use of bookmarklets. There are some really great resources already for creating your one bookmarklets. For more information look here, here, or just search the web.

One of the issues I’ve had to deal with is that many of my bookmarklet screens are using standard parts of my site just in a slightly different format.  Rather than duplicating controller logic or adding if statements everywhere I decided to add bookmarklets as a declared format.

The First step was to setup my custom format.  For me it was really just an alias for js (html might work better for you in your app). So I started by creating an initializer to setup the type alias:

Make sure to restart rails server in order to see this change. Now if we visit a regular page and add the format like this http://localhost:3000/user_sessions/new.bookmarklet You should see a template missing error.  Add the file user_sessions/new.bookmarklet.erb and you should see your custom response. Now at this point you can create different template files accommodating for your different view size, layout, etc. Still in order to drive this change you need to hard code the format in to the views which are posting/linking to your custom views. In order to DRY that up I added a before filter to automatically infer the format based upon referrer.

Here I use the bookmarklet_url, which is my base path as the test whether I should be formatting for embedding in the bookmarklet or not.

From here any action that I call from my bookmarklet I just add a.bookmarklet.erb to my views folder. Along with this I make heavy use of partials and have very little duplication between my views.

I was trying to run cucumber with profiles under rails when I got the following error:

“You have already activated builder 3.0.0, but your Gemfile requires builder 2.1.2. Consider using bundle exec. ”

Looking into it this seems to be a common problem not just limited to builder.  Option #1 according to stack overflow is:

bundle exec cucumber

That works fine, but I was looking for something more concise.  Fortunately bundler comes with an option to help out. If you add the binstubs option to your install command it will create a directory bin in your rails root that contains all executables from gems that are installed. So after running:

bundle install —-path .bundle/ —-binstubs

I can run

bin/cucumber —-profile wip

from my rails root. Not bad, but I force of habit I kept just typing cucumber (which on my system is under /usr/bin) and I was back to square one. So the final step I edited the /usr/bin/cucumber file so that it detected the <rails_root>/bin/cucumber and ran it if it existed. Here’s the new /usr/bin/cucumber:

And that’s it. Now cucumber runs using the bundle environment if it exists, otherwise it falls back to default behavior.

A while ago I had the need to add image dimensions (height & width) to my paperclip attachments. After a lot of research I came across this:

https://gist.github.com/343678

It did the job OK, but had a few problems:

• I hated adding the my_style attribute
• Dimension calculation was flaky and often produced ghost failures

So instead I rewrote the caching part to listen for paperclip processing events and use introspection to determine attachments and styles. I also took a suggestion (thanks Win) and moved it into an initializer.  Here’s the updated file:

https://gist.github.com/827827

In order to get this working you need to add a couple lines to the model that currently has the Paperclip::Attachment

https://gist.github.com/827533

Seems to work much more reliably now. Also, notice the lack of my style attributes. One more note, if you’re adding dimensions to your model where they weren’t before, make sure to add a <attachment_name>_dimensions column to your model (should be type :text on sql)

Compojure is a web framework for clojure. It is built on top of ring and provides a good starting point for web based clojure apps. It is not a direct replacement for a stateful web app framework like rails or spring webflow. Instead it allows you to quickly connect clojure code to the web and is ideal for building backend systems which can communicate over http.

It took me a bit of working around since compojure 0.4.0 came out to get it to work with a simple hello world project. Thanks to the incanter folks for getting me most of the way there.  First off if you’re working (or just playing) with clojure then leiningen is really a must.  It using a combination of maven & ant to manage dependancies and build your projects. First you can get started by installing lein using the instructions on their website.

Installing lein

Next from the command line make a new project:

lein new helloworld

Cd into your new project directory and take a look around. The main files we’ll be dealing with are project.clj and the src directory.  Open up the project.clj file. It should look something like this:

(defproject helloworld "1.0.0-SNAPSHOT"
  :description "FIXME: write"
  :dependencies [
      [org.clojure/clojure "1.1.0"]
      [org.clojure/clojure-contrib "1.1.0"]
])

If you’ve worked with Maven before this should look pretty familiar . First thing we need to do is add a couple of dependancies. Edit the file so it looks like the following:

(defproject helloworld "1.0.0-SNAPSHOT"
:description "FIXME: write"
:dependencies [[org.clojure/clojure "1.1.0"]
[org.clojure/clojure-contrib "1.1.0"]
[compojure "0.4.0"]
[ring/ring "0.2.3"]])

(ns simple_web_app
(:gen-class)
(:use compojure.core, ring.adapter.jetty)
(:require [compojure.route :as route]))

(defroutes webservice
(GET "/" [] "<h1>Hello World</h1>")
(route/not-found "<h1>Page not found</h1>"))
(defn -main [& args]
(run-jetty webservice {:port 8080})
)

(ns simple_web_app
(:gen-class)
(:use compojure.core, ring.adapter.jetty)
(:require [compojure.route :as route]))

(defroutes webservice
(GET "/" [] "<h1>Hello World</h1>")
(route/not-found "<h1>Page not found</h1>"))

(defn -main [& args]
(run-jetty webservice {:port 8080})
)

:main simple_web_app

(defproject helloworld "1.0.0-SNAPSHOT"
:description "FIXME: write"
:dependencies [[org.clojure/clojure "1.1.0"]
[org.clojure/clojure-contrib "1.1.0"]
[compojure "0.4.0"]
[ring/ring "0.2.3"]]
:main simple_web_app)

I’m all for using TextMate (or hopefully soon Ultraedit for osx) for straight rails dev. I’m starting on a project that uses both clojure and rails so I wanted something that would play nicer for cross-language development. My favorite IDE from the Java side of things is by far IntelliJ IDEA. They have a relatively new version out with ruby support so I thought I’d give it a go setting up a jruby based rails project. It ended up being a little more complicated than just download and install so here you go:

First head over to the jetbrain’s website and download a demo copy of IDEA 9. Make sure to get the ultimate edition since ruby dev is disabled in their community edition. After that snag the most recent version of jruby which as of this writing was 1.4.0. In order to follow along exactly you’ll want the straight bin version, but with a little modification to paths any version should do.

I keep all of my (non-maven) java libraries in /Library/JavaDev just swap that out for whatever directory you use.

Place the downloaded jruby-bin in /Library/JavaDev/

cd /Library/JavaDev/
curl http://jruby.kenai.com/downloads/1.4.0/jruby-bin-1.4.0.tar.gz > jruby-bin-1.4.0.tar.gz
tar xzvf jruby-bin-1.4.0.tar.gz

make sure your user has permissions to execute files in the bin directory. As a matter of good house keeping, I also like to setup a symlink

ln -s jruby-1.4.0 jruby

next lets see where ruby is currently running from type

which gem

It most likely is running under /usr/bin/gem. We want to change that temporarily so we can update the gem system. Lets change our path and update the gem system

export JRUBY_HOME=/Library/JavaDev/jruby
export PATH=$JRUBY_HOME/bin:$PATH
which gem

the last command should now read /Library/JavaDev/jruby/bin/gem. If at any point you now close your terminal window you’ll need to run the last three commands again to make sure you’re working with the right gem base. Time to update the gems on your system:

gem install rails
gem update system
gem install activerecord-jdbcsqlite3-adapter
gem install jdbc-sqlite3
gem install activerecord-jdbcmysql-adapter

You will probably see a message “JRuby limited openssl loaded. gem install jruby-openssl for full support.” As long as this is just a development station don’t worry about it.

Now on to IDEA. Open the dmg and copy IDEA to your application folder. Run the app and create a new Ruby project. Once you get to the point that you need a project SDK click configure and add. The file IDEA is looking for is $JRUBY_HOME/bin/jruby. It will detect the lib folders from there.

Last step make sure you’re using the appropriate sql adapter in your database.yml file. For sqlite your dev should look something like:

development:
adapter: jdbcsqlite3
database: db/development.sqlite3
pool: 5
timeout: 5000

Notice the adapter is jdbcsqlite3 rather than the standard sqlite3

That’s it. If you want to use a db other than sqllite, you’ll need to install the appropriate gem. IDEA will warn you & let you fix it if you forget to do this so no big deal. For reference:

• OSX 10.6.2
• Jruby 1.4.0
• Rails 2.3.5
• Idea Ultimate 9.0.1

I love pbpaste and pbcopy. They are utilities on osx that allow you to write to and read from the copy buffer. They really have unlimited use in productivity scripts for transforming information. My most frequent use though is when I want to open a new tab in Terminal and stay in the same directory. In order to do this in Terminal I:

1. type “pwd | pbcopy”
2. Apple-T
3. cd <paste>

This is even snazzier with a touchstream.

I used to run in to the problem all the time where some one would send me a unc style path and I needed to open it up on my mac. To get around this I wrote the following script:

### unc2smb.sh ###
#!/bin/sh

pbpaste|sed -e "s,[\\][\\],smb://," -e  "s,[\\],/,g" -e "s,\(.*\)/\([^/]*\),\1/,"|pbcopy

Now just copy your unc style (\\server\folder) path and run unc2smb.sh. Now when you use a standard paste, you’ll instead paste the osx friendly smb style (smb://). One last step, I added unc2smb.sh to quciksilver. Now when I get a unc style path in an email I:

1. Select & copy the path
2. run unc2smb.sh from the quicksilver command menu (or you could bind it to a key)
3. Connect to server
4. <Paste>

Still not a great process, but better than I had before. Anyways combining pbpaste, pbcopy, & sed can really give you a huge amount of control