A few days ago we launched Bingodisk, which is a 100GB WebDAV powered disk in the sky, with a public folder. Useful for storing just about anything and serving it up (or not) to the public.

Building Bingo! is a lot of fun, the bingodisk you’ll get is sitting on a ‘Thumper’ which is just a lovely piece of monster storage hardware and the actual frontend application is a Rails application that talks to the Thumpers via a distributed interface I wrote.

The really nice thing is that it uses a WebDAV interface which means that it’s possibly to mount it as a disk in almost every modern operating system (as usual, we had to jump through hoops to get it working properly in windows, but it does). WebDAV also supports things such as resource locking, and easily moving and/or copying resources. Now, WebDAV is a set of HTTP extensions, which means that it’s easy to talk to from an application. Let’s see how we could do that from a Ruby script using Net::HTTP. Unfortunately Net::HTTP doesn’t support Digest authentication out of the box (Basic auth won’t work with Bingo), but we can fairly easily add that, based on a snippet I found by Eric Hodel.

First you’ll need to get this file. It adds a digest_auth method to Net::HTTP.

Finding properties

To get a list of the resources available we’ll use the PROPFIND HTTP method, which will return a (rather large) chunk of XML, containing locking info, resource name and meta such as size and mtime. Here’s a script that lists the files at a given path:

  # list.rb
  require 'net_digest_auth'
  require 'rexml/document'
  include REXML

  abort("Usage #{$0} <username> <password>") unless ARGV.size==2

  ALLPROPS = <<EOS
  <?xml version="1.0" encoding="utf-8" ?>
  <D:propfind xmlns:D="DAV:">
    <D:allprop/>
  </D:propfind>
  EOS

  url = URI.parse("http://johan.bingodisk.com/bingo/")
  Net::HTTP.start(url.host) do |http|
    res = http.head(url.request_uri)
    req = Net::HTTP::Propfind.new('/bingo/tmp/', {'Depth' => '1'})
    req.digest_auth(ARGV[0], ARGV[1], res)
    response = http.request(req, ALLPROPS)

    puts "#{response.code} #{response.message}\n" 
    puts

    Document.new(response.body).elements.each("//D:response") do |r| 
      puts r.elements["D:href"].text
    end
  end

Apologies for the textile parsing error with the ARGV index

And the output:

  $ ruby list.rb johan@bingodisk.com secret
  207 Multi-Status

  /bingo/tmp/
  /bingo/tmp/mch.jpg
  /bingo/tmp/TextMateBook-beta.pdf

So what this does is that it first requests HEAD to get the things needed for the digest auth, then we create a new Net::HTTP::Propfind request instance and use the digest_auth_ method to set the user and password from the arguments given. Then fire off the request with a snippet (the ALLPROPS constant) of XML telling the DAV server we want to get all the props.

We’ll get back the “207 Multi-Status” HTTP request code and the XML describing the properties of the resources, on which it does a XPath query using REXML to get the filenames (the D:href element).

PUTting resources on the disk

Now let’s upload something, as expected we’ll want to use the PUT method, here’s a script that takes the username, password and a path for a file to upload into /bingo/public/code/:

  # upload.rb
  require 'net_digest_auth'

  abort("Usage: #{$0} <username> <password> <path/to/file/to/upload>") unless ARGV.size==3

  if File.exists?(ARGV[2])
    url = URI.parse("http://johan.bingodisk.com/bingo/")
    Net::HTTP.start(url.host) do |http|
      res = http.head(url.request_uri)
      req = Net::HTTP::Put.new("/bingo/public/code/#{File.basename(ARGV[2])}")
      req.digest_auth(ARGV[0], ARGV[1], res)
      response = http.request(req, File.read(ARGV[2]))
      puts response.code + " " + response.message
    end
  else
    puts "No such file #{ARGV[2].inspect}" 
  end

By running the script we’ll upload the net_digest_auth.rb file you:

  $ ruby upload.rb johan@bingodisk.com secret net_digest_auth.rb 
  201 Created

Nice and easy.

WebDAV might feel a bit more “bulky” than a straight up RESTful interface, but it’s really not that bad and the fact that it’s so well-supported in existing client programs is just friggin’ sweet.