David Cassel

The Typical Course of Events

There are two main use cases for this service. The first is a Tournament Director reporting the results of games. In this case, the TD will POST a PGN game to /games. The typical course here is that the service will translate the PGN to XML, use some of the fields to construct a URL, store the game, and returns its new URL. For this first version of the service, uploading a game is the only way to add data to the database. The other main use case is someone using the service to browse the contents of the database. Here, we expect that the consumer of the service will use a proper URL and the service will return a list of games that match. In the previous post, I worked out that the search results will be paginated.

As a secondary use case, the service allows a TD to upload a correction to a PGN record. This case leads to some interesting results. Naturally, if someone uses the service to retrieve an updated game, we want to show the updated version. The interesting part is what happens to the URL. Since we are constructing the game URLs from the PGN headers, updating a game record may well change the game’s URL. For instance, suppose that a TD POSTS a game with these PGN headers:

[Event "February 2010 Octet X"]
[Site "http://redhotpawn.com"]
[Date "2010.02.17"]
[Round "1"]
[White "David Cassel"]
[Black "sagator"]
[Result "1/2-1/2"]

The game will be accessible through the URL “/games/February+2010+Octet+X/http%3A%2F%2Fredhotpawn.com /2010.02.17/1/David+Cassel/sagator”. Later, the TD realizes that this site is usually recorded with the full address, including the “www”. The service will allow the TD to PUT the PGN with the corrected header ([Site "http://www.redhotpawn.com"]) to the URL created by the earlier POST. Two things happen as a result. First, the game will now be available at a URL based on the corrected headers: “/games/February+2010+Octet+X/%3A%2F%2Fredhotpawn.com/ 2010.02.17/1/David+Cassel/sagator”. But we also need to think about what to do if someone requests the original URL. After all, someone may have conducted a search or bookmarked the URL and may try to retrieve the game. HTTP code 301 Moved Permanently is the way to go here.

These seems like a good time to point out one of the simplifying assumptions I’ve made: that there’s no need to log in. In a real system, I would limit who is allowed to POST or PUT data, and might limit search and retrieval, depending on the intended use of the service. That means I’d need to think through how to handle security, how to model my users, how to respond to unauthorized access attempts, and so forth. MarkLogic Server provides handy ways to handle all that, but it’s not what I want to focus on right now. Maybe I’ll add that to the service in a future post.

Consider What Can Go Wrong

The steps listed by the book’s authors includes trying to anticipate what might go wrong while the service is being used. A little defensive programming can go a long way. So… what could possibly go wrong?

Let’s start with an obvious case: someone requesting a URL that doesn’t correspond to anything in the database. Is it worthwhile to list something basic like this? Personally, I think so. Listing such cases makes you decide whether you should return 404 Not Found, or simply an empty list with a 200 Success (you probably wouldn’t want this, but I’ve seen it happen). If someone asks the service for a game that isn’t in the database, the service will return a 404, but a search that has no results will get a 200 with an empty <results/> node. It also helps you to remember to address those cases in the code, so you don’t accidentally end up throwing a 500 Server Error.

Another potential error is a conflict — a TD POSTing a game with PGN fields that match those of a game already in the database. If this happens, we’ll return 409 Conflict, along with the URL of the conflicting game. That way, the TD will be able to look at the game that’s already in the database. It might be that the TD accidentally POSTed the same game twice, or it may be a mistake in one of the fields. By providing useful information, the consumer of the service will be able to figure out what went wrong.

Any time you expect a certain format for the data your users send you, you need to be prepared for the data not to match the format. In the chess service case, the PGN might be invalid, or it might be valid PGN that is missing some fields we require. We’ll treat these cases the same way, responding with a 400 Bad Request and a message stating the requirement of valid PGN and a list of required fields.

One way that I like to think of how things go wrong is to glance over the list of HTTP response codes. Sometimes it helps me think of a case that I might have overlooked otherwise. Based on a look at the list, I remembered that there are only a small number of permitted methods (GET on a particular game, event, player, or search; POST on /games; PUT on a particular game). Outside of those, we’ll return 405 Method Not Allowed.

I think that about covers it. I’ve now worked through the design for my RESTful chess service, informed in a couple places by some exploratory coding. Next time, we’ll look at how to implement the service with MarkLogic Server.

I found the procedure listed out by Richardson & Ruby to be a helpful one. Even though I’ve made some simplifying assumptions for this exercise, having a set of steps helped me go about the design in an organized way.

Do you have a particular approach you use to design a service?

This is part four in a series of posts walking through the process of building a RESTful chess service. I’ve been slacking off a bit in the pace of my posts, but hopefully that will pick back up — I have a bit more travel coming up, and evenings spent hanging out in hotel rooms are good for getting posts written!

The previous post in this series laid out representations of individual games. As part of that, I picked URLs for events and players, rooted at /events/ and /players/.

I think I’ve mentioned that I intend this blog as learning-in-public sort of exercise. Here’s a case in point: I don’t like the way I set up the event URLs in my last post. I have pictured that the representation of an event would list the participating players and the games. However, I set up the URL to only mention the name of the event. The problem with that is that the name of an event (“First Saturday Quads”) does not uniquely identify a specific tournament. To do that, you need to add at least the site and the date(s) on which the tournament was played. As such, I’m now revising the URL of event to take the form: /events/<event name>/<site name>/<date range>. I’m allowing (but not requiring) a date range, because while some events are over in one day, some last for a few days, weeks, or even (in the case of postal chess) years.

Representing Events and Players

Under the use cases I’ve identified so far, there is no way to enter event data other than the game information that shows up in the PGN. An event, under my setup so far, is simply a collection of games. I’m going to run with that for now, but I can picture storing addition infomation such as the name of the Tournament Director and the tournament winner. Eventually, that would require accepting a new representation for a tournament.

Since we don’t have much data about events, the representation won’t be that complex. Here’s the representation for /events/February 2010 Octet X/http%3A%2F%2Fwww.redhotpawn.com/2010.02.17-2010.04.01:

<event>
  <name>February 2010 Octet X</name>
  <site>http://www.redhotpawn.com</site>
  <date-start>2010-02-17</date-start>
  <date-end>2010-04-01</date-end>
  <players>8</players>
</event>

Likewise, what we know about players comes from the PGN information about individual games. For our player representation, we’ll give the name and what we know about the player — not much at this point.

<player>
  <name>Cassel, David</name>
  <counts events="2" games="100"/>
  <rating type="highest">1546</rating>
  <rating type="latest">1542</rating>
</player>

One more thing before we move on. I mentioned earlier that if we wanted to add, say, the Tournament Director to the representation of an event, we’d have to allow a consumer of this service to PUT a new representation. But notice that what’s contained in the representations right now is extracted from the actual game data. What would we do if someone uploaded a representation that contradicted what we knew from the game data? That’s an application-specific decision; we could either decide to overwrite what we saw in the game PGN data, or we could decide it is involatile. If we go with the latter approach, we could disregard contradicting information or trigger an error. Since I’m not going to allow PUTs to Event and Player resources, I don’t have to worry about it — but when you’re designing your service, watch out for gotchas like that and think through what should happen. (That’s step 9 in our authors’ procedure, for those keeping track.)

Representing Search

I’m going to offer two ways to search. The first will look pretty normal. I’ll use a URL like this: /search?q=. Is this RESTful? How does “search” count as a resource? Simply enough: search is an algorithm; the results of that algorithm are a resource, complete with a representation.

This is a free text search. Results may include games, players and events.

/search?q=quads
<results>
  <game>/games/First+Saturday+Quads/West+Chester+Chess+Club/2000.04.01/1/Joe+Abner/David+Cassel</game>
  <game>/games/First+Saturday+Quads/West+Chester+Chess+Club/2003.02.01/1/Joe+Demetrick/David+Cassel</game>
  <game>/games/First+Saturday+Quads/West+Chester+Chess+Club/2000.04.01/1/David+Cassel/Charles+Jay</game>
  <game>/games/First+Saturday+Quads/West+Chester+Chess+Club/2000.04.01/1/David+Cassel/Fred+Austin</game>
  <event>/events/First+Saturday+Quads/West+Chester+Chess+Club/2000.04.01</event>
</results>

We can see now how the decision to use transparent URLs is helpful: we can present these results a user, who will be able to determine something about them without having to load up the contents of the individual documents. But wait, there’s more! Let’s also set up search by path variable: we’ll let the user specify the parts they know about and use “any” for the remainder. We’ll only apply this style to game searches for now. I’ll also disallow wildcards, but you can probably see how that would be a useful extension. Let’s look at an example:

/games/First+Saturday+Quads/West+Chester+Chess+Club/any/David+Cassel/any

This URL will return a list of all games in which I played white in the West Chester Chess Club’s First Saturday Quads event, regardless or date or opponent. Of course, in a large database this could yield a lot of results, especially if you used “any” for a lot of fields. So let’s add two more fields in order to allow for paging: page number and page size.

/games/First+Saturday+Quads/West+Chester+Chess+Club/any/David+Cassel/any/1/10

<results>
  <game>/games/First+Saturday+Quads/West+Chester+Chess+Club/2000.04.01/1/Joe+Abner/David+Cassel</game>
  <game>/games/First+Saturday+Quads/West+Chester+Chess+Club/2003.02.01/1/Joe+Demetrick/David+Cassel</game>
  <game>/games/First+Saturday+Quads/West+Chester+Chess+Club/2000.04.01/1/David+Cassel/Charles+Jay</game>
  <game>/games/First+Saturday+Quads/West+Chester+Chess+Club/2000.04.01/1/David+Cassel/Fred+Austin</game>
  <!-- six more <game/> results, then: -->
  <page>1</page>
  <link rel="next">/games/First+Saturday+Quads/West+Chester+Chess+Club/any/David+Cassel/any/2/10</link>
</results>

Now we’re asking for the same set of results as before, but we want the first page of 10 results. The <page/> element toward the bottom identifies the page and the <link/> element shows how to get to the next page. After the first page, the service would also provide a <link rel=”prev”/> element. This is part of being RESTful: a representation of a resource provides links to other things.

At this point, I’ve walked through the first seven steps of the process. The remaining steps are to consider the typical course of events, and to consider error conditions. I’ll pick that up in the next post. After that, it will be time to start whipping up some code!

In part 2 of this series, I defined the data set and resources for my service. I’ve been following the procedure listed in RESTful Web Services for how to lay out the service. Here are those steps again:

1. Figure out the data set
2. Split the data set into resources

For each resource:

3. Name the resources with URIs
4. Expose a subset of the uniform interface
5. Design the representation(s) accepted from the client
6. Design the representation(s) served to the client
7. Integrate this resource into existing resources, using hypermedia links and forms
8. Consider the typical course of events
9. Consider error conditions

Naming the Resources

I’ve finished the first two steps, and in doing so I identified Events, Players, Games, and search as resources that I want to expose through the service. Let’s take a look at the Games resource first. I need to decide how I will build the URI for a game. Let’s take another look at the PGN headers for my sample game again:

Of these fields, I believe you would need to specify Event, Site, Date, Round, White, and Black to be sure you’ve uniquely identified a game. This would cover cases where two players are playing a series of matches, with multiple games in the same day (I think this would be an unusual case, but it’s best to be sure). I can picture a couple different ways to identify this game with a URI. A very transparent way to do it would be to include each piece needed for uniqueness as a path variable:

/games/First+Saturday+Quads/West+Chester+Chess+Club/2003.02.01/1/Joe+Demetrick/David+Cassel

Another approach, which is a bit more compact, is to take the hash of the needed values and use that:

/games/372fa3184dc56e2910cc91195baccb4b

If I could count on each game having a game id field, I could just use that, of course, but that does not appear to be a widely used field, so let’s think about the two options I’ve laid out. Clearly, the first is more informative to a human reader. That’s handy, but there is something I’m not wild about with this scheme: the slashes typically imply a hierarchy; here, no such hierarchy is really there. The authors use other punctuation to deal with cases where there isn’t actually a hierarchy. For instance, when they establish URIs for maps, they separate the latitude and longitude values with commas instead of slashes. The slash-based structure implies some hierarchy where none really exists.

On the other hand, the hash certainly doesn’t imply an hierarchy. In fact, it’s completely impenetrable: all you know from looking at the URI is that it identifies one particular game. Let’s think about how this service might be used. Picture doing a search and getting back a list of games. If you’re building an application that will present this list to the user, you won’t want to just give a list of hashes. You’ll want your user to have some way to decide what links to click. Using the first approach gives a lot of information about the game. Using the second approach, conversely, before a system could present a list it would need to retrieve each of the individual games in order to get the basic metadata. Because it will be more helpful for building applications around the service, I’m going to go with the more descriptive URI.

Exposing a Subset of the Uniform Interface

The Uniform Interface refers to the set of HTTP methods. The most commonly used are GET, POST, PUT, and DELETE. The protocol also supports HEAD, OPTIONS, TRACE, and CONNECT, but my service will not support any of these. In a nutshell, GET is used to retrieve a resource’s representation, POST and PUT are used to create and update resources, and DELETE removes a resource. For the use cases I’ve described, there is only one way data changes through my service: a tournament director uploads a game. The way I will support this is to let the TD POST a game to /games/. Doing so will return the URI of the game. I will also allow a TD to PUT to a particular game URI, replacing the previous resource, as a way of correcting mistakes, as well as DELETE to simply remove a game.

I want to let users retrieve games, of course, or this service wouldn’t be all that useful. As such, I’ll support GET on a game to return a representation of that game. Here, I’ll throw in a little wrinkle: I want to offer the PGN version of the game, as that’s the standard representation, but I also want to offer an XML version, to be machine friendly. One of the hallmarks of a RESTful service is links between various resources. PGN doesn’t typically have links, but my XML representation will. With this in mind, I’ll make one change the URI used to retrieve a game: I’ll add an extension, which must be either .xml or .pgn.

Designing the Representation Accepted from the Client

Although my service will offer two representations for a game, I’ll only accept one from the client: the PGN standard. My service will expect to find the fields used to construct the URI; any other fields will be stored too.

Designing the Representations Served to the Client

The PGN representation needs no design work: when requested, a PGN representation will follow the standard format. The XML representation will take a little bit of thought, however.

<game>
  <headers>
    <Event>First Saturday Quads</Event>
    <Site>West Chester Chess Club</Site>
    <Date>2003-02-01</Date>
    <Round>1</Round>
    <White>Joe Demetrick</White>
    <Black>David Cassel</Black>
    <Result>1/2-1/2</Result>
    <WhiteELO>1337</WhiteELO>
    <ECO>D13b</ECO>
  </headers>
  <moves>
    <move num="1" color="w">d4</move>
    <move num="1" color="b">d5</move>
    <move num="1" color="w">c4</move>
    <move num="1" color="b">c6</move>
    ...
  </moves>
</game>

So far, this looks like an XMLized version of the PGN, with the one exception that I changed the date from 2003.02.01 to 2003-02-01. This simple change is to make the date a valid XML date. I’ll make a couple more changes in the next section.

One last comment before we move on, however. I took a quick look for XML representations of chess games. It seems none have caught on, simply because PGN has been so successful at capturing the information. So why am I bothering? Two reasons: 1) to provide the links that a good RESTful service needs to connect the resources, and 2) because for this exercise, I want to show multiple representations.

Integrate the Resource into Existing Resources

One of the key elements of a RESTful service is links to guide a consuming application from resource to resource. PGN doesn’t provide a way to do that, but our XML representation can. Let’s add a few changes:

<game>
  <link rel="alternate" type="application/x-chess-pgn"
    href="/games/First+Saturday+Quads/West+Chester+Chess+Club/2003.02.01/1/Joe+Demetrick/David+Cassel.pgn"/>
  <headers>
    <Event>
      <nameFirst Saturday Quads</name>
      <link uri="/events/First+Saturday+Quads"/>
    </Event>
    <Site>West Chester Chess Club</Site>
    <Date>2003-02-01</Date>
    <Round>1</Round>
    <White>
      <name>Joe Demetrick</name>
      <link uri="/players/Joe+Demetrick"/>
    </White>
    <Black>
      <name>David Cassel</name>
      <link uri="/players/David+Cassel"/>
    </Black>
    <Result>1/2-1/2</Result>
    <WhiteELO>1337</WhiteELO>
    <ECO>D13b</ECO>
  </headers>
  <moves>
    <move num="1" color="w">d4</move>
    <move num="1" color="b">d5</move>
    <move num="1" color="w">c4</move>
    <move num="1" color="b">c6</move>
    ...
  </moves>
</game>

For this step, I needed to jump ahead a bit and define the URIs for events and players. In most organizations, players would likely be given a unique id, which I would certainly want to use in the players URIs to avoid name collisions. Since PGN does typically use names, though, I’m going to stick with that for this exercise.

This post is getting a bit long, so I’m going to wrap this one up. In the next post, I’ll take a look at the representations for events and players as well as looking at seach.

I recently got a copy of RESTful Web Services, a book in the O’Reilly series that I read some time ago. I first read it when I was introduced to REST as an architectural style. It helped me get my head around a number of the concepts.

I decided that I want to refresh my memory about some aspects of designing RESTfully and to explore how to build a RESTful service using MarkLogic Server, so this post is actually the introduction to a miniseries. The book has a suggested method for going through the design steps, so I’ll walk through those steps and the implementation over a few posts. I invite you to read along and see how the process unfolds.

First things first: I need to decide what need I’m trying to satisfy. Being a chess player, I decided to implement a service that would be used by a chess organization such as the US Chess Federation, FIDE or the Free Internet Chess Server. These groups hold lots of tournaments, some of which are major international events while others are small events held by community chess clubs. In all cases, the tournament director needs to report the results to the organization, which will record the results and update players’ ratings.

Besides reporting, my service will support searching. The service should provide representations of players, events, and individual games. We’ll return player and event information as XML, but we’ll give users the choice between XML and PGN, the standard for chess notation, for individual games. We’ll need some search capability, in addition to browsing.

RESTful Web Services gives a 9-step Generic ROA (Resource Oriented Architecture) Procedure:

1. Figure out the data set
2. Split the data set into resources

For each resource:

3. Name the resources with URIs
4. Expose a subset of the uniform interface
5. Design the representation(s) accepted from the client
6. Design the representation(s) served to the client
7. Integrate this resource into existing resources, using hypermedia links and forms
8. Consider the typical course of events
9. Consider error conditions

In this post, I’ve introduced the series and laid out roughly what I want to do. Next time I’ll tackle the first two steps in the process. See you then!