MarkLogic and the Amazon API

Author: Dave Cassel  |  Category: Software Development

I’ve been working with MarkLogic Server for more than a year now, and one thing I’ve enjoyed is how easy it makes some things that used to be hard (or at least, harder). But there are some thing that the MLS API doesn’t provide. What to do? Run back to LAMP? No, no, no. Delegate. If a different language is the right tool for the job, use it.

I faced this situation recently when I wanted to use the Amazon Product Advertising API. Amazon conveniently makes tremendous amounts of information about their inventory available through their API, all in XML. Here’s one challenge, though: a few years ago, they changed the API to require a signature, using a SHA-256 hash. MLS’ API doesn’t provide a means to do that. How can we resolve this easily?

Ladies and gentlemen, for those who haven’t met, let me introduce MLJAM, described as a library for evaluating Java code. In a nutshell, it’s a simple servlet to which you can pass some Java-ish code, have it evaluated in a JVM, and get back the results. Need to calculate an SHA-256 hash? No sweat — pass that job to MLJAM and move on. This post lays out how to do it.

First, a little homework assignment. Others have described how to install MLJAM, so I’m not going to repeat that.

Second, we’ll use a little Java code to encapsulate what we need done — the signing. An important note here: to use the Amazon API, you have to sign up. Sorry, friends, not my rules, but there are benefits: this is connected to the Amazon Associates program. So when someone clicks on your link and buys something from Amazon, you get a piece of it. Good for you, good for them. Anyway, the Java code I’m going to post will require you to put in your own Access Key and Secret Key. But if you’re really interested in this post, odds are you have those or would be happy to get them. Onward. Here’s the code (modified every-so-slightly for my purposes; change .txt to .java). (Note: when this requirement was first announced, there was much wailing and gnashing of teeth. Before long, some standard signing solutions were offered in common languages, everyone had a nice cup of tea, and people returned to fretting about the competition.)

So you’ve got MLJAM set up, and you have some Java source code. Good start. To deploy, I put SignedRequestHelper.class in Tomcat’s webapps/mljam/WEB-INF/classes/com/trovz/security/ directory (matching the package name that I used). Sweet, now we have a servlet to help us send requests to the API. The last step is to call this servlet from XQuery. I put together a function that signs a request as part of a library module. Let’s take a look:

declare function amazon:sign($map as map:map) as xs:string
{
    jam:start("http://localhost:8080/mljam/mljam", (), ()),
    jam:set("url", "foo"),
    jam:set("fields", ("Service", "Version", map:keys($map))),
    jam:set("values", ("AWSECommerceService", "2009-01-06", for $key in map:keys($map) return map:get($map, $key))),
    jam:eval('
        import com.trovz.security.SignedRequestsHelper;
        com.trovz.security.SignedRequestsHelper helper = new com.trovz.security.SignedRequestsHelper();
        Hashtable hashtable = new Hashtable();
        for (var i=0; i<fields.length; i++) {
            hashtable.put(fields[i], values[i]);
        }
        url = helper.sign(hashtable);
    '),
xs:string(jam:get("url")),
jam:end()
};

Most of the Java code here consists of building a hashmap to hold the keys and values that we want to send to Amazon. The SignedRequestHelper builds up a full URL, including the signature. So to do a search against Amazon, we can use this function:

declare function amazon:lookup($asin as xs:string)
{
    let $map := map:map()
    let $_ := map:put($map, "Operation", "ItemLookup")
    let $_ := map:put($map, "ItemId", $asin)
    let $_ := map:put($map, "ResponseGroup", "ItemAttributes,EditorialReview,Images")
    let $url := amazon:sign($map)
    return xdmp:http-get($url)[2]
};

Once we get the response back from Amazon, XQuery really shines. I remember the hassle I went through parsing the responses when I implemented this stuff using LAMP. Could have been worse, but it doesn’t get much easier than handling this in XQuery.

The functions above are available in my amazon-model library (change from .txt to .xqy).

Tags: , ,

Leave a Reply