Catalyst Model #3: Cover images via Amazon.com’s APA

Published · Wednesday, 15 July 2009 (Updated · 7 March 2010)

This one is going to get much more involved than the first two. We’re going to introduce a couple of things we’ve skipped so far: configuration variables and calls for data external to the application.

It isn’t always obvious but most all models are external to your application. If you are using a database, you are dealing at a minimum with the file system and probably with a server via some client code. A DBI call to MySQL is doing things outside your application. That database can be thought of as a service and there is effectively no difference between this kind of local service and a web service. Only the transport layer, through local sockets or over TCP, differs.

The moral: don’t think about models as part of your application. Think about them as services your application will consume.

To run this example, you will need an Amazon.com developer web services account. Many of the services, including the ones in the APA—Amazon Product Advertising API—are free. You can sign-up for your account here—http://aws.amazon.com/. Then you’ll need to sign-in and find the access identifiers page with this info–

Access Key ID and Secret Access Key

In the model configuration below, secret corresponds to the “Secret Access Key” and key to “Access Key ID.”

This excellent little utility is the entire engine for our model to interact with Amazon’s APA web service: URI::Amazon::APA. Well, that and XML::LibXML to handle the service’s responses.

Here is the documentation for the API. You won’t need to know it for the example to work but you’ll certainly be able to modify it and mess around with it better if you do. Amazon often breaks these URIs… If this doesn’t work, let me know and I’ll update it.

Install the required modules

URI::Amazon::APA, LWP::UserAgent, XML::LibXML, XML::LibXML::XPathContext, and HTML::Entities.

cpan URI::Amazon::APA HTML::Entities LWP::UserAgent

You may need to get libxml installed to do the next batch. It’s worth it. It is fast, reliable, and fun once you get the hang of it.

cpan XML::LibXML XML::LibXML::XPathContext

Back to the Catalyst

First, please kill the conf formatted file if you didn’t already while following the set-up. I really dislike this format. It’s harder to read and edit than YAML and makes certain simple things rather difficult. The only defensible reason the core Cat devs moved to the config general format is that YAML is entirely whitespace sensitive and Pod doesn’t allow whitespace to be respected correctly which makes copy/paste quite error prone which makes supporting it into a helpdesk chore when newbies adopt Catalyst. So, be aware, the YAML needs to be exactly as shown.

rm myapp.conf

Then open up a YAML config file and edit the model configuration.

emacs myapp.yml

This is what you need to add–

---
Model::Amazon::APA:
  service_uri: http://ecs.amazonaws.com/onca/xml
  service: AWSECommerceService
  operation: ItemSearch
  search_index: Books
  associate_tag: apv-20
  response_group: Large
  signature:
    key: YOU_MUST_GET_YOUR_OWN_FROM_AMAZON
    secret: ditto/thisizrlyasekretushudnotnaodig

Anything included here will be passed to MyApp::Model::Amazon::APA->new as its second argument. Ends up executing something like this to create your model–

$hash_ref = { service => AWSECommerceService, …et cetera… };
MyApp::Model::Amazon::APA->new('MyApp',$hash_ref);

The new is done automatically in the background because we’ve subclassed Catalyst::Model. You can override or extend the new method but don’t. The right way, generally, is to use Catalyst::Model::Adaptor instead. We’ll cover that in an up-coming article.

Anything inside the hash ref is available in the model object (hash based) as regular data keys. So, $self->{operation} will return ItemSearch; by default, it’s regular Perl data so it can be overwritten.

Note, that is my associate_tag in there so only include it if a) you don’t have one of your own to include, b) you don’t mind potentially sending users to Amazon—the code doesn’t do that as it stands—whose purchases give referral fees that buy me a cup of coffee now and then.

Create the new model

./script/myapp_create.pl model Amazon::APA
created "/Users/jinx/depot/sites/MyApp/script/../lib/MyApp/Model/Amazon"
 exists "/Users/jinx/depot/sites/MyApp/script/../t"
created "/Users/jinx/depot/sites/MyApp/script/../lib/MyApp/Model/Amazon/APA.pm"
created "/Users/jinx/depot/sites/MyApp/script/../t/model_Amazon-APA.t"

Fill the model up with goodies

Inside the model we have access to all the configuration information we put into myappl.yml.

emacs lib/MyApp/Model/Amazon/APA.pm
package MyApp::Model::Amazon::APA;
use strict;
use warnings;
use parent 'Catalyst::Model';
use URI::Amazon::APA;
use LWP::UserAgent;
use XML::LibXML;
use XML::LibXML::XPathContext;
use HTML::Entities qw( encode_entities );
use Carp;

sub covers_for_title {
    my $self = shift;
    my $title = shift || croak "You must provide a title string";
    my $mode = shift || $self->{search_index}
        || croak "You must provide a mode, e.g., Book";
    my $uri = URI::Amazon::APA->new($self->{service_uri});
    $uri->query_form(
                     Service       => $self->{service},
                     Operation     => $self->{operation},
                     AssociateTag  => $self->{associate_tag},
                     ResponseGroup => $self->{response_group},
                     Title         => encode_entities($title),
                     SearchIndex   => $mode,
                     );

    $uri->sign( %{ $self->{signature} } );

    my $ua = LWP::UserAgent->new;
    my $r = $ua->get($uri);

    my $doc = XML::LibXML->new->parse_string($r->decoded_content);
    my $xc = XML::LibXML::XPathContext->new($doc);
    $xc->registerNs('amzn', $doc->getDocumentElement->namespaceURI);

    if ( $r->is_success )
    {
        return map { $_->textContent }
            $xc->findnodes('//amzn:MediumImage/amzn:URL');
    }
    elsif ( my @error = $xc->findnodes('//amzn:Error') )
    {
        my $error = join("", map { $_->textContent } @error );
        die $error, "\n";
    }
    else
    {
        croak("Unexpected error in ", __PACKAGE__,
              " request to ",
              encode_entities($uri), "\n\n",
              encode_entities($r->as_string)
              );
    }
}

1;

You can see that, even though there is an external web service call and XML parsing, this is simple and straightforward. It’s not a good design template for a real model. We’d use a lot of the same code but it would be broken up differently. You can see in this snippet that xpath queries are going to be a large part of the data functionality.

return map { $_->textContent }
    $xc->findnodes('//amzn:MediumImage/amzn:URL');

Once this model ends up covering a large amount of the services that the APA provides we’d have at least a dozen model methods just like this which duplicated half of its code in each. If we didn’t generalize the data gathering we’d probably end up with a few hundred. It would be stupid to do the same error checking, the same XML namespace set-up, the same request procedure and so on; DRY.

This is nice for a tight example but if you were doing the model for real you could abstract the data fetch out to xpath definitions and the page fetching, error checking, and URI signature into master utility methods. Any given data fetching method would then be simple and could easily take advantage of caching or being repurposed or even subclassed.

Make a controller to consume the model’s data

./script/myapp_create.pl controller Cover
 exists "/Users/jinx/depot/sites/MyApp/script/../lib/MyApp/Controller"
 exists "/Users/jinx/depot/sites/MyApp/script/../t"
created "/Users/jinx/depot/sites/MyApp/script/../lib/MyApp/Controller/Cover.pm"
created "/Users/jinx/depot/sites/MyApp/script/../t/controller_Cover.t"
emacs lib/MyApp/Controller/Cover.pm
package MyApp::Controller::Cover;
use strict;
use warnings;
use parent 'Catalyst::Controller';
use HTML::Entities qw( encode_entities );

sub index :Path :Args(0) {
    my ( $self, $c ) = @_;
    $c->response->body('Matched MyApp::Controller::Cover in Cover.');
}

sub search: Local Args(1) {
    my ( $self, $c, $title ) = @_;
    my $safe_title = encode_entities($title);
    my $images = join "\n",
            map { qq{<img src="$_" alt="$safe_title" />} }
        $c->model("Amazon::APA")->covers_for_title($title);

    $images ||= "No images found searching on \x{201C}$safe_title\x{201D}";

    $c->response->content_type("text/html; charset=utf-8");

    $c->response->body(<<"");
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en-US" xml:lang="en-US">
<head>
<title>Book Covers!</title>
<style type="text/css" media="screen">
  img { margin: 25px; border: 1px solid #aaa; }
</style>
</head>
<body>
  <p>$images</p>
</body>
</html>

}

1;

Fire it up and try some searches

./script/myapp_server.pl -d -r -p 3000

Tomorrow we’ll do Catalyst Model #4: Random numbers and really-o, truly-o random numbers.



digg stumbleupon del.icio.us reddit Fark Technorati Faves

« Catalyst Model #2: Moon phase data · Catalyst Model #4: Random numbers and really-o, truly-o random numbers »
« 10 Catalyst models in 10 days1 »