In my last post, I introduced a Google gadget that I wrote to display graphs of Pachube datastreams. This gadget takes advantage of the Pachube API to add a few new features that I couldn’t find in other Pachube gadgets, like sizable graphs and support for local time zones.
In this article, I’d like to explore parts of the Javascript code in the gadget, and describe how they interface with the Pachube API.
Viewing the Source Code
As with any iGoogle Gadget, the source code can be viewed by clicking the “View source” link in the bottom right of the gadget’s iGoogle Gadget Directory page – my gadget’s page is here.
However, this will only show you the XML definition for the gadget, since I’ve put the Javascript in a separate file. My gadget has 2 different views defined in the XML file, normal and “canvas” (i.e. maximized), as you can see at the bottom of the XML file. Using a separate Javascript file allows the 2 views to use the same code. Here are direct links to the XML and Javascript files.
- XML: http://www.gigamegablog.com/gadgets/gm-pachube.xml
- Javascript: http://www.gigamegablog.com/gadgets/gm-pachube.js. (It’s OK to click on it. Since this Javascript can’t execute outside of Google’s gadget container, your browser will just display the source code instead of running it. Or, you can just right-click the link and save it as a file, you big baby.
)
The Pachube API
Pachube’s API documentation is quite good overall, but it can be a little difficult to find the details for a particular feature.
The graphing API I’m using, for example, is in the “Read datastream – GET v2/feeds/<feed_id>/datastreams/<datastream_id>” section. It’s just barely there, actually – a passing reference to the all-important PNG parameter, along with a table of parameters you can pass to the API. (A better way to learn the graphing API is to use Pachube’s Graph Builder – see the next section).
There are a lot of options for setting the time range of a graph, including some that my gadget doesn’t support, such as a specific end date. These options are described in the “Historical Queries” section tucked away at the end of that documentation page.
To get a list of datastreams in a feed, I use the “Read feed” API, described here. As you can see from the JSON example in that section, this is a very handy API that returns a lot of information about the feed and each datastream.
An API key must be passed with most API calls, even when accessing Public feeds. The mechanism for passing the key is described on a separate page, the API Overview. That page also describes the timezone parameter. You’ll find more information on the API key and time zones later in this post.
Pachube Graph Builder
The best way to learn the API syntax for generating a graph is Pachube’s Graph Builder. To display it, go to your Feed’s page on Pachube’s web site, then click on the gear icon in the lower right of the graph for a datastream, as shown below.
The Graph Builder page allows you to interactively try out various graph parameters, then copy a URL that calls the API to generate that graph.
Pachube Graph API
The URL that you’ll get from the Graph Builder will look something like this:
https://api.pachube.com/v2/feeds/37080/datastreams/JadePlant.png? width=500&height=300&colour=%238024f1&duration=3hours &title=Jade Plant Moisture&show_axis_labels=true &detailed_grid=true &timezone=Eastern Time (US & Canada)
){kind=link}
All Pachube API URLs have the same basic layout, following the REST standard:
- https://api.pachube.com/v2 – All API URLs start with this. (You can use the non-secure http:// prefix if your coding platform doesn’t support https, such as an Arduino.)
The V2 refers to Version 2 of the API. Version 1, which uses a different syntax, is “deprecated” (i.e. still supported, but use is discouraged). Some of the examples you’ll find on the Web use the V1 API: if there is one /v2 in the URL, it’s a version 1 call. - feeds/37080 - All API URLs for a specific feed will identify the feed # this way.
- Datastreams/JadePlant.png – All API URLs which deal with a specific datastream will identify it using its ID field (in this case, “JadePlant”). The “.png” part is what tells the API that you want a graph. If you replace that with “.xml”, you’ll get back XML data about your datastream instead.
- ?<parms> – The parameters that follow the question mark are specific to the API call you are making.
Pachube’s API is pretty easy-going about these parameters: if you pass ones that don’t apply to the API call, or even ones that don’t exist at all, the API will just ignore them.
If you’re familiar with URLs, you might have noticed that the above URL example (copied from the Graph Builder) contains an invalid parameter: the &timezone parameter contains spaces and an ampersand that need to be properly “escaped” when including in a URL. The API doesn’t tell you this, it just ignores the timezone parameter. (And, as a result, the X axis of the graph shows UTC times, the default, rather than EST times).
API Keys
I have to admit that I don’t have a firm grasp on which APIs and feeds require authentication. I’m tempted to just write “always pass an API key – it can’t hurt”. But that’s way too simple for this blog, so here’s my best guess explanation of how it works…
If you tried pasting URLs from the Graph Builder into your browser, you probably noticed that no login was required, and no API key is being passed. As far as I know, an API key is never required for a graph of a public datastream.
However, the other API used by the gadget, which retrieves a list of datastreams for a specified feed, does require authentication, even if the feed is public. If you have a neglected browser from which you’ve never logged into Pachube (that blue E icon will probably do), try accessing the following URL
http://api.pachube.com/v2/feeds/37080.json? timezone=Eastern%20Time%20%28US%20%26%20Canada%29
You should get a pop-up authentication prompt.
Private feeds always require authentication, even for a graph. For example, the following should definitely display an authentication prompt, unless you are a nefarious hacker.
http://api.pachube.com/v2/feeds/39460.json? timezone=Eastern%20Time%20%28US%20%26%20Canada%29
If your application were to try doing the same thing, it would get a “404 – Authentication Error”. When trying to graph a private datastream without an API key, the application gets back a nice “Server error” graphic, instead
To avoid that, the code needs to pass an API key. The gadget doesn’t ask you to fill in an API key of your own: instead, I’ve hard-coded one that has read access to public feeds, but update access to nothing (including my own feeds).
The key is passed in an HTML header field, as specified in the Pachube API documentation. Various programming languages have various methods for setting the contents of the header field – the Google Gadgets approach is described in their documentation, and shown below:
var params = {};
params[gadgets.io.RequestParameters.HEADERS] = {
"X-PachubeApiKey": "<your API key>"
};
Incidentally, if you’re using the gadget to graph datastreams in a private feed, you might wonder how my API key has access to your private feed. Relax, it doesn’t. Through some kind of under-the-covers-cached-browser-authentication-cookie-machination (my words, not Pachube’s), the gadget is allowed to access your private feed on a browser in which you’ve previously logged into Pachube. I’ve noticed intermittent “authentication failed” errors when using the gadget with my own private feed, presumably when my cached authentication expires (or the cookie-machinations go awry).
Time Zones
According to the API documentation on time zones :
There are two places where you can specify the time zone: in your user profile on the website and via a parameter in an API request.
I think the intention is that your user profile’s timezone setting is automatically used by the API unless overridden, but it currently doesn’t work that way for graphs. Perhaps that’s a bug that will be fixed at some point, but for now you have to pass the timezone parameter to the graph API call, or the X axis will display UTC times.
I had originally planned to have a “UTC offset” field in the gadget settings, where you would fill in the # of hours difference between your time zone and UTC: for example, -5 for Eastern Standard Time.
However, when testing I noticed something odd: timezone=-5 gave me UTC minus 5 hours, as expected, but timezone=-4 gave me UTC minus 3 hours.
As described in this post in the Pachube Community forum, if you set the timezone parameter to a numeric value, the API will select the first matching entry in Pachube’s (rather eclectic) list of geographic time zone settings.
For example, timezone=-4 is the same as specifying “Atlantic Time (Canada), and timezone=-5 is the same as specifying “Bogota”. Rather than being dependent on the local customs of Atlanteans and Bogatians, I gave in and included the full list of geographic timezones in the the gadget’s Edit Settings page. (If you have no idea which one on those time zones you are in, ask Pachube, not me.)
As shown in the Graph Builder section above , another tricky thing to consider when working with timezones is proper encoding of non-URL characters, like spaces and ampersands. Javascript makes this easy: the gadget’s code for handling the conversion is simply:
// replace And with & (Google's Gadget API hates ampersands)
timeZone = timeZone.replace(" and ", " & ");
// use escape function to encode spaces and ampersands in URL
urlSuffix = "timezone=" + escape(timeZone);
One last pothole to avoid: Google’s Gadget API seems to have a particular distaste for ampersands, however they’re encoded: the dropdown list of timezones wouldn’t display until I took them out.
Getting the List of Datastreams in a Feed
Pachube makes it easy to get the list of datastreams in a feed: just one API call returns a variety of properties, including the timestamp and value of the last datapoint received by each datastream.
The Javascript code for doing this is shown below, most of which is from the gadget’s getDatastreamStatus function:
var urlPrefix = "https://api.pachube.com/v2/";
var urlSuffix = "timezone=" + escape(timeZone);
var url = urlPrefix + "feeds/" + feedID + ".json" + "?" + urlSuffix;
var params = {};
params[gadgets.io.RequestParameters.CONTENT_TYPE] = gadgets.io.ContentType.JSON;
// NOTE - this API key has READ access only
params[gadgets.io.RequestParameters.HEADERS] = {
"X-PachubeApiKey": "nBGUIT63Onke5-sfOoARTvKvkwl45b85wjz5PKDIZkc"
// refresh data if longer than 5 minutes since last request
makeCachedRequest(url, getDatastreamsResponse, params, 300);
Yes, that’s an actual, genuine API key that I’ve published – my own, in fact. As mentioned earlier, this one is defanged, limited to read-only access to public feeds.
The “.json” in the URL specifies that the data is to be returned in JSON format: other options are XML and CSV. Your choice of format depends primarily on the coding platform you’re using. Google’s Gadget API supports both XML and JSON, but I went with JSON because it is more light-weight (i.e. faster), a consideration if you’re working with a lot of datastreams. The CSV option is intended for platforms that don’t support XML or JSON, such as an Arduino.
The code for makeCachedRequest is taken directly from Google’s API documentation. The code allows you to override Google’s default caching algorithm for gadget data, which retrieves updated data from the source (i.e. Pachube in this case) every 60 minutes, regardless of how often the user clicks the browser refresh button. I’m overriding the interval to be every 5 minutes.
Note that the gadget doesn’t automatically update every 5 minutes: the update only occurs when the user first opens the web page containing the gadget, or clicks the refresh button, or when iGoogle refreshes the gadget. (iGoogle refreshes each gadget once an hour, if the iGoogle page is in the foreground tab). As far as I know, your gadget code has no way of forcing a refresh to occur.
The response to the API request is returned to the callback function passed as a parameter to makeCachedRequest, a simplified version of which is:
function getDatastreamsResponse(obj)
{
if (obj.data === null || typeof obj.data === "undefined")
{
// something’s wrong – check the other obj properties, like obj.error and obj.rc
. . .
}
jsonData = obj.data;
if (!jsonData.datastreams)
{
displayErrorMsg("No datastreams found in feed");
return;
}
datastreamList = jsonData.datastreams;
feedTitle = jsonData.title;
feedStatus = jsonData.status;
feedDesc = jsonData.description;
The meat of the response is contained in the .data property of the parm passed to the callback function. If that property is empty, then an error occurred. Generally the error information is found in obj.error and/or obj.rc. The former contains an error message as a raw JSON text string, and the latter contains one of the standard HTTP return codes as listed in Pachube’s documentation.
The data object is bristling with useful information about the feed and datastream. As shown in the code, I’m getting the feed’s title, status and description from there, as well as a list of datastreams, each of which is itself a JSON object with properties. The screenshot below shows some of the other properties in this data object as seen in the Google Chrome Javascript console. For a full list of all of the available properties, see the example in Pachube’s Get Feeds documentation.
Getting the Datastream Graphs
The Javascript which handles the calls to the Graph API, displayGraphs(), basically just pieces together the URL for the Graph API, as explained in the “Pachube Graph API” section above.
There are a couple of issues handled here that I didn’t describe earlier.
Unbeknownst to the user (wouldn’t want to worry their pretty little heads, would we?), the code scales down the graph if necessary to fit within Pachube’s limit 300K pixels.
var MAX_GRAPH_SIZE = 300000; // Pachube's maximum graph size, in pixels
. . .
var defaultWidth = 320; // in normal gadget mode, 320 is the recommended width
if (graphHeight * graphWidth > MAX_GRAPH_SIZE)
{
// set to maximum size that will fit within Pachube's limit
if (graphWidth > defaultWidth)
{
graphWidth = defaultWidth;
}
graphHeight = Math.floor(MAX_GRAPH_SIZE / graphWidth);
}
This is done somewhat crudely, basically yanking the graph width back to 320 pixels, then yanking the height down as needed to fit within the 300K.
Also admittedly quite crude is the support for increasing the graph size when the gadget is maximized.
// in canvas mode, double the graph size
if (gadgets.views.getCurrentView().getName() === "CANVAS")
{
graphHeight *= 2;
graphWidth *= 2;
defaultWidth *= 2;
}
I’m a little surprised that most gadgets ignore the maximized setting altogether, just displaying the same content in the same cramped size. I think the problem is one of “discoverability”, to use a Googley term: the feature is disabled by default, and Google’s documentation on the feature confusingly insists on referring to it as “canvas” mode.
As shown in the API documentation, enabling “canvas” mode is just a matter of pasting some boilerplate XML settings into your gadget, like so:
<Content type="html" view="home"> <![CDATA[ <div id="content_div"></div> . . . <script src="http://www.gigamegablog.com/gadgets/gm-pachube.js" type="text/javascript"></script> ]]> </Content> <Content type="html" view="canvas"> <![CDATA[ <div id="content_div"></div> . . . <script src="http://www.gigamegablog.com/gadgets/gm-pachube.js" type="text/javascript"></script> ]]> </Content>
Perhaps Google would prefer that your gadget take on an entirely new form when maximized, since the XML allows for a separate block of Javascript for canvas mode (the “content_div” section). I wasn’t that ambitious, so instead I use a script tag to feed the same Javascript code into both the normal and canvas mode XML definitions, then changed the code to check which mode it’s in, as shown in the earlier code sample.
Wrapping It Up
If you’ve read the Programmer’s Show and Tell for my Nimbits gadget, you’ll notice that this time I’ve done much less ranting about how frustrating Google’s Gadget API is to work with. The API hasn’t changed, but once you know where the potholes are, the drive is a lot smoother. If you intend to write your own gadget I’d suggest you read my earlier rant/article first, since it points out a lot of those potholes.
I’m still dubious about the future of Google gadgets given Google’s newfound emphasis on monetization and focusing on core products. The iGoogle Developer Blog hasn’t had any new posts since a May post asking “Did you know we are continuously adding new features” ? (In fairness, they were referring to the Gadget Dashboard – they stopped adding new features to the gadget API long before May).
Methinks the best browser-based platform for making use of the Pachube API lies elsewhere, perhaps a Google App Engine project like this one. . The App Engine would give your code a lot more elbow room, and there is so much more to the Pachube API than what I’ve covered here.