A Nimbits Gadget: Programmer’s Show And Tell

Updated June 4: I’ve corrected a couple of things, as marked in blue.

Updated Dec 3: I made a small code change to fix the autoscale feature, as marked in green below.

In my last post, I gave a user’s overview of a Google Gadget I wrote to display graphs from the Nimbits data logger.

This post is directed at developers. I’ll explain how the gadget interacts with the Nimbits API and the Google Charts API, and give a few tips to those who are getting started with Gadget development.

Actually, my tips would only be useful to gadget noobies, since I’m a gadget noob myself. Fortunately, Google’s Gadget development framework is pretty easy to get started with. This is partly due to the abundance of sample code – you can examine the source code for any gadget by clicking the “View Source” link near the bottom right of the gadget’s page in the Google Gadget directory.

However, another reason why the Gadget framework is easy to learn is that there isn’t that much to it. The limitations can be frustrating, and it doesn’t seem like the framework is under active development by Google any longer.

That’s fine with me, though: the gadget is just the container, and the contents are supplied by the Nimbits and Google Charts APIs, both of which are very much moving forward.

Viewing the Source Code

If you click on the “View source” link for the Nimbits Gadget , the first thing you’ll notice about the code is that there isn’t any. The link displays the Gadget’s XML definition. The JavaScript code is often embedded in the XML file (that’s actually what Google recommends for better performance), but I decided to put it in a separate .js file to avoid duplicating the code in “canvas” view.

So, the code is actually here: http://www.gigamegablog.com/gadgets/nimbits.js. (Yeah, I know your Mother told you not to run strange JavaScript, but if you click this link it will just display the code, not run it.)

Most of the JavaScript code is a series of functions that build up 2 calls to REST APIs:

The Nimbits Chart API

A good introduction to the Chart API is Benjamin Sautner’s blog post. . His post gives a couple of examples of calls to the Chart API, and the 2nd one is very similar to the one that I ended up using in the gadget:

 

http://app.nimbits.com/service/chartapi?point=TempF&email=bsautner@gmail.com&cht=lc&chs=200×100&chxt=x,y&chxl=0:|Apr|May|June|1:||100+F

 

If you aren’t familiar with REST APIs, then you might wonder why this looks like a URL. That’s the whole idea of REST: API calls are URLs with all of the parameters embedded in the URL string. Ideally, the URL’s parameters tells you (and, most importantly, the server) everything you need to know to handle the request, with no dependencies on previous requests, etc.

Authentication aside, if you and I paste a REST URL string into a browser, we should both get the exact same result. (And, for the purposes of this gadget, I basically tossed authentication aside – that’s why it only works with Public data points).

If you paste the above URL into a browser, you’ll get back a graph in a .png file. The Nimbits Charts API will always return a PNG. This makes it easy to use trial-and-error to experiment with the Charts API. Other Nimbits APIs, and most REST APIs in general, return XML – we’ll see an example of that in the CurrentValue API, below.

Before I get into the code I used in the gadget to build the chart…

How To Build A Nimbits Gadget in 5 Minutes

Mine took a little longer, actually, but displaying a Nimbits chart in a gadget is quite simple:

1. Open the Google Gadgets Editor: basically a Notepad that saves files on one of Google’s servers.

Replace the “Hello, world!” line with


<img
src="http://app.nimbits.com/service/chartapi?point=TempF&email=bsautner@gmail.com&cht=lc&chs=200x100&chxt=x,y&chxl=0:|Apr|May|June|1:||100+F"
/>

2.Click File, then Save. In the Save File dialog, change the filename from helloworld.xml to nimbits.xml. It should now look like this

3.Click File, then Publish. The Gadget Editor will nag you about some of the things that are missing, but that’s OK because you’re only going to publish it to your own iGoogle page, not Vanity Fair. Click OK.

4.The “Publish Your Gadget” dialog will be displayed. Click on the “Add to my iGoogle page” option.

5.Another page is opened, with a big blue “Add Hello World! To iGoogle” button. Click it.

6.If all goes well, you should see a brand new gadget on your iGoogle page, as below.

If you jumped ahead and started changing the settings in the Gadget Editor to see what would happen then, hey, sorry about that whole nervous breakdown thing. I should have mentioned that Google automatically caches your gadget’s code, so your changes won’t be automatically applied to your iGoogle page. There is a way to turn that off – see the Caching section below.

Your shiny new gadget will faithfully report the temperature in Benjamin Sautner’s aquarium. But what if you actually wanted your gadget to display something else? That’s where the code comes in…

Nimbits Chart API Options

The code block below is taken from the displayChart() function of the gadget’s JavaScript. It builds an HTML img tag similar to the one you pasted into your gadget, but getting its parameters from variables instead.

The variables are loaded from settings in the User Preferences dialog. I won’t cover that code here, since this section of Google’s Gadget Developer docs tells you all you’ll need to know.

html += '<img src=http://' + serverName + '/service/chartapi?'
  + 'points=' + pointsParm;
html += '&email=' + userEmail;
html +=  '&chds=a&cht=lc&chco=' + colourParm + '&chxt=y&chf=bg,s,EFEFEF';
html += '&chs=' + graphWidth + 'x' + graphHeight;
if (spanType == "Hours")
{
	html += "&st=-" + timeSpan + "h&et=now";
} else
{
	// # of readings
	html += "&count=" + timeSpan;
}

 if (displayLegend)
{
	// display it at top listing all data points
	html += '&chdl=' + legendParm + '&chdlp=t';
}
var gridStyle = "&chg=";
if (displayGrid == "Vertical")
{
   gridStyle += "10,0,1,0";
} else if (displayGrid == "Horiz")
{
	gridStyle += "0,10,1,0";
} else if (displayGrid == "Both")
{
	gridStyle += "10,10,1,0";
} else
{
	// no grid
	gridStyle = "";
}

Let’s break this block of code into parts

html += '<img src=http://' + serverName + '/service/chartapi?'
			  + 'points=' + pointsParm;

Every call to the Nimbits Charts API needs a server name and at least 1 DataPoint. If you want to include multiple Data Points, they must be comma-delimited. If your Data Point names include spaces, they must be replaced with plus signs (which is a Googly standard for handling spaces in URLs). For example, if there are 2 data points named “Plant Light” and “Home Meter 2”, then pointsParm is set to “Plant+Light,Home+Meter+2″.

html += '&email=' + userEmail;

Unless the user has already been authenticated (logged into iGoogle isn’t enough), the e-mail address is a mandatory parameter for the Nimbits REST API calls. Surprisingly, there is no longer a way for a Gadget to automatically determine the user’s Gmail address.

html += '&chds=a&cht=lc&chco=' + colourParm + '&chxt=y&chf=bg,s,EFEFEF';

Autoscale is a very useful setting – so useful that you can’t turn it off in my gadget.  Update Dec 3: Well, autoscale *was* a very useful feature, until it stopped working.  I’ve replaced it with the Google Chart API’s “&chds=a” parameter, which appears to have the exact same effect.

It tells Google Charts to automatically scale and label the Y axis based on the value of the points you are graphing. Without it, the graph is always plotted on a scale of 1 to 100, regardless of the actual value of the data.

For example, here is a graph of 2 Data Points with very different scales: one for a relatively low-power set of devices (DuinoFarm) and one for a big fat PC.

Here is the same graph with autoscale turned off. The scale defaults to 1-100, and data points above 100 are cut off.

The &cht parm is another mandatory one: it specifies the type of graph, with “lc” meaning line chart. Leave it out, and you’ll get the default “polar bear in a snowstorm” chart. There are a bunch of other types of graphs available, as shown in the Google Chart Gallery, but line charts are how I ride.

The &chco parm specifies the colours to be used for the lines in the graph, assigned in the same order as the data points in the points parm. The values are RGB hex codes: for example &chco=FF0000,00FF00, for red and green. Without this parm, everything is orange. (Well, Google being Google, probably whatever colour they feel like using that day. Today, it’s orange).

The &chxt is another one that I always use: it provides the vertical scale. Without it, the graph looks like this:


The last parm in this section, &chf=bg,s,EFEFEF, is responsible for that lovely gray background you see in the screenshots.

html += '&chs=' + graphWidth + 'x' + graphHeight;

This one’s pretty self explanatory. The actual value used in the gadget looks something like &chs=320×200, with the 320 part being hard-coded. The Chart API is quite happy to serve up PNGs that are much larger than this, so if you are displaying a graph outside of a gadget container (such as the “canvas” view displayed when you maximize a gadget) then you can go crazy with megapixel monstrosities.

if (spanType == "Hours")
{
    html += "&st=-" + timeSpan + "h&et=now";
} else
{
    // # of readings
    html += "&count=" + timeSpan;
}

Unlike the last few parms, this one is a Nimbits API parm, not Google Charts. It tells Nimbits which set of data point values to pass along to Google Charts. I’m using a relative timespan, where the end point is always now, and the start point is whatever offset the user specifies in the Settings panel (in hours). The Nimbits API supports a lot of other formats for the start and end time parms, allowing you to chart any time period you like, as documented in the TimeSpan page of the Nimbits Wiki, and a more flexible time span setting is definitely on the To Do list for this gadget.

The remaining settings in the code are self-explanatory, controlling the legend and grid used in the graph.

The resulting tag is plopped into the gadget’s content area, where it gets magically replaced with a PNG of the actual graph. It really is magical, since the response received from REST APIs are usually not so easy to handle.

Which brings us to the Status line…

The Nimbits CurrentValue API

The Status line displays the data point name, its most recent value, and the date and time it was received. All of that information is returned by the Nimbits CurrentValue API. This API’s main purpose is to record new values, not return existing ones, but since it always returns the latest value it’s handy for seeing when the last data was received.

For example, try pasting the following URL into a browser

http://app.nimbits.com/service/currentvalue?point=HomeServer&email=gigamegabot@gmail.com&format=json

You’ll get back a series of tags and values that looks like this :

{"id":7865660,"lat":0.0,"lng":0.0,"d":57.9303189764,"timestamp":"2011-03-15T00:15:03 +0000","pointFK":4405630,"note":""}

This is in JSON format, which is a simplified variant of XML that is handy for Javascript code (and most other languages) to work with.

Unlike the graph, this return string isn’t something we can plop into the gadget’s HTML and display unaltered to the user. Instead, we need to process it with code. And that means that we have to invoke the REST API in a way that passes the result to a piece of code.

In the Google Gadgets framework, this is done by calling one of their APIs:

gadgets.io.makeRequest(url, callback, params);

The “callback” parameter is the name of the Javascript function that will receive the response, and the “params” parameter are primarily used to override Google’s caching algorithm, as explained later. I took the code that invokes makeRequest pretty much verbatim from Google’s documentation for calling REST APIs.

The code which processes the data returned by the CurrentValue API is in the buildStatusString() function:

if (obj.data == null || typeof obj.data == "undefined")
{
	// something went wrong
	if (typeof obj.text == "undefined" || obj.text == null || obj.text == "")
	{
		return obj.error;
	} else
	{
		return obj.text;
	}
}
jsondata = obj.data;

var alldata = "";
for (var key in jsondata) {
	var value = jsondata[key];
	alldata += key + ":" + value + ", ";
 }
 var time1 = jsondata['timestamp'];
 var date1 = new Date(time1);
 // retrieve values as local time
 //var dateAndTime = date1.getMonth() + "/" + date1.getDate() + " " + date1.getHours() + ":" + date1.getMinutes();
 var monthNames = [ "January", "February", "March", "April", "May", "June",
	"July", "August", "September", "October", "November", "December" ];
 var dateAndTime = monthNames[date1.getMonth()].substr(0,3) + " " + date1.getDate() + "  " + date1.toLocaleTimeString();

 var value1 = jsondata['d'];
 value1 = Math.round(100 * value1) / 100;

 return dateAndTime + ' -- ' + value1;

Admittedly, this code is anything but beautiful. It got real ugly real fast when I tried to grapple with API requests that fail.

The code starts off with an attempt to figure out if the response was data or an error message. The object returned from the REST API contains 3 properties. If all goes well, the only one that you have to worry about is .data. Otherwise, .error and/or .text may contain error messages.

JavaScript’s handling of null/undefined variables must make sense to someone, but not me!

If you got data, then JSON makes processing it quite easy. The tags that you saw displayed in the browser are keys to an array, so to retrieve the timestamp value you use code like this:

var time1 = jsondata['timestamp'];
var date1 = new Date(time1);

The time1 variable is actually automatically handled as a date/time object by JavaScript, so why feed it into the Date function afterwards?

If you look at the value that is returned, it looks like this: “:”2011-03-15T00:15:03 +0000″. This is a UTC date and time string, so to make it more meaningful to the user you’ll want to convert it to their local time. In some languages, like Python, this conversion is a major pain.

JavaScript makes it easy: just feed that string into the Date function, and it automatically converts it to your local time when you access methods like date1.getDate() and toLocaleTimeString(). (Locale is different from local, by the way – toTimeString() also converts to the user’s local time.)

Google Gadgets and Caching

Google’s automatic caching of gadget code and data is something that trips up most first-time developers – it certainly got me.

Caching affects gadgets in 2 ways:

1. Gadget code is cached. So, if you change the code, then click refresh on the iGoogle page, an old copy of your code will still run, not the change you just made. This is a major performance benefit for users, but a major pain for developers.

Google feels your pain, sort of. They document this behavior, way down at the bottom of the FAQ, and offer a “My Gadgets” gadget that lets you disable this caching. The gadget has a tendency to flip from its initial state (it’s supposed to look like this screenshot in Google’s documentation) to a minimalistic “Go to developer mode” link – I’m still not sure if that’s a bug or a feature – but “Going To Developer Mode” actually does disable the caching.

2. Remote data is cached. So, for example, calls to the Nimbits CurrentValue API are intercepted by the API framework, which returns the same data as you got last time. This is another performance boost, but at least Google has documented this one properly.

The default behaviour is to cache any request that is within 60 minutes of the last time your gadget got data. You override this by passing an extra parameter in the call to the REST API – in my case, I set it to 5 minutes.

Note that this doesn’t mean your gadget will automatically update itself every 5 minutes – the update will only occur if the user refreshes the browser page, or if iGoogle automatically refreshes itself.

Other Google Gadget Gotchas

Where to host your code?

This is one of the first things you’ll have to decide, since you can’t test your code unless iGoogle can get to it through a web server. A good place to start is let Google host it – this is what happens if you enter your code into the Google Gadget Editor. Google gives you 4M to play around with, and the GGE makes it easy to add the gadget to your iGoogle page while you are testing it.

Google’s Gadget framework isn’t picky though – your gadget just needs to be an .xml file with a URL.

The one hitch with hosting it outside of the Google Gadget Editor is how to add it to iGoogle without publishing it to the world. I find the most reliable method is the Gadget Directory page for Google’s “My Gadgets” gadget . Not the gadget itself, whose “Add a gadget” feature inexplicably disappears, but the web page for the gadget, which has a convenient if equally inexplicable text box labeled “Add a gadget”. Works every time.

Edit Settings: Google’s Playground

As I discovered to my dismay after releasing the Nimbits gadget, you shouldn’t get too attached to the layout of your Edit Settings dialog. The layout is automatically generated by Google’s gadget framework, and you have no control over that layout.

Google can, and will, change that layout without warning or explanation. Other developers have experienced hidden settings that pop suddenly into view, and deleted settings that refuse to go away.

So, don’t go overboard with the options.

Debugging your JavaScript

For years and years, I heard web developers raving about Firefox’s Firebug plug-in without fully appreciating what they were raving about. Actually, I still don’t. But I can now say I’ve used it.

You’ll definitely need FireBug, or a JavaScript console of some sort ,when debugging your gadget. The console is used for catching errors in your JavaScript (it will report both the error and line number it occurred at), and for logging debug information using the JavaScript console.log() function.

I’ve left some console.log() calls in my gadget code because they are a handy way of seeing how the underlying REST API is being called. In the screenshot below, you can see the call being made to the Nimbits CurrentValue API, the JSON-formatted response a few lines later, and the call to the Nimbits Chart API at the bottom. (The console output also contains an error logged by an, ahem, “competing product”. Just kidding – Pachube’s great too, and I’m sure their request wasn’t that bad!)

Actually, FireBug’s functionality is so indispensible that Chrome and Internet Explorer have built similar features right into the browser. Having this built-in Javascript console leads to another common pitfall: don’t assume that a call to console.log() is going to work for other users, who might be running an older or non-mainstream browser. To ensure that console.log() won’t cause the script to fail, you can add the following to the JavaScript (I took the code from Alvin Abad’s blog.)

if (typeof console == 'undefined') {
    var console = {};
    console.log = function(msg) {
        return;
    };
}

Wrapping It Up

As mentioned in the introduction to this article, I’m not terribly impressed with Google’s Gadget framework.

I like the fact that it’s a small set of tools with a relatively shallow learning curve. However, the simplicity of the framework becomes a weakness when you inevitably bump up against some limitation that has no apparent coding workaround.

The documentation is so-so, which is admittedly a notch above the average web development platform’s documentation.

As someone who has been a user of iGoogle on a daily basis for many years, I’m a little alarmed to see how the sausage is made. I can’t shake the feeling that Google has lost interest in moving the platform forward. There haven’t been any significant changes to the API or user interface in a couple of years, and they haven’t bothered to expand, or even clean up, their documentation.

Google Gadgets are a fun place to visit, but I wouldn’t want to make a living there.

This entry was posted in Electronics, Programming and tagged , . Bookmark the permalink.

Comments are closed.