Tweet-A-Watt: Beyond the Twitter, Part Deux

This article continues my earlier post about web-based services for monitoring your electricity usage using Tweet-A-Watt. Last time I focused on Pachube – today I’ll explain the process of connecting a Tweet-A-Watt to Nimbits.

Nimbits Overview

Nimbits has a lot of similarities to Pachube: it is a “cloud-based” data repository with a browser interface for configuring your feeds and viewing your data. Both services provide a REST API to allow applications to input and output data, though for the purposes of Tweet-A-Watt you can get by with just barely scratching the surface of what the API offers.

If you’re a developer and/or paranoiac, a key advantage of Nimbits is that it is an open source project based on the Google App Engine. It is quite easy to install Nimbits on your own Google App Engine account, as I’ll show you below. This allows you to “own” the data, updating it as frequently as you like and keeping it for as long as you like.

GAE does have limits on how much data you can toss about without paying. According to Google’s “What is Google App Engine” page: “All applications can use up to 500 MB of storage and enough CPU and bandwidth to support an efficient app serving around 5 million page views a month, absolutely free”. Data compression is one of the key strengths of the Nimbits software, so it’s safe to assume you aren’t going to comes close to that limit unless Rebecca Black decides to sing about your electricity usage. (It’s also safe to assume that she won’t).

The Google App Engine is widely used, and extensively documented, so it’s certainly a solid foundation for experimenting with ways to process and display your Tweet-A-Watt data. The Nimbits code is all written in Java. So, if you know GAE and Java, the sky’s the limit for what you can do with Nimbits. Personally I know neither, but as with Pachube it’s quite easy to get up-and-running with Nimbits with a bit of configuration and a few lines of code.

Configuring a Nimbits Account

You have 2 options for hosting your data: use the public Nimbits Server at app.nimbits.com, or create your own using a Google App Engine account. You’ll probably want to use the first option to test your Tweet-A-Watt connection and get familiar with the Nimbits interface, then graduate to the 2nd option later. I’ll go through both processes, beginning with the public server.

1. Open app.nimbits.com in a browser. You’ll be prompted to login using a Gmail user-ID (or if you are already logged in, to confirm that you want to use that login ID).

Wait, what? Nimbits is part of the Google empire?

No, but Google App Engine is, and it not surprisingly uses Gmail as the primary type of authentication. In fact, some parts of Nimbits (and GAE) are closely tied to Gmail – your alerts can only be sent to that Gmail account.
However, if you’re a developer you might want to use a secondary Google account here. Your applications will connect with either Gmail authentication or with a Nimbits-specific API key. While the API key works fine for all Nimbits features, if you delve deeper into the underlying Google App Engine framework you might start experimenting with GMail authentication. Hard-coding your primary Gmail password into a program is like looking death in the eyes!
Incidentally, if you are generally logged into Gmail all day, you’ll find the “Private Browsing” (aka “Incognito Window” aka “Porn Mode”) setting of your browser to be a handy way of logging in with a secondary Gmail account without logging out of your main one.
2. Your browser will display the standard Nimbits user interface, as shown below. To configure your Tweet-A-Watt feed, Click” New Data Point” in the left frame.

A Nimbits Data Point is like a Pachube Datastream. But — get this — it has a name instead of a number. Far out! Actually, you can make things a little easier on the coding end (but harder on the viewing end) by setting the Data Point name to the sensor # in your wattcher.py script (i.e. “0” for the 1st Tweet-A-Watt sensor, and so on). You can’t rename a Data Point, though, so choose carefully.

3. The new Data Point will be added to the frame on the left. Further configuration is optional – the default settings will allow you to save Tweet-A-Watt readings and view them in a graph.  However, there are some neat features built into the configuration screen, so lets have a look around. 

4. Double-click on the Data Point.  A couple of windows will open in the main frame, as shown below.  Click Point Settings on the menu bar, then the Data Point name. 

5. The “General” tab of the configuration dialog is shown below, with the settings I use.  By the way, this dialog is also the only way to delete a Data Point (using the toolbar icon). 

6. The default Compression is 0.1, meaning that a new reading that is within 0.1 of the previous reading won’t be stored.  This setting is handy if your data source is bombarding the server with several readings per second, but your Tweet-A-Watt will likely be sending  just 1 reading every 5 minutes.  If your electrical usage tends to be rock steady for long periods, you can turn off Compression by entering zero.

7.  The default History of 90 days means only the most recent 3 months of data will be stored.   You’ll probably want to increase this at some point, especially if you setup your own Google App Engine account 

8. The Unit of Measure for Tweet-A-Watt is Watts, or Volt-Amps if you insist.

9. I left the setting at Public. Don’t worry – this doesn’t mean that the unwashed masses can post data to your feed; just that they can view your feed through the REST API.  (Don’t worry – they won’t.) The Public setting makes playing around with the API a little easier, especially when connecting through 3rd party software like Excel, so I’d suggest you leave it on until you’ve got things working.

10. The Messaging options can be used to inflict all of your data readings on your friends or unusually social enemies.   This might be a good way of converting the former to the latter.

11. In the Alerts tab, you can configure a high and/or low threshold at which an alert will be triggered.  Here, the messaging options become more useful, and I use the Instant Message setting for my Tweet-A-Watt feeds.  (See the “Google Talk” section, below).

12. The other configuration setting that I use is the Idle Alarm, which was recently added to Nimbits.  This lets you know when your Tweet-A-Watt configuration stops working, which I find usually means my (highly butchered) Python “wattcher” script has stopped.  Back when my Tweet-A-Watt merely twittered, I would sometimes go a couple of weeks before realizing that the tweets had stopped. (Strange that one of my, ahem, 2 followers didn’t complain.)

13. That’s it for Data Point configuration. Click on Save.

13. If you have multiple Tweet-A-Watt units, you can create other Data Points for each one.   Remember, the Data Point name uniquely identifies the data stream.

14. You probably noticed that there is something called a “Category”, which is a group of related Data Points (the equivalent of Pachube’s “Feed”).  Categories are optional, and are most useful if using the API to “walk” through the server’s Data Points.  To create a Category, click on the Green folder icon, then drag each Data Point into the new Category in the left frame.  You may have to refresh the browser page before the drag-and-drop works. 

15. One last thing before you are done with the Nimbits side of the configuration: create an API key.  As mentioned earlier, this allows you to post data to your feed without dangling your Google password out to the Internet.  Click on the Secret Key icon on the toolbar, then follow the instructions.  Note that, unlike Pachube, there is only one Key for your account – generating a new one invalidates the previous one.

Tweet-A-Watt configuration

The modifications needed to the Tweet-A-Watt wattcher.py script are a little different from those for Pachube, as described in the previous article.  For Nimbits we’ll invoke the REST API directly, while for Pachube we used a separate library (eeml) as an interface.

Add the following code to  update_graph method, in the same section that the Pachube code was added in the previous article.  You should, of course, replace [Data Point Name], [e-mail address] and [secret key] with the values used to configure your Nimbits account (without the square brackets) [got it?].

try:
  urllib.urlopen("http://app.nimbits.com/service/currentvalue?value=" +
     str(avgwattsused) +
     "&point=[Data Point Name]&email=[e-mail address]&secret=[secret key]")
except IOError:
    print 'Error sending to nimbits

 

If you have multiple Tweet-A-Watt sensors, and you decided not to use the sensor # as the Data Point name, you’ll have to add some Python code to set the “Data Point Name” parm based on the sensorhistory.sensornum variable.

After restarting your wattcher.py script, switch back to the Nimbits page in the browser.  If everything’s working, the “Current Values” window in the main frame will display a value and timestamp sent by your Tweet-A-Watt within a few minutes.

The graph on the Nimbits page takes a little getting used to.  For one thing, it uses Adobe Flash, so forget above viewing it on your iPhone. By default, it displays the last hour of data.  Well, actually, it displays an hour of data – the hour specified in the 2 text boxes at the top of the graph.  Clicking the Refresh icon doesn’t refresh the graph, because the textboxes aren’t changed.   Similarily, clicking the “1d” link doesn’t display the last day of data, because the freakin’ textboxes aren’t changed.  Wah!

Don’t blame Nimbits for that, blame Google. It’s their graph.  Officially, it’s an “Annotated Time Line” (without annotations), as documented on the Google Vizualization API site .

So, here’s how it works: enter a date and time range in the text boxes, then click the graph’s refresh button.  You can now use the links (1h, 1d, etc.) to zoom in on a window of data of that length.  You can then use the slider at the bottom of the graph to scroll a zoomed view of your data over the full time range.  In the screenshot below, the line graph shows electricity usage for an hour ending about 11 pm on March 14, while the bottom part of the chart allows you to use the slider to scroll though the full time range between Mar 12 and Mar 14.  (Hey, don’t judge!  My ‘duinos have special needs.  And 2 plant lights.)

If that doesn’t float your boat, other charts and graphs are available.   The Google Visualization API Gallery shows the full list, any of which can be displayed through the REST API.  The Nimbits page has just 2 built in, the Annotated TimeLine and the Motion Chart.  I’m not entirely sure how the latter works, but if you click on it enough it starts to act like a game of Asteroids.  (The Motion Chart appears to have been dropped from Nimbits 3.1, which is good since I suck at Asteroids.) 

Google Talk

I was initially skeptical of the social networking features, since I’m antisocial.  However, I’ve found the Instant Messenger feature to be quite handy, since it doesn’t require any actual friends.  (Pachube doesn’t offer any prepackaged Instant Messenger connectivity at present).

This feature leverages the Google App Engine’s interface to “any XMPP compatible instant messenger client”.  The only one that I’ve tried is Google’s own “Google Talk”.  I think this is the best route, since it is integrated with your Gmail account – if you aren’t online when your Nimbits server has something it wants to send you, it will e-mail you instead.   You’ll need to install the Google Talk desktop software, and login to Google Talk with the same Gmail ID with which you log into Nimbits.  You’ll also need to enable the Instant Messenger option in Nimbits, as described in this article on the Nimbits blog.

There are basically 3 ways that you can use the Instant Messenger interface:

  1. Receive alerts when your data passes a specified threshold.  This is configured in the “Alerts” and “Idle Alarm” tabs, as shown earlier.  Although it isn’t obvious from the “Idle Alarm” dialog, it appears to use the same settings as Alerts to determine where to send the Alarm.
  2. The interactive interface.  The commands are described in the Nimbits blog post.  This is particularly useful if you are using a non-Android smartphone (there is a Nimbits app for Android).  Incidentally, I didn’t have much luck on my iPhone using Google’s iPhone-specific Google Talk web page, but there are a bunch of apps that support Google Talk.  I use the IM+ app for this.
  3. Receive every new data reading.  This is what will happen if you enable Instant Messenger on the “General” tab.  Possibly useful for getting excused from (or kicked out of) meetings and awkward social occasions. 

Rolling Your Own

Once you are up-and-running with the public Nimbits Server, you should consider configuring your own AppSpot site using the Google App Engine.  It is surprisingly easy to do, and almost entirely maintenance-free.   About the only time you’ll have to fiddle with the controls is if you decide to update your version of Nimbits.

The browser interface and API is identical, so there’s no additional learning curve.  On the down side, there is no easy way to export your data from the public server to your own.

The instructions here (beginning with the “Getting Started” section) do a good job of walking you through the process – it really is that simple.  I’ll add a few observations on the process:

  • The suggestion about using a secondary Gmail account also applies to registering for Google App Engine. If you are a developer, there are some things that your code can do only when authenticating through Google, and you definitely don’t want to be a position of hard-coding your main Gmail password in any app.
  • When you enter your application name in the Google App Engine page, be sure to use only lower-case characters.    When I was setting up mine, I couldn’t believe that every application name I tried was taken according to the “Check Availability” button.  “C’mon, someone used GigaMegaAlphaBeta!  Do I have a stalker?”  Turns out that upper case letters aren’t allowed.  (They’re evil).
  • Be sure to download the Java App Engine SDK, not the Python one.  Even if you won’t be doing any developing, you’ll need the Java deployment tool – it’s the “install program” for Nimbits in this case.
  • If this is the first time you’ve deployed Nimbits to GAE, you should find that the most Nimbits version is already set as the default.  If you later install an update of Nimbits, then you’ll have to manually change the version in GAE – the new version doesn’t automatically go “live”.
  •  In the “ServerSettings” page, the only setting I had to change was admin (which should be your Gmail address).  The rooturl setting can be left at http://app.nimbits.com.

After finishing with the ServerSettings, you can open <yourappname>.appspot.com in your browser, and you should be presented with the exact same interface as at app.nimbits.com.  You’ll have to recreate your Data Points, using the same process as described earlier.  At the Tweet-A-Watt end, just replace “app.nimbits.com” with “<yourappname>.appspot.com” in the wattcher.py script

e.g.

 urllib.urlopen(<a href=http://gigamegaalphabeta.appspot.com/service/currentvalue...

Note: I didn’t actually use gigamegaalphabeta.  It’s all yours. Stalker.

Wrapping Up

I had originally planned to write a bit about the Nimbits .Net SDK. Unlike Pachube, Nimbits provides a .Net DLL, as well as supporting calls through a REST API.  The .DLL uses an intriguing technology that I hadn’t encountered before, making its Java SDK library accessible to .Net through IKVM.  I’ve been tinkering with it for awhile, and it’s quite cool to see Java data types presented as .Net classes.

There isn’t a lot of documentation on the .Net API, so I plan to give some code examples in a future blog post.  I’ll also explore what I think will be a common coding requirement for Tweet-A-Watt users: importing historical data from the TAW .csv file into the Nimbits server.

This entry was posted in Electronics, Programming and tagged , , , . Bookmark the permalink.

One Response to Tweet-A-Watt: Beyond the Twitter, Part Deux

  1. Great posting Dan, thanks! I may include a link to this in the next Nimbits Newsletter.

    I laughed about being party of the Google Empire. Call me Darth Ben. I’m actually working with they guys at AppScale to support running Nimbits locally and on ubuntu / amazon clouds.

    Ben

    PS – the zoom buttons on the annotated timeline work but only after you select a date range. i.e select a large date range and click the refresh button. Then use the 1h, 5h etc in the chart to zoom into the selected data.