Netduino Meets World

[Updated August 6: Added a couple of tips to the section on using minicom to configure the XBee.]

[Updated August 7: Added "Windows Weirdness" section with workarounds for problems with pySerial on Windows]

Note: This article is an installment in a series about building a plant light controller (and beyond) on the Netduino microcontroller platform. For previous articles in the series, click here.

When last we saw it, the Netduino project was a pretty smart plant light controller but a pretty dumb clock. The main problem is that it can’t figure out the time on its own – when it’s powered on I have to manually set the time using its menu.

C’mon, Netduino, you don’t need me for that! Haven’t you heard of the Internet?

An Inconvenient Truth

Give a hoot, don't compute!
Give a hoot, don't compute!
As you probably know, it is quite possible to use a Netduino Plus, or a regular Netduino with an Ethernet shield, to connect a Netduino directly to the Internet. In fact, O’Reilly recently published a book starring the Netduino Plus which shows you how to do that.

Instead of Ethernet, I decided to use an Xbee Series 1 connection to a PC acting as an intermediary, partly because I already had that configuration in place to support Tweet-A-Watt monitoring, but also because of certain advantages:

  • Wired Ethernet connections aren’t practical for most parts of the home
  • XBee is cheaper than Wi-Fi, especially if you plan to use multiple Netduinos in various parts of the home (which we will, once we add sensor monitoring to the mix)
  • The Netduino is a relatively constrained development platform compared to a PC. So, it helps to limit the Netduino’s role to what’s required on the “front lines”, and have much of the code running on a PC acting as “command and control”. This, too, will become a bigger advantage when there are multiple Netduinos involved.

The big downside to this approach is that you’re leaving a PC powered on 24/7. That adds up to a lot of wasted electricity, considering how low its workload is going to be. Dedicating a 120W+ PC to this project is absurdly excessive – do you really want to make Al Gore cry?  (Well, yeah, me too – but you know what I mean.)

Fortunately, there are an increasing number of ultra low power devices that are capable of acting as the PC for this purpose. As long as it has a USB port, an Internet connection and can run Python, then it’s good to go.

I’ve written about the Beagleboard in the past. When running Linux and the Python script in this project, a Beagleboard XM uses about 3W (as measured by a Kill-A-Watt meter).

By comparison, my “energy efficient” 2 year old Windows PC averages about 120W. My older PC averages about 200W. Considering that it’s able to serve as a full-fledged home server, the Beagleboard is incredible, and hopefully an indication of the future direction of home PCs.

Incidentally, the Netduino/Arduino/LCD combination used in this project burns well less than 1W, so using *duinos in your home projects is definitely an energy-friendly alternative.

TonidoPlug Computer
TonidoPlug Computer
A less expensive option, and the one that I’m using in this project, is the $99 TonidoPlug. It comes with Ubuntu already installed, and a built-in Ethernet and USB Port. You can connect the XBee to the USB port directly if you want, and that will work fine for the purposes of this project.  However, if you want to run the Tonido software then you’ll need a powered USB hub — the Tonido software requires a USB drive, and there’s only the 1 built-in port.

Like the Beagleboards I’ve found the TonidoPlug to be very reliable device, and it’s just as power efficient. It’s not easy to upgrade or replace the OS, but the built-in version of Ubuntu is more than adequate for the projects I have in mind.

If you decide to go with a Beagleboard, one of my past articles covers installing Angstrom and Python, and this eLinux page is gospel if you’re installing Ubuntu.

As for the TonidoPlug, it comes with Ubuntu 9.04 (Jaunty Jack), Python 2.5, and an SSH Server out of the box. All you need to add for this project is the Python serial library and the GNU Screen utility for running the Python app in a background terminal session.

To install them, login as root through SSH, create then directory used by the apt package manager, then install a couple of packages:

mkdir -p /var/cache/apt/archives/partial
apt-get update
apt-get install python-serial
apt-get install screen

Connecting an XBee to a PC

Point-to-point communication ain’t no fun with just one point, so we’ll need to connect an XBee to both the PC and the Netduino.

At the PC end, whether you use a Beagleboard, TonidoPlug, or a full PC, you’ll need a USB adapter for the XBee.

There are many options out there. I’m using an Adafruit XBee Adapter and a USB TTL-232 cable. However, unless you happen to have a spare cable this is a relatively expensive option, since the adapter costs $10 and the cable costs $20. Other options that I’ve used are the $20 Parallax XBee Adapter, and SeeedStudio’s versatile $20 UartSBee.

Before connecting the XBee to the Netduino, you should configure the 2 XBees so that they will talk to one another.

By default, the XBee adapters are all configured as adapter 0 in PAN (network ID) 3332, and they’ll send their data to other adapter 0s. While this works fine if one XBee only sends, and the other XBee only receives, we’ll soon have the Netduino and PC chatting merrily away to one another. So, you should assign one of the XBees to have a different adapter ID (number 1 works fine), then set the other XBee’s destination address to be that same ID.

This page at Adafruit covers the 2 options for configuring an XBee: the Windows-only X-CTU program, and entering AT commands from a serial terminal.

Whichever configuration method you use, you need to change the “MY” setting on the first XBee to 1, and the the “DL” setting on the other XBee to 1.  That’s it: the other defaults are fine.

I’d recommend using the X-CTU program if you’re getting started with XBees: it’s much more user friendly and foolproof than a serial terminal.   While you are in there, you should update the firmware of the XBees, something you can’t do from a serial terminal.

If you’re hardcore about using a TonidoPlug or Beagleboard as a PC, or want to avoid running X-CTU for other reasons, than a serial terminal is the way to go.

Personally, I’ve used minicom on a Beagleboard and TonidoPlug for this. In order to talk to the XBee you’ll need to run minicom as root and change some of the default settings. Connect an XBee in its adapter to the USB port, then:

  1. At the Linux command line, enter “minicom -s”.  If you plan to save your settings in a profile (a good idea), you should run as root: “sudo minicom -s”
  2. Select “Serial port setup”
  3. The serial port settings should be as shown in the screenshot below.  Press “E” to change the Bps to 9600, and press “F” to set Hardware Flow Control to “No”.  You might also need to change the Serial Device setting: its generally “/dev/ttyUSB0″, but if you have other USB devices connected: it might be /dev/ttyUSB1 or /dev/ttyUSB2, etc.
  4. Press Enter to return to the minicom configuration menu, then select “Screen and keyboard”
  5. Press “P” to turn on the Add Linefeed feature, and press “Q” to turn on Local Echo.
  6. Press Enter to return to the configuration menu again.  To save these settings, choose one of the “Save setup” options.  To display the serial terminal,  select “Exit”.
  7. The XBee is initially ready to receive and send data.  You’ll need to put it in command mode instead.  To enter command mode, press the “+” key 3 times (i.e. +++) but don’t press Enter.  The XBee should display OK.

Minicom settings for XBee configuration
Minicom settings for XBee configuration

The correct terminal commands for the first XBee are:

+++
ATMY 1
ATWR

And for the second XBee

+++
ATDL 1
ATWR

The XBee should send back “OK” after each command.

If you aren’t able to see what you are typing, try turning on Local Echo again: press Ctrl-A then Z to display the menu, then press E.

Note that the XBee will automatically (and silently) exit configuration mode if you wait too long between commands, so if you aren’t getting the OK after an AT command, try entering +++ again.

Connecting an XBee to a Netduino

The hardware connection to the Netduino is relatively simple, because the Netduino uses the same 3.3V voltage level as the XBee and the Netduino’s serial pins talk the same language as the XBee.

The only complication is that the XBee uses an usually tight spacing for its pins, just 2mm apart.  This makes it difficult to connect jumper cables to the pins, and impossible to plug the XBee into a breadboard.  Most people use a breakout board to solve this problem, such as Sparkfun’s XBee Breakout Board or Adafruit’s XBee Adapter PCB (which is the PCB used in their Adapter Kit – it functions well as a breakout board).

As shown in the photo below, I plugged the XBee into a small breadboard using an Adafruit XBee Adapter.  This makes it easy to share the Netduino’s 3.3V pin with other devices besides the XBee, and it allows you to connect an LED to some of the XBee’s pins for troubleshooting:

XBee Pin Connection
1 3.3V
2 Netduino RX (e.g D0)
3 Netduino TX (e.g. D1)
6 LED (lit when receiving, optional)
10 Ground

XBee To Netduino Connections
XBee To Netduino Connections

Running the Python Software

Once the hardware is in place, the next step is to start the Python software running.

You’ll need to have pySerial installed.  If you followed my instructions for configuring Angstrom Linux on a Beagleboard, or configuring Ubuntu on a TonidoPlug (earlier in this article), then you’re all set.  Otherwise, see the install instructions on the pySerial page.

Download the Python code from my Google Code project. Save the file to any folder on the host PC – if you are using a TonidoPlug or Beagleboard running Linux, then your home folder is fine.

If you are using Linux, you’ll probably need to run the Python code as root in order to access the serial port:

i.e.

sudo python TimeServer.py

The command line parameters for the program are all optional:

TimeServer.py [-nl] [-d] [-port xxxx]
  • -nl – turns off logging.  Otherwise, any errors will be written to the current directory in a file named controller.txt.  Hopefully this file won’t get large, but turning off logging is necessary if you are running the code from a read-only directory.
  • -d – turns on debug messages written to the console.
  • -port – specifies the serial port that the XBee is connected to.  The default is /dev/ttyUSB0. On a TonidoPlug or Beagleboard, that should be the right port. On a Windows PC, the port will have a totally different name, such as COM3 – check Device Manager if you don’t know which one to use.  If you are using Windows, see the “Windows Weirdness” section below.

If you launch the program when connecting through SSH, you should first use the GNU Screen utility to open a virtual command prompt, and run the program from there. Otherwise, the program will stop running when you disconnect from SSH.

To run screen, just type “screen”. You’ll be brought to a new command prompt, at which you run TimeServer.py as above.

When TimeRelay is up and running, you can disconnect from your screen session by pressing CTRL-A, then D.  This leaves the Python code running in the background.   To reconnect to the screen session, enter screen –r –d.

GNU Screen is handy for running a background process like this because you can reconnect at any time, from any place, and run screen –r –d to see the current output from the TimeServer.py program.  This is really helpful for debugging, and for restarting the Python code if (heaven forbid!) it should start misbehaving.

Windows Weirdness

Running the Python code on Windows presents a couple of other complications, besides that whole “global warming melting glaciers” thing.

If Python throws an “The system cannot find the file specified” Exception when it tries to open the serial port, you may have run into the problem described on StackOverflow here. Try specifying the COM port as ”\.COMxx”

e.g.

-port \.COM10

Hey, don’t ask me — try it.

Also, opening a serial port sometimes results in an “Access denied” exception on Windows if you don’t close it before opening it.  Like I said, don’t ask me.  Since closing the serial port before opening it doesn’t seem to cause any problems, I’ve updated the Python code with this fix.

Running the Netduino Code

You can get the Netduino code in one of 2 ways:

  1. If you are familiar with Subversion, you use any SVN client(my favorite is TortoiseSVN) to get a copy of all the project files.  The SVN URL is https://gigamega-micro.googlecode.com/svn/trunk. If you want the older version of the code to match one of the past articles, get the corresponding revision. (See the revision list here)
  2. Or, download the good old .zip file from the Downloads section of the Google Code project. Here is a direct link to the code for this article.

If the Python program is running, then you’ll know within a minute of powering on the Netduino whether everything is working correctly: the date and time will automatically be set correctly via the XBee connection. Tah-dah!

OK, OK, maybe that wasn’t worth a Tah-dah.  But, that’s not all: the LED you connected to the XBee’s receive pin will light up once a minute. Tah-dah!

Hmm, tough audience.

About the only other change to the Netduino software this time is that I finally added a Daylight Savings Time setting to the Netduino’s menu: you can turn DST off or on in the “Date” menu. (Tah-dah!) This is used when calculating the “Sun Up” and “Sun Down” times, if you are basing the plant light timer on those times. Purists will point out that an Internet-connected Netduino shouldn’t have to rely on a manual setting to know when DST is in effect, but we’ll ignore those purists for now.

Programmer’s Show and Tell

The Python code is admittedly very simple: all the better to learn Python programming from, right?

The XBee’s serial port connection is opened during startup. If the connection can’t be opened, the program will display an error message and terminate by calling the Python sys.exit function. If you had some problems getting the serial port working earlier, you’re no doubt well acquainted with this error message.

try:
    ser = serial.Serial(SERIALPORT, BAUDRATE, timeout=TIMEOUT)
    ser.close() # workaround for known problem when running on Windows
    ser.open()
except:
    print "Unable to open serial port ", SERIALPORT, ". See ya."
    sys.exit(1)

The main loop checks whether the time has changed (just the minute, actually – we’ll let the Netduino handle the seconds). If it has, it calls the sendDateTime() function. It then sleeps for awhile before repeating the loop.

def displayTime():
    lastMin = -1
    while 1==1:
        try:
            currTime = datetime.datetime.now()
            if DEBUG:
                print "in displayTime, currTime = " + currTime.strftime("%Y-%m-%d %H:%M:%S")
            currMin = currTime.minute
            if lastMin != currMin:
                lastMin = currMin
                sendDateTime()
            time.sleep(15);
        except Exception:
            LogMsg("Error in displayTime")
            print sys.exc_info()[0]

Note that that the Exception handler is different from the one used when opening the serial port: it logs the error, displays some information about the error to the console (using the Python sys.exc_info function), then optimistically keeps going. If there was no Exception handler, any exception in this loop, or in the functions called by this loop, would cause the application to terminate.

This “log it and keep going” approach is generally frowned upon in software design, but I’d recommend it for a simple program like this that runs in the background.  You should find that this application, and Python code that communicates via XBee in general, is very low maintenance: you can just start it, then forget about it.  You can occasionally check on the code using GNU Screen or by looking at the contents of the controller.txt error log.

The sendDateTime function just converts the date and time to the format that the Netduino is expecting it, then writes it to the serial port. Note that the “ser.flush()” function is used to make sure that the data is sent right away rather than being buffered – this is literally time-sensitive data.

def sendCommand(strPrefix, strCommand):
    if ser:
        ser.write(strPrefix + ":" + strCommand + chr(0))
        ser.flush()

def sendDateTime():
    timeStr = time.strftime("%Y-%m-%d %H:%M")
    sendCommand("D", timeStr)

On the Netduino side, surprisingly little new code is required to talk to the XBee. We’re using the XBee as a serial port device, rather than making use of the XBee’s API (a more versatile option that I might explore in a future article). From the Netduino’s perspective, the XBee might be some guy sitting at a serial terminal, occasionally typing in a”set time” command. We’ve already implemented that support, back in the “Netduino: Time and Weather” article .

The XBee connection is opened during startup, using almost the exact same code as was used to open a serial connection to a PC terminal:

// last parm should be true for UART terminal (to send
//   user confirmation), but false for UART XBee
uart = new clsUART(strUARTPort, 9600, 512, false);

The event handler is the same one that was used for the PC terminal: the event is raised when our generic serial port class, clsUART, receives some data. In fact, if you’re using the SparkFun Serial LCD Kit, this event handler is still used when the Netduino receives data from the Arduino, as explained in the “A Little Help From Its Friends” article .

Sharing a serial command event handler between Arduino and the XBee works well, and is easy to implement in .Net. The Netduino doesn’t know, or need to know, who’s sending the command, and the .Net Micro framework ensures there isn’t any conflict between the 2 serial ports.

The only thing that’s changed here is that the XBee sends the date and time together in the same command:

D: 2011-07-25 18:01

So, I modified clsUART to check to see if the string is in this format, and it raises both a “date set” and “time set” event if it is.

if (strDateParts.Length == 5)
{
    intHours = Int32.Parse(strDateParts[3]);
    intMinutes = Int32.Parse(strDateParts[4]);
    intSeconds = 0;
    blnTimeSet = true;
}
...
// tell rest of application that the date has been set
raiseCommand(strCommand);
if (blnTimeSet)
{
    // raise the time event too
    raiseCommand("T:" + Program.Int_ToZeroPrefixedString(intHours, 2) + ":" + Program.Int_ToZeroPrefixedString(intMinutes, 2));
}

The event handler already knew how to handle date changes and time changes, so no code change was required there.

The only other significant change was modifying the menu to add a “Daylight Savings Time” switch. Since the menu code was designed to be easily extended – as explained in the Ordering Off the Menu article  – this required just 1 new line to the strMenuItems array:

"2-1", "Hour `HH`", "_UD-`HH`",
"2-1", "Min `mm`", "_UD-`mm`",
"2-1", "DST `DS`", "_DST", // NEW - Daylight Savings Time setting

The format looks rather cryptic, but it will make more sense if you read the “Programmer Show and Tell” section of Ordering Off the Menu.

The code in clsLCDMenu passes the “`DS`” keyword to its callback method for getting the current DST setting, and it passes the “_DST” keyword to the callback method for handling a new setting entered by the user.

For example, the code added to the setValueFromMenu() callback method for the “_DST” keyword is:

else if (menuCode == "_DST")
{
    // toggle DST
    blnUseDaylightSavingsTime = !blnUseDaylightSavingsTime;
    calculateSunriseAndSunset(DateTime.Now);
    if (blnUseDaylightSavingsTime)
    {
        SaveSettingToEEPROM(ADDR_DST, 2);
    }
    else
    {
        SaveSettingToEEPROM(ADDR_DST, 1);
    }
}

This recalculates the Sunrise and Sunset times (and updates the plant light timers if they are based on these times), then passes the new setting to the Arduino in the Sparkfun LCD Kit to save it to EEPROM.

I love it when a plan comes together!

Wrapping Up

I’ve found the XBee to be a very dependable add-on to the Netduino. I actually wrote the code used in this article back in May and have been running it continuously since then, and I’ve yet to have any cases where the XBee connection permanently drops or stops responding. In my experience, this is a big advantage over using an Ethernet connection – when the Netduino’s Ethernet connection drops it tends to stay down.

So, if the Xbee’s so great, why am I using it only to set the time?

Good question.

If you browse through the main Program.cs class in the source code, you’ll notice a new method called uploadSensorData, that I didn’t address in this article.

Stay tuned for the next step in the project, in which the Netduino talks back, and I draw together a couple of previously separate topics in my blog: the Netduino and Nimbits.

Posted in .Net, Electronics, Programming | Tagged , , , , , | Leave a comment

A Nimbits Gadget: Programmer’s Show And Tell

Updated June 4: I’ve corrected a couple of things, as marked in blue.

Updated Dec 3: I made a small code change to fix the autoscale feature, as marked in green below.

In my last post, I gave a user’s overview of a Google Gadget I wrote to display graphs from the Nimbits data logger.

This post is directed at developers. I’ll explain how the gadget interacts with the Nimbits API and the Google Charts API, and give a few tips to those who are getting started with Gadget development.

Actually, my tips would only be useful to gadget noobies, since I’m a gadget noob myself. Fortunately, Google’s Gadget development framework is pretty easy to get started with. This is partly due to the abundance of sample code – you can examine the source code for any gadget by clicking the “View Source” link near the bottom right of the gadget’s page in the Google Gadget directory.

However, another reason why the Gadget framework is easy to learn is that there isn’t that much to it. The limitations can be frustrating, and it doesn’t seem like the framework is under active development by Google any longer.

That’s fine with me, though: the gadget is just the container, and the contents are supplied by the Nimbits and Google Charts APIs, both of which are very much moving forward.

Viewing the Source Code

If you click on the “View source” link for the Nimbits Gadget , the first thing you’ll notice about the code is that there isn’t any. The link displays the Gadget’s XML definition. The JavaScript code is often embedded in the XML file (that’s actually what Google recommends for better performance), but I decided to put it in a separate .js file to avoid duplicating the code in “canvas” view.

So, the code is actually here: http://www.gigamegablog.com/gadgets/nimbits.js. (Yeah, I know your Mother told you not to run strange JavaScript, but if you click this link it will just display the code, not run it.)

Most of the JavaScript code is a series of functions that build up 2 calls to REST APIs:

The Nimbits Chart API

A good introduction to the Chart API is Benjamin Sautner’s blog post. . His post gives a couple of examples of calls to the Chart API, and the 2nd one is very similar to the one that I ended up using in the gadget:

 

http://app.nimbits.com/service/chartapi?point=TempF&email=bsautner@gmail.com&cht=lc&chs=200×100&chxt=x,y&chxl=0:|Apr|May|June|1:||100+F

 

If you aren’t familiar with REST APIs, then you might wonder why this looks like a URL. That’s the whole idea of REST: API calls are URLs with all of the parameters embedded in the URL string. Ideally, the URL’s parameters tells you (and, most importantly, the server) everything you need to know to handle the request, with no dependencies on previous requests, etc.

Authentication aside, if you and I paste a REST URL string into a browser, we should both get the exact same result. (And, for the purposes of this gadget, I basically tossed authentication aside – that’s why it only works with Public data points).

If you paste the above URL into a browser, you’ll get back a graph in a .png file. The Nimbits Charts API will always return a PNG. This makes it easy to use trial-and-error to experiment with the Charts API. Other Nimbits APIs, and most REST APIs in general, return XML – we’ll see an example of that in the CurrentValue API, below.

Before I get into the code I used in the gadget to build the chart…

How To Build A Nimbits Gadget in 5 Minutes

Mine took a little longer, actually, but displaying a Nimbits chart in a gadget is quite simple:

1. Open the Google Gadgets Editor: basically a Notepad that saves files on one of Google’s servers.

Replace the “Hello, world!” line with


<img
src="http://app.nimbits.com/service/chartapi?point=TempF&email=bsautner@gmail.com&cht=lc&chs=200x100&chxt=x,y&chxl=0:|Apr|May|June|1:||100+F"
/>

2.Click File, then Save. In the Save File dialog, change the filename from helloworld.xml to nimbits.xml. It should now look like this

3.Click File, then Publish. The Gadget Editor will nag you about some of the things that are missing, but that’s OK because you’re only going to publish it to your own iGoogle page, not Vanity Fair. Click OK.

4.The “Publish Your Gadget” dialog will be displayed. Click on the “Add to my iGoogle page” option.

5.Another page is opened, with a big blue “Add Hello World! To iGoogle” button. Click it.

6.If all goes well, you should see a brand new gadget on your iGoogle page, as below.

If you jumped ahead and started changing the settings in the Gadget Editor to see what would happen then, hey, sorry about that whole nervous breakdown thing. I should have mentioned that Google automatically caches your gadget’s code, so your changes won’t be automatically applied to your iGoogle page. There is a way to turn that off – see the Caching section below.

Your shiny new gadget will faithfully report the temperature in Benjamin Sautner’s aquarium. But what if you actually wanted your gadget to display something else? That’s where the code comes in…

Nimbits Chart API Options

The code block below is taken from the displayChart() function of the gadget’s JavaScript. It builds an HTML img tag similar to the one you pasted into your gadget, but getting its parameters from variables instead.

The variables are loaded from settings in the User Preferences dialog. I won’t cover that code here, since this section of Google’s Gadget Developer docs tells you all you’ll need to know.

html += '<img src=http://' + serverName + '/service/chartapi?'
  + 'points=' + pointsParm;
html += '&email=' + userEmail;
html +=  '&chds=a&cht=lc&chco=' + colourParm + '&chxt=y&chf=bg,s,EFEFEF';
html += '&chs=' + graphWidth + 'x' + graphHeight;
if (spanType == "Hours")
{
	html += "&st=-" + timeSpan + "h&et=now";
} else
{
	// # of readings
	html += "&count=" + timeSpan;
}

 if (displayLegend)
{
	// display it at top listing all data points
	html += '&chdl=' + legendParm + '&chdlp=t';
}
var gridStyle = "&chg=";
if (displayGrid == "Vertical")
{
   gridStyle += "10,0,1,0";
} else if (displayGrid == "Horiz")
{
	gridStyle += "0,10,1,0";
} else if (displayGrid == "Both")
{
	gridStyle += "10,10,1,0";
} else
{
	// no grid
	gridStyle = "";
}

Let’s break this block of code into parts

html += '<img src=http://' + serverName + '/service/chartapi?'
			  + 'points=' + pointsParm;

Every call to the Nimbits Charts API needs a server name and at least 1 DataPoint. If you want to include multiple Data Points, they must be comma-delimited. If your Data Point names include spaces, they must be replaced with plus signs (which is a Googly standard for handling spaces in URLs). For example, if there are 2 data points named “Plant Light” and “Home Meter 2”, then pointsParm is set to “Plant+Light,Home+Meter+2″.

html += '&email=' + userEmail;

Unless the user has already been authenticated (logged into iGoogle isn’t enough), the e-mail address is a mandatory parameter for the Nimbits REST API calls. Surprisingly, there is no longer a way for a Gadget to automatically determine the user’s Gmail address.

html += '&chds=a&cht=lc&chco=' + colourParm + '&chxt=y&chf=bg,s,EFEFEF';

Autoscale is a very useful setting – so useful that you can’t turn it off in my gadget.  Update Dec 3: Well, autoscale *was* a very useful feature, until it stopped working.  I’ve replaced it with the Google Chart API’s “&chds=a” parameter, which appears to have the exact same effect.

It tells Google Charts to automatically scale and label the Y axis based on the value of the points you are graphing. Without it, the graph is always plotted on a scale of 1 to 100, regardless of the actual value of the data.

For example, here is a graph of 2 Data Points with very different scales: one for a relatively low-power set of devices (DuinoFarm) and one for a big fat PC.

Here is the same graph with autoscale turned off. The scale defaults to 1-100, and data points above 100 are cut off.

The &cht parm is another mandatory one: it specifies the type of graph, with “lc” meaning line chart. Leave it out, and you’ll get the default “polar bear in a snowstorm” chart. There are a bunch of other types of graphs available, as shown in the Google Chart Gallery, but line charts are how I ride.

The &chco parm specifies the colours to be used for the lines in the graph, assigned in the same order as the data points in the points parm. The values are RGB hex codes: for example &chco=FF0000,00FF00, for red and green. Without this parm, everything is orange. (Well, Google being Google, probably whatever colour they feel like using that day. Today, it’s orange).

The &chxt is another one that I always use: it provides the vertical scale. Without it, the graph looks like this:


The last parm in this section, &chf=bg,s,EFEFEF, is responsible for that lovely gray background you see in the screenshots.

html += '&chs=' + graphWidth + 'x' + graphHeight;

This one’s pretty self explanatory. The actual value used in the gadget looks something like &chs=320×200, with the 320 part being hard-coded. The Chart API is quite happy to serve up PNGs that are much larger than this, so if you are displaying a graph outside of a gadget container (such as the “canvas” view displayed when you maximize a gadget) then you can go crazy with megapixel monstrosities.

if (spanType == "Hours")
{
    html += "&st=-" + timeSpan + "h&et=now";
} else
{
    // # of readings
    html += "&count=" + timeSpan;
}

Unlike the last few parms, this one is a Nimbits API parm, not Google Charts. It tells Nimbits which set of data point values to pass along to Google Charts. I’m using a relative timespan, where the end point is always now, and the start point is whatever offset the user specifies in the Settings panel (in hours). The Nimbits API supports a lot of other formats for the start and end time parms, allowing you to chart any time period you like, as documented in the TimeSpan page of the Nimbits Wiki, and a more flexible time span setting is definitely on the To Do list for this gadget.

The remaining settings in the code are self-explanatory, controlling the legend and grid used in the graph.

The resulting tag is plopped into the gadget’s content area, where it gets magically replaced with a PNG of the actual graph. It really is magical, since the response received from REST APIs are usually not so easy to handle.

Which brings us to the Status line…

The Nimbits CurrentValue API

The Status line displays the data point name, its most recent value, and the date and time it was received. All of that information is returned by the Nimbits CurrentValue API. This API’s main purpose is to record new values, not return existing ones, but since it always returns the latest value it’s handy for seeing when the last data was received.

For example, try pasting the following URL into a browser

http://app.nimbits.com/service/currentvalue?point=HomeServer&email=gigamegabot@gmail.com&format=json

You’ll get back a series of tags and values that looks like this :

{"id":7865660,"lat":0.0,"lng":0.0,"d":57.9303189764,"timestamp":"2011-03-15T00:15:03 +0000","pointFK":4405630,"note":""}

This is in JSON format, which is a simplified variant of XML that is handy for Javascript code (and most other languages) to work with.

Unlike the graph, this return string isn’t something we can plop into the gadget’s HTML and display unaltered to the user. Instead, we need to process it with code. And that means that we have to invoke the REST API in a way that passes the result to a piece of code.

In the Google Gadgets framework, this is done by calling one of their APIs:

gadgets.io.makeRequest(url, callback, params);

The “callback” parameter is the name of the Javascript function that will receive the response, and the “params” parameter are primarily used to override Google’s caching algorithm, as explained later. I took the code that invokes makeRequest pretty much verbatim from Google’s documentation for calling REST APIs.

The code which processes the data returned by the CurrentValue API is in the buildStatusString() function:

if (obj.data == null || typeof obj.data == "undefined")
{
	// something went wrong
	if (typeof obj.text == "undefined" || obj.text == null || obj.text == "")
	{
		return obj.error;
	} else
	{
		return obj.text;
	}
}
jsondata = obj.data;

var alldata = "";
for (var key in jsondata) {
	var value = jsondata[key];
	alldata += key + ":" + value + ", ";
 }
 var time1 = jsondata['timestamp'];
 var date1 = new Date(time1);
 // retrieve values as local time
 //var dateAndTime = date1.getMonth() + "/" + date1.getDate() + " " + date1.getHours() + ":" + date1.getMinutes();
 var monthNames = [ "January", "February", "March", "April", "May", "June",
	"July", "August", "September", "October", "November", "December" ];
 var dateAndTime = monthNames[date1.getMonth()].substr(0,3) + " " + date1.getDate() + "  " + date1.toLocaleTimeString();

 var value1 = jsondata['d'];
 value1 = Math.round(100 * value1) / 100;

 return dateAndTime + ' -- ' + value1;

Admittedly, this code is anything but beautiful. It got real ugly real fast when I tried to grapple with API requests that fail.

The code starts off with an attempt to figure out if the response was data or an error message. The object returned from the REST API contains 3 properties. If all goes well, the only one that you have to worry about is .data. Otherwise, .error and/or .text may contain error messages.

JavaScript’s handling of null/undefined variables must make sense to someone, but not me!

If you got data, then JSON makes processing it quite easy. The tags that you saw displayed in the browser are keys to an array, so to retrieve the timestamp value you use code like this:

var time1 = jsondata['timestamp'];
var date1 = new Date(time1);

The time1 variable is actually automatically handled as a date/time object by JavaScript, so why feed it into the Date function afterwards?

If you look at the value that is returned, it looks like this: “:”2011-03-15T00:15:03 +0000″. This is a UTC date and time string, so to make it more meaningful to the user you’ll want to convert it to their local time. In some languages, like Python, this conversion is a major pain.

JavaScript makes it easy: just feed that string into the Date function, and it automatically converts it to your local time when you access methods like date1.getDate() and toLocaleTimeString(). (Locale is different from local, by the way – toTimeString() also converts to the user’s local time.)

Google Gadgets and Caching

Google’s automatic caching of gadget code and data is something that trips up most first-time developers – it certainly got me.

Caching affects gadgets in 2 ways:

1. Gadget code is cached. So, if you change the code, then click refresh on the iGoogle page, an old copy of your code will still run, not the change you just made. This is a major performance benefit for users, but a major pain for developers.

Google feels your pain, sort of. They document this behavior, way down at the bottom of the FAQ, and offer a “My Gadgets” gadget that lets you disable this caching. The gadget has a tendency to flip from its initial state (it’s supposed to look like this screenshot in Google’s documentation) to a minimalistic “Go to developer mode” link – I’m still not sure if that’s a bug or a feature – but “Going To Developer Mode” actually does disable the caching.

2. Remote data is cached. So, for example, calls to the Nimbits CurrentValue API are intercepted by the API framework, which returns the same data as you got last time. This is another performance boost, but at least Google has documented this one properly.

The default behaviour is to cache any request that is within 60 minutes of the last time your gadget got data. You override this by passing an extra parameter in the call to the REST API – in my case, I set it to 5 minutes.

Note that this doesn’t mean your gadget will automatically update itself every 5 minutes – the update will only occur if the user refreshes the browser page, or if iGoogle automatically refreshes itself.

Other Google Gadget Gotchas

Where to host your code?

This is one of the first things you’ll have to decide, since you can’t test your code unless iGoogle can get to it through a web server. A good place to start is let Google host it – this is what happens if you enter your code into the Google Gadget Editor. Google gives you 4M to play around with, and the GGE makes it easy to add the gadget to your iGoogle page while you are testing it.

Google’s Gadget framework isn’t picky though – your gadget just needs to be an .xml file with a URL.

The one hitch with hosting it outside of the Google Gadget Editor is how to add it to iGoogle without publishing it to the world. I find the most reliable method is the Gadget Directory page for Google’s “My Gadgets” gadget . Not the gadget itself, whose “Add a gadget” feature inexplicably disappears, but the web page for the gadget, which has a convenient if equally inexplicable text box labeled “Add a gadget”. Works every time.

Edit Settings: Google’s Playground

As I discovered to my dismay after releasing the Nimbits gadget, you shouldn’t get too attached to the layout of your Edit Settings dialog. The layout is automatically generated by Google’s gadget framework, and you have no control over that layout.

Google can, and will, change that layout without warning or explanation. Other developers have experienced hidden settings that pop suddenly into view, and deleted settings that refuse to go away.

So, don’t go overboard with the options.

Debugging your JavaScript

For years and years, I heard web developers raving about Firefox’s Firebug plug-in without fully appreciating what they were raving about. Actually, I still don’t. But I can now say I’ve used it.

You’ll definitely need FireBug, or a JavaScript console of some sort ,when debugging your gadget. The console is used for catching errors in your JavaScript (it will report both the error and line number it occurred at), and for logging debug information using the JavaScript console.log() function.

I’ve left some console.log() calls in my gadget code because they are a handy way of seeing how the underlying REST API is being called. In the screenshot below, you can see the call being made to the Nimbits CurrentValue API, the JSON-formatted response a few lines later, and the call to the Nimbits Chart API at the bottom. (The console output also contains an error logged by an, ahem, “competing product”. Just kidding – Pachube’s great too, and I’m sure their request wasn’t that bad!)

Actually, FireBug’s functionality is so indispensible that Chrome and Internet Explorer have built similar features right into the browser. Having this built-in Javascript console leads to another common pitfall: don’t assume that a call to console.log() is going to work for other users, who might be running an older or non-mainstream browser. To ensure that console.log() won’t cause the script to fail, you can add the following to the JavaScript (I took the code from Alvin Abad’s blog.)

if (typeof console == 'undefined') {
    var console = {};
    console.log = function(msg) {
        return;
    };
}

Wrapping It Up

As mentioned in the introduction to this article, I’m not terribly impressed with Google’s Gadget framework.

I like the fact that it’s a small set of tools with a relatively shallow learning curve. However, the simplicity of the framework becomes a weakness when you inevitably bump up against some limitation that has no apparent coding workaround.

The documentation is so-so, which is admittedly a notch above the average web development platform’s documentation.

As someone who has been a user of iGoogle on a daily basis for many years, I’m a little alarmed to see how the sausage is made. I can’t shake the feeling that Google has lost interest in moving the platform forward. There haven’t been any significant changes to the API or user interface in a couple of years, and they haven’t bothered to expand, or even clean up, their documentation.

Google Gadgets are a fun place to visit, but I wouldn’t want to make a living there.

Posted in Electronics, Programming | Tagged , | Leave a comment

A Nimbits Gadget: Minding the Data Store

Updated June 4: I’ve corrected a couple of things, as marked in blue. Also, developers might be interested in my follow-up blog entry A Nimbits Gadget: Programmer’s Show And Tell

Updated Dec 3/11: I fixed a broken feature: autoscaling of the Y axis was no longer working, resulting in values above 100 being cut-off.  

Updated Dec 14/11: I’ve added a couple of new settings: Graph Width and Gadget Height.  For details, see the “Settings” section below.

As mentioned in my post about the data-logging service Pachube, I found the best way to monitor my datastreams was with the Pachube Google Gadget.  Stick it on your iGoogle page, and it allows you to quickly and conveniently keep an eye on your data without having to go out of your way to find it.

When I started logging data to the open source Nimbits system, the first thing I went looking for was some equally convenient way to keep tabs on my Nimbits data. While Nimbits offers a lot of built-in methods for monitoring your data, such as e-mail, Google Talk, and Twitter, I decided that nothing worked as well for me as that Gadget. So, I decided to write a Nimbits Gadget.

The Gadget displays up to 3 line graphs, with each line graph containing as many Data Points (i.e. datastreams) as you like. You can optionally include a Status line showing the last data point received.

You can grab the gadget from its Google Directory page.


In this post I’ll describe the settings for the Gadget, and in the next post I’ll write a bit about the Gadget code and the underlying Nimbits API.

What You’ll Need

If you’ve never used Nimbits before, you’ll need to start posting data to it before the Gadget will do you any good. The Gadget is display-only – it won’t help you get data into the system. The process of creating and configuring a Nimbits account and adding Data Points to it was described in an earlier post.

One major caveat: the Data Points that the Gadget displays must be Public. Admittedly, this is just to keep the coding relatively simple. Private data points require authentication, something which can be tricky from a Javascript application like a Gadget, though I expect (hope?) that this will be streamlined somewhat by the shared Googly underpinnings of Nimbits and the gadget. If it works, I’ll add authentication in a future update of the code.

Settings

The settings dialog, shown below, is longer than the average gadget’s. Not to brag, or anything.

 

Most of the options are there so that you can tinker with the gadget appearance and features to see what suits you best. The defaults should work fine, but there are a few settings which you’ll need to supply. Let’s start with those.

Required Settings

Server

If you are using the public Nimbits server, then leave this at the default, app.nimbits.com. If you’ve installed your own copy of the Nimbits code on the Google App Engine, then fill in the name of that GAE instance (usually your-app-name.appspot.com)

Data Points

In Nimbits, a Data Point is the unique name assigned to a specific datastream. For example, in the screenshot to the right there are 2 Data Points, Home Meter and Plant Light. (Power Usage is a category – the Gadget only works with Data Points, not Categories).

Enter the name of a Data Point in the textbox then click the Add button. You’ll then see it listed below the textbox.

To add a second Data Point to the same graph, enter its name in the textbox then click the Add button. (You have to add them one at a time, rather than typing them in all at once).

To display a Data Point in a separate graph, click the “Display Graph 2?” checkbox to turn it on. Fill in the name of the Data Point in its textbox then click the Add button.

The following screenshot shows 2 Data Points added to the 1st graph, and 1 Data Point in the second graph.

To remove a Data Point from a graph, click the “X” next to the Data Point name. To remove a graph entirely, turn off its checkbox. (OK, here’s the thing: You can’t remove a graph by removing all of its Data Points – Google’s idea, not mine. The gadget “list” interface pretends to remove them, then adds back the last item when you’re not looking. Apparently, that is not evil.)

Your Email Address

This field is mandatory when using the public Nimbits server (i.e. when the Server fields is app.nimbits.com). That’s how Nimbits figures out which Data Points belong to you.
The field is optional if you’ve setup your own Nimbits server using the Google App Engine.


Update June 4: Actually, the e-mail address is optional when using your own Nimbits server only if you’re authenticated with the backend Google App Engine. Since I’m not sure what triggers that authentication, you should just go ahead and fill in your e-mail address.

Graph Height

This determine the height, in pixels, of the graph. The default of 200 will only allow one full graph to fit. A size of 80 will allow 2 graphs with Status lines to fit.

Graph Width   [Added: Dec 14 2011]

This determines the width, in pixels, of each graph.  The default of 310 pixels fits snugly in the gadget container on my desktop PC, but the optimal width will vary depending on the browser and device you’re using.

Gadget Height   [Added: Dec 14 2011]

This determines the height, in pixels, of the gadget container.  The default setting is 240 pixels, which is big enough for a couple of graphs and status lines.

Other Settings

Time Span

The amount of data displayed on the graph is specified in the Time Span field. You can enter it either as hour or as # of readings. In either case, you’ll get the most recent data: that is, the most recent x hours of data, or the most recent x readings.

By default, this is set to display the last 24 hours of data.

Legend

This option determines whether a list of the Data Points and their colours are displayed at the top of the graph. This option is particularly useful if you have multiple Data Points in a graph. Turning the legend off frees up more vertical space for the data in the graph.

Grid Style

This option determines the grid lines displayed on the graph: None, Vertical, Horizontal or Both. You can have any number of grid lines you like, as long as that number is 10.

Show Status

This option determines whether a status line is displayed above each graph, giving the name of the first Data Point and the date and time of the last reading. The status is particularly useful to identify that your data stream has stopped working, since the line graph won’t make that outage apparent. (Integrating downtime into the graph is definitely a top priority for future updates of the gadget.)

The downside to displaying the status is that it slows down the refresh time of the gadget, and (ahem) the code doesn’t always work. See the Troubleshooting section below if the Status line displays something other than a status.

Troubleshooting

Blank or missing graphs

When something goes wrong, you generally end up with a blank graph rather than an error message. That’s kind of by design: if there is no data matching the settings, the Nimbits API will dutifully provide an empty set of data to Google Charts, which will dutifully display it.

If the graphs are missing entirely, then the gadget likely couldn’t find a Nimbits server, so check your Server setting.

If blank graphs are displayed, try turning on the Legend and Status options in the settings – they may give an indication of what went wrong.

If the legend is labeled “Doesn’t exist”, then the Data Point might not have been correctly entered in the settings.

Status line errors

The Status might be set to one of the following errors:

  • The requested URL /service/currentvalue was not found on this server. Check the server name in the settings. This error means the server exists, but Nimbits isn’t running on it.
  • Could not identify user. Check the email address field in the settings.
  • Status unknown or Undefined. Unfortunately, there seem to be a variety of causes to this one. The problem isn’t with the settings – it’s likely something I can fix in the code. So, don’t blame yourself, blame me. Fixing this is a priority for future versions of the gadget.

Slow updates

If the Gadget takes too long to update itself, try turning off the Status option in the settings, and/or putting multiple Data Points on the same graph. Each graph and each status field require an extra call to the Nimbits server. Three data points on one graph take much less time to appear than 3 graphs.

Data isn’t updated often enough

Chill, dude! Updates to the graphs and status will only appear every 5 minutes, no matter how much you click the browser Refresh button. This is by design. Google caches the data used by the gadgets, and I’ve set the cache length for 5 minutes.

If you don’t manually refresh the browser page, iGoogle will update all of the gadgets just once per hour.

Some graphs don’t fit

I left the Gadget height at its default, and there is no way for the user to increase the size of a gadget in iGoogle. You can fit more graphs in the space by reducing the Graph Height in the Settings dialog. The default of 200 will fit just one graph; a Graph Height of 80 should fit two graphs with their status fields.

You can see an expanded view of all graphs by clicking Maximize control button on the titlebar, or by selecting the Nimbits gadget in the list on the left of the iGoogle screen.

Wrapping Up

If you have any problems, or requests for features in future versions of the gadget, please leave a comment below.

Posted in Gadgets | Tagged , , | Leave a comment

Netduino: A Little Help From Its Friends

Note: This article is an installment in a series about building a plant light controller (and beyond) on the Netduino microcontroller platform.  For previous articles in the series, click here.

If you’ve made the switch to programming with .Net Micro Framework devices like the Netduino and GHI Panda, you inevitably find yourself asking the question: “What the heck do I do with all my Arduinos?”  They have a minimalistic IDE and no debugger to speak of.  They make lousy cat toys and anniversary gifts. (Unless you’re receiving one, that is.)  Who needs them?

Well, actually, you do. You might think that .Net’s “managed code” runtime environment is there to usher your program past the hardware, but it’s more like a bouncer protecting the hardware from your crummy code.   It’s great that your code can’t mess with the hardware, unless you intentionally want to mess with the hardware.

I ran into this problem when trying to upgrade the Light Controller’s analog temperature sensor with a digital “one-wire” sensor.  These types of temperature sensors have become quite common, and are generally much more accurate than analog sensors. While the Netduino’s firmware doesn’t officially support the one-wire standard yet (as of Version 4.1.0 Update 6), it is available in the GHI firmware, and I was planning to incorporate it into a Panda version of the Light Controller.

Unfortunately, not all “one wire” temperature sensors are created equally, and the one I have doesn’t follow the Dallas Semicondutors / Maxim 1-wire standard. It’s a Seeed Studios Temperature and Humidity Brick (now replaced by a Twig version) , both based on the DHT11/AM2302 chipset.

The interface to this standard is well documented (quite enthusiastically documented, in fact).  As demonstrated in the Arduino sample code, communicating with the chipset requires toggling a pin high-and-low at specific sequence of microseconds.

That’s microseconds, not milliseconds.  .Net MF doesn’t let you send signals at the microsecond level.  That’s dangerous, son.  Might shoot your eye out.

Wouldn’t it be nice if we could combine the productivity of managed C# code with the versatility of the Arduino’s AVR-C code library? Mush them together somehow, with solder and duct tape.  Hmmm…

Sadly, I happened to be out of duct tape, so I turned my attention to a Serial Enabled LCD Kit that I had recently bought from Sparkfun.  I was looking to replace the parallel LCD that comes with the Seeed Grove Starter Bundle with a serial LCD to free up some pins.

I hadn’t considered the upgradable firmware on this model as a big benefit – there isn’t much that you can do with an LCD beyond the basic set of commands that all serial LCDs support.  The kit actually incorporates an ATMega328 with the Arduino bootloader — a full Arduino, essentially.  The Arduino code reads the occasional LCD update command and passes it along to the LCD.  Between LCD updates, it sits about, twiddling its thumbs, waiting for something to do.  Hmmm…

Maybe we could use the Kit’s ATMega328 as an interface to the temperature sensor?  That’s so crazy, it might just work.

The Sparkfun Serial Enabled LCD Kit

And it does work.  (Most of the time, as I explain below.)

The Sparkfun Kit’s design makes it quite easy to implement this type of project.  It breaks out the standard 5 pins used by the FTDI cable (the serial-to-USB cable I previously used with the Light Sensor’s dearly departed UART interface), and also breaks out all of the Arduino pins not used by the LCD.

The LCD interface “firmware” is just a standard Arduino program, nothing weird, so it’s easy to merge in the sample code for reading the DHT11 sensor – mostly just copy and paste.

While I was at it, I also added some code to interface to the Arduino’s built-in EEPROM memory.

So, the end result is that the Plant Light Controller can now display the humidity, a more accurate temperature, and save the settings from the menu too.

Actually, the “more accurate” temperature is debatable.  The DHT11 chip always returns temperature as an integer, and always in Celcius.  It appears to truncate the actual temperature, rather than rounding it.  So, the temperature can be off by .9C, or 1.6F.  Humidity is similarly truncated to an integer.  But, hey, I’m not complaining – at $5, Seeed’s sensor is still a steal.

Pin Assignments

From the Netduino’s standpoint, not much changes from the previous installments of this project.  All communication with the Arduino goes through the same serial pins used by the LCD interface.  However, since the LCD now talks back, we need to connect a Netduino Serial Rx pin to the Serial LCD’s Kit TX pin, which is part of the FTDI breakout on the end of the backpack’s circuit board (the right end of the photo below).  My code uses Netduino’s COM2 (pins D2 and D3) for this.

Sparkfun Serial LCD and Seeed Temperature/Humidity Sensor
Sparkfun Serial LCD and Seeed Temperature/Humidity Sensor

From the Serial Kit’s standpoint, the following pins are used:

-          The white JST plug contains the 5V, ground and RX pins, which we connect to the 5V, ground and TX (D3) pins of the Netduino

-          The TX pin at the end of the board is connected to the RX pin (D2) of the Netduino

-          The breakout pins at the bottom of the Serial Kit are used to connect to the Temperature/Humidity Sensor: specifically, the 5V, Ground and Analog 0 pins.

User Interface

Not much has changed here from the last installment of the project, either.  As shown in the photo below, I’ve used the leftover space on the Temperature row of the LCD to display the humidity instead of the Sunset time.

The number in parentheses is the temperature value from the sensor – the other temperature value is from the analog sensor, as added to the project in an earlier installment.  Which one is more accurate depends on how well your analog sensor is behaving – as mentioned, the digital sensor’s temperature can be off by 0.9C, while my analog sensor is apparently a little high.

Demonic LCD displaying strangely moderate temperature and humidity
Demonic LCD displaying strangely moderate temperature and humidity

I’ve also added a couple of new items to the “Timers” section of the menu, allowing you to disable the plant light timer.  This became necessary once I started saving the menu settings to EEPROM, since that ruined the handy Disable-the-timer-by-turning-the-gadget-off-and-on ™ feature.

Programmer Show And Tell

The Serial LCD Kit is now the 4th LCD interface supported by the code, joining the parallel LCD, the Matrix Orbital Serial LCD, and the Sparkfun Serial Backpack.  I considered making the LCD Kit fully compatible with the Sparkfun Backpack’s command set by changing the Arduino “firmware” code.  The 2 command sets are quite similar, differing only when in the cursor positioning and backlight control commands.

However, I like the Serial LCD Kit’s API better, so I subclassed the Backpack’s clsLCD_SF class and overrode the commands that differed.

Subclassing is supported the same way in .Net Micro Framework as in the full blown .Net:


public class clsLCD_SFKit : clsLCD_SF

Overriding a class method is also done the standard C#, using the “new” keyword.  So, to replace the cursor positioning method with code that works with the LCD Kit’s firmware:

new public void SelectLine(byte intLine, bool blnClearLine)

I added some new commands to the LCD class, used to receive data from the Serial Kit’s Arduino.  Since its unlikely that the other LCDs are going to do much talking, I didn’t add these  methods to the common ILCD Interface.  Instead, when the main program wants to check to see if the LCD is of the Serial Kit variety, it uses the “is” keyword, again just like in the full .Net Framework:

if (lcd is clsLCD_SFKit)
{
// get the humidity
((clsLCD_SFKit)lcd).RequestHumidityAndTemp();

Data from the Serial Kit comes in through the Netduino’s serial RX pin, just like we used to read terminal commands through the UART interface.  So much like it, in fact, that I used the same event handler.  When a humidity reading or EEPROM setting comes back from the Serial Kit, the code raises a uart_Command event, as described  back in the Time and Weather article.

There is actually one big difference between the Serial Kit’s behaviour and the UART Telnet terminal, though.  The Telnet terminal could send us a command at any time, so an asynchronous interface using an event was necessary.  The LCD, on the other hand, sends data only when we ask for it.  It would have been possible, therefore, to read the data synchronously after we send the request, using code like the following:

public void RequestHumidityAndTemp()
{
byte[] command = new byte[2];
command[0] = CMD_SPECIAL;
command[1] = 71;
lcd.Write(command, 0, command.Length);
// give the LCD time to retrieve the data
Thread.Sleep(1000);
intBytesRead = lcd.Read(readBuffer, intBufferOffset, lcd.BytesToRead);
. . .
}

This has the advantage of making the context of the data we are reading obvious, but the disadvantage of introducing timing problems.  What if the LCD doesn’t respond fast enough, or the response isn’t received at all?  How long should we wait for the LCD to respond, and at what point does the Netduino decide that a response isn’t going to be received, and carries on?

Asynchronous communication does away with that problem.  In fact, you can unplug the LCD altogether, and the Netduino will faithfully carry on with its other duties, turning on and off the plant light.  It won’t receive any humidity readings, and its request to save menu settings won’t be performed (good luck using the menu without an LCD, by the way), but the Netduino doesn’t notice.

The downside of asynchronous communication is that data doesn’t necessarily come back when you want it.  You’ll see this when you first plug in the Netduino – although it requests the humidity from the Arduinio during startup, the response may not be received before the first “display the temperature” timer fires.  As a result, it may take another 30 seconds (the interval of the temperature timer) before the humidity is displayed.

The code I added to the Arduino to save settings to EEPROM takes advantage of the Arduino EEPROM library – the library makes it simple.  One minor twist is how to handle the very first interaction between Netduino and Arduino.  The Arduino has no idea whether it has any settings saved – it just returns whatever happens to be in the address you’ve requested.  The first time, this will be 0s.  So, all settings are saved as non-zero values: I add 1 to the values where necessary.

One other design decision worth mentioning is the issue of which settings to save, and when to save them.  I decided to save the date, but not the time, since it’s likely that the date will be unchanged, or very slightly changed, when you power the Netduino back on, but the time will almost certainly have changed, perhaps by hours.

The other downside of saving the time is that you’d have to do it every minute, putting wear and tear on the EEPROM.  It is best to write to EEPROM as little as possible: in this case, only when the user changes a setting in the menu.

Fortunately, I already had a callback method for responding to the user’s menu selections (as described in the Ordering Off the Menu article), so I added code like the following to that method.

static void setValueFromMenu(string menuCode, Int16 intValue)
{
. . .
if (menuCode == "_CELCIUS")
{
. . .
SaveSettingToEEPROM(ADDR_TEMP_FORMAT, 2); // 2 means Celcius

With Friends Like That…

This configuration actually did a good job of proving the point about unmanaged code: it’s awfully easy to crash the microcontroller when running it.

When I got things up and running, I found that the LCD would freeze at some point, generally a few hours after powering it on.  The Netduino soldiered on, faithfully turning the plant light off and on, so clearly it was the Arduino that dropped the ball.

Commenting out the code that requests the temperature and humidity reading caused the problem to go away, so whatever was going wrong involved the DHT11 interface.

The right thing to do would be to debug the Arduino’s DHT11 code, but the easy thing to do was to just “reboot” the Arduino on a regular basis.

Crude, but effective.  I’ve had this configuration running for over a month now, and it works like a charm.

The “rebooting” is done by using a transistor to turn the Arduino’s power off and on.  This requires a hardware change: instead of connecting the Sparkfun LCD Kit’s ground pin directly to the Netduino, I used a small breadboard to route the connection through a transistor.  The transistor is controlled using the D12 pin.

Specifically, the configuration I used (shown in the photo) is:

-          Collector pin of 2N2222 transistor to LCD Kit’s ground

-          Emitter pin of 2N2222 to Netduino ground

-          Base pin of 2N2222 to D12 pin of Netduino

-         Base pin of 2N2222 to ground through pushbutton – this can be used to manually boot the LCD Kit Arduino if necessary

Arduino Reset Circuit
Arduino Reset Circuit

The Arduino is powered up during the Netduino code’s startup:

arduino = new OutputPort(SecretLabs.NETMF.Hardware.Netduino.Pins.GPIO_PIN_D12, false);
arduino.Write(true); // turn on Arduino
Thread.Sleep(2000); // give Arduino a chance to do its initialization
lcd = new clsLCD_SFKit(SerialPorts.COM2, 9600, 2, 16);

I added a new method to reset the power to the LCD Kit’s Arduino by temporarily setting D12 low:

static void ToggleArduino()
{
arduino.Write(false);
Thread.Sleep(1000);
arduino.Write(true);
// let Arduino do its initialization
Thread.Sleep(2000);
// restore the brightness setting
lcd.SetBrightness(bytBrightness);

The new method is called when the Netduino hasn’t received a response from the Arduino for over an hour.  It asks for the temperature and humidity every 30 seconds, so an hour is obviously lenient.  If you need the temperature/humidity readings to be more reliable, you should shorten this interval:

// if we haven't received any data from the Arduino for over an hour, reboot it
if (datlastArduinoRead != DateTime.MinValue)
if (DateTime.Now.Subtract(datlastArduinoRead).Hours > 0
datlastArduinoRead = DateTime.MinValue; // prevent another reboot
ToggleArduino();

The voltage out of the Netduino’s digital pins is 5 volts, so why bother with the transistor – why not power the Arduino directly from D12?  The problem is current: the Netduino (and pretty much every microcontroller under the sun) only allows a trickle of current to flow through its pins.  You can power an LED with it, but anything more power hungry needs to get its voltage another way.

Wrapping Up

This installment of the Light Controller project is admittedly off on somewhat of a tangent: connecting a temperature sensor to a Netduino via a Serial LCD interface is not a common configuration.

However, offloading some of the Netduino’s responsibilities onto another processor is actually quite a handy technique.  There are a lot of microcontrollers out there, each with its own libraries of code capable of doing some things that a Netduino can’t (yet), so when looking to add a new feature it’s smart to think “outside the box”.

One other improvement I’ve made to the project is to put the source code into a proper repository, rather than just linking each article to a .zip file.  I’ve created a Google Code project named gigamega-micro.

You can get the code for each of my Netduino articles (starting with Let There Be Light) in 2 ways:

-          If you are familiar with Subversion use SVN to get the latest code.  If you want the older version of the code to match one of the past articles, get the corresponding revision (see the revision list here http://code.google.com/p/gigamega-micro/source/list)

-          Or, download the good old .zip file from the Downloads section of the Google Code project. Here is a direct link to the code for this article.

The modified Arduino code for the Sparkfun LCD Kit can also be found in the Downloads section of the Google Code page (direct link to .zip file).  90% of that code is copied from the original Sparkfun firmware or from Sheepdog Software’s DHT11 page.  I’ve left the original authors’ headers in the code.

Don’t worry, I haven’t forgotten the “road map” of hooking the Netduino up to an XBee.  We’ll get back on track in a future article.

Posted in .Net, Electronics, Programming | Tagged , , | 1 Comment

Tweet-A-Watt: Beyond the Twitter, Part Deux

This article continues my earlier post about web-based services for monitoring your electricity usage using Tweet-A-Watt. Last time I focused on Pachube – today I’ll explain the process of connecting a Tweet-A-Watt to Nimbits.

Nimbits Overview

Nimbits has a lot of similarities to Pachube: it is a “cloud-based” data repository with a browser interface for configuring your feeds and viewing your data. Both services provide a REST API to allow applications to input and output data, though for the purposes of Tweet-A-Watt you can get by with just barely scratching the surface of what the API offers.

If you’re a developer and/or paranoiac, a key advantage of Nimbits is that it is an open source project based on the Google App Engine. It is quite easy to install Nimbits on your own Google App Engine account, as I’ll show you below. This allows you to “own” the data, updating it as frequently as you like and keeping it for as long as you like.

GAE does have limits on how much data you can toss about without paying. According to Google’s “What is Google App Engine” page: “All applications can use up to 500 MB of storage and enough CPU and bandwidth to support an efficient app serving around 5 million page views a month, absolutely free”. Data compression is one of the key strengths of the Nimbits software, so it’s safe to assume you aren’t going to comes close to that limit unless Rebecca Black decides to sing about your electricity usage. (It’s also safe to assume that she won’t).

The Google App Engine is widely used, and extensively documented, so it’s certainly a solid foundation for experimenting with ways to process and display your Tweet-A-Watt data. The Nimbits code is all written in Java. So, if you know GAE and Java, the sky’s the limit for what you can do with Nimbits. Personally I know neither, but as with Pachube it’s quite easy to get up-and-running with Nimbits with a bit of configuration and a few lines of code.

Configuring a Nimbits Account

You have 2 options for hosting your data: use the public Nimbits Server at app.nimbits.com, or create your own using a Google App Engine account. You’ll probably want to use the first option to test your Tweet-A-Watt connection and get familiar with the Nimbits interface, then graduate to the 2nd option later. I’ll go through both processes, beginning with the public server.

1. Open app.nimbits.com in a browser. You’ll be prompted to login using a Gmail user-ID (or if you are already logged in, to confirm that you want to use that login ID).

Wait, what? Nimbits is part of the Google empire?

No, but Google App Engine is, and it not surprisingly uses Gmail as the primary type of authentication. In fact, some parts of Nimbits (and GAE) are closely tied to Gmail – your alerts can only be sent to that Gmail account.
However, if you’re a developer you might want to use a secondary Google account here. Your applications will connect with either Gmail authentication or with a Nimbits-specific API key. While the API key works fine for all Nimbits features, if you delve deeper into the underlying Google App Engine framework you might start experimenting with GMail authentication. Hard-coding your primary Gmail password into a program is like looking death in the eyes!
Incidentally, if you are generally logged into Gmail all day, you’ll find the “Private Browsing” (aka “Incognito Window” aka “Porn Mode”) setting of your browser to be a handy way of logging in with a secondary Gmail account without logging out of your main one.
2. Your browser will display the standard Nimbits user interface, as shown below. To configure your Tweet-A-Watt feed, Click” New Data Point” in the left frame.

A Nimbits Data Point is like a Pachube Datastream. But — get this — it has a name instead of a number. Far out! Actually, you can make things a little easier on the coding end (but harder on the viewing end) by setting the Data Point name to the sensor # in your wattcher.py script (i.e. “0” for the 1st Tweet-A-Watt sensor, and so on). You can’t rename a Data Point, though, so choose carefully.

3. The new Data Point will be added to the frame on the left. Further configuration is optional – the default settings will allow you to save Tweet-A-Watt readings and view them in a graph.  However, there are some neat features built into the configuration screen, so lets have a look around. 

4. Double-click on the Data Point.  A couple of windows will open in the main frame, as shown below.  Click Point Settings on the menu bar, then the Data Point name. 

5. The “General” tab of the configuration dialog is shown below, with the settings I use.  By the way, this dialog is also the only way to delete a Data Point (using the toolbar icon). 

6. The default Compression is 0.1, meaning that a new reading that is within 0.1 of the previous reading won’t be stored.  This setting is handy if your data source is bombarding the server with several readings per second, but your Tweet-A-Watt will likely be sending  just 1 reading every 5 minutes.  If your electrical usage tends to be rock steady for long periods, you can turn off Compression by entering zero.

7.  The default History of 90 days means only the most recent 3 months of data will be stored.   You’ll probably want to increase this at some point, especially if you setup your own Google App Engine account 

8. The Unit of Measure for Tweet-A-Watt is Watts, or Volt-Amps if you insist.

9. I left the setting at Public. Don’t worry – this doesn’t mean that the unwashed masses can post data to your feed; just that they can view your feed through the REST API.  (Don’t worry – they won’t.) The Public setting makes playing around with the API a little easier, especially when connecting through 3rd party software like Excel, so I’d suggest you leave it on until you’ve got things working.

10. The Messaging options can be used to inflict all of your data readings on your friends or unusually social enemies.   This might be a good way of converting the former to the latter.

11. In the Alerts tab, you can configure a high and/or low threshold at which an alert will be triggered.  Here, the messaging options become more useful, and I use the Instant Message setting for my Tweet-A-Watt feeds.  (See the “Google Talk” section, below).

12. The other configuration setting that I use is the Idle Alarm, which was recently added to Nimbits.  This lets you know when your Tweet-A-Watt configuration stops working, which I find usually means my (highly butchered) Python “wattcher” script has stopped.  Back when my Tweet-A-Watt merely twittered, I would sometimes go a couple of weeks before realizing that the tweets had stopped. (Strange that one of my, ahem, 2 followers didn’t complain.)

13. That’s it for Data Point configuration. Click on Save.

13. If you have multiple Tweet-A-Watt units, you can create other Data Points for each one.   Remember, the Data Point name uniquely identifies the data stream.

14. You probably noticed that there is something called a “Category”, which is a group of related Data Points (the equivalent of Pachube’s “Feed”).  Categories are optional, and are most useful if using the API to “walk” through the server’s Data Points.  To create a Category, click on the Green folder icon, then drag each Data Point into the new Category in the left frame.  You may have to refresh the browser page before the drag-and-drop works. 

15. One last thing before you are done with the Nimbits side of the configuration: create an API key.  As mentioned earlier, this allows you to post data to your feed without dangling your Google password out to the Internet.  Click on the Secret Key icon on the toolbar, then follow the instructions.  Note that, unlike Pachube, there is only one Key for your account – generating a new one invalidates the previous one.

Tweet-A-Watt configuration

The modifications needed to the Tweet-A-Watt wattcher.py script are a little different from those for Pachube, as described in the previous article.  For Nimbits we’ll invoke the REST API directly, while for Pachube we used a separate library (eeml) as an interface.

Add the following code to  update_graph method, in the same section that the Pachube code was added in the previous article.  You should, of course, replace [Data Point Name], [e-mail address] and [secret key] with the values used to configure your Nimbits account (without the square brackets) [got it?].

try:
  urllib.urlopen("http://app.nimbits.com/service/currentvalue?value=" +
     str(avgwattsused) +
     "&point=[Data Point Name]&email=[e-mail address]&secret=[secret key]")
except IOError:
    print 'Error sending to nimbits

 

If you have multiple Tweet-A-Watt sensors, and you decided not to use the sensor # as the Data Point name, you’ll have to add some Python code to set the “Data Point Name” parm based on the sensorhistory.sensornum variable.

After restarting your wattcher.py script, switch back to the Nimbits page in the browser.  If everything’s working, the “Current Values” window in the main frame will display a value and timestamp sent by your Tweet-A-Watt within a few minutes.

The graph on the Nimbits page takes a little getting used to.  For one thing, it uses Adobe Flash, so forget above viewing it on your iPhone. By default, it displays the last hour of data.  Well, actually, it displays an hour of data – the hour specified in the 2 text boxes at the top of the graph.  Clicking the Refresh icon doesn’t refresh the graph, because the textboxes aren’t changed.   Similarily, clicking the “1d” link doesn’t display the last day of data, because the freakin’ textboxes aren’t changed.  Wah!

Don’t blame Nimbits for that, blame Google. It’s their graph.  Officially, it’s an “Annotated Time Line” (without annotations), as documented on the Google Vizualization API site .

So, here’s how it works: enter a date and time range in the text boxes, then click the graph’s refresh button.  You can now use the links (1h, 1d, etc.) to zoom in on a window of data of that length.  You can then use the slider at the bottom of the graph to scroll a zoomed view of your data over the full time range.  In the screenshot below, the line graph shows electricity usage for an hour ending about 11 pm on March 14, while the bottom part of the chart allows you to use the slider to scroll though the full time range between Mar 12 and Mar 14.  (Hey, don’t judge!  My ‘duinos have special needs.  And 2 plant lights.)

If that doesn’t float your boat, other charts and graphs are available.   The Google Visualization API Gallery shows the full list, any of which can be displayed through the REST API.  The Nimbits page has just 2 built in, the Annotated TimeLine and the Motion Chart.  I’m not entirely sure how the latter works, but if you click on it enough it starts to act like a game of Asteroids.  (The Motion Chart appears to have been dropped from Nimbits 3.1, which is good since I suck at Asteroids.) 

Google Talk

I was initially skeptical of the social networking features, since I’m antisocial.  However, I’ve found the Instant Messenger feature to be quite handy, since it doesn’t require any actual friends.  (Pachube doesn’t offer any prepackaged Instant Messenger connectivity at present).

This feature leverages the Google App Engine’s interface to “any XMPP compatible instant messenger client”.  The only one that I’ve tried is Google’s own “Google Talk”.  I think this is the best route, since it is integrated with your Gmail account – if you aren’t online when your Nimbits server has something it wants to send you, it will e-mail you instead.   You’ll need to install the Google Talk desktop software, and login to Google Talk with the same Gmail ID with which you log into Nimbits.  You’ll also need to enable the Instant Messenger option in Nimbits, as described in this article on the Nimbits blog.

There are basically 3 ways that you can use the Instant Messenger interface:

  1. Receive alerts when your data passes a specified threshold.  This is configured in the “Alerts” and “Idle Alarm” tabs, as shown earlier.  Although it isn’t obvious from the “Idle Alarm” dialog, it appears to use the same settings as Alerts to determine where to send the Alarm.
  2. The interactive interface.  The commands are described in the Nimbits blog post.  This is particularly useful if you are using a non-Android smartphone (there is a Nimbits app for Android).  Incidentally, I didn’t have much luck on my iPhone using Google’s iPhone-specific Google Talk web page, but there are a bunch of apps that support Google Talk.  I use the IM+ app for this.
  3. Receive every new data reading.  This is what will happen if you enable Instant Messenger on the “General” tab.  Possibly useful for getting excused from (or kicked out of) meetings and awkward social occasions. 

Rolling Your Own

Once you are up-and-running with the public Nimbits Server, you should consider configuring your own AppSpot site using the Google App Engine.  It is surprisingly easy to do, and almost entirely maintenance-free.   About the only time you’ll have to fiddle with the controls is if you decide to update your version of Nimbits.

The browser interface and API is identical, so there’s no additional learning curve.  On the down side, there is no easy way to export your data from the public server to your own.

The instructions here (beginning with the “Getting Started” section) do a good job of walking you through the process – it really is that simple.  I’ll add a few observations on the process:

  • The suggestion about using a secondary Gmail account also applies to registering for Google App Engine. If you are a developer, there are some things that your code can do only when authenticating through Google, and you definitely don’t want to be a position of hard-coding your main Gmail password in any app.
  • When you enter your application name in the Google App Engine page, be sure to use only lower-case characters.    When I was setting up mine, I couldn’t believe that every application name I tried was taken according to the “Check Availability” button.  “C’mon, someone used GigaMegaAlphaBeta!  Do I have a stalker?”  Turns out that upper case letters aren’t allowed.  (They’re evil).
  • Be sure to download the Java App Engine SDK, not the Python one.  Even if you won’t be doing any developing, you’ll need the Java deployment tool – it’s the “install program” for Nimbits in this case.
  • If this is the first time you’ve deployed Nimbits to GAE, you should find that the most Nimbits version is already set as the default.  If you later install an update of Nimbits, then you’ll have to manually change the version in GAE – the new version doesn’t automatically go “live”.
  •  In the “ServerSettings” page, the only setting I had to change was admin (which should be your Gmail address).  The rooturl setting can be left at http://app.nimbits.com.

After finishing with the ServerSettings, you can open <yourappname>.appspot.com in your browser, and you should be presented with the exact same interface as at app.nimbits.com.  You’ll have to recreate your Data Points, using the same process as described earlier.  At the Tweet-A-Watt end, just replace “app.nimbits.com” with “<yourappname>.appspot.com” in the wattcher.py script

e.g.

 urllib.urlopen(<a href=http://gigamegaalphabeta.appspot.com/service/currentvalue...

Note: I didn’t actually use gigamegaalphabeta.  It’s all yours. Stalker.

Wrapping Up

I had originally planned to write a bit about the Nimbits .Net SDK. Unlike Pachube, Nimbits provides a .Net DLL, as well as supporting calls through a REST API.  The .DLL uses an intriguing technology that I hadn’t encountered before, making its Java SDK library accessible to .Net through IKVM.  I’ve been tinkering with it for awhile, and it’s quite cool to see Java data types presented as .Net classes.

There isn’t a lot of documentation on the .Net API, so I plan to give some code examples in a future blog post.  I’ll also explore what I think will be a common coding requirement for Tweet-A-Watt users: importing historical data from the TAW .csv file into the Nimbits server.

Posted in Electronics, Programming | Tagged , , , | 1 Comment

Tweet-A-Watt: Beyond the Twitter

As I’ve written before, I really like the idea behind Adafruit’s Tweet-A-Watt project, which uses a modified Kill-A-Watt box and an Xbee transmitter to create an inexpensive home energy monitor. I’ve been running one 24/7 for over a year, and it has proven to be remarkably stable.

However, while the eponymous Twitter feature is (was?) novel and catchy, it’s not the most convenient way to track your energy usage. The Tweet-A-Watt project page includes a well written tutorial for using Google App Engine to graph your energy usage. It looks like a great weekend project, especially if you want an excuse to learn Python or the Google App Engine API, but I never got around to freeing up a weekend for it.

As an alternative, you can hook up your Tweet-A-Watt to an existing data tracking service with just a few lines of code. I’ve used a couple of “cloud-based” services for this, Pachube and Nimbits. They each have some features and advantages worth considering. I’ll cover Pachube in this article, and Nimbits in the next one.

Pachube

Pachube is a good choice for those who aren’t DYI software enthusiasts. It’s designed to get you up-and-running quickly, with as little software configuration as possible. Their site is easy to use, well documented, and filled with tutorials and examples. It has been around for a few years and is now widely used, so you can Google “Pachube” to find a lot of videos and articles showing how to use the platform.

The downside to using Pachube is that your data is stored for only a month, unless you sign up for one of their (fairly expensive) paid plans. There are also limits on how much data you frequently you can log or retrieve data, though those limits won’t affect Tweet-A-Watt usage. Still, there is nothing to stop you from using Pachube for day-to-day monitoring, and other software (such as Nimbits) for historical archiving and other more advanced usage.

Getting Connected

Here’s a walkthrough of how to get your Tweet-A-Watt connected to Pachube:

1. Register at Pachube.com.

2. You’ll need an API key, which you’ll paste into the Tweet-A-Watt wattcher.py script. While you can use your Master API Key, it is much safer to generate a separate API key for use by Tweet-A-Watt. (Wouldn’t want evil environmentally-conscious hackers posting their Tweet-A-Watt data to your feed, would you?)

Go to “My Settings” page in Pachube. Click on “New API Key”, then select the “all” checkbox as shown in the screenshot below. (There are other options that restrict how the API key is used, such as setting the source domain to your ISP, but when I played with those options in February the only one that worked for me was plain old “All”).

Pachube API Keys window
Pachube API Keys window

3. Register a feed. There is a link for this near the top right of most Pachube pages when you are logged in. For a Tweet-A-Watt feed, you should set the Feed Type to “Manual”. Most of the other fields are optional and up to you: I filled in a title, picked my location using the map, and left the other fields alone. There is an “Add a datastream” button near the bottom of the page. You can use it to manually create a datastream for each of your Tweet-A-Watts. However, since new streams from the wattcher.py script are automatically added here, I think it’s easiest to add your tags later, after the data shows up.

When you’re finished, Pachube will display a page listing various URLs for your feed. For the Tweet-A-Watt Python script you won’t actually use any of these – all you need to know is the unique numeric ID of your feed. For example, if your feed URL is http://api.pachube.com/v2/feeds/12345.xml, then your numeric ID is 12345.

Pachube Feed ID. That's my blur - go get your own.
Pachube Feed ID. That's my blur - go get your own.

4. That’s it for the Pachube end. Now for the Tweet-A-Watt end.

The least intuitive part of the process is getting the data you send into the format that Pachube prefers, called EEML. To do this, you’ll need to install a small piece of software, python-eeml on the computer running the Python wattcher.py script.

Go to the python-eeml page and click the Downloads button. Unzip (or “tar –xzvf”) the download file. Alternatively, if you have the git client installed (which you can do on Ubuntu by installing the git-core package), you can download the python-eeml by running the command “git clone http://github.com/petervizi/python-eeml.git”. Either way, you then open a command prompt, change to the “python-eeml” directory and run “python setup.py install”.

5. As a test, save the following Python code into a file (e.g. eemltest.py), fill in your API Key and Feed ID where indicated, and try running it (e.g. python eemltest.py).

import eeml

# parameters
# the key is a string, so include the quotes
API_KEY = '[the API key from step 2 above]'
# this isn’t a real URL, just a number, so don’t put quotes around it
API_URL = [the numeric ID of your feed from step 3 above]

pac = eeml.Pachube(API_URL, API_KEY)
# 99 is an ID for your senso
# 123.45 is the value in watts
# tags can be anything you like such as a text description of your sensor location
pac.update([eeml.Data(99, 123.45, tags=['Test1', ‘Tweet-A-Watt’], unit=eeml.Watt())])
pac.put()

Run the script. If you get an error, make sure you’ve got the data types right: API_KEY Is a string, API_URL is a 5-digit number, and the sensor ID (99) and watt value (123.45) are also just numbers without quotes. The “eeml.Watt” thing isn’t freeform text – it needs to be entered exactly as above, with no quotes.

If you experimented with API Key settings in step #2, an error might indicate that the settings you tried aren’t working – generate an API Key with the “All” setting to see if that’s the problem.

If the script runs without errors, then switch back to the Pachube web page, go to the “My Feeds” page, and click on the “Edit Feed” link. You should see a “datastream” 99 at the bottom of the page. Note that the tags you specified in the python code are displayed here – this is a good way to give a text description for your sensor ID.

6. Modify the Tweet-A-Watt wattcher.py script:

Add the import eeml statement to the top of the file:

import serial, time, datetime, sys
from xbee import xbee
import eeml  # add this!!

Add constants with your API key and URL number anywhere in the script – I added mine after the NUMWATTDATASAMPLE line:

CURRENTNORM = 15.5  # conversion to amperes from ADC
NUMWATTDATASAMPLES = 1800 # how many samples to watch in the plot window, 1 hr @ 2s samples

# Add this!!  Pachube settings:
API_KEY = '[the API key from step 2 above]'
API_URL =  [the numeric ID of your feed from step 3 above]

Then add the code to pass your Tweet-A-Watt data to Pachube. A good place to put this code is in the area where it is currently writing to the log file, which by default happens every 5 minutes.

Search for the following code in wattcher.py, which is in the update_graph method:

# Lets log it! Seek to the end of our log file
if logfile:
    logfile.seek(0, 2) # 2 == SEEK_END. ie, go to the end of the file
    logfile.write(time.strftime("%Y %m %d, %H:%M")+", "+
                  str(sensorhistory.sensornum)+", "+
                  str(sensorhistory.avgwattover5min())+"n")
    logfile.flush()

# Or, send it to the app engine
if not LOGFILENAME:
    appengineauth.sendreport(xb.address_16, avgwattsused)

The variables sensorhistory.sensornum and avgwattsused are what we want to send to Pachube. Add the following new lines underneath. (If you’ve never written Python code, the whitespace at the start of the line has to be exactly right – the “try:” line should start at the same column as the “if not LOGFILENAME” line above. The alignment of the other lines, relative to it, should be exactly as shown.

try:
    pac.update([eeml.Data(int(PACHUBE_ID), avgwattsused, unit=eeml.Watt())])
    pac.put()
except:
    print 'Error sending to Pachube'

The try/pass is included in case something temporarily goes wrong (like your Internet connection drops) – the try/pass ensures that wattcher.py will keep truckin’ rather than displaying an Exception message and terminating.

Run the wattcher.py script, give it 5 minutes to do its thing, then, go back to your feed page on Pachube. If everything is working, your data should show up as datastreams. You can fill in the descriptive fields for your datastreams as you like: whatever you type in won’t be overwritten by the incoming data.

Pachube Datastreams list
Pachube Datastreams list

Once things are up and running, you can consider logging data to Pachube more frequently (the limit is 10 updates per minute). To do this, locate the code in the main loop (where it prints the data to the screen every time it gets data from an Xbee). However, you’ll have to use a counter or timer to ensure you don’t write to Pachube more than 10 times per minute. If you put the code there, you also have the option of logging other data to Pachube, like voltage or amperage or the RSSI signal strength of the XBee.

Output Feeds

By default, the feed’s page shows the most recent value and a tiny graph of the last 24 hours’ readings for each datastream. (Actually, they won’t resemble graphs until you’ve logged about half an hour of data). These graphs are quite useful for spotting any unexpected energy usage, but you probably don’t want to leave them tucked away in your feed page on Pachube.

Pachube Google Gadget.  The dotted line indicates downtime (at my end in this case).
Pachube Google Gadget. The dotted line indicates downtime (at my end in this case).

One quick and easy way to make these graphs more accessible is as a Google Gadget, which you can get from the Gadget gallery, or create and customize your own version using this page on Pachube. I have the gadget embedded in my iGoogle home page, where the data can be conveniently monitored.

There are variety of other graphs and notifications that can be configured for your data. You can read about them on Pachube’s Use A Feed page. Some of them can be used just by filling in a form, such as the SMS notifications, but most of them require some “assembly”.

As a test, or a 1-time view, you can manually enter a URL in a browser like the examples below. However, these URLs are intended to be embedded in a web page on your own site or blog, or used by software to retrieve data. For programmers there are a lot of possibilities available through Pachube’s extensive API.

Most of the request formats include 2 parameters: interval (how far apart the readings are) and duration (how many readings you want to display). If you are posting Tweet-A-Watt data every 5 minutes, like my sample code, then the shortest interval is 5 minutes. With this interval, the longest duration you can request is 5 days. Pachube restricts the amount of data you can request at once, so going beyond this will result in a somewhat confusing “Forbidden” web page.

For example, to see the raw data with timestamps over the last 24 hours for datastream 1 (replace “12345″ with your 5 digit feed ID):

http://api.pachube.com/v2/feeds/12345/datastreams/1?duration=24hour&interval=300

This will return data in JSON format, suitable for processing by other software. Note that the times are all UTC (aka Greenwich Mean Time) – that’s the default format, which you can change on your Pachube settings page:


{"at":"2011-02-09T23:10:14.035059Z","tags":["DuinoFarm","Tweet-A-Watt"],"current_value":"72.2982427511","max_value":"164.251237339","unit":{"type":"derivedSI","label":"Watt","symbol":"W"},"min_value":"12.2650462797","id":"1","datapoints":[{"value":"32.5504473426","at":"2011-02-09T22:15:13.503066Z"}

To return data for all datastreams for the last hour in CSV format

http://api.pachube.com/v2/feeds/12345.csv?duration=1hour&interval=0

which returns (datastream #, timestamp, value):


0,2011-02-09T22:25:03.179157Z,58.6875577135
1,2011-02-09T22:25:13.480201Z,77.7378909685
0,2011-02-09T22:30:05.235568Z,55.9543778625
1,2011-02-09T22:30:12.307615Z,75.1620522158
0,2011-02-09T22:35:02.070760Z,58.0097389617
1,2011-02-09T22:35:13.445684Z,76.802283106

As mentioned in the introduction, an alternative to Pachube is Nimbits. (The name is apparently unrelated to the Canadian delicacy Timbits, which, contrary to what my brother claimed when we are kids, are apparently not manufactured from bits of the desiccated corpse of restauranteur/hockey player Tim Horton. But I digress.)

Nimbits uses a similar interface for posting data, and has similar functionality for accessing your data, but with some extra prepackaged options, like sending updates through Google Chat or Twitter. (Hey, posting your Tweet-A-Watt to Twitter — there’s an idea!). I’ll cover Nimbits in my next blog post.

Posted in Electronics, Programming | Tagged , , , | 3 Comments

Netduino: Ordering Off the Menu

The source code for this article can be downloaded here: LightControllerWithMenu.zip.

This is the 5th in a series of projects for the Netduino and Seeed Studio’s Grove Starter Bundle.

The first four articles in the series were:

  1. Netduino Enters the Grove
  2. Netduino: Time and Weather
  3. Netduino: Let There Be Light
  4. Netduino: Here Comes The Night

Today, my friends, the voice of the people has been heard, and the reign of an oppressive dictator shall finally end.

For too long (about a month-and-a-half, to be specific, since my Netduino: Time and Weather post), we’ve had only one way to enter the date and time and other settings: the user-hostile serial connection. It forced us to drag our notebooks over to the Netduino, open serial terminals, enter obscure commands, and other things too loathsome to mention. Is this any way to live?

The people want freedom. The freedom to just walk up to a Netduino and set the date and time at the click of a button. The freedom to choose Celcius or Fahrenheit, at the click of a button.  (Though only imperialists use Fahrenheit, you know.) The freedom to turn off the LCD backlight – well, you get the idea.

In short, the people want a menu system.

This new menu system will follow the orders of one person only: you the user. Well, technically, also me, the programmer. And if you choose to fix my bugs customize my code, then also you (again), the programmer. Oh, and the reviled serial connection is going to remain in, um, a strictly unofficial capacity to ensure an orderly transition.

Pin Assignments

The pin assignments are unchanged from the previous blog post, Netduino: Here Comes The Night. The changes in this installment of the project are purely software.

User Interface

To invoke the snazzy new menu system, take the Red pill — I mean, press the Red button. (For those who aren’t using the Seeed Grove hardware system, the red button is connected to D7, and the green to D8. In Grove terminology, this is a button “twig” connected to the D7/D8 plug on the Grove stem).

Throughout the menu system, the buttons are used as follows (aaah – a user manual!):

  • Red button click – scroll down
  • Green button click – scroll up
  • Red button double-click – select a menu item
  • Green button double-click – exit the current menu

Some of the menus allow you to change a numeric value, like the hour of the day. In these menus:

  • Red button click – increase the value by 1
  • Green button click- decrease the value by 1
  • Red or Green button press and hold – repeatedly increase or decrease the value of the number
  • Red button double-click – save the current value of the number and exit the current menu
  • Green button double-click – exit the current menu without saving

Programmer Show And Tell

The heart of this update to the project, and the part you are most likely to want to change, is the menu configuration. I didn’t want to reinvent the wheel, so I carefully looked through the work that had already been done on creating a menu system for .Net Micro Framework devices.  When I couldn’t find what I wanted on the first page of the Google results, I got annoyed and reinvented the wheel.

Near the top of Program.cs, you’ll find the following array definition:

static string[] strMenuItems = new string[] {
"1", "Date `yy`-`MM`-`dd`", "1-1",
"1", "Time `HH`:`mm`", "2-1",
"1", "Timers", "3-1",
"1", "Settings", "4-1",
"1-1", "Year `yy`", "_UD-`yy`",
"1-1", "Month `MM`", "_UD-`MM`",
"1-1", "Day `dd`", "_UD-`dd`",
"2-1", "Hour `HH`", "_UD-`HH`",
"2-1", "Min `mm`", "_UD-`mm`",
"3-1", "Light On `HH:mm+`", "3-2",
"3-1", "Light Off `HH:mm-`", "3-3",
"3-2", "Manual `HH:mm+`", "3-2-1",
"3-2", "Sunset `HH:mm_`", "_SUNSET",
"3-3", "Manual `HH:mm-`", "3-3-1",
"3-3", "Sunrise `HH:mm*`", "_SUNRISE",
"3-2-1", "Hour `HH+`", "_UD-`HH+`",
"3-2-1", "Min `mm+`", "_UD-`mm+`",
"3-3-1", "Hour `HH-`", "_UD-`HH-`",
"3-3-1", "Min `mm-`", "_UD-`mm-`",
"4-1", "Temp Fmt `TD`", "4-1-1",
"4-1", "Backlight `LB`", "_BACKLITE",
"4-1", "Brightness `BR`", "_UD-`BR`",
"4-1-1", "Celcius", "_CELCIUS",
"4-1-1", "Fahrenheit", "_FAHR"
};

One thing you’ll notice right away is that this should be a 2 dimensional array (i.e. string[,]), but isn’t. Oddly, there is no such thing as a 2-dimensional array in the .Net Micro Framework. Instead, you can have arrays of arrays (e.g. string[][] strMenuItems = new string[20][], where the 2nd-level array can be of a different size in each element of the main array. This weirded me out, so I went with a 1-dimensional array instead.

Each menu item consists of 3 strings:
- A menu ID – this must be the same for each item in that level of the menu
- A caption, which can contain embedded keywords delimited with the backquote character (`)
- The result of selecting (double-clicking) that menu item, which can be the ID of a submenu to display, or an action (actions are prefixed with an underscore).

So, for example, the first level menu has 4 items that appear on the LCD like this:

Date 11-02-14
Time 18:05
Timers
Tools

The first 2 items contain keywords that are replaced at run-time with the current date and time.
Selecting the first item in this menu displays sub-menu “1-1”, which looks like

Year 11
Month 02
Day 14

Selecting any of these menus selects an action, such as “_UD-`yy`” — this displays an “Up/Down” menu for setting the year. More on that below.

I’ve added a new class to the project, clsLCDMenu, and given it sole responsibility for displaying the menu and handling all button input. This class is intended to be a generalized menu handler: it knows  how to display lines of a menu and allow the user to navigate through the menu using the buttons, but it knows nothing about how to handle the actions selected by the user.

This design results in a fairly extendable and reusable menu. There are no hard-coded limits on the # of elements in a menu of the # of levels of submenus. To add new menu items, or create an entirely different menu system, I won’t have to change any of the code in clsLCDMenu. Instead, I just change the strMenuItems array, above, and add some “action handler” logic to some callback methods in Program.cs, described below.

Since clsLCDMenu “owns” the LCD and buttons when the menu is up, I have to make sure that the existing code never tries to write to the LCD or respond to a button click when the menu is up. I do this by turning off the timers and “unhooking” the button event handlers, as follows:

static void launchMenu()
{
  // turn off the timers that update the display
  timerTemperature.Dispose();
 timerTime.Dispose();

  //unhook the button event handlers
  buttonRed.OnInterrupt -= buttonRed_OnInterrupt;
  buttonGreen.OnInterrupt -= buttonGreen_OnInterrupt;

  // buttonGreen = up, buttonRed = down
  menu = new clsLCDMenu(strMenuItems, lcd, buttonGreen, buttonRed, getStringForMenu, getValuesForMenu,     setValueFromMenu);
}

The “-=” statement that unhooks the event handlers will look freaky to anyone who hasn’t encountered it in .Net before, since neither side of the “-=” is a numeric value. This syntax removes the main class’s button event handlers from a virtual “list” of handlers for the event. It isn’t all that commonly used in .Net, since event handlers are automatically “unhooked” when the class that contains the event handlers is destroyed, such as when a form is closed.

The constructor for the clsLCDMenu class looks like this:

public clsLCDMenu(string[] strItems, ILCD lcd, InterruptPort buttonUp, InterruptPort buttonDown,
MenuCallback getCaption, MenuGetValue getValue, MenuSetValue setValue)

The caller passes a reference to the LCD and “up” (green) and “down” (red) buttons, along with 3 callback methods. The callback methods add the “business logic” to the menu system, if you will: they fill in the keywords in the menu captions, and handle the menu actions. Specifically, there are 3 types of callbacks:

  • MenuCallback – gets passed a menu caption containing keywords (e.g. “Light On `HH:mm+`”), and passes back the caption to be displayed to the user (e.g. Light On 20:30”.
  • MenuGetValue – gets passed an “up/down” menu definition (e.g. “_UD-`MM`”) and passes back the current value, minimum value and maximum value to be used for the up/down menu.
  • MenuSetValue – gets passed an action that needs to be taken: either the changed value from an up/down menu (e.g. the user has changed the hour setting on the clock), or some other action such as “_SUNSET” (the user has set the “Light On” timer to sunset.)

Callback methods in .Net require 2 things:
1) A “delegate” definition. For example, the definitions of the 3 callback methods used by clsLCDMenu are:

public delegate string MenuCallback(string s);
public delegate bool MenuGetValue(string menuType, out Int16 start, out Int16 min, out Int16 max, out byte increment);
public delegate void MenuSetValue(string menuType, Int16 value);

2) The actual method to be executed when the callback is invoked. References to these 3 methods are passed to the clsLCDMenu constructor by the launchMenu() method, shown earlier. The constructor saves these references in variables of the “delegate” type. For example, the MenuCallback is saved by clsLCDMenu as:

private MenuCallback menuGetCaption;

…

public clsLCDMenu(string[] strItems, ILCD lcd, InterruptPort buttonUp, InterruptPort buttonDown,
MenuCallback getCaption, MenuGetValue getValue, MenuSetValue setValue)

…

this.menuGetCaption = getCaption;

When clsLCDMenu needs to call one of the callback methods, it uses its reference variable, calling it as if it was a method. For example:

string strMenuText = menuGetCaption(((menuItem)lstMenu[intMenu]).StrCaption);

I won’t give an explanation of what the callback methods do. They are actually quite straightforward: just a bunch of “if” and “switch” statements that examine the parms to figure out what kind of data they are dealing with, then returning the corresponding settings or taking the corresponding action.

Two other features that are worth explaining, though, are the buttons’ double-click and auto-repeat handlers. These both rely on timers, which are a pleasure to work with in .Net compared to the alternatives that you need to hack together in other microcontroller platforms.

The double-click is a click followed by another click (duh). As every new (or elderly) PC user learns the hard way, the difference between a double-click and 2 single clicks is the amount of time in between.

So, the code which handles the first click sets a timer. If the timer goes off before another click occurs, then that was a single click, and the code handles it accordingly. If a second click occurs before the timer expires, that’s a double click.

The code which sets the double-click timer is:

timerButtonDown = new Timer(new TimerCallback(buttonClick), "Down", DOUBLECLICK_DELAY, -1);

The DOUBLECLICK_DELAY is a constant, in milliseconds, which determines how fast Granny has to move to pull off a double-click. I ended up setting it to 400 ms, which was short enough that I didn’t accidentally double-click when I wanted to make 2 separate clicks.

The “buttonClick” timer event handler now contains the logic that was previously in the button interrupt event: a click isn’t a click until the timer goes off:

private void buttonClick(object data)
{
  if ((string)data == "Down")
  {
    // cancel the button click timer
    if (timerButtonDown == null)
    {
      // oops, someone already cancelled it, so we should ignore it
      return;
    }
    timerButtonDown.Dispose();
    timerButtonDown = null;
    blnButtonDownClicked = false; // no longer waiting for double-click
    goDown();

You’ll notice some seemingly unnecessary null handling.  Here’s the thing…

If the user clicks the button a second time, then we want to cancel this timer event: it’s a double-click action, not a single-click. In theory, disposing a timer should cancel the timer event. However, when testing that code I found that double-clicking would often result in both the double-click and single-click action being performed, since the timer event wasn’t cancelled by disposing the timer object. Setting the timer to null didn’t cancel the event either.

So, I ended up adding code to the timer event to see if the timer had been set to null, and aborting the event if it had. Crude, but effective, and a lot more reliable than polling the button a la Arduino.

To implement button auto-repeat, I used the button “push-and-hold” timer (as described in the Netduino: Let There Be Light article), and had it re-invoke the timer if the button was still being held down:

private void buttonHold(object data)
{
  if ((string)data == "Down")
  {
  …
  if (!buttonDown.Read())
  {
    // not being held down anymore
    return;
  }
  intDownRepeatCount += 1;
  int delay = BUTTONREPEAT_DELAY;
  if (intDownRepeatCount >= 4)
    delay = TURBOREPEAT_DELAY;
    timerButtonDown = new Timer(new TimerCallback(buttonHold), "Down", delay, -1);

As you can see, I implemented auto-repeat the same way as most alarm clocks: the number advances relatively slowly at first (BUTTONREPEAT_DELAY is 500 ms), then more quickly after a couple of seconds (TURBOREPEAT_DELAY) is 200 ms.

Next Steps

At this point in the project, we have a reasonably functional plant light controller (and clock and temperature display).

Although using the new menu to set the date and time is definitely a lot more user friendly than the deposed serial port system (boo, hiss), it’s still a pain to have to re-enter each menu setting after a power outage. So, a battery backup and/or saving the menu settings to flash memory would be a nice feature, and I’ll definitely be adding that at some point.

However, the more exciting enhancement is to give the Netduino something to display on its LCD other than the time and temperature. There is a world of data swirling around the Netduino: twitterers tweeting, bloggers blogging, weather forecasters making stuff up forecasting. Wouldn’t it be cool if our Netduinos could dip into that datastream and display some of that info on its LCD?

There are 2 main options for implementing this: using a Netduino-compatible Ethernet shield to connect directly to the Internet, or using a wireless XBee device to connect to a PC. The first option has the very strong advantage of being a standalone solution, but I’ll be going down the second path – it has the even stronger advantage of me having already written most of the code.

So, if you’re interested in following me there, you’ll want to invest in a couple of XBees, and a couple of hardware interfaces for connecting the XBees to your Netduino and your PC.  More on that in my next article.

The source code for this article can be downloaded here: LightControllerWithMenu.zip. As before, it contains a reference to the Micro Liquid Crystal library, so you should download it as well, and place it in a folder that is side-by-side with my LightControllerWithMenu folder (e.g. c:projectsLightControllerWithMenu and c:projectsMicroLiquidCrystal).

Posted in .Net, Electronics, Programming | Tagged , , | 6 Comments

Netduino: Here Comes The Night

This is the 4th in a series of projects for the Netduino and Seeed Studio’s Grove Starter Bundle.

The first three articles in the series were:

  1. Netduino Enters the Grove
  2. Netduino: Time and Weather
  3. Netduino: Let There Be Light

As I hinted at the end of the previous article, there is an easy way to automatically set the plant light to turn on when it starts to get dark. It’s a light-sensing photoresistor (like this), and we’re not going to use it because

  1. There isn’t one in the Grove Starter Pack
  2. I kinda thought the easiest way would be to calculate the sunset time instead

Using an algorithm to calculate sunset time is actually relatively foolproof, being unaffected by storm clouds, cat attacks, or awestruck bystanders staring at your way-cool plant light controller.

It does however, require 2 additional pieces of information:
- your geographic location
- your timezone, including daylight savings time

The geographic location is, specifically, the longitude and latitude. There are zillions of sources for that on the Internet. One of the handiest is Wolfram Alpha, the Google of calculations. Just type in ” latitude longitude [your city]“, and it will spit out the numbers. Type in “[your city]sunset”, and it will tell you what result the algorithm should be giving you for today.

We all know what timezone we are in, but calculating the start and end date for Daylight Savings Time is too convoluted for most programmers, including me. It becomes pretty easy to determine if your Netduino is Internet connected, but since we aren’t there yet I’ll follow the same approach as the prestigious National Research Council Canada’s Sunrise/Sunset calculator: leave it unimplemented.  (As they put it: “Add one hour to listed times when Daylight Saving Time is in effect.”)

I decided to use the sunset calculation algorithm from this CodeProject article: Solar Calculator.  This API is relatively simple and self-contained. The “LongituteTimeZone” parameter mentioned in the article threw me for a loop – I fruitlessly Googled “Longitute” before realizing that was a typo. This parameter represents your time zone: it is the offset from UTC (aka Greenwich Mean Time) in hours, times 15.    (Turns out each timezone represents 15 degrees of longitude – who knew!)

The algorithm makes extensive use of the trigometry calculations from the .Net Math class, none of which exist in the .Net Micro Framework. Fortunately, all of them do exist, with the exact same parameters, in Elze Kool’s exMath library. This the same library that I used to calculate the temperature in the first article of this series.

So, to change my code to work correctly for your location, you’ll need to change the following 4 values, which are near the beginning of Program.cs in the source code:


// settings for Toronto Canada
const double MY_LONGITUDE = -79.38;
const double MY_LATITUDE = 43.65;
const int UTC_OFFSET = -5;
const bool USE_DAYLIGHT_SAVINGS = false;

Note that the longitude and latitude are in decimal format. If you’ve forgotten your high school geography, negative values are for south of the equator and west of the Prime Meridian, and to convert “minutes” of longitude and latitude to decimals, divide them by 60.

Pin Assignments

If you are using a Parallel LCD, then nothing changes from the list of pin assignments in the Let There Be Light article.

If you are using a Serial LCD, then you probably already changed my pin assignments, since they weren’t at all suitable for a Serial LCD. (As before, you should comment out one of the #define statements at the top of Program.cs to select your LCD type). I changed the code to use:

  • UART Serial connection – this changes to COM1 (pins 0 and 1)
  • LCD Serial connection – this uses COM2
  • Relay twig – this moves from D0 to D13

User Interface

The sunrise calculation is done after the Netduino knows the current date and time. Both of these are still entered using Terminal commands sent over the UART serial connection, as described in the Time and Weather article.

By default, the Light On time is set to 15 minutes before sunset, and the Light Off time is set to 11 pm. You can, of course, override either these using the “+” and “-” Terminal commands described in the Let There Be Light article.

If you have an 16×2 LCD (or larger), it will display the Sunset time on the 2nd row, next to the temperature. This has little practical value other than to confirm that the sunset times are being calculated correctly (and impress friends who are easily impressed).  You can comment it out if you like in the displayTemperature() method.

Speaking of the accuracy of the sunset calculation: it isn’t all that accurate for me (in Toronto Canada). Depending on the day of the year, it is 5-10 minutes earlier than what Wolfram Alpha and other web sites calculate. I get the same result when I run the CodeProject code on a Windows PC, so whatever is responsible isn’t MathEx or the .Net Micro Framework. For the purposes of a plant light, 5-10 minutes is close enough.

Programmer Show and Tell

The most interesting code change this time is something that didn’t work as expected. As described in the Let There Be Light article, I’m using the exotic and mysterious ExtendedTimer class to raise events for turning the light on and off. This class has a Period parm, which is how often you want the timer event to repeat. Since we are now recalculating the “light on” time on a daily basis, I wanted to turn off the period. The MSDN docs for the ExtendedTimer constructor say you do this with Timeout.Infinite

tmrLightOn = new ExtendedTimer(new TimerCallback(SetLightState),
  true, datAlarm,
  new TimeSpan(Timeout.Infinite));

Unfortunately, this doesn’t work, not on a Netduino anyway. It throws an “Argument out of range” exception. The best workaround I could find was to set it to the maximum TimeSpan value.

TimeSpan tsRepeat = TimeSpan.MaxValue;

if (!blnLightOnAtSunset)
{
   // repeat timer at same time every day
   tsRepeat = new TimeSpan(1, 0, 0, 0);
}

tmrLightOn = new ExtendedTimer(
  new TimerCallback(SetLightState),
  true, datAlarm, tsRepeat);

The other piece of code that I’ll point out is the use of a custom event to notify the main class that the user has entered a date and time. The clsUART class has a UART_DataReceived event that fires every time something is entered by the user in a serial terminal. It handles things that it knows how to do (like set the system date and time), then fires off an event to notify the rest of the application that the command was received. This allows you to cleanly separate the serial port logic in clsUART from the “plant light” logic in the main application class.

For example, the code in clsUART that handles the Date command includes the following:

DateTime currentDayTime = new DateTime(intYear,
  Int16.Parse(strDateParts[1]),
  Int16.Parse(strDateParts[2]),
  DateTime.Now.Hour,
  DateTime.Now.Minute, DateTime.Now.Second);

Utility.SetLocalTime(currentDayTime);

// tell rest of application that the date has been set
raiseCommand(strCommand);

if (blnSendResponse)
{
  WriteToUART("OK");
  return true;
}

The raiseCommand method looks like this:

public event CommandEventHandler Command;

private bool raiseCommand(string strCommand)
{
   CommandArgs args = new CommandArgs(strCommand);
   // raise event using Command delegate
   Command(this, args);
   return args.blnHandled;
}

It uses a parm passed to the CommandEventHandlers to determine if some other part of the program recognized the command and handled it. The return code is ignored in the above “date command” logic (since clsUART already handled it), but it is important for commands that clsUART doesn’t know how to handle, like setting turning the plant light on or off. If it gets a false parm back, it displays an error message to the user. (Well, it actually displays a “?” and buzzes a little, but you get the idea.)

Currently, the only CommandEventHandler is in Program.cs. It is created during startup by the following:

uart = new clsUART(strUARTPort, 9600, 50, true);
uart.Command += new CommandEventHandler(uart_Command);

The event handler includes this:

static void uart_Command(object sender, CommandArgs e)
{
   if (command == '-')
   {
      e.blnHandled = setLightTime(e.StrCommand.Substring(2), false);
   }
   else if (command == 'D')
   {
      blnDateSet = true;
       // set default light timer if date and time have been set for the first time
      setDefaultTimers();
  }

So, the code uses the e parm to return a confirmation that it took care of the command (if it did).

That’s it for this installment – a simple change that could have been simpler.  (Clearly George Vernon Hudson wasn’t a programmer.)

Next up, I’ll be adding a menu system for things like setting the date and time.  I notice that the current version (1.0b) of the Grove Starter Kit now includes a 16×2 LCD, rather than the tiny 8×2 model that the beta Starter Kit shipped with, so I’ll use that as the minimum screen size for the menu.

The source code for this article can be downloaded here: LightControllerV2.zip. As before, it contains a reference to the Micro Liquid Crystal library, so you should download it as well, and place it in a folder that is side-by-side with my LightController folder (e.g. c:projectsLightControllerV2 and c:projectsMicroLiquidCrystal).

Posted in .Net, Electronics, Programming | Tagged , , | Leave a comment

Netduino: Let There Be Light

This is the 3rd in a series of projects for the Netduino and Seeed Studio’s Grove Starter Bundle.

This installment is intended to create a plant light that automatically switches on and off at a pre-set time of the day (well, more usefully, the night).  It is inspired by the “Garduino” project, which was published in Make Magazine and can also be found on Instructables.

This project builds on the foundation set by the earlier two articles. If you want to actually get the project working yourself, you’ll probably need to browse the earlier articles, especially the part about the serial connection to the PC.

The first two articles  in the series were:

Netduino Enters the Grove

and

Netduino: Time and Weather

As the project develops it will soon be necessary to move beyond the Grove’s components (that 8×2 display, in particular, is cramping my style).  First, though, we’re going to bring another Grove component into the act: the Relay twig.

The Relay twig is used to switch the plant light off and on.   To connect the relay to the plant light, we first need to splice open an AC extension cord.

Now, I know that Edison fellow has been spreading the word about AC power being dangerous and what-not.  Sounds scary, don’t it?  Well, it’s true!  Ask Topsy the Elephant.

Seriously, though, various unpleasant things can happen if you hook the relay up incorrectly, but fortunately the risk is mainly to the relay, and your self-esteem.  However, since you don’t want your relay Twig to end up like poor Topsy,  I’ll spend much of this article covering the process of  making that connection.

You actually have a couple of options: build or buy.

Option 1 – Modify an Extension Cord

Here’s the plan: The Relay twig acts as a on/off switch for the extension cord.  The twig has two little green screw connectors.  The “live” wire of the extension cord needs to be cut open, and the 2 exposed ends need to be inserted into those 2 green connectors.  Sounds simple, and it is.

The following photo shows a modified 2-prong extension.  It also shows the connection to the Relay twig, but don’t worry about that for now – it is explained in the “Connecting the Relay Twig” section, below.

Two-pronged extension cord connected to Grove Relay
Two-pronged extension cord connected to Grove Relay

The process of modifying the extension cord isn’t complicated or dangerous. Anything you do wrong will be evident before you get to the stage where you’ve got the extension cord plugged into the wall. However, it does requires some soldering and some common sense.  If you’re not confident about your abilities at either of these, consider Option 2, below.

The extension cord you use doesn’t need to be anything special – the cheapest one at Wal-Mart is good enough.  If your plant light has a three prong plug then, of course, you need to buy a three prong extension cord.

The best guide to what’s required (and the one I originally followed) is the one in the Make magazine Garduino article. The Instructables version of Garduino refers you to 2 other sites.  Personally, I found the Hobby Robotics article to be confusing (so many wires!) and the Sparkfun article to be downright intimidating (you don’t have to pull a power receptacle out of the wall for this project).  As far as I know the Make Magazine article isn’t available to non-subscribers, but it just so happens that someone (not me) has posted a PDF of this article here.  To modify the extension cord for use with Relay twig, follows steps 7a-7c.

The Make magazine article shows how to modify a two-prong extension cord.  If your plant light requires a 3-prong cord, then the only difference is the process of selecting the right wire to cut and splice.  

It must be the “live” one, which for both 2- and 3-prong cords is the one connected to the narrower blade of the plug.  Generally, the corresponding wire in the cord is aligned with the blade of the plug (i.e. if the blade is on the left, you need to cut the cord on the left).   However, to make sure that you’ve selected the right one before cutting it, nick the cord to expose the copper underneath, then use a multimeter to check connectivity with the narrower blade of the plug.  (If you picked the wrong cord, it’s safe to cover the nick with electrical tape, as I did with the 3-prong extension cord in the photo below.)

Three-pronged extension cord connected to Seeed Relay Brick
Three-pronged extension cord connected to Seeed Relay Brick

Option 2 – Buy a Netduino-Friendly Extension Cord

A product called the “PowerSwitch Tail” recently became available.  It is specifically intended to be hooked up to microcontroller circuits, such as the Arduino and Netduino.  It contains a 5V relay similar to the one in the Grove starter kit – if you go this route, you can ignore the Grove relay.  You’ll have to connect a wire from the Powerswitch to the D0 pin of the Netduino (or modify my code to use whichever pin you’d like).  Then, just skip ahead to the “First Test” section below.  You can read more about the PowerSwitch on the Adafruit product page and in Adafruit’s forums.

I haven’t tried a PowerSwitch, but I’m 99.9% sure it will work fine with the Netduino and this project.

Connecting the Relay Twig

You should have two wires with exposed tips coming from your extension cord.  As you probably figured out, you need to connect each wire into one of the screw connectors on the Relay twig.  Ah, but which goes into which?

Unfortunately, Seeed didn’t put any markings on the v0.9 Relay twig to help you out.  The Datasheet and tutorial on Seeed’s web site don’t offer any guidance on this either.  (“If in doubt contact a professional such as a licensed electrician for help“.  Uh, yeah, good luck with that.)

However, the HLS8L-DC5V relay happens to be the exact same model as used in Seeed’s Electronic Brick relay, and the Brick relay is well documented in this Instructables article and in this Seeed Forum post.

The Grove relay has removed one of the Brick’s 3 screw connectors, meaning that the only way you can hook it up is “normally open”.  This means that the extension cord won’t (yeah, “open” means “off” – go figure) conduct electricity when your Netduino project is powered off.

The 2 screw connectors are, then, are the “Common” and “NO” connectors shown in the Instructables article.  As with its big brother, the Seeed Relay Brick, the twig’s Common connector is on the left (when the screw connectors are facing you) and Normally Open is on the right.  See the photos in this post.

Important Note: this is true for the “0.9″ version of the twig, which is the first version that was sold.  If your twig doesn’t say 0.9 on it, then I’d strongly suggest you double-check this information, either by using a multimeter or by checking the Seeed product page or forum.  Seeed recently posted an article about version 1.0 of the Grove Starter Bundle.  The Relay twig photo on that page looks more like a Relay forest, so if your twig is bristling with relays then all bets are off!

To check continuity with a multimeter, you want to confirm that the “Common” left-hand screw connector is connected to the middle pin at one end of the black relay box, and the other screw connector is connected to one of the 2 pins at the other end of the black relay box. I think the best illustration of the orientation of the pins for this relay is the one in the Seeed forum thread.

Grove Relay Brick - hooked up and ready to rock
Grove Relay Brick - hooked up and ready to rock

OK, so at this point you should have an UNPLUGGED extension cord that is connected to an unplugged relay twig.   This is the time to ensure that you’ve firmly and securely connected the extension cord to the relay.  Tug  hard on the relay twig to make sure that it doesn’t come loose from the extension cord.  

Carefully inspect the points at which the wires from the extension cord are plugged into the screw connectors of the Grove twig.  There must not be any exposed metal from the wire.  If there is, insert the wire further into the screw connector or, if the exposed metal is simply too long, snip it shorter.

That cord-to-relay connection is the only dangerous (to you) part of the setup, and you definitely don’t want to have a wire come loose after you plug it in!

Incidentally, if you Googled for information about the HLS8L-DC5V relay, you may have been unsettled to find that there isn’t much out there, aside from a bunch of pretend-datasheet-sites gaming Googling search engine.  However, I’ve had 2 of the Seeed bricks with this relay in daily service for about 6 months now, and never had a problem with either of them.  So, for what its worth, I’m pretty comfortable with this relay, safety-wise.

Initial Test

I’ve written a small test project, RelayTester, that you can use to test turning the relay off and on.  Pressing the green button (connected to D8) toggles the relay.

Start by unplugging everything from the Grove shield.  Reconnect to the Grove shield just 2 twigs: the one with the red and green buttons (connected to plug D7/D8) and the Relay twig (connected to D0, and to an UNPLUGGED extension cord).  Yes, there, is such a thing as D0 on the Grove shield, off to one end, as shown in the photo in the “Final Configuration” section, below.

Deploy the test project to the Netduino.  For this part of the test, you can leave the Netduino plugged into the USB port of the PC.

The red light on the relay may briefly go on and then off when the Netduino boots up.  Press the green button on the button twig, and the red light should come on and stay on.  Press the green button again to turn it off.

Do you hear a fairly loud clicking noise from the black relay box each time the red light goes on or off?  If not, something is wrong.

If the red light on the relay works, but there is no click, try using a different cable between the Relay twig and the Netduino.  If that doesn’t help, try a different connector than D0 (you’ll have to change the test program code accordingly, of course).

If the red light doesn’t come on at all, check that the connectors are plugged into the Grove shield correctly (buttons in D7/D8, relay in D0, and nothing else connected), and that the project was deployed to the Netduino correctly.  (It will display messages to the Visual Studio Debug window when turning the relay on or off).

If that’s not the problem, then you might have a bad relay twig and should check the Seeed forums or e-mail them.

Live Test

I don’t think there is much risk to the Netduino if things aren’t hooked up correctly.  The Netduino isn’t exposed to the AC current – that’s the whole idea of the relay.  The biggest risk is to the relay itself, and possibly to whatever you have plugged into your extension cord.  However, as an extra precaution, I’d suggest you have as little as possible connected to the Netduino: the Grove shield, the button twig and the relay twig.

Also, I’d suggest you power the Netduino from a DC adapter rather than the PC’s USB port.  The concern isn’t that the relay is power-hungry, but that you want to protect your PC’s USB port if things go wrong.    (If you haven’t run the Netduino with that DC adapter before, rerun the above “Initial Test” with the extension cord still unplugged from the wall.)

OK, so it’s all been fun and games so far, but no fooling around for this test. Remember Topsy!

  1. Plug the plant light (or a cheap test light) into the extension cord.
  2. Plug the extension cord into the wall.
  3. Plug the DC adapter into the Netduino.
  4. Press the green button.  Let there be light!
  5. Leave the light turned on for awhile, periodically touching the extension cord, the wires leading from the extension cord to the relay, and black relay box to make sure it isn’t getting hot.  The cord and wires shouldn’t get warm at all; the relay box will be very slightly warm.

One last (I promise!) comment about safety: ideally you should never make any adjustment to the relay/extension cord/light setup with the extension cord plugged in.  When hooking things up, and when adjusting the setup, unplug the extension cord first.  Keep an eye on those 2 screw terminals on the Relay twig and the wires inserted into them — that’s the danger zone.  Don’t pile anything on top of them, don’t let those wires get snagged on something.

Final Configuration

OK, let’s bring the rest of the Grove twigs back on stage.  (Not you, Prototype Twig, we aren’t ready for you yet.  Oh, and Tilt Switch twig, um, you can run along home.  We’ll keep you in our files, and we’ll call if we need to do any tilting).

The configuration is the same as in the previous project, Time and Weather, with the addition of the Relay twig:

  • D0/D1 – Relay
  • D2 – jumper to TX pin of serial header on breadboard
  • D3 – jumper to RX pin of serial header on breadboard
  • D4/D5 – right side of LCD stem, lower connector
  • D6/D7 – piezo buffer twig (only D6 is used)
  • D7/D8 – button twig
  • D9/D10 – bottom of LCD stem, right connector
  • D11/D12 – bottom of LCD stem, left connector
  • D13 – right side of LCD stem, upper connector
  • A0 – modified potentiometer (unused for Grove LCD, or any other LCD without a software backlight control)
  • A2 – temperature twig
Final Configuration of the Netduino and Grove components.  Can't see the forest for the trees!
Final Configuration of the Netduino and Grove components. Can't see the forest for the trees!

By now, you’ve probably noticed that I have a strange aversion to pin D1.  What’s up with that?

Well, I have 2 Netduinos, one for development and one for testing.  My test Netduino suffered an unfortunate accident, resulting in it being, um, “differently abled” on the D1 pin.

Wait, what?   The guy who’s  giving you safety tips electrocuted his Netduino?

Well, not on this project – honest!  I’m actually not sure at what point it stopped working, but I suspect it occurred while experimenting with the not-particularly-dangerous-but-not-Netduino-friendly Danger Shield.  (See my “Spiderman: The Musical” project for details).

The source code for the LightController project can be downloaded here.

As with the test project, press (and, this time, hold) the green button to manually turn the relay (and light) on and off.  You can also program it to turn off and on at a certain time, using the handy serial port connection from the previous project.  (Stop groaning, we’ll get rid of the serial port in a later project, I promise.)

The 2 new commands are:

+:hh:mm

Sets the time at which the light will come on
-:hh:mm

Sets the time at which the light will go out.

There is no default on and off time – you have to set them manually for now.  (Since you have to set the Netduino’s clock manually, there wouldn’t be much point to having default timer settings).

I’ve also added some new button features, pretty much exhausting the possibilities for the 2-button twig:

  • Press the red button: Toggles LCD between the regular time/temperature display, and a display of the  Light On and Light Off times.   If all you see is “+” and “-”, with no times, then you need to set the times using the serial terminal “+” and “-” commands, as described above.
  • Press and hold red button: Toggles the serial port off and on, as described in the “Time and Weather” article.
  • Press the green button: toggles the first row of the LCD between the current time and a countdown to the next light “event”.  That is, if the light is currently on, it shows a countdown to when the light will go off.  And vice versa.
  • Press and hold green button: toggles the light off and on.
  • Press and hold red and green button: clears the light on and light off timer.  Also makes your fingers hurt.

Programmer Show And Tell

The new code has a few features worth mentioning.

The light on/off timers are implemented using the ExtendedTimer class.  This is the lesser known of the 2 .Net Framework timers.  It is better suited than the Timer class for time events that occur at specific times of day, rather than at intervals.

For example, the code to set the “light on” timer is:

// set the timer to go off at this date, and every 24 hours after that
tmrLightOn = new ExtendedTimer(new TimerCallback(SetLightState), true, datAlarm, new TimeSpan(1, 0, 0, 0));

Since we want the light to come on at the same time each day, the ExtendedTimer constructor does all the work. There is no need to reset the timer for the next day when the timer event fires.

The countdown timer takes advantage of the fact that, in .Net, subtracting one date from another returns a TimeSpan that gives the difference between them:

// relay is off, display time to next light-off
string strCountdown;
if (datNextOn == DateTime.MinValue)
{
    strCountdown = "Not set";
} else
{
   TimeSpan timeTo = datNextOn.Subtract(DateTime.Now);
   strCountdown = timeTo.ToString();
}
lcd.SelectLine(1, true);
lcd.WriteStringToLCD(strCountdown);

Note that DateTime (and TimeSpan) objects can’t be null – an unset DateTime has a value of DateTime.MinValue.

And lastly, this projects makes extensive use of the 2 pushbuttons. I really like the fact that all the button features can be implemented in the .Net MF without polling.  

Buttons can be configured as InterruptPorts.  The InterruptPort constructor has a parm that lets you specify when the event should be triggered — when you press the button, when you release the button, or both. The following code specifies both:

buttonRed = new InterruptPort(Pins.GPIO_PIN_D7, true,
   SecretLabs.NETMF.Hardware.Netduino.ResistorModes.Disabled,
   SecretLabs.NETMF.Hardware.Netduino.InterruptModes.InterruptEdgeBoth);

The event handler needs to handle 2 cases: a short push and a push-and-hold.  The handler can’t know which it is until the button is released.  On the other hand, the user would like to have some confirmation of the push-and-hold action before letting go.  The answer is to set a timer (the regular kind) on the button press.  If the button is released before the timer goes off, that’s a short push.  If the timer expires first, then that’s a push-and-hold.  I use a 2-second timer.

Here’s the button’s event handler. The same event handler is fired when the button is pressed and when it is released. The state parm tells you which event it is.

static void buttonRed_OnInterrupt(uint port, uint state, DateTime time)
{

  if (state == 1)
  {
     // start the button hold timer
    timerButtonRed = new Timer(new TimerCallback(buttonHold),
       buttonRed, BUTTON_DELAY, BUTTON_DELAY);
  }
  else
  {
     // cancel the button hold timer
     if (timerButtonRed != null)
     {
        timerButtonRed.Dispose();
      }
      if (blnButtonHeld)
      {
         // timer went off, so ignore the release
         blnButtonHeld = false;
         return;
      }

      // normal button press - handle it

      . . .

Note that .Net MF timers are turned off using the .Dispose method – there is no property for setting a timer’s active state.

Last, here’s the timer’s event handler.  I decided to have the red and green button share the same button timer, so a parm is passed to indicate which button it is.  

A boolean variable is used to tell the button’s event handler to ignore the button’s release event when the user eventually lets go.  

Note that, in order to implement the double-button-hold feature for clearing the timers, the code checks the state of the other button using the .Read method, same as you would if you used polling to check the buttons.

private static void buttonHold(object data)
{

  // tell the button release event to ignore it - we're handling it
  blnButtonHeld = true;

  if ((InterruptPort)data == buttonRed)
  {
     // cancel the button hold timer - otherwise, this event will go off again if user keeps button pressed
     timerButtonRed.Dispose();

     // if both buttons are held down, clear the light timers
     if (buttonGreen.Read())
     {
        // cancel the green button's timer
        timerButtonGreen.Dispose();

        clearTimers();
     }
     . . .

So what’s next for this project?  Needless to say, this project is badly in need of an improved user interface, so a menu system can be used to replace (or supplement) the serial port commands.  This will require that we say bye-bye to the tiny 8×2 Grove LCD.

But first, we’ll let Mother Nature handle one of the settings.
Posted in .Net, Electronics, Programming | Tagged , , | Leave a comment

Beagleboard XM / Angstrom Next Steps: NTP Time Daemon and Cron

At about 8 am on January 1st, I was awakened by the beep-beep-beep of an unhappy UPS system.  The power had gone off, and it stayed off for about half an hour. Mercifully (this being 8 am on January 1), my UPS bleated for only about 15 minutes before going to sleep, as did I soon afterwards.

At about 8:30, my Beagleboard was rebooted and promptly resumed its role as Tweet-A-Watt server.  You’ll have to take my word for that, since it abandoned its role as Tweet-A-Watt Twitterer.  I didn’t notice this latter fact until earlier today.  So, if you read my Beagleboard XM And Angstrom – Getting the Big Dog to Run At Full Speed
post sometime between Jan 1 and Jan 3, you might have wondered if the Big Dog had, indeed, begun to fight with Angstrom.  But, rest assured, the failure was mine, not the Beagleboard’s or Angstrom’s.

Hello, Dali
Hello, Dali
What I had failed to do was configure an NTP daemon, so that it would automatically get the correct date and time in just such an event.  Oddly, it ended up being about 6 days off – when I checked it earlier today the system date was December 28th.  This led to a “date out of range” error from the Twitter API when it attempted to tweet.

There is a post on the Mechomaniac blog which explains the process of configuring your timezone and installing the NTP package.  After following those instructions, you can enter ntpdate pool.ntp.org, and your system date and time should be set correctly.  (If the time is off, you probably didn’t set the timezone correctly – as stated in that blog entry, the /etc/timezone file used by some Linux distros seems to be ignored by Angstrom.)

Although I didn’t include the NTP configuration in my initial article about setting up Angstrom the Beagleboard XM, I actually had done it.  I also took the next logical step, which is to configure ntpdate to be automatically run periodically, so that your system date is updated after, say, a power outage.

Or, so I thought.

What I did was

opkg install cron

[Update Aug 13: Oh, hai! Dan from the Future, here. Sorry to interrupt your reading, but I just installed a new Angstrom image generated with Narcissus, and I found that the ntpdate entry was placed in the correct crontabs file after installing cron: /var/cron/tabs/root. So, if you're configuring NTP using an up-to-date cron package, then you're probably done! However, I'm sure you'll want to continuing reading, to see what madcap adventures Dan from the Past got himself into this time...]

Period.  The next step should have been to configure cron to run ntpdate.  However, when I checked to see how cron was configured, I noticed that a folder named /etc/cron/crontabs existed, and in there was a file named root that contained:

30 * * * *    /usr/bin/ntpdate -s -u pool.ntp.org

And, furthermore, the cron package automatically added a file to etc/init.d so that cron was automatically started when the system booted.

Hey yeah! That was easy, especially by Angstrom standards.  Surprisingly easy.  Suspiciously easy.  Hmmm.

On January 1 I belatedly learned that it isn’t quite that easy.  The problem is that /etc/cron/crontabs isn’t where the configuration tables for cron are stored, at least not in the world of Angstrom.  They are in /var/cron/tabs.  One way to configure cron to run ntpdate is to copy the file from /etc/cron/crontabs to here. Then, restart the cron daemon

cp /etc/cron/crontabs/root /var/cron/tabs/
/etc/init.d/cron restart

In the interests of Linux orthodoxy, I should mention that this isn’t the “proper” way.  The proper way is to make use of two crontab commands.

To see what is currently configured (for root – for another user’s crontab, replace “root” in the following commands with the user-ID):

crontab -u root -l

If you are already root (which you should be for the purpose of configuring ntpdate), you can leave out the user name:

crontab -l

To change what is configured

crontab -e

This last one will launch vi so that you can edit the file.

Oops, maybe I should have mentioned that earlier. If you are now staring dumbfounded at the world’s most user-hostile editor, press the Esc key, enter :quit! and take a deep breath. Then, copy the root file from the crontabs folder, as specified earlier. Should you need to change the contents of this file, you can just edit the “root” file using your preferred editor, then copy the resulting file into /var/cron/tabs.  If you go this route, you’ll need to manually restart cron afterwards, whereas changes made with crontab -e are automatically detected by cron.

Actually, its possible to configure crontab to use another editor — Google it if you’re interested.

The above crontab line will run ntpdate at the 30th minute of every hour (i.e. 12:30, 1:30).  That’s probably fine, but if you want to stand out from the crowd you can change it to something less common like 13 * * etc, or change ntp.pool.org to your favorite neighbourhood NTP server.

Not working? Not sure that it’s working, and anal retentive? By default cron will log its results to /etc/cron/log, in a format like this

root (01/03-17:03:00-1237) CMD (-s -u pool.ntp.org )
root (01/03-17:03:00-1236) MAIL (mailed 27 bytes of output but got status 0x0001)

root (01/03-17:08:00-1259) CMD (/usr/bin/ntpdate -s -u pool.ntp.org )

The “MAIL” entry is how it handles errors – presumably you have to configure mail on your Linux box in order for that error message to actually go somewhere.  It wouldn’t matter much in the case of this ntpdate command, since the “-s” tells it to log its output to syslog.  And syslog is another thing that works differently in Angstrom (and doesn’t work at all by default). Sigh. We won’t go there.

If you want to know what went wrong when ntpdate goes wrong, you’ll have to change the crontab entry to something like the following:

13 * * * *  /usr/bin/ntpdate -u pool.ntp.org ›› /home/root/ntpdatelog.txt

This will append the results of the ntpdate command to a text file.

Oh, and a belated Happy New Year from my Beagleboard!

Posted in Gadgets | Tagged , | Leave a comment