John's Ramblings

Arduino reading a CSV pachube feed with the ENC28J60

john — Tue, 24 Aug 2010 12:12:14 +0000

The question of how to do GET requests with the ENC28J60 arose recently so here is an example.
This is using the etherShield from Nuelectronics and @andrewdlindsay‘s etherShield library.

The code assumes some familiarity with the ethershield examples.

#include 

static uint8_t mymac[6] = {0x54,0x55,0x58,0x10,0x00,0x28};
static uint8_t myip[4] = {192,168,1,25};
// Default gateway. The ip address of your DSL router. It can be set to the same as
// websrvip the case where there is no default GW to access the
// web server (=web server is on the same lan as this host)
static uint8_t gwip[4] = {192,168,1,1};

//============================================================================================================
// Pachube declarations
//============================================================================================================
#define PORT 80                   // HTTP

// the etherShield library does not really support sending additional info in a get request
// here we fudge it in the host field to add the API key
// Http header is
// Host: 
// X-PachubeApiKey: xxxxxxxx
// User-Agent: Arduino/1.0
// Accept: text/html
#define HOSTNAME "www.pachube.com\r\nX-PachubeApiKey: xxxxxxxxxxxxxxxxxxxxxxxxxxx"          // my API key
static uint8_t websrvip[4] = {173,203,98,29};  // www.pachube.com
#define HTTPPATH "/api/9999.csv"      // The feed

EtherShield es=EtherShield();

#define BUFFER_SIZE 500
static uint8_t buf[BUFFER_SIZE+1];

void browserresult_callback(uint8_t statuscode,uint16_t datapos){
 Serial.print("Received data, status:"); Serial.println(statuscode,DEC);
 if (datapos != 0)
 {
  Serial.println((char*)&buf[datapos]);
  // now search for the csv data - it follows the first blank line
  // I'm sure that there is an easier way to search for a blank line - but I threw this together quickly
  // and it works for me.
  uint16_t pos = datapos;
  while (buf[pos])    // loop until end of buffer (or we break out having found what we wanted)
  {
   while (buf[pos]) if (buf[pos++] == '\n') break;   // find the first line feed
   if (buf[pos] == 0) break; // run out of buffer
   if (buf[pos++] == '\r') break; // if it is followed by a carriage return then it is a blank line (\r\n\r\n)
  }
  if (buf[pos])  // we didn't run out of buffer
  {
   pos++;  //skip over the '\n' remaining
   Serial.print("CSV line is:");
   Serial.println((char*)&buf[pos]);
  }
 }
}

void setup(){
 Serial.begin(9600);
 Serial.println("Ethershield Get example");

 /*initialize enc28j60*/
 es.ES_enc28j60Init(mymac);

 //init the ethernet/ip layer:
 es.ES_init_ip_arp_udp_tcp(mymac, myip, PORT);

 // init the web client:
 es.ES_client_set_gwip(gwip);  // e.g internal IP of dsl router

 es.ES_client_set_wwwip(websrvip);  // target web server

}

void loop()
{
 static uint32_t timetosend;
 uint16_t dat_p;
 while(1){
  // handle ping and wait for a tcp packet - calling this routine powers the sending and receiving of data
  dat_p=es.ES_packetloop_icmp_tcp(buf,es.ES_enc28j60PacketReceive(BUFFER_SIZE, buf));
  if (dat_p == 0)
  {
   if (millis() - timetosend > 10000)  // every 10 seconds
   {
    timetosend = millis();
    Serial.println("Sending request");
    // note the use of PSTR - this puts the string into code space and is compulsory in this call
    // second parameter is a variable string to append to HTTPPATH, this string is NOT a PSTR
    es.ES_client_browse_url(PSTR(HTTPPATH), NULL, PSTR(HOSTNAME), &browserresult_callback);
   }
  }
 }
}

Hacking the pink IM-Me Part 1

john — Tue, 03 Aug 2010 11:15:32 +0000

This is the first of several posts about the IM-Me pink wireless toy from Mattell. The aim of this post is to collect all of the relevant links I have found useful from the web.

You can currently get one (or more) of these toys for £8.99 at Amazon
The IM-Me is based upon the TI CC1110F32 chip, datasheet
Dave’s hacks – im-me hacking
Dave’s hacks – im-me LCD interface hacked
Travis Goodspeed’s blog on the IM-Me
Michael Ossmans blog – a $16 spectrum analyser
CC-flasher – open source programmer – most useful for the SDCC open source tool chain for the CC1110
From TI:To develop software, program and debug the CC1110, the IAR Embedded Workbench for 8051 is recommended. An evaluation version of IAR EW8051 is included in the kit. This free evaluation version and a free code size limited version, can be downloaded from the web, see www.iar.com/ew8051 this is limited to 4K of code, this can be increased to 16K by following the instructions in section 9.1 of this document
Details of the CC1110 debugging interface and how to program it

The next post on this topic will look at modifying the IM-Me so that the debug interface is easily used, setting up an Arduino to program the IM-Me and the Arduino software required to interface to the IM-Me.

Further posts will then look at programming the IM-Me using the IAR EW8051 IDE www.iar.com/ew8051 (I use the 4kb code limited Kickstart edition, increased to 16kb).

MISO and Arduino Ethernet Shields

john — Thu, 24 Jun 2010 12:03:23 +0000

Both of the major Arduino ethernet shields (standard Arduino and NueElectronics) have issues with the MISO signal which prevent them being used with other SPI devices.

The standard Arduino shield uses the Wiznet5100 chip – this chip has a flaw in that the CS signal does not tri-state the MISO line. The Wiznet application note can be found here. The relevant part is below.

Quote
Basically, multiple SPI slave usage is the same as single SPI slave usage. One difference between other SPI slave devices compared to the W5100 is that the MISO output is continuously driven in the W5100 whether the /SCS is asserted as high or as low. As well, when the 5100 /SCS is asserted as high when using multiple slaves, other SPI devices cannot be read or written by the SPI master on the SPI BUS simultaneously. These problems will continue unless the recommendations listed below are followed.

Recommendations:
- When accessing another device on the SPI BUS rather than the W5100, assert the SEN pin in the W5100 as low first, then access the other devices.
- When accessing the W5100, the SEN pin should be high.
…
The W5100?s SEN signal is input from the /SCS through the inverter. If you don?t want to use the inverter, you can control each signal through the I/O port of the MCU.
end quote

So by inverting the SS line (Arduino digital 10) and driving SEN from that fixes the issue. Here is a photo of my rather crude fix – using a CMOS 4011 quad 2 input NAND gate chip. Note this is a DFRobot clone – the pads for the PROG jumper (one connected to SEN) are reversed from the normal shield. The wire running underneath connects to the chip select line (Ardunio 10) – I went underneath for this to allow my to cut the line later if I needed another pin for chip select. Small rant – all SPI shields seem to assume they can use CS on pin10 – why cannot this come out to a solder jumper or pads so that I can select any pin as chip select. Using multiple SPI devices is a real pain because of this.
By request the circuit diagram for this is also shown. Note that in the board pics I placed the 4011 upside down (I could then glue it to the board). Also all unused inputs of the 4011 (I only use one gate out of 4) are grounded (because this is good practice with CMOS devices), even here I made one of my many mistakes – the diagram is wrong in fact pins 5&6 are grounded not 4&5.
Any inverter would have done, I just happened to have a 4011 in my ‘bits’ box.

———————————————————————————

The NueElectronics shield uses a 74HCT08 chip to boost the MISO and INT signals to 5V levels, this chip is not tri-state and so the MISO line is always driven.

Replacing the 74HCT08D with a 74AHC125D is an almost direct replacement – cut the 74HCT08D off and tidy up the pads,

the connection between pins 1&2 and pins 4&5 needs to be broken and then the 74AHC125D soldered in the same place. Then add two small jumper wires: pin1 connected to GND (for the INT line), p4 connected to SS (Arduino digital 10) for the MISO line (Arduino digital 12).

I2C LCD and keypad controller using ATTiny2313

john — Sun, 18 Apr 2010 15:57:30 +0000

A while back @andrewoke was bemoaning the availability of an open source HD4470 I2C LCD and keypad controller. So I wrote one – the code fits into an Attiny2313 (just). ATTiny2313 chips are available for under $2.
The AVRDude command line I use to program the ATTiny2313 with Adafruit’s USBTiny programmer is: (note the fuse change – this is to run the 2313 at 8MHz)
avrdude -p attiny2313 -c USBTiny -u -U lfuse:w:0xE4:m -U flash:w:I2C_LCD.hex -U eeprom:w:I2C_LCD.eep

Downloads:

Source and hex files
Documentation
Arduino library – not ready yet.

Wire up like this:

New Home Monitoring – part 2

john — Sun, 18 Apr 2010 15:47:31 +0000

*********
Notice – the protocol to be used has pretty much completely changed. I am adopting the aProtocol described on the OpenKontrol web site

aProtocol is essentially a hub polling devices, it does allow for devices to announce events but in the main it is polling.
The aProtocol is still work in development, I have joined the team to help make it into a robust home monitoring and automation protocol.
*********

Now that I have decided on the hardware (JeeNode V4 for nodes and Arduino for data logger) I have started to look at the data format and protocols between the nodes/sensors and the data logger. It is clear that there are three basic types of data to be dealt with

a) Current reading (sensor to data logger ). This includes events (e.g. PIR trigger or Battery Low).

b) Totals & counts (sensor to data logger). Note: totals/counts are never cleared down (unless the node is reset), the receiver of the data needs to keep track of the last total and subtract it. This makes it easy to get total for last 15 minutes, total for day etc.

c) Requests and instructions (data logger to sensor)

Low level protocol will ensure message integrity (CRC check or similar). If the sending message requests it then the receiver will ack the message (most messages will not use this – see message descriptions).

Some means to assess transmission quality (number of missing packets) is needed. The low level transmission protocol will send an incrementing sequence number (byte) with every message. Each node will also track the last sequence number received from the data logger. The data logger will keep a table; for every node the last sequence number sent and received will be stored.

The basic mode of operation is that nodes will send out data at predetermined time intervals (say every x seconds – but may differ for some sensor types), if a message goes astray or is corrupted then this is not important and we will get the value next time. Whilst this is not completely accurate (for example for energy readings) in that some values may be lost it is good enough for monitoring purposes. We require to know if some messages have arrived successfully so these will request an ack, (and re-transmission if no ack is received) in these cases the receiver should be able to accept the same message more than once without issues as the ack may be lost.

All sensor to data logger messages will be broadcast (destination node 0) – this facilitates having more than one listener – although only the data logger will ack (if ack is requested).

All messages will be in the following format : 12 bytes, human readable ascii to facilitate debugging (32 bit values will be sent as hex). Numeric information will be zero padded to the left (e.g. 123 will be sent as 00000123).
Message Format

Byte #	Name	Meaning
0	msg_Type	Message type – list below
1	msg_NodeID	Sending node ID (A-Z)
2	msg_SnsID	Sensor ID within node (A-Z) + special types
3	msg_SnsType	Sensor type – list below
4	msg_Data1	Data to be sent
5	msg_Data2
6	msg_Data3
7	msg_Data4
8	msg_Data5
9	msg_Data6
10	msg_Data7
11	msg_Data8

Message Types

Type	Meaning
A
B
C
D
E	Echo – Request a I’m here message
F	I’m here
G
H
I
J
K
L
M
N
O
P
Q
R	Sensor reading
S	Request reading
T	Request total
U
V
X	Node (re)started
Y
Z

Some sensor IDs are reserved for special use. These are: (lowercase letters)

ID	Meaning
a	Total of missing received sequence numbers (16 bit)
b	Last sequence number received
c	Last sequence number transmitted
d	Battery low indicator



z	All sensors (e.g. to request totals).

Sensor Types

Type	Meaning
A	Generic analogue reading (0-124)
B	Generic digital reading (0/1)
C	Counter state change (e.g. door, window)
D	Counter total (16 bit)
E	Energy use (Watts) – Gas or Electricity
F	Energy use total Watts (32 bit)
G
H	Humidity
I
J
K
L	Light Level
M
N
O
P	PIR sensor triggered
Q
R
S
T	Temperature (tenths of a degree C)
U
V
X	Display (output) cursor position (row,col)
Y	Display (output) command (blink, colour etc)
Z	Display (output) data (8 bytes)

Arduino XBee shield issue

john — Sat, 13 Mar 2010 11:02:58 +0000

The Arduino XBee shield uses a resistor divider to connect the TTL 5V Arduino Tx to the XBee Din line. This has the side effect of pulling the Din line to GND if the Arduino is not asserting the Tx line. Problem is that the Arduino does not have control of the TX line during the brief power up sequence.
Series one (802.15.4) Xbees use a low on DIn during power up to force entry into command mode at 9600 baud – an escape for when all else fails.
This means that you cannot use the XBee for about 10 seconds after power up. To avoid this you need to pull up the Arduino Tx line by attaching a 10K resistor between Tx and 5V. Here are a couple of shields I modified

New home monitoring setup – part 1

john — Tue, 23 Feb 2010 21:54:12 +0000

Updated 26/2/2010
I have been looking at revisiting my home monitoring setup for some time. Whilst I have considered plug computers and other slugs as the hub for this I keep coming back to the Arduino. An Arduino has enough CPU power, is cheap, very low power and I can maintain it should it develop a fault.

For the sensor communications I have decided to standardize on the JeeLabs JN4. Most sensors will connect directly to the sensor controllers, but for the CC128 then I will use a bare board Arduino as a data concentrator.

For the first part of this project I will build the data storage unit. This will have:

1) Seeeduino 328 – chosen due to the additional 3.3v power available.
Update 24/2/10: I am revising this decision as the Seeeduino has placed the 500mA fuse on the 5v line (not on the USB 5v line), I need more than this (it is the LCD back light that draws most of this) and the fuse keeps cutting power. I will probably end up with a standard Duemilanove and put a 3.3v regulator in for the 3.3v devices.
Update 25/2/10: Measured current required – 305mA (180mA without the LCD) – so it must be the regulator cutting out because the heat sink is not adequate. Moved to Duemilanove, with 100mA 3.3v LDO regulator and although the 5v regulator does get hot I do not have any cutout problems.

2) Arduino ethernet shield

3) XBee-pro 802.15.4 for serial wireless communications connecting to the Arduino via a XBee shield.
Update: Using the JeeLabs JN4 with RFM12B operating at 868MHz for the wireless communications

4) 128Kbyte I2C FRAM (RAMTRON) this will be used to hold message queues – this is 3.3v

5) 2GB SD card – initially with uALFAT controller (I2C 3.3v), but I intend to use a SD card directly for the final version.

6) I2C serial LCD – based upon my own ATTiny2313 I2C Serial LCD adapter

7) I2C battery backed RTC to use as a time stamp for all readings.

I am using my own level converter (based upon this) to drive the 3.3v I2C bus from 5v.

I have gathered the necessary hardware and will start detailed design at the weekend.

Calke Abbey

john — Mon, 22 Feb 2010 10:30:19 +0000

A week ago we went for a walk round Calke Abbey grounds. Calke Abbey is National Trust and about 10 miles south of Derby.
As is usual I took a few (well a lot) of photos.

I can see a face in this gnarled old Oak trunk.

Cannot post reliably – keep getting errors

john — Sun, 21 Feb 2010 09:39:47 +0000

I am having great difficulty in posting anything that is not straight text – I suspect a WordPress bug (recently upgraded to 2.9) but I cannot narrow it down. It appears to be when I have links to external sites. Wonder if WordPress introduced a new security feature?
Got it – any post created with the twitter upload plugin active and WP 2.9 or greater gets garbled. Disable and delete the plugin, then create new posts – all is good.
Set up with different plugins and theme.

FIO (Arduino and XBEE) door, doorbell and others monitor

john — Wed, 03 Feb 2010 11:01:28 +0000

I have been extending my range of inputs to the CurrentCost, Ardunio, Xport pachube logger.
In order to do this I wanted a wireless sensor module.
After looking round I chose the FIO, has an XBEE socket and ATMega168, very little else. Can be powered from a rechargeable LiPo battery.
The next step was to look at the power requirements. I am using a 1000mAH battery and with the normal FIO consumption of 65mA this would last about 16 hours – not very long. After some experimentation I found that if I ran the XBEE in pin controlled sleep mode 2 (less power used than mode 1 at 3.3V) and the ATMega in full sleep I could get the consumption down to 270uA. Switching off the ADC and the analog comparator got this down further to 170uA. Assuming that we have 30 minutes active per day (I’m hoping for much less) this will run for just under 1 month. In practice (this has been in place for the last 6 months) it runs for about 4 months before it needs recharging.
I am using a low value on digital inputs 2 or 3 to wake up the ATMega and then the ATMega wakes up the XBEE.

To be continued …