Friday, September 19, 2014

Arduino sensors to Raspberry Pi using 433 MHz radios

Finally getting to wrap up my first end-to-end project using Arduino and Raspberry Pi talking via inexpensive 433 MHz radios - the distance over which they work is pretty small but this instructables explains how to add an antenna which dramatically improved the distance. The main idea was to have multiple Arduinos with TX radios, each equipped with DHT and PIR sensors; they all send the data to the Raspberry Pi which posts it to the web and also writes it to a local database. In addition to the values from sensors, I am also sending the voltage level, hoping that this way I can monitor the discharge of the batteries (I am not sure if this works or not, I didn't use an analog pin for this, I am simply getting the voltage on the MCU, using code from this great instructables).

The code for both the sender (Arduino) and the receiver (Raspberry Pi) is in my github repo. The comments in the code pretty much explain everything (how it works, how to connect the sensors and radios, and so on). Here are just some quick ideas that may be interesting.
  • I wanted to transmit all the data in one go so I combined 4 numbers into one long, as described in both sender and receiver code.
  • The transmission between the radios is pretty flaky, that's why I am repeating the send a lot on the Arduino and there is code on the receiver to ignore identical values coming in a 30s interval. This is OK because transmissions from different stations have different station codes so even if the sensor values are the same, the actual overall value is different. Same for a motion sensor: the ON value is different than the OFF value so even if they happen a second apart, both will be received.
  • I started by posting the data to SparkFun's data service, then added Dweet.io and in the end settled on ThingSpeak and I'm also saving all the data to a local SQLite database. See some notes about this later in this post.
  • The code in github is ready to use: just change the stationCode for each Arduino station and use the right API keys in the Raspberry Pi code.
Notes about posting to the web: there are a lot of great IoT services out there with free tiers, to start with I chose the few mentioned above that have a simple API and that accept GET. Each free service has certain rate limits so I need to consider the data volume I will be posting. With one station only getting the temperature sensor data every 3 minutes, this means 5 posts in 15 min. The motion data is a bit more intensive, roughly 2 posts 5 sec apart (ON and OFF) every time the motion sensor is triggered which at the most (sensor triggered all the time) means 24 posts a minute. Because there is potentially a lot more motion data, I am using 2 different data streams (channels, feeds): one for temperature, humidity and voltage and one for motion; even so with a lot of stations there is the risk of missing data (according to each service rate limits as described below) so I decided to also implement a local SQLite database to store the data and eventually process it later and post it (maybe using node-red).

Rate limits:
  1. SparkFun's data service is limited to making 100 log requests every 15 minutes. Given the 5 posts in 15 min, the temperature data stream can accommodate 20 stations without losing data which is more than I need. The motion data though may accommodate only 1 or 2 stations (more if there is not a lot of activity around them) without starting to lose data. I will post all the data I can, the remainder being lost.
  2. Dweet.io holds on to the last 500 dweets over a 24 hour period so while there won't be data loss, the motion data will probably cover only a few hours at the most, even for just a couple stations.
  3. ThingSpeak has a rate limit of an update per channel every 15 seconds so more than one station per channel will more than likely run into this limit (even 2 temp stations sending data every 15 minutes can happen to send data less than 15 sec apart); I could wait and retry until the post goes through but this would delay the code and risk missing data sent by the stations. Because of this, I decided to tie the local db data to this service: every time I post to ThingSpeak, I check the response: if > 0, the post was successful, I will log the data to the db with a flag of posted=true (1); if = 0, the post was not successful so I will log the data to the db with a flag of posted=false (0). Later I may create a node-red flow to get all the db data with posted=false and repost it to ThingSpeak; each row records the UTC time of the actual insert so I can send it along with the data, the measurements getting recorded at the actual time they took place not the time I post them.
If you find anything useful in this post and the code I am very happy. Most of this stuff is on the web but putting it together in a real project took some work; I am happy I did it and learned so much in the process. When I started I had no idea how to use a radio on either Arduino or Raspberry Pi, how to work with a PIR sensor and how to use interrupts, how to write any real C code, to work with sqlite and post to REST services on Raspberry Pi and much more. I used a lot of info from the web, from too many people and websites to thank individually but a big thank you! goes to everyone that provided help and information in a way or another.

15 comments:

Ray Scicluna-Kerry said...

This is absolutely great! Since getting my Raspberry Pi two months ago, I was looking to do some home automation / security and incorporate some sensor logging. I wasn't really interested in copying anyone elses solutions but yours fits the bill so perfectly I can't see the point in reinventing the wheel. One question (for now) the data word that the Arduino sends are all of the bits used or is there spare capacity to add other sensors onto each node.
Thanks Ray

merlin13 said...

Thanks a lot for your kind words!

As far as your question, I am using all the bits so for more sensors you may need more than one transmission.

Gabriela Alexandra said...

Hi,can you tell me the code that you use for receiving data from arduino,because my raspberry pi shows only some numbers

merlin13 said...

Gabriela, most of the code is in my repo, mentioned in the article. Is there anything in particular I can help you with?

Man said...
This comment has been removed by the author.
Man said...

i got problem only receive 268435457 1-0-0-1 and 268435456 1-0-0-0 from ./RFRcvCmplxData
never get temperature,humidity and voltage..

but arduino serial monitor output
Humidity: 69.00 % Temperature: 29.00 *C 84.20 *F Heat index: 90.58 *F
batt: 93
DHT: 268435549
PIR: 268435457
10000000000000000000000000001
Humidity: 69.00 % Temperature: 29.00 *C 84.20 *F Heat index: 90.58 *F
batt: 93
DHT: 268435549
PIR: 268435457
10000000000000000000000000001

Man said...

resolve the issue adding humid=h; and temp=t;

float hi = dht.computeHeatIndex(f, h);

Serial.print("Humidity: ");
Serial.print(h);
humid=h;
Serial.print(" %\t");
Serial.print("Temperature: ");
Serial.print(t);
temp=t;
Serial.print(" *C ");
Serial.print(f);
Serial.print(" *F\t");
Serial.print("Heat index: ");
Serial.print(hi);
Serial.println(" *F");

}

merlin13 said...

Sorry I didn't have time to look into it, I am happy you found a solution.

Man said...

right now got problem MQTT error 14 after long run ./RFRcvCmplxData

dont know how to figure out adding mosquitto_loop_start if connection to MQTT broker drop and reconnect

Man said...

many thanks to you for sharing this project....i so interested :)

Man said...
This comment has been removed by the author.
merlin13 said...

I had problems as well using mqtt, since this was my first time using the library I figured I am missing something and never had time to go back to find out what was wrong. If you do find a solution, please post a link to a gist or your github repo. Thanks!

Man said...

im already figure out that error 14...im combine script in 433Mhz-Arduino/RFRcvCmplxData.cpp + /mqtt/RFRcvCmplxData.cpp so it send MQTT plus update thingspeaks at once

res variable used 2 times so it conflict...now rename it to res1 for all curl res variable.

now im trying to solve issue auto reconnect to mqtt broken...

testing stop and then start my mqtt broker it seem it not auto reconnect to broker

for ur info im new to programming and linux

Unknown said...

Awsome project!
Thanks for sharing it. It took me quite some time to figure a lot of stuff out, but I'm almost up and running.
I'm running in to a small problem with the MQTT part of it.
when running RFMqttRcvCmplxData It's kind of stuck at 'Client client_1 sending CONNECT' and I would like some pointers on how to proceed. Mosquitto seems to be working fine.

merlin13 said...

Unfortunately, I don't have any idea why your message is not sent. As I commented in my code, I've seen notes on the web about using mosquitto_loop() to make sure all your messages are sent but I never got to look into the details. I hope you will be able to make it work.

Thanks a lot for your kind words!