Yafa! How to make a fermentation controller using an Arduino Yun.: November 2013

Saturday, November 30, 2013

Temperature reading

red = 5v
white = signal
metal= gnd

setResolution does not seem to retain value after powerdown???

Bridge Communication - Part 4

The new IDE 1.5.5 was released with Bridge bug fixes. This solved so far all my Bridge issues :-)

Thursday, November 28, 2013

Good to knows - 2

int freeRam () 
{
  extern int __heap_start, *__brkval; 
  int v; 
  return (int) &v - (__brkval == 0 ? (int) &__heap_start : (int) __brkval); 
}

Sometimes the Bridge seems to fail, decided to make a simple stress test:

The basic idea of the test is:

For a certain message length=M do:
- reset MCU
- cpu puts random message of length M and a single char ID (key1)
- mcu constinuously checks for a new ID, if it finds a new ID, it puts the message back to the cpu (key2)
- cpu checks (after a couple of seconds) if it gets the message it had sent.
- if YES: repeat (max 200 times for now)
- if NO: failure and decrease M
Results get written in a file.

To rule out timing issues I put quite some delays in the process (2 second wait on the cpu side, 2*250ms on the mcu side).

The result is imho quite amazing (not all lengths tested for now):

msg_length=60 FAILURE after 21 messages
msg_length=55 FAILURE after 21 messages
msg_length=50 FAILURE after 21 messages
msg_length=45 FAILURE after 21 messages
msg_length=40 FAILURE after 21 messages
msg_length=35 FAILURE after 21 messages
msg_length=30 FAILURE after 21 messages
msg_length=25 SUCCES 199 messages
msg_length=20 SUCCES 199 messages
msg_length=15 SUCCES 199 messages

Note that the buffer size on the mcu side is identical for each message length.

So all message lengths above or equal to 30 seem to fail, and surprisingly each time after 21 message exchanges??
Below that length no failures were detected.

Code:

#include "Bridge.h"

void setup() 
{
  Bridge.begin();
}

void loop() 
{
      unsigned int len;
      char buffer[64];
      char prevID='-';
      while(true)
      {
          Bridge.get("key",buffer,64);
          if(buffer[0]!=prevID)
          {
              // new message
              delay(250);
              Bridge.put("key2",&buffer[1]);
              prevID=buffer[0];
          }
          delay(250);
      }
}

and

#!/usr/bin/python
import sys    
import time
import string
import random

def rand_generator(size=6, chars=string.ascii_uppercase + string.digits):
    return ''.join(random.choice(chars) for x in range(size))

sys.path.insert(0, '/usr/lib/python2.7/bridge/') 
from bridgeclient import BridgeClient as bridgeclient
bc = bridgeclient()                              

from subprocess import call

f=open('test_result.txt','w')

cnt4=0
msg_id='a'
n_loops=200

for msg_length in range(60,10,-5):
   print "msg_length set to ",msg_length
   #reset mcu
   print "resetting mcu now"
   call(["reset-mcu"])
   time.sleep(5)
   for loop in range(1,n_loops):
       time.sleep(2)
       if cnt4==0:
         msg_id='a'
         cnt4=1
       elif cnt4==1:
         msg_id='b'
         cnt4=2
       elif cnt4==2:
         msg_id='c'
         cnt4=3
       else:
         msg_id='d'
         cnt4=0
       msg=rand_generator(msg_length)  
       toput=msg_id+msg
       print "put",toput
       bc.put('key',toput)
      
       # wait for identical returned value
       trial=1
       while trial<4:
           time.sleep(1)
           r = bc.get('key2')
           print "get ",r
           if r is None:
             print "   No answer key found yet"
           else:
             if r==msg:
                 print "   OK after",trial," trials"
                 break
             else:
                 print "   FAIL"
                 trial=trial+1
       if trial>=4:
         f.write("msg_length=")
         f.write(str(msg_length))
         f.write(" ")
         f.write("FAILURE after ")
         f.write(str(loop))
         f.write(" messages\n")
         break
       elif loop==n_loops-1:
         f.write("msg_length=")
         f.write(str(msg_length))
         f.write(" ")
         f.write("SUCCES ")
         f.write(str(loop))
         f.write(" messages\n")
f.close()

also available on http://forum.arduino.cc/index.php?topic=201484.msg1484792#msg1484792

Sunday, November 24, 2013

Good to knows - 1

http://forum.arduino.cc/index.php?topic=200732.msg1480813#msg1480813

Do not CTRL-C a python script that uses the bridge. Stuff may hang. Use kill-bridge instead.

To reset the 32u4 from linino, you can use command reset-mcu

Sunday, November 17, 2013

Bridge communication - Part2

The message size that I need will be well below 266, more in the range of 30 characters or so. So I am continuing without caring why it crashes if we try to send long messages.

In my setup, the Linino side will be the master, sending out commands to the Sketch side. For example a command to set the temperature. Most commands will be accompanied by a value (e.g. the temperature).

Given the messages stay in the Bridge data store after a get, and the Sketch will be likely polling to see if there is a new command I need something to identify if a message is a new one or not. I will just use 1 character in the message, that should change for every new command.

So the message that the Linino side will put looks as follows:
1 message ID char + command + value

I made a C++ class BridgeComm.h that on top of the bridge, offers this functionality. It has a method that can check for a new message, if so it returns true + the null-terminated command and value.

Maybe the Mailbox class that is shipped with the Arduino does similar things, but it was just less time to make it using the Bridge.

For now the command will be 8 characters, the value 16. But that are just consts that can be changed in whatever I need.

My Yun sketch now looks as follows:

#include <Bridge.h>
#include <Console.h>
#include <BridgeComm.h>

BridgeComm bc;
char command_buff[BridgeComm::REQ_CMD_BUFF_LEN];
char value_buff[BridgeComm::REQ_VAL_BUFF_LEN];

void setup() {
  bc.setup();
  Console.begin();
  while(!Console)
  {
    ;
  }
  Console.println("Connected to the Console!");
}

void loop() {
  if(bc.check_for_command(command_buff,value_buff))
  {
     Console.print("Command received=");
     Console.print(command_buff);
     Console.print(", Value recieved=");
     Console.println(value_buff);       
  }
  delay(200);
}

I made a simple Python script that every 2 seconds sends another command (Temp-Max,Temp-Min,Temp) and a temperature value.
The output of the Yun console looks as follows:

Command received=Temp-Max, Value recieved=22.5
Command received=Temp-Min, Value recieved=23.5
Command received=Temp    , Value recieved=24.5
Command received=Temp    , Value recieved=25.5
Command received=Temp    , Value recieved=26.5
Command received=Temp    , Value recieved=27.5
Command received=Temp    , Value recieved=28.5

Another step in the right direction.

Bridge communication - part1

Some first trials with the Arduino Bridge. Starting simple with put and gets.
A first trial:

the Linino puts random characters using key1 (then waits 3 seconds)
the Sketch side gets on key1, and puts the same message on key2 (should happen in the 3 seconds)
the Linino gets on key2 and compares with what it had put on key1. If OK, it restarts with a longer message. If it fails it tries another get.

At a message length of 266 (always that length) it seems to FAIL:

Trying length 262
put AZORBF9S9C8E1V2YR531TS4GOFRN5YHOLCI5T7KAP1E6SLVH3UPK4SVABTLB7RC80JTCHBPEZHK5ZFE183G5GMHHD552DUBTKJD2PPTU5VANVBHFANI01SIDCEK2PN88REU7V0RJX0T0PS8JPDJX988PBF78Y37XYWHHBOL0FGCZ7AXJPXFKF0MKUH5I2EN89EAN1I9NPWEYVKB5DWCL7NWKTDBDXHQ62MCOOHP348SGNO2J5LPHW8TFVAW9E913T9UGN7
got AZORBF9S9C8E1V2YR531TS4GOFRN5YHOLCI5T7KAP1E6SLVH3UPK4SVABTLB7RC80JTCHBPEZHK5ZFE183G5GMHHD552DUBTKJD2PPTU5VANVBHFANI01SIDCEK2PN88REU7V0RJX0T0PS8JPDJX988PBF78Y37XYWHHBOL0FGCZ7AXJPXFKF0MKUH5I2EN89EAN1I9NPWEYVKB5DWCL7NWKTDBDXHQ62MCOOHP348SGNO2J5LPHW8TFVAW9E913T9UGN7
OK
Trying length 263
put DA4VIB5FV9B8K669U3POPSNQHO1U8NI3JWLBG1LW9YW9028ME309HHPCGW39ZQZQYJ43TIVN367MIIX8N74RPS45V3TFSBX5ZIQGO8FDTP40B10X7A7CLJ3DOGZMDSHNO6V4WWRN1WHAR3GHA094UQ69OKYN46GENJG77C8E322DC16Q3SOTQ5PWE12QD7LOF450ZU5ZUFTLJZCZCL5TA5MBGUCYIEQS9W02F5KP78NI6L4BJT6QCMG9IHQJIS26M9Z64R0
got DA4VIB5FV9B8K669U3POPSNQHO1U8NI3JWLBG1LW9YW9028ME309HHPCGW39ZQZQYJ43TIVN367MIIX8N74RPS45V3TFSBX5ZIQGO8FDTP40B10X7A7CLJ3DOGZMDSHNO6V4WWRN1WHAR3GHA094UQ69OKYN46GENJG77C8E322DC16Q3SOTQ5PWE12QD7LOF450ZU5ZUFTLJZCZCL5TA5MBGUCYIEQS9W02F5KP78NI6L4BJT6QCMG9IHQJIS26M9Z64R0
OK
Trying length 264
put LKAR6LQO7LV74BLY6O4FXM8WJA9X0CP5ZKZLXAT19YM07EM3IBTEV4IOWVFQLUQZ7KSC14I96PAK9WY2DXPXP6UJ0ZIIU6IBU65G1PMOMNSJN2HZXF2DEMID6ZKIE3XZ4O5ZJ80UVP45AQF9TYBU8PNYRFC47NF9O5SM40VXO1ANYVII8IHZ8HQZBJ1NZZLOY1N8XVH7MXVCN5XJ7CKN0VSD19Q6Y9QMOS4H7BOHZNZ3SIU8T8BTRHY3I4C354BAH5EMU2WK
got LKAR6LQO7LV74BLY6O4FXM8WJA9X0CP5ZKZLXAT19YM07EM3IBTEV4IOWVFQLUQZ7KSC14I96PAK9WY2DXPXP6UJ0ZIIU6IBU65G1PMOMNSJN2HZXF2DEMID6ZKIE3XZ4O5ZJ80UVP45AQF9TYBU8PNYRFC47NF9O5SM40VXO1ANYVII8IHZ8HQZBJ1NZZLOY1N8XVH7MXVCN5XJ7CKN0VSD19Q6Y9QMOS4H7BOHZNZ3SIU8T8BTRHY3I4C354BAH5EMU2WK
OK
Trying length 265
put WY8XFBQL8ND0WMVVE1TU409M90YJXC2VMUAD6FSVFUNWD112NPD5ZP389X7SNOIF6D6Z46EY1Z2NBL3I46244MF88NBPGRUIRREXZWZBS5TFI4XMBZXZVIUR2VLYV9FSHBCMWWQ23TNQ8229RKFWX3Q27APKLWCLP80IDVH0NNQU4HJF2DPG3RN1EYH3FGUFHDS2SH34JH34TITEPPA9IB7QWXV7AMEMNXBK5NMF91026XV2BE9FS1PQPW93CUY4QDOUA29KI
got WY8XFBQL8ND0WMVVE1TU409M90YJXC2VMUAD6FSVFUNWD112NPD5ZP389X7SNOIF6D6Z46EY1Z2NBL3I46244MF88NBPGRUIRREXZWZBS5TFI4XMBZXZVIUR2VLYV9FSHBCMWWQ23TNQ8229RKFWX3Q27APKLWCLP80IDVH0NNQU4HJF2DPG3RN1EYH3FGUFHDS2SH34JH34TITEPPA9IB7QWXV7AMEMNXBK5NMF91026XV2BE9FS1PQPW93CUY4QDOUA29KI
OK
Trying length 266
put MAJPFJVK0PEHM866Z170SQR0Z12Z5HI1TGW5THNUY6P0PQ1CEZ1PACJC7ZFJFAIKH7M0L2DZY0GS5CE9Q38XSZ95E1LF2KI7NZKUNSDSR09SVIQV9DMVI43APXHNYKQ5KE6TYF21CU6P4HPQG8YFPRA3PKRULIG5MH4XD9PXQDE0HH06X2FOXUQ1P1MOP9FGESU2LHHJ49L79CJEJSGPWJLRR2NHY75BK1NATEQGNNYD4A3IEOJ0P311KT20A6X4ZG1TU4DKEK
got WY8XFBQL8ND0WMVVE1TU409M90YJXC2VMUAD6FSVFUNWD112NPD5ZP389X7SNOIF6D6Z46EY1Z2NBL3I46244MF88NBPGRUIRREXZWZBS5TFI4XMBZXZVIUR2VLYV9FSHBCMWWQ23TNQ8229RKFWX3Q27APKLWCLP80IDVH0NNQU4HJF2DPG3RN1EYH3FGUFHDS2SH34JH34TITEPPA9IB7QWXV7AMEMNXBK5NMF91026XV2BE9FS1PQPW93CUY4QDOUA29KI
FAIL
got WY8XFBQL8ND0WMVVE1TU409M90YJXC2VMUAD6FSVFUNWD112NPD5ZP389X7SNOIF6D6Z46EY1Z2NBL3I46244MF88NBPGRUIRREXZWZBS5TFI4XMBZXZVIUR2VLYV9FSHBCMWWQ23TNQ8229RKFWX3Q27APKLWCLP80IDVH0NNQU4HJF2DPG3RN1EYH3FGUFHDS2SH34JH34TITEPPA9IB7QWXV7AMEMNXBK5NMF91026XV2BE9FS1PQPW93CUY4QDOUA29KI
FAIL
got WY8XFBQL8ND0WMVVE1TU409M90YJXC2VMUAD6FSVFUNWD112NPD5ZP389X7SNOIF6D6Z46EY1Z2NBL3I46244MF88NBPGRUIRREXZWZBS5TFI4XMBZXZVIUR2VLYV9FSHBCMWWQ23TNQ8229RKFWX3Q27APKLWCLP80IDVH0NNQU4HJF2DPG3RN1EYH3FGUFHDS2SH34JH34TITEPPA9IB7QWXV7AMEMNXBK5NMF91026XV2BE9FS1PQPW93CUY4QDOUA29KI
FAIL

From that length onwards, I keep on receiving the previous message...did the Arduino side crash??
If I reset the Arduino side, things are ok again.

This is the python script:

def id_generator(size=6, chars=string.ascii_uppercase + string.digits):
    return ''.join(random.choice(chars) for x in range(size))

sys.path.insert(0, '/usr/lib/python2.7/bridge/')
from bridgeclient import BridgeClient as bridgeclient
bc = bridgeclient()

for x in range(250,300):
myrand = id_generator(x)
print "Trying length",x
print "put", myrand
bc.put('to_slave',myrand)
# wait for identical returned message
while True:
     time.sleep(3)
     r = bc.get('from_slave')
     if r is None:
        print "No answer key found yet"
     else:
        print "got", r
        if r==myrand:
            print "OK"
            break
        else:
            print "FAIL"

This is the sketch:

#include <Bridge.h>

char value[512];

void setup() {
  pinMode(12,OUTPUT);
  Bridge.begin();
}

void loop() {
  Bridge.get("to_slave",value,512);
  Bridge.put("from_slave",value);
  delay(100);  
}

Saturday, November 16, 2013

Mounting the Yun SD card on my Linux machine using sshfs

The editor on the Yun terminal is rather limited.

Using sshfs you can mount the SD card to a directory on your Linux machine:

sshfs root@arduino.local:/mnt/sda1/arduino /home/<your user name>/Yun

You first need to install openssh-sftp-server on the arduino Yun:

opkg update
opkg install openssh-sftp-server

Obviously you can mount any directory like this, it does not need to be a directory on the SD card.

Note: how to create partitions and format them on the SD card (to be done prior to what is described above is explained at http://myyafa.blogspot.be/2014/05/create-and-format-partitions-on-yun-sd.html

Sunday, November 10, 2013

Mail body parsing

Now that I can receive emails, the next step will be to parse the mail body.
Although I have no Python experience it seems to be well supported by the Yun, so I will try that road...

The mail body should be an xml document. For now, I will just try to parse a mail body that contains:

<yafadoc>
<temp> 22.3 </temp>
</yafadoc>

I will not worry about error handling, etc.... that is an important aspect, but for later.

I am going to use ElementTree for parsing. When I tried to build a tree from my body string as follows:

tree = ET.ElementTree(ET.fromstring(body))

I got the error:

ImportError: No module named expat; use SimpleXMLTreeBuilder instead

I solved this by:

opkg update
opkg install python-expat

Sunday, November 3, 2013

Getting the arduino Yun ready to receive and read google mails - part5

Now the procmail part. For now I just want all mails sent to me@yafasomething.com to come in a certain file. Again I want all my files on the SD card.

on the SD card create folders (in arduino):

mail.
mail/mailboxes/
mail/mailboxes/yafa/
mail/mailboxes/default

create the file mail/procmailrc containing:

SHELL=/bin/sh
MAILDIR=/mnt/sda1/arduino/mail/mailboxes
DEFAULT=$MAILDIR/default
LOGFILE=$MAILDIR/procmail.log


:0:/mnt/sda1/arduino/mail/procmail.lock
* ^TOme@yafasomething.com
| cat - > yafa/yafa_msg.txt

you can test procmail by e.g.

procmail ./procmailrc < testmail2.txt

where testmail2.txt contains e.g.

From: me@example.com
To: me@yafasomething.com
X-info: I'm just testing

BODY OF MESSAGE SEPARATED BY EMPTY LINE
txt txt txt txt txt txt txt txt txt txt

Now in our fetchmailrc change the procmail line to:

mda "/usr/bin/procmail /mnt/sda1/arduino/mail/procmailrc"

Now if a mail is sent to me@yafasomething.com and we run fetchmail -vk the message should arrive in the specified yafa_msg.txt file. To avoid procmail errors I also created the folder /var/spool/mail, but it is not the goal that something arrives in here.

This topic is continued here.

Useful links & libraries

Mail reading:

http://partmaps.org/era/procmail/quickref.html

http://fetchmail.berlios.de/fetchmail-man.html

http://wiki.openwrt.org/doc/howto/wget-ssl-certs

http://pm-doc.sourceforge.net/doc/#dry_run_testing

Temperature sensor (DS18B20):

http://www.pjrc.com/teensy/td_libs_OneWire.html (library here)

http://milesburton.com/Main_Page?title=Dallas_Temperature_Control_Library (library 372beta here)

Getting the arduino Yun ready to receive and read google mails - part4

Time to work on the ssl certificates

openssl s_client -connect pop.gmail.com:995 -showcerts

returns

CONNECTED(00000003)
depth=2 C = US, O = GeoTrust Inc., CN = GeoTrust Global CA
verify error:num=20:unable to get local issuer certificate
verify return:0
---
Certificate chain
 0 s:/C=US/ST=California/L=Mountain View/O=Google Inc/CN=pop.gmail.com
   i:/C=US/O=Google Inc/CN=Google Internet Authority G2
-----BEGIN CERTIFICATE-----
MIIEdDCCA1ygAwIBAgIICnqidmpojoAwDQYJKoZIhvcNAQEFBQAwSTELMAkGA1UE
BhMCVVMxEzARBgNVBAoTCkdvb2dsZSBJbmMxJTAjBgNVBAMTHEdvb2dsZSBJbnRl
cm5ldCBBdXRob3JpdHkgRzIwHhcNMTMwOTEwMDc1NzM5WhcNMTQwOTEwMDc1NzM5
WjBnMQswCQYDVQQGEwJVUzETMBEGA1UECAwKQ2FsaWZvcm5pYTEWMBQGA1UEBwwN
TW91bnRhaW4gVmlldzETMBEGA1UECgwKR29vZ2xlIEluYzEWMBQGA1UEAwwNcG9w
LmdtYWlsLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMTnolk6
Aqy4DpU20E++g1VGBVKiypiIpTEytbYmzTkI59gyKuSu8psO1Cto4STQ+0vBZaAa
+I71tE1QsUCBTibOubv0g9FQlYhc/3yuC8+0fwqhfgT0Cx81qxuBUP2RLUKkXZ/3
EPauychxFFALnwwRoyuJDE+QdkCgsVJa5d2y34aT3ojhPTAswm1SUsdcU041LkEY
UM5kWFlYC5fTd4IQXoBDkzd7p6Danbf5XLVog/GcYqxIBWVhPkbTLqpB5Coil1TD
G6K2aLa+ivniQvk7B95K+HRAcXkEcolIgwJ7l+vYatVrZqTlMgL6m6JTG7+AwmZG
yvXkHUTMw8vtinMCAwEAAaOCAUAwggE8MB0GA1UdJQQWMBQGCCsGAQUFBwMBBggr
BgEFBQcDAjAYBgNVHREEETAPgg1wb3AuZ21haWwuY29tMGgGCCsGAQUFBwEBBFww
WjArBggrBgEFBQcwAoYfaHR0cDovL3BraS5nb29nbGUuY29tL0dJQUcyLmNydDAr
BggrBgEFBQcwAYYfaHR0cDovL2NsaWVudHMxLmdvb2dsZS5jb20vb2NzcDAdBgNV
HQ4EFgQUayhrKRpwxMMZ48nhS2Owk1m2UB8wDAYDVR0TAQH/BAIwADAfBgNVHSME
GDAWgBRK3QYWG7z2aLV29YG2u2IaulqBLzAXBgNVHSAEEDAOMAwGCisGAQQB1nkC
BQEwMAYDVR0fBCkwJzAloCOgIYYfaHR0cDovL3BraS5nb29nbGUuY29tL0dJQUcy
LmNybDANBgkqhkiG9w0BAQUFAAOCAQEAUTIgKAlAHYOFNTPI8+zRWFUZDqpg2nA0
oUprmReIq94DZA9oOG87glHZXfkzcjXb2QPRFqnIZIQFizWUM9JKRvNsmWT/qoB6
pqBWTHOM1x1DUDGdl9V9068H8BuX4gIZSiWwINAJCTcmpr+01dPafO9Fti69j3Oo
O64AeRQsYQM9eWcUGE8PlGoHK0SeOWHVXhhA9KOeoGzG9IcpihuXTt1xQFIOiW6I
pP1RDJJabLDXB61w8V34w8+B9SfsL9vmpbnsesOkW0Hs4E3tVCXusGhHbUw5Sz0i
3Ut5JUDcUUXsaafv+WSYJjEU4Zq6yCVoHAJ192YZA8LvMwZ3S2U5TQ==
-----END CERTIFICATE-----
 1 s:/C=US/O=Google Inc/CN=Google Internet Authority G2
   i:/C=US/O=GeoTrust Inc./CN=GeoTrust Global CA
-----BEGIN CERTIFICATE-----
MIIEBDCCAuygAwIBAgIDAjppMA0GCSqGSIb3DQEBBQUAMEIxCzAJBgNVBAYTAlVT
MRYwFAYDVQQKEw1HZW9UcnVzdCBJbmMuMRswGQYDVQQDExJHZW9UcnVzdCBHbG9i
YWwgQ0EwHhcNMTMwNDA1MTUxNTU1WhcNMTUwNDA0MTUxNTU1WjBJMQswCQYDVQQG
EwJVUzETMBEGA1UEChMKR29vZ2xlIEluYzElMCMGA1UEAxMcR29vZ2xlIEludGVy
bmV0IEF1dGhvcml0eSBHMjCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB
AJwqBHdc2FCROgajguDYUEi8iT/xGXAaiEZ+4I/F8YnOIe5a/mENtzJEiaB0C1NP
VaTOgmKV7utZX8bhBYASxF6UP7xbSDj0U/ck5vuR6RXEz/RTDfRK/J9U3n2+oGtv
h8DQUB8oMANA2ghzUWx//zo8pzcGjr1LEQTrfSTe5vn8MXH7lNVg8y5Kr0LSy+rE
ahqyzFPdFUuLH8gZYR/Nnag+YyuENWllhMgZxUYi+FOVvuOAShDGKuy6lyARxzmZ
EASg8GF6lSWMTlJ14rbtCMoU/M4iarNOz0YDl5cDfsCx3nuvRTPPuj5xt970JSXC
DTWJnZ37DhF5iR43xa+OcmkCAwEAAaOB+zCB+DAfBgNVHSMEGDAWgBTAephojYn7
qwVkDBF9qn1luMrMTjAdBgNVHQ4EFgQUSt0GFhu89mi1dvWBtrtiGrpagS8wEgYD
VR0TAQH/BAgwBgEB/wIBADAOBgNVHQ8BAf8EBAMCAQYwOgYDVR0fBDMwMTAvoC2g
K4YpaHR0cDovL2NybC5nZW90cnVzdC5jb20vY3Jscy9ndGdsb2JhbC5jcmwwPQYI
KwYBBQUHAQEEMTAvMC0GCCsGAQUFBzABhiFodHRwOi8vZ3RnbG9iYWwtb2NzcC5n
ZW90cnVzdC5jb20wFwYDVR0gBBAwDjAMBgorBgEEAdZ5AgUBMA0GCSqGSIb3DQEB
BQUAA4IBAQA21waAESetKhSbOHezI6B1WLuxfoNCunLaHtiONgaX4PCVOzf9G0JY
/iLIa704XtE7JW4S615ndkZAkNoUyHgN7ZVm2o6Gb4ChulYylYbc3GrKBIxbf/a/
zG+FA1jDaFETzf3I93k9mTXwVqO94FntT0QJo544evZG0R0SnU++0ED8Vf4GXjza
HFa9llF7b1cq26KqltyMdMKVvvBulRP/F/A8rLIQjcxz++iPAsbw+zOzlTvjwsto
WHPbqCRiOwY1nQ2pM714A5AuTHhdUDqB1O6gyHA43LL5Z/qHQF1hwFGPa4NrzQU6
yuGnBXj8ytqU0CwIPX4WecigUCAkVDNx
-----END CERTIFICATE-----
 2 s:/C=US/O=GeoTrust Inc./CN=GeoTrust Global CA
   i:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority
-----BEGIN CERTIFICATE-----
MIIDfTCCAuagAwIBAgIDErvmMA0GCSqGSIb3DQEBBQUAME4xCzAJBgNVBAYTAlVT
MRAwDgYDVQQKEwdFcXVpZmF4MS0wKwYDVQQLEyRFcXVpZmF4IFNlY3VyZSBDZXJ0
aWZpY2F0ZSBBdXRob3JpdHkwHhcNMDIwNTIxMDQwMDAwWhcNMTgwODIxMDQwMDAw
WjBCMQswCQYDVQQGEwJVUzEWMBQGA1UEChMNR2VvVHJ1c3QgSW5jLjEbMBkGA1UE
AxMSR2VvVHJ1c3QgR2xvYmFsIENBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB
CgKCAQEA2swYYzD99BcjGlZ+W988bDjkcbd4kdS8odhM+KhDtgPpTSEHCIjaWC9m
OSm9BXiLnTjoBbdqfnGk5sRgprDvgOSJKA+eJdbtg/OtppHHmMlCGDUUna2YRpIu
T8rxh0PBFpVXLVDviS2Aelet8u5fa9IAjbkU+BQVNdnARqN7csiRv8lVK83Qlz6c
JmTM386DGXHKTubU1XupGc1V3sjs0l44U+VcT4wt/lAjNvxm5suOpDkZALeVAjmR
Cw7+OC7RHQWa9k0+bw8HHa8sHo9gOeL6NlMTOdReJivbPagUvTLrGAMoUgRx5asz
PeE4uwc2hGKceeoWMPRfwCvocWvk+QIDAQABo4HwMIHtMB8GA1UdIwQYMBaAFEjm
aPkr0rKV10fYIyAQTzOYkJ/UMB0GA1UdDgQWBBTAephojYn7qwVkDBF9qn1luMrM
TjAPBgNVHRMBAf8EBTADAQH/MA4GA1UdDwEB/wQEAwIBBjA6BgNVHR8EMzAxMC+g
LaArhilodHRwOi8vY3JsLmdlb3RydXN0LmNvbS9jcmxzL3NlY3VyZWNhLmNybDBO
BgNVHSAERzBFMEMGBFUdIAAwOzA5BggrBgEFBQcCARYtaHR0cHM6Ly93d3cuZ2Vv
dHJ1c3QuY29tL3Jlc291cmNlcy9yZXBvc2l0b3J5MA0GCSqGSIb3DQEBBQUAA4GB
AHbhEm5OSxYShjAGsoEIz/AIx8dxfmbuwu3UOx//8PDITtZDOLC5MH0Y0FWDomrL
NhGc6Ehmo21/uBPUR/6LWlxz/K7ZGzIZOKuXNBSqltLroxwUCEm2u+WR74M26x1W
b8ravHNjkOR/ez4iyz0H7V84dJzjA1BOoa+Y7mHyhD8S
-----END CERTIFICATE-----
---
.
.
.
    Start Time: 1383482982
    Timeout   : 300 (sec)
    Verify return code: 20 (unable to get local issuer certificate)
---
+OK Gpop ready for requests from ....

Looks like we need 3 certificates...let's make a directory in our fetchmail folder:

mkdir certs

Then get the root certificates from http://www.geotrust.com/resources/root-certificates/index.html
We need:

Root 1 - Equifax Secure Certificate Authority
Root 2 - GeoTrust Global CA

You can download them to your certs folder, or just copy paste the content (e.g. from a browser) in the files. So in the certs folder we will have a file called Equifax_Secure_Certificate_Authority.pem with as content:

Do the same for the GeoTrust_Global_CA.pem file.
Finally create a pop.gmail.com.pem with as content:

Then it is time to make the hashes. I have seen typically there is a c_rehash command, but I don't find it for Linino/OpenWrt. we can do this manually:

openssl x509 -hash -noout -in pop.gmail.com.pem

This returns a number (09cc127e). Make a link using this number (adding .0):

ln -s pop.gmail.com.pem 09cc127e.0

Do the same for the other 2. Now add to the fetchmailrc file the location of our certs:

sslcertpath /mnt/sda1/arduino/fetchmail/certs/

If we now run fetchmail -vk we get something like:

fetchmail: WARNING: Running as root is discouraged.
fetchmail: 6.3.26 querying pop.gmail.com (protocol POP3) at Sun, 03 Nov 2013 16:52:40 +0100 (CET): poll started
Trying to connect to 173.194.70.108/995...connected.
fetchmail: Server certificate:
fetchmail: Issuer Organization: Google Inc
fetchmail: Issuer CommonName: Google Internet Authority G2
fetchmail: Subject CommonName: pop.gmail.com
fetchmail: Subject Alternative Name: pop.gmail.com
fetchmail: pop.gmail.com key fingerprint: 91:63:CF:6F:DF:03:3C:B9:3E:19:3B:1C:B6:EB:3C:25
fetchmail: POP3< +OK Gpop ready for requests from 91.179.156.171 n2pf31788823eef.22
fetchmail: POP3> CAPA
fetchmail: POP3< +OK Capability list follows
fetchmail: POP3< USER
fetchmail: POP3< RESP-CODES
fetchmail: POP3< EXPIRE 0
fetchmail: POP3< LOGIN-DELAY 300
fetchmail: POP3< TOP
fetchmail: POP3< UIDL
fetchmail: POP3< X-GOOGLE-VERHOEVEN
fetchmail: POP3< X-GOOGLE-RICO
fetchmail: POP3< .
fetchmail: POP3> USER xxxxxx
fetchmail: POP3< +OK send PASS
fetchmail: POP3> PASS *
fetchmail: POP3< +OK Welcome.
fetchmail: POP3> STAT
fetchmail: POP3< +OK 0 0
fetchmail: No mail for xxxxxxxx at pop.gmail.com
fetchmail: POP3> QUIT
fetchmail: POP3< +OK Farewell.
fetchmail: 6.3.26 querying pop.gmail.com (protocol POP3) at Sun, 03 Nov 2013 16:52:41 +0100 (CET): poll completed
fetchmail: normal termination, status 1

In this case there was no mail (status 1).

Ideas

Ideas to have a look at:

use MQTT
init script on Linino
use xmpp?

TODO list:

procmail: better handling of mail that is not directed to the right address (if any)
procmail: handle >1 arrival of mail (maybe already ok)
xml parsing in python: look into SAX
in general: 'error handling'
check if both sides + bridge are alive: if not reset
linino checks if mcu is still ok: if not reset-mcu
minimize #writes, also on SD card (fetchmail, procmail, log/lock)
look deeper into DallasTempRead:
- avoid eeprom writes, etc...
using dtostrf to convert float to string. Still wondering about exact buffer size needed. For now on safe side.
use watchdog
use google feed to detect if new mail has arrived (instead of checkign every 5 min or so)

Saturday, November 2, 2013

Getting the arduino Yun ready to receive and read google mails - part3

I changed to users homedir to reside on the micro-SD card. In /etc/passwd, just add /mnt/sda1/arduino/ in front of the existing path for the self-created users. Obviously also move the existing user dirs to that location.

A bit later I discovered that there isn't good support for running sketches as non-root on the Yun yet.
Furthermore to be able to have user owned directories on the SD card, the SD card should be formatted in Linux format. For now I am giving up on running Yafa as a normal user. I'll do things as root for now.

However I want to avoid fetchmail writing in the /root folder, so I want all my fetchmail stuff to reside on the SD card. Because the .fetchmailrc needs specific permissions, I need to format the SD card in linux format anyway:

opkg update
opkg install e2fsprogs
opkg install mount-utils
umount /dev/sda1
mkfs.ext4 /dev/sda1

I didn't succeed in mounting it again. A reboot of the Yun took care of that.

cd /dev/sda1
mkdir arduino
cd arduino
mkdir fetchmail
cd fetchmail
touch fetchmail.log
touch fetchmailrc
chmod 600 fetchmailrc

In fetchtmailrc I have put:

set idfile /mnt/sda1/arduino/fetchmail/fetchids
#set logfile /mnt/sda1/arduino/fetchmail/fetchmail.log
set pidfile /mnt/sda1/arduino/fetchmail/fetchmail.pid
poll pop.gmail.com
proto pop3
user "me@yafasomething.com"
password "mypassword"    
mda "/usr/bin/procmail -d %T"
ssl
sslcertck

I could run fetchmail with the -f option, but it is easier to set an environment variable, which will also make sure the SD card is used (could probably remove the first lines from fetchmailrc) add to /etc/profile the following line (and execute it or log in again):

export FETCHMAILHOME="/mnt/sda1/arduino/fetchmail/"

Note that when running fetchmail with the 2nd line not commented out, the messages do not come on the screen anymore, but come in the log-file. For now I want them to come to standard out.

Friday, November 1, 2013

Getting the arduino Yun ready to receive and read google mails - part2

I noticed that the first 'return message' that I got from fetchmail is:

gethostbyname failed for Arduino
Name or service not knownCannot find my own host in hosts database to qualify it!
Trying to continue with unqualified hostname.
DO NOT report broken Received: headers, HELO/EHLO lines or similar problems!
DO repair your /etc/hosts, DNS, NIS or LDAP instead.

Arduino is the hostname of my Arduino Yun (it is the name you can choose on the Arduino's web-interface). For now I got this message to disappear by changing the content of /etc/hosts from

127.0.0.1 localhost

127.0.0.1 localhost
127.0.1.1 Arduino

fetchmail also suggested to add sslcertck to the setup (I think this is more strict). So I did add this to the .fetchmailrc file.