Wednesday, October 30, 2013

Getting the arduino Yun ready to receive and read google mails - part1

I found a lot of examples of how to send mail with the Yun, or how to fetch the google atom feed, but I didn't find any example of how to really receive the full emails with the Yun.

I have a google mail account, so I am focussing only on accessing the google mail (gmail).
Below I will assume my google mail user has the following 'properties':
username:  me@yafasomething.com
password:  mypassword

Disclaimer: I have no experience with setting up all this email stuff, so there might be better ways of doing this. What I did is based on a lot of googling.

I asked on the arduino forum for some pointers. I got a suggestion to look into the google python API's, but I got the impression that this is not for free at some point. So I abandoned that route.
I started reading about linux mail 'clients'. I learned a lot of things. I now know more or less what an MTA, MDA, MUA, etc... is :-)

I had no idea if I should use POP3 or IMAP. I went for POP3 for now. So first thing to do is to enable POP3 in your google account (I assume you know how to do this).

Next steps are on the arduino Yun:

login as root and run the following commands:

opkg update
opkg install openssl-util
opkg install fetchmail
opkg install procmail
opkg install shadow-useradd

This installs the packages that I think I will need.

I wasn't sure if I should eventually run my app as root, or if I can do it as another user.
I will give it a try as non-root. I decided to create a user called MrYafa:

mkdir /home 
useradd -m -c "Mr Yafa" MrYafa
passwd MrYafa

Maybe the first command isn't needed. In the last command you obviously need to choose a password.

Then edit the /etc/passwd file   (e.g vi /etc/passwd - I will assume you know how to work with vi)
You should see the newly created user. At the end of that line add /bin/ash

Now exit as root and try to login as MrYafa:

ssh MrYafa@arduino.local

(on windows you might need to enter the IP address as arduino.local might not work)

You should be able to login an arrive in your home directory.

pwd

should return

/home/MrYafa

now create a file .fetchmailrc in that directory (vi .fetchmailrc) containing:

poll pop.gmail.com
with proto pop3
user "me@yafasomething.com"
there with password "mypassword"     
is "MrYafa" here 
mda "/usr/bin/procmail -d %T"
options
ssl

Now, if you give the command

fetchmail -vk

you should see the communication with your google mail on the screen, ending with (in my case):

fetchmail: POP3> QUIT
fetchmail: POP3< +OK Farewell.
fetchmail: 6.3.26 querying pop.gmail.com (protocol POP3) at Wed, 30 Oct 2013 18:51:33 +0100 (CET): poll completed
fetchmail: normal termination, status 1

One warning is that the connection is insecure. I know that you should use certificates, but haven't figured out yet how to do so. On mu Ubuntu PC I have ca-certificates installed, and there it seems OK. But the ca-certificates package does not seem to be available for Linino. So I need to find another way of making it more secure.

That's it for now. Not so difficult when you know what to do, but it took a while to find that out.

Monday, October 28, 2013

accessing Yun from Linux

ssh root@arduino.local

Tuesday, October 22, 2013

Changing the directory of imported arduino libraries

Standard imported libraries are placed somewhere in /users/Documents/...

This isnt't where want them to reside.

You can change this location by:
File --> Preferences
and changing the sketchbook location. I find this a bit a weird name though...

Monday, October 21, 2013

runShellCommand

I just learned that there exists a runShellCommand as well :-)

The difference with the addParameter is that it does not escape characters. But that is no issue at all.

The clumsy code has just become:


p.runShellCommand("date > /mnt/sda1/arduino/time.txt");

p.runShellCommand("curl -T /mnt/sda1/arduino/time.txt ftp://username:password@webpage/ -a");


Sunday, October 20, 2013

ftp the current time and date to a webpage

The first thing I want to try is uploading something to a webpage. Later this will be processed sensor data. For now I will try to log the current time date.

For the ftp-ing I looked at the pre-installed curl package.
I didn't immediately find how I could pass a string or environment variable to curl, so I decided to put the time and date in a file, and ftp this file to my webpage.

I created a folder arduino on my SD card. Using the terminal I could immediately see the folder /mnt/sda1/arduino. Writing to the file using simple linux commands was a bit more difficult. I read about the process library and its begin/run/addparameter methods. I simply wanted to call

date > /mnt/sda1/arduino/time.txt

I tried several combinations of what strings to put in begin()/run() but didn't manage to get it to work :-(
As alternative I made a simple shell script (go.sh) that does just that:


#!/bin/sh
date > /mnt/sda1/arduino/time.txt
echo "hoi"

Calling this script from the sketch is as simple as:

Process p;        // Create a process and call it "p"
p.begin("/mnt/sda1/arduino/go.sh");
p.run();

Great :-)

Now uploading the file to my webbage. Doing this on the terminal was pretty straightforward, but I again struggled with the begin/addparameter syntax. Eventually I got it to work using the following piece of code:

p.begin("curl");
p.addParameter("-T");
p.addParameter("/mnt/sda1/arduino/time.txt");
p.addParameter("ftp://username:password@my-web-location");
p.addParameter("-a");
p.run();

The -a is to append the file.
I don't know why I had to use 4 addParameter calls though...but OK it works.
Probably I could do something similar for the file creation.

These 2 codes together and a delay of about 2 minutes in between create a file on my webpage looking as follows:

Sun Oct 20 20:40:58 CEST 2013
Sun Oct 20 20:43:04 CEST 2013
Sun Oct 20 20:45:10 CEST 2013
Sun Oct 20 20:47:16 CEST 2013
Sun Oct 20 20:49:21 CEST 2013
Sun Oct 20 20:51:28 CEST 2013
Sun Oct 20 20:53:34 CEST 2013
Sun Oct 20 20:55:40 CEST 2013
Sun Oct 20 20:57:46 CEST 2013
Sun Oct 20 20:59:52 CEST 2013
Sun Oct 20 21:01:58 CEST 2013
Sun Oct 20 21:04:04 CEST 2013
Sun Oct 20 21:06:10 CEST 2013
Sun Oct 20 21:08:15 CEST 2013




First ideas regarding Yun-Internet communication

Challenge: observe and control the setup over internet.

I want to be able to follow the fermentation (temperature, CO2 production, etc...) online. But I also want to be able to control or setup the device remotely.
I do not have a lot of expertise with this, so I will start with something very simple.

My first idea was to run a webserver or so on the Yun and store everything on the SD-card.
But I have no idea how I can do this, especially how to give (safe) internet access of the Yun through my (home) router/firewall, and how to manage my non-static IP-address.
So I will start with something more simple.
I have solar-panels on my house with a Solarlog. This box regularly uploads (via ftp) production numbers to my site. Using any browser (and some Java I believe) you can get graphics about the production.
Something similar is good enough for now.

The reverse (controlling the Yun) looks more difficult. I saw the Yun should be able to read emails. This seems like a good start: to control the Yun you just sent it an email with some configuration file (probably XML). This will allow me to start by just manually sending emails. Later I can easily make a windows or android GUI to create and send the emails more human friendly.

First experiments with my Yun

I actually started with an Arduino Uno. Just to make sure I wasn't destroying my somewhat more expensive Yun with some early trials.

But anyway...hereby a picture of the Yun with a breadboard and some very simple circuitry.


It is USB powered. Bottom right is the SD card that you can use as storage.

Getting started with the Arduino environment and the Yun is pretty simple. The getting started guide is very instructive.

But I did bump into one issue: you can program the Yun via Wifi or over USB. Surprisingly the programming over Wifi worked immediately, but I couldn't get the Yun recognized via the USB port. With the Uno this did work.

Eventually I saw that the drivers were not installed correctly. Quite a bit of googling told me that the combination of Windows7 64bit with USB3.0 and the Arduino Leonardo is problematic. Given the Yun is very similar to the Leonardo I tried installing the drivers using the USB2.0 port of my laptop.
And....success :-)

Saturday, October 19, 2013

A project name

Obviously this project needed a name. Coming up with a good one isn't something I am good at.
I already have a brewsoftware (for calculating recipes) called Yabs. I searched for something similar.
Eventually I stranded at Yafa.

It can stand for Yet Another Fermentation Assistent, or someting like Yun Android Fermentation ...
Or just the obvious slang meaning...


The start...

I like homebrewing, electronics and programming.

One important aspect of brewing is to control the temperature of the fermentation. The temperature has a huge influence on the taste of the produced beer. It is important that you can keep the temperature very stable. Also there are different phases in the fermentation. For example you want to start the fermentation at 20C, near the end of the main fermentation you might want to go to 22C, followed by a couple of days at 5C.

Many homebrewers have some kind of thermostat to control a cooler and a heater. Most solutions are pretty simple (dutch forum, flemish forum). The most advanced solution I have looked at is Henielms's TControl. I actually bought this solution, and I think it works very well. The main components are an IO-board with some relays, software (Labview based) and a CO2 volume sensor.

The IO-board and Labview do not allow me to make changes. Therefor (and just because it is fun to do) I decided to make my own version. I do plan to keep using the CO2 production sensor as I think it works very well.

I want to use as much as possible 'standard/off the shelf' components and free/open-source stuff.
That is how I ended up looking at the Arduino.

Some early project ideas/requirements were:
- minimal 'hardware' work - use off the shelf components
- use free/open source stuff
- measure temperature
- control 2 relays (one for heating, one for cooling)
- work with Henielma's CO2 sensor.
- follow the status from internet
- control over internet, eventually also using my android phone
- simple GUI

Obviously not everything will be there from the start...

Given the above requirements I ended up looking into the Arduino environment.

The Arduino Yun looked like a very good candidate for this project.
The appealing features of the Yun board for me were:
- microcontroller with enough IO
- Wifi on-board
- Linux on-board with support for easy internet access (curl, temboo, ...)
- relatively cheap (around 60 EUR)

From a hardware point of view, I think I only need the Yun, a temperature sensor, the CO2 sensor, a USB power source, 2 relays (plus maybe some resistors, LEDs, ...)

The rest of this blog will present the progress (hopefully) with this project. I also plan to use the blog for myself as a reference and log-book.