I actually started with an Arduino Uno. Just to make sure I wasn't destroying my somewhat more expensive Yun with some early trials.
But anyway...hereby a picture of the Yun with a breadboard and some very simple circuitry.
It is USB powered. Bottom right is the SD card that you can use as storage.
Getting started with the Arduino environment and the Yun is pretty simple. The getting started guide is very instructive.
But I did bump into one issue: you can program the Yun via Wifi or over USB. Surprisingly the programming over Wifi worked immediately, but I couldn't get the Yun recognized via the USB port. With the Uno this did work.
Eventually I saw that the drivers were not installed correctly. Quite a bit of googling told me that the combination of Windows7 64bit with USB3.0 and the Arduino Leonardo is problematic. Given the Yun is very similar to the Leonardo I tried installing the drivers using the USB2.0 port of my laptop.