Live Streaming Video With Raspberry Pi

IMG_0320

Much to my delight, I discovered that a pair of pigeons are nesting outside of my window. I decided to set up a live streaming webcam so I can watch the young pigeons hatch without disturbing the family. Instead of buying an off-the-shelf streaming solution, I used a Raspberry Pi and a USB webcam. Here is how I set up live streaming video using my Pi and Ustream.

For this project, I used a Raspberry Pi Model B+, a USB WiFi adapter, a microSD card, a USB webcam and a 5 volt power adapter. When selecting a USB webcam, try to get something on the list of USB webcams known to work with Raspberry Pi. It will save you a lot of headaches in the long run!

IMG_0319

To start, download the latest Raspbian image and load it onto the SD card. My favorite tool for doing this on a Mac is Pi Filler. It’s no-frills, easy to use and free! It may help to connect the Pi to a monitor and keyboard when first setting it up. Once the Pi first comes up, you will be prompted to set it up using raspi-config. At this time, it’s a good idea to expand the image to use the full card space and set the internationalization options to your locale so that your keyboard works properly.

Once the Raspberry Pi boots up, there are a few things that need to be updated and installed. First, it’s a good idea to update the Raspbian image with the latest software. I also like to install webcam software, fswebcam, so I can test that the webcam works before setting up video streaming. Finally, you’ll need ffmpeg, which is software capable of streaming video. The following commands will set up the Raspberry Pi:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install fswebcam
sudo apt-get install ffmpeg

After installing the software, it’s a good idea to check whether the webcam works with the Raspberry Pi. To do this, simply take a picture with the webcam using fswebcam. This will attempt to take a single photo from the webcam. You can do this by running the following command:

fswebcam photo.jpg

If the photo looks good, then you are ready to set up streaming video. First, set up a Ustream account. I set up a free account which works well despite all of the ads.  Once you set up your video channel, you will need the RTMP URL and stream key for the channel. These can be found in Dashboard > Channel > Broadcast Settings > Encoder Settings.

Next, set up video streaming on the Raspberry Pi. To do this, I used avconv. The documentation for avconv is very dense and there are tons of options to read through. I found this blog post which helped me get started. I then made some adjustments, such as using full resolution video, adjusting the frame rate to 10 frames per second to help with buffering issues and setting the log level to quiet as to not fill the SD card with logs. I also disabled audio recording so I wouldn’t stream the laments of my cat for not being allowed to ogle the pigeons. I wrote this control script for my streaming service:

#!/bin/bash 

case "$1" in 
   start)
      echo "Starting ustream"
      avconv -f video4linux2 -r 10 -i /dev/video0 - pix_fmt yuv420p -r 10 -f flv -an -loglevel quiet <YOUR RTMP URL>/<YOUR STREAM KEY> &
      ;;
   stop)
      echo "Stopping ustream"
      killall avconv
      ;;
   *)
      echo "Usage: ustream [start|stop]"
      exit 1
      ;;
esac

exit 0

Make sure the permissions of your control script are set to executable. You can then use the script to start and stop your streaming service. Before placing the webcam, it’s a good idea to see if you need to make any additional updates to the Raspberry Pi for your webcam to work. The webcam I chose, a Logitech C270, also required some modprobe commands to keep from freezing. Finally, it’s a good idea to add your control script to /etc/rc.local so that the streaming service automatically starts in case your Raspberry Pi accidentally gets rebooted.

And that’s it! There is a multiple second delay to the streaming service so within a minute you should see live streaming video on Ustream. One word of caution on working with the Raspberry Pi: be sure to shut down the Raspbian operating system before unplugging the Raspberry Pi. The SD card can become corrupted by just unplugging it. This will cause the operating system to go into kernel panic and refuse to boot.  Sadly, the only solution for this is to reinstall Raspbian and start all over again.

Once my webcam was up, I found that I had some issues positioning the camera effectively. To solve this, I bought a cheap mini camera tripod. I then dismantled the clip of my webcam and drilled a 1/4″ hole in the plastic so it would fit on the tripod. I put a 1/4″-20 nut on the top of the screw and I was good to go!

IMG_0330

I will be live streaming the pigeon nest for the next month or so on this Ustream channel. I’ve learned a lot about pigeons by watching them every day. The squabs should hatch during the upcoming week and I am excited to watch them grow!

IMG_0333

Master of Science

Things have been quiet on the project front recently as I have been busy finishing up one of my largest pursuits to date: a master’s degree in Computer Science from Courant Institute of Mathematical Sciences at New York University. I completed my degree part-time while working a full-time engineering job. It took me ten semesters to complete, which roughly translates to four academic years.

The quality of the education at NYU Courant was mostly good. I had some excellent professors who were experts in their respective fields. A few of my favorite classes were Realtime and Big Data Analytics, Operating Systems, and Statistical Natural Language Processing. Sadly, there were also a few classes that had some room for improvement. Some of my worse experiences included poorly organized professors and incredibly bland or irrelevant lectures. Despite those flaws, I felt that overall the program was challenging and interesting.

I met many fantastic people while getting my degree. During my time there I was able to be involved with NYU’s Women in Computing (WinC) group. WinC enabled me to be a part of a community of other women computer science students at all levels. I even gave a few talks on behalf of WinC about my experiences of being a woman in engineering, such as at the NYC Girls Computer Science and Engineering Conference at NYU and at the Women Chartering Technical Career Paths event at the Apple Store in SoHo.

Speaking at the Women Charting Technical Career Paths event at the Apple Store in SoHo
Speaking at the Women Charting Technical Career Paths event at the Apple Store in SoHo

So is it worthwhile to get a master’s degree? There are three things to consider when deciding whether to pursue graduate school: the value of the degree, the financial cost, and the time investment. First and foremost, it’s important to consider how much value the degree will add to your career. As far as technical skills, there are other ways of gaining the same skill set as an advanced degree. Many courses similar to my master’s program requirements can also be taken online through free class sites like Coursera and Udacity. Furthermore, the software industry tends to be a meritocracy in that your previous work experience can outweigh the name on your diploma. This means that a graduate degree may not add a lot of value if you already have an established career. Even with these considerations, having a master’s degree on your resume can open doors to opportunities that might not otherwise be available. Additionally, many companies prefer candidates with advanced degrees, especially at senior levels. The cost is another factor to be considered. My degree was $58,877, not including any books or materials. The financial price of the degree would have been prohibitively expensive if my company had not helped me pay for it. Finally, it’s important to consider how much time you have to invest in graduate school. Pursuing a degree full-time means that you will most likely not be earning wages for two years whereas a part-time program means that you will have limited free time for multiple years and the additional pressure of a career on top of graduate school. I had vastly underestimated how many weekends and late nights I would spend on class assignments. It meant making a lot of personal sacrifices and sitting inside working while everyone else was playing outside in the sunshine.

Speaking at the NYC Girls Computer Science and Engineering Conference  at NYU
Speaking at the NYC Girls Computer Science and Engineering Conference at NYU

I’ve considered whether it would have been better to work on my master’s degree right after finishing my bachelor’s degree. I think the years of industry experience had served me well in graduate school. My technical skills were more mature when I started my degree and I had a better idea of what topics I wanted to pursue. It would have been nice to have fully dedicated my time to the master’s program but after a few years of work it’s a hard decision to stop working to go to school full-time. Additionally, as my company was paying for my degree, I did not have the option to take time off. All said and done, I’m glad I decided to go part-time for the degree. A number of times my coursework lined up nicely with my professional work and I was able to apply what I had learned directly to my job.

Despite all of the personal sacrifices, I am still happy with my decision to get a master’s degree. It was quite the achievement but I am also happy that it is finally done. I have been learning what it’s like to have free time again and I am starting to tackle my ever-growing project list.

Spark Core

I’ve been spending some time playing with the Spark Core. This device is an open source ARM-based microcontroller with WiFi on-board. It belongs to the Spark OS ecosystem, which aims to be an easy, secure, and scalable solution for connecting devices to graphical interfaces, web services, and other devices. One interesting feature is how you interact with the Spark Core: it has support for mobile devices (iOS or Android), a Web Integrated Development Environment (IDE), and a command line.

The Spark Core devices (also known as “cores”) function in tandem  with the Spark Cloud service (also called the “cloud”) on the internet. The cloud is responsible for managing your cores, developing the core code, and loading applications on your core. Spark Cloud accounts are free and can be created on the Spark build page. Many cores can communicate with each other through a publish/subscribe messaging system made available through the cloud.

IMG_1340The Spark Core comes in a great package. The box promises that “when the internet spills over into the real world, exciting things happen.” Conveniently, the core comes with a breadboard and a micro USB cable right in the box. This all-inclusiveness makes it ideal for beginners. And it even comes with a sticker!

The easiest way to get your core up and running is to use your mobile device. Simply download the Spark mobile application and connect your mobile device to the same network that the core will use. Turn on your core and make sure it is in listening mode.  Next, use your mobile application to log into your cloud account. You will then be prompted for the network credentials to be used by the core. This will begin a search and registration process where the mobile device finds the core, connects it to the network, and registers the core to your cloud account. The RGB LED on the core shows the status of the internet connection. Once your core is online and registered to your account, you are ready to start playing it!

IMG_1341IMG_1344

First, I wanted to try interacting with my core from my mobile device. This can be done using a part of the Spark mobile application called Tinker.

IMG_1346Tinker is more of a prototyping app than it is a dedicated programming environment. It allows you to simulate analog and digital inputs and outputs on the core. Tinker can be integrated with code written for the core so that an application running on your core can interface with the Tinker application on your mobile device.  My experience with Tinker was only so-so as it crashed a number of times on my iPhone 6.

Next, I wanted to try programming my core from the web through the Spark Cloud build website. To do this, I simply logged on to my cloud account which automatically loaded the web IDE. I was curious about how easy it was to import and implement external libraries. To get a feel for this, I tried to connect my core to an LED strip and control it via the Tinker app.

Screen Shot 2014-10-22 at 8.02.45 PMThe web IDE is very clean and easy to use. There are mouse-over tips to help you navigate the environment. The controls (located on the left panel of the IDE) are as follows from top to bottom: flash, verify, save, code, libraries, docs, cores and settings. Double clicking any one of these icons expands and collapses the grey information pane.

The Spark Core language is Arduino compatible as it supports the functions defined in the Arduino language specification. It also includes some extra features that enable you to do things like interact with the network settings and subscribe to specific events from the cloud. Unfortunately, many of the Arduino libraries included in the Arduino IDE have not been implemented for the Spark platform. This may create some problems if you are trying to port your old Arduino code to a core.

Screen Shot 2014-11-05 at 8.58.16 PMIncluding the Adafruit NeoPixel library was very easy. I simply searched the available libraries and clicked the import button for the library I wanted to use. All of the necessary includes were automatically inserted into my code. The library display pane also allowed me to browse and/or import the sample code from the library I selected.

Once my code was complete and verified, I simply clicked the flash button and waited for the cloud to update my core. Success!

IMG_1362Finally, I tried connecting to my core with the Spark Command Line Interface (also called spark-cli). This package is an open source command line tool which uses node.js to program your core. It  works over both WiFi and USB (which is handy when the network is unavailable). The spark-cli tool is not packaged well and was a little tricky to install. After installing node.js, I kept getting compile failures. After some digging I finally got it to work by opening XCode and accepting some license agreements.

The spark-cli tool allows you to interact with your core in a more advanced way. The command line allows you log into the core and read any serial output being generated by the application. It also enables you to manage the application running on a core, such as compiling and uploading new applications or reverting the core to its factory state. Much like Tinker, the spark-cli allows you to simulate both analog and digital input or output. It also enables you to publish and subscribe to events in the cloud so that you can communicate with other cores.

On the hardware front, it is important to note that the internal WiFi chip uses an older version of the 802.11 standard. As the Spark Core uses 802.11b/g, it won’t connect with newer 802.11n networks. I ran into this issue when moving my core between networks. In this case, I had to connect to the core via USB and use a serial connection to enter my network credentials manually. I later discovered that this could also be done via the spark-cli tool.

spark_core_serialStoring all of your code in the Spark Cloud is both a blessing and a curse. Currently, there is no easy way to version your code or to determine what version of a library is available in the web IDE. I fumbled a bit programming the LED strip because I had to dig around to see which version of the NeoPixel library was available. Additionally, Having the code in a private remote location also makes it harder to share code with other people. Because the core is programmed over the internet, it takes longer to program. This can be too time consuming if you are doing rapid iterative development.  On the positive side, remote code storage and programming means that you can easily modify and upload your application to any core from any web browser. This means no more frantic searching for the correct cable, code version, library version and so on.

To give you an idea how the Spark Core stacks up to other ARM-based microcontrollers, I compared it to two other devices in my project box:

Feature Spark Core 1.0 Arduino Due Teensy 3.1
Processor 72 MHz ARM Cortex M3 84 MHz ARM Cortex M3 72 MHz ARM Cortex M4
Memory (Flash) 128KB 512 KB 256 KB
Memory (SRAM) 20 KB 96 KB 64 KB
Voltage 3.3v 3.3v 3.3v
Regulated output voltage 3.3v 3.3v and 5v 3.3v
Cost $39 $50 $20
Size 1.47" x 0.8" 4" x 2.1" 1.4" x 0.7"
Digital pins 18 54 34
Analog pins 8 12 21
5v tolerant input pins 7 0 21
SPI yes yes yes
UART (Tx/Rx) 1 4 3
I2C (SDA/SLC) 1 2 2
JTAG yes yes no
WiFi yes (802.11 b/g) no no
Programming environment Web and Mobile IDE (WiFi), command line (USB or WiFi) Arduino IDE (USB) Arduino IDE + Teensyduino (USB)

The online nature of this device makes it a good choice for people new to Arduino programming. Since the core is internet based, setup is easier than with an Arduino as there are no FTDI drivers to install or serial issues to debug. The RGB LED used for network status is a clever way to assist beginners with debugging connectivity issues. The Spark Core shields are a great starting point for many projects. The Shield Shield makes any Arduino shield compatible with the Spark Core layout, which allows you to take advantage of the large number of Arduino shields already out there. The Spark documentation is very clear and it has a helpful community of users in case you have any questions.

Veteran Arduino programmers can enjoy the advanced features of the Spark OS ecosystem. The distributed nature of the Spark OS makes it simple to connect devices together. The publish/subscribe messaging mechanism allows devices to interact with each other in real time. The RESTful API built into the Spark Cloud makes it easy for any web service to interact with any of your devices on the cloud. On the administrative front, the command line tool gives more power to the user. I was especially pleased that I could use the command line to remotely read the serial output while the core was running.

All in all, I think this is a great board for both beginners and advanced Arduino users. Just like any new device, the Spark Core has some growing pains to work through. Despite that, it offers some great features that make it easy to look past some of the shortcomings.  The on-board WiFi is a real game changer in the hobbyist microcontroller market. I look forward to more internet-enabled projects!

Fiber Optic Flowers

IMG_0080While visiting Québec, Canada, I saw this fantastic installation in Montmorency Park: a field of fiber optic flowers. This project was done to honor the 150th anniversary of the Québec Convention. The field was comprised of over 15,000 flowers which gradually changed color over time. The slight variations of the timing and color of each flower made this installation even more fascinating to watch.

The construction of the flowers was relatively simple. Each flower was composed of a fiber optic cable running through some  clear plastic tubing with a translucent plastic bulb on top.

IMG_9900The real magic came from the illuminated cable inside of the flowers. The fiber optic cable gave each blub a pleasant glow while the glowing cable running through the clear tubing created the subtle illusion of a stem.

IMG_0084It was possible to trace the fiber optic cables back to the light sources. As the sources were covered, it was hard to tell exactly what was powering them. The sources were running some sort of synchronized lighting program so that the large garden would simultaneously change color.

IMG_0079Although each flower was all slightly different, they all came together to make one beautiful glowing blanket covering the entire park. This has definitely inspired me to try to make some of my own fiber optic works of art. IMG_0069

DIY LED Strip Controller

On a whim I decided to add LED lighting to my desk hutch. I already had a reel of LED strips but nothing to control them. As I wanted to build a controller that afternoon, I constructed it from parts which could be purchased locally. The controller I made has an on/off switch and two knobs: one to control the brightness and the other to control the color of the lights. Here is how I built it!

This project required the following parts: two 10K ohm potentiometers with knobs, an on/off switch, a project box, a 5 volt power supply, a power jack, some wire, and a small Arduino compatible microcontroller. RadioShack sells the Arduino Micro, but I used a Teensy 2.0 I had on hand as it is a much cheaper alternative. Of course, you also need some programmable LED strips. I used some 5 volt WS2812B LED strips (similar to Adafruit’s NeoPixel strips). It’s also useful but not required to have some connectors for the LED strips so that they can be detached from the controller. I used some JST connectors from my project stash.

The first step is to make holes in the project box. I did this with my trusty Dremel. Drill five holes: two for the potentiometers, one for the power switch, one for the power jack and a one for where the LED strip wires will enter the project box. Once the box is drilled out, place the power switch, potentiometers and power jack into the project box. Solder the power wires to the components.

IMG_9702Next, I assembled the LED strips. If you are adding connectors to the LED strips, solder those on to the strips. If you’re connecting multiple strips, be sure that you bundle the wires together properly. I’ve found that using colored wire or marking wires with different colors of tape makes it easier to keep everything straight.

IMG_9722Next, solder the potentiometers and LED strips. Mark the data lines for the LED strip and the potentiometers so you know which wire corresponds to a given component. It makes coding the microcontroller easier.

IMG_9761Next, solder the microcontroller.  Keep track of the pins and their corresponding data lines. When soldering the potentiomenter to the microcontroller, make sure to connect the potentiometer data wire to pins that can support the analogRead function. These pins generally begin with the letter ‘A’.

IMG_9767Now it’s time to program the microcontroller. The simple code can be found here. Update the code to reflect the length of your LED strip and the pins that correspond to your components. Be sure to test everything!

IMG_9771Once you’ve verified that everything works, tape up any solder joints so that there are no shorts. Close up the box and you’re done!

IMG_9772