Category Archives: Meetups/Conventions

Track-a-Watt – IoT to the Database: code walkthrough

This is a companion post to my Track-a-Watt – IoT to the Database presentation.

If I missed you at GLOC 2017, you can still catch it at KScope 2017 or OpenWest 2017.

I’ve packed loads of stuff into this presentation, including: soldering (no software involved), Python, Javascript, HTML, PL/SQL and a little SQL (there has to be at least a little SQL in any application! :-)).

Even if I had a few hours of presentation time, it’d be hard to do justice to all these different scripts in their different languages, without losing lots of my audience somewhere along the way. So the presentation keeps things brief and to the point, and I will use this post to provide more depth for some of the code sections.

Python modules

sensorhistory.py

I mention that there are some names and labels used in this module that reference “5 minutes”.

I didn’t find any instances where a value for 5 minutes (300 seconds) is used in the functionality.  Five minutes is only used as labels and object names.

The declarations for these can be found on lines:

  • 103 – cumulative5mwatthr.
    A variable used to store the cumulative watts per hour readings since the timer was started.  We’ll call this total-watt-hours below.
  • 105 – fiveminutetimer.
    A variable used to store the time when the timer was initialized.  We’ll call this start-time below.
  • 119 – reset5mintimer.
    A function to reset start-time and total-watts.
  • 123 – avgwattover5min.
    A function that prints the current data and returns the calculated average watts per hour since the timer started.
  • 124 – fivetimer.
    A text label in the print statement.
  • 125 – 5mintimer and 5minwatthr
    Labels in the text returned by the __str__ function.

This is just a demo, so I didn’t rename these objects.  I only highlight these in case the names cause confusion after I change the timer to 10 seconds.

xbee.py

I only made one change in this module due to an error I received.  I have been running this project on both Windows 7 and Fedora 25 machines.  On one machine the values for p are passed in as Unicode and the other they are Strings.

The change here just checks to see if p is a String if so, convert it to Unicode otherwise accept it as is.  Thanks, Anthony Tuininga for making this clean and compact.

wattcher.py to wattcher-min.py

The original code for the Tweet-a-Watt project has some functionality that I don’t intend to use for my simple graph.  I created the wattcher-min.py module by stripping out most of these features.

Average Watts/Hour calculation

As far as I can prove with my (cough cough) math skills, the algorithm used to calculate watts per hour works for whatever time slice you want to track.

I have not gone through all of the code that leads up to this point, but as I understand it:

  • The kill-o-watt is collecting a constant stream of readings.
  • The kill-o-watt X-Bee transmits the reading to the computer every 2 seconds where the data is stored in the array, wattdata[].
  • This code calculates and stores the average watts used in the last second.
To calculate the average W/Hr during our current time slice:

  • Calculate the number of seconds since the last reading.
  • Multiply the average watts per second by the elapsed seconds then divide by 3600 (seconds in an hour).
  • Reset the last reading timer.
  • Print the data.
  • Add the calculated average W/Hr for this time slice to the running total.
Here’s a basic explanation:

When a chunk of data comes in, we calculate the average W/Hr for the first second of that chunk.  Multiply that value by the number of seconds since the previous reading.  This gives us the average W/Hr for a 2 second time slice.  If we were to collect those slices for one hour and add them together we would have X watts used in one hour.

The cumulative watts used will continue to accrue until we pass the limit of the timer we’re using to determine how often to send the data up to ORDS.

To calculate the average W/Hr during the last 10 seconds:

  • Multiply the cumulative watts used by 3600 (seconds in an hour).
  • Divide by the seconds since the last time we sent data to ORDS.
The short explanation is if we were getting a consistent reading of 5 watts per hour for every sample, every 10 seconds this calculation would come out to 5 W/Hr during the last 10 seconds.  However, it’s not likely that we will get the same 5 W/Hr every reading so this function will give us the average W/Hr during the last 10 seconds.

I can understand if you’re a bit confused at this point. There seem to be a couple extra steps here than what should be needed for my simple graph.  I had to work out a simulation in a spreadsheet before I could accept that it was working.  However, I left the calculation code alone assuming that it may be needed for some of the more advanced versions of the project.

If your math skills are better than mine and you find that my explanation is wrong or you can explain it better, please leave a comment.

Oracle Jet

The Oracle Jet graph used in the example is the basic Line with Area Chart.  I’m using the Y axis for the W/Hr data and the X axis for the timestamps.

The graph has the capability to track multiple series of data which would be useful for multiple kill-a-watts, but I’m only using one in the presentation.

The relationship between the X and Y axises is positional using the array position for the data elements in two arrays.

JavaScript

This is a typical jQuery ajax GET function.

Inside the success function:

  • Get the items array from the response.
  • Create a variable for the X-axis data.
  • Create a variable for the Y-axis data.  Since we’re only tracking one sensor we can define the name value and initialize an items array for it.
  • Loop through the response items.
  • Populate the items array for our sensor (Y axis).
  • Populate the timestamp array (X axis).
  • Set the ko.observable objects for the two axises of the graph.

Next is a short function to call getData() every 5 seconds.

HTML

We copy the HTML from the cookbook for just the graph component.

Since we’re not using the additional functionality from the Jet Cookbook example we remove the highlighted lines (14, 15).

Go try something new

The goal of this presentation is to encourage people to go out and try something a little out of their comfort zone.  If you think your soldering skills are lacking find a maker group in your area and take a class.  If you are strong in one programming language try another.

This is a great project to experiment with, there are a few different skills all mixed together, each of them is fairly close to entry level and they are popular enough that there should be a lot of help available.

As always, if you run into issues feel free to leave a comment here or hit me up on twitter and I’ll be glad to help get you going.

I plan to update this post as questions arise.  If you’d like to see it all running together catch one of my upcoming sessions.

Leave your comfort zone behind: head to an open conference

If you are a(n) [insert language here] programmer, you probably attend a(n) [insert language here] conference when you’re able.  That’s great, since conferences are an awesome way to improve your skills in [insert language here] .

However, I also recommend that you try to attend an “open (source, hardware, data, ….)” conference if and when you can.  The broader the scope, the better.  Odds are, there will still be something related to your [insert language here] skills that you will find useful (and can use as justification to attend, if need be) but the real value comes from semi-random association with new stuff. Stuff you’ve never seen before, maybe never thought about before.

Recently, I attended OpenWest and of course there were plenty of sessions on technologies I use regularly (NodeJS, Python, Raspberry Pi,  Docker,  Privacy/Security,  and a ton of stuff I don’t know how to do …. yet).  I attended a few of these and learned some new tricks.  I also attended some sessions that were out of my comfort zone.  I learned a bit about Bitcoin and Block Chain, tried to keep up with some Linux server discussions, was completely out of my league listening to a session on Recent Advances in Microcontrollers and, after struggling through some basic “hacker” challenges, I got to dust off my soldering skills and make a cool LED badge.

Why does this matter?  Diversity is the key to sustained growth.  This famously applies to your financial investments, but it’s also important to your growth as a technologist.  If you continue to do the same/similar thing over and over, with the same technology, you will undoubtedly get really good at it.  The potential downside, however, is that you may be perceived as that person who only has a hammer, so everything looks like a nail.

If you enjoy being the go-to person when there’s a problem to be solved in technology X and no expert in X in sight, you need to keep adding to your bag of tricks. You need to keep challenging yourself.

This doesn’t mean that you have to have all of the answers. Sometimes it’s enough to know that a better solution exists “out there” somewhere, and help get the team pointed in the right direction.

Before going to OpenWest, I had experimented with my Raspberry Pi, but only in software ways like OSMC and OpenHab.  I do have an Arduino and I’ve read through a few projects that look fun, but I’ve hesitated to get started, assuming that it’s going to take too much time to ramp up my skills and I’ll probably burn it up unless I practice my soldering skills…. a lot.

Now, I’ve learned that the hardware components are more rugged than I assumed and I’m not going to burn them up with a minor solder error and once again I’m experiencing with how much fun it is to be the “new kid” learning cool, new things.

It is definitely time to go buy a good soldering iron.

Collaborate 15

Last week I had the privileged to attend Collaborate 15 in Las Vegas.

The facilities at the Mandalay Bay were top notch.  The food was very good and there were plenty of quiet comfortable places where you could catch your breath, relax and catch up.

I was only able to attend for a couple days at the end, but I still learned a lot and made a few great connections. I came away with new useful information from every session I attended, even the one I jumped into late just for a place to sit down. A couple of my favorite sessions had lively discussions that were focused, on topic and helpful.

I think most people prefer substance over flash when they chose a conference, at least I know I do, but some flash is fun.  I really enjoyed the mix they put together. The floor exhibits were interesting and there were a lot of them.  The 60s party was a blast and the band was awesome. Just when I would reach the “my brain is full” point, there was something shiny to focus on while my mind absorbed the info and I could relax before loading up the next chunk of information.

Overall I thought it was a great conference and I would definitely recommend it.