Track-a-Watt – IoT to the Database: code walkthrough

This is a companion post to my Track-a-Watt – IoT to the Database presentation.

If I missed you at GLOC 2017, you can still catch it at KScope 2017 or OpenWest 2017.

I’ve packed loads of stuff into this presentation, including: soldering (no software involved), Python, Javascript, HTML, PL/SQL and a little SQL (there has to be at least a little SQL in any application! :-)).

Even if I had a few hours of presentation time, it’d be hard to do justice to all these different scripts in their different languages, without losing lots of my audience somewhere along the way. So the presentation keeps things brief and to the point, and I will use this post to provide more depth for some of the code sections.

Python modules

sensorhistory.py

I mention that there are some names and labels used in this module that reference “5 minutes”.

I didn’t find any instances where a value for 5 minutes (300 seconds) is used in the functionality. Five minutes is only used as labels and object names.

The declarations for these can be found on lines:

103 – cumulative5mwatthr.
A variable used to store the cumulative watts per hour readings since the timer was started. We’ll call this total-watt-hours below.
105 – fiveminutetimer.
A variable used to store the time when the timer was initialized. We’ll call this start-time below.
119 – reset5mintimer.
A function to reset start-time and total-watts.
123 – avgwattover5min.
A function that prints the current data and returns the calculated average watts per hour since the timer started.
124 – fivetimer.
A text label in the print statement.
125 – 5mintimer and 5minwatthr
Labels in the text returned by the __str__ function.

This is just a demo, so I didn’t rename these objects. I only highlight these in case the names cause confusion after I change the timer to 10 seconds.

xbee.py

I only made one change in this module due to an error I received. I have been running this project on both Windows 7 and Fedora 25 machines. On one machine the values for p are passed in as Unicode and the other they are Strings.

The change here just checks to see if p is a String if so, convert it to Unicode otherwise accept it as is. Thanks, Anthony Tuininga for making this clean and compact.

def init_with_packet(self, p):
    #   p = [ord(c) for c in p]
        p = [ord(c) for c in p] if isinstance(p, str) else [c for c in p]

def init_with_packet(self, p):

# p = [ord(c) for c in p]

p = [ord(c) for c in p] if isinstance(p, str) else [c for c in p]

wattcher.py to wattcher-min.py

The original code for the Tweet-a-Watt project has some functionality that I don’t intend to use for my simple graph. I created the wattcher-min.py module by stripping out most of these features.

Average Watts/Hour calculation

As far as I can prove with my (cough cough) math skills, the algorithm used to calculate watts per hour works for whatever time slice you want to track.

I have not gone through all of the code that leads up to this point, but as I understand it:

The kill-o-watt is collecting a constant stream of readings.
The kill-o-watt X-Bee transmits the reading to the computer every 2 seconds where the data is stored in the array, wattdata[].
This code calculates and stores the average watts used in the last second.

# sum up power drawn over one 1/60hz cycle
avgwatt = 0
# 16.6 samples per second, one cycle = ~17 samples
for i in range(17):
    avgwatt += abs(wattdata[i])
avgwatt /= 17.0

127

128

129

130

131

132

# sum up power drawn over one 1/60hz cycle

avgwatt = 0

# 16.6 samples per second, one cycle = ~17 samples

for i in range(17):

avgwatt += abs(wattdata[i])

avgwatt /= 17.0

To calculate the average W/Hr during our current time slice:

Calculate the number of seconds since the last reading.
Multiply the average watts per second by the elapsed seconds then divide by 3600 (seconds in an hour).
Reset the last reading timer.
Print the data.
Add the calculated average W/Hr for this time slice to the running total.

# add up the delta-watthr used since last reading
# Figure out how many watt hours were used since last reading
elapsedseconds = time.time() - sensorhistory.lasttime
dwatthr = (avgwatt * elapsedseconds) / (60.0 * 60.0)  # 60 seconds in 60 minutes = 1 hr
sensorhistory.lasttime = time.time()
print("\t\tWh used in last {} seconds: {}".format(elapsedseconds,dwatthr))
sensorhistory.addwatthr(dwatthr)

146

147

148

149

150

151

152

# add up the delta-watthr used since last reading

# Figure out how many watt hours were used since last reading

elapsedseconds = time.time() - sensorhistory.lasttime

dwatthr = (avgwatt * elapsedseconds) / (60.0 * 60.0) # 60 seconds in 60 minutes = 1 hr

sensorhistory.lasttime = time.time()

print("\t\tWh used in last {} seconds: {}".format(elapsedseconds,dwatthr))

sensorhistory.addwatthr(dwatthr)

Here’s a basic explanation:

When a chunk of data comes in, we calculate the average W/Hr for the first second of that chunk. Multiply that value by the number of seconds since the previous reading. This gives us the average W/Hr for a 2 second time slice. If we were to collect those slices for one hour and add them together we would have X watts used in one hour.

The cumulative watts used will continue to accrue until we pass the limit of the timer we’re using to determine how often to send the data up to ORDS.

# Determine the minute of the hour (ie 6:42 -> '42')
currminute = (int(time.time())/60) % 10
print(int(time.time()))
# Figure out if its been five minutes since our last save
if (((time.time() - sensorhistory.fiveminutetimer) >= 60.0)
    and (currminute % 5 == 0)
    ):

154

155

156

157

158

159

160

# Determine the minute of the hour (ie 6:42 -> '42')

currminute = (int(time.time())/60) % 10

print(int(time.time()))

# Figure out if its been five minutes since our last save

if (((time.time() - sensorhistory.fiveminutetimer) >= 60.0)

and (currminute % 5 == 0)

# Determine the minute of the hour (ie 6:42 -> '42')
currminute = (int(time.time())/60) % 10
print(int(time.time()))
# Figure out if its been five minutes since our last save
if (((time.time() - sensorhistory.fiveminutetimer) >= 10.0)
   # and (currminute % 5 == 0)
    ):

154

155

156

157

158

159

160

# Determine the minute of the hour (ie 6:42 -> '42')

currminute = (int(time.time())/60) % 10

print(int(time.time()))

# Figure out if its been five minutes since our last save

if (((time.time() - sensorhistory.fiveminutetimer) >= 10.0)

# and (currminute % 5 == 0)

To calculate the average W/Hr during the last 10 seconds:

Multiply the cumulative watts used by 3600 (seconds in an hour).
Divide by the seconds since the last time we sent data to ORDS.

def avgwattover5min(self):
    print("cumulative: %f, time.time: %f, fivetimer: %f" % (self.cumulative5mwatthr, time.time(), self.fiveminutetimer))
    return self.cumulative5mwatthr * (60.0*60.0 / (time.time() - self.fiveminutetimer))

123

124

125

def avgwattover5min(self):

print("cumulative: %f, time.time: %f, fivetimer: %f" % (self.cumulative5mwatthr, time.time(), self.fiveminutetimer))

return self.cumulative5mwatthr * (60.0*60.0 / (time.time() - self.fiveminutetimer))

The short explanation is if we were getting a consistent reading of 5 watts per hour for every sample, every 10 seconds this calculation would come out to 5 W/Hr during the last 10 seconds. However, it’s not likely that we will get the same 5 W/Hr every reading so this function will give us the average W/Hr during the last 10 seconds.

I can understand if you’re a bit confused at this point. There seem to be a couple extra steps here than what should be needed for my simple graph. I had to work out a simulation in a spreadsheet before I could accept that it was working. However, I left the calculation code alone assuming that it may be needed for some of the more advanced versions of the project.

If your math skills are better than mine and you find that my explanation is wrong or you can explain it better, please leave a comment.

Oracle Jet

The Oracle Jet graph used in the example is the basic Line with Area Chart. I’m using the Y axis for the W/Hr data and the X axis for the timestamps.

The graph has the capability to track multiple series of data which would be useful for multiple kill-a-watts, but I’m only using one in the presentation.

The relationship between the X and Y axises is positional using the array position for the data elements in two arrays.

JavaScript

This is a typical jQuery ajax GET function.

Inside the success function:

Get the items array from the response.
Create a variable for the X-axis data.
Create a variable for the Y-axis data. Since we’re only tracking one sensor we can define the name value and initialize an items array for it.
Loop through the response items.
Populate the items array for our sensor (Y axis).
Populate the timestamp array (X axis).
Set the ko.observable objects for the two axises of the graph.

Next is a short function to call getData() every 5 seconds.

var getData = function() {
  $.ajax({
    type: "GET",
    url: 'https://oda-a425020.db.us2.oraclecloudapps.com/apex/kaw_demo/power/usage',
    success: function(res) {
      //Get the items from the response
      var resData = res.items;
      var groupData = [];
      var areaSeries = [{name: "Sensor 1", items:[]}];

      for (var i = 0; i < resData.length; i++) {
        //push the watt value to the areaSeries[0] items array
        //  We are only tracking one sensor so it's always areaSeries[0]
        areaSeries[0].items.push(resData[i].watt);

        //push the created_on value to the groupData array
        groupData.push(resData[i].created_on);
      }

      //set the observables to the proper values
      self.areaSeriesValue(areaSeries);
      self.areaGroupsValue(groupData);
    },
    failure: function(jqXHR, textStatus, errorThrown) {
      alert(textStatus);
    }
  });
};

//Get the data every 5 seconds.
setInterval(function() {
  getData();
}, 5000);

var getData = function() {

$.ajax({

type: "GET",

url: 'https://oda-a425020.db.us2.oraclecloudapps.com/apex/kaw_demo/power/usage',

success: function(res) {

//Get the items from the response

var resData = res.items;

var groupData = [];

var areaSeries = [{name: "Sensor 1", items:[]}];

for (var i = 0; i < resData.length; i++) {

//push the watt value to the areaSeries[0] items array

// We are only tracking one sensor so it's always areaSeries[0]

areaSeries[0].items.push(resData[i].watt);

//push the created_on value to the groupData array

groupData.push(resData[i].created_on);

}

//set the observables to the proper values

self.areaSeriesValue(areaSeries);

self.areaGroupsValue(groupData);

failure: function(jqXHR, textStatus, errorThrown) {

alert(textStatus);

}

});

};

//Get the data every 5 seconds.

setInterval(function() {

getData();

}, 5000);

HTML

We copy the HTML from the cookbook for just the graph component.

Since we’re not using the additional functionality from the Jet Cookbook example we remove the highlighted lines (14, 15).

<div id="lineAreaChart" data-bind="ojComponent: {
        component: 'ojChart',
        type: 'lineWithArea',
        series: areaSeriesValue,
        groups: areaGroupsValue,
        animationOnDisplay: 'auto',
        animationOnDataChange: 'auto',
        orientation: orientationValue,
        stack: stackValue,
        hoverBehavior: 'dim'
     }"
     style="max-width:1000px;width:100%;height:350px;">
</div>

<div id="lineAreaChart" data-bind="ojComponent: {

component: 'ojChart',

type: 'lineWithArea',

series: areaSeriesValue,

groups: areaGroupsValue,

animationOnDisplay: 'auto',

animationOnDataChange: 'auto',

orientation: orientationValue,

stack: stackValue,

hoverBehavior: 'dim'

style="max-width:1000px;width:100%;height:350px;">

</div>

Go try something new

The goal of this presentation is to encourage people to go out and try something a little out of their comfort zone. If you think your soldering skills are lacking find a maker group in your area and take a class. If you are strong in one programming language try another.

This is a great project to experiment with, there are a few different skills all mixed together, each of them is fairly close to entry level and they are popular enough that there should be a lot of help available.

As always, if you run into issues feel free to leave a comment here or hit me up on twitter and I’ll be glad to help get you going.

I plan to update this post as questions arise. If you’d like to see it all running together catch one of my upcoming sessions.

Learn Code Share

Track-a-Watt – IoT to the Database: code walkthrough

Python modules

sensorhistory.py

xbee.py

wattcher.py to wattcher-min.py

Average Watts/Hour calculation

Oracle Jet

JavaScript

HTML

Go try something new

Related

Leave a Reply Cancel reply

Python modules

sensorhistory.py

xbee.py

wattcher.py to wattcher-min.py

Average Watts/Hour calculation

Oracle Jet

JavaScript

HTML

Go try something new

Share this:

Related

Leave a Reply Cancel reply