Analyzing Home Heating and Cooling Data Using Nest: Data Collection

By Jason Thompson, co-founder, 33 Sticks
 
As home automation becomes more of a consumer reality, the data available to home owners to monitor, analyze, and optimize the home will become increasingly valuable. In this multi-part series, we will be examining home automation analytics, specifically focusing on data acquired through home heating and cooling automation using the Nest Thermostat. This three part series will address data collection, data visualization, and analysis, insights, & optimization.
 
In part one of this series, we will examine the data that is available from the Nest and how to collect and store that data for future use.
 
Before we can determine what data we want to collect, we must first understand what data is available. The following data elements can be accessed from your Nest Thermostat:
 

• Current and Target Temperatures & Humidity
• Time to Target Temperature



• Target Temperature Mode, Fan Mode
• AC, Heat, and Fan Status; On/Off
• Manual and Automatic Away Mode
• Location Information
• Network Information
• Currently Active Schedule (by Day)
• Next Scheduled Event
• Last 10 Days Energy Report

 
Step 1 – Build a Database
The first step is to determine what data elements you are interested in capturing and then build a corresponding database table that will store the information.
 
We decided to capture the following elements: Location, Outside Temperature, Away Status, Current Temperature, Current Humidity, Temperature Mode, Target Temperature, Time to Target Temperature, Heat Status, and AC Status. In addition to the data retrieved from the Nest, we are also capturing a date and timestamp that aligns with the date and time the script retrieves the data from the Nest.
 
Once we had determined the data we wanted to capture, it was easy to create a simple table within mySQL to hold the data. We created a database tabled named ‘nest’ that has the following structure:
 

 
Step 2 – Capture Nest Data Into Database
In order to query your Nest data and insert the data into your newly created database table, you must create a script that has two parts to it. The first part authenticates to your Nest and retrieves the requested data that is returned in JSON format. Our base script is making use of a NEST API that was developed using PHP. The API is available on GetHub.
 
The second part of the script parses the JSON data and inserts it into the database table we created in Step 1.
 
Rather than just dump all the code out into this blog post, we have created a template file that can be downloaded and modified as needed. The file is well documented and allows you to create a database connection, create a Nest object, query Nest, and insert values into a database table.
 

• Nest PHP Template
• Nest PHP Class

 
Step 3 – Automate Date Capture
Once you have coded your script and everything is working as expected, the next step is to automate the execution of your script to run on an ongoing basis to continuously capture and log data from your Nest.
 
We chose to execute our script every 5 minutes, at this point that number is completely arbitrary, you could choose to run it every minute or every 30 minutes. We will have a better idea on a proper interval once we have had a chance to spend some quality time analyzing the data we are currently capturing.
 
To automate the execution of our script, we are making use of a cron job.
 
As we have a Raspberry Pi laying around, we thought it would make the perfect server to manage our Nest data capture script.
 
Once you have decided where you want to run your script from, if you are comfortable with it, it is pretty easy to create a new cron job directly from the command line by creating a new crontab file for your user. This is done by accessing the command line. Type the following into the command line:
 
crontab -e
 
Determine how often you want to run your script and add a new entry to your crontab file. As mentioned, we are running our script every 5 minutes, so our entry looks like:
 
*/5 * * * * /home/nest/nest.php
 
If you are hosting your script using a service like GoDaddy, while you will have less control over what you can do with your cron job, they make very nice UI tools that make creating a new cron job super simple. Here is a link to the GoDaddy help file on setting up new cron jobs.
 
* * * * *
 
Visually, our three step data collection process looks like the following:

 
And that is it, once you have completed those 3 steps, you will be capturing data from your Nest Thermostat.
 
In the next part of this series, we will be covering techniques to visualize your Nest data. Following that, we will go in-depth on analyzing the Nest data and illustrating how the insights gleaned from that analysis can be put into use to lower overall energy consumption.
 
Note, this project is very much a work-in-process, if you have any suggestions on how we can optimize this process, from the database setup to the execution of the script, please add your suggestions below.
 
We hope you enjoy this series and if you have any questions, please don’t hesitate to contact us.