Pardon the coy title. But I love “NCIS”. And some days, I feel a lot like Gibbs: I have a pet project in the basement. And whenever I get bored or frustrated, I work on my project. But as of now, I think that my home automation project is finally sea-worthy.

My little Pi is a beast. I am running a long list of software on this device. I’m running Home Assistant with the following add-ons: DuckDNS, Let’s Encrypt, Mosquitto, Node-RED, Samba, and SSH. With this combination, I can monitor assets within the home. I can determine whether my wife and I are in the home or outside of the home. I can build automated tasks based upon data collected within the home. I can manage the assets and the compute infrastructure in the home. And I can secure it against exploitation by ‘bad actors’.

And now, I’ve finally gotten around to configuring the data collection and graphing infrastructure. The package of tools that I am using for this includes Grafana and InfluxDB. After installing the components, I got about the work of configuring the software.

InfluxDB is a time-series data repository. It is designed much like a NoSQL tool; data is written in series but isn’t updated after it is originally written. Later, the data is read serially and used in graphing and/or statistical studies. Fortunately, I was able to configure InfluxDB with very little incident. I think that years of econometric studies made this part relatively simple to implement.

Once done with the database, I turned my attention to Grafana itself. And it was very difficult to grok this tool. First, I ran into quite a bit of difficulty installing needed plugins. After poring over the logs (and consulting written guides on the Internet), I found that the “behind the curtain” instance running in a container was having difficulty downloading the ‘plugin’ components on the fly.

While scratching my head, I saw a quick popup about my dynamic address being updated. That’s when the light came on. For whatever reason, I had been having trouble with a sporadic inability to log into my system. The symptoms were that the login would just wait, and wait, and wait. I finally remembered that some applications really dislike running inside a VPN tunnel. And worse still, I wondered if the IP address recorded in DNS reflected a potentially changing DNS entry.

So I disconnected from my VPN. That’s when things just started to work. It was quite odd, though. I could finally add the plugins. But I had changed the network on my Windows system – and not the network on the Pi. So there has to be something flowing through the browser. I’ll have to dig into that. But the problem had been solved.

I also found that I needed to update the DNS on my little server. Simply put, I had been using the Pi-hole (an ad blocking DNS server) to fulfill the DNS requests for the Home Assistant Pi system. I suspected that certain key DNS requests returned with null results. Therefore, I needed to clean up the DNS config on the Home Assistant Pi.

Once both of these tasks (i.e., the VPN and the Pi-hole DNS) were resolved, the plugins started to install. So my Grafana installation could proceed.

And then I hit the learning curve of Grafana itself.

Grafana is a very cool tool. But its user interface is not very intuitive. It took a few hours to figure out just how to add variables and select the right graphing interval before real data started to emerge. But once I learned these little tricks, the graphs became easy. Last night, I began the quest to graph all of the data that I could graph. This compulsion is a learned experience; I spent many years being driven by capacity and performance data. And I wasn’t harnessing the data that is coming from my home sensors. So I am now inspired to build all sorts of data models and graphs. [Note: I really love it when my past experiences can inform my current and future activities.]

All in all, I’ve spent a few hundred hours over a few months on this home automation project. And I have learned so much about home automation, container technologies, and web security at the edge of the network. Now I’m left with one nagging thought – and an irresistible question: How does anyone expect the average homeowner to know these things. Moreover, how can we expect consumers to care enough to learn these things? Most people want the “iPhone experience” where they can spend a lot and have someone else do the integration for them.

So which are you? Are you a maker/builder/integrator? Or are you a buyer?