Building an IoT setup with NodeMCU and Azure IoT Hub on macOS

Overview

In this post we’ll walk through how to get Arduino development with the NodeMCU for a simple IoT (Internet of Things) setup started. As the IoT device, we use the popular ESP8266-based NodeMCU platform and then hook it up to Azure IoT Hub. From there, we archive all our message into Azure Blob. The whole setup will look something like this:
Overview
Overview
While there are plenty of tutorials out there on how to get this working under Windows, I want to focus explicitly on macOS, as the Arduino IDE has some challenges with bringing code to NodeMCU boards under Apple’s operating system.
Our NodeMCU board
Our NodeMCU board
One word of caution:
I used the NodeMCU Amica in this tutorial. In case the USB connection to the board does not work, I advise to try several USB-to-MicroUSB cables. Using an USB Hub with USB-C connectivity on my Macbook Pro 13″ (Late 2017) did not work. However, using an Anker USB to USB-C adapter with a high-quality USB-to-MicroUSB cable (e.g., one from a cellphone or Kindle), worked smoothly.

Getting Started

First install the Arduino IDE. I used version 1.8.5 under macOS High Sierra (10.13.3).
In order to be able to connect the NodeMCU board to our Macbook, we need to install the proper USB driver for establishing a serial connection. In our case, the Silicon Labs driver should work fine. We might need to reboot macOS afterwards.
Fire up the Arduino IDE. There are plenty of tutorials for setting this up with the NodeMCU, so we’ll just look at the high-level steps:
Form the menu, pick Arduino -> Preferences and add http://arduino.esp8266.com/stable/package_esp8266com_index.json as a Additional Board Manager URLs. While we’re in there, we probably want to turn on Display line numbers.
Arduino Preferences
Arduino Preferences
Next, we need to load our libraries for connecting the NodeMCU to Azure IoT Hub. For this we select Sketch -> Include Library -> Manage Libraries... and then search and install the following libraries:
  • AzureIoTHub
  • AzureIoTUtility
  • AzureIoTProtocol_MQTT
  • Adafruit Unified Sensor
  • DHT sensor library

Getting everything working under macOS

In theory, we would be ready to start now, but as of March 2018, there are a few things that will (most liekly) not work on macOS:
  • Flashing the NodeMCU chip
  • Proper compilation of some logging code in the Azure IoT Hub library
Luckily, there is a workaround for each problem!

Solution 1: Flashing the NodeMCU from macOS

The Arduino IDE uses a binary called esptool to push binary code from our machine to the NodeMCU board. However, on macOS this doesn’t seem to work well. This seems to be a NodeMCU specific problem, as other platforms (e.g., Wemos D1 Mini) worked out of the box for. To fix this, we can replace esptool with a Python-based esptool.py.
Firstly, let’s fire up our friend pip and install esptool.py (for more details, look here):
$ pip install esptool
In case the Arduino IDE is open, close it. Then fire up your favorite editor for editing Arduino IDE’s platform.txt so we use the python script instead of the native esptool:
# Your path might differ a bit
vi /Users/csiebler/Library/Arduino15/packages/esp8266/hardware/esp8266/2.4.1/platform.txt
Look for the following line of code:
# Around line 124
tools.esptool.upload.pattern="{path}/{cmd}" {upload.verbose} \
-cd {upload.resetmethod} -cb {upload.speed} -cp "{serial.port}" \
{upload.erase_cmd} -ca 0x00000 -cf "{build.path}/{build.project_name}.bin"
Comment it out, and add:
tools.esptool.upload.pattern="/usr/local/bin/esptool.py" \
  --port "{serial.port}" write_flash 0x00000 "{build.path}/{build.project_name}.bin"
Save it and we’re done!

Solution 2: Getting logging in Azure IoT Hub library working

From here on, we could happily continue, flash our code to the NodeMCU board and even run it, but we might run into some weird exceptions during code execution of the Azure IoT Hub library. According to this GitHub issue, there is a workaround for this:
First, open
$ vi ~/Documents/Arduino/libraries/AzureIoTUtility/src/azure_c_shared_utility
and locate the following code block:
#define LOG(log_category, log_options, FORMAT, ...) { \
  const char* __localFORMAT = PSTR(FORMAT); \
  os_printf(__localFORMAT, ##__VA_ARGS__); \
  os_printf("\r\n"); \
}

#define LogInfo(FORMAT, ...) { \
  const char* __localFORMAT = PSTR(FORMAT); \
  os_printf(__localFORMAT, ##__VA_ARGS__); \
  os_printf("\r\n"); \
}

#define LogError LogInfo
Comment it out and replace it with:
#define LOG(log_category, log_options, FORMAT, ...) { \
  char tmp[256]; \
  az_c_strncpy_P(tmp, PSTR(FORMAT), 256); \
  os_printf(tmp, ##__VA_ARGS__); \
  os_printf("\r\n"); \
}

#define LogInfo(FORMAT, ...) { \
  char tmp[256]; \
  az_c_strncpy_P(tmp, PSTR(FORMAT), 256); \
  os_printf(tmp, ##__VA_ARGS__); \
  os_printf("\r\n"); \
}

#define LogError LogInfo
Save and done. From here on, we can start deploying an IoT Hub in Azure and hook up our NodeMCU to it!

Setting up Azure IoT Hub

For deploying the IoT Hub, we can use Azure CLI with the IoT Extension, which can be installed easily:
$ az extension add --name azure-cli-iot-ext
$ az extension list
[
  {
    "extensionType": "whl",
    "name": "azure-cli-iot-ext",
    "version": "0.4.1"
  }
]
Next, let’s create a resource group for our IoT setup:
$ az group create --name iot-hub-demo --location westeurope
and then our IoT Hub with the F1 free tier (note: only one possible per subscription):
$ az iot hub create --name clemensdemohub \
--sku F1 \
--resource-group iot-hub-demo
As our NodeMCU will be registered as an IoT device in IoT Hub, we need to create a device-identity for it:
$ az iot hub device-identity create \
    --hub-name clemensdemohub \
    --device-id clemensnodemcu \
    --resource-group iot-hub-demo
In a real world setup, this would obviously be an UUID.
Lastly, we need to query the connection string, so that our device can authenticate with IoT Hub:
$ az iot hub device-identity show-connection-string \
    --hub-name clemensdemohub \
    --device-id clemensnodemcu
{
  "cs": "HostName=clemensdemohub.azure-devices.net;DeviceId=clemensnodemcu;SharedAccessKey=xxxxxxx="
}
Last but not least, we can fire up the Arduino IDE and get everything plugged together.

Connecting our NodeMCU board to Azure IoT Hub

For getting started, we can rely on the example repository for hooking up the NodeMCU to IoT Hub:
$ git clone https://github.com/Azure-Samples/iot-hub-c-huzzah-getstartedkit.git
In the Arduino IDE, choose File -> Open and pick:
iot-hub-c-huzzah-getstartedkit/simplesample_mqtt/simplesample_mqtt.ino
A few tabs will open up. In iot_configs.h we can set our wifi credentials, as well as the connection string for linking the NodeMCU as device to IoT Hub:
#define IOT_CONFIG_WIFI_SSID "myssid"
#define IOT_CONFIG_WIFI_PASSWORD "supersecret"
#define IOT_CONFIG_CONNECTION_STRING "HostName=clemensdemohub.azure-devices.net;DeviceId=clemensnodemcu;SharedAccessKey=xxxxxx="
Lastly, connect the NodeMCU and select the correct board via Tools -> Board -> NodeMCU 1.0 and the connection port via Tools -> Port -> /dev/cu.SLAB_USBtoUART.
Finally, hit the checkmark button (first button) in the Arduino IDE UI to compile the project, and then the arrow (second button) for flashing to the device.
Flashing our NodeMCU board
Flashing our NodeMCU board
Once completed, NodeMCU will restart and boot the code.

Verifying that everything works

As NodeMCU should be running our code now, we can go to Tools -> Serial Monitor, set the rate to 115200 baud, and we should see some log messages:
Showing log messages through Serial Monitor
Showing log messages through Serial Monitor
It might take up to a minute until the NodeMCU synced with NTP and starts sending its first message.
Judging from the log output, NodeMCU should have sent one message to Azure IoT Hub. If we go to the landing page to our IoT Hub instance, we should see that it already received some messages:
Azure IoT Hub registered some messages
Azure IoT Hub registered some messages
Looks good! From here on, we can forward our messages into Azure Stream Analytics, Azure Databricks (which is based on Apache Spark) or just push them into Azure Blob for archival.
For storing all device messages in Azure Blob, we navigate into our IoT Hub, then select Endpoints and click Add. From here on, we define the name of the endpoint and choose to output the data into an Azure Blob container:
Configuring IoT Hub to archive messages in Blob
Configuring IoT Hub to archive messages in Blob
Lastly, we need to add a route for forwarding our device messages to the configured Blob endpoint. We can simply do this by selecting Routes and clicking Add:
Adding a routing rule to Blob
Adding a routing rule to Blob
After 1-2 minutes, the first messages will come visible in our Blob container, aggregated by 60 second or 100MB chunks, whichever is reached first.

Summary

NodeMCU is a great platform for experimenting with IoT and taking some first steps. Using Azure IoT Hub in the backend keeps the barrier of entry low, and later allows to easily scale to millions of devices. We also showed how to attach an Azure Blob container for long-term archival of device messages.
However, for the macOS developers among us, there are a few pitfalls like flashing code onto the NodeMCU and getting logging with the Azure IoT Hub library working. With the two workarounds outlined here, we were able to get our end-to-end IoT setup working in under 30 minutes.
A big thanks to the authors of the following two blog posts: Getting Started with the ESP8266 on OSX and
ESP8266 Arduino IDE on Mac OS X. If you have any questions, feel free to reach out to me on Twitter @clemenssiebler.

Leave a Reply

Your email address will not be published. Required fields are marked *