ESP8266 + DHT22 + Homie -> Home-Assistant

So now that I had a Home-Assistant (HA) setup working, an esp8266 booting and that I can write software for, it was time to start making the two talk to each other and make something that can become unseful based on all this.

The choice went to the simplest element I had at hand, a DHT22 temperature and humidity sensor.

There were 5 steps to do this:

  • wire the dht22 on a test setup
  • find how to use it in a Homie firmware, and flash it
  • configure the device
  • configure HA for MQTT, which led me to rewrite the whole configuration
  • test it

So let's get to this step by step

Wiring the DHT22

It can be pretty simple, without pullup resistors and capacitor, I found this on instructable

dht22
wiring

I simply put this to another GPIO on my esp12f, but that's pretty much it. The result on the breadboard looks like this:

dht22 breadboard

That's about it, for testing of course, for a final sensors, I'll have to make it cleaner than that...

Homie firmware

I started looking up for code on how to use homie with the dht22, there are libs to make use of it, running:

platformio lib search dht

Showed me the DHT sensor library that is esspressif compatible, I simply installed it via:

platformio lib install 19

After that I looked online at examples to use it, and actually found this code which does exactly what I expected. At first I started writting each lines trying to include only the needed one. But I think this is really close to the bare minimum to get something fully working, so I simply renamed it to raton-dht and switch the version to 0.0.1.

Jumping a little off-topic, the version number has an interest, because Homie support OTA, using the pretty cool homie-ota, which includes a listing of firmware and an inventory of devices, which is pretty awesome for handling yor homie devices and update them if need be. I haven't tried it yet myself, but I definitely will once I get real devices up and running in the house. This really looks like a must-have to me. Once again, thanks jpmens!

Back on tracks, not having the OTA setup yes, I simply switched my GPIO0 to low and ran:

platformio run -t upload

And the next boot the dht sensors firmware was sitting in configuration mode.

Homie configuration

In the previous post, I didn't explain how to configure the homie devices, I actually did not discover it at this step, I played with this on the previous useless firmware to understand how it works.

First it is worth noting that the homie-esp8266 documentation has been moved from github to readme.io, there on the http json api page, you can find the 3 options that you can use to configure your device in configuration mode:

  • the Web UI
  • the android app which apparently is a a webview directly to the webui (at least that's my guess)
  • a curl example, but you'll need to know what the config.json looks like like described in the dedicated page

In any case, you'll need to be connected to the AP the homie device will create when in configuration mode, for me, working on a workstation, and not having my laptop at hand (and feeling too lazy to grab it, plug it as it isn't charged right now, configure another wireless device via wpa_supplicant), that only left me with my android phone... But there is a catch: android checks that it can reach outside to actually let you use your wifi connection.

I found here that on a rooted device you can run:

settings put global captive_portal_detection_enabled 0

This did the trick for me, so I ran that, connected to the Homie-XXXXX AP, and fired the android app, which found the device, and allow me to configure it, the configuration is pretty straightforward and short, once this is done, you can finally reboot the homie device in normal mode, where it connects to your AP/router and start trying to talk to the mqtt broker. Now on to the HA setup.

Home-Assistant MQTT configuration

As said in the previous post, I was planning on using the embedded MQTT broker in HA. According to the MQTT component page, to enable MQTT support in HA, we have to add

mqtt:

To the configuration file, other parameters are for specifying the broker, to use the embedded one, without further configuration, that's it. Restarting HA with that enables the MQTT broker.

Next we need to configure the sensors, but when I started to look to add this I realized how messy my HA configuration had become, with so few things it looked already pretty cluttered with useless things, comments that are only usefull when beginning to use it. So i decided to go for a huge cleanup and rewrite.

First step, version control, I planned on splitting it in multiple files, and this won't be any easy to handle and maintain if not in git, so I created a ha-cfg github repo

I went from this to that

It took some fiddling to understand the !include which is not totally obvious to use, but I'm pretty happy with the final result.

Now in my sensors.yaml I simply added

# temperature test
- platform: mqtt
  state_topic: devices/01397be0/temperature/degrees
  name: "Test temperature"
  qos: 0
  unit_of_measurement: "°C"

# humidity test
- platform: mqtt
  state_topic: devices/01397be0/humidity/relative
  name: "Test humidity"
  qos: 0
  unit_of_measurement: "%"

And I was all set.

Testing

After that and restarting HA with the new configuration, I booted the esp again and could see:

** Booting into normal mode **
{} Stored configuration:
   Hardware device ID: 01397be0
   Device ID: 01397be0
   Boot mode: normal
   Name: dht#1
   Wi-Fi
     SSID: SFR-5458
     Password not shown
   MQTT
     Host: 192.168.0.6
     Port: 1883
     Base topic: devices/
     Auth? no
     SSL? no
   OTA
     Enabled? yes
     Host: 192.168.0.17
     Port: 80
     Path: /ota
     SSL? no
 Attempting to connect to Wi-Fi...
 Wi-Fi connected
Triggering HOMIE_WIFI_CONNECTED event...
 Attempting to connect to MQTT...
Connected
Sending initial information...  OK
Subscribing to topics...  OK
 MQTT ready
Triggering HOMIE_MQTT_CONNECTED event...
Calling setup function...
Sending Wi-Fi signal quality (100%)...  OK
Sending uptime (8s)...  OK

And on HA web-ui I know have test humidity and test temperature sensors showing up on the top row.

ha-ui

And that's it!


21 août 2016 12:39:25 -- tags [ esp8266 , dht22 , home-assistant , homie ]

0 Comments

esp8266 with Homie and PlatformIO

As I talked about in previous posts, I got some ESP8266 devices I wanted to play with, one has been setup on a breadboard for testing for a while, able to boot, but never got to try anything more on it yet.

Protocol and firmware base choice

As my goal is to do some connected devices for home automation, I wanted to make firmwares for these that could talk to Home-Assistant. Even through HA supports quite a lot of way to talk to devices, there were 2 main options I considered:

  • http: simply having a http on the esp which exposes status, and can get commands via POST for example
  • mqtt: it is meant for IoT, the main drawback was I actually didn't want to run a broker

That's when I realized HA MQTT component has an embedded broker. Which apparently is not getting much love as everyone is using mqtt with HA for owntracks, and the embedded broken is incompatible with it. But given I don't plan on using owntracks myself for now, that should be sufficient for my need.

A large part of my fiddling for home automation has been triggered by Jan-Piet Mens and he actually loves MQTT, and played around with clients and told a few times (on twitter if my memory is any good) that Homie for esp8266 was well thought and stable, as I generally find his choices, advices and tastes to match pretty well mine, I decided to go for this.

So here we are, if all goes well, my esp8266 will be running a Homie based firmware to talk to home-assistant via MQTT.

Tools

Then the question of the tools arised, most of the things I read about esps were all talking about Arduino IDE, which I found comments before even trying it, saying to sum it up, as working, but not great. For those who know me, you know I don't like UI, graphical interfaces, and mouse compliant interfaces. I was not trilled to have to use this.

There is quite some talk about esptool which apparently is a great command line tool written in python to flash you esps. That could have done it, but as a complete beginner in all this, I had no clue how to build a project to create the firmware before flashing it with esptool.

Homie github README talks about PlatformIO. I had no clue what this was, and one of the first thing you notice on their webpage is the IDE word, and screenshot of Atom sporting a toolbar to interact with PlatformIO. But after discading it in the first place, and coming back to their website and taking some time to actually read things, I noticed: PlatformIO is an open source ecosystem for IoT development. There is an IDE, but it is more an ecosystem, and everything can be done from the command line, and that was a lot more appealing to me, so I went for it.

Platform.IO

Being a python project, you can install it via pip, some python -c directly from curl as suggested in the Super-Quick Installation Guide.

I went with archlinux's AUR package, there also is a -git one, if need arises later on.

Once installed I went through this page and followed their example:

platformio init --board nodemcu

The nodemcu are based on ESP12s, as I own an esp12f, and I read somewhere it was generally working pretty fine with nocemcu as a board, I decided to go for it.

The previous command create an .ini and a lib and src folders. There also is a .pioenvs folder that contains everything PlatformIO will need to build your project, libs, the output binaries etc.

Then as explained on Homie's getting started guide I installed the lib, given I'm a bit curious I looked it up before typing what the doc suggests.

platformio lib search homie
platformio lib install 555

I then dropped the so called Bare mininum sketch

#include <Homie.h>

void setup() {
      Homie.setup();
}

void loop() {
      Homie.loop();
}

in src/test.c, and tried to compile using the command:

platformio run

But that failed because of missing PubSubClient.h in Homie includes, as PlatformIO is said to handle depedancies, that surprised me but well, whatever:

platformio lib search pubsubclient
platformio lib install 89

Then tried to compile again... this time a real compilation error arised:

In file included from .pioenvs/nodemcu/Homie_ID555/Homie/MqttClient.hpp:3:0,
from .pioenvs/nodemcu/Homie_ID555/Homie.hpp:3,
from .pioenvs/nodemcu/Homie_ID555/Homie.h:4,
from src/test.c:1:
.pioenvs/nodemcu/ESP8266WiFi/ESP8266WiFi.h:27:8: error: expected identifier
or '(' before string constant
extern "C" {
^

Given the presence of .hpp in the error, and the extern "C" section I guess I should not be assume I'm writing C, so I move src/test.c to src/test.cpp, launched the compilation, and this time it built fine.

If you have your FTDI adapter plugged in, and your ESP in flash mode, you can add --target upload to the run command line, and PlatformIO will flash your device for you.

I simply wired GPIO0 to GND and rebooted the esp, launch the run with upload flag, and my esp was flashed. I then rewired GPIO to VCC and rebooted, and the AP from Homie's configuration mode showed up in my phone wifi... I didn't yet took the time to configure it via json and test it further.

But that's already a good starting point, and that's it for now!


18 août 2016 22:59:54 -- tags [ esp8266 , wifi , homie , platformio , IoT , MQTT ]

0 Comments

Home Automation (part 3)

My current Home-Assistant setup

Currently my HA homepage looks like this:

ha-ui

Let's first go over the various parts of the UI, and how my setup is currently working befor diving into the configuration file details.

The web UI

  • At the top, we have the status of various sensors
    • Update notification
    • Sun status, useful sensor to automate things based on sunset/sunrise
    • Device tracker status: My wife's and my phone are both at home
    • Z-Wave embedded switch current power consumption
    • Current Weather
    • Speedtest: download, ping, upload
    • Z-Wave embedded switch power consumption sum: so we know since we started using HA the living room light consumed 5.62kWh
    • Transmisison torrent download and upload speed
  • Weblink section are custom links to whatever URL you want, here my transmission webui
  • Input Select shows a custom dropdown I created: Mode, I am using it to select the current working mode of my HA setup
    • Jour (day): when switching to it, switch off the living room light, no automation
    • Soir (evening): light set to a medium dimming level (90/255), media center playing will dim the light to 30/255 for better atmosphere
    • Soirée (party): As for soir setting, light 90/255, but no automation
    • Nuit (night): set living room light to very very low level (10/255)
  • Switch: as you may have guessed, that section list the switches that are not for light control
    • PC raton is a wake-on-lan switch to wake my computer up, unfortunately not working as the FreeBSD jail that runs HA won't let the WOL packet out...
  • Light: the switches to control lights, for dimming you can click on the name and get a slider to adjust level.
    • Lampadaire is my wall plug on which I plug a standing light
    • Salon is the living room wall switch controlled lights
  • Kodi: is my Openelec installation, my main media center
  • Ratoncast: is a chromecast, that we mainly use for netflix

By default there can be a lot of other things showing up, with pretty awful names, for example the Fibaro Z-Wave switch has various sensors and parameters, they all appear in there but you can rename them to friendly names and hide the unused ones in your configuration.

The Automation

I have only a few automation for now, but they are actually pretty handy, most of them are based on the custom Input Select.

  • At sunset - 2.5h set the mode to evening: That switch the living room on, and enable the media centers automation
  • At 1:30am switch the mode to night: light to minimum in living room and no automation based on media centers
  • At sunrise set the mode to day: switch off the living room if needed and no automation based on media centers

If the automation based on media centers is enable there is: when a media player starts playing dim the light when paused or stop, raise the light to its usual level

It may sound like I didn't setup much, but just that is actually really nice, we actually don't use the wall switch anymore, and only on a very few occasion change settings manually, it just works and we don't have to do anything, that's pretty awesome.

The configuration

Now for the ugly, if you just wanted to know what I did with HA quickly, that's about it, you can stop reading here :).

The configuration isn't very hard to understand, but it still does require a bit of fiddling and tweaking before getting things right.

Most of the components pages explains pretty well what to do and how to configure you devices. In addition there is the Examples or Cookbook page where you can find configuration from other people and snippets of automation for some commonly used cases, like dimming the lights when playing a movie.

The configuration is in YAML and can be splitted in multiple files using !include, I didn't do it myself yet, but will probably have to go this way if it gets any more complex than it currently is.

Basics

homeassistant:
  # Name of the location where Home Assistant is running
  name: Home
  # Location required to calculate the time the sun rises and sets
  latitude: 48.80569
  longitude: 2.130699
  # C for Celsius, F for Fahrenheit
  temperature_unit: C
  # Pick yours from here: http://en.wikipedia.org/wiki/List_of_tz_database_time_zones
  time_zone: Europe/Paris

# Discover some devices automatically
discovery:

# Allows you to issue voice commands from the frontend
#conversation:

# View all events in a logbook
logbook:

# Enables the frontend
frontend:

# Track the sun
sun:

# Prediction of weather
sensor:
   platform: yr

# Show links to resources in log and frontend
#introduction:

# Enables support for tracking state changes over time.
history:

# Checks for available updates
updater:

You give your setup a name, a location (to fetch sunrise/sunset), unit for temperature and your timezone.

Then in my case: enable discovery (that's how the chromecast is found), disable conversation: you can give voice command to HA from the webui, but I'm not a native english speaker, so I don't like talking to my phone in english, plus I do hope to do a better voice control system (always on) at some point. enable logbook, frontend, history and update checker enable sun tracking, based on location, that will create the sensor on top of the web UI and allow automation based on sunset and sunrise times * disable introduction: that is a nice to have at first when learning HA, but after that it mostly get annoying to me.

When using Z-Wave, you have to configure your dongle, as explained on the Z-Wave page:

zwave:
  usb_path: /dev/cuaU0
  config_path: /usr/local/lib/python3.4/site-packages/libopenzwave-0.3.1-py3.4-freebsd-10.3-RELEASE-amd64.egg/config
  polling_interval: 60000

Customization

As explained earlier, by default, all sensors, switches, and their various settings will show up once HA detects them, the annoying part is that can create a lot of clutter and be pretty ugly as the default names of some devices are pretty awful.

To solve this HA provides the customize: section, where you can hide some devices, and change the friendly_name too. In the configuration you will still have to reference the original name, but at least for the UI you'll get a nice name.

Note that this is a subsection of the homeassistant: section, this one would be a good candidate for the first !include in my configuration file.

  customize:
    sensor.fibaro_system_fgd212_dimmer_2_energy_3:
      friendly_name: Total Salon
    sensor.fibaro_system_fgd212_dimmer_2_power_3:
      friendly_name: Instant. Salon
    sensor.yr_symbol:
      friendly_name: Meteo
    sensor.everspring_ad147_plugin_dimmer_module_alarm_level_2:
      hidden: true
    sensor.everspring_ad147_plugin_dimmer_module_alarm_type_2:
      hidden: true
    sensor.everspring_ad147_plugin_dimmer_module_power_management_2:
      hidden: true
    sensor.everspring_ad147_plugin_dimmer_module_sourcenodeid_2:
      hidden: true
    sensor.fibaro_system_fgd212_dimmer_2_alarm_level_3:
      hidden: true
    sensor.fibaro_system_fgd212_dimmer_2_alarm_type_3:
      hidden: true
    sensor.fibaro_system_fgd212_dimmer_2_heat_3:
      hidden: true
    sensor.fibaro_system_fgd212_dimmer_2_power_management_3:
      hidden: true
    sensor.fibaro_system_fgd212_dimmer_2_sourcenodeid_3:
      hidden: true
    sensor.fibaro_system_fgd212_dimmer_2_system_3:
      hidden: true

    light.everspring_ad147_plugin_dimmer_module_level_2:
      friendly_name: Lampadaire
    light.fibaro_system_fgd212_dimmer_2_level_3:
      friendly_name: Salon
    light.fibaro_system_fgd212_dimmer_2_level_3_2:
      hidden: true
    light.fibaro_system_fgd212_dimmer_2_step_size_3:
      hidden: true
    light.fibaro_system_fgd212_dimmer_2_step_size_3_2:
      hidden: true

I don't think it is worth going over every part here, it is pretty clear, and definetily won't match anyone else setup, it is just here as a reference.

Sensors

Here I use the sensor word in as a generic term, I'll include quite a list of various things within it, first the device tracker: I currently don't use it but it can be a great way to automate things based on weither you're home or not.

device_tracker:
  platform: nmap_tracker
  hosts: 192.168.0.1/24
  home_interval: 10
  track_new_devices: no

There are other possibilities as you can see on the Presence Detection section of the components, one of the most popular being owntracks. But I went for a simple nmap on my local subnet, this will create a known_devices.yaml containing the list of the known hosts. As you can see I used the track_new_devices: no option, to avoid having any new device showing up, a friend who borrow your wifi for example etc, by default they will all show at the top of your homepage, that's why I used this option and the track: no option in known_devices.yaml file to limit to only tracking our two phones.

I also added speedtest to test our link speed periodically, and transimission to see if torrents are actually doing anything:

sensor 2:
  platform: speedtest
  minute: 30
  hour:
    - 0
    - 6
    - 12
    - 18
  monitored_conditions:
    - ping
    - download
    - upload

sensor 3:
  platform: transmission
  host: 192.168.0.2
  port: 9091
  monitored_variables:
    - 'download_speed'
    - 'upload_speed'

And that's it.

Media Players

The chromecast is automatically detected via the discover: module and there is no configuration entry related to it, except for the automation which is another part of this post.

So I only had to configure our Openelec media center, which is based on Kodi, which have its own component:

media_player:
  platform: kodi
  host: http://192.168.0.X
  port: 80
  name: Kodi

Custom Switches

I have 2 switches added manually in the configuration, the wake-on-lan for my computer:

switch 1:
  platform: wake_on_lan
  mac_address: "XX:XX:XX:XX:XX:XX"
  name: PC raton
  host: "192.168.0.X"

The host ip is used to check if it is up for the switch in the web UI.

The second one is the input select to choose mode:

input_select:
  current_mode:
    name: "Mode"
    options:
      - Jour
      - Soir
      - Soiree
      - Nuit
    initial: Jour
    icon: mdi:power-settings

It creates a current_mode device you can refer to in the automation, you simply list the options and the initial setting to use when restarting HA, I added an icon, but that's optional.

You can also add sliders or boolean input, they are listed in the Automation components page.

Weblinks

If like me you use HA as your browser homepage, it can be useful to add links to some often used website or services, this is where you can use the weblink component, I simply added my transmission url:

weblink:
  entities:
    - name: Torrents
      url: http://192.168.0.X:9091
      icon: mdi:download

Automation

That's it about the devices and sensors we can now dive into the automation, I'm not sure it is really worth to detail everything, it can help getting a grasp of how it works, but every setup will be different, and most Examples will cover this better than I'll be able to. So what I'll do is simply explain what part of the automation I'm talking about, and paste the configuration part related to it.

All the automation rules will reside in the automation: section (who whould have guessed?)

Mode Switching Events

So, most of my automation is based on the mode the setup is running at, and when switching from one to another changing the light accordingly.

Switching to day mode, turn off all the lights. Currently we only have 2 lights both in the living room, so it make sense to simply use the service light witouth parameter so it act on all the light:

  - alias: Jour
    trigger:
      - platform: state
        entity_id: input_select.current_mode
        state: "Jour"
    action:
      service: light.turn_off

Switching to evening or party mode, turn the living room above and behing tv to 90/255 setting, only difference is that the media playing automation won't trigger if mode isn't evening:

  - alias: Soir
    trigger:
      - platform: state
        entity_id: input_select.current_mode
        state: "Soir"
    action:
      service: light.turn_on
      data:
        entity_id: light.fibaro_system_fgd212_dimmer_2_level_3
        brightness: 90

  - alias: Soiree
    trigger:
      - platform: state
        entity_id: input_select.current_mode
        state: "Soiree"
    action:
      service: light.turn_on
      data:
        entity_id: light.fibaro_system_fgd212_dimmer_2_level_3
        brightness: 90

Switching to night mode, set the light to the lowest level, we generally leave it running overnight, at this level, the consumption is about 2W, which really isn't much:

  - alias: Nuit
    trigger:
      - platform: state
        entity_id: input_select.current_mode
        state: "Nuit"
    action:
      service: light.turn_on
      data:
        entity_id: light.fibaro_system_fgd212_dimmer_2_level_3
        brightness: 10

Mode Switching Triggers

To automate switching from one mode to another, we have 3 time triggers, one for the evening, starting 2.5 hours before sunset

  - alias: "evening time"
    trigger:
      - platform: sun
        event: sunset
        offset: "-02:30:00"
    action:
      service: input_select.select_option
      data:
        entity_id: input_select.current_mode
        option: "Soir"

One for the night, with a hard limit set at 1:30am:

  - alias: "night time"
    trigger:
      - platform: time
        hours: 1
        minutes: 30
        seconds: 0
    action:
      service: input_select.select_option
      data:
        entity_id: input_select.current_mode
        option: "Nuit"

And one to go back to daytime mode, using the sunrise time:

  - alias: "day time"
    trigger:
      - platform: sun
        event: sunrise
    action:
      service: input_select.select_option
      data:
        entity_id: input_select.current_mode
        option: "Jour"

Now we have the mode switching automatically, let's handle the dimming of lights when watching movies!

Media Playing

Now based of if we are in the evening mode or not, we want the media players to trigger dimming the light. I tried to add a check on the content type for kodi, to avoid the dimming when listening to music, but it apparently dosen't work, and I didn't find out why yet.

For kodi:

  - alias: "Kodi paused/stopped"
    trigger:
      - platform: state
        entity_id: media_player.kodi
        from: 'playing'
    condition:
      - condition: state
        entity_id: input_select.current_mode
        state: Soir
      - condition: template
        value_template: '{{ states.media_player.kodi.media_content_type != "video" }}'
    action:
      service: light.turn_on
      entity_id: light.fibaro_system_fgd212_dimmer_2_level_3
      data:
        brightness: 90

  - alias: "Kodi playing"
    trigger:
      - platform: state
        entity_id: media_player.kodi
        to: 'playing'
    condition:
      - condition: state
        entity_id: input_select.current_mode
        state: Soir
      - condition: template
        value_template: '{{ states.media_player.kodi.media_content_type != "video" }}'
    action:
      service: light.turn_on
      entity_id: light.fibaro_system_fgd212_dimmer_2_level_3
      data:
        brightness: 30

For chromecast:

  - alias: "chromecast paused/stopped"
    trigger:
      - platform: state
        entity_id: media_player.ratoncast
        from: 'playing'
    condition:
      - condition: state
        entity_id: input_select.current_mode
        state: Soir
    action:
      service: light.turn_on
      entity_id: light.fibaro_system_fgd212_dimmer_2_level_3
      data:
        brightness: 90

  - alias: "chromecast playing"
    trigger:
      - platform: state
        entity_id: media_player.ratoncast
        to: 'playing'
    condition:
      - condition: state
        entity_id: input_select.current_mode
        state: Soir
    action:
      service: light.turn_on
      entity_id: light.fibaro_system_fgd212_dimmer_2_level_3
      data:
        brightness: 30

Future

Currently one thing I would like to automate is running a script that would switch off the tv/amp when both media player have been idling for 30 minutes and if my wife computer isn't on.

For the light, I would like to replace the modes/timeslots in most cases except the night by a lux detection, and adjust brightness based on how much light there is in the living room.

I want to add other light in this room for other situation, I would love to have some sensors for temperature and humidity both inside and outside, automate the curtains too, and add some kind opened door/window detection too.

The possibilities are endless, only money, and ability in diy is the limits!

And that wraps it up for this serie of post on home automation, I'll try to keep posting the changes and hopefully my DIY sensors/controller based on ESP8266 in the near future.


31 juillet 2016 12:42:45 -- tags [ home-automation , home-assistant , light , dimming , configuration , yaml ]

0 Comments

Home Automation (part 2)

Note: Part 1 is here

Electrical setup for living room light

Our new place doesn't have any ceiling lights in the living room, but one switch activated wall plug. It is driven by a 2 way setup of 2 switches, and I decided to put the fibaro dimmer in one of them, and replace the standard switch by a push button.

The fibaro dimmer2 has 2 switch inputs, so I took a 2 button ones, and set it up in place of one of the switches, and use a single for the other one.

Currently it works like this:

  • entrance: single switch
  • near tv: double switch
  • single push: light on, same level of dimming as last use, another push to power off
  • double push: light on, full brightness
  • long press: starts at lower power, and slowly rise to allow control of dimming

Using the z-wave grouping, the second switch controls the dimming plug, where I have a floor lamp plugged for now.

Software

First thing first, the usb z-wave dongles are generally seen simplay as usb serial ports, so it will work on almost any system, as long as the software you wan to use runs on it too.

Current setup

I currently plugged the dongle on my NAS, running FreeNAS, in a jail, this caused some issues I'll explain later when detailing the softwares I tested. The computer is an AMD-E350 which is a dual core 1.6GHz, with 8GB of ram.

Likely futur setup

I will probably move the home automation to a separate machine, a lot of people use raspberrypi for this, but I will probably take an odroid-c2, The reason being I do plan on adding voice control, and latency will play a big part in the feeling of it, faster cores will ilkely help a lot in this. We will see.

Testing the softwares

There are various open source options out there to setup your own home automation box:

  • OpenHAB: v1 and v2 (in beta), a pretty huge beast, in java, but very powerful, with a lot of features
  • Domoticz: written in C, it was pretty appealing to me at first (see below)
  • Home-Assistant: written in python, pretty easy to start with
  • Jeedom: opensource but limited if you don't subscribe

I didn't try Jeedom as I wasn't fond of the subscription thing but I did try, or at least try to try, the others.

NOTE: There are other options, I didn't search everything, and didn't had time to try them all, so I only mention the ones I planned to test at first.

OpenHAB

OpenHAB is kinda the go to home automation software, it is well known, well supported, have a lot of features, the version2 has better way to handle things from the UI instead of fiddling with the configuration file, but the issue was that I run FreeBSD/FreeNAS: it comes with packages lib/modules/whatever the java world called them, including the one that handles talking to the serial port.

Unfortunately, there is a bug on FreeBSD with the shipped version, and I didn't dare rebuild everything myself just to patch this lib, so after a while I gave up for now. I still plan to give it a shot if things get easier to get it to work.

If you happen to have more motivation than I and want to try, there is a page about making openhab working on FreeBSD.

Domoticz

Domoticz was the second one I tried, mostly because it is written in C and I felt like it would be easier to fiddle inside it if needed. And I did. I had to fiddle in the code almost straight away, as the USB serial port showed up as /dev/ttyU0 but also as /dev/cuaU0. For some reason, the ifdefs in domoticz code are choosing ttyU for FreeBSD, but on my setup, this one wasn't working, wehereas the cuauU one worked, so I patched it and went with this.

I did set it up, and started using it. One nice thing it had too, is the z-wave network tools, monitongi, inclusion mode etc. That is pretty nice.

I was pretty happy until about 24h later, when for some reason it became pretty slow, like it took about 30 seconds to turn the lights on. So after a quick search, and not finding anything obvious, I went to try another one.

Home Assistant

Home Assistant (HA) is the one I still run today, it is pretty slick, setting it up was fairly easy and I didn't had to patch it. But it requires that you install the python-OpenZWave pacakge, but everything is explained on the getting-started/z-wave page, and it wasn't too hard to do. It has some services about the z-wave network that you can call, but it isn't as integrated as it is in domoticz, I actually didn't had to play with it, as the z-wave itself was already setup at this stage, I simply enable the z-wave component in HA, configured the serial port, and fire it up, and it detected the devices and they worked like a charm.

At first, it is a bit hard to understand what goes where in the configuration, or what the services/stats things in the web-ui are, but once you get a grasp of the basic principles, it is pretty easy to understand, and quite powerful.

Python-lovers will be confortable with the yaml configuration and the use of templates (as in jinja-templates) for some advanced usage.

Let's stop here for this part, the third one will talk about my current HA configuration.


16 juillet 2016 13:32:21 -- tags [ home-automation , z-wave , openhab , domoticz , home-assistant ]

0 Comments

Home Automation begins (part 1)

Introduction

So we moved, again, back to Versailles. I had the bad idea to start looking into home automation before we even moved, and started wondering what I wanted to have or to do.

Defining the need

The thing I was interested in at first was simply controlling the lights, I wanted to be LED based and dimmable. I like LED strips, but 12 or 24v things means you need a converter, and the dimming should be done at this level, which kind of make the controlling remotely in software harder to achieve.

Wandering online I discovered there now are 220v LED strips, you can't cut them anywhere like most 12v, but that was fine by me, so I got a 10m of it, and a few connectors and stuff to go with it.

I then looked at the various protocol used, and pretty quickly found that there currently is more devices available for z-wave and decided I'd go with it.

At first I was about to buy an already made box, and realized there were pretty amazing open source software available and could be installed on a computer or a raspberrypi with an usb dongle.

Looking online, I then realized all the things you could do with the so called Home Automation, and ended up with a kinda crazy goal:

  • light control
  • temperature and humidity monitoring
  • heaters control
  • power consumption monitoring
  • door and windows opening monitoring
  • alarm
  • smoke detector

The biggest drawback on z-wave is its price, every module is expensive, when you sum up all the thing I listed before, the bill was fairly high and I decided to start small instead of getting everything right away.

First order

So I ordered 4 things:

Once I received this and the LED strip, I started playing with it, simply with the plug as we didn't move yet.

It does flicker (the bypass was for the fibaro module, not the plug), but it works, first test was encouraging, first step completed.

Continue to Part 2


15 juillet 2016 12:48:03 -- tags [ home-automation , z-wave , dimmer , dimmable , led ]

0 Comments

Decipher ESP packets with Scapy

I'm currently working on implementing support for hardware cryptographic engine on a board we support at work. Currently it is not working and I was trying to debug it, thus trying to see what is happening. I tcpdumped the packets received, and tried to have a look at it. Sadly, for some reason, the SAs I added in wireshark do not match and he doesn't want to use them.

I then discovered that scapy now has support for that, and it was actually added by a colleague... Should have started there! I only found the issues on bitbucket showing how to use it so I thought it would be worth it to write a quick how-to, once got it down it is pretty straightforward, but knowing what to look for may not be obvious in the first place.

So here is a an example usage for an AES-CBC with no auth packet:

pkts = rdpcap("path/to/file.pcap")
pkts[0].show()

Here is the ESP packet:

###[ Ethernet ]###
  dst= 56:1e:7d:aa:e0:b3
  src= 68:05:ca:15:d1:99
  type= IPv4
###[ IP ]###
     version= 4L
     ihl= 5L
     tos= 0x0
     len= 140
     id= 0
     flags= DF
     frag= 0L
     ttl= 64
     proto= esp
     chksum= 0x2544
     src= 10.125.0.2
     dst= 10.125.0.1
     \options\
###[ ESP ]###
        spi= 0x22
        seq= 1
        data= '\xa7\[...]'

Then create the SA and decrypt:

sa = SecurityAssociation(ESP, spi=0x22, crypt_algo='AES-CBC', crypt_key='your_key')
res = sa.decrypt(pkts[0].getlayer(IP))
res.show()

The result showing the icmp packet:

###[ IP ]###
  version= 4L
  ihl= 5L
  tos= 0x0
  len= 104
  id= 0
  flags= DF
  frag= 0L
  ttl= 64
  proto= ipv4
  chksum= 0x2596
  src= 10.125.0.2
  dst= 10.125.0.1
  \options\
###[ IP ]###
     version= 4L
     ihl= 5L
     tos= 0x0
     len= 84
     id= 36470
     flags= DF
     frag= 0L
     ttl= 63
     proto= icmp
     chksum= 0x9805
     src= 10.200.0.1
     dst= 10.100.0.1
     \options\
###[ ICMP ]###
        type= echo-request
        code= 0
        chksum= 0x7cd1
        id= 0x9df
        seq= 0x1
###[ Raw ]###
       load= '\x9fTMW[...]'

And here you go.


31 mai 2016 13:35:34 -- tags [ scapy , python , IPsec , cipher , crypto ]

0 Comments

.vimrc

As a reference, here is my current .vimrc file, stripped from a few things specific to my current job:

" ---------------
" Start Plug
" ---------------
call plug#begin('~/.vim/plugged')

" git
Plug 'tpope/vim-fugitive'

" utils
Plug 'rking/ag.vim'         " ag fro grepping in vim
Plug 'tpope/vim-commentary' " gcc gc etc for commenting
Plug 'tpope/vim-surround'   " change surrounding chars
Plug 'tpope/vim-unimpaired' " [<space> and more
Plug 'tpope/vim-sensible'   " default settings that makes sense
"Plug 'tpope/vim-rsi'        " shell/emacs binding in insert mode
Plug 'tpope/vim-tbone'      " tmux integration in vim
Plug 'junegunn/fzf', { 'dir': '~/.fzf', 'do': './install --all' }
Plug 'junegunn/fzf.vim'

" wiki + tw
Plug 'vimwiki/vimwiki'
Plug 'tbabej/taskwiki'

" appearance
Plug 'powerline/powerline'
Plug 'noah/fu'

" Add plugins to &runtimepath
call plug#end()

" ---------------
" Behavior
" ---------------
syntax on
set nobackup
set undodir=~/etc/vim/undo
set history=1000
set undolevels=1000
set wildignore=*.swp,*.bak,*.pyc,*.class
set hidden " allow changing buffer from unsaved

set ai
set cinoptions=(0
set cindent
set nocompatible
set ruler
set hlsearch
set wildmode=list:longest "bash like completion

set mouse=vn
set clipboard=unnamed

filetype plugin on
if has("autocmd")
  autocmd BufRead *.txt set tw=78 sw=2 ts=2 et
  autocmd BufRead *.py set tw=80 sw=4 ts=4 et
  autocmd BufRead *.sh set ts=4 sw=4 et noai nocin
  autocmd Bufread *.wiki set tw=80 ts=4 sw=4 et ai
endif

" spellcheck in rst
autocmd FileType rst setlocal spell spelllang=en_us
autocmd Filetype rst set tw=80 sw=2 ts=2 et

" ---------------
" Appearance
" ---------------
set term=xterm-256color
set t_Co=256
set background=dark
set listchars=tab:▸\ ,eolcolorscheme fu

set laststatus=2
set cursorline
set nocursorcolumn
set colorcolumn=80
highlight ColorColumn ctermbg=16
highlight CursorLine ctermbg=23 term=bold cterm=bold

" Syntax error hilighting, this should be after colorscheme
syntax match Error "\(^\t.*\n\)\@<= \+"
syntax match Error "\(^ \+.*\n\)\@<=\t\+"
autocmd FileType c* syntax match Error '\%>80v.\+'

" enable powerline
set rtp+=/home/bleader/.vim/plugged/powerline/powerline/bindings/vim

" enable fzf
set rtp+=~/.fzf

" ---------------
" Binds
" ---------------

" use , for map leader, feel easier than \
let mapleader=","

set pastetoggle=<F9>

" edit/reload vimrc quickly
nmap <silent> <leader>ev :e $MYVIMRC<CR>
nmap <silent> <leader>sv :so $MYVIMRC<CR>

" allow use of ; in place of :
nnoremap ; :

" clear search hilight
nmap <silent> <leader>/ :nohl<CR>

" Error list binds
nnoremap <leader>n :cn<CR>zz
nnoremap <leader>p :cp<CR>zz

" compute line through rcpy https://github.com/bleader/rcpy
map <F4> yypV:!rcpy<CR>

" enable/disable list
imap <F1> <C-o>:set nolist!<CR>
map <F1> <ESC>:set nolist!<CR>

" moving around centering
nnoremap n nzz
nnoremap N Nzz
nnoremap * *zz
nnoremap # #zz
nnoremap g* g*zz
nnoremap g# g#zz
nnoremap [[ [[zz
nnoremap ]] ]]zz
nnoremap {{ {{zz
nnoremap }} }}zz

" debug printf in C
map <leader>dd oprintf("%s:%d\n", __func__, __LINE__);<Esc>==

" paste selection to haste
vnoremap Y <esc>:'<,'>:w !haste<CR>

" ---------------
" GIT
" ---------------
nnoremap <leader>gd :Gvdiff<CR>
nnoremap <leader>gs :Gstatus<CR>
nnoremap <leader>gl :Glog --<CR>:copen<CR>
nnoremap <leader>gL :Git l<CR>
nnoremap <leader>gg :Ggrep <cword><CR>
nnoremap <leader>ag :Ag <cword><CR>
nnoremap <leader>gc :Gcommit -s -v<CR>
nnoremap <leader>gS :Git show<CR>
nnoremap <leader><leader>gS :Git show <cword><CR>
autocmd BufReadPost fugitive://* set bufhidden=delete

let g:git_branch_status_head_current=1 
let g:git_branch_status_text="branch "
let g:git_branch_status_nogit="no git"
let g:git_branch_status_around="{}"

fu! GitReview()
    Glog --reverse origin/master.. -- .
    copen
endf
command Greview call GitReview()
nnoremap <leader>gr :Greview<CR>

" ---------------
" Cscope bindings and configuration (autoload of cscope.out)
" ---------------
nmap <leader>cs :cs find s <C-R>=expand("<cword>")<CR><CR>
nmap <leader><leader>cs :vert scs find s <C-R>=expand("<cword>")<CR><CR>
nmap <leader>cg :cs find g <C-R>=expand("<cword>")<CR><CR>
nmap <leader><leader>cg :vert scs find g <C-R>=expand("<cword>")<CR><CR>
nmap <leader>cc :cs find c <C-R>=expand("<cword>")<CR><CR>
nmap <leader><leader>cc :vert scs find c <C-R>=expand("<cword>")<CR><CR>
nmap <leader>ct :cs find t <C-R>=expand("<cword>")<CR><CR>
nmap <leader><leader>ct :vert scs find t <C-R>=expand("<cword>")<CR><CR>
nmap <leader>ce :cs find e <C-R>=expand("<cword>")<CR><CR>
nmap <leader><leader>ce :vert scs find e <C-R>=expand("<cword>")<CR><CR>
nmap <leader>cf :cs find f <C-R>=expand("<cfile>")<CR><CR>
nmap <leader><leader>cf :vert scs find f <C-R>=expand("<cfile>")<CR><CR>
nmap <leader>ci :cs find i ^<C-R>=expand("<cfile>")<CR>$<CR>
nmap <leader><leader>ci :vert scs find i ^<C-R>=expand("<cfile>")<CR>$<CR>
nmap <leader>cd :cs find d <C-R>=expand("<cword>")<CR><CR>
nmap <leader><leader>cd :vert scs find d <C-R>=expand("<cword>")<CR><CR>

if has("cscope") && filereadable("/usr/bin/cscope")
   set csprg=/usr/bin/cscope
   set csto=0
   set cst
   set nocsverb
   " add any database in current directory
   if filereadable("cscope.out")
      cs add cscope.out
   " else add database pointed to by environment
   elseif $CSCOPE_DB != ""
      cs add $CSCOPE_DB
   endif
   set csverb
endif

fu! CscopeReload()
   cscope kill 0
   !cscope -R -b
   cscope add cscope.out
endf

nmap <leader>cR :call CscopeReload()<CR>

" edit .configs
map <leader>uu ^i# <esc>$xxA is not set<esc>
map <leader>ss ^xxEld$a=y<esc>

" FZF
let g:fzf_command_prefix="Fzf"
nnoremap <leader>f :FzfFiles<cr>
nnoremap <leader>b :FzfBuffers<cr>
inoremap <expr> <c-x><c-k> fzf#vim#complete#word({'left': '15%'})

30 décembre 2015 13:26:14 -- tags [ vim , configuration ]

0 Comments

Review git-to-be-pushed changes in vim

First, this assume you use fugitive.vim. You don't? I really think you should. If you're still not convinced, have a quick look at the readme, take some time to go over one of the screencast linked there. I personally understood a lot about git index by watching the screencast Working with the git index.

I personnally like to have a look at my patches before giving them to my colleagues, there are a lot of options available to achieve this:

  • starting a gitweb
  • git log -p
  • git show of each commits
  • using tig
  • using fugitive directly in vim

As you may have guessed, my favorite is the last one, for this, I use a small function plus a bind:

fu! GitReview()                                                                 
        Glog --reverse origin/master.. -- .                                     
        copen                                                                   
endf                                                                            
command Greview call GitReview()                                                
nnoremap <leader>gr :Greview<CR>

This creates the GitReview() function, allows you to call it via :Greview or by the bind gr.

It will do a fugitive :Glog from the head of your current branch to origin/master and show it in reverse order so you start on the first commit after origin/master. This could be improved to do it based on the remotely tracked branch or accept a parameter to specify which branch to start from.

Once you called this, the first patch will be opened, likely with each files touched by the patch folded so you can review them one by one if you want. Following patches are loaded in the quickfixlist, that will likely have opened at the bottom of your vim. If you are not familiar with the quickfix list you can always :h quickfix. Thing to know, you can move forward and backward in this list be it shown or hidden, with :cn and :cp (Next/Prev).

And that's about it for this one. Hope it can be useful to someone!


30 décembre 2015 13:15:28 -- tags [ vim , git , fugitive , tips ]

0 Comments

Git command typos

Recently beorn shared two git typos on IRC, and I felt it was worth a small post about possible typos and their possible meaning, if any.

add

git ass

For when you need to get some?

bisect

git insect

Ok, this one may be a little far-fetched... still, it il likely you're searching for a bug, aren't you?

commit

git vommit

That one is for when you know your code is really dirty. (you better cleanup before pushing)

diff

git duff

At this stage, its time for a beer. Oh, yeah!

log

git lag

Maybe too much git duff, or you are just checking how far behing origin/master you are.

pull

git pill

Will you take the red or the blue pill?

push

git pish

After some git duff you'll have to take a pish, that is how it goes.

show

git dhow

Doh'! You are about to check how this commit looks like now that you've sober up, not feeling good about it, eh?

status

git statue

Just having a look at the greatness of what you're actually building, that was worth the efforts!

tag

git tab

Better keep an eye on your tab at Moe's, and make sure Barney hasn't been ordering on your tab.

That's about it, looking at a keyboard, other commands didn't gave me any idea, some are already quite far-fetched and may not have ended up in here, but here they are.


29 octobre 2013 18:41:40 -- tags [ git , command , typos , fun ]

0 Comments

Imagemagick convert and groups

I was willing to make some combined images to show off Camille Dolls, but I wanted to have good alignement and spacing between the pictures. I gave a shot at doing it by hand with Gimp but making alignement and spacing by hand was kind of a hassle. I'm pretty sure I missed some features which may have made it easier, but I decided to search another solution. Scribus seems to be made for this kind of things, but tends to be restrictive toward the final output given its main purpose being to design final paper documents.i

In the end, I decided to play with convert and other Imagemagick tools, which proved quite efficient once I got the grasp of groups.

When issuing a convert command for example, you can group set of option and they will be treated like an image. This allows more flexibility in the order to achieve your final result in less command and without, or with less intermediate images.

To stack 2 pictures on top of each other with a 10 pixels separation and a 10 pixels border around the whole thing, the steps are:

  • take first image
  • extent it by its height + 10
  • take second image
  • stack them
  • add border to whole thing

A useful command is identify which can give you output information about an image, and allows calculations to be done, to get the height + 10:

identify -format 'x%[fx:H+10]' in.png

To group commands in convert you put them between parenthesis, that needs to be escaped so your shell don't interpret them:

convert \( \( command1 \) \( command2 \) \) command_on_result out.png

So for our example:

img1=A.png
img2=B.png
convert \
    \( \ # group for resulting images stacked
        \( $img1 -extent $(identify -format 'x%[fx:H+10]' $img1) \) \ # take first pic and add 10px
        $img2 \
    \) \
    -append # stack images
    -border 10x10 \ # add border around stacked images
    out.png

A more complete thing I did was stack 2 pictures on the left, for a total of 800px, and then add another picture of 800px on the right, so to achieve this:

  • resize first picture to 395px height
  • add 10px space
  • resize the second picture to 395px height
  • stack them
  • add 10px to width to the result, you need to know the width of the resized images
  • add last picture to the right
  • add border

In term of convert:

convert \( \
  \( \
    \( -geometry x395 img1.png -extent x405 \) \ # resize + add 10px margin
    img2.png  -append \
  \) -extent 405x \ # add 10px to width
  img3.png
  \) +append \
  -border 10x10 \
  out.png

I hope this can be of some help to someone, you can check out my results.


29 juin 2013 00:22:48 -- tags [ imagemagick , convert , groups , images , combine , identify , convert ]

0 Comments