chester's blog

technology, travel, comics, books, math, web, software and random thoughts

Voice control for a non-smart TV (with Google Home, a Raspberry Pi, LIRC, nginx, Lua and IFTTT)

10 Dec 2017

Despite my privacy concerns, I could not resist the low price of the Google Home Mini. It is really convenient to control the ChromeCast with it, but turning my (non-smart) TV on/off, or switching the input between different devices still required reaching the remote…

…until I hacked a bit!

In a nutshell: IFTTT turns Google Home commands into HTTPS requests towards a Raspberry Pi. There, nginx triggers some Lua code that runs LIRC, which generates IR signals into a transistor that amplifies them to two IR LEDs. Complicated, but works!

Hardware: Raspberry Pi + IR LED (+ a few extras)

TVs with HDMI-CEC can be controlled and input-switched by Google Home via ChromeCast. Mine doesn’t, so I resorted to something that could duplicate the (IR) light signals sent by the remote control.

Sure, I could just add an IR LED to an Arduino or a Raspberry Pi (with a resistor, just like we do with regular LEDs on those “blinking LED” tutorials). But this simple circuit strengthens the signal just by adding a transistor and second resistor. I liked that, and went with it for my initial breadboard experiment:

initial prototype

Once I realized I also wanted to control the sound bar, the amplifier helped: just added a second LED in parallel, and it worked just as well as the single one, no changes needed to the circuit.

The final version was soldered on a tiny piece of protoboard - small enough to fit inside the Raspberry Pi case. The IR LEDs were connected with 22 AWG black wire, which stays in place when twisted, and blends well with the black TV and sound bar.

final, multi-led version

IR programming: LIRC

One reason I chose a Raspberry Pi over an Arduino was LIRC, an open-source IR remote control software. Took some time to install because most tutorials don’t include the (Raspbian-specific) step of editing boot/config.txt, in which the following line must be uncommented and point to the pin connected to the 10K resistor:

dtoverlay=lirc-rpi,gpio_out_pin=22

Another hurdle: LIRC’s remotes database did not include either my TV or my sound bar. Had to create configuration files for them.

To do so, I added an infrared receiver (TSOP38238) to another pin (pinout here), adding it as a gpio_in_pin at all places that I previously added the IR led as gpio_out_pin.

With this new (and temporary) hardware setup, irrecord guided me into pressing each button on the remote and generating a config file. The TV remote was straightforward, but the IR bar one required raw mode (-f), then converting the results to regular codes (-a).

It was a bit tedious, so I sent the files to the database maintainer for the benefit of future owners of those devices. Until they actually publish it (or in case that they never do), here are my remote code files:

With these in place, I could submit commands such as:

irsend SEND_ONCE Sharp_LCDTV-845-039-40B0 KEY_MENU

and see the menu appearing on the TV, just as if I had pressed the MENU key.

final, multi-led version

The source switching was another challenge: my TV requires entering an input menu, navigating with arrows and pressing enter. But I could pack the sequence of irsend commands (with proper sleeps to give the slow TV time to react) in a script, like this:

irsend SEND_ONCE Sharp_LCDTV-845-039-40B0 INPUT
sleep 1.5
irsend SEND_ONCE Sharp_LCDTV-845-039-40B0 KEY_UP
sleep 0.5
irsend SEND_ONCE Sharp_LCDTV-845-039-40B0 KEY_UP
sleep 0.5
irsend SEND_ONCE Sharp_LCDTV-845-039-40B0 KEY_ENTER

It wasn’t over yet: the number of KEY_UPs for a given input depends on which one is already selected. The final solution involved using the KEY_PC button (which switches the VGA input) and using that as a starting point. Not super fast, but works.

Opening (safely) to the outside world: nginx

initial prototype

The other reason I chose a Raspberry Pi for the project was that the commands from IFTTT would come as web requests.

Security is always a concern with outside requests, so trusty nginx was the tool for the job. Once you properly secure your Raspberry Pi, it can be installed with:

sudo apt-get install nginx

I had to forward ports 80 and 443 from my router to the Pi (also giving it a permanent IP lease), then opening the same ports on ufw (you did enable the Linux firewall when you secured it, right?), allowing requests to my current IP to reach nginx.

Adding a dynamic DNS service (such as no-ip) and an ssl certificate (either a self-signed one or - my favorite option - a fully trusted one with Let’s Encrypt) allowed the the Pi to respond at a fixed URL with full TLS (“https”) encryption.

Hardening the configuration with something like this (in /etc/nginx/sites-enabled/my.domain, where my.domain is the domain from the dynamic DNS provider) makes both myself and security checkers happy:

server {
  listen 80;
  listen 443 ssl;
  listen [::]:443 ssl;

  ssl_certificate /etc/letsencrypt/live/my.domain/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/my.domain/privkey.pem;

  ssl_session_timeout 1d;
  ssl_session_cache shared:SSL:50m;
  ssl_session_tickets off;

  ssl_protocols TLSv1.1 TLSv1.2;
  ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256';
  ssl_prefer_server_ciphers on;

  # HSTS (ngx_http_headers_module is required) (15768000 seconds = 6 months)
  add_header Strict-Transport-Security max-age=15768000;

  # OCSP Stapling ---
  ssl_stapling on;
  ssl_stapling_verify on;

  error_page 500 502 503 504 /500.html;
  client_max_body_size 4G;
  keepalive_timeout 10;

  if ($request_method !~ ^(GET|HEAD|PUT|PATCH|POST|DELETE|OPTIONS)$ ){
    return 405;
  }
  ...

Triggering the IR from the web: Lua

In the Apache days, I’d set it up CGI scripts and be done with it, but nginx doesn’t support that. To my surprise, Raspbian Stretch Lite came with Lua - a nifty scripting language, perfect for the job at hand! All I had to do was to add this module:

sudo apt-get install libnginx-mod-http-lua

and I could add some scripting in the same config file where I set up the server:

location /remote {
  lua_need_request_body on;
  content_by_lua_block {
    local args, err = ngx.req.get_post_args()
    if not args then
      return
    end
    if args["secret-key"] ~= "some_generated_secret" then
      return
    end
    if args["action"] == "tv_power" then
      os.execute("irsend SEND_ONCE Sharp_LCDTV-845-039-40B0 KEY_POWER");
...
    elseif args["action"] == "soundbar_volume_up" then
      os.execute("irsend SEND_ONCE Insignia_RMC-SB314 KEY_VOLUMEUP");
...
    end
  }
}

With this, an HTTPS POST message containing the secret-key and the action triggers the remote. The secret key will be fully encrypted within the message payload, keeping it safe.

It can be tested with curl:

curl -d "secret-key=some_generated_secret&action=tv_power" https://my.domain/remote

Putting it all together: IFTTT

IFTTT example

IFTTT (“IF This, Then That”) is a neat website that executes actions (“that”) in response to triggers (“this”).

You can tell it to do things like “if I receive an email from this address, then post the contents on Facebook”, or “if the stocks for company XYZ change, add the value to a Google Spreadsheet”. It is only limited by the available services (but there are a lot of them).

By using Google Assistant as a trigger and Maker Webhooks as an action, it was super easy to trigger the HTTPS endpoint defined above by a voice command.

I just created a new applet (“applet” is how IFTTT calls the “if this then that” statements), picking Google Assistant as “this”. After a quick, first-time-only setup it allowed me to tell what phrase(s) will trigger the command and what Google Home should say.

For “that”, I chose Webhooks, using application/x-www-form-urlencoded for Content Type, POST for method and the arguments of the curl -d command above (minus quotes) as body and URL, respectively.

Created an applet for each desired command, and that was it, done. Look ma, no hands!

Comments


Austin Acton

This was very helpful, and encouraging. There is a nice little IR daughterboard available on Amazon that I might try out if I can't get CEC working with my ca. 2010 Samsung LCD. And Home Assistant, which runs on RPi might be able to make some of what you did easier to install, if way more bloated. Then there's Stringify too. So much to explore. Thanks!

chesterbr

Hi! Glad if any of this can be useful. Indeed, running Home Assistant on the RPi seems like an interesting alternative! Also didn't know about Stringify, another cool tool to keep in mind, thanks for sharing, and let us know how your experiment ends up! All the best!


oneil bogle

oneil bogle

Great tutorial i have a google aiy kit that i would like to experiment with but in this instructions u added IR LEDS are they required

chesterbr

Thanks! Not sure what the Google kit provides, but you only need it if you don't have any other way of controlling your TV. Good luck with your experiment, let me know!


Bryan Natera

Bryan Natera

This is such a great thread! I'm trying to implement this on my Magic Mirror project. My goal is to make my mirror entirely Google integrated. However, I can't seem to understand the configuration process. Can you please explain the "my.domain" part. Is this a web domain that you own? I don't understand how the IR blaster is triggered. I'm fairly new to coding. Please bare with me. :(

Bryan Natera

Bryan Natera

I'm currently using a Raspberry Pi 3 b+ running Rasbian. (If that helps... :c )
I already recorded my TV remote buttons and just stuck on the actual triggering process.

Thank you,
Bryan

chesterbr

Hello Bryan!

First of all "my.domain" there is the (example of a) free name I registered with https://www.no-ip.com (so that IFTTT can reach my network). So if, say, you go there and register `coolname.ddns.net` and want to set up nginx to respond to that, name your file `/etc/nginx/sites-enabled/coolname.ddns.net` and also replace the "my.domain" occurrences with `coolname.ddns.net`

The 3 B+ should be super fine (I'm using a *way* olderRaspberry Pi 1 B), and I also use Raspbian, so your setup is good.

If you recorded the remote commands with the `irrecord` utility, you must have saved them a file ending with `.lirc.conf` (see http://www.lirc.org/html/ir... "Description" session for details). Copy that file to the `/etc/lirc/lircd.conf.d/` directory and restart lirc (easiest way is to reboot your Pi via Raspbian), and you should be able to send commands using:

`irsend SEND_ONCE device_name command`

where `device_name` is the name you supplied to `irrecord` (it will be on the `name` section of the config file it generated) and `command` is the key you want to send (they are all listed after `begin codes` in that file, e.g., `KEY_POWER`)

Let me know if you have any other questions!

Bryan Natera

Bryan Natera

First of all, thank you so much for the reply! I really appreciate your assistance on this project I'm working on. :)

Ohh I see. So, for example, if I change the my.domain to "mirror.ddns.net", is it correct in the /etc/nginx/sites-enabled/mirror.ddns.net file I have in the picture below??

Also, is an SSL certificate required for this? If so, how can I set it up? I used your link and I had to set it up with Certbot, however, I'm not quite sure what to do after. Am I done from there, or is there something else I need to do?

https://uploads.disquscdn.c... https://uploads.disquscdn.c...

chesterbr

Yup, the replacement is correct. You will need a certificate to ensure communications are encrypted (otherwise someone could grab your secrets and control your outlets - not much incentive in that, but there is the possibility) - not sure if it *must* be a "real" one (from the likes of letsencrypt) or a self-signed would do (depends on IFTTT playing nice).

You should be done if you can can access https://mirror.ddns.net (or whichever is your name) from a browser outside your network - even if it shows some standard nginx "you need to configure this" message. A self-signed cert will show some "!"s on the browser bar but should still be good, but if you have done the certbot setup, https://www.ssllabs.com/ssl... will both check that your RPi is accessible and secure, from an SSL point of view.


chesterbr

Hi! Unfortunately a *local* IP won't do, because those IPs are only unique inside your local network (e.g., my computer may have the same local IP as yours, and IFTTT, being outside our homes, needs to know which of us to talk to).

But you **can** use your router's public IP address (sites like https://whatismyipaddress.com/ will tell which one it is) as long as you configure the router to redirect that IP and designated port to your local IP. It will work until your IP changes (which is why I suggest registering a dynamic DNS), but it should be enough for a local test.


geo667

Hello! Many thanks for this post.
I try to implement it on my raspberry pi3 B+.

I make it step by step so I first try to implement nginx to receive http request instead of https.
So, in /etc/nginx/sites-available/default file, I add :


server {
listen 80;

# HSTS (ngx_http_headers_module is required) (15768000 seconds = 6 months)
add_header Strict-Transport-Security max-age=15768000;

error_page 500 502 503 504 /500.html;
client_max_body_size 4G;
keepalive_timeout 10;

if ($request_method !~ ^(GET|HEAD|PUT|PATCH|POST|DELETE|OPTIONS)$ ){
return 405;
}

location /remote {
lua_need_request_body on;
content_by_lua_block{
local args, err = ngx.req.get_post_args()
if not args then
os.execute("echo noargs > /home/pi/Desktop/lua_out");
end
if args["action"] == "action1"
then
os.execute("echo action1 > /home/pi/Desktop/lua_out");
end
}
}
}

I restart nginx and try this configuration with this url directly inside chromium raspberry browser :
http://127.0.0.1/remote/

Unfortunately I never obtain a lua_out file on my Desktop...
I try with this configuration before with a file /home/pi/Desktop/remote/index.html that contain small hello world write in HTML and it was working, so I guess I have an error on lua script ?


server {
listen 80;
listen [::]:80;

root /home/pi/Desktop/;

location /remote {
try_files $uri $uri/ =404;
}
}

Can you help me a little please ? I'm deadlocked...

chesterbr

Hi! Of course! I did some tests here, and found a couple things:

First of all, I'm not sure you can simply run `echo` (an internal shell command) or do a console redirect (`>`) directly with `os.execute` - I may be wrong (my Lua-fu is not really good), but I believe it will try (and fail) to start a process with an executable named `echo`.

To fix that, we need to run a shell (e.g.,) `/bin/bash`, asking it to execute the command (`-c`, for bash). I'd also single-quote the command, include the redirect inside and change it from `>` to `>>`, so you can capture the output of multiple runs)

The other thing: I'm not entirely sure the nginx process (which runs as the ultra-restricted `www-data` user, to mitigate the damage that any privilege escalation might cause) would be able to write to your `pi` user's `Desktop`, so instead I'd write to `/tmp`, which often allows anyone to read and write.

So I've put this modified version in my `default` file:


location /remote {
lua_need_request_body on;
content_by_lua_block{
local args, err = ngx.req.get_post_args()
if not args then
os.execute("/bin/bash -c 'echo noargs >> /tmp/lua_out'");
end
if args["action"] == "action1" then
os.execute("/bin/bash -c 'echo action1 >> /tmp/lua_out'");
end
}
}

And once I call it with `action=action1` in the body (becuase we are using `get_post_args`, it won't work for regular querystring):


curl -i tv-raspberrypi/remote --data "action=action1"

we get "action1" added to `/tmp/lua_out`!

Hope this gets you unstuck - in any case, keep this thread posted!


dantheman1988

dantheman1988

This tutorial is great! I've got it also right, but for some reason I get a 404 return on the applet and when running curl --insecure action=tv_volume_up https://my.domain/remote (I've used a self-signed certificate so curl -d doesn't like it). I can access https://my.domain which gives me:

"If you see this page, the nginx web server is successfully installed and working. Further configuration is required.
For online documentation and support please refer to nginx.org.
Commercial support is available at nginx.com.
Thank you for using nginx."

My /etc/nginx/sites-enabled/my.domain contents are as follows:

Server {
listen 80;
listen 443 ssl;
listen [::]:443 ssl;

ssl_certificate /etc/ssl/certs/nginx-selfsigned.crt;
ssl_certificate_key /etc/ssl/private/nginx-selfsigned.key;

ssl_session_timeout 1d;
ssl_session_cache shared:SSL:50m;
ssl_session_tickets off;

ssl_protocols TLSv1.1 TLSv1.2;
ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH';
ssl_prefer_server_ciphers on;

# HSTS (ngx_http_headers_module is required) (15768000 seconds = 6 months)
add_header Strict-Transport-Security max-age=15768000;

# OCSP Stapling ---
ssl_stapling on;
ssl_stapling_verify on;

error_page 500 502 503 504 /500.html;
client_max_body_size 4G;
keepalive_timeout 10;

if ($request_method !~ ^(GET|HEAD|PUT|PATCH|POST|DELETE|OPTIONS)$ ){
return 405;
}
}
...

location /remote {
lua_need_request_body on;
content_by_lua_block {
local args, err = ngx.req.get_post_args()
if not args then
return
end
if args["secret-key"] ~= "some_generated_secret" then
return
end
if args["action"] == "tv_volume_up" then
os.execute("irsend SEND_ONCE Samsung_BN59-01175N KEY_VOLUMEUP");
...
elseif args["action"] == "tv_volume_down" then
os.execute("irsend SEND_ONCE Samsung_BN59-01175N KEY_VOLUMEDOWN");
...
end
}
}

I'm really stuck as to what could be causing the 404 error. Any ideas?

Thanks,

Dan

chesterbr

Hello Dan! Not sure what could be, but the first thing that comes to mind: --insecure just skips the certificate check. You may still need "-d" followed by the parameters, that is, something like:


curl --insecure -d "action=tv_volume_up&secret-key=some_generated_secret" https://my.domain/remote

Although TBH failure on that should likely only generate a blank 200 page, not 404. So I'd start debugging by removing some pieces. E.g., replacing the location /remote { ... with a simpler thing like:


location /remote {
content_by_lua_block {
os.execute("/bin/bash -c 'echo hello >> /tmp/lua_out'");
}
}

If a simple request to /remote writes "hello" to the /tmp/lua.out file, it is indeed something with parameter management on the code, or the curl call.

If it doesn't, nginx isn't triggering the lua interpreter.

In either case, it gives a more specific place to look at. Other things that can give you insight:

- Does http (instead of https) work? Since you have a listener on port 80, you could try http (and rule out/nail in any certificate/https problem);
- Do other HTTP verbs work? (curl default is GET, adding -d changes it to post, but you can specify -x VERB. I'd try each command (including your current setup) with GET, POST and PUT to see if I get any difference on them (if you do, parameter handling is the culprit)
- Finally, curl is my go-to tool for these things, but other tools may give you better insight (the browser is usually opaque, but the Network tab inside Firefox/Chrome's debugger gives all the details). Ah, curl also have --verbose, which will tell you exactly what was sent/received and often reveals details.

Good luck and let us know how it worked for you!

dantheman1988

dantheman1988

So I started from scratch again with a fresh install of Rasbian Stretch. I made some modifications to the /etc/nginx/sites-allowed/my.domain file to streamline things as per the self-cert guide referenced. It now reads:


server {
listen 80;
listen [::]:80;
server_name my.domain;
return 301 https://$server_name$request_uri;
}

server {

#SSL configuration

listen 443 ssl;
listen [::]:443 ssl;
server_name my.domain;
include snippets/self-signed.conf;
include snippets/ssl-params.conf;

# HSTS (ngx_http_headers_module is required) (15768000 seconds = 6 months)
add_header Strict-Transport-Security max-age=15768000;

if ($request_method !~ ^(GET|HEAD|PUT|PATCH|POST|DELETE|OPTIONS)$ ){
return 405;
}

location /remote {
lua_need_request_body on;
content_by_lua_block {
local args, err = ngx.req.get_post_args()
if not args then
return
end
if args["secret-key"] ~= "some_generated_secret" then
return
end
if args["action"] == "tv_power" then
os.execute("irsend SEND_ONCE Samsung_BN59-01175N KEY_POWER");
...
elseif args["action"] == "tv_volume_up" then
os.execute("irsend SEND_ONCE Samsung_BN59-01175N KEY_VOLUMEUP");
...
end
}
}
}

And I am able to access https://my.domain on my local network and externally with the generic message from nginx

If you see this page, the nginx web server is successfully installed and working. Further configuration is required.

For online documentation and support please refer to nginx.org.
Commercial support is available at nginx.com.

Thank you for using nginx.

If I navigate to https://my.domain/remote then the browser downloads a file called "remote". The same also happened when I tried your suggestion of something simple like:


location /remote {
content_by_lua_block {
os.execute("/bin/bash -c 'echo hello >> /tmp/lua_out'");
}
}

I ran curl -k --verbose "secret-key=some_generated_secret&action=tv_power" https://my.domain/remote and received the response:


* Rebuilt URL to: secret-key=some_generated_secret&action=tv_power/
* Could not resolve host: secret-key=some_generated_secret&action=tv_power
* Closing connection 0
curl: (6) Could not resolve host: secret-key=some_generated_secret&action=tv_power
* Trying (domain IP)...
* TCP_NODELAY set
* Connected to my.domain (IP address) port 443 (#1)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: C=**; ST=***; L=***; O=***; OU=***; CN=my.domain; emailAddress=***
* start date: Mar 8 16:04:04 2019 GMT
* expire date: Mar 7 16:04:04 2020 GMT
* issuer: C=***; ST=***; L=***; O=***; OU=***; CN=my.domain; emailAddress=****
* SSL certificate verify result: self signed certificate (18), continuing anyway.
> GET /remote HTTP/1.1
> Host: my.domain
> User-Agent: curl/7.52.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx/1.10.3
< Date: Fri, 08 Mar 2019 17:34:05 GMT
< Content-Type: application/octet-stream
< Transfer-Encoding: chunked
< Connection: keep-alive
< Strict-Transport-Security: max-age=63072000; includeSubdomains
< X-Frame-Options: DENY
< X-Content-Type-Options: nosniff
<
* Curl_http_done: called premature == 0
* Connection #1 to host my.domain left intact

The only thing I can think of is that nginx isn't triggering the lua interpreter, but I'm really stumped as to how I could fix this.

Apologies for the MASSIVE post, just wanted to include all of the info!

Cheers,

Dan

chesterbr

Sorry for the late reply. When you run the small version, does the file get created? I mean, if you ssh into the pi and type

cat /tmp/lua_out

what do you get?

chesterbr

asking that because it seems that *something* happens when you hit /remote... and that is crucial for us to know if there is an nginx/lua config issue or something else going on