Restrict Access to Raspberry Pi Web Server Using a Certificate, Dynamic DNS and Port Forwarding

Published

Introduction: Restrict Access to Raspberry Pi Web Server Using a Certificate, Dynamic DNS and Port Forwarding

*** 29APR2018 - I AM CURRENTLY UPDATING THIS INSTRUCTABLE PLEASE CHECK BACK IN A FEW DAYS - TO DOS: GO THROUGH EACH STEP AND ENSURE IT WORKS ***

My Home Automation projects allow control via a web server over the internet. However, only authorized devices are allowed. While internet access is very useful, it is a security risk. Reduce this risk by using only HTTPS on your web server, creating a certificate authority, server-side certificate, and client-side certificate (or cert). Restrict access to only those devices that have the certificate installed.

This instructable was originally written using SSL and SHA-1. Since then, TLS has replaced SSL and SHA-256 replaced SHA-1. I have updated the instructable accordingly.

Certificates facilitate encryption and trust. Trust allows a browser to validate the authenticity of a web site. For this instructable, client-side certificates with a private key can be used by a web server to authenticate a cell phone, laptop or tablet. A server and client-side TLS private key certificate pair can prevent unauthorized users from accessing home servers.

One of my web servers allows the garage door to be opened and closed from a cell phone. For a cell phone to open the garage door, it must have the client-side certificate installed. I can put this cert on any cell phone allowed to open the garage door.

A potential problem with self-signed certs, such as those generated with openSSL, is the certificate authority (CA) may not be recognized by a browser, which may result in several threatening messages, or a browser may even block you from getting to your website.

When I started to update this instructable, my goal was to use a recognized cert authority. I was going to use Let's Encrypt. However, Let's Encrypt doesn't issue client-side, private authentication certs. Let's Encrypt only issues domain certs.

By itself, this instructable isn't very useful. This instructable is used in conjunction with larger projects. This instructable is referenced from my other instructables. It was first referenced in creating a garage door opener. I am currently creating a home automation web server using flask, python and lighttpd.

The instructable has an extensive appendix, which includes references, updates, troubleshooting and other items that might be of interest but are not required to complete the instructable.

Step 1: Gather Parts

For this instructable, I am using:

  • Raspberry Pi 3 Model B+ running DietPi
    • Any Raspberry Pi model can be used, and any Raspbian or other Raspberry Pi OS can be used
    • If you are new to the Raspberry Pi, then I recommend a CanaKit that includes a Raspberry Pi, case, power supply, HDMI cable, and MicroSD Card (Class 10)
  • AT&T GigaPower (Arris) Gateway
    • Any ISP's gateway with port forwarding may be used
    • Any circuit speed can be used
  • MacBook Pro Laptop (terminal window and ssh)
    • A Windows PC can be used. If using windows, then use putty with ssh, instead of a mac terminal window and ssh
  • Dynamic DNS
    • There are many vendors of dynamic DNS. Pick the one you like best
  • Web Server Stack with: Linux (DietPi), lighttpd (web server), sqlite3 (database), python3 with Flask
    • A stack isn't necessary for the instructable and those steps can be skipped
    • Certs can be used with any stack. A typical stack is LAMP (Linux, Apache, MySQL and PHP).
    • The stack above is lighter weight and runs well on a Raspberry Pi.
    • Flask is a web server framework in python

Step 2: My Instructable Standards

The following conventions or standards are used in my instructables:

  • I don't share my passwords, host names, IP addresses and other personal data on the internet. Text enclosed in clubs, such as, replace-this, should be replaced with your value. Of course, remove the clubs.
  • The instructable editor messes with http links converting them to short cuts. If you copy-and-paste a command with a link, then it may not work. So, text enclosed in hearts, such as, copy-and-paste-link, is a link. Copy and paste the command and copy-and-paste the link to use it.
  • I have difficulty using the instructable editor to store certain types of code, especially, HTML or CSS. So, all code is stored in github with appropriate links.
  • Each instructable has an appendix, which may include:
    • References: Contains links to web pages used or consulted in writing the instructable
    • Updates: I use my instructables and periodically update them. My intent is to track my changes and explain why I made each change
    • Troubleshooting: Either during development or afterwards, I find issues or others may report problems and I attempt to document how to troubleshoot those issues
    • Automated install script: Some of my instructables contain a long list of commands. Since I use the instructable, it is much easier to write a script to implement all of the commands, rather than copy-and-paste each command. Also, the install script is supposed to fix common errors.
  • Most of my complex instructables are based on Raspberry Pi. Instead of including steps common to many instructables, I reference an instructable for, say, setting up Raspberry Pi and its Operating System.
  • For the most part, it doesn't matter which operating system you use. It's a personal preference. Almost all of my projects run headless (no GUI). So, I prefer DietPi.
  • In general, I develop the instructable as I am making the project. Once complete, I run through the instructions again to try and ensure I haven't forgotten anything.
  • I use a MacBook instead of a windows PC for many reasons, but the primary reason is that its base OS linux. So, similar commands can be used in a Mac terminal window and also on the Raspberry Pi.
  • Improvements in the instructable editor seem to cause new and inexplicable problems. The new format (~2018) doesn't seem to be able to size images correctly, and just makes them into a mess. This instructable explains they need to be a 4:3 ratio with a canvas size of 600x450

Step 3: Setup Raspberry Pi With DietPi

A web server runs as a headless machine, which means there is no display connected to it. You can interact with the raspberry pi either via a browser on a laptop, mobile device or tablet, or a terminal window on a laptop running ssh.

DietPi has several features/advantages that make it ideal for a headless server. But you can use any Raspberry Pi distribution with this instructable, such as, raspbian.

I try to keep the instructions detailed, yet simple. The image above shows etcher, which makes burning and copying images fool proof. Etcher will format and copy the image to the MicroSD card.

I am using a Raspberry Pi 3, but these instructions should work with any Raspberry Pi. Setup Raspberry Pi 3 using DietPi by following this instructable.

After following the instructable, the following will be setup:

  • raspberry pi 3 running dietpi with
    • username = pi
    • username = root
    • password = your-password
    • hostname = your-hostname
  • avahi-daemon, which allows you to login using: ssh pi@your-hostname.local
  • sqlite3, which is a database
  • lighttpd, which is a webserver
  • python3
  • python-setuptools
  • RPi.GPIO
  • openSSH
  • python pip
  • ssmtp

Check that lighttpd is working. Open a browser on a laptop and enter the following URL:

http://your-hostname.local

Step 4: Dynamic DNS

Google uses many IP addresses, but almost everyone enters google.com into a browser's url box rather than one of google's IP address because the name is easier to remember than the IP address. A home gateway (or router) also has an IP address.

I can't remember my home IP address. Luckily, I don't need to remember my home IP address. DNS is an internet service to translate a name, such as google.com, into an IP address. I can create a name for our home web server that is easy to remember.

In general, a home IP address does not change, but it can. Also, I do not own my home IP address. Anyways, dynamic DNS is a way for me to name an IP address that I do not own, and allows me to associate a name with an IP address that may change from time-to-time.

To make it easy to access the web server from a remote location, create and use a dynamic DNS name.

With dynamic DNS, I enter dynamic-dns-name in the URL field of a browser.

Note: The IP on home routers can change without notice. However, this rarely happens. If the IP address changes, you can update the IP address through your dynamic DNS provider's website. There are many dynamic DNS vendors. I chose to use DynDNS. I have also used GoDaddy Dynamic DNS. Pick which ever vendor you like.

You will need to find your router's public IP address:

  • On an AT&T Gigabit, open a browser and enter the URL 192.168.1.254. Look under the Broadband and Status tabs and get the: router's-public-ip
  • I have an AT&T Gigabit Arris router. Google: your-internet-serivce-provider "port forwarding" model
  • So for me, my google search is: att "port forwarding" arris

You can view your hostnames and DynDNS Pro account details. Here are a few tips to get started:

  • Create your Dynamic DNS hostname (1 of 30):
    • dynamic-hostname
    • Updater Key: dynamic-ip-updater-key
  • Install an update client to ensure your hostname is always pointed to the correct IP address.
    • Download and install on MacBook (or PC)
    • AT&T's IPs rarely change, so the updater above can be run whenever it does change
    • Follow the directions:
      • Add account
      • Enter username and password: dynamic-ip-usernamedynamic-ip-password
      • Select hostname
      • Click Configure Selected Hosts
      • The host should become active

Step 5: Port Forwarding

Configure your network to allow access to your device from the Internet. You can use our Dyn Wizard for some general instructions on how to do this, use resources such as PortForward.com, or contact the device's manufacturer for assistance.

For AT&T Routers, this is a good link for port forwarding.

See Appendix: Updates for how to port forward older/different AT&T Routers I have used.

To set up port forwarding on gigabit BGW210 router follow the directions in the link above, or:

  • Open a browser
  • Go to your gateway: 192.168.1.254
  • Select the Firewall tab.
  • Enter the Device Access Code found on the side of your gateway.
  • Select NAT/Gaming.
    • If you receive a warning message, you will need to visit the AT&T Port Forwarding tool to enable Port Forwarding on your account.
    • Then, you can continue with the steps in this solution.
  • Under Manage Custom Services, select: Custom Services
    • Create an entry
      • Port 5000 is used by flask
        • service name: webserver5000
        • Global Port Range: 5000-5000
        • Base Host Port: 5000
        • Protocol: TCP
      • Return to NAT/Gaming
  • Under Application Hosting Entry
    • select the webserver5000 service and
    • assign it to the appropriate server

Once this is done, the web server will be accessible from the internet. To verify, enter:

http://dynamic-dns-name:5000

and you should see your web server. However, this isn't secure.

Step 6: Lighttpd Simple Webpage

Port forwarding for your server must be set up before doing this step.

Open a terminal window and login as

$ ssh root@host-name.local

Run the following commands to create a very simple webpage:

# rm /var/www/index.lighttpd.html
# echo '<h1>lighttpd is working</h1>' > /var/www/index.html

Open a browser and enter host IP address, and you should see: lighttpd is working

If you run into any problems, here are some pointers:

  • Location of lighttpd config file: /etc/lighttpd/lighttpd.conf
  • Check if lighttpd config is okay: lighttpd -t -f /etc/lighttpd/lighttpd.conf
  • Location of daemon: /etc/init.d/lighttpd
  • To start (or stop) daemon: /etc/init.d/lighttpd start
  • Check error logs
    • systemctl status lighttpd.service
    • journalctl -xe

Port forwarding allows a home website to be accessible from the internet and a certificate (aka cert) prevents anyone else except those devices with the cert from accessing the website.

Step 7: Creating Server/client Certificate Pair Using OpenSSL

*** STOPPED HERE ***

*** SPLIT STEP: 1 generate certs, 2 add cert to web server ***

Open a terminal window on the Mac and login to Raspberry Pi:

$ ssh pi@raspberry-pi-ip

login: ♣raspberry-pi-password

Generate Certificate Authority (CA)

Before creating server/client certificate, setup a self-signed Certificate Authority (CA), which can be used to sign the server/client certificates. Once created, the CA cert will act as the trusted authority for both your server and client certificates (or certs).

$ sudo openssl req -newkey rsa:4096 -keyform PEM -keyout ca.key -x509 -days 3650 -outform PEM -out ca.cer 

pass phrase = ♣cert-password♣ 

Generates: ca.cer, ca.key


Generate Apache server SSL key and certificate

Generate server.key:

$ sudo openssl genrsa -out server.key 4096 

Generate a certificate generation request.

$ sudo openssl req -new -key server.key -out server.req 

Use the certificate generation request and the CA cert to generate the server cert

$ sudo openssl x509 -req -in server.req -CA ca.cer -CAkey ca.key -set_serial 100 -extensions server -days 1460 -outform PEM -out server.cer 

Clean up – now that the cert has been created, we no longer need the request.

$ sudo rm server.req 


Install the server certificate in Apache

Copy the CA cert to a permanent place. We’ll need to specify our CA cert in Apache since it is a self generated CA and not one that is included in operating systems everywhere.

$ sudo cp ca.cer /etc/ssl/certs/ 

Copy the server cert and private key to permanent place.

$ sudo cp server.cer /etc/ssl/certs/server.crt 
$ sudo cp server.key /etc/ssl/private/server.key 

Activate the SSL module in Apache.

$ sudo a2enmod ssl  

Activate the SSL site in Apache

$ sudo a2ensite default-ssl

Disable the HTTP site

$ sudo a2dissite default 

Edit the config file for the SSL enabled site

$ sudo nano /etc/apache2/sites-enabled/000-default-ssl 

and add the lines below:

SSLCACertificateFile /etc/ssl/certs/ca.cer 
SSLCertificateFile /etc/ssl/certs/server.crt 
SSLCertificateKeyFile /etc/ssl/private/server.key 

Apply the config in Apache.

$ sudo service apache2 restart 

Right now if you visit your https site, you will get an SSL error similar to “SSL peer was unable to negotiate an acceptable set of security parameters.” That is good – it means your site won’t accept a connection unless your browser is using a trusted client cert. We’ll generate one now.


Generate a client SSL certificate

Generate a private key for the SSL client.

$ sudo openssl genrsa -out client.key 4096 

Use the client’s private key to generate a cert request.

$ sudo openssl req -new -key client.key -out client.req 

Issue the client certificate using the cert request and the CA cert/key.

$ sudo openssl x509 -req -in client.req -CA ca.cer -CAkey ca.key -set_serial 101 -extensions client -days 365 -outform PEM -out client.cer 

Convert the client certificate and private key to pkcs#12 format for use by browsers.

$ sudo openssl pkcs12 -export -inkey client.key -in client.cer -out client.p12 

Clean up – remove the client private key, client cert and client request files as the pkcs12 has everything needed.

$ sudo rm client.key client.cer client.req

Step 8: Add Cert to Web Server

Step 9: Add Cert to Lighttpd

*** THIS DOESN'T WORK BECAUSE NOT USING CERTBOT

*** ADD CERT TO LIGHTTPD***

*** END ADD CERT TO LIGHTTPD ***

In the browser try, your dynamic DNS name and you should get: lighttpd is working

lighttpd expects a combined cert

# cd /etc/letsencrypt/live/♣dynamic-dns-name♣/
# cat privkey.pem cert.pem > combined.pem

Enable TLS and point lighttpd to the cert.

Open a terminal window and enter the command to create and then edit a file:

# nano /etc/lighttpd/conf-enabled/letsencrypt.conf

To save and exit the editor, CTRL-o, ENTER, CTRL-x

and add:

$SERVER["socket"] == ":443" {
	ssl.engine = "enable"
	ssl.pemfile = "/etc/letsencrypt/live/♣dynamic-dns-name♣/combined.pem"
	ssl.ca-file =  "/etc/letsencrypt/live/♣dynamic-dns-name♣/fullchain.pem"
	ssl.cipher-list = "ECDHE-RSA-AES256-SHA384:AES256-SHA256:HIGH:!MD5:!aNULL:!EDH:!AESGCM"
	ssl.honor-cipher-order = "enable"
	ssl.use-sslv2 = "disable"
	ssl.use-sslv3 = "disable"
}

To save and exit the editor, type CTRL-o, ENTER, CTRL-x

Restart lighttpd

# /etc/init.d/lighttpd force-reload

In your browser go to: https://♣dynamic-dns-name♣

You will see some error messages, click through these

Step 10: Add Client-side Certificate to Devices

Import the client.p12 file into your browser.

To copy client.p12 from the Raspberry Pi to a Mac, open a terminal window and enter the command:

$ pwd

/Users/your-username

$ scp pi@raspberry-pi-ip:client.p12 /Users/your-username

Double click the file to import into the operating system’s keystore that will be used by IE and Chrome.

For Firefox, open the Options → Advanced → Certificates → View Certificates → Your Certificates and import the certificate.

For Android phones, the browser must be Chrome.

Email client.p12 as an attachment to your device.

Open the email on the Android phone and save the attachment to downloads

Go to home screen and open Settings → Security → Credential Storage → Install from device storage → Open the client.p12 file

Enter pass phrase: cert-password

For Apple phones, email the cert and double click on it, then follow the directions.

Email client.p12 and ca.cer as attachments

Step 11: ​Disable HTTP in Apache

$ sudo nano /etc/apache2/ports.conf

Comment out these lines:

NameVirtualHost *:80
Listen 80

So it looks like:

# NameVirtualHost *:80
# Listen 80
Listen 443
NameVirtualHost *:443
<VirtualHost *:443>
	ServerName ♣raspberry-pi-hostname♣
	Redirect permanent https://♣u-verse-gateway-ip♣
</VirtualHost>

Save the file (CTRL-o, ENTER, CTRL-x)

Restart Apache

$ sudo service apache2 restart

Step 12: Automatic Cert Renewal

*** THIS DOESN"T WORK, NOT USING LETSENCRYPT, BUT STILL APPLIES TO OPENSSL CERTS

Let's Encrypt certificates are only good for 90 days. Create a script and a cron tab to automatically renew the cert.

Run the command

# sudo /root/certbot/renew.sh

and add the lines

#!/bin/bash
# Renew Let's Encrypt cert
/root/certbot/certbot-auto renew

# Rebuild the cert
cd /etc/letsencrypt/live/♣dynamic-dns-name♣/
cat privkey.pem cert.pem > combined.pem

# Reload
/etc/init.d/lighttpd force-reload

Test automatic renewal for your certificates by running this command:

# sudo /root/certbot/certbot-auto renew --dry-run 

If that works, then create a cron job to automatically renew the cert

# crontab -e

and add the lines at the end

# renew daily by running the script and writing to log file
0 0 * * * sudo /root/certbot/renew.sh >> /root/certbot/renew.log

Step 13: Other Things to Add or Check

add cert to iphone - Step 5: Add Client-side Certificate to Devices

disable http on gateway Step 7: Disable HTTP on U-verse Gateway

disable http in lighttpd Step 6: Disable HTTP in Apache

*** NEED TO MOVE or identify path to FLASK to /var/www/***name***

*** NEED TO ADD FLASK stuff

*** ENSURE BROWSER DOESN'T GET STUPID WARNING MESSAGES, CHECK DELICIOUS BRAINS link in Appendix

need to add support for flask

Step 14: Disable HTTP on U-verse Gateway

HTTPS is secure and uses port 443. HTTP is insecure and uses port 80. My internet service provider's (ISP) U-Verse 2-Wire gateway provides a firewall.

Login to U-verse 2-Wire Gateway.

On MacBook, open browser and enter: ♣u-verse-gateway-ip-address♣. My gateway's IP is 192.168.1.254.

Go to: Settings → Firewall → Applications, Pinholes and DMZ

And, select ♣raspberry-pi-hostname♣

Remove applications that allow port 80 through the web server

Keep or add port 443 on HTTPS Server

Now, visit your website with the browser where you imported the client certificate. You’ll likely be prompted for which client certificate to use – select it. Then you’ll be authenticated and allowed in!

Check each device.

If they all work, then you are done!

*** FOLLOWING NEEDS TO BE MERGED WITH ABOVE

Redirect all requests on port 80 to 443.

Run the command

# nano /etc/lighttpd/conf-enabled/redirect.conf

and the lines:

$HTTP["scheme"] == "http" {
    # capture vhost name with regex condition -> %0 in redirect pattern
    # must be the most inner block to the redirect rule
    $HTTP["host"] =~ ".*" {
        url.redirect = (".*" => "https://%0$0")
    }
}

Reload lighttpd

# /etc/init.d/lighttpd force-reload

In your browser go to: http://♣dynamic-dns-name♣

You should be redirected to https

Step 15: Appendix: References

Step 16: Appendix: Updates

07APR2018: Added instructable standards and an image. Corrected some typos and incorrect reformatting added by the instructable editor.

29APR2018: Multiple changes, including changing port forwarding to my new gateway

Step 17: Appendix: Port Forwarding on My Old Gateway

Port Forwarding for AT&T 5268ac

  • 5268ac does not support loop back, so Garage Opener server is not accessible from computers on the LAN. For a MacBook to access the garage opener web server do the following:
$ sudo nano /etc/hosts
  • and add the line

192.168.1.64 server's-external-domain-name

  • Save and exit, CTRL-o, CTRL-x
$ sudo killall -HUP mDNSResponder
  • The 5268ac's built-in port forwarding rules, such as, HTTPS server, HTTP Server and Web Server do not work. To get port forwarding to work:
    • Open a browser, and go to 192.168.1.254
    • Login
    • Select Settings, Firewall, Applications, pinholes and DMZ.
    • Select the garage opener web server
    • Select User-defined
    • Click a new user-defined application
      • In Application Profile Name enter: PortForwarding
      • In Create Application Definition add each of the following and click Add to List:
        • TCP from 80 to 80, map to host port 443
        • TCP from 443 to 443 map to host port 443
        • UDP from 443 to 443, map to host port 443
        • UDP from 80 to 80, map to host port 443
    • Click back
    • Reselect your web server
    • Select User-defined
    • Add PortForwarding
    • Click Save

Port Forwarding for AT&T U-verse 2-Wire 3801HGV.

  • Open browser and go to raspberry-pi-ip
  • Password: password
  • Settings
  • Firewall
    • Choose raspberry-pi-hostname
    • Servers
      • Web Server port 80
      • HTTPS Server – port 443
    • Add
  • Save

Step 18: Appendix: Troubleshooting

Let's Encrypt:

I tried to get Let's Encrypt to work, until I realized Let's Encrypt doesn't issue private certs. I ran into issues with a GoDaddy dynamic DNS address. When running certbot-auto, I kept getting the error: "Error getting validation data". So, I ran whois on both, which pointed to something being wrong. Next, I pinged my GoDaddy dynamic DNS and instead of returning my gateway's IP address it returned some other IP address, which I assumed was a GoDaddy IP address. Pinging the DynDNS name returned my gateway's IP address. When I switched to the DynDNS name and followed the same instructions certbot-auto worked. I am not quite sure why one would work and the other would not.

Step 19: Appendix: Script

Create a script

Step 20: ​Generate Certificate - Delete ???

*** DELETE THIS PAGE ??? - MOVE TO END FOR NOW ***

*** why have this step, the next step deletes the certs created - maybe enabling ssl is needed ***

*** REDO TITLE AND SPLIT UP THE PAGE ***

*** USE NEWER OPTIONS FOR SHA256 ... ***

Open a terminal window on the MacBook and login into the Raspberry Pi using its IP address.

$ ssh pi@raspberry-pi-ip

login: raspberry-pi-password

Generate a certificate key for the web server. You'll be asked to enter a pass phrase. This pass phrase can be anything, such as, cert-password. Re-enter ♣cert-password♣.

$ sudo openssl genrsa -des3 -out server.key 1024
$ sudo openssl rsa -in server.key -out server.key.insecure 

The next command generates the certificate, and asks several questions (only the fully qualified domain name or FQDN matters):

$ sudo openssl req -new -key server.key -out server.csr 
$ sudo openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt

Copy the certificate (cert) into the SSL directory

$ sudo cp server.crt /etc/ssl/certs 
$ sudo cp server.key /etc/ssl/private 

Enable SSL

$ sudo a2enmod ssl
$ sudo a2ensite default-ssl 

Restart apache (I am not sure which one to use. So, I did both)

$ sudo /etc/init.d/apache2 restart 
$ sudo service apache2 restart

Open a browser and in the url field enter:

https://raspberry-pi-ip

A warning about the certificate not being from a trusted source will appear.

Press Continue. Make an exception and it should work.

Share

    Recommendations

    • Oil Contest

      Oil Contest
    • Clocks Contest

      Clocks Contest
    • Water Contest

      Water Contest

    2 Discussions

    0
    user
    Linwiz

    2 years ago

    Have you heard of "Let's Encrypt"? It's free and you can use it to create a certificate in step 2 that will produce a valid certificate (not self signed, so no warning about the certificate not being from a trusted source will appear)

    1 reply