Overview
In a cloud/containerized environment, it can be useful to provide a single access point for all www traffic (e.g. to reduce the number of public IP’s required and to load-balance across many www servers). A ‘reverse proxy’ server running where the external traffic enters the system can detect which site should receive the traffic and then forward the requests to one or more concerned servers.
The reverse proxy can also manage the SSL encryption/decryption associated with HTTPS:// requests. In fact, the reverse proxy must be able to decode the initial HTTPS:// traffic, and provide the necessary SSL handshakes required by the external clients. For this reason, the reverse proxy must have the SSL certificated required for each site being served/managed.
If your internal network is trusted, the traffic passed from the reverse proxy to the site server(s) can be http only – so the reverse proxy removes any need for the www sites themselves to consider https traffic – managing only http.
AWS – stop reading now
As a caveat here – I ended up not using this setup as AWS is able to provide free certificates for your domains hosted in Route53, and AWS load-balancers can be configured with those certificates to provice exactly the same functioniality as using Nginx
Software
Many www server software packages can also act as a reverse proxy. Often NGINX is used in this role due to it’s performance and small footprint.
Nginx – Install
Initial install of Nginx is straightforward using standard package managers (yum/apt, etc).
Nginx – HTTP reverse proxy configuration
Configuration for use as a simple reverse proxy/load-balancer for standard HTTP traffic is straightforward.
A configuration file is created for each site to be proxied. Within the file, the back-end servers where the www requests will be received are defined.
Nginx – HTTPS traffic – SSL Certificates
HTTPS encrypted www traffic relies on SSL ceryificates (encryption keys) provided by a generally trusted 3rd party. Historically it has been necessary to pay for these certificates and pay for annual renewals. More recently an industry-funded organization is providing trustworthy SSL certificates free of charge. Letsencrypt ( https://letsencrypt.org/ ) provides certificates that anyone can generate, providing that they can prove ownership of the domain(s) where the certificate will be used.
The ownership is proven by showing that the person requesting the certificate is able to manipulate the DNS entries for the domain, or able to install a file to be served via www from the domain.
Nginx – HTTPS traffic – Generating Cerificates
1) Install Letsencrypt
# yum -y install yum-utils # yum-config-manager --enable rhui-REGION-rhel-server-extras rhui-REGION-rhel-server-optional # yum -y install certbot python2-certbot-nginx
2) Request A Certificate
letsencrypt certonly -a webroot --webroot-path=/srv/www/letsencrypt/ -d www.example.com
The command above will fail, as the most common ‘proof of ownership’ relies on the letsencrypt command being able to write a file, which is then retrieved from the www domain by the letsencrypt servers.
In the case of a basic reverse-proxy configuration, the letsencrypt command cannot write a file to the www server itself, as that is likely to be a completely different server from the proxy.
The solution here is to modify the Nginx configuration for the domain so that requests targeted at the specific www location where letsencrypt will retrieve the ‘proof’ file will be server from a folder on the Ngnx server, and not proxied.
3) Nginx/Letsencrypt configuration
Adjust the domain configuration as shown here – this will ensure that all requests to the location /.well-known/acme-challenge/ will be server from the local /srv/www/letsencrypt folder.
(do create this folder on your system).
location ^~ /.well-known/acme-challenge {
allow all;
root /srv/www/letsencrypt;
default_type text/plain;
}
Possible issues here will be 404 & 403 errors when trying to retrieve the test file. Put a file (test.html) in the /.well-known/acme-challenge/ folder and try to retrieve it from a browser.
Review the following:
- Possible selinux related issues. Test by using ‘setenforce 0’ and re-trying. If it works, you must work with selinux to solve the problem (don’t just leave selinux off).
# setsebool -P httpd_can_network_connect on # hcon -Rt httpd_sys_content_t /srv/www
- File ownership issues. Determine the user that is running the nginx process (ps aux) and chown your www files accordingly.
- File/Folder permission issues. Folders in the www hierarchy should have 755 permissions. Files should be 644.
find /srv/www -type d -exec chmod 755 {} \;
cd /serv/www
chmod -R 644 * - Location-related issues. Look at the nginx log (/var/log/nginx/error-log) to see specific details. IF you are seeing 404 – it may be that Ngnx is trying to server the file from an unexpected location (due to configuration issues). Use the letsencrypt debug options to ensure that it is writing the challenge file to the directory that you expect.
Once these issues are resolved, the letsencrypt command should be successful and you will find any new certificates in the /etc/letsencrypt folders.
Adding SSL to Nginx
Next you need to update the domain configuration files in Nginx to allow it to server HTTPS traffic (port 443) using the new SSL certificates.
Update config file as shown here.
Trusted internal network
In this scenario – traffic sent by the proxy to the back-end servers can remain unencrypted (http – port 80) – so the task is complete.
Untrusted internal network
Where the traffic between the reverse-proxy and back-end www servers may still be open to inspection, it is recommended that HTTPS encryption should be used on this 2nd leg.
We could use similar letsencrypt generated certificates here, but then would face the problem of key distribution across all servers in the cluster.
This can be avoided by using less secure, but still functional, self-certified keys.
References:
Where I found a lot of this information
https://wiki.alpinelinux.org/wiki/Nginx_as_reverse_proxy_with_acme_(letsencrypt) https://letsencrypt.org/