Automatic domain validation TLS certificates with acme.sh and let's encrypt with own DNS servers

04 May 2017 - tsp
Last update 16 Apr 2019
Reading time 10 mins

The following is a short introduction how to use acme.sh as client for ACME services like let’s encrypt to obtain domain validated certificates for your services that I’ve written because there have been many interests how to achieve such a deployment in a potential sane way.

First I’ve to say that I’m not a big fan of using ACME for certificates. This is because without DNSSEC in place there is no protection that somebody spoofs DNS records for your domain during the validation process - there is also no protection against redirecting IP traffic to another host during the process. Then there is no validation of the entities controlling the domain. It’s simply a validation that tries to proof that you currently have control over a host or your DNS zone. Nonetheless it’s currently the only remaining free method of obtaining SSL/TLS certificates for small hobbyist or non profit services. If you want to host an E-commerce solution or something similar you should clearly take the more expensive route and buy an EV certificate from one of the known certificate authorities (they normally start at about 150$ per year if you do not require wildcards except for .onion domains).

Note that you will have to change the DNS challenge update and deploy scripts to fit your own permission system and potentially have to update your sudoers file. Other possibilities would include to post messages via AMQP to other services in case to perform zone updating and certificate deployment. This is just a quick and dirty example of how these scripts work, not a solution that I’d deploy on a real production system exactly like presented here, but it should be sufficient for any small development or test system - and also for a small hobbyists web- and mailserver.

Requirements

First you need any Unix or Linux system with sh available - this should be the case on any current Unix or Linux system.
Curl or wget has to be already installed on the system.
Second you need to install acme.sh. How to do this will be outlined below
If you use HTTP-01 validation (details see below) you require write access to your webroot from the script user, if you use DNS-01 validation you have to have full control over your DNS zone either by being able to manipulate your zone files or using one of the supported APIs. I will concentrate on how to do updates if you host your zone yourself.

How is domain validation performed?

Currently acme.sh supports two different validation techniques:

HTTP-01 Validation. This is used if you want to perform validation only via HTTP and have to ability to modify files inside your webroot.
DNS-01 Validation. This method can be used for any service, for internal and external hosts and for any hostname that can be created inside your DNS zone.

If you want to create certificates for appliances or have web applications or API gateways in place that don’t allow modification of your webroot (or you don’t want acme.sh to access your webroot for whatever reasons) you have to take the DNS-01 route and deploy scripts via custom deployment scripst, which is exactly what will be described below.

Setup of acme.sh

There is the possibility of piping the script from get.acme.sh directly into your sh but this is something i would never recommend. In case the hosting service would have been compromised or in case your IP connection gets redirected it would be possible to download and install malware - if you run these commands against recommendation as root this may compromise your whole system. In case you don’t mind you can simply execute one of the following lines:

curl https://get.acme.sh | sh
wget -O -  https://get.acme.sh | sh

First create an user that has necessary permissions on your system (you can of course also run everything following as root, but i don’t really recommend doing anything as root - some systems may even don’t have an superuser any more). If you want to install the script as another user than your current one use appropriate sudo (for example sudo -u certbot command) as prefix on any of the following commands.

Download acme.sh by cloning it from the git repository:

git clone https://github.com/Neilpang/acme.sh.git

If you want you can inspect the scripts now before executing the automatic installation procedure.

cd ./acme.sh
./acme.sh --install

This will install acme.sh into ~/.acme.sh/, register a crontab entry for automatic certificate renewal and create an shell alias so that you can run acme.sh simply by typing “acme.sh” anywhere. If you want to run the script without the alias it can be executed with ~/.acme.sh/acme.sh.

DNS hook, automatic serial increment and DNSSEC zone signing

The following section assumes that you’ve some kind of setup where you are editing your DNS zone files by hand or created them automatically by usage of scripts and then sign them (eventually using a shellscript like described in my previous note about simple DNSSEC zone signing)

acme.sh uses hook scripts for manipulating DNS. These are installed in ~/.acme.sh/dnsapi/. They are realized as simple shellscripts so one can really extend acme.sh in an easy way without tampering with the code. To support multiple challenges without resigning the zone too often one can modify the acme.sh script like described later on. They are always named dns_XXX.sh where XXX is the name of the API. Every script contains at least two functions dns_XXX_add() and dns_XXX_rm() where XXX has to be the same as in the filename. The following code snippets will call the script dns_tspi.sh.

To write acme challenge responses into the zonefile i’m simply using an include inside the zonefile itself:

$include "./example.com/acme.master"

After each update we have to increment the serial. Because i know that my generated zonefiles always contain the serial on line 2 the script is pretty simple. The SOA of my zonefiles always have the same structure:

$TTL 1800                               ; 30 Minuten TTL
example.com.  IN SOA ns1.example.com. dnsadmin.example.com. (
   2015070504 ; Serial
   3600 ; Slave refresh (1h)
   7200 ; Slave retry (2h)
   3600000 ; Expire (1000h)
   120  ; Negative Caching TTL
)

This allows for a simple increment script serial.sh which I’m storing in my /usr/local/etc/named/master/ directory where I’m also keeping all master zonefiles with associated directories for DNSSEC keys and acme.master files:

#!/bin/sh

# Increment serial in zonefile (IF serial is contained on line 2 ...)
#
#       serial.sh zonename zonefile tempfilename

if [ "$#" -lt 3 ]; then
        echo "Missing arguments"
        exit 200
fi
ZONENAME=${1}
ZONEFILE=${2}
TEMPFILE=${3}

echo "Incrementing serial for zone ${ZONENAME} contained in ${ZONEFILE}. Using temporary ${TEMPFILE}"

SERIALLINE=`cat ${ZONEFILE} | head -n 3 | tail -n 1 | cut -d ';' -f 1 `
NEXTSERIAL=`expr ${SERIALLINE} + 1`

if [ ${SERIALLINE} -lt 2014000000 ]; then
        echo "Serial seems to be suspicious"
        exit 200
fi
echo "Incrementing serial from ${SERIALLINE} to ${NEXTSERIAL}"

cat ${ZONEFILE} | head -n 2 > ${TEMPFILE}

echo "     ${NEXTSERIAL}           ; Serial (Auto Incremented)" >> ${TEMPFILE}

cat ${ZONEFILE} | tail -n +4 >> ${TEMPFILE}

echo "Checking zone"
set +e
LASTLINE=`named-checkzone ${ZONENAME} ${ZONEFILE} | tail -n 1`
set -e

if [ ${LASTLINE} = "OK" ]; then
        echo "Ok"
        cp ${TEMPFILE} ${ZONEFILE}
else
        echo "Failed"
        exit 200
fi

To perform signing I’m using an extra script called signnow.sh. This script simply performs the actions described in my previous note on DNSSEC with bind (and manual signing).

#!/bin/sh
set -e

cd /etc/namedb/master
# Increment serials
./serial.sh example.com example.com.master serialtemp.master

# Sign zone
dnssec-signzone -a -t -o example.com -k ./example.com/Kexample.com.+005+51601 example.com.master ./example.com/Kexample.com.+005+22950
mv dsset-example.com. example.com/
mv keyset-example.com. example.com/
chown bind example.com.master.signed
echo "Zone example.com resigned"

# Restart named
rndc reload

If you have rndc not configured or have an configuration that does not allow rndc to perform the reload you could also kill and restart the server:

killall named
/usr/local/etc/rc.d/named start

while true; do
   /usr/local/etc/rc.d/named status
   if [ “$?” -eq 0 ]; then
      echo “Named running again”
      break
   fi
   echo “Waiting for named to start. Retrying in 10 seconds”
   sleep 10
   /usr/local/etc/rc.d/named start
done

Note that this script of course enters an endless loop in case there are any configuration errors and is not capable of signal or detect this situation. The hook script now only has to write the responses for the DNS-01 Challenges into the acme.master file for the given zone and call the signing script and call our signing script. This is the script contained in ~/.acme.sh/dnsapi/my_tspi.sh:

#!/bin/sh

dns_tspi_add() {
   fulldomain=$1
   txtvalue=$2
   echo “Adding challenge for ${fulldomain}”
   grep -v ‘^${fulldomain}’ /usr/local/etc/namedb/master/example.com/acme.master > /usr/local/etc/namedb/master/example.com/acme.master.new
   mv /usr/local/etc/namedb/master/example.com/acme.master.new /usr/local/etc/namedb/master/example.com/acme.master
   echo “${fulldomain}. IN TXT \”${txtvalue}\”” >> /usr/local/etc/namedb/master/example.com/acme.master

   # Now sign now OR we have to modify the acme.sh script
   # as described in the next section
   /usr/local/etc/namedb/master/dosign.sh

   return 0
}

dns_tspi_rm() {
   fulldomain=$1
   txtvalue=$2 
   echo “Removing challenge for ${fulldomain}”
    grep -v ‘^${fulldomain}’ /usr/local/etc/namedb/master/example.com/acme.master > /usr/local/etc/namedb/master/example.com/acme.master.new    mv /usr/local/etc/namedb/master/example.com/acme.master.new /usr/local/etc/namedb/master/example.com/acme.master 

   # There is no need to perform the resign now because this happens
   # on a regular basis anyways and lingering signatures are only relevant
   # if we have some “confidential” hostnames which should not be discoverable
   # via DNSSEC zonewalking - in this case they should not enter public DNS
   # anyway ...
}

Requesting a certificate

To request a certificate one can now use the issue command:

acme.sh --issue --dns dns_tspi -d hostname1.example.com -d example.com

One can specify as much aliases as one needs with additional -d parameters. After the issue has been requested the acme.sh script periodically updates the scripts and later invokes deploy scripts for the given certificates.

If one wants to test before issuing real certificates one can use the --staging parameter to use the let’s encrypt testing environment which is not affected by usage limits.

Deploy Scripts

Deploy scripts contained in ~/.acme.sh/deploy have a similar layout as DNS hook scripts. They are called after a new certificate has been issued. There exact operation depends on your local server configuration, how you want to deploy the certificate, etc.

A simple script that updates keys for an Apache vhost may look like the following:

#!/bin/sh
# Filename: example.sh

example_deploy() {
   _cdomain=”$1”
   _ckey=”$2”
   _ccert=”$3”
   _cca=”$4”
   _cfullchain=”$5”
   DEPLOY=0
   if [ ! -e /usr/www/www.example.com/conf/le_${_cdomain}.cert ]; then
      DEPLOY=1
   else
      if [ /usr/www/www.example.com/conf/le_${_cdomain}.cert -ot ${_cdomain} ]; then
         DEPLOY=1
      fi
   fi

   if [ “${DEPLOY}” -lt 1 ]; then
      echo “Not deploying. Current deployed certificate is newer than staged certificate”
   else
      echo “Deploying certs for ${_cdomain}”
      echo “   Key file: ${_ckey}”
      echo “   Cert file: ${_ccert}”
      echo “   Fullchain: ${_cfullchain}”
      # Maybe use sudo if you require a privileged or other users command!
      cp ${_ckey} /usr/www/www.example.com/conf/le_${_cdomain}.key
      cp ${_ccert} /usr/www/www.example.com/conf/le_${_cdomain}.ccert
      cp ${_cfullchain} /usr/www/www.example.com/conf/le_${_cdomain}.chain
      # Add commands to patch permissions if required (and it should be ...)

      echo “Reloading apache configuration”
      sudo -u apache apachectl graceful
   fi
   return 0
}