Troubleshooting Let's Encrypt Errors
Manually debugging /.well-known/acme-challenge/letsencrypt_12345
When trying to install a Let's Encrypt certificate, if you're hitting this error for :
Getting challenge for domain.com from acme-server...
Error:
http://domain.com/.well-known/acme-challenge/letsencrypt_1472046285
is not reachable. Aborting the script.
Please make sure /.well-known alias is setup in WWW server.
we can manually test this to see what's going on.
First, ensure that you have letsencrypt=1 in your directadmin.conf. You can use this guide to help you do this.
Next, we'll want to confirm the Alias /.well-known has been added to the file
/etc/httpd/conf/extra/httpd-alias.conf
If this is not set, add it using
da build rewrite_confs
and the build script should add it into the httpd-alias.conf file for you.
- To manually test things, create a file like this:
echo "test" >> /var/www/html/.well-known/acme-challenge/test.txt
and then test it via:
http://domain.com/.well-known/acme-challenge/test.txt
which should show you "test" in the output if all is working.
- If that works, but you still get the error, then try out exactly what curl is running. Login to ssh as root, and type:
curl -I -L -k -X GET http://domain.com/.well-known/acme-challenge/test.txt
where we'd be testing with the test.txt, while the letsencrypt.sh is testing on the letsencrypt_12345678, which is just a unix timestamp used to keep the file name moderately unique for the test.
Let's Encrypt: Manually forcing the automated renewal to test for errors
Sometimes you might want to force DirectAdmin to think a Let's Encrypt certificate needs to be renewed. We can do this by manually changing the certificate's creation time file to an older time (e.g., 61 days prior):
Assume:
Directadmin User:
Domain:
and that the Let's Encrypt SSL is currently valid with a renewal time somewhere in the future.
Once that info is filled out, type:
cd /usr/local/directadmin
echo `date +%s --date='61 days ago'` > "data/users/user/domains/domain.com.cert.creation_time"
/usr/local/directadmin/directadmin taskq --run 'action=rewrite&value=letsencrypt&domain=domain.com' --debug 3100
Let's Encrypt certificate creation works, but takes 15 minutes
Report that if your system **has IPv6 **present, but is not on a valid IPv6 network, Let's Encrypt calls may try and used IPv6, and timeout after 15 minutes, then using the IPv4.
Solution for the DirectAdmin setup is to edit:
/usr/local/directadmin/scripts/letsencrypt.sh
and add the -4 option to all curl calls.
The simplest way to do this is to add -4 into the ${CURL} variable, after it's been checked for existence. Find this code:
CURL=/usr/local/bin/curl
if [ ! -x ${CURL} ]; then
CURL=/usr/bin/curl
fi
and right below it, add a line, so that it looks like this:
CURL=/usr/local/bin/curl
if [ ! -x ${CURL} ]; then
CURL=/usr/bin/curl
fi
CURL="${CURL} -4"
so that all calls using ${CURL} will have the -4 option added.
Note, the file will be rewritten with the next letsencrypt update. Better to fix IPv6 or disable it completely.
Nonce is empty. Exiting. dig output of acme-v01.api.letsencrypt.org
When creating a certificate, if you get the following error:
Getting challenge for server.yourhost.com from acme-server...
Nonce is empty. Exiting. dig output of acme-v01.api.letsencrypt.org:
api.letsencrypt.org.edgekey.net.
e981.dscb.akamaiedge.net.
1.2.3.4
Full nonce request output:
Reports shows that it can be due to the following command
"FULL_NONCE="`${CURL} ${CURL_OPTIONS} --silent -I ${API}/directory`": curl --connect-timeout 15 -k -I https://acme-v01.api.letsencrypt.org/directory
throwing the error
curl: (43) CURLOPT_SSL_VERIFYHOST no longer supports 1 as value!
which likely means your version of curl is old and should be updated. You can do this with CustomBuild like so:
da build curl
A similar report is related to the CustomBuild curl linking against an RPM-based curl library, e.g.,
[root@server scripts]# curl --connect-timeout 15 -k -I https://acme-v01.api.letsencrypt.org/directory
curl: (48) An unknown option was passed in to libcurl
[root@server scripts]# ldd /usr/local/bin/curl | grep curl
libcurl.so.4 => /lib64/libcurl.so.4 (0x00007fdd1411a000)
where it should be /usr/local/lib/libcurl.so.4. To resolve that, remove the libcurl-devel RPM, re-compile curl and run ldconfig:
rpm -e libcurl-devel
da build curl
ldconfig
Now, confirm this worked:
[root@server custombuild]# ldd /usr/local/bin/curl | grep curl
libcurl.so.4 => /usr/local/lib/libcurl.so.4 (0x00007f17c3cd5000)
Reported error:
Requesting new certificate order...
Nonce is empty. Exiting. dig output of acme-v02.api.letsencrypt.org:
prod.api.letsencrypt.org.
ca80a1adb12a4fbdac5ffcbc944e9a61.pacloudflare.com.
172.65.32.248
Full nonce request output:
HTTP/2 200
server: nginx
date: Thu, 10 Oct 2019 05:10:04 GMT
cache-control: public, max-age=0, no-cache
link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index"
replay-nonce: 00013TyemkZQGnX2K1N4l76MLUN-WybdRuqsJjKWrYpsHWA
x-frame-options: DENY
strict-transport-security: max-age=604800
The solution to this one is simply to grab an updated letsencrypt.sh script:
da build lego
Reported solution for Debian/Ubuntu Edit
/etc/ld.so.conf
and change the order of the paths, so that /usr/local/lib is higher up (so the file looks like this):
/usr/local/lib/
include /etc/ld.so.conf.d/*.conf
Save, then run
ldconfig
We've not tested this, as it could affect the libraries that system binaries use. Be sure to fully test things, especially SSHd (restart the SSHd server and test a login), before logging out of the current SSH session.
"Let's Encrypt request successful" but contains an error message for non-success
We had a few reports where Debian systems were returning a success message for Let's Encrypt calls, while the message itself contained errors, e.g.,
Subject: Let's Encrypt request successful
Requesting new certificate order...
Processing authorization for fakedomain.com...
Error: http://fakedomain.com/.well-known/acme-challenge/letsencrypt_1560468322 is not reachable. Aborting the script.
dig output for fakedomain.com:
Please make sure /.well-known alias is setup in WWW server.
The message itself isn't relevant, as the domain(s) in question does not exist.
The issue was that the subject was reporting success, when the letsencrypt.sh script was returning an error code.
Debugging the issue found /bin/dash (the Debian shell) to be the cause, in that it was returning code 0, when the script it was calling was returning code 1.
Simple solution: Use bash.
- Install bash on your system, if it's not present:
apt-get install bash
and confirm the binary exists at:
ls -la /bin/bash
- /bin/bash is present, change /bin/sh to use it:
ln -sf bash /bin/sh
which changes the old "/bin/sh -> dash" link to point to the properly behaving bash binary.
"type": "urn:acme:error:unauthorized", "detail": "Must agree to subscriber agreement before any further actions"
When creating a certificate, if you run into this error:
{
"type": "urn:acme:error:unauthorized",
"detail": "Must agree to subscriber agreement before any further actions",
"status": 403
}. Exiting...
For User , delete your file:
/usr/local/directadmin/data/users/username/letsencrypt.key
and try again. The issue relates to a change in policy that must be approved, and the old letsencrypt.key file has to be wiped and re-generated as a sign of approval.
How to revoke a Let's Encrypt hostname certificate
To revoke the certificate, please run the following (if the cert is still in-place):
cd /usr/local/directadmin/scripts
./letsencrypt.sh revoke $(hostname -f)
To stop the auto-renewal:
rm -f /usr/local/directadmin/conf/cacert.pem.creation_time
CAA record prevents issuing the certificate
This error indicates either a DNS error or a CAA record exists for one of the domains included in the request and prevents the Certificate Authority "letsencrypt.org" from being authorized to issue an SSL.
Check the following for each domain/subdomain included in the SSL request to troubleshoot this error:
- Check for an existing CAA record that would prohibit the SSL to be issued by "letsencrypt.org" by running the following command in a terminal (change 'DOMAIN.TLD' to your actual domain):
dig caa +short DOMAIN.TLD
Keep in mind that a subdomain can have its own CAA record that overrides the parent domain. Check any other domains included in the request for the existence of this record, too.
- If no CAA records exist for the domains included in the SSL request, you should move on to check the DNS for the domains. Run the same CAA dig this time without the "+short" and look for "SERVFAIL" in the output. If this exists in the output, it signifies a DNS problem for the domain checked. At this point, you should check for the following:
- invalid DNSSEC
- general DNS errors, such as mismatched nameservers, for example.
- check your domain's overall DNS health using IntoDNS.com
A vast compilation of tools for checking your configuration exists here:
https://community.letsencrypt.org/t/third-party-tools-to-check-your-configuration/76272
- Make sure that you are running the latest letsencrypt.sh that uses POST-as-GET requests and that you have LEGO installed:
da build letsencrypt
- Run the command via the command line via debug and staging modes. If you had previously requested the SSL via the DirectAdmin SSL interface, you should use the DirectAdmin-generated san_config file containing all of your selections made via the DirectAdmin SSL GUI (make sure to replace "DOMAIN.TLD" & "USERNAME" below with your actual domain and username):
bash -x /usr/local/directadmin/scripts/letsencrypt.sh request 'DOMAIN.TLD' 4096 /usr/local/directadmin/data/users/USERNAME/domains/DOMAIN.TLD.san_config /var/www/html
Now, review the output and check for errors. Pay close attention to the domains that are being tested. For example, a domain request may fail with a SERVFAIL because of an expired domain pointer included in the request. The actual domain may exhibit no DNS issues itself when checked, but if the domain pointer does, it will prevent the issuance of the SSL.
- If you still can't figure it out, feel free to submit a ticket to DirectAdmin for examination. Please include all tests and output from said tests in your request.