Manually debugging /.well-known/acme-challenge/letsencrypt_12345

When trying to install a Let's Encrypt certificate, if you're hitting this error for :

Getting challenge for domain.com from acme-server...
Error:
http://domain.com/.well-known/acme-challenge/letsencrypt_1472046285
is not reachable. Aborting the script.
Please make sure /.well-known alias is setup in WWW server.
1
2
3
4
5

we can manually test this to see what's going on.

  1. First, ensure that you have letsencrypt=1 in your directadmin.conf. You can use this guide to help you do this.

  2. Next, we'll want to confirm the Alias /.well-known has been added to the file

/etc/httpd/conf/extra/httpd-alias.conf
1

If this is not set, add it using

cd /usr/local/directadmin/custombuild
./build update
./build rewrite_confs
1
2
3

and the build script should add it into the httpd-alias.conf file for you.

  1. To manually test things, create a file like this:
echo "test" >> /var/www/html/.well-known/acme-challenge/test.txt
1

and then test it via:

http://domain.com/.well-known/acme-challenge/test.txt
1

which should show you "test" in the output if all is working.

  1. If that works, but you still get the error, then try out exactly what curl is running. Login to ssh as root, and type:
/usr/local/bin/curl -I -L -k -X GET http://domain.com/.well-known/acme-challenge/test.txt
1

where we'd be testing with the test.txt, while the letsencrypt.sh is testing on the letsencrypt_12345678, which is just a unix timestamp used to keep the file name moderately unique for the test.

  1. Another testing tool is to run bash in -x mode and enabling staging so rate limit will not be hit, e.g.,
staging=yes bash -x ./letsencrypt.sh request domain.com
1

which will dump all calls and helps in the debug process.

Let's Encrypt: Manually forcing the automated renewal to test for errors

Sometimes you might want to force DirectAdmin to think a Let's Encrypt certificate needs to be renewed. We can do this by manually changing the certificate's creation time file to an older time (e.g., 61 days prior):

Assume:

Directadmin User:

Domain:

and that the Let's Encrypt SSL is currently valid with a renewal time somewhere in the future.

Once that info is filled out, type:

cd /usr/local/directadmin
echo `date +%s --date='61 days ago'` > "data/users/user/domains/domain.com.cert.creation_time"
echo 'action=rewrite&value=letsencrypt&domain=domain.com' > data/task.queue; ./dataskq d3100
1
2
3

Let's Encrypt certificate creation works, but takes 15 minutes

Report that if your system **has IPv6 **present, but is not on a valid IPv6 network, Let's Encrypt calls may try and used IPv6, and timeout after 15 minutes, then using the IPv4.

Related discussionopen in new window

Solution for the DirectAdmin setup is to edit:

/usr/local/directadmin/scripts/letsencrypt.sh
1

and add the -4 option to all curl calls.

The simplest way to do this is to add -4 into the ${CURL} variable, after it's been checked for existence. Find this code:

CURL=/usr/local/bin/curl
if [ ! -x ${CURL} ]; then
       CURL=/usr/bin/curl
fi
1
2
3
4

and right below it, add a line, so that it looks like this:

CURL=/usr/local/bin/curl
if [ ! -x ${CURL} ]; then
       CURL=/usr/bin/curl
fi
CURL="${CURL} -4"
1
2
3
4
5

so that all calls using ${CURL} will have the -4 option added.

Note, the file will be rewritten with the next letsencrypt update. Better to fix IPv6 or disable it completely.

Nonce is empty. Exiting. dig output of acme-v01.api.letsencrypt.org

When creating a certificate, if you get the following error:

Getting challenge for server.yourhost.com from acme-server...
Nonce is empty. Exiting. dig output of acme-v01.api.letsencrypt.org:
api.letsencrypt.org.edgekey.net.
e981.dscb.akamaiedge.net.
1.2.3.4
Full nonce request output:
1
2
3
4
5
6

Reports shows that it can be due to the following command

"FULL_NONCE="`${CURL} ${CURL_OPTIONS} --silent -I ${API}/directory`": /usr/local/bin/curl --connect-timeout 15 -k -I https://acme-v01.api.letsencrypt.org/directory
1

throwing the error

curl: (43) CURLOPT_SSL_VERIFYHOST no longer supports 1 as value!
1

which likely means your version of curl is old and should be updated. You can do this with CustomBuild like so:

cd /usr/local/directadmin/custombuild
./build update
./build curl
1
2
3

A similar report is related to the CustomBuild curl linking against an RPM-based curl library, e.g.,

[root@server scripts]# /usr/local/bin/curl --connect-timeout 15 -k -I https://acme-v01.api.letsencrypt.org/directory
curl: (48) An unknown option was passed in to libcurl
[root@server scripts]# ldd /usr/local/bin/curl | grep curl
       libcurl.so.4 => /lib64/libcurl.so.4 (0x00007fdd1411a000)
1
2
3
4

where it should be /usr/local/lib/libcurl.so.4. To resolve that, remove the libcurl-devel RPM, re-compile curl and run ldconfig:

rpm -e libcurl-devel
cd /usr/local/directadmin/custombuild
./build curl
ldconfig
1
2
3
4

Now, confirm this worked:

[root@server custombuild]# ldd /usr/local/bin/curl | grep curl
       libcurl.so.4 => /usr/local/lib/libcurl.so.4 (0x00007f17c3cd5000)
1
2

Reported error:

Requesting new certificate order...
Nonce is empty. Exiting. dig output of acme-v02.api.letsencrypt.org:
prod.api.letsencrypt.org.
ca80a1adb12a4fbdac5ffcbc944e9a61.pacloudflare.com.
172.65.32.248
Full nonce request output:
HTTP/2 200
server: nginx
date: Thu, 10 Oct 2019 05:10:04 GMT
cache-control: public, max-age=0, no-cache
link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index"
replay-nonce: 00013TyemkZQGnX2K1N4l76MLUN-WybdRuqsJjKWrYpsHWA
x-frame-options: DENY
strict-transport-security: max-age=604800
1
2
3
4
5
6
7
8
9
10
11
12
13
14

The solution to this one is simply to grab an updated letsencrypt.sh script:

cd /usr/local/directadmin/custombuild
./build update
./build letsencrypt
1
2
3

Reported solution for Debian/Ubuntu Edit

/etc/ld.so.conf
1

and change the order of the paths, so that /usr/local/lib is higher up (so the file looks like this):

/usr/local/lib/
include /etc/ld.so.conf.d/*.conf
1
2

Save, then run

ldconfig
1

We've not tested this, as it could affect the libraries that system binaries use. Be sure to fully test things, especially SSHd (restart the SSHd server and test a login), before logging out of the current SSH session.

"Let's Encrypt request successful" but contains an error message for non-success

We had a few reports where Debian systems were returning a success message for Let's Encrypt calls, while the message itself contained errors, e.g.,

Subject: Let's Encrypt request successful

Requesting new certificate order...
Processing authorization for fakedomain.com...
Error: http://fakedomain.com/.well-known/acme-challenge/letsencrypt_1560468322 is not reachable. Aborting the script.
dig output for fakedomain.com:
Please make sure /.well-known alias is setup in WWW server.
1
2
3
4
5
6
7

The message itself isn't relevant, as the domain(s) in question does not exist.

The issue was that the subject was reporting success, when the letsencrypt.sh script was returning an error code.

Debugging the issue found /bin/dash (the Debian shell) to be the cause, in that it was returning code 0, when the script it was calling was returning code 1.

Simple solution: Use bash.

  1. Install bash on your system, if it's not present:
apt-get install bash
1

and confirm the binary exists at:

ls -la /bin/bash
1
  1. /bin/bash is present, change /bin/sh to use it:
ln -sf bash /bin/sh
1

which changes the old "/bin/sh -> dash" link to point to the properly behaving bash binary.

"type": "urn:acme:error:unauthorized", "detail": "Must agree to subscriber agreement before any further actions"

When creating a certificate, if you run into this error:

{
"type": "urn:acme:error:unauthorized",
"detail": "Must agree to subscriber agreement before any further actions",
"status": 403
}. Exiting...
1
2
3
4
5

For User , delete your file:

/usr/local/directadmin/data/users/username/letsencrypt.key
1

and try again. The issue relates to a change in policy that must be approved, and the old letsencrypt.key file has to be wiped and re-generated as a sign of approval.

Related forum threadopen in new window

Provided agreement URL [https://letsencrypt.org/....pdf] does not match current agreement URL

If you're trying to use Let's Encrypt but get an error similar to:

{
"type": "urn:acme:error:malformed",
"detail": "Provided agreement URL [https://letsencrypt.org/documents/LE-SA-v1.0.1-July-27-2015.pdf] does not match current agreement URL [https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf]",
"status": 400
}.
1
2
3
4
5

Then you'll need to update your letsencrypt.sh script.

CustomBuild 2.0 now has the ability to do this, assuming you've got **letsencrypt=1 **set in your directadmin.conf.

cd /usr/local/directadmin/custombuild
./build update
./build letsencrypt
1
2
3

If that's giving you issues, you can do it manually with wget

wget -O /usr/local/directadmin/scripts/letsencrypt.sh http://files.directadmin.com/services/all/letsencrypt.sh
1

Forum thread: http://forum.directadmin.com/showthread.php?t=53564open in new window

Change to Custombuild: https://www.directadmin.com/features.php?id=1867open in new window

How to revoke a Let's Encrypt hostname certificate

To revoke the certificate, please run the following (if the cert is still in-place):

cd /usr/local/directadmin/scripts
./letsencrypt.sh revoke `hostname -f`
1
2

To stop the auto-renewal:

rm -f /usr/local/directadmin/conf/cacert.pem.creation_time
1

CAA record prevents issuing the certificate

This error indicates either a DNS error or a CAA record exists for one of the domains included in the request and prevents the Certificate Authority "letsencrypt.org" from being authorized to issue an SSL.

Check the following for each domain/subdomain included in the SSL request to troubleshoot this error:

  1. Check for an existing CAA record that would prohibit the SSL to be issued by "letsencrypt.org" by running the following command in a terminal (change 'DOMAIN.TLD' to your actual domain):
dig caa +short DOMAIN.TLD
1

Keep in mind that a subdomain can have its own CAA record that overrides the parent domain. Check any other domains included in the request for the existence of this record, too.

  1. If no CAA records exist for the domains included in the SSL request, you should move on to check the DNS for the domains. Run the same CAA dig this time without the "+short" and look for "SERVFAIL" in the output. If this exists in the output, it signifies a DNS problem for the domain checked. At this point, you should check for the following:

A vast compilation of tools for checking your configuration exists here:

https://community.letsencrypt.org/t/third-party-tools-to-check-your-configuration/76272open in new window

  1. Make sure that you are running the latest letsencrypt.sh that uses POST-as-GET requests and that you have LEGO installed:
cd /usr/local/directadmin/custombuild
./build update
./build letsencrypt
1
2
3
  1. Run the command via the command line via debug and staging modes. If you had previously requested the SSL via the DirectAdmin SSL interface, you should use the DirectAdmin-generated san_config file containing all of your selections made via the DirectAdmin SSL GUI (make sure to replace "DOMAIN.TLD" & "USERNAME" below with your actual domain and username):
staging=yes bash -x /usr/local/directadmin/scripts/letsencrypt.sh request 'DOMAIN.TLD' 4096 /usr/local/directadmin/data/users/USERNAME/domains/DOMAIN.TLD.san_config /var/www/html
1

Now, review the output and check for errors. Pay close attention to the domains that are being tested. For example, a domain request may fail with a SERVFAIL because of an expired domain pointer included in the request. The actual domain may exhibit no DNS issues itself when checked, but if the domain pointer does, it will prevent the issuance of the SSL.

  1. If you still can't figure it out, feel free to submit a ticket to DirectAdmin for examination. Please include all tests and output from said tests in your request.
Last Updated: 6/23/2021, 9:36:08 PM