Version 1.44.1
Released: 2013-12-01
Add success=1 to user_restore_post.sh new
If you want to know if a restore worked or not, a new variable:
success=1
will be added to the user_restore_post.sh.
If there is an issue, you'd see sucecss=0.
Also, a variable called:
result_string
will hold the string displayed in DA, as to what went wrong, if anything.
Related:
Backup Monitor (API) (SKINS) new
Dec 10, 2013: Bug found if you use track_task_queue_processes=2:
backup monitor may cause dataskq segfault
Fixed in the pre-release binaries, or 1.44.2 or greater.
Thread:
https://forum.directadmin.com/threads/47417
Ability to track backup progress (and possibly any task in the dataskq or "directadmin", depending how this goes)
You must reload your CMD_ADMIN_BACKUP (Admin Backup/Transfer) page after the backup has started to have the table show up.
We may change this to have an ajax reload in the future, but for now, if you're waiting to see a backup, just hit F5 a few times.
Default internal variable:
track_task_queue_processes=1
which gives a process overview of who's being backed up, how many Users there are, and what part of the backup progress is being made for each User.
I'll add a progress bar that moves long slowly using ajax for an accurate measure.
You can also set this in your directadmin.conf:
track_task_queue_processes=2
to get a much more detailed track, in addition to the above information.
With 2, the dataskq will dump any tracked process location to a log file, which will scroll in DA as it goes.
Include API-ability.
SKINS
admin/admin_backup_monitor.html
files_admin.conf:
CMD_ADMIN_BACKUP_MONITOR=admin/admin_backup_monitor.html
JSON - updated August 2, 2018
If you call:
CMD_ADMIN_BACKUP?json=yes
you'll get the usual admin backup page, but if a backup is running, you'll also get a JSON entry:
"current_backups":
{
"0":
{
"pid": "555",
"id": "",
"start":
{
"dayofmonth": "",
"dayofweek": "",
"hour": "",
"minute": "",
"month": ""
},
"who":
{
"select":
[
"cronuser"
],
"who": "selected"
},
"where": "<span class='green_lock'>🔒</span> /home/admin/admin_backups2",
"progress": "0.00%",
"details": "CMD_ADMIN_BACKUP?action=monitor&pid=555"
},
"info":
{
"columns":
{
"pid": "1",
"id": "2",
"start": "3",
"who": "4",
"where": "5",
"progress": "6",
"details": "7"
},
"current_page": "1",
"ipp": "99999",
"rows": "1",
"total_pages": "1"
}
},
With that information, you can then use action=monitor, if you want to just view that process:
CMD_ADMIN_BACKUP?action=monitor&pid=555&json=yes
{
"backup_info":
{
"0":
{
"pid": "555",
"id": "",
"owner": "admin",
"start": "1533247621",
"who":
{
"select":
[
"cronuser"
],
"who": "selected"
},
"where": "<span class='green_lock'>🔒</span> /home/admin/admin_backups2"
},
"info":
{
"columns":
{
"pid": "1",
"id": "2",
"owner": "3",
"start": "4",
"who": "5",
"where": "6"
},
"current_page": "1",
"ipp": "50",
"rows": "1",
"total_pages": "1"
}
},
"track_task_queue_processes": "2"
}
STREAM
Note: SSE Stream format is supposed in DA 1.59.5+:
Show Restore Tracking info in Admin Backup/Transfer
And one lever further if you want the play-by-play:
CMD_ADMIN_BACKUP?action=monitor&pid=555&get=tracking_data
where it slowly generates JS code, which you'd have your side run (or just parse/display).. sort of a hacky way about doing it, but works for the skin (isn't json-ified, as it's js-ified)
Note that the track_task_queue_processes=2 will give much much more output than the 1 value, where 1 is set by default, so this is more than usual.
As mentioned, DA will trickle this data slowly, as it happens.
Related: Backup Monitor (API) (SKINS)
You can find the client-side JS in enhanced: admin/admin_backup_monitor.html
sample streamed get=tracking_data output:
document.getElementById('dynamic_text').innerHTML='Backup User cronuser';
document.getElementById('dynamic_details').innerHTML='user_backup_pre.sh';
add_process_info('User('admin')::readFile() : uuf');
add_process_info('getlock(./data/users/admin/user.usage, 'ConfigFile::readFile') : start');
add_process_info('getlock(./data/users/admin/user.usage, 'ConfigFile::readFile') : finished');
add_process_info('User('admin')::readFile() : dlf');
add_process_info('getlock(./data/users/admin/domains.list, 'ListFile::readFile') : start');
add_process_info('getlock(./data/users/admin/domains.list, 'ListFile::readFile') : finished');
add_process_info('User('admin')::readFile() : ucf');
add_process_info('getlock(./data/users/admin/user.conf, 'ConfigFile::readFile') : start');
add_process_info('getlock(./data/users/admin/user.conf, 'ConfigFile::readFile') : finished');
add_process_info('User('admin')::readFile() : finished');
add_process_info('getlock(./data/users/admin/reseller.usage, 'ConfigFile::readFile') : start');
add_process_info('getlock(./data/users/admin/reseller.usage, 'ConfigFile::readFile') : finished');
add_process_info('getlock(./data/users/admin/users.list, 'ListFile::readFile') : start');
add_process_info('getlock(./data/users/admin/users.list, 'ListFile::readFile') : finished');
add_process_info('getlock(./data/users/admin/reseller.conf, 'ConfigFile::readFile') : start');
add_process_info('getlock(./data/users/admin/reseller.conf, 'ConfigFile::readFile') : finished');
add_process_info('getlock(./data/users/admin/reseller.allocation, 'ConfigFile::readFile') : start');
add_process_info('getlock(./data/users/admin/reseller.allocation, 'ConfigFile::readFile') : finished');
add_process_info('getlock(./data/admin/admin.conf, 'ConfigFile::readFile') : start');
add_process_info('getlock(./data/admin/admin.conf, 'ConfigFile::readFile') : finished');
add_process_info('getlock(./data/admin/admin.usage, 'ConfigFile::readFile') : start');
add_process_info('getlock(./data/admin/admin.usage, 'ConfigFile::readFile') : finished');
add_process_info('getlock(./data/admin/reseller.list, 'ListFile::readFile') : start');
add_process_info('getlock(./data/admin/reseller.list, 'ListFile::readFile') : finished');
add_process_info('getlock(./data/admin/admin.list, 'ListFile::readFile') : start');
add_process_info('getlock(./data/admin/admin.list, 'ListFile::readFile') : finished');
add_process_info('getlock(./data/admin/admin.allocation, 'ConfigFile::readFile') : start');
add_process_info('getlock(./data/admin/admin.allocation, 'ConfigFile::readFile') : finished');
add_process_info('getDirFilesAndDirs(./data/users, *tlf, *tdlf, (null))');
add_process_info('getDirFilesAndDirs(./data/users, *tlf, *tdlf, (null)) : done');
add_process_info('getlock(./data/users/cronuser/user.conf, 'ConfigFile::readFile') : start');
add_process_info('getlock(./data/users/cronuser/user.conf, 'ConfigFile::readFile') : finished');
add_process_info('User('cronuser')::readFile() : uuf');
add_process_info('User('cronuser')::readFile() : dlf');
add_process_info('User('cronuser')::readFile() : ucf');
add_process_info('User('cronuser')::readFile() : finished');
add_process_info('User::setQuota(0) : cronuser');
add_process_info('getHomeDir : cronuser');
add_process_info('getHomeDir : cronuser : done');
add_process_info('custom script: /usr/local/directadmin/scripts/custom/user_backup_pre.sh : 'file=/home/admin/admin_backups2/user.admin.cronuser.tar.gz&reseller=admin&username=cronuser'');
set_percent('0.00');
set_percent('0.00');
set_percent('0.00');
set_percent('0.00');
set_percent('0.00');
set_percent('0.00');
set_percent('0.00');
set_percent('0.00');
set_percent('0.00');
set_percent('0.00');
set_percent('0.00');
set_percent('0.00');
set_percent('0.00');
set_percent('0.00');
set_percent('0.00');
set_percent('100');
document.getElementById('dynamic_text').innerHTML='Backup Complete';
document.getElementById('dynamic_details').innerHTML='';
Note the menthod used to geneated this backup output was to add a very long sleep in the user_backup_pre.sh, hence it's sitting at 0% for a long time.
It then jumps to the end, as the User was small.
If you were to load the page after the backup was done:
CMD_ADMIN_BACKUP?action=monitor&pid=555&json=yes&get=tracking_data
you'll get this output, which can be used to determine if it's done.. but you must know the PID first.
document.getElementById('dynamic_text').innerHTML='Cannot find that pid number. Backup may already be finished.';
UserDir banwidth in breakdown, and homdir.log rotation new
If you go to the "Bandwidth Breakdown" page for a given User, the hover-over popup will now show the UserDir (IP/~username) usage for that given day.
It was always included in the total Apache downloaded amount, and still is included in the Apache total, but now is shown to get a better idea of where bandwidth might have come from.
As well, the /var/log/homedir.log, where this banwidth is taken from will be rotated, instead of truncated, eg:
/var/log/homedir.log
/var/log/homedir.log.1
/var/log/homedir.log.2
etc..
up to the "Number of Apache logs to keep", specified in the Admin Settings.
Lastly, the bytes logs:
/var/log/httpd/domains/domain.com.bytes
will also be rotated, so old copies can be seen, eg:
/var/log/httpd/domains/domain.com.bytes
/var/log/httpd/domains/domain.com.bytes.1
/var/log/httpd/domains/domain.com.bytes.2
..
The rotation of the homedir.log and domain.com.bytes do rename the numbered files, but the original file is copied and truncated.
This is so that we don't need to wait for a full restart of apache... as per this old change:
Delete apache logs in place without changing link
Added \".Deleted Messages\" to auto-purge list new
Admin Level -> Admin Settings -> Automatically Purge Spambox/Trash data
The list now contains:
.INBOX.spam
.INBOX.Trash
.Trash
.Deleted Messages
CA Root Certificate to save to server.ca for Admins new
When logged in as an Admin that uses the Server IP, the CA Root Certificate page in DA was saving the CA to:
/usr/local/directadmin/data/users/username/admin/domains/domain.com.cacert
This change will let that GUI page save to:
/etc/httpd/conf/ssl.crt/server.ca
The files:
/etc/httpd/conf/extra/httpd-vhosts.conf
/etc/nginx/conf/nginx.conf
will also have lined added in CustomBuild 2.0, pointing to this server.ca by default, so a file must exist there.
The ./build rewrite_confs in CB2 will automatically download this file.
which is a generic bundle for the major SSL authorities (it may even work as a CA for a purchased cert, but probably won't)
The main purpose of this file/download is so that apache has something to use, so it starts up.
The ips_virtual_host.conf and nginx_ips.conf will already set the server.ca if set, as per:
SSLCACertificateFile in VH's in ips.conf
Predict non-readable restore files new
Similar to this previous feature:
Admin Level Restore: Check that Reseller exists
DA will make an attempt to notice that a tar.gz file is not readable by the restoring Reseller/Admin.
If it's not readable by that Admin or Reseller, then the tar.gz file in the restore list will appear red, and a hover-over popup will tell you why.
It would show the permissions of the file or folder that needs adjusting.
It doesn't work for readability on the file of the account being restored (which is a different read) because the account might not yet exist, so I've also added a check if the extraction fails during the restore, to give a similar list of files/paths that need to be adjusted in the return message.
However, DA should automatically set 640 and chgrp the file to the restored account.. so the only case that would happen here would be if the admin_backups or user_backups is not set to 711.. or the /home/admin or /home/reseller are not set to 711.
Suspend an automated suspended account to set account=off new
Let's say a User went over bandwidth, and this suspended the User.
DA will suspend the account.
In this state, upon the reset at the first of the month, the account will be unsuspended, as the account=ON is still set.
Now, when you manually suspend an account, it sets account=OFF such that it's not unsuspended upon the reset.
The issue is when you want to manually suspend an account to set account=OFF if the account was automatically suspended for bandwidth.
DA wasn't able to set account=OFF.
This change will allow you to manually suspend an already automated suspend, in order to set account=OFF
DNSSEC (SKINS) new
Guide to enable DNSSEC:
https://help.directadmin.com/item.php?id=651
Documentation:
For now, it's only at the Admin Level -> DNS Admin in the Enhanced skin.
After testing, it will be added to the other skins, and to the User Level.
directadmin.conf option:
dnssec=0
which is the internal default.
To enable it add:
dnssec=1
and restart DA.
Global token:
HAVE_DNSSEC=yes|no
if it's enabled.
SCRIPT
new script:
/usr/local/directadmin/scripts/dnssec.sh
./dnssec.sh install
./dnssec.sh keygen <domain>
./dnssec.sh sign <domain>
The "install" may need you to manually add bits to your named.conf.
MONTHLY RESET
will automatically re-sign all zones that have keys and already signed.
If a zone has keys, but is not signed, this domain will be skipped
TASK.QUEUE
echo "action=rewrite&value=dnssec" >> /usr/local/directadmin/data/task.queue
will issue a re-signing, with the same rules as with the monthly reset.
That being said, this not to be used for the initial key/signing. (it won't do anything without the key/signs already in place)
You do not need to add this as a cronjob.
The monthly reset will re-sign the zones automatically.. this is just a manual way to re-sign the zones if needed. (requires that they already have keys)
MULTI-SERVER SETUP
by default, this will be the directadmin.conf value:
dnssec_mss_use_signed_zone=1
meaning if you're using the MSS (multi-server setup), DA will send over the signed zone to the remote box, rather than the raw zone you'd be editing.
Note, DA will also not let you edit a signed zone on a remote box because it's format is fairly different and DA can't read it (at this time)
plus going backwards to a raw zone would just get very messy.
For DNSSEC always edit from the main dns server to send the signed zones (from where the keys live)
SKINS
admin/dns_admin_control.html
New token for the dnssec table, added below the Add Record table.
|DNSSEC_TABLE|
NOT USED, but FYI:
The named.db template has a non-used token:
|INCLUDE_DNSSEC_KEYS|
which, if added, and the ksk and zsk keys exist, DA will add:
$include /var/named/domain.com.zsk.key;
$include /var/named/domain.com.ksk.key;
This method was droped, and the $include lines are instead only added to a temp copy of the zone, rather than the main file.
This keeps is cleaner, so that the zone can exist without the keys.
lost_password_pre.sh new
Script, run before all calls of the CMD_LOST_PASSWORD command.
If you exit with a non-zero value, the script will abort the process.
Create your script here:
/usr/local/directadmin/scripts/custom/lost_password_pre.sh
Ability to suppress login key creation/modification notifications new
When creating a login key, the owner of that key will be notified via message system.
If the API is used, this may not be desirable.
You can shut off the notifications by adding:
login_keys_notify_on_creation=0
to your directadmin.conf and restart DA.
The internal default is:
login_keys_notify_on_creation=1
Ability to disable all database functions new
If you don't want to have DA manage anything regarding MySQL, a new option is available in the directadmin.conf
mysql=0
and restart DA, to prevent DA from doing anything with MySQL at all.
The internal default is:
mysql=1
Inodes to enhanced skin (SKINS) new
Added usage bar for inodes, if limit is not unlimited.
As well as entry in the table below the status bars.
SKINS
user/template_user_bottom.html
|*if HAVE_INODE="yes"|
show_bar("<b>|LANG_INODES|</b><br>", "|USERINODE|", "|USERINODEMAX|", "inode_bar");
|*endif|
and near the bottom:
|*if HAVE_INODE="yes"|
<tr>
<td width="33%" class=bar-row2>|LANG_INODES|</td>
<td width="33%" class=bar-row2 align="center">|USERINODE|</td>
<td width="34%" class=bar-row2 align="center">|USERINODEMAX|</td>
</tr>
|*endif|
Ability to tokenize script output to set tokens new
This feature will give an embedded tokenizer script the ability to have it's output tokenized.
The main benefit of this is to allow the setting of tokens for use farther down in the output.
For example, if you want a |DOMAIN| token to be set via GET.. but it's not available, say for example you're at the Admin Level, where a DOMAIN isn't set, you could use the API to get this info, then have your script output a standard token variable setting code.
Note, that because the script is tokenized piror to running, you must make sure that any strings you set are not seen by DA as a token, else it will be swapped with "none" prior to the script running.
An example skin would be:
|?domain=testing.com|
|$/usr/local/bin/php
<?php
echo "|";
echo "?domain=domain.com|\";
?>
DONE|
Domain is |domain|<br>
Which will output:
Domain is domain.com
Note, how we set the "domain" token to testing.com, just so that we can see for sure that the output from the script is overriding the previous setting (setting it to testing.com is not actually needed, it's just there to ensure the script output parsing is working).
Also, note how the echo "|"; is set onto it's own line, before the next echo.
This is because this code:
echo "|?domain=domain.com|";
would be seen by the tokenizer prior to the script actually running.. and would be either "none" or completely blank, eg ""; hence the need to fool the tokenizer on the first pass.. and output a valid tokenizable output for the 2nd post-script pass.
By default this feature is disabled. It may be enabled in the future.
The internal default is:
tokenize_script_output=0
to enable it, add:
tokenize_script_output=1
to your directadmin.conf and restart directadmin.
Added /usr/local/phpXX/lib/php to ODB (TEMPLATES) new
template changes to:
user_virtual_host.conf
virtual_host2.conf
virtual_host2_secure.conf
virtual_host2_sub.conf
virtual_host2_secure_sub.conf
For CustomBuild 2 setups, the extra:
/usr/local/php54/lib/php
path is needed (also for other versions php53, php54, php55)
The open basedir setting in these templates has been changed to include extra paths.
The change below basically read.. if PHPX_RELEASE is set to a php version, then add /usr/local/phpXX/lib/php to the ODB settings.
Applies to both php1_release and php2_release.
|?OBDP1=|
|*if PHP1_RELEASE!="0.000000"|
|?OBDP1=:/usr/local/php\`PHP1_RELEASE\`/lib/php/|
|*endif|
|?OBDP2=|
|*if PHP2_RELEASE!="0.000000"|
|?OBDP2=:/usr/local/php\`PHP2_RELEASE\`/lib/php/|
|*endif|
|?OPEN_BASEDIR_PATH=\`HOME\`/:/tmp:/var/tmp:/usr/local/lib/php/\`OBDP1\`\`OBDP2\`|
Allow space in the middle of password new
Spaces are now allowed in passwords, with the restriction that the password does not being, nor end with a space.
API MX record bug for old format fixed
Relating to this change:
Subdomain MX records (SKINS) (API)
The API calls to CMD_API_DNS_ADMIN and CMD_API_DNS_CONTROL should have converted the new format from:
domain.com. MX 10 mail
to show:
mail=10
but it was showing:
mail=10 mail
========
Similar report in the dns_control.html:
Bug was:
|?FULL_MX_NAME=\`domain\`.|
changed to:
|?FULL_MX_NAME=\`ptr_val\`.|
so that the MX record editing of domain pointers worked correctly.
Plugins Location redirect bug fixed
The plugin downloader supports Location redirects, but https was not being correctly parsed from the header, so it kept hammering away on the http version (often getting into a loop, up to 10 times). Fixed to correctly allow https in the Location redirect header.
Also added a Location redirect to be allowed in the version_url variable in the plugin.conf, so you can redirect the version info to another url.
Backup removing linked IPs from cluster dns servers fixed
If you are using the Linked-IP feature, and you create backups (I would hope so), and you are also using an external dns server with the Multi-Server Setup, then you're affected by this bug.
When a backup is created, a copy of the domain.com.db is added to the backups/ folder, to await compression.
However, the domain.com.db should not have any of the extra linked IPs in there.. DA removes them.
If the new box has linked IPs, then they'll be added there.
After the linked IPs are removed, the zone file is re-written to disk (still in the backups folder).
The bug was that this write did not shut off the clustering... so, since it's a dns write, this triggered the cluster sync, and sent over the db file without the linked IPs in it.
But was to ensure no clustering happened when writing the backup db file.
non_readable_files not created with 0 files but more than 0 directories fixed
If you've got a folder which is chmod to 0, but all files under the ~/domains path are all readable by the User, then you'll get an error during the backup that looks like this:
Error Compressing the backup file user.admin.username.tar.gz : /bin/tar: non_readable_files: Cannot stat: No such file or directory
/bin/tar: Error exit delayed from previous errors
The bug was that the copying was aborted if the files list was empty.
But the abort should not have happened if the non-readable folders list had data.
mod_ruid2 change to RGroups (SECURITY) fixed
Changed:
RGroups apache access
to be:
RGroups @none
so that the additional groups are cleared.
Only the username should be sufficient to access all files and folders under /home/username.
nginx pointed to wrong servert.crt and server.key fixed
The shared server certificate for nginx was pointing to:
/etc/httpd/conf/ssl.crt/server.crt
/etc/httpd/conf/ssl.key/server.key
/etc/httpd/conf/ssl.crt/server.ca
Fixed DA to point to the proper location:
/etc/nginx/ssl.crt/server.crt
/etc/nginx/conf/ssl.key/server.key
/etc/nginx/conf/ssl.crt/server.ca
new directadmin.conf options:
nginx_cert
nginx_key
nginx_ca
which, when nginx=1 is used, will also reset these values to the nginx values:
apachecert
apachekey
apacheca
so as to save me the need to change all references in the code.
CMD_EMAIL_POP Password/Quota linking to outlook.reg file fixed
The "change" link for password/quota on the DA User account was pointing to the CMD_EMAIL_REG URL.. and the "Download" link for the Outlook Settings was pointing nowhere.
Was introduced when the table was converted to be a variable length columned table (ability to hide the outlook column)
The insert of the <a href> for the system account was still hardcoded to column 6, when it should have been set to the "outlook_column" variable, like the other accounts already had.
Selective data not storing backup_options.list without domains fixed
Relating to feature:
Backup option to Admin backups (SKINS)
but was that if "Domains" was not selected, the backup_options.list was not saved.
Without a backup_options.list, DA assumed it was an old-school backup where everything was in the tar.gz.
Since this was not the case, a restore was done, trying to find data that didn't exist in the backup, causing unexpected results.
Related thread:
https://forum.directadmin.com/threads/47641
exim.pl version 15 fixed
VERSION=15
https://files1.directadmin.com/services/exim.pl.15
Fixes a bug where extra bcc/cc/to values are not counted.
Issue was the message ID is the same for all sends of each copy, so the exim.pl thought they were just retries.
Change the /etc/virtual/user_ids to have the message ID as a folder, istead of a file, and have the designation user-domain as a file in the message ID folder, as the index for "already counted". The user-domain is going to be base64 encoded, if MIME::Base64 exists in perl (usually does). If not, it will revert to just user-domain so it still works.
In a similar fashion, DirectAdmin will also include the local_part and domain variables as unique indexes, along with the message ID, to determine the number of sends.
DA will not update this file for you.
If you want the exim.pl version 15, grab it:
wget -O /etc/exim.pl https://files1.directadmin.com/services/exim.pl.15
once more testing is done, it will replace the main exim.pl, rather than exim.pl.15.
nginx to use reload instead of restart fixed
DA was calling restarts to the nginx boot script.
This is somewhat forceful and not required, so the task.queue will now be filled with requests to reload nginx instead.
*** FreeBSD *** Manual update required
If you are using nginx with FreeBSD, please grab a new nginx boot script, because the reload function uses killproc, which FreeBSD doesn't support.
wget -O /usr/local/etc/rc.d/nginx https://files1.directadmin.com/services/custombuild/nginx.boot.freebsd