Integrate zstd compression into DirectAdmin developed by Facebook

Today AZDIGI will show you how to integrate zstd compression into DirectAdmin developed by the Facebook team. From there, compressing & decompressing directly on DirectAdmin is much better than gzip in terms of space usage, as well as compression and decompression performance.

The new zstd compression format will be tar.zst instead of tar.gz like Gzip‘s.

I. What is ZSTD?

Zst (zstd) Standard is a lossless data compression algorithm developed by Yann Collet at Facebook. Zstd is the reference implementation in C. Version 1 was released as freeware on August 31, 2016.

Currently, the latest version of zstd is 1.4.5, you can track the version of zsdt at the following link:

• GitHub – zstd.

II. Integrate zstd compression into DirectAdmin

Note: The integration of zstd compression into DirectAdmin is only possible if the DirectAdmin version you are using is 1.61.4 or higher.

If your DirectAdmin version is lower, you can upgrade according to the following instructions:

• How to upgrade the DirectAdmin version.

To integrate zstd compression into DirectAdmin, we follow these 3 steps.

Step 1: SSH into your DirectAdmin system

To integrate zstd compression into DirectAdmin, we first need to SSH or access your VPS/server with root privileges.

After successfully SSH, we continue with step 2 to execute commands to integrate zstd compression into DirectAdmin.

Step 2: Commands to integrate zstd compression into DirectAdmin

To integrate zstd compression into DirectAdmin, we run the following 3 commands:

cd /usr/local/directadmin/custombuild
./build update
./build zstd

Explanation of the above commands:

• Command 1: Move to the custombuild folder.
• Command 2: Update custombuild script.
• Command 3: Install zstd.

Below is my installation process:

So we have completed the steps to integrate zstd compression into DirectAdmin, but currently, this feature is not allowed to be used on File Manager. The only application of zstd is to integrate with the backup/restore feature on DirectAdmin to help compress backup data more efficiently, save more storage space as well as extract backed-up data faster.

We continue with the third step to request the backup/restore feature using zstd.

Step 3: Enable zstd into the backup/restore feature on DirectAdmin

To request the backup/restore feature using zstd, we run the following 3 commands:

cd /usr/local/directadmin
./directadmin set zstd 1
./directadmin set backup_gzip 2

Explanation of the above commands:

• Command 1: Move to the directadmin folder.
• Command 2: Notify directadmin that you already have zstd.
• Command 3: Enable backup priority mode with gzip of 2 if zstd is unavailable.

Below are images of the execution of 3 commands above:

In this article, I experimented with a user’s backup with the default WordPress installation before installing zstd and created another backup after installing zstd.

The result is not surprising as the backup from zstd ( tar.zstformat) is lower in size than normal Gzip compression. Although the lower capacity is not much, it is probably because this user only has the default WordPress installation and hardly any media data.

After that, I tried it with another user with a capacity of nearly 6GB, and I achieved extremely unexpected results as follows:

The file size compressed with zstd aggregate is only half that of compressed with Gzip. And because this user size is large, I also noticed one more thing if compressed with zstd for extremely fast, about at least 3 times faster.

III. Disable zstd during backup/restore

In case zstd does not suit your needs, you can disable the integration of zstd compression into DirectAdmin with the following commands:

cd /usr/local/directadmin
./directadmin set zstd 0
./directadmin set backup_gzip 1

Just as simple as that.

IV. Summary

So AZDIGI has shown you how to integrate zstd compression into DirectAdmin. Although we have not been able to use it on the File Manager interface, the usefulness level is still very significant. For a system that has to be backed up regularly, it will relieve the burden on storage hardware and reduce the use of transmission lines if you need to backup to the cloud or a separate backup storage system.

However, it will be a bit inconvenient when downloading the tar.zst to your personal computer will not be able to decompress. Force you to install additional software to be able to decompress, like PeaZip to decompress.

In case you still want to use the old format but want to speed up backup creation, you can refer to the following tutorial:

• Speed up DirectAdmin backups with PIGZ.

Hope this article will be of help to you.

Wishing you success!