HOWTO : Highest Secured Hiawatha Web Server 10.6 on Ubuntu Server 16.04 LTS

(A) Introduction



Hiawatha Web Server is designed with security in mind. It already built-in some security features to protect common attacks, such as SQLi, XSS, CSRF. Meanwhile, it can be configured to prevent scanning from vulnerability scanners too.



Hiawatha is a light weight and fast as well as secure web server in the market. Hiawatha is working well with PHP and MySQL. The following guide is showing how to configure Hiawatha in a very high secure way on Ubuntu Server LTS.



(B) Software Prerequisite



The current version as at the time of this writing :

(1) Ubuntu Server 16.04.2 LTS

(2) CMake 3.8.1

(3) Hiawatha 10.6



(C) Installation of PHP7.0 and MySQL



sudo apt-get install php7.0-cgi php7.0 php7.0-cli php7.0-mysql php7.0-curl php7.0-gd php7.0-intl php7.0-imap php7.0-mcrypt php7.0-pspell php7.0-recode php7.0-sqlite3 php7.0-tidy php7.0-xmlrpc php7.0-xsl apache2-utils php7.0-fpm php-memcache php-imagick php-cache mysql-server mysql-client php7.0-mbstring



(D) Installation of Hiawatha



sudo apt-get install libc6-dev libssl-dev dpkg-dev debhelper fakeroot libxml2-dev libxslt1-dev



(a) Install CMake



wget https://cmake.org/files/v3.8/cmake-3.8.1.tar.gz

tar -xvzf cmake-3.8.1.tar.gz

cd cmake-3.8.1

./configure

make

sudo make install



(b) Install Hiawatha



wget https://github.com/hsleisink/hiawatha/archive/v10.6.tar.gz

tar -xzvf v10.6.tar.gz



cd hiawatha-10.6/extra



./make_debian_package



cd ..



sudo dpkg -i hiawatha_10.6_amd64.deb



(E) Configuration of PHP7.0



sudo nano /etc/php/7.0/fpm/php.ini



Make changes as is.



allow_url_fopen = Off

session.cookie_httponly = 1

disable_functions = [EXIST_FUNCTION],system, show_source, symlink, exec, dl, shell_exec, passthru, phpinfo, escapeshellarg, escapeshellcmd,



* [EXIST_FUNCTION] is the functions that already at "disable_functions" of php.ini



(F) Let's Encrypt on Hiawatha



(a) Configuration of Hiawatha



sudo mkdir -p /etc/hiawatha/enable-sites

sudo mkdir -p /etc/hiawatha/disable-sites



Edit "cgi-wrapper.conf".



sudo nano /etc/hiawatha/cgi-wrapper.conf



Change the following as is.



CGIhandler = /usr/bin/perl

CGIhandler = /usr/sbin/php7.0-fpm

CGIhandler = /usr/bin/python

CGIhandler = /usr/bin/ruby

CGIhandler = /usr/bin/ssi-cgi



Wrap = jail_mysite ; /var/www/mysite ; www-data:www-data



Change the ownership of the log files.



cd /var/log/hiawatha

sudo chown www-data:www-data access.log

sudo chown www-data:www-data error.log

sudo chown www-data:www-data exploit.log

sudo chown www-data:www-data garbage.log

sudo chown root:root system.log



Change the ownership of the web application files.



cd /var/www/mysite

sudo chown -R root:root *



The following are examples of "hiawatha.conf" and "mysite.com".



/etc/hiawatha/hiawatha.conf example :





/etc/hiawatha/enable-sites/mysite.com example :





sudo systemctl restart php7.0-fpm

sudo systemctl restart hiawatha

sudo systemctl enable php7.0-fpm

sudo systemctl enable hiawatha



(b) Self Signed SSL Certificate Generation



To generate SSL certificate for the web root.



openssl genrsa -out default.pem 4096

openssl req -new -x509 -days 3650 -key default.pem -out server.crt

echo "" >> default.pem

cat server.crt >> default.pem

echo "" >> default.pem

rm -f server.crt

sudo mkdir -p /etc/hiawatha/tls

sudo cp default.pem /etc/hiawatha/tls

sudo chown root:root /etc/hiawatha/tls/default.pem

sudo chmod 600 /etc/hiawatha/tls/default.pem

sudo chmod 600 -R /etc/hiawatha/tls



(c) Let's Encrypt Generation and Configuration



(1) First time install Let's Encrypt :



Make sure port 80 is workable as Let's Encrypt script will use it to generate the SSL/TLS certificates.



Change ~/hiawatha-10.6/extra/letsencrypt/letsencrypt.conf :



nano ~/hiawatha-10.6/extra/letsencrypt/letsencrypt.conf



Change "ACCOUNT_EMAIL_ADDRESS" to your email address. Let's Encrypt will alert you when the SSL/TLS certificate is going to expire via this email address.



ACCOUNT_EMAIL_ADDRESS = samiux@gmail.com



Change "CERTIFICATE_RSA_KEY_SIZE" to 4096.



CERTIFICATE_RSA_KEY_SIZE = 4096



Change "RENEWAL_REUSE_KEY" to true. The server private key/public key will be used for the SSL/TLS certificate renewal.



RENEWAL_REUSE_KEY = true



Comment out the "Testing" LE_CA_HOSTNAME and uncomment "Production" LE_CA_HOSTNAME.



LE_CA_HOSTNAME = acme-v01.api.letsencrypt.org # Production

#LE_CA_HOSTNAME = acme-staging.api.letsencrypt.org # Testing



Run the Hiawatha 10.6 letsencrypt script to generate server private key and server certificate as well as Let's Encrypt X3 certificate :



cd ~/hiawatha-10.6/extra/letsencrypt

sudo ./letsencrypt register



A "account.key" will be generated at the ~/hiawatha-10.6/extra/letsencrypt. Make sure keep this "account.key" in a safe space.



Then generate the SSL/TLS certificate of your server :



sudo ./letsencrypt www.mysite.com



A "www.mysite.com.pem" will be generated at /etc/hiawatha/tls/.



Rename the generated file :



sudo -sH

cd /etc/hiawatha/tls

mv www.mysite.com.pem www.mysite.com-privkey.pem

cp www.mysite.com-privkey.pem www.mysite.com.pem



Make sure keep the private key file in a safe space and generate the server public key :



openssl rsa -in www.mysite.com-privkey.pem -pubout -out pubkey.pem



Replace pubkey.pem content to the first block of code "PRIVATE KEY" at www.mysite.com.pem.



Insert Let's Encrypt X4 certificate :



wget https://letsencrypt.org/certs/lets-encrypt-x4-cross-signed.pem.txt

echo "" >> www.mysite.com.pem

cat lets-encrypt-x4-cross-signed.pem.txt >> www.mysite.com.pem



chmod 600 www.mysite.com.pem

chmod 600 www.mysite.com-privkey.pem



Then configure VirtualHost at /etc/hiawatha/enable-sites/mysite.com.



Add the following to the VirtualHost at /etc/hiawatha/enable-sites/mysite.com :



RequireTLS = yes, 180d; includeSubDomains; preload

TLScertFile = /etc/hiawatha/tls/www.mysite.com-privkey.pem

PublicKeyPins = /etc/hiawatha/tls/www.mysite.com.pem,60d



Make sure to delete the private key at www.mysite.com.pem file



sudo cp /etc/hiawatha/tls/www.mysite.com.pem /etc/hiawatha/tls/www.mysite.com.pem-BACKUP

sudo sed '/-----BEGIN PRIVATE KEY-----/,/-----END PRIVATE KEY-----/d' /etc/hiawatha/tls/www.mysite.com.pem



sudo systemctl restart hiawatha



(2) Renew SSL/TLS certificate :



Make sure your proxy setting is not set and port 80 can be accessed. Meanwhile, make sure disable TLS related setting before renew. You need to reset the TLS setting back after the renew.



cd ~/hiawatha-10.6/extra/letsencrypt

sudo ./letsencrypt renew



Make sure to delete the private key at www.mysite.com.pem file



sudo cp /etc/hiawatha/tls/www.mysite.com.pem /etc/hiawatha/tls/www.mysite.com.pem-BACKUP

sudo sed '/-----BEGIN PRIVATE KEY-----/,/-----END PRIVATE KEY-----/d' /etc/hiawatha/tls/www.mysite.com.pem



sudo systemctl restart hiawatha



* You can consider to write an auto renew script on cronjob for automatically update.



(3) Revoke SSL/TLS certificate : (Optional)



cd ~/hiawatha-10.6/extra/letsencrypt

sudo ./letsencrypt revoke /etc/hiawatha/tls/www.mysite.com.pem



sudo systemctl restart php7.0-fpm

sudo systemctl restart hiawatha/code>



(G) Hardening of Ubuntu Server



(a) sysctl



sudo nano /etc/sysctl.d/60-hiawatha.conf







sudo sysctl /etc/sysctl.d/60-hiawatha.conf -p



(b) Apparmor



sudo apt-get install apparmor-profiles apparmor-utils

sudo nano /etc/apparmor.d/usr.sbin.hiawatha







sudo aa-enforce hiawatha



If you have change some settings, you should reload the profile.



sudo apparmor_parser -r < /etc/apparmor.d/usr.sbin.hiawatha



If you want to disable this profile.



sudo ln -s /etc/apparmor.d/usr.sbin.hiawatha /etc/apparmor.d/disable/

sudo apparmor_parser -R < /etc/apparmor.d/usr.sbin.hiawatha



If you want to re-enable this profile after it has been disabled.



sudo rm /etc/apparmor.d/disable/usr.sbin.hiawatha

sudo apparmor_parser -r < /etc/apparmor.d/usr.sbin.hiawatha



Remarks :



If you encounter "500 Internal Server Error", you may consider to make the Apparmor to "Complain mode".



sudo aa-complain hiawatha



After several days browsing the website, you may consider to turn the Apparmor to "Enforce mode".



sudo aa-logprof



sudo aa-enforce hiawatha



It is because the captioned usr.sbin.hiawatha may not 100% work for you.



(c) Linux Malware Detect (Optional)



Linux Malware Detect Installation



* the captioned link may be out-dated and it is for your reference only



(d) MySQL



Create Normal User on MySQL



(e) fail2ban



sudo apt-get install fail2ban



Change the setting at /etc/fail2ban/jail.conf when necessary.



(H) Storage Performance Tuning



It is recommended to use SSD for the storage instead of hard drive for the excellent performance.



(a) SSD



Verify TRIM is supported :



sudo hdparm -I /dev/sda | grep TRIM



If the output is similar to the below which is supported :



* Data Set Management TRIM supported (limit 1 block)



If you install your Ubuntu in LVM, the TRIM is usually enabled by default. You can confirm it :



cat /etc/lvm/lvm.conf | grep issue_discards



If the output is similar to the below which is enabled :



issue_discards = 1



Then set the following to "deadline" if it is not done yet.



cat /sys/block/sda/queue/scheduler



noop [deadline] cfq



If not, set it :



sudo nano /etc/rc.local


Insert the following before "exit 0" :



echo 2048 > /sys/block/sda/queue/read_ahead_kb

echo 2048 > /sys/block/sda/queue/nr_requests

echo deadline > /sys/block/sda/queue/scheduler



* make sure your device is sda (or sdb ...)



To reload it or reboot your system :



sudo bash /etc/rc.local



After that, you need to edit the partition table (/etc/fstab) :



To make it looks like the following :



/dev/mapper/ubuntu--vg-root / ext4 noatime,nodiratime,norelatime,errors=remount-ro 0 1



sudo mount -a

sudo mount -o remount /



If you have an error after running the captioned commands, DO NOT reboot your system. You should correct the typo before doing so; otherwise, you cannot boot to your system again.



(b) Hard Drive



sudo nano /etc/rc.local



Insert the following before "exit 0" :



echo 2048 > /sys/block/sda/queue/read_ahead_kb

echo 2048 > /sys/block/sda/queue/nr_requests



* make sure your device is sda (or sdb ...)



To reload it or reboot your system :



sudo bash /etc/rc.local



After that, you need to edit the partition table (/etc/fstab) :



To make it looks like the following :



ext4 noatime,nodiratime,norelatime,errors=remount-ro 0 1



sudo mount -a

sudo mount -o remount /



If you have an error after running the captioned commands, DO NOT reboot your system. You should correct the typo before doing so; otherwise, you cannot boot to your system again.



(I) Redis for PHP Session



The PHP sessions are stored in the memory will increase the speed of a web site.



sudo apt-get install php-redis redis-server



sudo nano /etc/php/7.0/fpm/php.ini



Change to following to :



session.save_handler = redis

session.save_path = "tcp://127.0.0.1:6379"



To restart Hiawatha and PHP :



sudo systemctl restart hiawatha

sudo systemctl restart php7.0-fpm



To confirm if it is working or not :



redis-cli

127.0.0.1:6379> keys *



The result will be similar to :



1) "PHPREDIS_SESSION:038gl83953j9bfnf02ksts52q5"

2) "PHPREDIS_SESSION:p53j1t43mbdp49cvaq1nv37o97"

3) "PHPREDIS_SESSION:kuop27qq6g6q265gu29000ee21"

4) "PHPREDIS_SESSION:84n96cba8colp73td8mslnjgq2"



Type "quit" to exit.



(J) Optional



To further hardening Ubuntu Server, you may consider to set up firewall (UFW/iptables) and place the Ubuntu Server behind Unified Threats Management System (UTM) or Intrusion Prevention System (IPS).



Reference

Qualys SSL Labs

High-Tech Bridge

securityheaders.io

URL Rewrite for Hiawatha



That's all! See you.