Ubuntu server
Installing and configuring the Ubuntu server 16.04
Return to the list of tool configurations
Define a password for root user
ubuntu is the root user. The password can be useful in case SSH doesn't work any more, for recovery with VNC.
sudo su
passwd ubuntu
Mount the 2nd hard drive in /var/www
sudo mkfs.xfs -f /dev/vdb
sudo mkdir /var/www
sudo mount /dev/vdb /var/www
sudo blkid
sudo nano /etc/fstab
Remplacer l'UUID par celui donné par blkid pour /dev/sdb
LABEL=cloudimg-rootfs / ext4 defaults 0 0
UUID=4a4004ff-9191-432c-8693-75dbf533f061 /var/www xfs defaults 0 2
Set Time zone
dpkg-reconfigure tzdata
mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -uroot mysql -p
mysql -u root -p
SET GLOBAL time_zone = "Europe/Paris";
exit
And restart the system or just MySQL : service mysql reload
Set Locale
Schedule server restart every month automaticaly
crontab -e
Add :
#Restart server every first Monday of every month
45 2 1-7 * * [ "$(date '+\%u')" = "1" ] && /sbin/shutdown -r +5
Security
Firewall
sudo ufw allow OpenSSH
sudo ufw deny 21
sudo ufw enable
sudo ufw status
OSSEC
Newest version : https://github.com/ossec/ossec-hids/ (Server/Agent Unix)
sudo su
apt install build-essential inotify-tools sendmail
if stops at ``
pidof apt
kill xxxx
cd /etc/mail/tls
sudo openssl dsaparam -out sendmail-common.prm 2048
sudo chown root:smmsp sendmail-common.prm
sudo chmod 0640 sendmail-common.prm
sudo dpkg --configure -a
wget https://github.com/ossec/ossec-hids/archive/2.9.0.tar.gz
tar xzvf 2.9.0.tar.gz
cd ossec-hids-2.9.0
./install.sh
choose local (not server) enter email choose default for all others options
nano /var/ossec/etc/ossec.conf add :
<white_list>IPOfTheSever</white_list>
<white_list>MyIP</white_list>
And check email_to and email_from
nano /var/ossec/rules/local_rules.xml add :
<rule id="100013" level="0">
<if_sid>31103</if_sid>
<url>^/phpmyadmin/</url>
<description>Ignoring phpMyAdmin events.</description>
</rule>
<rule id="100014" level="0">
<if_sid>31533</if_sid>
<url>^/s/|^/phpmyadmin/|^/mail/mail/|^/live/poll|^/socket.io|^/index.php?date=|^/xhr_send?t=</url>
<description>Ignoring Humhub Polls module activation events, phpMyAdmin, HackMd (socket.io), Piwik and OnlyOffice.</description>
</rule>
<rule id="100015" level="0">
<if_sid>31151</if_sid>
<url>^/mail/mail/|^/socket.io</url>
<description>Ignoring Humhub Ajax when disconnected, CodiMd (socket.io)</description>
</rule>
Automaticaly add IP in white list
- create approved_humhub_list and chmod 666 in order to apache to be able to edit this file :
echo "" >/var/ossec/lists/approved_humhub_list
chmod 666 /var/ossec/lists/approved_humhub_list
- edit
nano /var/ossec/etc/ossec.confand in<rules></rules>add:
<list>lists/approved_humhub_list</list>
service ossec restart- In
/var/ossec/rules/local_rules.xml, add:
<rule id="100016" level="0">
<if_sid>31100, 31103, 31533</if_sid>
<list field="srcip" lookup="address_match_key">lists/approved_humhub_list</list>
<description>Quiet: Approved admin</description>
</rule>
id="100016" must be unique, change it if necessary ! 6. Update automaticaly /var/ossec/lists/approved_humhub_list.cdb every minutes adding this in root crontab:
* * * * * /var/ossec/bin/ossec-makelists >/dev/null 2>&1
Call this script in your website if current user is admin :
$file = '/var/ossec/lists/approved_humhub_list';
$ip = $_SERVER['REMOTE_ADDR'];
$username = Yii::$app->user->identity->username;
// read file
$content = file_get_contents($file);
// If user IP is not in white list
if (!strpos($content, $ip)) {
$lines = explode("\n", $content);
// delete previous IPs corresponding to the current username
foreach ($lines as $key => $line) {
if (strpos($line, $username) || $line == '')
unset ($lines[$key]);
}
// add new IP
$lines[] = $ip.":".$username;
// Update file
file_put_contents($file, implode("\n", $lines));
}
GUI
Newest version : https://github.com/ossec/ossec-wui/releases
su web
cd /home/web/www
wget https://github.com/ossec/ossec-wui/archive/0.9.tar.gz
tar xzvf 0.9.tar.gz
rm 0.9.tar.gz
mv ossec-wui* ossec
cd ossec
sudo su
./setup.sh
choose www-data
/etc/init.d/apache2 restart
/etc/init.d/ossec start
http://serverIp/ossec
Fail2ban
sudo su
apt install fail2ban
nano /etc/fail2ban/jail.conf
ignoreip = 127.0.0.1/8 cramf.mooo.com
bantime = 900
maxretry = 3
[ssh]
enabled = true
port = ssh,sftp
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
[apache-w00tw00t]
enabled = true
filter = apache-w00tw00t
action = iptables[name=Apache-w00tw00t,port=80,protocol=tcp]
logpath = /var/log/apache2/access*.log
maxretry = 1
[apache-noscript]
enabled = true
[apache-overflows]
enabled = true
[apache-badbots]
enabled = true
nano /etc/fail2ban/filter.d/apache-w00tw00t.conf
[Definition]
failregex = ^<HOST> -.*"GET \/w00tw00t\.at\.ISC\.SANS\.DFind\:\).*".*
ignoreregex =
fail2ban-client reload
fail2ban-client status
Monitoring
apt install apache2 libcgi-fast-perl libapache2-mod-fcgid
a2enmod fcgid
Munin
apt install munin munin-node munin-plugins-extra
chown -R munin:munin /var/cache/munin/www
nano /etc/munin/munin.conf
Uncomment :
dbdir /var/lib/munin
htmldir /var/cache/munin/www
logdir /var/log/munin
rundir /var/run/munin
contact.someuser.command mail -s "Munin notification"
And replace email address
nano /etc/munin/apache.conf
Replace :
Order allow,deny
Allow from localhost 127.0.0.0/8 ::1
Allow from all
Options None
By :
Require all granted
Options FollowSymLinks SymLinksIfOwnerMatch
cd /etc/munin/plugins
sudo ln -s /usr/share/munin/plugins/mysql_
sudo ln -s /usr/share/munin/plugins/mysql_bytes
sudo ln -s /usr/share/munin/plugins/mysql_queries
sudo ln -s /usr/share/munin/plugins/mysql_slowqueries
sudo ln -s /usr/share/munin/plugins/mysql_threads
service munin-node restart
service apache2 reload
Go to http://IPAdress/munin If it doesn't work, create a Vhost in /etc/apache2/sites-available :
<VirtualHost *:80>
# Enable this for template generation
ServerName munin.domain.ext
ServerAdmin name@domain.ext
DocumentRoot /var/cache/munin/www
# Enable this for cgi-based templates
#Alias /munin-cgi/static /var/cache/munin/www/static
#ScriptAlias /munin-cgi /usr/lib/munin/cgi/munin-cgi-html
#<Location /munin-cgi>
# Order allow,deny
# Allow from localhost 127.0.0.0/8 ::1
# AuthUserFile /etc/munin/munin-htpasswd
# AuthName "Munin"
# AuthType Basic
# require valid-user
#</Location>
<Directory /var/cache/munin/www>
Require all granted
Options FollowSymLinks SymLinksIfOwnerMatch
# This file can be used as a .htaccess file, or a part of your apache
# config file.
#
# For the .htaccess file option to work the munin www directory
# (/var/cache/munin/www) must have "AllowOverride all" or something
# close to that set.
#
# AuthUserFile /etc/munin/munin-htpasswd
# AuthName "Munin"
# AuthType Basic
# require valid-user
# This next part requires mod_expires to be enabled.
#
# Set the default expiration time for files to 5 minutes 10 seconds from
# their creation (modification) time. There are probably new files by
# that time.
#
<IfModule mod_expires.c>
ExpiresActive On
ExpiresDefault M310
</IfModule>
</Directory>
# Enables fastcgi for munin-cgi-html if present
#<Location /munin-cgi>
# <IfModule mod_fastcgi.c>
# SetHandler fastcgi-script
# </IfModule>
#</Location>
#<Location /munin-cgi/static>
# SetHandler None
#</Location>
# Enables fastcgi for munin-cgi-graph if present
ScriptAlias /munin-cgi/munin-cgi-graph /usr/lib/munin/cgi/munin-cgi-graph
<Location /munin-cgi/munin-cgi-graph>
Require all granted
# AuthUserFile /etc/munin/munin-htpasswd
# AuthName "Munin"
# AuthType Basic
# require valid-user
<IfModule mod_fcgid.c>
SetHandler fcgid-script
</IfModule>
<IfModule !mod_fcgid.c>
SetHandler cgi-script
</IfModule>
</Location>
ScriptAlias /munin-cgi/munin-cgi-html /usr/lib/munin/cgi/munin-cgi-html
<Location /munin-cgi/munin-cgi-html>
Require all granted
# AuthUserFile /etc/munin/munin-htpasswd
# AuthName "Munin"
# AuthType Basic
# require valid-user
<IfModule mod_fcgid.c>
SetHandler fcgid-script
</IfModule>
<IfModule !mod_fcgid.c>
SetHandler cgi-script
</IfModule>
</Location>
</VirtualHost>
Monit
sudo su
apt install monit
systemctl enable monit
nano /etc/monit/monitrc
set daemon 120
set mailserver localhost
set alert name@domain.ext only on { timeout, icmp }
set httpd port 2812 and
allow admin:monit # require user 'admin' with password 'monit'
systemctl restart monit
ufw allow 2812
Got to http://IpAdress:2812
user 'admin' / password 'monit'
LAMP
Follow this documentation except the PHP paragraph
Install PHP7.1 (instead of 7.0 in Ubuntu 16.04)
sudo su
apt install -y python-software-properties (pour Ubuntu 16.04)
apt install software-properties-common (pour Ubuntu 18.04)
add-apt-repository ppa:ondrej/php
apt update
sudo apt-get install php7.1 libapache2-mod-php7.1 php7.1-mcrypt php7.1-mysql php7.1-bcmath php7.1-bz2 php7.1-ctype php7.1-curl php-date php7.1-dom php7.1-enchant php7.1-exif php7.1-fileinfo php7.1-ftp php7.1-gd php7.1-gettext php7.1-iconv php7.1-imagick php7.1-imap php7.1-mongodb php7.1-dev php7.1-intl php7.1-json php7.1-mbstring php7.1-mcrypt php7.1-memcache php7.1-memcached php7.1-mysql php7.1-mysqli php7.1-mysqlnd php7.1-pgsql php7.1-posix php7.1-redis php7.1-sockets php7.1-sqlite3 php7.1-tokenizer php7.1-wddx php7.1-xml php7.1-xmlreader php7.1-xmlrpc php7.1-xmlwriter php7.1-xsl php7.1-zip php7.1-ldap php7.1-apcu git curl zip unzip imagemagick
a2enmod php7.1
update-alternatives --config php
Select 7.1 version
List enabled apache2 modules : apachectl -t -D DUMP_MODULES Enable URL rewriting module : a2enmod rewrite
nano /etc/php/7.1/apache2/php.ini :
post_max_size = 20M
upload_max_filesize = 20M
max_file_uploads = 50
max_input_time = 180
memory_limit = 256M
date.timezone = Europe/Paris
session.gc_maxlifetime = 2592000
session.gc_maxlifetime enables to stay connected 1 month
sudo nano /etc/apache2/apache2.conf (replace xxx.xxx.xxx.xxx by IP server address) : Add to the end of the file : ServerName xxx.xxx.xxx.xxx
service apache2 reload
A bug can make Apache crash. Workaround : nano /etc/logrotate.d/apache2 Replace reload with restart
PHP 5.6 (for Communect)
https://phpraxis.wordpress.com/2016/05/16/install-php-5-6-or-5-5-in-ubuntu-16-04-lts-xenial-xerus/
sudo apt-add-repository -y ppa:ondrej/php
sudo apt-get -y update
sudo apt-get -y install php5.6-mysql php5.6-cli php5.6-curl php5.6-json php5.6-sqlite3 php5.6-mcrypt php5.6-curl php5.6-mcrypt php5.6-curl php5.6-gd php5.6-xml php5.6-xmlrpc php5.6-mbstring libapache2-mod-php5.6 php5.6-zip php5.6-dev php5.6-mongo php5.6-mongodb
sudo a2dismod php7.1 ; sudo a2enmod php5.6 ; sudo service apache2 restart ; echo 1 | sudo update-alternatives --config php
Optimize MySQL
systemctl edit mysql
[Service]
LimitNOFILE=infinity
LimitMEMLOCK=infinity
systemctl daemon-reload
service mysql reload
Changing values for Ubuntu 16.04
Report for optimization software
phpMyAdmin
https://www.digitalocean.com/community/tutorials/how-to-install-and-secure-phpmyadmin-on-ubuntu-16-04 : use the same password for phpMyAdmin as for MySQL
sudo nano /etc/mailname
Enter domain name here (ex : domain.ext)
sudo nano /etc/aliases
# See man 5 aliases for format
postmaster: root
root: webmaster@mail.ext
monit: root
Access to the server with authentication (SSH and SFTP)
On your local computer, if you don't already have, create a public (~/.ssh/id_rsa.pub) and a private key (~/.ssh/id_rsa) (give a long pass phrase), protect your private key, save the pass phrase to avoid typing it at each connexion and display your public key :
ssh-keygen -t rsa
chmod 400 ~/.ssh/id_rsa
ssh-add ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub
Add the public key on the distant server file ~/.ssh/authorized_keys by pasting it in a new line. Connect by SSH or SFTP : ssh web@83.166.144.90 or ssh ubuntu@83.166.144.90
If you want to create your keys in specific files :
ssh-keygen -t rsa -f ~/.ssh/id_rsa_name_of_your_choice
nano ~/.ssh/config
Host 83.166.144.90
User web (or ubuntu)
IdentityFile /Users/username/.ssh/id_rsa_name_of_your_choice
www-data account
sudo su
adduser web
apt-get -y install bindfs
mkdir -p /home/web/www
chown web:web /home/web/www
chmod 755 /home/web/www
nano /etc/fstab
add
bindfs#/var/www /home/web/www fuse force-user=web,force-group=web,create-for-user=www-data,create-for-group=www-data,create-with-perms=0755,chgrp-ignore,chown-ignore,chmod-ignore 0 0
mount /home/web/www
nano /etc/rc.local
Add this line just before exit 0
mount /home/web/www &
IMPORTANT : Now, to write in /var/www as www-data user, you must use the web user and access /var/www threw /home/web/www
DNS
https://wiki.gandi.net/fr/dns/zone/a-record
and then
https://www.digitalocean.com/community/tutorials/how-to-set-up-apache-virtual-hosts-on-ubuntu-16-04
SSL for HTTPS with Letsencrypt
sudo su
add-apt-repository ppa:certbot/certbot
apt update
apt install python-certbot-apache
••ufw allow 443
crontab -e
Add : 15 3 * * * /usr/bin/certbot renew --quiet
To create a certificate to a domain (must be setup on apache2 with a2ensite /etc/apache2/sites-avalaible/domain.ext.conf; service apache2 reload;)
certbot --apache -d domain.ext
If doesn't work :
certbot --authenticator webroot --installer apache -d domain.ext
webroot is the folder where the website is installed
A new file is created : /etc/apache2/sites-available/domain.ext-le-ssl.conf
Node.Js
Each web app needs a specific version of Node.js. So create a new user for each web app and install Node.Js locally.
- Install NVM (Node version manager) : https://github.com/creationix/nvm#install-script
- edit the package.json file of the web app, look at the node version, and install node :
nvm install 6.x.x; nvm use 6.x.x;. If you want the stable version :nvm install stable; nvm use stable;. LTS (recommanded) :nvm install --lts; nvm use --lts; - Node.Js and NPM (Node Package Manager) will be installed
But if Node.js is executed by apache (using the system() or exec() command), Node.Js needs to be installed globally :
sudo apt install nodejs
LDAP
Not finalized, to resume
Warning before apt-get install phpldapadmin ! Check afterwards weather apache server still works, otherwise you have to do an apt-get remove --purge apache2 then reinstall, add modules that are missing a2enmod and show the list ls /etc/apache2/mods-available/ , add sites-availables, reconfigure HTTPS
Warning : it will appear only at install, it is necessary to choose a subdomain not to interact with another site, but I'm not sure.
First create the subdomain under apache a2ensite ldap.make.social.conf and publish it in HTTPS : certbot --apache -d ldap.make.social
If apache does not work anymore after the installation : https://support.plesk.com/hc/en-us/articles/213946305-Apache-crashes-on-reload-and-websites-show-502-Bad-Gateway-seg-fault-or-similar-nasty-error-detected-in-the-parent-process
local backups
HackMd
Note the postgress container ID :
sudo docker ps -a
Create a backup script in the file /var/www/backups/scripts/hackmd.sh :
#!/bin/sh
DATE=`date +"%Y-%m-%d_%H-%M-%S"`
docker exec 80c6a4ef581a pg_dump hackmd -U postgres > /var/www/backups/hackmd/$DATE.sql
find /var/www/backups/hackmd/* -mtime +30 -exec rm {} \;
if find /var/www/backups/hackmd/$DATE.sql -type f -size +10c 2>/dev/null | grep -q .; then
echo "The backup has run successfully"
else
echo "The backup has was unsuccessful"
echo "La sauvegarde de HackMd a echoue. C est probablement l identifiant du conteneur posgress du docker qui a change" | mail -s "Probleme de sauvegarde sur le serveur Infomaniak" contact@marc.fun
fi
Then schedule the daily backups of the database :
sudo crontab -e
and add :
1 3 * * * /var/www/backups/scripts/hackmd.sh
MongoDb
Create a backup script in the file /var/www/backups/scripts/mongodb.sh :
#!/bin/sh
DATE=`date +"%Y-%m-%d_%H-%M-%S"`
mongodump --out /var/www/backups/mongodb/$DATE --gzip
find /var/www/backups/mongodb/* -mtime +30 -exec rm -rf {} \;
Then schedule the daily backups of the database :
sudo crontab -e
and add :
3 3 * * * /var/www/backups/scripts/mongodb.sh
Nextcloud and the other PHP/MySQL applications
su ubuntu
mkdir /home/web/www/backups-full-local
mkdir /home/web/www/backups
cd /home/web/www/backups
mkdir weekly
mkdir weekly/MySql
mkdir weekly/Zip
mkdir daily
mkdir daily/MySql
mkdir daily/Zip
mkdir hourly
mkdir hourly/MySql
mkdir hourly/Zip
cd /home/ubuntu
git clone https://gitlab.com/funkycram/files-and-sql-backup.git
cd files-and-sql-backup/configs
cp example.inc.php weekly-backup.inc.php
cp example.inc.php daily-backup.inc.php
cp example.inc.php hourly-backup.inc.php
chmod 700 *
Edit the file *.inc.php
sudo crontab -e :
5 4 * * 3 zip /var/www/backups-full-local/etc-backup.zip /etc -r
10 4 * * 3 zip /var/www/backups-full-local/home-and-var-www-backup.zip /home -r -x /home/web/www/backups/\*
5 3 * * 7 php /home/ubuntu/files-and-sql-backup/backup.php config=weekly-backup.inc.php
5 3 * * 1-6 php /home/ubuntu/files-and-sql-backup/backup.php config=daily-backup.inc.php
5 0 * * * php /home/ubuntu/files-and-sql-backup/backup.php config=hourly-backup.inc.php
Backups on the Hubic cloud server
- Create an account Hubic (we use the account contact@openappecosystem.cc)
- Create a folder called "backups"
On the server as root user :
- Install rclone
- Configure rclone for Hubic (leave blank "Hubic Client Id" and "Hubic Client Secret" and open to the URL asked at the end of the process in a second SSH terminal using elinks)
-
crontab -e:
5 5 * * * rclone --checkers=1 sync /var/www/backups remote:default/backups --dump-bodies
Interesting article if you want to encrypt backups : http://nogues.pro/blog/backup-hubic-duplicity-rsync.html
Copies of system files
su root; crontab -e :
15 3 * * * /usr/bin/certbot renew --quiet
20 3 * * * /usr/bin/certbot renew --quiet --renew-hook /usr/local/bin/renew.sh
1 3 * * * /var/www/backups/scripts/hackmd.sh
3 3 * * * /var/www/backups/scripts/mongodb.sh
5 3 * * * php /var/www/backups/files-and-sql-backup/backup.php config=all.inc.php
5 4 * * * rclone sync /var/www/backups remote:default/backups
su web; crontab -e :
*/2 * * * * curl https://co2.make.social/co2/test/docron
15 3 * * * curl https://wiki.communecter.org/fr/update-gitbook.php
17 3 * * * curl https://wiki.communecter.org/en/update-gitbook.php
20 3 * * * curl https://doc.cosystem.cc/update-gitbook.php
30 * * * * /home/web/www/ecolesinnovantes.info/public_html/protected/yii cron/hourly >/dev/null 2>&1
00 18 * * * /home/web/www/ecolesinnovantes.info/public_html/protected/yii cron/daily >/dev/null 2>&1
sudo nano /etc/rc.local :
mount /home/web/www &
/var/www/keycloak/keycloak-server/bin/standalone.sh &
docker-compose -f /var/www/docker/docker-hackmd/docker-compose.yml up &
#su ubuntu -c 'cd /var/www/wiki.communecter.org; node wiki start'
su dialoguea -c "/var/www/dialoguea.co.tools/dialoguea/start-server.sh" &
su cryptpad -c "/var/www/cryptpad.co.tools/cryptpad/start-server.sh" &
exit 0
Increase partition size
If disk space has been increased with Infomaniak, the partition must be increased to the new size (check if partition system is XFS) :
xfs_growfs -d /var/www
Remove old kernels
uname -r
dpkg --list | grep linux-image
apt --purge remove linux-image-XXX* (XXX is old kernel number)
dpkg --list | grep linux-headers
apt --purge remove linux-headers-XXX*
dpkg --list | grep linux-modules
apt --purge remove linux-modules-XXX*
update-grub2
List largest installed packages
dpkg-query -Wf '${Installed-Size}\t${Package}\n' | sort -n••