Info
Content
Books
4 - Technical document...
Configuring tools for ...
Ubuntu server

Ubuntu server

Installing and configuring the Ubuntu server 16.04

Define a password for root user

ubuntu is the root user. The password can be useful in case SSH doesn't work any more, for recovery with VNC.

sudo su
passwd ubuntu

Mount the 2nd hard drive in /var/www

sudo mkfs.xfs -f /dev/vdb
sudo mkdir /var/www
sudo mount /dev/vdb /var/www
sudo blkid
sudo nano /etc/fstab

Remplacer l'UUID par celui donné par blkid pour /dev/sdb

LABEL=cloudimg-rootfs   /        ext4   defaults        0 0
UUID=4a4004ff-9191-432c-8693-75dbf533f061    /var/www           xfs    defaults        0       2

Set Time zone

dpkg-reconfigure tzdata
mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -uroot mysql -p
mysql -u root -p

SET GLOBAL time_zone = "Europe/Paris";
exit

And restart the system or just MySQL : service mysql reload

Set Locale

sudo /usr/share/locales/install-language-pack fr_FR

Security

Firewall

sudo ufw allow OpenSSH
sudo ufw deny 21
sudo ufw enable
sudo ufw status

OSSEC

Newest version : https://github.com/ossec/ossec-hids/ (Server/Agent Unix)

sudo su
apt install build-essential inotify-tools sendmail

if stops at ``

pidof apt
kill xxxx
cd /etc/mail/tls
sudo openssl dsaparam -out sendmail-common.prm 2048
sudo chown root:smmsp sendmail-common.prm
sudo chmod 0640 sendmail-common.prm
sudo dpkg --configure -a

wget https://github.com/ossec/ossec-hids/archive/2.9.0.tar.gz
tar xzvf 2.9.0.tar.gz
cd ossec-hids-2.9.0
./install.sh

choose local (not server) enter email choose default for all others options

nano /var/ossec/etc/ossec.conf add :

    <white_list>IPOfTheSever</white_list>
    <white_list>MyIP</white_list>

And check email_to and email_from

nano /var/ossec/rules/local_rules.xml add :

<rule id="100013" level="0">
    <if_sid>31103</if_sid>
    <url>^/phpmyadmin/</url>
    <description>Ignoring phpMyAdmin events.</description>
</rule>

<rule id="100014" level="0">
        <if_sid>31533</if_sid>
        <url>^/s/|^/phpmyadmin/|^/mail/mail/|^/live/poll|^/socket.io|^/index.php?date=|^/xhr_send?t=</url>
        <description>Ignoring Humhub Polls module activation events, phpMyAdmin, HackMd (socket.io), Piwik and OnlyOffice.</description>
</rule>

<rule id="100015" level="0">
    <if_sid>31151</if_sid>
    <url>^/mail/mail/</url>
    <description>Ignoring Humhub Ajax when disconnected</description>
</rule>

Automaticaly add IP in white list

Source

  1. create approved_humhub_list and chmod 666 in order to apache to be able to edit this file : 
    echo "" >/var/ossec/lists/approved_humhub_list
chmod 666 /var/ossec/lists/approved_humhub_list
  2. edit nano /var/ossec/etc/ossec.conf and in <rules></rules> add: 
     <list>lists/approved_humhub_list</list>
  3. service ossec restart
  4. In /var/ossec/rules/local_rules.xml, add: 
    <rule id="100016" level="0">
<if_sid>31100, 31103, 31533</if_sid>
<list field="srcip" lookup="address_match_key">lists/approved_humhub_list</list>
<description>Quiet: Approved admin</description>
</rule>
    id="100016" must be unique, change it if necessary !
  5. Update automaticaly /var/ossec/lists/approved_humhub_list.cdb every minutes adding this in root crontab: ```
          • /var/ossec/bin/ossec-makelists >/dev/null 2>&1 ```

Call this script in your website if current user is admin :

        $file = '/var/ossec/lists/approved_humhub_list';
        $ip = $_SERVER['REMOTE_ADDR'];
        $username = Yii::$app->user->identity->username;

        // read file
        $content = file_get_contents($file);

        // If user IP is not in white list
        if (!strpos($content, $ip)) {
            $lines = explode("\n", $content);

            // delete previous IPs corresponding to the current username
            foreach ($lines as $key => $line) {
                if (strpos($line, $username) || $line == '')
                    unset ($lines[$key]);
            }

            // add new IP
            $lines[] = $ip.":".$username;

            // Update file
            file_put_contents($file, implode("\n", $lines));
        }

GUI

Newest version : https://github.com/ossec/ossec-wui/releases

su web
cd /home/web/www
wget https://github.com/ossec/ossec-wui/archive/0.9.tar.gz
tar xzvf 0.9.tar.gz
rm 0.9.tar.gz
mv ossec-wui* ossec
cd ossec
sudo su
./setup.sh

choose www-data

/etc/init.d/apache2 restart
/etc/init.d/ossec start

http://serverIp/ossec

More details here

Fail2ban

sudo su
apt install fail2ban
nano /etc/fail2ban/jail.conf

ignoreip = 127.0.0.1/8 cramf.mooo.com
bantime  = 900
maxretry = 3

[ssh]

enabled = true
port    = ssh,sftp
filter  = sshd
logpath  = /var/log/auth.log
maxretry = 3

[apache-w00tw00t]
enabled = true
filter = apache-w00tw00t
action = iptables[name=Apache-w00tw00t,port=80,protocol=tcp]
logpath = /var/log/apache2/access*.log
maxretry = 1

[apache-noscript]
enabled  = true  

[apache-overflows]
enabled  = true 

[apache-badbots]
enabled  = true

nano /etc/fail2ban/filter.d/apache-w00tw00t.conf

[Definition]

failregex = ^<HOST> -.*"GET \/w00tw00t\.at\.ISC\.SANS\.DFind\:\).*".*

ignoreregex =

fail2ban-client reload
fail2ban-client status

Monitoring

apt install apache2 libcgi-fast-perl libapache2-mod-fcgid
a2enmod fcgid

Munin

apt install munin munin-node munin-plugins-extra
chown -R munin:munin /var/cache/munin/www
nano /etc/munin/munin.conf

Uncomment :

dbdir /var/lib/munin
htmldir /var/cache/munin/www
logdir /var/log/munin
rundir /var/run/munin
contact.someuser.command mail -s "Munin notification"

And replace email address

nano /etc/munin/apache.conf

Replace :

        Order allow,deny
        Allow from localhost 127.0.0.0/8 ::1
        Allow from all
        Options None

By :

        Require all granted
        Options FollowSymLinks SymLinksIfOwnerMatch

cd /etc/munin/plugins
sudo ln -s /usr/share/munin/plugins/mysql_
sudo ln -s /usr/share/munin/plugins/mysql_bytes
sudo ln -s /usr/share/munin/plugins/mysql_queries
sudo ln -s /usr/share/munin/plugins/mysql_slowqueries
sudo ln -s /usr/share/munin/plugins/mysql_threads
service munin-node restart
service apache2 reload

Go to http://IPAdress/munin If it doesn't work, create a Vhost in /etc/apache2/sites-available :

<VirtualHost *:80>
# Enable this for template generation
    ServerName munin.domain.ext

    ServerAdmin name@domain.ext
    DocumentRoot /var/cache/munin/www


# Enable this for cgi-based templates
#Alias /munin-cgi/static /var/cache/munin/www/static
#ScriptAlias /munin-cgi /usr/lib/munin/cgi/munin-cgi-html
#<Location /munin-cgi>
#    Order allow,deny
#    Allow from localhost 127.0.0.0/8 ::1
#    AuthUserFile /etc/munin/munin-htpasswd
#    AuthName "Munin"
#    AuthType Basic
#    require valid-user
#</Location>

<Directory /var/cache/munin/www>
     Require all granted
     Options FollowSymLinks SymLinksIfOwnerMatch
    # This file can be used as a .htaccess file, or a part of your apache
    # config file.
    #
    # For the .htaccess file option to work the munin www directory
    # (/var/cache/munin/www) must have "AllowOverride all" or something 
    # close to that set.
    #

    # AuthUserFile /etc/munin/munin-htpasswd
    # AuthName "Munin"
    # AuthType Basic
    # require valid-user

    # This next part requires mod_expires to be enabled.
    #

    # Set the default expiration time for files to 5 minutes 10 seconds from
    # their creation (modification) time.  There are probably new files by
    # that time. 
    #

    <IfModule mod_expires.c>
        ExpiresActive On
        ExpiresDefault M310
    </IfModule>

</Directory> 

# Enables fastcgi for munin-cgi-html if present
#<Location /munin-cgi>
#    <IfModule mod_fastcgi.c>
#        SetHandler fastcgi-script
#    </IfModule>
#</Location>

#<Location /munin-cgi/static>
#    SetHandler None
#</Location>

# Enables fastcgi for munin-cgi-graph if present
ScriptAlias /munin-cgi/munin-cgi-graph /usr/lib/munin/cgi/munin-cgi-graph
<Location /munin-cgi/munin-cgi-graph>
        Require all granted
    # AuthUserFile /etc/munin/munin-htpasswd
    # AuthName "Munin"
    # AuthType Basic
    # require valid-user
    <IfModule mod_fcgid.c>
        SetHandler fcgid-script
    </IfModule>
    <IfModule !mod_fcgid.c>
        SetHandler cgi-script
    </IfModule>
</Location>

ScriptAlias /munin-cgi/munin-cgi-html /usr/lib/munin/cgi/munin-cgi-html
<Location /munin-cgi/munin-cgi-html>
        Require all granted
    # AuthUserFile /etc/munin/munin-htpasswd
    # AuthName "Munin"
    # AuthType Basic
    # require valid-user
    <IfModule mod_fcgid.c>
        SetHandler fcgid-script
    </IfModule>
    <IfModule !mod_fcgid.c>
        SetHandler cgi-script
    </IfModule>
</Location>
</VirtualHost>

Monit

sudo su
apt install monit
systemctl enable monit
nano /etc/monit/monitrc

set daemon 120
set mailserver localhost
set alert name@domain.ext only on { timeout, icmp }
set httpd port 2812 and
allow admin:monit # require user 'admin' with password 'monit'

systemctl restart monit
ufw allow 2812

Got to http://IpAdress:2812

user 'admin' / password 'monit'

LAMP

Follow this documentation except the PHP paragraph

Install PHP7.1 (instead of 7.0 in Ubuntu 16.04)

sudo su
apt install -y python-software-properties
add-apt-repository ppa:ondrej/php
apt update
sudo apt-get install php7.1 libapache2-mod-php7.1 php7.1-mcrypt php7.1-mysql php7.1-bcmath php7.1-bz2 php7.1-ctype php7.1-curl php-date php7.1-dom php7.1-enchant php7.1-exif php7.1-fileinfo php7.1-ftp php7.1-gd php7.1-gettext php7.1-iconv php7.1-imagick php7.1-imap php7.1-mongodb php7.1-dev php7.1-intl php7.1-json php7.1-mbstring php7.1-mcrypt php7.1-memcache php7.1-memcached php7.1-mysql php7.1-mysqli php7.1-mysqlnd php7.1-pgsql php7.1-posix php7.1-redis php7.1-sockets php7.1-sqlite3 php7.1-tokenizer php7.1-wddx php7.1-xml php7.1-xmlreader php7.1-xmlrpc php7.1-xmlwriter php7.1-xsl php7.1-zip php7.1-ldap php7.1-apcu git curl zip unzip imagemagick
a2enmod 7.1
update-alternatives --config php

Select 7.1 version

List enabled apache2 modules : apachectl -t -D DUMP_MODULES Enable URL rewriting module : a2enmod rewrite

nano /etc/php/7.1/apache2/php.ini :

post_max_size = 20M
upload_max_filesize = 20M
max_file_uploads = 50
max_input_time = 180
memory_limit = 256M
date.timezone = Europe/Paris

sudo nano /etc/apache2/apache2.conf (replace xxx.xxx.xxx.xxx by IP server address) : Add to the end of the file : ServerName xxx.xxx.xxx.xxx

service apache2 reload

A bug can make Apache crash. Workaround : nano /etc/logrotate.d/apache2 Replace reload with restart

PHP 5.6 (for Communect)

https://phpraxis.wordpress.com/2016/05/16/install-php-5-6-or-5-5-in-ubuntu-16-04-lts-xenial-xerus/

sudo apt-add-repository -y ppa:ondrej/php
sudo apt-get -y update
sudo apt-get -y install php5.6-mysql php5.6-cli php5.6-curl php5.6-json php5.6-sqlite3 php5.6-mcrypt php5.6-curl php5.6-mcrypt php5.6-curl php5.6-gd php5.6-xml php5.6-xmlrpc php5.6-mbstring libapache2-mod-php5.6 php5.6-zip php5.6-dev php5.6-mongo php5.6-mongodb
sudo a2dismod php7.1 ; sudo a2enmod php5.6 ; sudo service apache2 restart ; echo 1 | sudo update-alternatives --config php

Optimize MySQL

systemctl edit mysql

[Service]
LimitNOFILE=infinity
LimitMEMLOCK=infinity

systemctl daemon-reload
service mysql reload

More info

Changing values for Ubuntu 16.04

Report for optimization software

phpMyAdmin

https://www.digitalocean.com/community/tutorials/how-to-install-and-secure-phpmyadmin-on-ubuntu-16-04 : use the same password for phpMyAdmin as for MySQL

Email

https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-postfix-as-a-send-only-smtp-server-on-ubuntu-16-04

sudo nano /etc/mailname

Enter domain name here (ex : domain.ext)

sudo nano /etc/aliases

# See man 5 aliases for format
postmaster:    root
root:          webmaster@mail.ext
monit: root

Access to the server with authentication (SSH and SFTP)

On your local computer, if you don't already have, create a public (~/.ssh/id_rsa.pub) and a private key (~/.ssh/id_rsa) (give a long pass phrase), protect your private key, save the pass phrase to avoid typing it at each connexion and display your public key :

ssh-keygen -t rsa
chmod 400 ~/.ssh/id_rsa
ssh-add ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub

Add the public key on the distant server file ~/.ssh/authorized_keys by pasting it in a new line. Connect by SSH or SFTP : ssh web@83.166.144.90 or ssh ubuntu@83.166.144.90

If you want to create your keys in specific files :

ssh-keygen -t rsa -f ~/.ssh/id_rsa_name_of_your_choice
nano ~/.ssh/config
Host 83.166.144.90
User web (or ubuntu)
IdentityFile /Users/username/.ssh/id_rsa_name_of_your_choice

www-data account

source

sudo su
adduser web
apt-get -y install bindfs
mkdir -p /home/web/www
chown web:web /home/web/www
chmod 755 /home/web/www
nano /etc/fstab

add

bindfs#/var/www /home/web/www fuse force-user=web,force-group=web,create-for-user=www-data,create-for-group=www-data,create-with-perms=0755,chgrp-ignore,chown-ignore,chmod-ignore 0 0

mount /home/web/www
nano /etc/rc.local

Add this line just before exit 0

    mount /home/web/www &

IMPORTANT : Now, to write in /var/www as www-data user, you must use the web user and access /var/www threw /home/web/www

DNS

https://wiki.gandi.net/fr/dns/zone/a-record

and then

https://www.digitalocean.com/community/tutorials/how-to-set-up-apache-virtual-hosts-on-ubuntu-16-04

SSL for HTTPS with Letsencrypt

source

sudo su
add-apt-repository ppa:certbot/certbot
apt update
apt install python-certbot-apache
ufw allow 443
crontab -e

Add : 15 3 * * * /usr/bin/certbot renew --quiet

To create a certificate to a domain (must be setup on apache2 with a2ensite /etc/apache2/sites-avalaible/domain.ext.conf; service apache2 reload;)

certbot --apache -d domain.ext

If doesn't work :

certbot --authenticator webroot --installer apache -d domain.ext

webroot is the folder where the website is installed

A new file is created : /etc/apache2/sites-available/domain.ext-le-ssl.conf

Node.Js

Each web app needs a specific version of Node.js. So create a new user for each web app and install Node.Js locally.

  1. Install NVM (Node version manager) : https://github.com/creationix/nvm#install-script
  2. edit the package.json file of the web app, look at the node version, and install node : nvm install 6.x.x; nvm use 6.x.x;. If you want the stable version : nvm install stable; nvm use stable;. LTS (recommanded) : nvm install --lts; nvm use --lts;
  3. Node.Js and NPM (Node Package Manager) will be installed

But if Node.js is executed by apache (using the system() or exec() command), Node.Js needs to be installed globally :

sudo apt install nodejs

LDAP

Not finalized, to resume

Warning before apt-get install phpldapadmin ! Check afterwards weather apache server still works, otherwise you have to do an apt-get remove --purge apache2 then reinstall, add modules that are missing a2enmod and show the list ls /etc/apache2/mods-available/ , add sites-availables, reconfigure HTTPS

Warning : it will appear only at install, it is necessary to choose a subdomain not to interact with another site, but I'm not sure.

First create the subdomain under apache a2ensite ldap.make.social.conf and publish it in HTTPS : certbot --apache -d ldap.make.social

https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-openldap-and-phpldapadmin-on-ubuntu-16-04

If apache does not work anymore after the installation : https://support.plesk.com/hc/en-us/articles/213946305-Apache-crashes-on-reload-and-websites-show-502-Bad-Gateway-seg-fault-or-similar-nasty-error-detected-in-the-parent-process

local backups

HackMD

Note the postgress container ID :

sudo docker ps -a

Create a backup script in the file /var/www/backups/scripts/hackmd.sh :

#!/bin/sh
DATE=`date +"%Y-%m-%d_%H-%M-%S"`
docker exec 80c6a4ef581a pg_dump hackmd -U postgres > /var/www/backups/hackmd/$DATE.sql
find /var/www/backups/hackmd/* -mtime +30 -exec rm {} \;

if find /var/www/backups/hackmd/$DATE.sql -type f -size +10c 2>/dev/null | grep -q .; then
    echo "The backup has run successfully"
else
    echo "The backup has was unsuccessful"
    echo "La sauvegarde de HackMd a echoue. C est probablement l identifiant du conteneur posgress du docker qui a change" | mail -s "Probleme de sauvegarde sur le serveur Infomaniak" contact@marc.fun
fi

Then schedule the daily backups of the database :

sudo crontab -e

and add :

1 3  * *  * /var/www/backups/scripts/hackmd.sh

MongoDB

Create a backup script in the file /var/www/backups/scripts/mongodb.sh :

#!/bin/sh
DATE=`date +"%Y-%m-%d_%H-%M-%S"`
mongodump --out /var/www/backups/mongodb/$DATE  --gzip
find /var/www/backups/mongodb/* -mtime +30 -exec rm -rf {} \;

Then schedule the daily backups of the database :

sudo crontab -e

and add :

3 3  * *  * /var/www/backups/scripts/mongodb.sh

Nextcloud and the other PHP/MySQL applications

su ubuntu
mkdir /home/web/www/backups-full-local
mkdir /home/web/www/backups
cd /home/web/www/backups
mkdir weekly
mkdir weekly/MySql
mkdir weekly/Zip
mkdir daily
mkdir daily/MySql
mkdir daily/Zip
mkdir hourly
mkdir hourly/MySql
mkdir hourly/Zip
cd /home/ubuntu
git clone https://gitlab.com/funkycram/files-and-sql-backup.git
cd files-and-sql-backup/configs
cp example.inc.php weekly-backup.inc.php
cp example.inc.php daily-backup.inc.php
cp example.inc.php hourly-backup.inc.php
chmod 700 *

Edit the file *.inc.php

sudo crontab -e :

5 4  * * 3 zip /var/www/backups-full-local/etc-backup.zip /etc -r
10 4  * * 3 zip /var/www/backups-full-local/home-and-var-www-backup.zip /home -r -x /home/web/www/backups/\*
5 3  * * 7 php /home/ubuntu/files-and-sql-backup/backup.php config=weekly-backup.inc.php
5 3  * * 1-6 php /home/ubuntu/files-and-sql-backup/backup.php config=daily-backup.inc.php
5 0  * * * php /home/ubuntu/files-and-sql-backup/backup.php config=hourly-backup.inc.php

Backups on the Hubic cloud server

  1. Create an account Hubic (we use the account contact@openappecosystem.cc)
  2. Create a folder called "backups"

On the server as root user :

  1. Install rclone
  2. Configure rclone for Hubic (leave blank "Hubic Client Id" and "Hubic Client Secret" and open to the URL asked at the end of the process in a second SSH terminal using elinks)
  3. crontab -e : 
    5 5  * *  * rclone --checkers=1 sync /var/www/backups remote:default/backups --dump-bodies

Interesting article if you want to encrypt backups : http://nogues.pro/blog/backup-hubic-duplicity-rsync.html

Copies of system files

su root; crontab -e :

15 3 * * * /usr/bin/certbot renew --quiet
20 3 * * * /usr/bin/certbot renew --quiet --renew-hook /usr/local/bin/renew.sh
1 3  * *  * /var/www/backups/scripts/hackmd.sh
3 3  * *  * /var/www/backups/scripts/mongodb.sh
5 3  * *  * php /var/www/backups/files-and-sql-backup/backup.php config=all.inc.php
5 4  * *  * rclone sync /var/www/backups remote:default/backups

su web; crontab -e :

*/2 * * * * curl https://co2.make.social/co2/test/docron
15 3 * * * curl https://wiki.communecter.org/fr/update-gitbook.php
17 3 * * * curl https://wiki.communecter.org/en/update-gitbook.php
20 3 * * * curl https://doc.cosystem.cc/update-gitbook.php
30 * * * * /home/web/www/ecolesinnovantes.info/public_html/protected/yii cron/hourly >/dev/null 2>&1
00 18 * * * /home/web/www/ecolesinnovantes.info/public_html/protected/yii cron/daily >/dev/null 2>&1

sudo nano /etc/rc.local :

mount /home/web/www &
/var/www/keycloak/keycloak-server/bin/standalone.sh &
docker-compose -f /var/www/docker/docker-hackmd/docker-compose.yml up &
#su ubuntu -c 'cd /var/www/wiki.communecter.org; node wiki start'
su dialoguea -c "/var/www/dialoguea.co.tools/dialoguea/start-server.sh" &
su cryptpad -c "/var/www/cryptpad.co.tools/cryptpad/start-server.sh" &
exit 0

Increase partition size

If disk space has been increased with Infomaniak, the partition must be increased to the new size : xfs_growfs -d /var/www

Details
Revision #1
Created 1 week ago by Tom
Updated 1 week ago by Tom
Actions
Export
Back to top