Tag: linux

Installing a software RAID 10 Debian system with LUKS disk encryption

Post author By Theodotos Andreou
Post date February 3, 2018
13 Comments on Installing a software RAID 10 Debian system with LUKS disk encryption

In this guide we will be installing Debian 9 (aka stretch) on a physical server with 4 disks. The role of this machice is to be used as a Storage/NAS system. We will create a software RAID 10 setup, with LVM and LUKS full disk encryption. Our goals:

Install a Debian 9 with RAID10/LVM/LUKS.
Secure SSH.
Enable Firewall (UFW).
Setup bonding with the two network cards.
Setup remote system unlock with dropbear and initramfs.
Setup disk-monitoring with smartmontools and mdadm.
Setup kexec for faster reboots.

Computer specs

Type: HP ProLiant MicroServer
CPU: AMD Turion(tm) II Neo N40L Dual-Core Processor
RAM: 2GB
Disks: 4x3TB SATA
Network:
- Broadcom Limited NetXtreme BCM5723 Gigabit Ethernet PCIe (on board)
- Intel Corporation 82574L Gigabit Network Connection (extra)

Assumptions

Server IP: 192.168.1.10
Netmask: 255.255.255.0
Gateway IP: 192.168.1.1
DNS IP: 192.168.1.1
Hostname: storage.example.com

Install Debian stretch

Basic Settings

It would probably be more clear if there were screenshots for each step, but this was an installation on a physical server and taking photos for each step, opposes my laziness :). Just follow the instructions and you will be fine.

Choose: Install
Language: English
Country: other
Europe: Cyprus
Country to base default locale settings: United States
Keymap to use: American English
Primary network interface: enp3s0: Broadcom Limeted NetXtream BCM5723 Gigabit Ethernet PCIe
Let it get an IP from DHCP
Hostname: storage
Domain name: example.com
Root password: SomethingBigAndUnpredictable
Re-enter password to verify: SomethingBigAndUnpredictable
Full name: Sysadmin
Username: admin
Choose a password for the new user: AlsoSomethingBigAndUnpredictable
Re-enter password to verify: AlsoSomethingBigAndUnpredictable
Select your time zone: Asia/Nicosia
Partitioning method: Manual

Feel free to adjust the above according to your own preferences.

Partitioning

There are 4 disks of 3TB each (3.0 TB SATA):

SCSI1 (0,0,0) (sda)
SCSI2 (0,0,0) (sdb)
SCSI3 (0,0,0) (sdc)
SCSI4 (0,0,0) (sdd)

Not really SCSI but SATA in fact.

Partition the devices

Then create a raid partition for /boot:

Select the free space of sda and ‘Enter’
Create a new partition
New partition size: 512 MB
Location of new partition: Beginning
Use as: physical volume for RAID
Done setting up the partition

Lastly create the raid partition to be used by the encrypted volume:

Select the free space of sda and ‘Enter’
Create a new partition
New partition size: 3.0 TB
Location of new partition: Beginning
Use as: physical volume for RAID
Done setting up the partition

Repeat the above steps for sdb, sdc and sdd.

Setup Software RAID 10

First select ‘Configure software RAID’ and follow these steps:

Write the changes to the storage devices and configure RAID? Yes

The we create the software RAID (MD) devices. First we create device md0 for /boot:

Create MD Device
RAID10
Number of active devices inn the RAID10 array: 4
Number of spare devices inn the RAID10 array: 0
Active devices for the RAID10 array (use ‘Space bar’ to select)
- /dev/sda2
- /dev/sdb2
- /dev/sdc2
- /dev/sdd2
Press ‘Continue’ when done.

Then we create the software RAID device to be used for the encrypted volume (md1):

Create MD Device
RAID10
Number of active devices inn the RAID10 array: 4
Number of spare devices inn the RAID10 array: 0
Active devices for the RAID10 array (use the ‘Space bar’ to select)
- /dev/sda3
- /dev/sdb3
- /dev/sdc3
- /dev/sdd3
Press ‘Continue’ when done.
Press ‘Finish’ when done.

Create the /boot volume

When done press ‘Finish partitioning and write changes to disk’.`

When finished you will see a ‘RAID10 device #0 1GB Software RAID device’:

Select: #1 1.0GB
Use as: Ext4 journaling file system
Mount point: /boot
Done setting up the partition

Setup the encrypted volume

We will be using the software RAID /dev/md1 device for the encrypted volume.

Now select ‘Configure encrypted volumes’ and follow these steps:

Write the changes to disk and configure encrypted volumes? Yes

Create encrypted volumes

Select: /dev/md1

Erase data: yes (this will take a long time)

Done setting up the partition

Write the changes to disk and configure encrypted volumes? Yes

Finish

Encryption passphrase: MyVeryLongEncryptionPassphrase

Re-enter the passphrase to verify: MyVeryLongEncryptionPassphrase

Setup LVM

Next we select the ‘Configure the Logical Volume Manager’ option and follow these steps:

Write the changes to disks and configure LVM? Yes

Create volume group

Volume group name: VG00

Devices for the new volume group:

/dev/mapper/md1_crypt

Then we create the Logical Volumes (LV). First let’s create a SWAP volume:

Create logical volume
Volume group: VG00
Logical volume name: SWAP
Logical volume size: 2048MB (2G is more than enough for this system)

Lastly we create the system (ROOT) volume. On an enterprise installation we may want to use different volumes for /usr, /home, /var, etc but for a home installation we will be fine to use just one.

Create logical volume
Volume group: VG00
Logical volume name: ROOT
Logical volume size: 5996818MB (All available space)

Press ‘Finish’ when done.

Start the installation

After all the steps are completed these Logical Volumes will be present on the system:

LVM VG VG00, LV ROOT 0 6.0 TB
LVM VG VG00, LV SWAP 0 2.0 GB

Create the ROOT filesystem

Under the ‘LVM VG VG00, LV ROOT 0 6.0 TB’ line select the ‘#1 6.0TB’ option:

Use as: Ext4 journaling file system
Mount point: /
Done setting up this partition

Create the SWAP space

Under the ‘LVM VG VG00, LV SWAP 0 2.0 GB’ line select the ‘#1 2.0GB’ option:

Use as: swap area
Done setting up this partition

Now we are ready to write the changes and start the installation. Press the ‘Finish partitioning and write changes to disk’ option to continue:

Write the changes to disks? Yes

Wait for the base install to finish. Then select a country close to you. No debian mirrors in Cyprus so I use UK:

Debian archive mirror country: United Kingdom
Debian archive mirror: ftp.uk.debian.org
HTTP proxy: (none)

Wait for the APT configuration to Finish.

Participate in the package usage survey: no

Choose software to install:

SSH server
standard system utilities

Wait while software is installing

Install the GRUB boot loader to the master boot record.
Device for boot loader installation:
- /dev/sda

Wait for the installation to finish and reboot. Remember to remove the USB during the reboot cycle.

Post install steps

During start-up you will see the ‘Please unlock md1_crypt’ prompt. Type your LUKS passphrase to unlock the disk and continue.

Update and Upgrade

# apt update && apt -y dist-upgrade

Install essential packages

# apt -y install vim htop multitail ntp byobu ufw unattended-upgrades downtimed

Secure ssh

You need to generate an SSH key pair on you PC, if you don’t have one (you should!):

$ ssh-keygen -b 4096

Copy the public key:

$ cat ~/.ssh/id_rsa.pub

Paste the public key at the end of the /root/.ssh/authorized_keys file in your server and try to login from your PC:

$ ssh root@192.168.1.10

Some final adjustments on your SSH config (/etc/ssh/sshd_config). Change these values:

Port 2233
PasswordAuthentication no

Restart SSH:

# systemctl restart ssh.service

Enable the UFW firewall

We are using port 2233 for SSH so we need to allow that and enable the firewall:

# ufw allow 2233/tcp
# ufw enable

Setup bonding

Since we have two ethernet cards, we may take advantage of thr Linux bonding feature and join them as one. We will be using the Adaptive load balancing mode which provides load balancing of transmit, load balancing of receive for IPv4 and requires no configuration from the switch side.

First we need to install ifenslave:

# apt -y install ifenslave

Set up this in /etc/network/interfaces:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# enp2s0 is manually configured, and slave to the "bond0" bonded NIC
auto enp2s0
iface eth0 inet manual
    bond-master bond0

# enp3s01 is also manually configured, thus creating a 2-link bond.
auto enp3s0
iface eth1 inet manual
    bond-master bond0

# bond0 is the bonded NIC and can be used like any other normal NIC.
# bond0 is configured using static network information.
auto bond0
iface bond0 inet static
    address 192.168.1.10
    gateway 192.168.1.1
    netmask 255.255.255.0

    # bond0 uses adaptive load balancing
    bond-mode 6
    bond-miimon 100
    bond-slaves enp2s0 enp3s0

An ifup bond0 should bring the bonded interface up. Or you can just reboot.

Setup remote-unlock with dropbear

The server will be a headless system, located in a difficult to access location. So we need a way to unlock it when a power failure occurs. The most convenient way to do this is to use a mandos server but convenience comes at a cost. A safer and easier way is to use dropbear during boot (initrd). The weak point of this solution is that the server will be basically offline until the sysadmin manually unlocks it, to boot.

First we install dropbear for initrd:

# apt -y install dropbear-initramfs

Then we set a custom ssh port for dropbear. This better be different than the custom ssh port we used earlier. Change the dropbear port to 2244 in /etc/dropbear-initramfs/config:

DROPBEAR_OPTIONS="-p 2244"

Add the static IP in the initramtools configuration (/etc/initramfs-tools/initramfs.conf):

IP=192.168.1.10::192.168.1.1:255.255.255.0:storage:enp3s0:off

Copy the authorized_keys file in /etc/dropbear-initramfs:

# cp /root/.ssh/authorized_keys /etc/dropbear-initramfs/

Regenerate the initrd file:

# update-initramfs -u

Now reboot and ssh to it to test it:

$ ssh -p 2244 root@192.168.1.10

If your pubkeys are in place you will enter a busybox shell. Enter the crypt-unlock command, supply your unlock passphrase and the system will boot to the encrypted system.

Setup a local MTA for notifications

We will be using our main mailserver as a smarthost for mail to go through.

Install the postfix MTA and the mail utility:

# apt -y install postfix mailutils

Answer these questions:

General type of mail configuration: Internet with smarthost
System mail name: storage.example.com
SMTP relay host (blank for none): smtp.example.com

Test it:

# echo 'Testing #1' | mail -s 'Test #1' user@example.com

If you get a mail in your mailbox then everything is set. If not extra configuration may be needed on the smarthost. Contact the sysadmin of the smarthost, or check the logs if you access to it.

Setup pro-active disk monitoring

Install smartmontools:

# apt -y install smartmontools

Enable S.M.A.R.T, offline testing, attribute autosave, short and long test on all 4 devices. Add these lines in /etc/smartd.conf:

/dev/sda -a -d sat -o on -S on -s (S/../.././02|L/../../6/03) -m user@example.com -M exec /usr/share/smartmontools/smartd-runner
/dev/sdb -a -d sat -o on -S on -s (S/../.././02|L/../../6/03) -m user@example.com -M exec /usr/share/smartmontools/smartd-runner
/dev/sdc -a -d sat -o on -S on -s (S/../.././02|L/../../6/03) -m user@example.com -M exec /usr/share/smartmontools/smartd-runner
/dev/sdd -a -d sat -o on -S on -s (S/../.././02|L/../../6/03) -m user@example.com -M exec /usr/share/smartmontools/smartd-runner

Restart smartmontools:

# systemctl restart smartmontools.service

Setup Software RAID10 monitoring

We also need to setup monitoring for the software raid. Add your email address in the /etc/mdadm/mdadm.conf file:

MAILADDR user@example.com

Restart the mdmonitor service:

# systemctl restart mdmonitor.service

Setup kexec for faster reboots

Kexec is a Linux kernel mechanism that can load a fresh kernel from a running system. This results in a “reboot” without in fact rebooting the computer. The system loads a new kernel, the system appears “rebooted” but skipping the BIOS?UEFI initialization, thus resulting in faster reboots.

Install kexec-tools:

# apt -y install kexec-tools

The ‘Should kexec-tools handle reboots (sysvinit only)?’ question is related only to sysvinit systems. Since we are using systemd, it has no effect in our case.

Now if you want to reboot instead of running reboot you can run systemctl kexec. The latter command will reboot the system without going though BIOS/UEFI, POST etc and your system downtime is minimized.

And we are done! Store your server in a protected location, add a UPS for power backup and you are ready.

References

https://wiki.debian.org/Bonding
https://help.ubuntu.com/community/UbuntuBonding
https://www.theo-andreou.org/?p=1579
https://wiki.recompile.se/wiki/Mandos
http://forums.ayksolutions.com/forum/documentation/knowledgebase/general-server-questions/641-proactively-monitoring-hard-drive-health-using-smartd

Tags bonding, debian, disk, drobbear, encryption, initramfs, linux, luks, lvm, mdadm, raid, raid10, smartmontools

Linux

Installing Ceph using ceph-deploy

Post author By Theodotos Andreou
Post date June 10, 2017
No Comments on Installing Ceph using ceph-deploy

Ceph is a distributed open source storage solution that supports Object Storage, Block Storage and File Storage.

Other open source distributed storage systems are GlusterFS and HDFS.

In this guide, we describe how to setup a basic Ceph Cluster for Block Storage. We have 25 nodes on our setup. The masternode is a MASS Region and Rack controller. The rest of the nodes are Ubuntu 16.04 deployed through MAAS. The recommended filesystem for Ceph is XFS and this is what is used on the nodes.

This guide is based on the Quick Installation guide from the Ceph Documentation. This guide uses the ceph-deploy tool which is a relatively quick way to setup Ceph, especially for newbies. There is also the Manual Installation, deployment through Ansible and juju.

Prerequisites

Topology

1 deploy node (masternode). MAAS region and rack controler is installed plus Ansible
3 monitor nodes (node01,node11,node24). Ubuntu 16.04 on XFS deployed through MAAS
20 OSD nodes (node02-10,12-23).

Create an Ubuntu user on masternode

It would be of convenience to create an ubuntu user on the masternode. with passwordless sudo access:

$ sudo useradd -m -s /bin/bash ubuntu

Run visudo and give passwordless sudo access to the ubuntu user:

ubuntu  ALL=NOPASSWD:ALL

Generate an SSH key pair for the ubuntu user:

$ ssh-keygen -b 4096
Generating public/private rsa key pair.
Enter file in which to save the key (/home/ubuntu/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/ubuntu/.ssh/id_rsa.
Your public key has been saved in /home/ubuntu/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:t1zWURVk7j6wJPkA3VmbcHtAKh3EB0kyanORVbiiBkU ubuntu@masternode
The key's randomart image is:
+---[RSA 4096]----+
|       .E +**B=*=|
|        ..o==oOo+|
|       .+.o.o=.=.|
|      .. oo.o....|
|       .S..=oo.. |
|        oo += +  |
|       .  o  o o |
|                .|
|                 |
+----[SHA256]-----+

Deploy the /home/ubuntu/.ssh/id_rsa.pub pubkey on all the nodes (append in /home/ubuntu/.ssh/authorized_keys). You could add this pubkey on the MAAS user before deploying Ubuntu 16.04 on the nodes.

Set /etc/hosts

$ for ID in {01..24}; do echo "$(dig +short node${ID}.maas @127.0.0.1) node${ID}.maas node${ID}"; done > nodes.txt

It should look like this:

192.168.10.28 node01.maas node01
192.168.10.29 node02.maas node02
192.168.10.30 node03.maas node03
192.168.10.31 node04.maas node04
192.168.10.32 node05.maas node05
192.168.10.33 node06.maas node06
192.168.10.34 node07.maas node07
192.168.10.35 node08.maas node08
192.168.10.36 node09.maas node09
192.168.10.37 node10.maas node10
192.168.10.38 node11.maas node11
192.168.10.39 node12.maas node12
192.168.10.40 node13.maas node13
192.168.10.41 node14.maas node14
192.168.10.42 node16.maas node16
192.168.10.43 node17.maas node17
192.168.10.44 node18.maas node18
192.168.10.45 node19.maas node19
192.168.10.46 node20.maas node20
192.168.10.47 node21.maas node21
192.168.10.48 node22.maas node22
192.168.10.49 node23.maas node23
192.168.10.50 node24.maas node24

Now you can append the result in /etc/hosts:

$ cat nodes.txt | sudo tee -a /etc/hosts

Ansible setup

Use this setup in /etc/ansible/hosts on masternode:

[masternode]
masternode

[nodes]
node01
node02
node03
node04
node05
node06
node07
node08
node09
node10
node11
node12
node13
node14
node15
node16
node17
node18
node19
node20
node21
node22
node23
node24

[ceph-mon]
node01
node11
node24

[ceph-osd]
node02
node03
node04
node05
node06
node07
node08
node09
node10
node12
node13
node14
node15
node16
node17
node18
node19
node20
node21
node22
node23

Install python on all the nodes

$ for ID in {01..24}
> do
>  ssh node${ID} "sudo apt -y install python-minimal"
> done

Ensure time synchronization of the nodes

Install the theodotos/debian-ntp role from Ansible Galaxy:

$ sudo ansible-galaxy install theodotos.debian-ntp

Create a basic playbook ntp-init.yml:

---
- hosts: nodes
  remote_user: ubuntu
  become: yes
  roles:
     - { role: theodotos.debian-ntp, ntp.server: masternode }

Apply the playbook:

$ ansible-playbook ntp-init.yml

Verify that the monitor nodes are time synchronized:

$ ansible ceph-mon -a 'timedatectl'
node11 | SUCCESS | rc=0 >>
      Local time: Fri 2017-04-28 08:06:30 UTC
  Universal time: Fri 2017-04-28 08:06:30 UTC
        RTC time: Fri 2017-04-28 08:06:30
       Time zone: Etc/UTC (UTC, +0000)
 Network time on: yes
NTP synchronized: yes
 RTC in local TZ: no

node24 | SUCCESS | rc=0 >>
      Local time: Fri 2017-04-28 08:06:30 UTC
  Universal time: Fri 2017-04-28 08:06:30 UTC
        RTC time: Fri 2017-04-28 08:06:30
       Time zone: Etc/UTC (UTC, +0000)
 Network time on: yes
NTP synchronized: yes
 RTC in local TZ: no

node01 | SUCCESS | rc=0 >>
      Local time: Fri 2017-04-28 08:06:30 UTC
  Universal time: Fri 2017-04-28 08:06:30 UTC
        RTC time: Fri 2017-04-28 08:06:30
       Time zone: Etc/UTC (UTC, +0000)
 Network time on: yes
NTP synchronized: yes
 RTC in local TZ: no

Check also the OSD nodes:

$ ansible ceph-osd -a 'timedatectl'

Install Ceph

Install ceph-deploy

On masternode:

$ sudo apt install ceph-deploy

Create a new cluster and set the monitor nodes (must be odd numbered):

$ ceph-deploy new node01 node11 node24

Install ceph on master node and all other nodes:

$ ceph-deploy install masternode node{01..24}

Deploy the monitors and gather the keys:

$ ceph-deploy mon create-initial

Prepare the OSD nodes

Create the OSD directories

Create the OSD directories on the OSD nodes:

$ I=0;
$ for ID in {02..10} {12..14} {16..23}
> do 
>  ssh -l ubuntu node${ID} "sudo mkdir /var/local/osd${I}"
>  I=$((${I}+1))
> done;

Verify that the OSD directories are created:

$ ansible ceph-osd -a "ls /var/local" | cut -d\| -f1 | xargs -n2 | sort
node02 osd0
node03 osd1
node04 osd2
node05 osd3
node06 osd4
node07 osd5
node08 osd6
node09 osd7
node10 osd8
node12 osd9
node13 osd10
node14 osd11
node16 osd12
node17 osd13
node18 osd14
node19 osd15
node20 osd16
node21 osd17
node22 osd18
node23 osd19

Nodes 01, 11 and 24 are excluded because those are the monitor nodes.

Fix OSD permissions

Because of some bug we need to change the OSD directories owneship to ceph:ceph. Otherwise you will get this:

** ERROR: error creating empty object store in /var/local/osd0: (13) Permission denied

Change the ownership of the OSD directories on the OSD nodes:

$ I=0;
$ for ID in {02..10} {12..14} {16..23}
> do 
>   ssh -l ubuntu node${ID} "sudo chown ceph:ceph /var/local/osd${I}"
>   I=$((${I}+1))
> done;

Prepare the OSDs

$ I=0
$ for ID in {02..10} {12..14} {16..23}
> do
>   ceph-deploy --username ubuntu osd prepare node${ID}:/var/local/osd${I}
>   I=$((${I}+1))
> done

Activate the OSDs

For nodes 02 – 10:

$ I=0
> for ID in {02..10} {12..14} {16..23}
> do
>   ceph-deploy --username ubuntu osd activate node${ID}:/var/local/osd${I}
>   I=$((${I}+1))
> done

Deploy the configuration file and admin key

Now we need to deploy the configuration file and admin key to the admin node and our Ceph nodes. This will save us from having to specify the monitor address and keyring every time we execute a Ceph cli command.

$ ceph-deploy admin masternode node{01..24}

Set the keyring to be world readable:

$ sudo chmod +r /etc/ceph/ceph.client.admin.keyring

Test and verify

$ ceph health
HEALTH_WARN too few PGs per OSD (9 < min 30)
HEALTH_ERR clock skew detected on mon.node11, mon.node24; 64 pgs are stuck inactive for more than 300 seconds; 64 pgs stuck inactive; 64 pgs stuck unclean; Monitor clock skew detected

Out newly build cluster is not healthy. We need to increase Placement Groups. The formula is the number_of_minimum_expected_PGs (30) times the number_of_OSDs (20) and rounded to the closest power of 2:

30x20=500 => pg_num=512

Increase PGs:

$ ceph osd pool set rbd pg_num 512

Now we run ceph health again:

$ ceph health
HEALTH_WARN pool rbd pg_num 512 > pgp_num 64

Still some tweaking needs to be done. We need to adjust pgp_num to 512:

$ ceph osd pool set rbd pgp_num 512

And we are there at last:

$ ceph health
HEALTH_OK

Create a Ceph Block Device device

Check the available storage:

$ ceph df
MapGLOBAL:
    SIZE       AVAIL      RAW USED     %RAW USED 
    11151G     10858G         293G          2.63 
POOLS:
    NAME     ID     USED     %USED     MAX AVAIL     OBJECTS 
    rbd      0       306         0         3619G           4

Now we need to create a RADOS Block Device (RBD) to hold our data.

$ rbd create clusterdata --size 4T --image-feature layering

Check the new block device:

$ rbd ls -l
NAME         SIZE PARENT FMT PROT LOCK 
clusterdata 4096G          2

Map the block device:

$ sudo rbd map clusterdata --name client.admin
/dev/rbd0

Format the clusterdata device:

$ sudo mkfs -t ext4 /dev/rbd0

Mount the blobk device:

$ mkdir /srv/clusterdata
$ mount /dev/rbd0 /srv/clusterdata

Now we have a block device for data that is distributed among the 21 storage nodes.

Here’s is a summary of some useful monitoring and troubleshooting commands for ceph

$ ceph health
$ ceph health detail
$ ceph status (ceph -s)
$ ceph osd stat
$ ceph osd tree
$ ceph mon dump
$ ceph mon stat
$ ceph -w
$ ceph quorum_status --format json-pretty
$ ceph mon_status --format json-pretty
$ ceph df

If you run into trouble contact the awesome folks at the #ceph IRC channel, hosted on Open and Free Technology Community IRC network.

Start over

In case you messed up the procedure and you need to start over you can use the following commands:

$ ceph-deploy purge masternode node{01..24}
$ ceph-deploy purgedata masternode node{01..24}
$ ceph-deploy forgetkeys
$ for ID in {02..11} {11..23}; do ssh node${ID} "sudo rm -fr /var/local/osd*"; done
$ rm ceph.conf ceph-deploy-ceph.log .cephdeploy.conf

NOTE: this procedure will destroy your Ceph cluster along with all the data!

Conclusions

Using ceph-deploy maybe an easy way to get started with Ceph, but it does not provide much customization. For a more fine tuned setup you maybe better with the Manual Installation, even though there is a steeper learning curve.

References

http://docs.ceph.com/docs/master/start/
https://bugs.launchpad.net/ubuntu/+source/tzdata/+bug/1554806
http://docs.ceph.com/docs/jewel/rados/configuration/filesystem-recommendations/
http://www.virtualtothecore.com/en/adventures-ceph-storage-part-1-introduction/

Tags ansible, ceph, ceph-deploy, distributed, linux, maas, storage, ubuntu

Linux

Installing Wiki.js on Ubuntu 16.04

Post author By Theodotos Andreou
Post date June 10, 2017
8 Comments on Installing Wiki.js on Ubuntu 16.04

Wiki.js is an elegant looking wiki based on Markdown. It supports LDAP and many more authentication mechanisms. In this guide we describe how to install Wiki.js on Ubuntu 16.04.

Prerequisites

An Ubuntu 16.04 instance.

Install curl, Node.js v8.x and build-essential:

# apt -y install curl
# curl -sL https://deb.nodesource.com/setup_8.x | bash -
# apt -y install nodejs build-essential

Install MongoDB v3.4

# apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 0C49F3730359A14518585931BC711F9BA15703C6
# echo "deb [ arch=amd64,arm64 ] http://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.4 multiverse" | tee /etc/apt/sources.list.d/mongodb-org-3.4.list
# apt update
# apt -y install mongodb-org

Start MongoDB:

# systemctl start mongodb

Enable MongoDB at startup:

# systemctl enable mongodb

Install git

The version that comes with Ubuntu 16.04 fills the minimum requirements so there is no need to install it from upstream.

# apt -y install git

Install Wiki.js

# mkdir /srv/wiki.js
# cd /srv/wiki.js
# npm install wiki.js@latest

You will get this message:

> Browse to http://your-server:3000/ to configure your wiki! (Replaced your-server with the hostname or IP of your server!)
▐   ⠂    ▌ I'll wait until you're done ;)

Do as the message says. Let the wizard wait until we are done, and open another shell to work with.

Setup nginx

Install Nginx:

# apt -y install nginx

Create this VirtualHost configuration (/etc/nginx/sites-available/wiki.example.com.conf):

server {
    listen      [::]:80 ipv6only=off;
    server_name wiki.example.com;
    return      301 https://$server_name$request_uri;
}
server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name  wiki.example.com;

    ssl_session_timeout 1d;
    ssl_session_cache shared:SSL:50m;
    ssl_session_tickets off;

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers "EECDH+ECDSA+AESGCM EECDH+aRSA+AESGCM EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256 EECDH EDH+aRSA !RC4 !aNULL !eNULL !LOW !3DES !MD5 !EXP !PSK !SRP !DSS";
    ssl_prefer_server_ciphers on;

    ssl_certificate /etc/nginx/ssl/wiki.example.com.crt;
    ssl_certificate_key /etc/nginx/ssl/wiki.example.com.key;
    ssl_trusted_certificate /etc/nginx/ssl/CA.crt;

    location / {
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_pass http://127.0.0.1:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_next_upstream error timeout http_502 http_503 http_504;
    }
}

Enable the wiki.example.com VirtualHost:

# cd /etc/nginx/sites-enabled/
# ln -s ../sites-available/wiki.example.com.conf
# unlink default

Restart Nginx:

# systemctl restart nginx

Configure Wiki.js

After the installation you will be asked if you wish to run the configuration wizard. Select this and continue:

Yes, run configuration wizard on port 3000 (recommended)

Now browse to http://wiki.example.com/ and follow the installation wizard:

Welcome!: Start
System Check (if all good): Continue
General:
- Site title: ExampleWiki
- Host: https://wiki.example.com
- Port: 3000
- Site UI Language: English
- Public Access: Not selected
- Press: Continue
Important Considerations: Continue
Database: mongodb://localhost:27017/wiki
Database Check: Continue:
Paths:
- Local Data Path: ./data
- Local Repository Path: ./repo
Git Repository: Skip this step
Git Repository Check: Continue
Administrator Account
- Administrator Email: admin@example.com
- Password: MySecretCombination
- ConfirmPassword: MySecretCombination
Finalizing: Start

Enable Wiki.js on startup

# npm install -g pm2
# pm2 startup
# pm2 save

Setup LDAP

This is an optional step for those wishing to integrate Wiki.js in their LDAP infrastructure.

Trust CUT IST ISSUING CA

Connect to the LDAP (AD) server and get all certificates:

openssl s_client -showcerts -connect dcs03ist00.lim.tepak.int:636 | tee ldap.log

Hit ‘Ctrl-C’ to end the command.

The certificate with the ID ‘1’ in ldap.log is the ISSUING CA certificate. Extract the CUT IST ISSUING CA certificate and save it in cut_issuing_ca.crt:

Verify the certificate with:

openssl x509 -text -in cut_issuing_ca.crt

Add the CUT ISSUING CA in the trusted chain of the system:

cp cut_issuing_ca.crt /usr/local/share/ca-certificates/
update-ca-certificates

Configure LDAP for Wiki.js

Make these changes in /srv/wiki.js/config.yml:

  ldap:
    enabled: true
    url: 'ldap://ldap.example.com:389'
    bindDn: 'cn=wiki,ou=dsa,dc=example,dc=com'
    bindCredentials: 'MyLDAPCredentials'
    searchBase: 'ou=people,dc=example,dc=com'
    searchFilter: '(uid={{username}})'
    tlsEnabled: true
    tlsCertPath: '/etc/ssl/certs/ca-certificates.crt'

Give Access permissions to authenticated users

Visit the Admin URL:

https://wiki.example.com/admin

Click on ‘Users’. You will get a list of users. You can give ‘Read and Write’ access to them from the ‘Access Rights’ field and you can upgrade them to ‘Global Administrators’ from the ‘Role Override’ field.

NOTE: For LDAP the users need to login first before they are allowed to write.

Enjoy your newly created Wiki!

References

https://docs.requarks.io/wiki
https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions
https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/

Tags linux, mongodb, nginx, nodejs, ubuntu, wiki

Linux

Setting up a Dual NIC, LXD Container

Post author By Theodotos Andreou
Post date May 20, 2017
No Comments on Setting up a Dual NIC, LXD Container

LXD is a relatively new technology by Canonical. It is a container hypervisor that builds on top of LXC. It shares some similarities with Docker, but LXD focuses on full blown containerized systems, instead of containerized applications.

LXD/LXC creates lighter systems than VM hypervisors, and it is an easy way to experiment with different software without messing up with your system. You can also use it in production, to run Linux systems with much greater density than on VM hypervisors. The only drawback is that the container must be the same OS as the host (Linux in this case) and it depends on the host for kernel functionality (e.g. modules etc).

In this guide we will build an LXD container with two virtual NICs which are attached to the physical NIC of it’s host using bridged networking.

Prerequisites

An Ubuntu 16.04 host with two physical NICs

Prepare host networking

Install dependencies

# apt -y install bridge-utils

Setup networking on the host

There are two interfaces on the host enp3s0 and enp5s0. The first one, enp3s0 is the primary and connects to the main, DHCP enabled, internal network. The second one is connected to an isolated internal network where some critical services are running. There is no DHCP in the isolated network.

Prepare your /etc/network/interfaces configuration file like this:

# The loopback network interface 
auto lo
iface lo inet loopback

# The main Bridge
auto br0
iface br0 inet dhcp
    bridge-ifaces enp3s0
    bridge-ports enp3s0
    up ip link set enp3s0 up

# The primary network interface
iface enp3s0 inet manual

# The secondary Bridge
auto br1
iface br1 inet manual
        bridge-ifaces enp5s0
        bridge-ports enp5s0
        up ip link set enp5s0 up

# The secondary network interface
iface enp5s0 inet manual

Setup LXC/ LXD

Installation of LXD

# apt -y install lxd

Prepare a new profile

We need a profile with two NICs. First create a new profile:

# lxc profile copy default dualnic

Then edit the new profile with lxc profile edit dualnic:

config:
  user.network_mode: link-local
description: Dual nic profile
devices:
  eth0:
    name: eth0
    nictype: bridged
    parent: br0
    type: nic
  eth1:
    name: eth1
    nictype: bridged
    parent: br1
    type: nic
name: dualnic

The names of the virtual interfaces of the container will be eth0 for the primary and eth1 for the secondary.

Launch the container

Now we want to launch a Debian jessie container, which we will retrieve from the prepared images on the

# lxc launch images:debian/jessie mycontainer -p dualnic

List the container:

# lxc list
+-------------+---------+----------------------+------+------------+-----------+
|    NAME     |  STATE  |         IPV4         | IPV6 |    TYPE    | SNAPSHOTS |
+-------------+---------+----------------------+------+------------+-----------+
| mycontainer | RUNNING | 192.168.0.122 (eth0) |      | PERSISTENT | 0         |
+-------------+---------+----------------------+------+------------+-----------+

The IP listed above was set by the DHCP server in the primary network.

Connect to the container:

# lxc exec mycontainer bash

Setup container’s networking

Edit /etc/network/interfaces inside our container:

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp

auto eth1
iface eth1 inet static
        address 172.16.16.100
        netmask 255.255.255.0
        network 172.16.16.0
        broadcast 172.16.16.255
        up ip route add 10.10.10.0/24  via 172.16.16.1 dev eth1
        up ip route add 10.10.20.0/24  via 172.16.16.1 dev eth1

The routes 10.10.10.0/24 and 10.10.20.0/24 are internal routes that need to be reached though the secondary network.

Take a clean snapshot

Before you start working on your container, it is a good idea to get a clean snapshot, so you can revert to it, in case you mess something up.

# lxc snapshot mycontainer

If you want to learn more about the and features I urge you to read the excellent introduction to LXD 2.0 by Stéphane Graber

References

https://linuxcontainers.org/
https://www.ubuntu.com/containers/lxd
https://insights.ubuntu.com/2016/03/14/the-lxd-2-0-story-prologue/
https://stgraber.org/2016/03/11/lxd-2-0-blog-post-series-012/
https://insights.ubuntu.com/2015/11/10/converting-eth0-to-br0-and-getting-all-your-lxc-or-lxd-onto-your-lan/

Tags containers, debian, linux, lxc, lxd, ubuntu

Linux

Installing Rocket.Chat on Debian

Post author By Theodotos Andreou
Post date January 6, 2017
4 Comments on Installing Rocket.Chat on Debian

Rocket.Chat is a free/open source software for team collaboration. It is a full featured platform and an ideal alternative to Slack, for organizations that are dedicated to the Free Software philosophy.

In this guide we will demonstrate how you can setup Rocket.Chat for your organization.

Prerequisites

A Debian stretch VM or server.
A FQDN pointing to the system’s IP. We will be using rocket.example.com throughout this guide.

Installation of rocket.chat

Install the Snappy package manager

Snappy is not installed on Debian by default, so we need to install it:

# apt install snapd ca-certificates

Install Rocket.Chat

# snap install rocketchat-server

Check its status:

# systemctl status snap.rocketchat-server.rocketchat-server.service
● snap.rocketchat-server.rocketchat-server.service - Service for snap application rocketchat-server.rocketchat-server
   Loaded: loaded (/etc/systemd/system/snap.rocketchat-server.rocketchat-server.service; enabled; vendor preset: enabled)
   Active: active (running) since Tue 2017-01-03 12:27:50 PST; 12min ago
 Main PID: 24891 (node)
    Tasks: 10 (limit: 4915)
   CGroup: /system.slice/snap.rocketchat-server.rocketchat-server.service
           └─24891 node /snap/rocketchat-server/326/main.js

Jan 03 12:27:50 rocket systemd[1]: Started Service for snap application rocketchat-server.rocketchat-server.

Check its port (default is 3000):

# ss -lnptu | grep 3000
tcp    LISTEN     0      128       *:3000                  *:*                   users:(("node",pid=24891,fd=13))

Looks OK

Configure TLS

Install nginx

The core application does not support TLS so we will be setting up an nginx reverse proxy on top of it.

Install NGINX and certbot from Let’s Encrypt:

# apt -y install nginx python-certbot-nginx

Create the VirtualHost

We need to create this file: /etc/nginx/sites-available/rocket.example.com:

# Upstreams
upstream backend {
    server 127.0.0.1:3000;
}

# HTTP Server
server {
    listen 80;
    server_name rocket.example.com;

    error_log /var/log/nginx/rocketchat.access.log;

    location / {
        proxy_pass http://127.0.0.1:3000/;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $http_host;

        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forward-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forward-Proto http;
        proxy_set_header X-Nginx-Proxy true;

        proxy_redirect off;
    }
}

Enable the VirtualHost:

cd /etc/nginx/sites-enabled/
ln -s ../sites-available/rocket.example.com
systemctl reload nginx

Generate the certificate

certbot --nginx run -d rocket.example.com

Enter email address: admin@example.com
Agree to the ToS
Enforce HTTPS: Secure

The VirtualHost file (/etc/nginx/sites-available/rocket.example.com) should look like this after the creation of the Let’s Encrypt certificate:

# Upstreams
upstream backend {
    server 127.0.0.1:3000;
}

server {
    listen 80;
    server_name rocket.example.com;

    error_log /var/log/nginx/rocketchat.access.log;

    location / {
        proxy_pass http://127.0.0.1:3000/;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $http_host;

        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forward-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forward-Proto http;
        proxy_set_header X-Nginx-Proxy true;

        proxy_redirect off;
    }

    listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/rocket.example.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/rocket.example.com/privkey.pem; # managed by Certbot
ssl_session_cache shared:le_nginx_SSL:1m; # managed by Certbot
ssl_session_timeout 1440m; # managed by Certbot

ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # managed by Certbot
ssl_prefer_server_ciphers on; # managed by Certbot

ssl_ciphers "ECDHE-ECDSA-AES128-GCM-SHA256 ECDHE-ECDSA-AES256-GCM-SHA384 ECDHE-ECDSA-AES128-SHA ECDHE-ECDSA-AES256-SHA ECDHE-ECDSA-AES128-SHA256 ECDHE-ECDSA-AES256-SHA384 ECDHE-RSA-AES128-GCM-SHA256 ECDHE-RSA-AES256-GCM-SHA384 ECDHE-RSA-AES128-SHA ECDHE-RSA-AES128-SHA256 ECDHE-RSA-AES256-SHA384 DHE-RSA-AES128-GCM-SHA256 DHE-RSA-AES256-GCM-SHA384 DHE-RSA-AES128-SHA DHE-RSA-AES256-SHA DHE-RSA-AES128-SHA256 DHE-RSA-AES256-SHA256 EDH-RSA-DES-CBC3-SHA"; # managed by Certbot

    if ($scheme != "https") {
        return 301 https://$host$request_uri;
    } # managed by Certbot
}

Now visit the website to test it: http://rocket.example.com (it should redirect you to https://rocket.example.com)

Creating an admin account

WARNING: the setting site URL is configured to http://localhost and you are accessing from https://rocket.example.com. do you want to change to https://rocket.example.com?

You should ofcourse answer ‘Yes’.

This first user created is an admin user. You can set global preferences from https://rocket.example.com/admin. Now users can visit the website and register with it. You may wish to Allow Notifications when you are prompted by your browser on your first visit to the site. Rocket.Chat supports many other authentication backends, including LDAP which is described in the next step.

Besides the web service you can also download native applications for Linux, Windows, Mac OS X, Android and iOS.

Configure LDAP

This is an optional step, but recommended if your organization has an LDAP or Active Directory setup. In this example we are using the Fusiondirectory setup from our previous guide.

Create a service account for rocket.chat, using the DSA module of Fusiondirectory:

Username: cn=rocketchat,ou=dsa,dc=example,dc=com
Password: MySecretCombination

Now go to https://rocket.example.com/admin using an admin account and set these:

Enable: True
Login Fallback: True
Host: ldap.example.com
Port: 389
Encryption: StartTLS
CA Cert: Paste the contents of your internal ROOT CA certificate (example.com-rootCA.crt for example)
Reject Unauthorized: True
Domain Base: ou=people,dc=example,dc=com
Use Custom Domain Search: False
Domain Search User: cn=rocketchat,ou=dsa,example,dc=com
Domain Search Password: MySecretCombination
Domain Search User ID: uid
Domain Search Object Class: person
Domain Search Object Category: Leave Empty
Username Field: Leave Empty

Leave the default settings for the rest and press the SAVE CHANGES button

You can use the TEST CONNECTION button to test the connection.

You can now try to login using your LDAP username and password.

References

https://rocket.chat/
http://snapcraft.io/docs/core/install#debian
https://rocket.chat/docs/installation/manual-installation/ubuntu/snaps
https://rocket.chat/docs/installation/manual-installation/configuring-ssl-reverse-proxy
https://rocket.chat/docs/administrator-guides/authentication/ldap/

Tags collaboration, debian, linux, rocket.chat

Linux

Creating an authoritative-only DNS system for your domain

Post author By Theodotos Andreou
Post date December 12, 2016
No Comments on Creating an authoritative-only DNS system for your domain

Most registrars they provide a DNS service when you purchase a domain name. But having your own DNS servers means more control to you. And if you are an aspiring sysadmin, you will find it fun and educational. We will be using BIND 9 which is, by far the most popular DNS implementation.

Prerequisites

We will need two systems (VMs or containers) preferably on different geographic locations and different providers. One will be the master and the other system, the slave. This guide works for recent Debian or Ubuntu systems.

Some assumptions:

Master DNS:
- Hostname: ns1.example.com
- IP: 1.1.1.1
Slave DNS:
- Hostname: ns2.example.com
- IP: 2.2.2.2
Webserver:
- Hostname: example.com (www.example.com is an alias)
- IP: 3.3.3.3
Main mailserver:
- Hostname: mx1.example.com ({mail,smtp,pop,imap,webmail}.example.com are aliases)
- IP: 4.4.4.4
Backup mailserver:
- Hostname: mx2.example.com
- IP: 5.5.5.5

Preparations

Setup the /etc/hosts file

On master’s /etc/hosts:

1.1.1.1    ns1.example.com ns1

On slave’s /etc/hosts:

2.2.2.2    ns2.example.com ns2

Installing BIND

NOTE: all commands must be applied to both master and slave unless otherwise stated

apt-get -y install bind9 bind9utils

Allow port 53:

ufw allow 53

Configure the global options

Make the following changes in the /etc/bind/named.conf.options of both servers:

--- /etc/bind/named.conf.options        2016-12-12 14:44:57.163515708 -0500
+++ /etc/bind/named.conf.options.new    2016-12-12 14:52:29.749268250 -0500
@@ -1,5 +1,7 @@
 options {
        directory "/var/cache/bind";
+       recursion no;
+       allow-transfer { none; };

        // If there is a firewall between you and nameservers you want
        // to talk to, you may need to fix the firewall to allow multiple

Configure the local options

On master’s /etc/bind/named.conf.local:

--- /etc/bind/named.conf.local  2016-12-12 20:06:27.000000000 +0000
+++ /etc/bind/named.conf.local.new      2016-12-12 20:07:03.000000000 +0000
@@ -6,3 +6,8 @@
 // organization
 //include "/etc/bind/zones.rfc1918";

+zone "example.com" {
+    type master;
+    file "/etc/bind/db.example.com";
+    allow-transfer { 2.2.2.2; };
+};

On slave’s /etc/bind/named.conf.local:

--- /etc/bind/named.conf.local  2016-11-01 13:02:24.000000000 -0400
+++ /etc/bind/named.conf.local.new      2016-12-12 15:09:47.445235343 -0500
@@ -6,3 +6,8 @@
 // organization
 //include "/etc/bind/zones.rfc1918";

+zone "example.com" {
+    type slave;
+    file "db.example.com";
+    masters { 1.1.1.1; };
+};

Create the zone file

On master server create the zone file (/etc/bind/db.example.com):

$TTL 3600
@    IN    SOA    ns1.example.com.    root.example.com. (
                                            2016121201
                                            7200
                                            3600
                                            1209600
                                            180 )

; NS Records
@       300    IN      NS              ns1.example.com.
@       300    IN      NS              ns2.example.com.

; MX Records
@       300    IN      MX      10      mx1.example.com.
@       300    IN      MX      20      mx2.example.com.

; Address records
@       300    IN      A               3.3.3.3
ns1     300    IN      A               1.1.1.1
ns2     300    IN      A               2.2.2.2
mx1     300    IN      A               4.4.4.4
mx1     300    IN      A               5.5.5.5

; Alias (Canonical Name) records
www     300    IN      CNAME           example.com.
mail    300    IN      CNAME           mx1.example.com.
smtp    300    IN      CNAME           mx1.example.com.
imap    300    IN      CNAME           mx1.example.com.
pop     300    IN      CNAME           mx1.example.com.
webmail 300    IN      CNAME           mx1.example.com.

; TXT records
@       300    IN      TXT             "v=spf1 a mx ?all"

Verify the global configuration

Run the named-checkconf command on both servers. If you get any errors you should fix them before proceeding.

Verify the zone configuration

Run the following command on the master server:

named-checkzone theo-andreou.org /etc/bind/db.example.com 
zone example.com/IN: loaded serial 2016121201
OK

If you get ‘OK’ then your setup is correct, otherwise you should fix it before proceeding.

Restart bind and test your servers

systemctl restart bind9.service

Now test if your servers resolve your records.

The master:

dig smtp.example.com @1.1.1.1
...
;; ANSWER SECTION:
smtp.example.com.   300 IN  CNAME   mx1.example.com.
mx1.example.com.    300 IN  A   4.4.4.4
...

The slave:

dig smtp.example.com @2.2.2.2
...
;; ANSWER SECTION:
smtp.example.com.   300 IN  CNAME   mx1.example.com.
mx1.example.com.    300 IN  A   4.4.4.4
...

If you get any errors you can check your syslog file (/var/log/syslog).

Finalizing

After you have tested that everything is OK, it is time to tell your registrar about the new servers. This is called setting the glue records and you should consult the documentation of your registrar on how to do that. Ask support if you have any trouble.

References

https://www.digitalocean.com/community/tutorials/how-to-configure-bind-as-an-authoritative-only-dns-server-on-ubuntu-14-04

Tags bind, bind9, debian, dns, linux, nameserver, ubuntu

Linux

How to remotely decrypt a LUKS encrypted Debian/Ubuntu System

Post author By Theodotos Andreou
Post date October 12, 2016
12 Comments on How to remotely decrypt a LUKS encrypted Debian/Ubuntu System

In this guide we will show how you can remotely decrypt a headless Debian or Ubuntu Linux system, that has been encrypted with LUKS.

Prerequisites

A LUKS encrypted Debian jessie or Ubuntu xenial system
Keyboard and monitor for the initial system setup
Allow SSH root access on the decrypted system using public key authentication
Use a different port for ssh (assuming port 4422) on the decrypted system
NOTE: using a different port than the standard SSH port (22) serves a double purpose. For once, you will not received the scary WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! message every time you will try to remotely unlock the system and as an additional bonus you will get less SSH attacks on the active system.

Installing dropbear

Dropbear is a lightweight SSH server especially suitable for initial ramdisk (initrd) environments and other lightweight systems.

Install dropbear:

# apt -y install dropbear

Setup public key authentication for dropbear

Create the homedir for the root user and the SSH configuration directory:

# mkdir -p /etc/initramfs-tools/root/.ssh

Append your client SSH pubkey to authorized_keys:

# cat ~/.ssh/id_rsa.pub | ssh -p 4422 root@encrypted-system "cat >> /etc/initramfs-tools/root/.ssh/authorized_keys"

Setup a static IP for the unlock environment

This step is optional but highly recommended if you are setting up a static, permanent service. If you skip this step DHCP will kick in, provided you have a DHCP Server in your environment.

Run this command to update the /etc/initramfs-tools/initramfs.conf configuration file:

echo IP=10.0.0.67::10.0.0.1:255.255.255.0:encrypted-system:eth0:off >> /etc/initramfs-tools/initramfs.conf

Explanation of the different fields:
[host_ip]::[gateway_ip]:[netmask]:[hostname]:[device]:[autoconf]

NOTE: there are two successive colons (::) after the host_ip.

Setup the unlock script

Copy the following text in /etc/initramfs-tools/hooks/crypt_unlock.sh:

#!/bin/sh
#
# By Stinky Parkia
# https://stinkyparkia.wordpress.com/2014/10/14/remote-unlocking-luks-encrypted-lvm-using-dropbear-ssh-in-ubuntu-server-14-04-1-with-static-ipst/

PREREQ="dropbear"

prereqs() {
    echo "$PREREQ"
}

case "$1" in
    prereqs)
    prereqs
    exit 0
    ;;
esac

. "${CONFDIR}/initramfs.conf"
. /usr/share/initramfs-tools/hook-functions

if [ "${DROPBEAR}" != "n" ] && [ -r "/etc/crypttab" ] ; then
    cat > "${DESTDIR}/bin/unlock" < < EOF
#!/bin/sh
if PATH=/lib/unlock:/bin:/sbin /scripts/local-top/cryptroot; then
    kill \`ps | grep cryptroot | grep -v "grep" | awk '{print \$1}'\`
    # following line kill the remote shell right after the passphrase has
    # been entered.
    kill -9 \`ps | grep "\-sh" | grep -v "grep" | awk '{print \$1}'\`
    exit 0
fi
exit 1
EOF

    chmod 755 "${DESTDIR}/bin/unlock"

    mkdir -p "${DESTDIR}/lib/unlock"
    cat > "${DESTDIR}/lib/unlock/plymouth" < < EOF
#!/bin/sh
[ "\$1" == "--ping" ] && exit 1
/bin/plymouth "\$@"
EOF

    chmod 755 "${DESTDIR}/lib/unlock/plymouth"

    echo To unlock root-partition run "unlock" >> ${DESTDIR}/etc/motd
fi

Make the script executable:

# chmod +x /etc/initramfs-tools/hooks/crypt_unlock.sh

Apply the configuration

Apply the changes in the initial ramdisk:

# update-initramfs -u

Reboot the system:

# reboot

Remotely unlock the system

From your client, SSH into the initial ramdisk:

ssh root@encrypted-system

If everything is correct you will be greeted by this MOTD:

To unlock root-partition run unlock


BusyBox v1.22.1 (Ubuntu 1:1.22.0-15ubuntu1) built-in shell (ash)
Enter 'help' for a list of built-in commands.

Unlock the system and boot into it:

# unlock
Please unlock disk sda3_crypt:

You will get the following message and you will exit the remote shell if successful:

cryptsetup: sda3_crypt set up successfully
Connection to 10.0.0.67 closed.

You can now login to the active Linux system using the alternative port 4422:

ssh -p 4422 root@encrypted-system

If you can login successfully to your system you can remove the keyboard and monitor and hide your system somewhere where the Sun does not shine :).

Thanks to Stinky Parkia for the excellent guide and the brilliant unlock script.

References

https://stinkyparkia.wordpress.com/2014/10/14/remote-unlocking-luks-encrypted-lvm-using-dropbear-ssh-in-ubuntu-server-14-04-1-with-static-ipst/

Tags crypsetup, debain, dm-crypt, dropbear, encryption, howto, linux, luks, ubuntu

Linux

Setting up a Postfix/Dovecot mail system with an LDAP/Fusiondirectory backend

Post author By Theodotos Andreou
Post date September 22, 2016
8 Comments on Setting up a Postfix/Dovecot mail system with an LDAP/Fusiondirectory backend

This guide is about setting up a Postfix/Dovecot system using LDAP/Fusiondirectoory as a backend.

Prerequisites

Install an LDAP/Fusiondirectory infrastructure
A DNS A Record:
mail.example.com. 300 IN A 4.5.6.7
A DNS PTR Record:
7.6.5.4.in-addr.arpa. 300 IN PTR mail.example.com.
A DNS MX Record (Actually more than one):
example.com. 300 IN MX 5 mail.example.com.

Install FusionDirectory Plugins

Install the Mail plugin:

apt -y install fusiondirectory-plugin-mail fusiondirectory-plugin-mail-schema
fusiondirectory-insert-schema -i /etc/ldap/schema/fusiondirectory/mail-fd.schema
fusiondirectory-insert-schema -i /etc/ldap/schema/fusiondirectory/mail-fd-conf.schema

Install the Alias plugin:

apt -y install fusiondirectory-plugin-alias fusiondirectory-plugin-alias-schema
fusiondirectory-insert-schema -i /etc/ldap/schema/fusiondirectory/alias-fd-conf.schema
fusiondirectory-insert-schema -i /etc/ldap/schema/fusiondirectory/alias-fd.schema

Install the Postfix plugin:

apt -y install fusiondirectory-plugin-postfix fusiondirectory-plugin-postfix-schema
fusiondirectory-insert-schema -i /etc/ldap/schema/fusiondirectory/postfix-fd.schema

Setup mail services:

Enter the mail.example.com server configuration (Systems plugin)
From the ‘Services’ tab create a ‘Postfix (SMTP)’ and an ‘IMAP/POP3 generic’ service
Add mail capability to all normal users:
- Click the ‘Mail’ tab of a user and then press ‘Add Mail settings’
Fill in the ‘Primary Address. field and select mail.example.com as the server.

Add All necessary distribution lists and redirections from ‘Aliases’

Setup postfix

Create a service account for postfix using the ‘DSA’ plugin in *FusionDirectory’. It should look like this:

dn: cn=postfix,ou=dsa,dc=example,dc=com
cn: postfix
userPassword:: yeduyt2732tet87eoiewoi32t4873t4387f7gf47gf49i6=
structuralObjectClass: organizationalRole
entryUUID: 280427ce-9a54-1035-8e48-bf1fd814366b
creatorsName: cn=admin,dc=example,dc=com
createTimestamp: 20160419082627Z
objectClass: organizationalRole
objectClass: top
objectClass: simpleSecurityObject
entryCSN: 20160419082628.006263Z#000000#000#000000
modifiersName: cn=admin,dc=example,dc=com
modifyTimestamp: 20160419082628Z

Install postfix:
```
apt -y install postfix postfix-pcre postfix-ldap
```
When you are prompted about "General type of mail configuration:" select "No configuration"
Prepare the /etc/postfix/main.cf file:

smtpd_banner = $myhostname ESMTP $mail_name
biff = no
append_dot_mydomain = no
readme_directory = no
smtpd_tls_cert_file=/etc/ssl/certs/mail.example.com.crt
smtpd_tls_key_file=/etc/ssl/private/mail.example.com.key
smtpd_use_tls=yes
smtpd_tls_security_level = may
smtpd_tls_auth_only = yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtpd_tls_dh512_param_file = ${config_directory}/certs/dh_512.pem
smtpd_tls_dh1024_param_file = ${config_directory}/certs/dh_1024.pem
tls_random_source = dev:/dev/urandom
smtpd_tls_loglevel = 0
smtpd_client_new_tls_session_rate_limit = 10
smtpd_tls_exclude_ciphers =
    EXP
    EDH-RSA-DES-CBC-SHA
    ADH-DES-CBC-SHA
    DES-CBC-SHA
    SEED-SHA
    aNULL
    eNULL
    EXPORT
    DES
    RC4
    MD5
    PSK
    aECDH
    EDH-DSS-DES-CBC3-SHA
    KRB5-DES-CBC3-SHA
myhostname = mail.example.com
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = $myhostname, mail.example.com, localhost.localdomain, localhost
relayhost =
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 10.10.10.0/24
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
inet_protocols = all
local_transport = local
postscreen_greet_action = enforce
smtpd_recipient_restrictions =
    reject
    permit_mynetworks
    permit_sasl_authenticated
    warn_if_reject reject_non_fqdn_hostname
    warn_if_reject reject_non_fqdn_sender
    reject_invalid_hostname
    reject_unknown_sender_domain
    reject_unverified_recipient
    reject_unauth_destination
    reject_non_fqdn_sender
    reject_non_fqdn_recipient
    reject_non_fqdn_helo_hostname
    check_sender_ns_access cidr:/etc/postfix/drop.cidr
    check_sender_mx_access cidr:/etc/postfix/drop.cidr
    reject_rbl_client bl.spamcop.net,
    reject_rbl_client cbl.abuseat.org
    reject
smtpd_data_restrictions = reject_multi_recipient_bounce
smtpd_sender_restrictions =
    reject_non_fqdn_sender
    reject_unknown_sender_domain
smtpd_helo_restrictions =
    permit_mynetworks
    check_helo_access pcre:/etc/postfix/identitycheck.pcre
    reject_invalid_helo_hostname
disable_vrfy_command = yes
smtpd_helo_required = yes
smtpd_delay_reject = no
smtpd_client_restrictions = check_client_access cidr:/etc/postfix/drop.cidr
message_size_limit = 51200000

Prepare the rest of the configuration files:

Prepare the virtual domains file:

cat > /etc/postfix/virtual_domains &lt; &lt; EOF
 # Domain                Anything

example.com            OK
EOF

Prepare the virtual recipients file:

cat > /etc/postfix/ldap_virtual_recipients.cf &lt; &lt; EOF
bind = yes
bind_dn = cn=postfix,ou=dsa,dc=example,dc=com
bind_pw = NotTheRealPassword
server_host = ldap://ldap.example.com:389
search_base = ou=people,dc=example,dc=com
domain = example.com
query_filter = (mail=%s)
result_attribute = mail
start_tls = yes
version = 3
EOF

Prepare the virtual aliases file:

cat > /etc/postfix/ldap_virtual_aliases.cf < < EOF
bind = yes
bind_dn = cn=postfix,ou=dsa,dc=example,dc=com
bind_pw = NotTheRealPass
server_host = ldap://ldap.example.com:389
search_base = ou=alias,dc=example,dc=com
domain = example.com
query_filter = (mail=%s)
result_attribute = gosaMailAlternateAddress, gosaMailForwardingAddress
start_tls = yes
version = 3
EOF

Prepare the identity check file:

cat > /etc/postfix/identitycheck.pcre &lt; &lt; EOF
# Identity (RegEx)              Action

/^(mail\.example\.com)$/       REJECT Hostname Abuse: $1
/^(1\.2\.3\.4)$/                REJECT Hostname Abuse: $1
/^(\[1\.2\.3\.4\])$/            REJECT Hostname Abuse: $1
EOF

Prepare the blacklist file:

cat > /etc/postfix/drop.cidr &lt; &lt; EOF
# IP/CIDR                       Action

1.2.3.0/24                      REJECT Blacklisted
EOF

Generate the virtual domains hashmap:

postmap hash:/etc/postfix/virtual_domains

Start postfix:

systemctl start postfix

Verify:

ss -lnptu | grep master
tcp    LISTEN     0      100    *:25      *:*      users (("master",pid=15539,fd=12))

Enable support for smtps (port 465) and submission (port 587):

Uncomment the following lines from /etc/postfix/master.cf:

submission inet n       -       -       -       -       smtpd
smtps     inet  n       -       -       -       -       smtpd

Restart postfix:

systemctl restart postfix

Verify:

ss -lnptu | grep master
tcp    LISTEN     0      100    *:587     *:*      users:(("master",pid=15854,fd=16))
tcp    LISTEN     0      100    *:465     *:*      users:(("master",pid=15854,fd=19))
tcp    LISTEN     0      100    *:25      *:*      users:(("master",pid=15854,fd=12))

Take precautions for perfect forward secrecy:

mkdir /etc/postfix/certs
cd /etc/postfix/certs
openssl dhparam -2 -out dh_512.pem 512
openssl dhparam -2 -out dh_1024.pem 1024
chmod 600 dh_*

Lookup test:

postmap -q theo@example.com ldap:/etc/postfix/ldap_virtual_recipients.cf
theo@example.com

Open SMTP, SMTPS ans Submission ports:

ufw allow 25/tcp
ufw allow 465/tcp
ufw allow 587/tcp

Install and configure dovecot

Create a service account for dovecot using the ‘DSA’ plugin in *FusionDirectory’. It should look like this:

dn: cn=dovecot,ou=dsa,dc=example,dc=com
cn: dovecot
userPassword:: ljsdewd98dej932j98dxjud8x3jx9843xj8943j438439e3=
structuralObjectClass: organizationalRole
entryUUID: 4d0d7174-9a54-1035-8e49-bf1fd814366b
creatorsName: cn=admin,dc=example,dc=com
createTimestamp: 20160419082730Z
objectClass: organizationalRole
objectClass: top
objectClass: simpleSecurityObject
entryCSN: 20160419082730.138012Z#000000#000#000000
modifiersName: cn=admin,dc=example,dc=com
modifyTimestamp: 20160419082730Z

Install dovecot:

apt -y install dovecot-core dovecot-imapd dovecot-pop3d dovecot-lmtpd dovecot-ldap

Make the following changes in /etc/dovecot/conf.d/10-ssl.conf:

ssl = required
ssl_cert = 

service imap-login {
  inet_listener imap {
    port = 143
  }
  inet_listener imaps {
    port = 993
    ssl = yes
  }
}

service pop3-login {
  inet_listener pop3 {
    port = 110
  }
  inet_listener pop3s {
    port = 995
    ssl = yes
  }
}

service auth {
  unix_listener auth-userdb {
    mode = 0777
    user = dovecot
    group = dovecot
  }
}

Add the vmail user in the system:

addgroup --system --gid 5000 vmail
adduser --system --home /srv/vmail --uid 5000 --gid 5000 --disabled-password --disabled-login vmail

Restart dovecot:

systemctl restart dovecot

Verify:

netstat -lnptu | grep dovecot
tcp        0      0 0.0.0.0:993             0.0.0.0:*               LISTEN      20894/dovecot   
tcp        0      0 0.0.0.0:995             0.0.0.0:*               LISTEN      20894/dovecot   
tcp        0      0 0.0.0.0:110             0.0.0.0:*               LISTEN      20894/dovecot   
tcp        0      0 0.0.0.0:143             0.0.0.0:*               LISTEN      20894/dovecot   
tcp6       0      0 :::993                  :::*                    LISTEN      20894/dovecot   
tcp6       0      0 :::995                  :::*                    LISTEN      20894/dovecot   
tcp6       0      0 :::110                  :::*                    LISTEN      20894/dovecot   
tcp6       0      0 :::143                  :::*                    LISTEN      20894/dovecot

Check if you can login:

openssl s_client -connect localhost:993
<output ommitted...>
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN] Dovecot ready.
a1 LOGIN theo MyNotSoSecretPass
a1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE] Logged in
a3 LOGOUT
* BYE Logging out
a2 OK Logout completed.
closed

Enable IMAP, IMAPS, POP3 and POP3S ports:

ufw allow 110/tcp
ufw allow 143/tcp
ufw allow 993/tcp
ufw allow 995/tcp

Tell postfix to deliver mail using dovecot:

Add the following lines at the end of /etc/postfix/master.cf:

dovecot   unix  -       n       n       -       -       pipe
  flags=ODRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -e -f ${sender} -d ${recipient}

Add these attributes in /etc/postfix/main.cf:

virtual_transport = dovecot
dovecot_destination_recipient_limit = 1

Restart postfix and dovecot:
```
systemctl restart postfix dovecot
```

Install SASL for SMTP AUTH

Create a service account for saslauthd using the ‘DSA’ plugin in *FusionDirectory’. It should look like this:

dn: cn=saslauthd,ou=dsa,dc=example,dc=com
cn: saslauthd
userPassword:: ejdoedoifj9ewufd9843e9ejfd98je938jcr9843843=
structuralObjectClass: organizationalRole
entryUUID: 61143234-9a54-1035-8e4a-bf1fd814366b
creatorsName: cn=admin,dc=example,dc=com
createTimestamp: 20160419082803Z
objectClass: organizationalRole
objectClass: top
objectClass: simpleSecurityObject
entryCSN: 20160419082803.738357Z#000000#000#000000
modifiersName: cn=admin,dc=example,dc=com
modifyTimestamp: 20160419082803Z

Install SASL:
```
apt -y install libsasl2-2 sasl2-bin
```

Create the /etc/postfix/sasl/smtpd.conf file:

cat > /etc/postfix/sasl/smtpd.conf < < EOF
log_level: 3
pwcheck_method: saslauthd
mech_list: PLAIN LOGIN
EOF

Make the following changes in /etc/default/saslauthd:

START=yes
MECHANISMS="ldap"
OPTIONS="-c -m /var/spool/postfix/var/run/saslauthd"

Create the /etc/saslauthd.conf file:

cat > /etc/saslauthd.conf < < EOF
ldap_servers: ldap://ldap.example.com/
ldap_bind_dn: cn=saslauthd,ou=dsa,dc=example,dc=com
ldap_bind_pw: MySecretPass
ldap_timeout: 10
ldap_time_limit: 10
ldap_scope: sub
ldap_search_base: ou=people,dc=example,dc=com
ldap_auth_method: bind
ldap_filter: (&(uid=%u)(mail=*))
ldap_debug: 0
ldap_verbose: off
ldap_ssl: yes
ldap_starttls: no
ldap_referrals: yes
EOF

Add the postfix user in the sasl group:
```
usermod -aG sasl postfix
```

Add these attributes in /etc/postfix/main.cf:

smtpd_sasl_auth_enable = yes
broken_sasl_auth_clients = yes

Restart saslauthd and postfix:
```
systemctl restart saslauthd postfix
```

Test message delivery using SMTP AUTH:

First you need to install swaks:

apt -y install swaks

Then use swaks to test message delivery:

swaks --from noreply@example.com --to theo@example.com --server 127.0.0.1:25 --tls --auth plain --auth-user=theo

Test if the message has been delivered to inbox:
First you need to install mutt:
```
apt -y install mutt
```
Then use mutt to test if the message has been delivered to inbox:
```
mutt -f /srv/vmail/theo@example.com/Maildir/
```
NOTE: When you are asked to create a ‘mail’ folder just say ‘no’.

References

https://documentation.fusiondirectory.org/en/documentation/plugin/mail_plugin
https://documentation.fusiondirectory.org/en/documentation/plugin/alias_plugin
https://documentation.fusiondirectory.org/en/documentation/plugin/dovecot_plugin</output>

Tags debian, dovecot, fusiondirectory, howto, ldap, linux, postfix

Linux

Setting up an LDAP directory with a Fusiondirectory frontend

Post author By Theodotos Andreou
Post date September 20, 2016
3 Comments on Setting up an LDAP directory with a Fusiondirectory frontend

LDAP/Fusiondirectory setup

In this guide we will be setting up LDAP (openldap) along with the FusionDirectory web management tool.

Preparations

Install Debian jessie (Ubuntu 16.04 should work too) on your server or VM
Setup the DNS records in your DNS servers:
```
ldap.example.com.           3599    IN      A       1.1.1.1
```
NOTE: replace 1.1.1.1 with you actual internal or external IP
Allow ssh and web services on firewall:
```
apt -y install ufw
ufw allow 22/tcp
ufw allow 80/tcp
ufw allow 443/tcp
ufw enable
```
NOTE: It may be a good idea to change the default ssh port from 22 to something less predictable

Setup LDAP

Setup ldap.example.com in /etc/hosts:

127.0.2.1 ldap.example.com ldap

Install OpenLDAP and FusionDirectory
```
apt -y install slapd
```
Enter and verify the administrator password for slapd.
Create An Internal Certificate Authority for Example LTD:
- First install gnutls-bin:
```
apt -y install gnutls-bin
```
- Create the key for the internal CA for example.com:
```
certtool --generate-privkey --outfile example.com-rootCA.key
```
- Create a certificate for our internal CA:

certtool --generate-self-signed --load-privkey example.com-rootCA.key --outfile example.com-rootCA.crt
Generating a self signed certificate...
Please enter the details of the certificate's distinguished name. Just press enter to ignore a field.
Common name: Example LTD Internal ROOT CA
UID: 
Organizational unit name: IT
Organization name: Example LTD
Locality name: Limassol
State or province name: Limassol
Country name (2 chars): CY
Enter the subject's domain component (DC): 
This field should not be used in new certificates.
E-mail: admins@example.com
Enter the certificate's serial number in decimal (default: 6295758616856773074): 
The certificate will expire in (days): 7300

Activation/Expiration time.
The certificate will expire in (days): 7300

Extensions.
Does the certificate belong to an authority? (y/N): y
Path length constraint (decimal, -1 for no constraint): -1
Is this a TLS web client certificate? (y/N):
Will the certificate be used for IPsec IKE operations? (y/N): 
Is this a TLS web server certificate? (y/N): 
Enter a dnsName of the subject of the certificate: 
Enter a URI of the subject of the certificate: 
Enter the IP address of the subject of the certificate: 
Enter the e-mail of the subject of the certificate: 
Will the certificate be used to sign other certificates? (y/N): y
Will the certificate be used to sign CRLs? (y/N): 
Will the certificate be used to sign code? (y/N): 
Will the certificate be used to sign OCSP requests? (y/N): 
Will the certificate be used for time stamping? (y/N):
Enter the URI of the CRL distribution point: 
X.509 Certificate Information:
        Version: 3
        Serial Number (hex): 575f071b0d5a41d2
        Validity:
                Not Before: Mon Jun 13 19:19:27 UTC 2016
                Not After: Sun Jun 08 19:20:00 UTC 2036
        Subject: CN=Example LTD Internal ROOT CA,OU=IT,O=Example LTD,L=Limassol,ST=Limassol,C=CY,EMAIL=admins@example.com
        Subject Public Key Algorithm: RSA
        Algorithm Security Level: Medium (2048 bits)
                Modulus (bits 2048):
                        00:c0:75:c8:02:05:d0:0c:67:af:ac:0c:80:53:bf:cd
                        a3:80:76:cf:3e:14:19:99:5c:24:b4:fc:b0:42:8d:5a
                        03:5d:04:a5:85:c7:fe:e3:d4:30:6c:4c:26:90:76:c5
                        3e:a0:dc:a7:53:a7:eb:13:60:78:44:b3:0a:b2:77:0c
                        46:19:96:ea:d2:46:82:9c:11:2c:a5:e2:a1:57:38:f4
                        8e:4d:74:4f:f9:41:dd:11:f4:c2:f5:9f:b7:9a:93:7d
                        a7:f8:f3:dd:2e:08:6a:25:75:79:f3:63:e5:09:1f:bd
                        6a:38:45:85:f0:63:54:c0:08:68:41:15:66:a4:e3:84
                        49:7e:e5:c5:c7:6c:d3:c7:be:d5:5a:df:1a:1d:55:f8
                        35:73:bb:e3:ea:f7:66:af:d9:09:72:ca:17:5f:80:09
                        99:6a:49:e3:8b:f2:72:56:ac:f8:ba:60:49:d5:80:2a
                        07:e6:17:88:86:e4:3c:89:cd:af:2b:66:a1:af:53:f4
                        66:21:30:a3:22:af:a9:11:6e:98:e0:f7:6d:ef:8a:32
                        e9:0b:a4:82:7b:7b:db:2d:90:8e:bd:e4:54:04:a4:52
                        e8:cf:f6:2e:9b:97:46:ab:cb:38:06:23:33:db:42:0c
                        25:41:5a:d7:02:15:07:c6:e8:86:0b:a6:d7:7d:81:16
                        bd
                Exponent (bits 24):
                        01:00:01
        Extensions:
                Basic Constraints (critical):
                        Certificate Authority (CA): TRUE
                Key Purpose (not critical):
                        Time stamping.
                Key Usage (critical):
                        Certificate signing.
                Subject Key Identifier (not critical):
                        7a596f6dea4080e89c9e78a698d7126cd63dafa7
Other Information:
        Public Key ID:
                7a596f6dea4080e89c9e78a698d7126cd63dafa7
        Public key's random art:
                +--[ RSA 2048]----+
                |                 |
                |     . .         |
                |    . . .        |
                |   o .   .       |
                | . .+.  S o      |
                |  =o..o. + . .   |
                | o.o= .oo . o o  |
                | oo+.  .o  o o   |
                |o...  E+   .o    |
                +-----------------+

Is the above information ok? (y/N): y


Signing certificate...

Add the Example LTD Internal ROOT CA as trusted in ca-certificates:
```
mkdir /usr/share/ca-certificates/extra
cp example.com-rootCA.crt /usr/share/ca-certificates/extra
dpkg-reconfigure ca-certificates
```
Add the extra/example.com-rootCA.crt CA as a trusted CA.
- Configure slapd:
```
dpkg-reconfigure slapd
```
- Omit OpenLDAP server configuration? No
- DNS domain name: example.com
- Organization name: Example LTD
- Administrator password: ***************
- Verify password: *************
- Database backend to use: MDB
- Do you want the database to be removed when slapd is purged? No
- Move old database? Yes
- Allow LDAPv2 protocol? No
Configure TLS on LDAP:

Create a key for ldap.example.com:

certtool --generate-privkey --outfile ldap.example.com.key
Generating a 2048 bit RSA private key...

Create a certificate for ldap.example.com:

certtool --generate-certificate --load-privkey ldap.example.com.key --outfile ldap.example.com.crt --load-ca-certificate example.com-rootCA.crt --load-ca-privkey example.com-rootCA.key
Generating a signed certificate...
Please enter the details of the certificate's distinguished name. Just press enter to ignore a field.
Common name: ldap.example.com
UID: 
Organizational unit name: IT
Organization name: Example LTD
Locality name: Limassol
State or province name: Limassol
Country name (2 chars): CY
Enter the subject's domain component (DC): 
This field should not be used in new certificates.
E-mail: admins@example.com
Enter the certificate's serial number in decimal (default: 6295762607454361711):

Activation/Expiration time.
The certificate will expire in (days): 3650

Extensions.
Does the certificate belong to an authority? (y/N): 
Is this a TLS web client certificate? (y/N): 
Will the certificate be used for IPsec IKE operations? (y/N): 
Is this a TLS web server certificate? (y/N): 
Enter a dnsName of the subject of the certificate: 
Enter a URI of the subject of the certificate: 
Enter the IP address of the subject of the certificate: 
Enter the e-mail of the subject of the certificate: 
Will the certificate be used for signing (required for TLS)? (Y/n): 
Will the certificate be used for encryption (not required for TLS)? (Y/n):
X.509 Certificate Information:
        Version: 3
        Serial Number (hex): 575f0abc2f81186f
        Validity:
                Not Before: Mon Jun 13 19:35:45 UTC 2016
                Not After: Thu Jun 11 19:36:29 UTC 2026
        Subject: CN=ldap.example.com,OU=IT,O=Example LTD,L=Limassol,ST=Limassol,C=CY,EMAIL=admins@example.com
        Subject Public Key Algorithm: RSA
        Algorithm Security Level: Medium (2048 bits)
                Modulus (bits 2048):
                        00:d0:15:8e:02:90:5f:4a:9f:90:ea:1e:35:e6:4b:eb
                        a9:8c:e5:bf:68:ec:83:0e:49:5b:d1:f0:08:4b:ac:b0
                        31:d2:e0:a7:eb:18:d3:ee:b8:38:b7:c4:0a:cc:97:cc
                        b6:ac:2d:29:c8:a8:c4:7c:cc:f1:36:5a:e9:6b:52:f5
                        1e:e5:4f:90:67:34:1f:8c:a8:17:72:ee:40:87:ba:ae
                        8b:f8:4f:f8:be:51:ee:ea:d5:e4:17:63:79:22:41:c0
                        19:43:33:55:bb:46:80:5c:b8:16:18:fa:fb:17:58:c2
                        ed:d2:14:10:3b:57:5d:de:7f:29:ab:66:c2:81:87:05
                        f7:b7:27:78:a9:c0:8e:4f:1c:3f:66:6f:dd:43:26:9f
                        84:59:fb:c7:21:3c:62:4f:8d:4a:25:ab:7e:f0:5f:7e
                        df:97:f7:79:f8:c7:2d:c8:5a:7a:de:ea:5b:c7:bd:e9
                        12:17:56:d3:47:ff:eb:fa:b5:6f:d9:56:8f:c7:e8:7a
                        46:92:75:cc:ff:de:0e:88:49:7d:d7:dd:6e:8d:3f:57
                        fa:0a:7a:3b:80:ec:0e:10:dd:70:d5:9a:8d:91:ce:72
                        44:06:21:d2:9d:e9:b8:91:13:68:4c:fc:e2:bb:4d:a8
                        97:ed:e9:a4:98:5d:e7:c0:ef:3e:9d:30:28:de:bd:10
                        01
                Exponent (bits 24):
                        01:00:01
        Extensions:
                Basic Constraints (critical):
                        Certificate Authority (CA): FALSE
                Key Usage (critical):
                        Digital signature.
                        Key encipherment.
                Subject Key Identifier (not critical):
                        6d8a173de01efa11a892dda76ccd7abc609a2707
                Authority Key Identifier (not critical):
                        7a596f6dea408aa89c9e78a698d7126cd63dafa7
Other Information:
        Public Key ID:
                6d8a173de01efa00a892dda67ccd7abc609a2707
        Public key's random art:
                +--[ RSA 2048]----+
                |                 |
                |                 |
                |        .        |
                |       . +       |
                |        A =      |
                |       +Y= .     |
                |      oo+kk+     |
                |      iii==o*    |
                |       .=B**o.   |
                +-----------------+


Is the above information ok? (y/N): y

Signing certificate...

Enable LDAPS in /etc/default/slapd:

SLAPD_SERVICES="ldap:/// ldapi:/// ldaps:///"

Copy the ldap.example.com certficate and apply permissions

mkdir /etc/ldap/ssl
cp ldap.example.com.crt /etc/ldap/ssl
cp ldap.example.com.key /etc/ldap/ssl
chown -R openldap:openldap /etc/ldap/ssl    /

Create an olcSSL.ldif file:

mkdir /etc/ldap/custom_ldifs/
cd /etc/ldap/custom_ldifs/<br />
cat > olcSSL.ldif < < EOF
dn: cn=config
changetype: modify
add: olcTLSCACertificateFile
olcTLSCACertificateFile: /etc/ssl/certs/ca-certificates.crt
-
add: olcTLSCertificateKeyFile
olcTLSCertificateKeyFile: /etc/ldap/ssl/ldap.example.com.key
-
add: olcTLSCertificateFile
olcTLSCertificateFile: /etc/ldap/ssl/ldap.example.com.crt
EOF

Import the TLS configuration:

ldapmodify -Y EXTERNAL -H ldapi:/// -f olcSSL.ldif

Verify with:

slapcat -n0 | grep -i tls
olcTLSCACertificateFile: /etc/ssl/certs/ca-certificates.crt
olcTLSCertificateFile: /etc/ldap/ssl/ldap.example.com.crt
olcTLSCertificateKeyFile: /etc/ldap/ssl/ldap.example.com.key

Restart and verify slapd:

systemctl restart slapd
netstat -lnptu | grep slapd
tcp        0      0 0.0.0.0:636             0.0.0.0:*               LISTEN      27665/slapd     
tcp        0      0 0.0.0.0:389             0.0.0.0:*               LISTEN      27665/slapd     
tcp6       0      0 :::636                  :::*                    LISTEN      27665/slapd     
tcp6       0      0 :::389                  :::*                    LISTEN      27665/slapd

Setup Apache

Install Apache with PHP:

apt -y install apache2 libapache2-mod-php5 php5-ldap php5-mcrypt php5-intl php-pear

Setup Let’s Encrypt:

NOTE: this step is only necessary if you have a public facing ldap.example.com service

Setup Debian jessie backports:

cat >> /etc/apt/sources.list < < EOF
# jessie backports
deb http://ftp.debian.org/debian jessie-backports main
deb-src http://ftp.debian.org/debian jessie-backports main
EOF

Run apt update

Install Let’s Encrypt utility, certbot:

apt -y install python-certbot-apache -t jessie-backports

Generate a certificate for all the domain ldap.example.com:
```
certbot run -d ldap.example.com
```
In the TUI add the admins@example.com email and agree to the ToS. Use 000-default.conf for now.

Download and trust all Let’s Encrypt Root and Intermediate CA certificates:

wget https://letsencrypt.org/certs/isrgrootx1.pem
for i in {1..4}; do wget https://letsencrypt.org/certs/lets-encrypt-x$i-cross-signed.pem; done
cp *.pem /usr/share/ca-certificates/extra/
for f in *.pem; do cp -- "$f" "/usr/share/ca-certificates/extra/${f%.pem}.crt"; done
dpkg-reconfigure ca-certificates # Select all the newlly added CA certs

Setup FusionDirectory

Setup Fusiondirectory Repo:

gpg --keyserver keys.gnupg.net --recv-key 62B4981F 
gpg --export -a "Fusiondirectory Archive Manager <contact @fusiondirectory.org>" > FD-archive-key
apt-key add FD-archive-key
echo '# fusiondirectory repository' > /etc/apt/sources.list.d/fusiondirectory.list
echo 'deb http://repos.fusiondirectory.org/debian-jessie jessie main' >> /etc/apt/sources.list.d/fusiondirectory.list
echo 'deb-src http://repos.fusiondirectory.org/debian-jessie jessie main' >> /etc/apt/sources.list.d/fusiondirectory.list
apt update

Install FusionDirectory:

apt -y install fusiondirectory fusiondirectory-schema

Insert core FusionDirectory schemas into *slapd

fusiondirectory-insert-schema

Verify:

fusiondirectory-insert-schema -l
core
cosine
nis
inetorgperson
samba
core-fd
core-fd-conf
ldapns
recovery-fd

Create a Fusiondirectory Apache vhost (/etc/apache2/sites-available/ldap.example.com.conf):

<virtualhost *:80>
    ServerName ldap.example.com
    Redirect "/" "https://ldap.example.com/"
    ErrorLog ${APACHE_LOG_DIR}/ldap-error.log
    CustomLog ${APACHE_LOG_DIR}/ldap-access.log combined
</virtualhost>
<ifmodule mod_ssl.c>
    <virtualhost *:443>
            ServerName ldap.example.com
            ServerAdmin webmaster@example.com
            DocumentRoot /usr/share/fusiondirectory/html
            ErrorLog ${APACHE_LOG_DIR}/ldap-error.log
            CustomLog ${APACHE_LOG_DIR}/ldap-access.log combined
            SSLEngine on
            SSLCertificateFile      /etc/letsencrypt/live/ldap.example.com/cert.pem
            SSLCertificateKeyFile   /etc/letsencrypt/live/ldap.example.com/privkey.pem
            <filesmatch "\.(cgi|shtml|phtml|php)$">
                            SSLOptions +StdEnvVars
            </filesmatch>
            <directory /usr/lib/cgi-bin>
                            SSLOptions +StdEnvVars
            </directory>
            BrowserMatch "MSIE [2-6]" \
                            nokeepalive ssl-unclean-shutdown \
                            downgrade-1.0 force-response-1.0
            BrowserMatch "MSIE [17-9]" ssl-unclean-shutdown
            <ifmodule mod_php5.c>
            <location></location>
                php_admin_flag engine on
                php_admin_flag register_globals off
                php_admin_flag allow_call_time_pass_reference off
                php_admin_flag expose_php off
                php_admin_flag zend.ze1_compatibility_mode off
                php_admin_flag register_long_arrays off
                php_admin_value upload_tmp_dir /var/spool/fusiondirectory/
                php_admin_value session.cookie_lifetime 0
                #Include /etc/fusiondirectory/fusiondirectory.secrets

            </ifmodule>
            <directory /usr/share/fusiondirectory/html></directory>
                Order Allow,Deny
                # Insert your public IPs here
                Allow from 2.2.2.2
                Allow from 2.2.2.3

    </virtualhost>
</ifmodule>

Enable mod_ssl, ldap.example.com and disable 000-default and default-ssl:

a2enmod ssl
a2ensite ldap.example.com
a2dissite default-ssl
a2dissite 000-default
systemctl restart apache2

Setup FusionDirectory:
- Install optional prerequisities:
```
apt -y install php-mdb2
```
- Point your Browser to:
```
https://ldap.example.com/
```
- Create a temporary token for the setup (taken from the first setup webpage):
```
echo -n r9l1srnu0rvdeca4k826nq4e05 > /var/cache/fusiondirectory/fusiondirectory.auth 
```
  Click ‘Next’
- In the ‘Language setup’ select ‘English’ and press ‘Next’.
- If everything is ‘OK'(Green) on the ‘Installation check’ click ‘Next’
- On the ‘LDAP connection setup’:
  - Location name: default
  - Connection URI: ldap://ldap.example.com:389
  - TLS connection: Yes
  - Admin DN: cn=admin(,dc=example,dc=com)
  - Admin password: *************************
  - LDAP schema check:
    - Enable schema validation when logging in: Yes
- Keep defaults in ‘Look and feel’ except ‘Timezone’:
  - Timezone: America/Los_Angeles
- Keep all the defaults in ‘Password settings’ except this:
  - Password minimum length: 8
- In the ‘SSL’ field use these:
  - Key path: /etc/ldap/ssl/ldap.example.com.key
  - Certificate path: /etc/ldap/ssl/ldap.example.com.crt
  - CA certificate path: /etc/ssl/certs/ca-certificates.crt
  Click ‘Next’ when done.
- In the ‘LDAP inspection’ page:
  - Press ‘Migrate’ (twice) in the ‘Inspecting object classes in root object’ option
- Press ‘Create’ in the ‘Checking for super administrator’ option’, fill the fields, and ‘Apply’ when done:
  - User ID: ldapadmin
  - Password: ***************
  - Password (again): ***************
- In the ‘Checking for default ACL roles and groups’ field, press ‘Migrate’
From the ‘Finish – write the configuration file’, download the configuration file on your PC

Copy the configuration file from your PC to ldap.example.com:

scp fusiondirectory.conf root@ldap.example.com:/etc/fusiondirectory

Apply the correct permissions to the /etc/fusiondirectory/fusiondirectory.conf file:

fusiondirectory-setup --check-config
Checking FusionDirectory's config file
/etc/fusiondirectory/fusiondirectory.conf exists…
/etc/fusiondirectory/fusiondirectory.conf is not set properly, do you want to fix it ?:  [Yes/No]?
y

Click ‘Next’ when done.

Use encrypted passwords in fusiondirectory.conf:

Enable the headers module in Apache:

a2enmod headers
systemctl restart apache2

Encrypt passwords in fusiondirectory.conf:

fusiondirectory-setup --encrypt-passwords

Uncomment the following line in /etc/apache2/sites-available/ldap.example.com.conf:
```
Include /etc/fusiondirectory/fusiondirectory.secrets
```
Restart Apache:
```
systemctl restart apache2
```
If everything works as expected remove /etc/fusiondirectory/fusiondirectory.conf.orig
```
rm /etc/fusiondirectory/fusiondirectory.conf.orig
```

Allow ‘.’ (dot) in usernames:

After everything is set, login as ldapadmin
Go to ‘Configuration’
Press the ‘Edit’ button at the bottom of the page
Un-tick the ‘Strict naming policy’ option and save.

Installing essential FusionDirectory plugins

Setup LDAP related plugins:

apt -y install fusiondirectory-plugin-ldapdump fusiondirectory-plugin-ldapmanager

Setup the DSA plugin for managing service accounts:

apt -y install fusiondirectory-plugin-dsa fusiondirectory-plugin-dsa-schema
fusiondirectory-insert-schema -i /etc/ldap/schema/fusiondirectory/dsa-fd-conf.schema

Allow the service accounts to read and write the password. First create the ACL definition:

cat > /etc/ldap/custom_ldifs/service_accounts_acl.ldif < < EOF
dn: olcDatabase={1}mdb,cn=config
changetype: modify
delete: olcAccess
-
add: olcAccess
olcAccess: {0}to dn.subtree="dc=example,dc=com" attrs=userPassword
  by self write
  by dn.base="cn=admin,dc=example,dc=com" write
  by dn.children="ou=dsa,dc=example,dc=com" write
  by anonymous auth
  by * none
-
add: olcAccess
olcAccess: {1}to dn.subtree="ou=people,dc=example,dc=com"
  by self read
  by dn.base="uid=test.user,ou=people,dc=example,dc=com" write
  by dn.base="cn=admin,dc=example,dc=com" write
  by dn.children="ou=dsa,dc=example,dc=com" read
  by anonymous auth
  by * none
-
add: olcAccess
olcAccess: {2}to attrs=userPassword,shadowLastChange
  by self write
  by anonymous auth
  by dn="cn=admin,dc=example,dc=com" write
  by * none
-
add: olcAccess
olcAccess: {3}to dn.subtree="dc=example,dc=com"
  by self read
  by dn.base="cn=admin,dc=example,dc=com" write
  by dn.children="ou=dsa,dc=example,dc=com" write
  by * none
-
add: olcAccess
olcAccess: {4}to dn.base=""
  by * none
EOF

NOTE: Add two spaces before each ‘by‘ in the ldif above or you will spend endless hours in troubleshooting!

Apply the ACL

ldapadd -c -Y EXTERNAL -H ldapi:/// -f /etc/ldap/custom_ldifs/service_accounts_acl.ldif

Setup the Systems plugin:

apt -y install fusiondirectory-plugin-systems fusiondirectory-plugin-systems-schema
fusiondirectory-insert-schema -i /etc/ldap/schema/fusiondirectory/service-fd.schema
fusiondirectory-insert-schema -i /etc/ldap/schema/fusiondirectory/systems-fd-conf.schema
fusiondirectory-insert-schema -i /etc/ldap/schema/fusiondirectory/systems-fd.schema

Setup a new server:
- Name: ldap.example.com
- Description: Communications Server
- Location: My Datacenter
- Base: /
- IP-address: 10.134.31.94
- MAC-address: 04:01:05:d6:63:02
From the ‘Services’ tab setup an LDAP service on the ldap.example.com server

You are finished. You can now start connecting services to your LDAP setup like mail or asterisk.

References

https://documentation.fusiondirectory.org/en/start</contact>

Tags debian, fusiondirectory, howto, ldap, linux, openldap, ubuntu

Uncategorized

Building opendmarc for Debian Jessie

Post author By Theodotos Andreou
Post date April 30, 2016
No Comments on Building opendmarc for Debian Jessie

I have been using Skelleton‘s guide¹ to setup opendmarc on some mail server I am building.

My problem is the version of opendmarc that comes with Debian is 1.3.0 and this version has a bug² which fails to honor the IgnoreAuthenticatedClients directive.

This is an attempt to build³ the newest version, 1.3.1, for Debian jessie.

Prepare the environment

Install all necessary packages:

$ sudo apt -y install pbuilder debootstrap devscripts packaging-dev debian-keyring

Prepare a Debian jessie build environment:

$ sudo pbuilder create --debootstrapopts --variant=buildd --mirror http://ftp.us.debian.org/debian/ --distribution jessie --architecture amd64 --components main --debbuildopts -mJohn Doe \<john .doe@example.net\>

Prepare the Debian environment for opendmarc 1.3.1

Download and extract the package:

$ wget http://downloads.sourceforge.net/project/opendmarc/opendmarc-1.3.1.tar.gz
$ tar xvzf opendmarc-1.3.1.tar.gz
$ cd opendmarc-1.3.1/

Prepare for Debian packaging:

$ DEBFULLNAME="John Doe" DEBEMAIL="john.doe@example.net" dh_make -s -y --createorig

Rename the debian folder:
```
$ mv debian debian.orig
```

Shamelessly copy *debian/** from the original:

$ cd /tmp/
$ apt-get source opendmarc
$ cd opendmarc-1.3.1+dfsg/
$ cp -a debian/ ~/opendmarc-1.3.1

Optional steps:

Change the Author name to yours in debian/control
Restore the debian/changelog file:
```
$ cp debian.orig/changelog debian/
```
Get rid of the debian.orig folder:
```
$ mv debian.orig/ ..
```

Edit the debian/changelog file with dch -e:

opendmarc (1.3.1-1) unstable; urgency=medium<br />
  * Initial release: To fix the IgnoreAuthenticatedClients issue:
    http://www.trusteddomain.org/pipermail/opendmarc-users/2016-April/000636.html
 -- John Doe </john><john .doe@example.net>  Fri, 29 Apr 2016 13:43:22 +0300

Build opendmarc 1.3.1

Run pdebuild:
```
$ pdebuild
```

You cab find the resultant debs under /var/cache/pbuilder/result/:

$ ls -la /var/cache/pbuilder/result/
total 868
drwxr-xr-x 2 root      root        4096 Απρ  29 13:54 .
drwxr-xr-x 8 root      root        4096 Απρ  29 12:56 ..
-rw-r--r-- 1 theodotos theodotos  38534 Απρ  29 13:54 libopendmarc2_1.3.1-1_amd64.deb
-rw-r--r-- 1 theodotos theodotos  64210 Απρ  29 13:54 libopendmarc-dev_1.3.1-1_amd64.deb
-rw-r--r-- 1 theodotos theodotos   2348 Απρ  29 13:54 opendmarc_1.3.1-1_amd64.changes
-rw-r--r-- 1 theodotos theodotos  75890 Απρ  29 13:54 opendmarc_1.3.1-1_amd64.deb
-rw-rw-r-- 1 theodotos theodotos    846 Απρ  29 13:54 opendmarc_1.3.1-1.dsc
-rw-r--r-- 1 theodotos theodotos 663859 Απρ  29 13:54 opendmarc_1.3.1-1.tar.gz
-rw-r--r-- 1 theodotos theodotos  17136 Απρ  29 13:54 rddmarc_1.3.1-1_all.deb

You can now copy the debs over your mail server and test them.

Update: I did this before I had discovered that opendmarc 1.3.1 is in Debian jessie backports⁴. But an interesting drill nevertheless.

References

https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/ ↩︎
http://www.trusteddomain.org/pipermail/opendmarc-users/2016-April/000636.html ↩︎
http://www.theo-andreou.org/?p=1145 ↩︎
http://www.trusteddomain.org/pipermail/opendmarc-users/2016-April/000641.html</john> ↩︎

Tags build, debian, dmarc, jessie, linux, opendmarc, pbuilder