Categories
Linux

Setting up an LDAP directory with a Fusiondirectory frontend

LDAP/Fusiondirectory setup

In this guide we will be setting up LDAP (openldap) along with the FusionDirectory web management tool.

Preparations

  • Install Debian jessie (Ubuntu 16.04 should work too) on your server or VM
  • Setup the DNS records in your DNS servers:

    ldap.example.com.           3599    IN      A       1.1.1.1
    

    NOTE: replace 1.1.1.1 with you actual internal or external IP

  • Allow ssh and web services on firewall:

    apt -y install ufw
    ufw allow 22/tcp
    ufw allow 80/tcp
    ufw allow 443/tcp
    ufw enable
    

    NOTE: It may be a good idea to change the default ssh port from 22 to something less predictable

Setup LDAP

  • Setup ldap.example.com in /etc/hosts:
127.0.2.1 ldap.example.com ldap
  • Install OpenLDAP and FusionDirectory

    apt -y install slapd
    

    Enter and verify the administrator password for slapd.

  • Create An Internal Certificate Authority for Example LTD:

    • First install gnutls-bin:
      apt -y install gnutls-bin
      
    • Create the key for the internal CA for example.com:
      certtool --generate-privkey --outfile example.com-rootCA.key
      
    • Create a certificate for our internal CA:
certtool --generate-self-signed --load-privkey example.com-rootCA.key --outfile example.com-rootCA.crt
Generating a self signed certificate...
Please enter the details of the certificate's distinguished name. Just press enter to ignore a field.
Common name: Example LTD Internal ROOT CA
UID: 
Organizational unit name: IT
Organization name: Example LTD
Locality name: Limassol
State or province name: Limassol
Country name (2 chars): CY
Enter the subject's domain component (DC): 
This field should not be used in new certificates.
E-mail: admins@example.com
Enter the certificate's serial number in decimal (default: 6295758616856773074): 
The certificate will expire in (days): 7300

Activation/Expiration time.
The certificate will expire in (days): 7300

Extensions.
Does the certificate belong to an authority? (y/N): y
Path length constraint (decimal, -1 for no constraint): -1
Is this a TLS web client certificate? (y/N):
Will the certificate be used for IPsec IKE operations? (y/N): 
Is this a TLS web server certificate? (y/N): 
Enter a dnsName of the subject of the certificate: 
Enter a URI of the subject of the certificate: 
Enter the IP address of the subject of the certificate: 
Enter the e-mail of the subject of the certificate: 
Will the certificate be used to sign other certificates? (y/N): y
Will the certificate be used to sign CRLs? (y/N): 
Will the certificate be used to sign code? (y/N): 
Will the certificate be used to sign OCSP requests? (y/N): 
Will the certificate be used for time stamping? (y/N):
Enter the URI of the CRL distribution point: 
X.509 Certificate Information:
        Version: 3
        Serial Number (hex): 575f071b0d5a41d2
        Validity:
                Not Before: Mon Jun 13 19:19:27 UTC 2016
                Not After: Sun Jun 08 19:20:00 UTC 2036
        Subject: CN=Example LTD Internal ROOT CA,OU=IT,O=Example LTD,L=Limassol,ST=Limassol,C=CY,EMAIL=admins@example.com
        Subject Public Key Algorithm: RSA
        Algorithm Security Level: Medium (2048 bits)
                Modulus (bits 2048):
                        00:c0:75:c8:02:05:d0:0c:67:af:ac:0c:80:53:bf:cd
                        a3:80:76:cf:3e:14:19:99:5c:24:b4:fc:b0:42:8d:5a
                        03:5d:04:a5:85:c7:fe:e3:d4:30:6c:4c:26:90:76:c5
                        3e:a0:dc:a7:53:a7:eb:13:60:78:44:b3:0a:b2:77:0c
                        46:19:96:ea:d2:46:82:9c:11:2c:a5:e2:a1:57:38:f4
                        8e:4d:74:4f:f9:41:dd:11:f4:c2:f5:9f:b7:9a:93:7d
                        a7:f8:f3:dd:2e:08:6a:25:75:79:f3:63:e5:09:1f:bd
                        6a:38:45:85:f0:63:54:c0:08:68:41:15:66:a4:e3:84
                        49:7e:e5:c5:c7:6c:d3:c7:be:d5:5a:df:1a:1d:55:f8
                        35:73:bb:e3:ea:f7:66:af:d9:09:72:ca:17:5f:80:09
                        99:6a:49:e3:8b:f2:72:56:ac:f8:ba:60:49:d5:80:2a
                        07:e6:17:88:86:e4:3c:89:cd:af:2b:66:a1:af:53:f4
                        66:21:30:a3:22:af:a9:11:6e:98:e0:f7:6d:ef:8a:32
                        e9:0b:a4:82:7b:7b:db:2d:90:8e:bd:e4:54:04:a4:52
                        e8:cf:f6:2e:9b:97:46:ab:cb:38:06:23:33:db:42:0c
                        25:41:5a:d7:02:15:07:c6:e8:86:0b:a6:d7:7d:81:16
                        bd
                Exponent (bits 24):
                        01:00:01
        Extensions:
                Basic Constraints (critical):
                        Certificate Authority (CA): TRUE
                Key Purpose (not critical):
                        Time stamping.
                Key Usage (critical):
                        Certificate signing.
                Subject Key Identifier (not critical):
                        7a596f6dea4080e89c9e78a698d7126cd63dafa7
Other Information:
        Public Key ID:
                7a596f6dea4080e89c9e78a698d7126cd63dafa7
        Public key's random art:
                +--[ RSA 2048]----+
                |                 |
                |     . .         |
                |    . . .        |
                |   o .   .       |
                | . .+.  S o      |
                |  =o..o. + . .   |
                | o.o= .oo . o o  |
                | oo+.  .o  o o   |
                |o...  E+   .o    |
                +-----------------+

Is the above information ok? (y/N): y


Signing certificate...
  • Add the Example LTD Internal ROOT CA as trusted in ca-certificates:

    mkdir /usr/share/ca-certificates/extra
    cp example.com-rootCA.crt /usr/share/ca-certificates/extra
    dpkg-reconfigure ca-certificates
    

    Add the extra/example.com-rootCA.crt CA as a trusted CA.

    • Configure slapd:
    dpkg-reconfigure slapd
    
    • Omit OpenLDAP server configuration? No
    • DNS domain name: example.com
    • Organization name: Example LTD
    • Administrator password: ***************
    • Verify password: *************
    • Database backend to use: MDB
    • Do you want the database to be removed when slapd is purged? No
    • Move old database? Yes
    • Allow LDAPv2 protocol? No
  • Configure TLS on LDAP:

  • Create a key for ldap.example.com:

    certtool --generate-privkey --outfile ldap.example.com.key
    Generating a 2048 bit RSA private key...
    

  • Create a certificate for ldap.example.com:
certtool --generate-certificate --load-privkey ldap.example.com.key --outfile ldap.example.com.crt --load-ca-certificate example.com-rootCA.crt --load-ca-privkey example.com-rootCA.key
Generating a signed certificate...
Please enter the details of the certificate's distinguished name. Just press enter to ignore a field.
Common name: ldap.example.com
UID: 
Organizational unit name: IT
Organization name: Example LTD
Locality name: Limassol
State or province name: Limassol
Country name (2 chars): CY
Enter the subject's domain component (DC): 
This field should not be used in new certificates.
E-mail: admins@example.com
Enter the certificate's serial number in decimal (default: 6295762607454361711):

Activation/Expiration time.
The certificate will expire in (days): 3650

Extensions.
Does the certificate belong to an authority? (y/N): 
Is this a TLS web client certificate? (y/N): 
Will the certificate be used for IPsec IKE operations? (y/N): 
Is this a TLS web server certificate? (y/N): 
Enter a dnsName of the subject of the certificate: 
Enter a URI of the subject of the certificate: 
Enter the IP address of the subject of the certificate: 
Enter the e-mail of the subject of the certificate: 
Will the certificate be used for signing (required for TLS)? (Y/n): 
Will the certificate be used for encryption (not required for TLS)? (Y/n):
X.509 Certificate Information:
        Version: 3
        Serial Number (hex): 575f0abc2f81186f
        Validity:
                Not Before: Mon Jun 13 19:35:45 UTC 2016
                Not After: Thu Jun 11 19:36:29 UTC 2026
        Subject: CN=ldap.example.com,OU=IT,O=Example LTD,L=Limassol,ST=Limassol,C=CY,EMAIL=admins@example.com
        Subject Public Key Algorithm: RSA
        Algorithm Security Level: Medium (2048 bits)
                Modulus (bits 2048):
                        00:d0:15:8e:02:90:5f:4a:9f:90:ea:1e:35:e6:4b:eb
                        a9:8c:e5:bf:68:ec:83:0e:49:5b:d1:f0:08:4b:ac:b0
                        31:d2:e0:a7:eb:18:d3:ee:b8:38:b7:c4:0a:cc:97:cc
                        b6:ac:2d:29:c8:a8:c4:7c:cc:f1:36:5a:e9:6b:52:f5
                        1e:e5:4f:90:67:34:1f:8c:a8:17:72:ee:40:87:ba:ae
                        8b:f8:4f:f8:be:51:ee:ea:d5:e4:17:63:79:22:41:c0
                        19:43:33:55:bb:46:80:5c:b8:16:18:fa:fb:17:58:c2
                        ed:d2:14:10:3b:57:5d:de:7f:29:ab:66:c2:81:87:05
                        f7:b7:27:78:a9:c0:8e:4f:1c:3f:66:6f:dd:43:26:9f
                        84:59:fb:c7:21:3c:62:4f:8d:4a:25:ab:7e:f0:5f:7e
                        df:97:f7:79:f8:c7:2d:c8:5a:7a:de:ea:5b:c7:bd:e9
                        12:17:56:d3:47:ff:eb:fa:b5:6f:d9:56:8f:c7:e8:7a
                        46:92:75:cc:ff:de:0e:88:49:7d:d7:dd:6e:8d:3f:57
                        fa:0a:7a:3b:80:ec:0e:10:dd:70:d5:9a:8d:91:ce:72
                        44:06:21:d2:9d:e9:b8:91:13:68:4c:fc:e2:bb:4d:a8
                        97:ed:e9:a4:98:5d:e7:c0:ef:3e:9d:30:28:de:bd:10
                        01
                Exponent (bits 24):
                        01:00:01
        Extensions:
                Basic Constraints (critical):
                        Certificate Authority (CA): FALSE
                Key Usage (critical):
                        Digital signature.
                        Key encipherment.
                Subject Key Identifier (not critical):
                        6d8a173de01efa11a892dda76ccd7abc609a2707
                Authority Key Identifier (not critical):
                        7a596f6dea408aa89c9e78a698d7126cd63dafa7
Other Information:
        Public Key ID:
                6d8a173de01efa00a892dda67ccd7abc609a2707
        Public key's random art:
                +--[ RSA 2048]----+
                |                 |
                |                 |
                |        .        |
                |       . +       |
                |        A =      |
                |       +Y= .     |
                |      oo+kk+     |
                |      iii==o*    |
                |       .=B**o.   |
                +-----------------+


Is the above information ok? (y/N): y

Signing certificate...
  • Enable LDAPS in /etc/default/slapd:

    SLAPD_SERVICES="ldap:/// ldapi:/// ldaps:///"
    
    • Copy the ldap.example.com certficate and apply permissions
      mkdir /etc/ldap/ssl
      cp ldap.example.com.crt /etc/ldap/ssl
      cp ldap.example.com.key /etc/ldap/ssl
      chown -R openldap:openldap /etc/ldap/ssl    /
      
    • Create an olcSSL.ldif file:
mkdir /etc/ldap/custom_ldifs/
cd /etc/ldap/custom_ldifs/<br />
cat > olcSSL.ldif < < EOF
dn: cn=config
changetype: modify
add: olcTLSCACertificateFile
olcTLSCACertificateFile: /etc/ssl/certs/ca-certificates.crt
-
add: olcTLSCertificateKeyFile
olcTLSCertificateKeyFile: /etc/ldap/ssl/ldap.example.com.key
-
add: olcTLSCertificateFile
olcTLSCertificateFile: /etc/ldap/ssl/ldap.example.com.crt
EOF
  • Import the TLS configuration:
ldapmodify -Y EXTERNAL -H ldapi:/// -f olcSSL.ldif

Verify with:

slapcat -n0 | grep -i tls
olcTLSCACertificateFile: /etc/ssl/certs/ca-certificates.crt
olcTLSCertificateFile: /etc/ldap/ssl/ldap.example.com.crt
olcTLSCertificateKeyFile: /etc/ldap/ssl/ldap.example.com.key
  • Restart and verify slapd:
    systemctl restart slapd
    netstat -lnptu | grep slapd
    tcp        0      0 0.0.0.0:636             0.0.0.0:*               LISTEN      27665/slapd     
    tcp        0      0 0.0.0.0:389             0.0.0.0:*               LISTEN      27665/slapd     
    tcp6       0      0 :::636                  :::*                    LISTEN      27665/slapd     
    tcp6       0      0 :::389                  :::*                    LISTEN      27665/slapd 
    

Setup Apache

  • Install Apache with PHP:
    apt -y install apache2 libapache2-mod-php5 php5-ldap php5-mcrypt php5-intl php-pear
    

Setup Let’s Encrypt:

NOTE: this step is only necessary if you have a public facing ldap.example.com service

  • Setup Debian jessie backports:
cat >> /etc/apt/sources.list < < EOF
# jessie backports
deb http://ftp.debian.org/debian jessie-backports main
deb-src http://ftp.debian.org/debian jessie-backports main
EOF
  • Run apt update
  • Install Let’s Encrypt utility, certbot:
    apt -y install python-certbot-apache -t jessie-backports
    
  • Generate a certificate for all the domain ldap.example.com:
    certbot run -d ldap.example.com
    

    In the TUI add the admins@example.com email and agree to the ToS. Use 000-default.conf for now.

  • Download and trust all Let’s Encrypt Root and Intermediate CA certificates:

    wget https://letsencrypt.org/certs/isrgrootx1.pem
    for i in {1..4}; do wget https://letsencrypt.org/certs/lets-encrypt-x$i-cross-signed.pem; done
    cp *.pem /usr/share/ca-certificates/extra/
    for f in *.pem; do cp -- "$f" "/usr/share/ca-certificates/extra/${f%.pem}.crt"; done
    dpkg-reconfigure ca-certificates # Select all the newlly added CA certs
    

  • Setup FusionDirectory

    • Setup Fusiondirectory Repo:
      gpg --keyserver keys.gnupg.net --recv-key 62B4981F 
      gpg --export -a "Fusiondirectory Archive Manager <contact @fusiondirectory.org>" > FD-archive-key
      apt-key add FD-archive-key
      echo '# fusiondirectory repository' > /etc/apt/sources.list.d/fusiondirectory.list
      echo 'deb http://repos.fusiondirectory.org/debian-jessie jessie main' >> /etc/apt/sources.list.d/fusiondirectory.list
      echo 'deb-src http://repos.fusiondirectory.org/debian-jessie jessie main' >> /etc/apt/sources.list.d/fusiondirectory.list
      apt update
      
    • Install FusionDirectory:
      apt -y install fusiondirectory fusiondirectory-schema
      
    • Insert core FusionDirectory schemas into *slapd
      fusiondirectory-insert-schema
      

      Verify:

      fusiondirectory-insert-schema -l
      core
      cosine
      nis
      inetorgperson
      samba
      core-fd
      core-fd-conf
      ldapns
      recovery-fd
      
    • Create a Fusiondirectory Apache vhost (/etc/apache2/sites-available/ldap.example.com.conf):
      <virtualhost *:80>
          ServerName ldap.example.com
          Redirect "/" "https://ldap.example.com/"
          ErrorLog ${APACHE_LOG_DIR}/ldap-error.log
          CustomLog ${APACHE_LOG_DIR}/ldap-access.log combined
      </virtualhost>
      <ifmodule mod_ssl.c>
          <virtualhost *:443>
                  ServerName ldap.example.com
                  ServerAdmin webmaster@example.com
                  DocumentRoot /usr/share/fusiondirectory/html
                  ErrorLog ${APACHE_LOG_DIR}/ldap-error.log
                  CustomLog ${APACHE_LOG_DIR}/ldap-access.log combined
                  SSLEngine on
                  SSLCertificateFile      /etc/letsencrypt/live/ldap.example.com/cert.pem
                  SSLCertificateKeyFile   /etc/letsencrypt/live/ldap.example.com/privkey.pem
                  <filesmatch "\.(cgi|shtml|phtml|php)$">
                                  SSLOptions +StdEnvVars
                  </filesmatch>
                  <directory /usr/lib/cgi-bin>
                                  SSLOptions +StdEnvVars
                  </directory>
                  BrowserMatch "MSIE [2-6]" \
                                  nokeepalive ssl-unclean-shutdown \
                                  downgrade-1.0 force-response-1.0
                  BrowserMatch "MSIE [17-9]" ssl-unclean-shutdown
                  <ifmodule mod_php5.c>
                  <location></location>
                      php_admin_flag engine on
                      php_admin_flag register_globals off
                      php_admin_flag allow_call_time_pass_reference off
                      php_admin_flag expose_php off
                      php_admin_flag zend.ze1_compatibility_mode off
                      php_admin_flag register_long_arrays off
                      php_admin_value upload_tmp_dir /var/spool/fusiondirectory/
                      php_admin_value session.cookie_lifetime 0
                      #Include /etc/fusiondirectory/fusiondirectory.secrets
      
                  </ifmodule>
                  <directory /usr/share/fusiondirectory/html></directory>
                      Order Allow,Deny
                      # Insert your public IPs here
                      Allow from 2.2.2.2
                      Allow from 2.2.2.3
      
          </virtualhost>
      </ifmodule>
      
    • Enable mod_ssl, ldap.example.com and disable 000-default and default-ssl:
      a2enmod ssl
      a2ensite ldap.example.com
      a2dissite default-ssl
      a2dissite 000-default
      systemctl restart apache2
      
    • Setup FusionDirectory:
      • Install optional prerequisities:
        apt -y install php-mdb2
        
      • Point your Browser to:
        https://ldap.example.com/
        
      • Create a temporary token for the setup (taken from the first setup webpage):
        echo -n r9l1srnu0rvdeca4k826nq4e05 > /var/cache/fusiondirectory/fusiondirectory.auth 
        

        Click ‘Next’

      • In the ‘Language setup’ select ‘English’ and press ‘Next’.

      • If everything is ‘OK'(Green) on the ‘Installation check’ click ‘Next’

      • On the ‘LDAP connection setup’:

        • Location name: default
        • Connection URI: ldap://ldap.example.com:389
        • TLS connection: Yes
        • Admin DN: cn=admin(,dc=example,dc=com)
        • Admin password: *************************
        • LDAP schema check:
          • Enable schema validation when logging in: Yes
      • Keep defaults in ‘Look and feel’ except ‘Timezone’:
        • Timezone: America/Los_Angeles
      • Keep all the defaults in ‘Password settings’ except this:
        • Password minimum length: 8
      • In the ‘SSL’ field use these:
        • Key path: /etc/ldap/ssl/ldap.example.com.key
        • Certificate path: /etc/ldap/ssl/ldap.example.com.crt
        • CA certificate path: /etc/ssl/certs/ca-certificates.crt

        Click ‘Next’ when done.

      • In the ‘LDAP inspection’ page:

        • Press ‘Migrate’ (twice) in the ‘Inspecting object classes in root object’ option
      • Press ‘Create’ in the ‘Checking for super administrator’ option’, fill the fields, and ‘Apply’ when done:

        • User ID: ldapadmin
        • Password: ***************
        • Password (again): ***************
      • In the ‘Checking for default ACL roles and groups’ field, press ‘Migrate’
    • From the ‘Finish – write the configuration file’, download the configuration file on your PC

    • Copy the configuration file from your PC to ldap.example.com:

      scp fusiondirectory.conf root@ldap.example.com:/etc/fusiondirectory
      

    • Apply the correct permissions to the /etc/fusiondirectory/fusiondirectory.conf file:
    fusiondirectory-setup --check-config
    Checking FusionDirectory's config file
    /etc/fusiondirectory/fusiondirectory.conf exists…
    /etc/fusiondirectory/fusiondirectory.conf is not set properly, do you want to fix it ?:  [Yes/No]?
    y
    
    • Click ‘Next’ when done.
  • Use encrypted passwords in fusiondirectory.conf:

    • Enable the headers module in Apache:
      a2enmod headers
      systemctl restart apache2
      
    • Encrypt passwords in fusiondirectory.conf:
      fusiondirectory-setup --encrypt-passwords
      
    • Uncomment the following line in /etc/apache2/sites-available/ldap.example.com.conf:
      Include /etc/fusiondirectory/fusiondirectory.secrets
      
    • Restart Apache:
      systemctl restart apache2
      
    • If everything works as expected remove /etc/fusiondirectory/fusiondirectory.conf.orig
      rm /etc/fusiondirectory/fusiondirectory.conf.orig
      
  • Allow ‘.’ (dot) in usernames:
    • After everything is set, login as ldapadmin
    • Go to ‘Configuration’
    • Press the ‘Edit’ button at the bottom of the page
    • Un-tick the ‘Strict naming policy’ option and save.
  • Installing essential FusionDirectory plugins

    • Setup LDAP related plugins:
      apt -y install fusiondirectory-plugin-ldapdump fusiondirectory-plugin-ldapmanager
      
    • Setup the DSA plugin for managing service accounts:
      apt -y install fusiondirectory-plugin-dsa fusiondirectory-plugin-dsa-schema
      fusiondirectory-insert-schema -i /etc/ldap/schema/fusiondirectory/dsa-fd-conf.schema
      
      • Allow the service accounts to read and write the password. First create the ACL definition:
    cat > /etc/ldap/custom_ldifs/service_accounts_acl.ldif < < EOF
    dn: olcDatabase={1}mdb,cn=config
    changetype: modify
    delete: olcAccess
    -
    add: olcAccess
    olcAccess: {0}to dn.subtree="dc=example,dc=com" attrs=userPassword
      by self write
      by dn.base="cn=admin,dc=example,dc=com" write
      by dn.children="ou=dsa,dc=example,dc=com" write
      by anonymous auth
      by * none
    -
    add: olcAccess
    olcAccess: {1}to dn.subtree="ou=people,dc=example,dc=com"
      by self read
      by dn.base="uid=test.user,ou=people,dc=example,dc=com" write
      by dn.base="cn=admin,dc=example,dc=com" write
      by dn.children="ou=dsa,dc=example,dc=com" read
      by anonymous auth
      by * none
    -
    add: olcAccess
    olcAccess: {2}to attrs=userPassword,shadowLastChange
      by self write
      by anonymous auth
      by dn="cn=admin,dc=example,dc=com" write
      by * none
    -
    add: olcAccess
    olcAccess: {3}to dn.subtree="dc=example,dc=com"
      by self read
      by dn.base="cn=admin,dc=example,dc=com" write
      by dn.children="ou=dsa,dc=example,dc=com" write
      by * none
    -
    add: olcAccess
    olcAccess: {4}to dn.base=""
      by * none
    EOF
    

    NOTE: Add two spaces before each ‘by‘ in the ldif above or you will spend endless hours in troubleshooting!

    • Apply the ACL
      ldapadd -c -Y EXTERNAL -H ldapi:/// -f /etc/ldap/custom_ldifs/service_accounts_acl.ldif
      
      • Setup the Systems plugin:
      apt -y install fusiondirectory-plugin-systems fusiondirectory-plugin-systems-schema
      fusiondirectory-insert-schema -i /etc/ldap/schema/fusiondirectory/service-fd.schema
      fusiondirectory-insert-schema -i /etc/ldap/schema/fusiondirectory/systems-fd-conf.schema
      fusiondirectory-insert-schema -i /etc/ldap/schema/fusiondirectory/systems-fd.schema
      
      • Setup a new server:
        • Name: ldap.example.com
        • Description: Communications Server
        • Location: My Datacenter
        • Base: /
        • IP-address: 10.134.31.94

        • MAC-address: 04:01:05:d6:63:02
      • From the ‘Services’ tab setup an LDAP service on the ldap.example.com server

    You are finished. You can now start connecting services to your LDAP setup like mail or asterisk.

    References

    • https://documentation.fusiondirectory.org/en/start</contact>
    Categories
    Uncategorized

    Building opendmarc for Debian Jessie

    I have been using Skelleton‘s guide1 to setup opendmarc on some mail server I am building.

    My problem is the version of opendmarc that comes with Debian is 1.3.0 and this version has a bug2 which fails to honor the IgnoreAuthenticatedClients directive.

    This is an attempt to build3 the newest version, 1.3.1, for Debian jessie.

    Prepare the environment

    • Install all necessary packages:
      $ sudo apt -y install pbuilder debootstrap devscripts packaging-dev debian-keyring
      
    • Prepare a Debian jessie build environment:
      $ sudo pbuilder create --debootstrapopts --variant=buildd --mirror http://ftp.us.debian.org/debian/ --distribution jessie --architecture amd64 --components main --debbuildopts -mJohn Doe \<john .doe@example.net\>
      

    Prepare the Debian environment for opendmarc 1.3.1

    • Download and extract the package:
      $ wget http://downloads.sourceforge.net/project/opendmarc/opendmarc-1.3.1.tar.gz
      $ tar xvzf opendmarc-1.3.1.tar.gz
      $ cd opendmarc-1.3.1/
      
    • Prepare for Debian packaging:
      $ DEBFULLNAME="John Doe" DEBEMAIL="john.doe@example.net" dh_make -s -y --createorig
      
      • Rename the debian folder:
        $ mv debian debian.orig
        
    • Shamelessly copy *debian/** from the original:
      $ cd /tmp/
      $ apt-get source opendmarc
      $ cd opendmarc-1.3.1+dfsg/
      $ cp -a debian/ ~/opendmarc-1.3.1
      
    • Optional steps:
      • Change the Author name to yours in debian/control
      • Restore the debian/changelog file:

        $ cp debian.orig/changelog debian/
        

      • Get rid of the debian.orig folder:
        $ mv debian.orig/ ..
        
      • Edit the debian/changelog file with dch -e:
        opendmarc (1.3.1-1) unstable; urgency=medium<br />
          * Initial release: To fix the IgnoreAuthenticatedClients issue:
            http://www.trusteddomain.org/pipermail/opendmarc-users/2016-April/000636.html
         -- John Doe </john><john .doe@example.net>  Fri, 29 Apr 2016 13:43:22 +0300
        

    Build opendmarc 1.3.1

    • Run pdebuild:
      $ pdebuild
      
    • You cab find the resultant debs under /var/cache/pbuilder/result/:
      $ ls -la /var/cache/pbuilder/result/
      total 868
      drwxr-xr-x 2 root      root        4096 Απρ  29 13:54 .
      drwxr-xr-x 8 root      root        4096 Απρ  29 12:56 ..
      -rw-r--r-- 1 theodotos theodotos  38534 Απρ  29 13:54 libopendmarc2_1.3.1-1_amd64.deb
      -rw-r--r-- 1 theodotos theodotos  64210 Απρ  29 13:54 libopendmarc-dev_1.3.1-1_amd64.deb
      -rw-r--r-- 1 theodotos theodotos   2348 Απρ  29 13:54 opendmarc_1.3.1-1_amd64.changes
      -rw-r--r-- 1 theodotos theodotos  75890 Απρ  29 13:54 opendmarc_1.3.1-1_amd64.deb
      -rw-rw-r-- 1 theodotos theodotos    846 Απρ  29 13:54 opendmarc_1.3.1-1.dsc
      -rw-r--r-- 1 theodotos theodotos 663859 Απρ  29 13:54 opendmarc_1.3.1-1.tar.gz
      -rw-r--r-- 1 theodotos theodotos  17136 Απρ  29 13:54 rddmarc_1.3.1-1_all.deb
      

    You can now copy the debs over your mail server and test them.

    Update: I did this before I had discovered that opendmarc 1.3.1 is in Debian jessie backports4. But an interesting drill nevertheless.

    References


    1. https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/ ↩︎
    2. http://www.trusteddomain.org/pipermail/opendmarc-users/2016-April/000636.html ↩︎
    3. http://www.theo-andreou.org/?p=1145 ↩︎
    4. http://www.trusteddomain.org/pipermail/opendmarc-users/2016-April/000641.html</john> ↩︎
    Categories
    Uncategorized

    Installing a Simple ORCID Authentication Node

    These are the instructions for installing an ORCID1 authentication node. We will be using the simple-orcid-auth-node2 developed by the ORCID organization.

    Assumptions

    • An Ubuntu 16.04 server machine but works on 14.04x with some minor changes.
    • A FQDN, let’s say orcid.example.com.
    • Server IP is 10.2.2.2 in our case.
    • Create an orcid user: sudo useradd -r -m -d /var/www/html/orcid orcid.
    • For Ubuntu 14.04 it is better to use /var/www/orcid instead of /var/www/html/orcid.
    • Also use service servicename restart on 14.04.x instead of systemctl restart service.

    Installing simple-orcid-auth-node

    • Install necessary packages (as a privileged user):
      $ sudo apt -y install nginx nodejs npm
      

      NOTE: If you are using Ubuntu 14.04.x do not install the node package. This package is completely unrelated with nodejs.__3

    • Download and extract simple-orcid-auth-node (as the orcid user):

      sudo su - orcid
      wget https://github.com/rcpeters/simple-orcid-auth-node/archive/master.tar.gz
      tar xvzf master.tar.gz
      

    • Install the application:
      $ cd simple-orcid-auth-node-master/
      $ npm install
      
    • Test run the application (as the orcid user):
      $ nodejs client-app.js
      server started on 8000
      

      Looks OK. Now point your Hit CTRL^C and move on.

      NOTE: If you prefer using the legacy node client-app.js invocation, you need to install the nodejs-legacy package as well.

    Setting ORCID as an autostart service

    • Autostart using systemd4 (Ubuntu 16.04):

      • Create the /etc/systemd/system/orcid.service service definition (as the root user):
        $ cat > /etc/systemd/system/orcid.service < < EOF
        [Service]
        ExecStart=/usr/bin/nodejs /var/www/html/orcid/simple-orcid-auth-node-master/client-app.js
        WorkingDirectory=/var/www/html/orcid/simple-orcid-auth-node-master
        Restart=always
        StandardOutput=syslog
        StandardError=syslog
        SyslogIdentifier=orcid
        User=orcid
        Group=orcid
        Environment=NODE_ENV=production
        [Install]
        WantedBy=multi-user.target
        EOF
        
      • Reload systemd and start the service:
        $ sudo systemctl daemon-reload
        $ sudo systemctl start orcid.service
        
      • Verify that the service is started:
        $ sudo systemctl status orcid.service
        ● orcid.service
        Loaded: loaded (/etc/systemd/system/orcid.service; disabled; vendor preset: enabled)
        Active: active (running) since Wed 2016-04-27 09:00:16 UTC; 37s ago
        Main PID: 11141 (nodejs)
        Tasks: 5 (limit: 512)
        Memory: 24.1M
        CPU: 268ms
        CGroup: /system.slice/orcid.service
           └─11141 /usr/bin/nodejs /var/www/html/orcid/simple-orcid-auth-node-master/client-app.js
        Apr 27 09:00:16 orcid systemd[1]: Started orcid.service.
        Apr 27 09:00:16 orcid orcid[11141]: server started on 8000
        
    • Autostart using sysv-init (Ubuntu 14.04.x):
      • Prepare a sysv-init startup script or use mine for convinience:
        $ cd /etc/init.d
        $ wget https://raw.githubusercontent.com/theodotos/arena/master/orcid
        $ chmod +x orcid
        $ update-rc.d orcid enable
        $ update-rc.d orcid defaults
        

      Now orcid should be able to autostart after a reboot.

    Setting up nginx

    • Prepare this configuration:

      $ cat > /etc/nginx/sites-available/orcid < < EOF
      server {
          listen 80;
          listen [::]:80 ipv6only=on;
          server_name orcid.example.com;
          access_log  /var/log/nginx/orcid.access.log;
          error_log /var/log/nginx/orcid.error.log;
          location / {
              proxy_pass http://localhost:8000/;
              proxy_set_header Host \$host;
              proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
          }
      }
      EOF
      

    • Enable the orcid site:
      $ cd /etc/nginx/sites-enabled/
      $ sudo ln -s /etc/nginx/sites-available/orcid
      
    • Uncomment the following line in /etc/nginx/nginx.conf5:
          server_names_hash_bucket_size 64;
      
    • Restart nginx:
      $ sudo systemctl restart nginx.service
      
    • Verify nginx with sudo systemctl status nginx.service

    Now you can visit the http://orcid.example.com site and test your setup

    Going to production

    The default simple-orcid-auth-node is using the sandbox ORCID service which is ideal for testing. This is how the configuration file (helpers/config.js) looks like:

    module.exports = config = {
      // Config for OAuth2 
      CLIENT_ID: 'APP-O9TUKAPVLALU1SOJ',
      CLIENT_SECRET: '0eafb938-020e-45a6-a148-3c222171d9d8',
      AUTHORIZE_URI: 'https://sandbox.orcid.org/oauth/authorize',
      TOKEN_EXCHANGE_URI: 'https://api.sandbox.orcid.org/oauth/token',
      CODE_CALLBACK_URI: 'http://localhost:8000/authorization-code-callback',
      // General server config
      PORT: '8000',
      SERVER_IP: '127.0.0.1',
    }
    ...
    

    This setup will not work in production. You have to modify the CLIENT_ID and CLIENT_SECRET variables with your own credentials and change the AUTHORIZE_URI and TOKEN_EXCHANGE_URI to point to the production ORCID services:

    module.exports = config = {
      // Config for OAuth2 
      CLIENT_ID: 'APP-HSGSHJS335353GSGSG',
      CLIENT_SECRET: '56d4eb21-6622-8483-3422-f53f3fs53sfs35f',
      AUTHORIZE_URI: 'https://orcid.org/oauth/authorize',
      TOKEN_EXCHANGE_URI: 'https://api.orcid.org/oauth/token',
      CODE_CALLBACK_URI: 'http://localhost:8000/authorization-code-callback',
      // General server config
      PORT: '8000',
      SERVER_IP: '127.0.0.1',
    }
    ...
    

    Restart nginx and orcid when done:

    $ sudo systemctl restart nginx.service orcid.service
    

    References


    1. https://en.wikipedia.org/wiki/ORCID ↩︎
    2. https://github.com/ORCID/simple-orcid-auth-node ↩︎
    3. https://github.com/ORCID/simple-orcid-auth-node/issues/3 ↩︎
    4. https://www.digitalocean.com/community/tutorials/how-to-deploy-node-js-applications-using-systemd-and-nginx ↩︎
    5. http://charles.lescampeurs.org/2008/11/14/fix-nginx-increase-server_names_hash_bucket_size</service> ↩︎
    Categories
    Linux

    Setting up your own ebook library

    This article is about setting up you own ebook library with support for the OPDS catalog format . The COPS ebook catalog can read a Calibre library and expose it as a website. It is developed on PHP.

    Prerequisites

    • Prepare a Linux system. In this guide we are using Ubuntu 14.04.4.
    • Have a cops.example.com DNS A record pointing to the IP of your server.

    Install necessary packages

    $ sudo apt-get -y samba nginx php5-fpm php5-gd php5-sqlite php5-json php5-intl git
    

    Setup a SAMBA share

    Samba is needed for sharing the Calibre library directory with the library administrator. Alternatively you could use NFS or even OwnCloud.

    Note

    It is a security risk to expose Samba on a public server. Restrict it through firewall or use OwnCloud instead.

    • Tweak the Samba configuration file (/etc/samba/smb.conf):
      #  unix password sync = yes
         unix password sync = no
      
      #  pam password change = yes
         pam password change = no
      
    • Add a Samba share for calibre (/etc/samba/smb.conf):
      [calibre$]
      path = /srv/calibre
      valid users = calibre
      write list = calibre
      read only = no
      
    • Restart Samba services:
      $ sudo service smbd restart ; sudo service nmbd restart
      
    • Add the calibre user:
      $ useradd -m -d /srv/calibre -s /usr/sbin/nologin calibre
      $ smbpasswd -a calibre
      

    Setup Calibre

    • Now you should setup Calibre on your PC. On Debian/Ubuntu:
      $ sudo apt-get -y install calibre
      

      Windows users can download it from here:

      http://calibre-ebook.com/download_windows

    • Configure Calibre to use the calibre share as its catalog:

      • You can "mount" the calibre share on Linux or "map" the share on Windows.
      • Create an ebook Directory under the calibre share.
      • Create a new Calibre library inside the ebook directory. From the Calibre menu:
        • Callibre Library –> Switch/create library -> Create an empty Library at the new location and fill the path in the "New Location" text field.
      • If you get an error about "Corrupted database" just click ‘Yes" to rebuilt it.
      • Start adding books

    Setup COPS OPDS

    • Download the software:
      $ mkdir /var/www
      $ cd /var/www
      $ git clone https://github.com/seblucas/cops.git
      
    • Setup COPS:

      Under the /var/www/cops/ directory, copy the config_default.php.example to config_default.php:

      $ cp config_local.php.example config_local.php
      

      Make the following changes in config_default.php:

      < ?php
          if (!isset($config))
              $config = array();
      
          /*
           * The directory containing calibre's metadata.db file, with sub-directories
           * containing all the formats.
           * BEWARE : it has to end with a /
           */
          $config['calibre_directory'] = '/srv/calibre/ebooks/';
      
          /*
           * Catalog's title
           */
          $config['cops_title_default'] = 'My Ebook Portal';
      
          $config['calibre_internal_directory'] = '/ebooks/';
      
          $config['cops_full_url'] = 'cops.example.com';
      
          $config['cops_x_accel_redirect'] = 'X-Accel-Redirect';
      
          /*
           * use URL rewriting for downloading of ebook in HTML catalog
           * See README for more information
           *  1 : enable
           *  0 : disable
           */
          $config['cops_use_url_rewriting'] = '1';
      

    Setup nginx

    • Create an /etc/nginx/sites-available/cops.example.com
      server {
      
          listen 80 default_server;
          listen [::]:80 default_server ipv6only=on;
      
          server_name cops.example.com;
      
          access_log  /var/log/nginx/cops.access.log;
          error_log /var/log/nginx/cops.error.log;
          root   /var/www/cops;
          #index feed.php;
          index index.php;
      
          #Useful only for Kobo reader
          location /download/ {
                rewrite ^/download/(\d+)/(\d+)/.*\.(.*)$ /fetch.php?data=$1&db=$2&type=$3 last;
                rewrite ^/download/(\d+)/.*\.(.*)$ /fetch.php?data=$1&type=$2 last;
                break;
              }
      
              #Can break loading the images - if you don't see anything, comment
              location ~ ^/images.*\.(gif|png|ico|jpg)$ {
                      expires 31d;
              }
              #Can also break loading the images, comment if it happens
              location ~ .(js|css|eot|svg|woff|ttf)$ {
                      expires 31d;
              }
      
          #Not necessarily correct, it depends on distro.
          location ~ \.php$ {
             try_files $uri =404;
             include /etc/nginx/fastcgi_params;
             fastcgi_param   SCRIPT_FILENAME  $document_root$fastcgi_script_name;
             fastcgi_pass    unix:/run/php5-fpm.sock;
          }
      
          location /ebooks {
              root /srv/calibre;
              internal;
          }
      }
      

      Note:

      The feed.php setting was redirecting me to an XML site with this error: This XML file does not appear to have any style information associated with it. The document tree is shown below. So I am using index.php instead, as the index file.

    • Enable the cops.example.com site and disable the default:
      $ cd /etc/nginx/sites-enabled/
      $ sudo ln -s /etc/nginx/sites-available/cops.example.com
      $ sudo unlink default
      
    • Restart nginx and php5-fpm:
      $ sudo service php5-fpm restart ; sudo service nginx restart
      
    • Change permissions to let nginx write to the library:
      $ usermod -a -G calibre www-data
      $ chmod -R g+w /srv/calibre/ebooks/
      

    Now you can navigate to http://cops.example.com and enjoy your newly created ebook library!

    References

    • https://github.com/seblucas/cops
    • http://blog.slucas.fr/en/oss/calibre-opds-php-server
    • https://github.com/seblucas/cops/wiki/Full-example-with-Nginx
    Categories
    Uncategorized

    Upgrade or Reload OpenElec on an XstreamTec HTPC media player

    This guide describes how to upgrade your XstreamTec HTPC appliance from XBMC to KODI or re-install the system, in case of system corruption.

    XstreamTec does not appear to have an official website but I have contacted them via Amazon and they promptly replied with instructions.

    Prerequisities

    • An XstreamTec HTPC media player.
    • A Linux system with an SD card slot.
    • A Windows PC or VM (optional).
    • An empty SD card (4GB minimum).

    Install the recovery firmware

    1. Download the android recovery sd card image (use Google Chrome or Chromium):

      https://mega.co.nz/#!wExHTJgC!XPuoFyhF_S-roTEhsMhQ8BLrfH4R5tXhp2jwaLF_o1M

    • Convert the image to raw format:

      If you extract the downloaded file (MATRICOM_MX2_115_FULL_AUG_09_2013.rar) you will get the image file (MATRICOM_MX2_115_FULL_AUG_09_2013.imgc). Alas this file has a peculiar format (.imgc) which is some kind of compressed raw image. I couldn’t figure out how to convert this to a raw (suitable for dd) image on Linux. To convert this to raw you need a Windows only utility called HDDRawCopy. So you will need a Windows system (WINE does not work for this) and use HDDRawCopy to convert the .imgc file to Raw/dd format.

      If you do not have a Windows machine, you can download the image I converted already, in .bz2 format from here:
      MATRICOM_MX2_115_FULL_AUG_09_2013.dd.bz2. Just extract it like this to get the raw image:

      $ cd ~/Downloads
      $ bzip2 -d MATRICOM_MX2_115_FULL_AUG_09_2013.dd.bz2
      

    • Prepare the SD card:

      Note
      Make sure you don’t have any useful data on the SD card!

      First verify that the SD card is inserted:

        $ cat /proc/partitions 
        major minor  #blocks  name
      
            8        0  488386584 sda
            8        1     524288 sda1
            8        2     249856 sda2
            8        3  487610368 sda3
           11        0    1048575 sr0
         252        0  487608320 dm-0
         252        1  483524608 dm-1
         252        2    4067328 dm-2
         179        0    7761920 mmcblk0
         179        1      71680 mmcblk0p1
         179        2     124928 mmcblk0p2
         179        3    7559168 mmcblk0p3
      

      Unmount all partitions on the SD card:

      $ sudo umount /dev/mmcblk0p*

      Now use fdisk and delete all partitions from the SD card:

      $ sudo fdisk /dev/mmcblk0

    • Dump the image on the SD card:

      After preparing the SD card we need to dump the raw image on it:

      $ sudo dd if=MATRICOM_MX2_115_FULL_AUG_09_2013.dd of=/dev/mmcblk0

      Note
      Be very careful with dd! If you specify the wrong device on the of= parameter you may destroy useful data on other devices, even you main hard disk!

    Boot the recovery image

    Now we need to go though these steps to load the recovery image:

    1. Power off the XstreamTec appliance.
    2. Insert the SD card, in the SD slot of the XstreamTec.

    3. Press the reset button. This is somewhat tricky. The reset button is inside the AV port of the XstreamTec so you will need a toothpick to press it.

    4. Power on the appliance, while keeping the reset button pressed. The system will enter the recovery installation procedure. Release the reset button when the recovery starts.

    5. Follow the on-screen instructions to load the Android recovery image. Then reboot into android and see if it boots correctly.

    Install the newer version of the Android system

    Now we need to install a more recent version of Android on the XstreamTec.

    1. Download the newer Android firmware:
      $ cd ~/Downloads
      $ wget http://matriserver.com/files/MX2/firmware/MX2-full-1.1.6-firmware-RC.zip
      
    2. Prepare the SD card:
    • Insert the SD card in your PC and check if it is mounted:

      $ df -hT | grep mmc
      /dev/mmcblk0p1 vfat 3,7G 100M 3,6G 3% /media/user/0E99-207A

    • Delete all previous files from the SD card:

      $ cd /media/user/0E99-207A
      $ rm -fr *

      Note
      Once again make sure you don’t have any data you need on the SD card.

    1. Load the firmware on the SD card.

      Unzip the contents of the downloaded zip file (MX2-full-1.1.6-firmware-RC.zip) on the SD card:

      $ cd /media/user/0E99-207A
      $ unzip ~/Downloads/MX2-full-1.1.6-firmware-RC.zip
      

    Boot the new Android firmware

    Go through these steps agaon to load the new Android firmware:

    1. Power off the XstreamTec appliance.
    2. Insert the SD card in the SD slot of the XstreamTec.

    3. Press the reset button. Repeat the toothpick procedure as previously.

    4. Power on the appliance while keeping the reset button pressed. The system will enter the new Android firmware installation procedure. Release the reset button when the recovery starts.

    5. Follow the on-screen instructions to load the new Android firmware. Then reboot into the new Android and see if it boots correctly.

    One would be tempted to keep the Android system and use XBMC or install Kodi on it. This way you can have Android and XBMC/Kodi on your XstreamTec appliance. Alas the XBMC/Kodi experience for Android is not as good as the one on OpenELEC. Trust me, if you need the full Kodi experience you need to install OpenELEC.

    Install OpenELEC on the XstreamTec HTPC media player

    1. Download the OpenELEC firmware:
      $ cd ~/Downloads
      $ wget https://www.dropbox.com/s/toe4kxi9ivomlmp/OpenELEC-Amlogic.MX2.arm-5.0.3.zip
      

      Update:
      There is a newer version at github.

    2. Prepare the SD card:
    • Insert the SD card in your PC and check if it is mounted:

      $ df -hT | grep mmc
      /dev/mmcblk0p1 vfat 3,7G 100M 3,6G 3% /media/user/0E99-207A

    • Delete all previous files from the SD card:

      $ cd /media/user/0E99-207A
      $ rm -fr *

    1. Load the OpenELEC firmware on the SD card.

      Unzip the contents of the downloaded zip file (OpenELEC-Amlogic.MX2.arm-5.0.3.zip) on the SD card:

      $ cd /media/user/0E99-207A
      $ unzip ~/Downloads/OpenELEC-Amlogic.MX2.arm-5.0.3.zip
      

    Boot the OpenELEC firmware

    Once again repeat these steps to load the OpenELEC firmware:

    1. Power off the XstreamTec appliance.
    2. Insert the SD card in the SD slot of the XstreamTec.

    3. Press the reset button. Repeat the toothpick procedure as previously.

    4. Power on the appliance while keeping the reset button pressed. The system will enter the OpenELEC installation procedure. Release the reset button when the recovery starts.

    5. Follow the on-screen instructions to load the OpenELEC firmware. Then reboot into your freshly installed system and see if it works properly.

    Enjoy your Kodi experience!

    References

    • http://kodi.tv/
    • http://openelec.tv/
    Categories
    Linux

    Setup a new Ubuntu or Debian system on the Utilite

    This is a guide to setup a recent Ubuntu system on a Utilite ARM computer, to replace the official Ubuntu 12.04 OS.

    Prerequisites

    • You need to prepare a [kernel with cgroups](/?p=1073 "Building a new Linux kernel for your Utilite Computer") support.
    • An ARM bootable microSD or USB. You can use the official Ubuntu 12.04 [Linux Utilite Image](http://www.compulab.co.il/utilite-computer/wiki/index.php/Utilite_Linux_Image "Linux Utilite Image") on the removable device.
    • Backup any data you have on the original system!

    Preparations

    1. Boot from microSD or USB.
    2. Update the system and install debootstrap:

      $ sudo apt-get update
      $ sudo apt-get install debootstrap
      

    3. Mount the SATA root filesystem on /mnt:
      $ sudo mount /dev/sda2 /mnt
      
    4. Clear the filesystem:

      Note
      Make sure you save any data you need, before doing this!.

      $ sudo rm -fr /mnt/*
      

    Installation of an Ubuntu or Debian base system

    1. Use debootstrap to install the Ubuntu base system:
      $ sudo debootstrap --foreign --arch=armhf trusty /mnt http://ports.ubuntu.com/
      

      You can use vivid instead of trusty if you are feeling adventurous.

      For a Debian system use this command instead:

      $ sudo debootstrap --foreign --arch armhf jessie /mnt http://ftp.uk.debian.org/debian
      

      You can use stretch or sid instead of jessie if you are feeling lucky.

    2. Chroot into the new system:

      $ sudo mount -o bind /dev /mnt/dev
      $ sudo mount -o bind /dev/pts /mnt/dev/pts
      $ sudo mount -t sysfs /sys /mnt/sys
      $ sudo mount -t proc /proc /mnt/proc
      $ sudo cp /proc/mounts /mnt/etc/mtab
      $ sudo chroot /mnt
      

    3. Second stage debootstrap:

      After you enter the chroot jail, we need to complete the second stage of the base setup:

      # /debootstrap/debootstrap --second-stage
      

    Installing additional packages

    1. Setup the correct timezone:
      # dpkg-reconfigure tzdata
      
    2. Add repositories in /etc/apt/sources.list.
    • For Ubuntu:

      deb http://ports.ubuntu.com/ trusty main restricted universe multiverse
      deb http://ports.ubuntu.com/ trusty-security main restricted universe multiverse
      deb http://ports.ubuntu.com/ trusty-updates main restricted universe multiverse
      deb http://ports.ubuntu.com/ trusty-backports main restricted universe multiverse
      
      deb-src http://ports.ubuntu.com/ trusty main restricted universe multiverse
      deb-src http://ports.ubuntu.com/ trusty-security main restricted universe multiverse
      deb-src http://ports.ubuntu.com/ trusty-updates main restricted universe multiverse
      deb-src http://ports.ubuntu.com/ trusty-backports main restricted universe multiverse
      

    • For Debian:
      deb http://ftp.uk.debian.org/debian jessie main contrib non-free
      deb http://ftp.debian.org/debian/ jessie-updates main contrib non-free
      deb http://security.debian.org/ jessie/updates main contrib non-free
      
      deb-src http://ftp.uk.debian.org/debian jessie main contrib non-free
      deb-src http://ftp.debian.org/debian/ jessie-updates main contrib non-free
      deb-src http://security.debian.org/ jessie/updates main contrib non-free
      
    • Run update after adding the repositories:
      # apt-get update
      
    1. Install additional software.
    • We need to create a fake /sbin/initctl to prevent apt from breaking:
      # dpkg-divert --local --rename --add /sbin/initctl
      # ln -s /bin/true /sbin/initctl
      
    • Install the software you need:
      # apt-get -y install language-pack-en ssh isc-dhcp-client net-tools man lsof less
      
    • Remove the fake /sbin/initctl:
      # rm /sbin/initctl
      # dpkg-divert --local --rename --remove /sbin/initctl
      
    1. Setup users:
      # passwd root
      # useradd -m -s /bin/bash utilite
      # passwd utilite
      # usermod -a -G adm,cdrom,sudo,dip,plugdev utilite
      
    2. Setup a kernel:

      Now you need to deploy the [kernel we prepare earlier](/?p=1073#toc-deploy-the-tarball "Building a new Linux kernel for your Utilite Computer") and reboot.

    Choose a role for your system

    1. After you reboot you may find that networking is not working. If you are in a dhcp enabled network run dhclient to get an IP address:

      # dhclient
      

    2. Upgrade the system to the latest upgrades:
      # apt-get update
      # apt-get -y dist-upgrade
      
    3. Check the available roles
    • On Ubuntu you may want to install tasksel (already installed on Debian):

      # apt-get install tasksel

    • See the available roles on Ubuntu:

      # tasksel --list-tasks
      i server Basic Ubuntu server
      i openssh-server OpenSSH server
      u dns-server DNS server
      u lamp-server LAMP server
      u mail-server Mail server
      u postgresql-server PostgreSQL database
      u print-server Print server
      u samba-server Samba file server
      u tomcat-server Tomcat Java server
      u cloud-image Ubuntu Cloud Image (instance)
      u virt-host Virtual Machine host
      u ubuntustudio-graphics 2D/3D creation and editing suite
      u ubuntustudio-audio Audio recording and editing suite
      u edubuntu-desktop-gnome Edubuntu desktop
      u kubuntu-active Kubuntu Active
      u kubuntu-desktop Kubuntu desktop
      u kubuntu-full Kubuntu full
      u ubuntustudio-font-meta Large selection of font packages
      u lubuntu-desktop Lubuntu Desktop
      u lubuntu-core Lubuntu minimal installation
      u mythbuntu-frontend Mythbuntu frontend
      u mythbuntu-backend-master Mythbuntu master backend
      u mythbuntu-backend-slave Mythbuntu slave backend
      u ubuntustudio-photography Photograph touchup and editing suite
      u ubuntustudio-publishing Publishing applications
      u ubuntu-gnome-desktop Ubuntu GNOME desktop
      u ubuntu-desktop Ubuntu desktop
      u ubuntu-usb Ubuntu desktop USB
      u ubuntustudio-video Video creation and editing suite
      u xubuntu-desktop Xubuntu desktop
      u edubuntu-dvd-live Edubuntu live DVD
      u kubuntu-active-live Kubuntu Active Remix live CD
      u kubuntu-live Kubuntu live CD
      u kubuntu-dvd-live Kubuntu live DVD
      u lubuntu-live Lubuntu live CD
      u ubuntu-gnome-live Ubuntu GNOME live CD
      u ubuntustudio-dvd-live Ubuntu Studio live DVD
      u ubuntu-live Ubuntu live CD
      u ubuntu-usb-live Ubuntu live USB
      u xubuntu-live Xubuntu live CD
      u manual Manual package selection

    • See the available roles on debian:

      $ tasksel --list-tasks
      i desktop   Debian desktop environment
      u gnome-desktop GNOME
      u xfce-desktop  Xfce
      u kde-desktop   KDE
      u cinnamon-desktop  Cinnamon
      u mate-desktop  MATE
      i lxde-desktop  LXDE
      u web-server    web server
      u print-server  print server
      u ssh-server    SSH server
      u laptop    laptop
      

    1. Choose a role.

      If you would like a Desktop system, I suggest you choose something light like lubuntu-desktop or lxde-desktop.

    • Installing Lubuntu (Ubuntu):

      # tasksel --task-packages lubuntu-desktop | xargs apt-get -y install
      

      I prefer this method rather than the [TUI](https://en.wikipedia.org/wiki/Text-based_user_interface "Text-based User Interface") of tasksel, so as to have better overview and control of the process. Keep an eye on it every now and then, because it will ask you about keyboard-configuration and other questions. Select the default settings if you are unsure.

    • Installing LXDE Desktop (Debian):

      # apt-get install task-lxde-desktop network-manager
      

      For Desktop systems it may be a good idea to install network-manager as well.

    • Add the vivante GPU module on startup:

      # echo vivante >> /etc/modules
      

    After you finish you can reboot your system and start playing.

    References

    • https://github.com/umiddelb/armhf/wiki/Installing-Ubuntu-14.04-on-the-utilite-computer-from-scratch
    • https://wiki.debian.org/InstallingDebianOn/CompuLab/PC-Utilite/wheezy
    Categories
    Uncategorized

    Building a new Linux kernel for your Utilite Computer.

    The Utilite computer comes with an aging Ubuntu 12.04. If you try to upgrade it to to 14.04 or later, you will end up with a broken system because of their dependency, either fully or partially, on the systemd init system. The problem is that the stock kernel does not support cgroups which is a mandatory dependency for systemd. Type these commands on your Utilite, to verify the missing feature:

    $ sudo mount /dev/sda1 /boot
    $ /usr/src/linux-kernel/scripts/extract-ikconfig /boot/uImage-cm-fx6 | grep CGROUPS
    # CONFIG_CGROUPS is not set
    

    There is also another important feature: fhandle. A systemd Linux system without an fhandle enabled kernel, will fail to enable the swap space during boot.

    Most modern distros, with a few notable exceptions, have switched to this new initialization system. So if we want to use a recent version of Ubuntu, Debian or most other distros, we need to recompile the kernel with cgroups support.

    Prerequisites

    • A standard x86-based PC with Linux installed (I use Ubuntu 15.04 at present). This is going to be our build station.
    • A Utilite ARM Computer. This is our target.

    Prepare the cross-compile toolchain

    The necessary packages are available on recent Ubuntu releases and Debian unstable.

    First let’s install all necessary and optional packages on our build station:

    $ sudo apt-get -y install build-essential gcc-arm-none-eabi gcc-arm-linux-gnueabi gcc-arm-linux-gnueabihf libncurses5 libncurses5-dev libncursesw5 libncursesw5-dev lzop git u-boot-tools pkg-config
    

    Download the kernel sources

    1. You can download the sources from here:
      $ wget https://github.com/utilite-computer/linux-kernel-3.0/archive/master.tar.gz
      
    2. Then extract the tarball and change into the kernel source root:
      $ tar xvzf master.tar.gz
      $ cd linux-kernel-3.0-master/
      

    Building the kernel

    1. Prepare the environment:
      $ export ARCH=arm
      $ export CROSS_COMPILE=arm-linux-gnueabihf-
      
    2. Prepare the configuration for utilite:
      $ make utilite_defconfig
      
    • Check if cgroups and fhandle are enabled:
      $ grep CGROUPS .config
      # CONFIG_CGROUPS is not set
      $ grep FHANDLE .config
      # CONFIG_FHANDLE is not set
      
    • Since cgroups and fhandle, are not enabled we need to enter menuconfig to enable it:
      $ make menuconfig
      

      Navigate to General Setup and go down to Control Group support. Press the space bar and you will see an asterisk ([*]). That means the Control Group support feature is enabled to be compiled in the kernel. Now do the same with the open by fhandle syscalls option. Press exit and Yes to save your new configuration.

    • Make sure cgroups and fhandle are now enabled:

      $ grep CGROUPS .config
      CONFIG_CGROUPS=y
      $ grep FHANDLE .config
      CONFIG_FHANDLE=y
      

      Good. Let’s proceed with the build.

    1. Build the kernel and prepare a U-boot compatible image (uImage):

      $ make
      $ make uImage
      

      This will take some time depending on how powerful your build station is. Go for coffee, watch some movie, read a comic

    Prepare the tarball

    Now we need to package the kernel and modules together.

    1. Install the modules in the rootfs directory:
      $ mkdir -p rootfs/boot
      $ INSTALL_MOD_PATH=./rootfs make modules_install
      

      Ignore this error after make modules_install:

      make[1]: *** No rule to make target 'rootfs/lib/firmware/./', needed by 'rootfs/lib/firmware/ti_3410.fw'.  Stop.
      Makefile:1130: recipe for target '_modinst_post' failed
      make: *** [_modinst_post] Error 2
      
    2. Copy the image and prepare the tarball:
      $ cp arch/arm/boot/uImage rootfs/boot/uImage-cm-fx6
      $ cp arch/arm/boot/zImage rootfs/boot/zImage-cm-fx6
      $ cd rootfs
      $ tar cvzf ../linux-utilite-kernel-3.0.tar.gz .
      $ cd ..
      

    Deploy the tarball

    Now we are ready to deploy the kernel to the Utilite computer.

    1. Copy the kernel to the Utilite computer:
      $ scp linux-utilite-kernel-3.0.tar.gz utilite@utilite-desktop:
      
    2. Now connect to the Utilite and mount the boot partition:
      $ sudo mount /dev/sda1 /boot
      
    3. Backup the existing files and remove the old image and modules:
      $ tar cvzf linux-utilite-original-kernel.tar.gz /boot/ /lib/modules/3.0.35-cm-fx6-6.3/
      $ sudo rm /boot/uImage-cm-fx6
      $ sudo rm -fr /lib/modules/3.0.35-cm-fx6-6.3
      

      If something goes wrong you can boot from a microSD or a USB drive and restore the original kernel.

  • Deploy the kernel:

    Note
    Make sure the /boot directory is mounted on the /dev/sda1 filesystem before running the following command.

    $ sudo tar xvzf linux-utilite-kernel-3.0.tar.gz -C /
    $ sudo chown -R root:root /lib/modules/3.0.35-cm-fx6-6.4
    $ sudo depmod 3.0.35-cm-fx6-6.4
    

    The errors caused by tar are produced because FAT filesystem like /dev/sda1, do not support the ownership attribute. You can safely ignore them. chown is used to set the owneship of the modules to the root user, since the tarball was packaged under a normal user account and inherited its permissions.

  • You can now reboot the Utilite and see if everything works as expected. If not, you can boot with a microSD or a USB flash drive and restore the original kernel and modules.

    After we have successfully prepared a cgroups enabled kernel, we can move on to setup a new Ubuntu or Debian system on the Utilite.

    References

    • [1] http://www.compulab.co.il/utilite-computer/wiki/index.php/Utilite_Linux_Kernel_3.0
    Categories
    Linux

    Shrink a Linux partition to create Swap space

    I have mentioned in an earlier post that I am an owner of a Utilite ARM computer. One of the issues I experienced with the default setup, is that there is no swap partition. That causes Firefox some memory hungry applications to perform badly.

    Check for yourself, if swap is activated on your machine:

    $ free
                total       used       free     shared    buffers     cached
    Mem:       2006476     263752    1742724      20100      21428     107388
    -/+ buffers/cache:     134936    1871540
    Swap:            0          0          0
    

    As you can see in the output, no swap space is used. So we are going to see how we can resize a linux ext2/ext3/ext4 partition to make space for a swap partition. We are then going to create a new swap partition and activate it on boot.

    Prerequisites

    Checking the current setup

    After you boot using a removable medium, you can follow this procedure to check the current setup:

    $ sudo fdisk -l /dev/sda
    
    Disk /dev/sda: 29.8 GiB, 32017047552 bytes, 62533296 sectors
    Units: sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disklabel type: dos
    Disk identifier: 0xa5161f73
    
    Device     Boot  Start      End  Sectors  Size Id Type
    /dev/sda1  *      2048   206847   204800  100M  c W95 FAT32 (LBA)
    /dev/sda2       206848 62533295 62326448 29.7G 83 Linux
    
    • START_SECTOR = 206848
    • END_SECTOR = 62533295
    • TOTAL_SECTORS = 62326448

    As you can see, the main root partition (sda2) has a size of 29.7GB. We will reduce it to 28GB and you the rest of the space (1.7GB) for swap. That should be more than enough.

    Now in order to avoid tedious calculations, and make sure the filesystem on sda2 takes all available space on the partition we are going to use this trick:

    • Resize the sda2 filesystem to 27GB.
    • Resize the sda2 partition to 28GB.
    • Resize the sda2 filesystem to all available space of the sda2 partition (28GB).

    Temporarily resize sda2

    1. First force check the filesystem for errors:
      $ sudo e2fsck -f /dev/sda2
      

      If it finishes without errors move to the next step

    2. Resize the filesystem to 27GB:

      $ sudo resize2fs /dev/sda2 27G
      

    3. Verify the result:
      $ sudo dumpe2fs  /dev/sda2 | grep Block | head -2
      dumpe2fs 1.42.12 (29-Aug-2014)
      Block count:              7077888
      Block size:               4096
      
      $ echo '(7077888 * 4096)/1024/1024/1024' | bc -l
      27.00000000000000000000
      

      Exactly 27GB.

    Resize the sda2 partition

    To resize the sda2 partition we will delete it temporarily and recreate it using the same START_SECTOR but a different END_SECTOR.

    1. To calculate the new end sector you can use this formula:

      NEW_END_SECTOR = START_SECTOR + TOTAL_SECTORS * 28 / 29.7

      $ echo "206848+62326448*28/29.7" | bc -l
      58965788.87542087542087542087
      

      We can round up the NEW_END_SECTOR to 58965789.

  • Use fdisk to resize the partition:

    • Delete the sda2 partition:

      $ sudo fdisk /dev/sda
      Changes will remain in memory only, until you decide to write them.
      Be careful before using the write command.
      
        Command (m for help): d
           Partition number (1,2, default 2): 2
      
           Partition 2 has been deleted.
      

      Verify that sda2 is deleted:

      Command (m for help): p
      Disk /dev/sda: 29,8 GiB, 32017047552 bytes, 62533296 sectors
      Units: sectors of 1 * 512 = 512 bytes
      Sector size (logical/physical): 512 bytes / 512 bytes
      I/O size (minimum/optimal): 512 bytes / 512 bytes
      Disklabel type: dos
      Disk identifier: 0xa5161f73
      
      Device     Boot Start    End Sectors  Size Id Type
      /dev/sda1  *     2048 206847  204800  100M  c W95 FAT32 (LBA)
      

      It is deleted, but not really until you issue the write command.

    • Create a smaller sda2 partition:

      Command (m for help): n
      Partition type
         p   primary (1 primary, 0 extended, 3 free)
         e   extended (container for logical partitions)
      Select (default p): p
      Partition number (2-4, default 2): 
      First sector (206848-62533295, default 206848): 
      Last sector, +sectors or +size{K,M,G,T,P} (206848-62533295, default 62533295): 58965789
      
      Created a new partition 2 of type 'Linux' and of size 28 GiB.      
      

      For the end sector we use the value of the NEW_END_SECTOR we have calculated earlier. Now let’s verify the new partition:

      Command (m for help): p
      Disk /dev/sda: 29,8 GiB, 32017047552 bytes, 62533296 sectors
      Units: sectors of 1 * 512 = 512 bytes
      Sector size (logical/physical): 512 bytes / 512 bytes
      I/O size (minimum/optimal): 512 bytes / 512 bytes
      Disklabel type: dos
      Disk identifier: 0xa5161f73
      
      Device     Boot  Start      End  Sectors  Size Id Type
      /dev/sda1  *      2048   206847   204800  100M  c W95 FAT32 (LBA)
      /dev/sda2       206848 58965789 58758942   28G 83 Linux
      

    • Create a new partition of type swap:
      Command (m for help): n
      Partition type
         p   primary (2 primary, 0 extended, 2 free)
         e   extended (container for logical partitions)
      Select (default p): p
      Partition number (3,4, default 3): 
      First sector (58965790-62533295, default 58966016): 
      Last sector, +sectors or +size{K,M,G,T,P} (58966016-62533295, default 62533295):
      
      Created a new partition 3 of type 'Linux' and of size 1,7 GiB.
      
      Command (m for help): t
      Partition number (1-3, default 3): 3
      Partition type (type L to list all types): 82
      
      Changed type of partition 'Linux' to 'Linux swap / Solaris'.
      

      Verify that the new partition is created:

      Command (m for help): p
      Disk /dev/sda: 29,8 GiB, 32017047552 bytes, 62533296 sectors
      Units: sectors of 1 * 512 = 512 bytes
      Sector size (logical/physical): 512 bytes / 512 bytes
      I/O size (minimum/optimal): 512 bytes / 512 bytes
      Disklabel type: dos
      Disk identifier: 0xa5161f73
      
      Device     Boot    Start      End  Sectors  Size Id Type
      /dev/sda1  *        2048   206847   204800  100M  c W95 FAT32 (LBA)
      /dev/sda2         206848 58965789 58758942   28G 83 Linux
      /dev/sda3       58966016 62533295  3567280  1,7G 82 Linux swap / Solaris
      

      Seems OK.

    • Write changes and exit:

      Command (m for help): w
      The partition table has been altered.
      Calling ioctl() to re-read partition table.
      

      Remember that nothing is changed unless you issue the write command. If you make a mistake, just hit Ctrl+c and no harm done.

    1. Resize the filesystem to the full size of the partition:
    • Force check the filesystem

      $ sudo e2fsck -f /dev/sda2
      

    • Resize the filesystem to occupy all the available space in the partition:
      $ sudo resize2fs /dev/sda2
      

      Running resize2fs without a size definition, extends the size of the filesystem to the size of the partition.

    • Verify the new filesystem size:

      $ sudo dumpe2fs  /dev/sda2 | grep Block | head -2
      dumpe2fs 1.42.13 (17-May-2015)
      Block count:              7344867
      Block size:               4096
      

      The size in bytes is BLOCK_COUNT * BLOCK_SIZE i.e. 7344867 * 4096 = 30084575232. The size in GB is:

      $ echo '(7344867 * 4096)/1024/1024/1024' | bc -l
      28.01844406127929687500
      

    Restart into your internal drive

    At this point we need to restart into the Operating System installed on the internal SATA device and check if everything works:

    $ sudo fdisk -l /dev/sda
    Disk /dev/sda: 29,8 GiB, 32017047552 bytes, 62533296 sectors
    Units: sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disklabel type: dos
    Disk identifier: 0xa5161f73
    
    Device     Boot    Start      End  Sectors  Size Id Type
    /dev/sda1  *        2048   206847   204800  100M  c W95 FAT32 (LBA)
    /dev/sda2         206848 58965789 58758942   28G 83 Linux
    /dev/sda3       58966016 62533295  3567280  1,7G 82 Linux swap / Solaris
    

    The partition table appears to be correct. But we haven’t formatted the swap partition yet:

    $ free
                total       used       free     shared    buffers     cached
    Mem:       2006476     264380    1742096      20100      21568     107800
    -/+ buffers/cache:     135012    1871464
    Swap:            0          0          0
    

    We need to format the swap partition before we can use it.

    Setup the swap partition

    1. Format the swap partition:
      $ sudo mkswap /dev/sda3
      mkswap: /dev/sda3: warning: wiping old swap signature.
      Setting up swapspace version 1, size = 1,7 GiB (1826443264 bytes)
      no label, UUID=fa1e99ff-a9ab-4fd7-ba81-e5020f4e604b
      

      Take a note of the UUID: fa1e99ff-a9ab-4fd7-ba81-e5020f4e604b

    2. Activate swap:

      $ sudo swapon /dev/sda3
      

    3. Verify swap (in human readable form this time):
      $ free -m
                  total       used       free     shared    buffers     cached
      Mem:          1959        259       1700         19         21        105
      -/+ buffers/cache:        132       1826
      Swap:         1741          0       1741
      

      So we have a total of 1741MB of swap space.

    4. Setup persistent swap.

      To have swap activated persistently across reboots, we need to set it up in /etc/fstab:

      # echo 'UUID="fa1e99ff-a9ab-4fd7-ba81-e5020f4e604b" none swap defaults 0 0' >> /etc/fstab
      

      You need to run the above command as root.

    Now after you reboot you can check with free to see if the swap space is activated.

    Categories
    Linux

    Creating a new Debian package using pbuilder: gnujump

    Yet another practical packaging session, this ine on how to build a Debian package from scratch. This one too is based on the packaging tutorial and practical sessions of Lucas Nussbaum.

    We are going to download the source code of gnujump from upstream, and use the pbuilder tool to package it. The advantages of pbuider is that it creates a minimal chroot jail that helps you track any dependencies you might forgot to specify in debian/control. Also it gives you the opportunity to build Debian packages on Ubuntu and vice-versa. You can also build packages for releases other than your own.

    Prerequisites

    • A recent Debian or Ubuntu system.

    Preparation of the pbuilder jail

    Install pbuilder and relevant packages:

    $ sudo apt-get -y install pbuilder debootstrap devscripts packaging-dev debian-keyring ubuntu-archive-keyring
    

    NOTE: use ubuntu-keyring instead of ubuntu-archive-keyring if your build station is Ubuntu

    Prepare the target environment. For example for Ubuntu trusty use can use this setup:

    $ sudo pbuilder create --debootstrapopts --variant=buildd --mirror http://cy.archive.ubuntu.com/ubuntu --distribution trusty --architecture amd64 --components main --debbuildopts -mJohn Doe <john.doe@example.net>
    

    Prepare the new package

    1. Download the source:
      $ wget https://ftp.gnu.org/gnu/gnujump/gnujump-1.0.8.tar.gz
      
    2. Extract the archive and change into the source tree:
      $ tar xvzf gnujump-1.0.8.tar.gz
      $ cd gnujump-1.0.8/
      

    Prepare for debian packaging

    1. We are going to use the dh_make utility to prepare the debian directory and all the necessary files:
      $ DEBFULLNAME="John Doe" DEBEMAIL="john.doe@example.net" dh_make -s -y --createorig
      

      Check if all necessary files are there:

      $ find debian/
      debian/
      debian/compat
       debian/manpage.1.ex
      debian/preinst.ex
      debian/rules
      debian/gnujump.default.ex
      debian/README.Debian
      debian/copyright
      debian/gnujump.cron.d.ex
      debian/gnujump.doc-base.EX
      debian/changelog
      debian/README.source
      debian/control
      debian/menu.ex
      debian/manpage.sgml.ex
      debian/docs
      debian/init.d.ex
      debian/source
      debian/source/format
      debian/watch.ex
      debian/postrm.ex
      debian/prerm.ex
      debian/manpage.xml.ex
      debian/postinst.ex
      

      Now Check the contents of the debian/changelog, debian/rules and debian/control files.

    2. We will need to make some changes in the debian/control file:

      Source: gnujump
      Section: games
      Priority: optional
      Maintainer: John Doe <john .doe@example.net>
      Build-Depends: debhelper (>= 9), autotools-dev, libsdl1.2-dev, libsdl-image1.2-dev, libsdl-mixer1.2-dev
      Standards-Version: 3.9.5
      Homepage: http://gnujump.es.gnu.org
      ...
      

      The libsdl1.2-dev, libsdl-image1.2-dev and libsdl-mixer1.2-dev have been discovered by repeatedly compiling and failing until you get it right. The you can use the apt-cache search and apt-file search commands to discover the packages corresponding to the missing dependencies.

    3. Edit the debian/rules file to look like this:

      DH_VERBOSE = 1
      
      DPKG_EXPORT_BUILDFLAGS = 1
      include /usr/share/dpkg/default.mk
      
      export DEB_BUILD_MAINT_OPTIONS = hardening=+all
      
      export DEB_CFLAGS_MAINT_APPEND  = -Wall -pedantic
      export DEB_LDFLAGS_MAINT_APPEND = -Wl,--as-needed
      
      %:
              dh $@  --with autotools-dev
      
      CFLAGS += -lm
      

    4. Edit the debian/changelog, with dch -e, to look like this:
      gnujump (1.0.8-1) unstable; urgency=low
      
        * Initial release
      
       -- John Doe</john><john .doe@example.net>  Mon, 22 Jun 2015 17:41:22 +0300
      

    Build the package

    Run this command from the source tree:

    $ pdebuild
    

    Give your password when asked. pdebuild will download and install all dependencies in the pbuilder jail and then build the package. Changes will be lost from the jail, the next time you use it but if all goes well, you should see these files under /var/cache/pbuilder/result:

    $ ls -la /var/cache/pbuilder/result/
    total 5776
    drwxr-xr-x 2 root      root         4096 Ιούν 22 17:52 .
    drwxr-xr-x 9 root      root         4096 Ιούν 22 11:26 ..
    -rw-r--r-- 1 john john    1449 Ιούν 22 17:52 gnujump_1.0.8-1_amd64.changes
    -rw-r--r-- 1 john john 1560574 Ιούν 22 17:52 <strong>gnujump_1.0.8-1_amd64.deb</strong>
    -rw-rw-r-- 1 john john    9228 Ιούν 22 17:52 gnujump_1.0.8-1.debian.tar.xz
    -rw-rw-r-- 1 john john     885 Ιούν 22 17:52 gnujump_1.0.8-1.dsc
    -rw-rw-r-- 1 john john 2508641 Ιούλ 24  2012 gnujump_1.0.8.orig.tar.gz
    -rw-rw-r-- 1 john john 1814056 Ιούν 22 17:41 gnujump_1.0.8.orig.tar.xz
    

    Checking the result

    1. View the information related to the package file:
      $ dpkg -I /var/cache/pbuilder/result/gnujump_1.0.8-1_amd64.deb
      new debian package, version 2.0.
      size 1560574 bytes: control archive=5550 bytes.
          419 bytes,    11 lines      control              
        18390 bytes,   233 lines      md5sums              
      Package: gnujump
      Version: 1.0.8-1
      Architecture: amd64
      Maintainer: John Doe </john><john .doe@example.net>
      Installed-Size: 2355
      Depends: libc6 (>= 2.14), libgl1-mesa-glx | libgl1, libsdl-image1.2 (>= 1.2.10), libsdl-mixer1.2, libsdl1.2debian (>= 1.2.11)
      Section: games
      Priority: optional
      Homepage: http://gnujump.es.gnu.org
      Description: <insert up to 60 chars description>
       </insert><insert long description, indented with spaces>
      

    As you can see our work is not finished. We need to add a description in debian/control and rebuild it.

    1. Check the contents of the package file:
      $ dpkg -I /var/cache/pbuilder/result/gnujump_1.0.8-1_amd64.deb
      

    Install the package

    Now we can install the package:

    $ sudo dpkg -i /var/cache/pbuilder/result/gnujump_1.0.8-1_amd64.deb
    

    Now run the gnujump software and check if it works.

    This is not the whole story of course. You have to check the package against Lintian to see if it complies with the Debian Policy. And if it’s not compliant, you may need to use Quilt to patch it. You will need to fill debian/watch so you can track updates on the upstream with uscan. The work of a Debian packager never ends.

    References

    • https://wiki.ubuntu.com/PbuilderHowto
    • https://www.debian.org/doc/manuals/packaging-tutorial/packaging-tutorial.en.pdf</insert></john>
    Categories
    Linux

    Modifying an existing Debian package: grep

    This guide is a practical session on how to modify an existing debian package, when the version is not included in the current release. It is based on the excellent packaging tutorial and practical sessions of Lucas Nussbaum.

    We are going to download a source deb package, build it without modifications, modify it, patch it, rebuild it and in the end, install it in our system.

    Prerequisites

    • A recent Debian or Ubuntu system.

    Installation of development packages

    $ sudo apt-get -y install build-essential debhelper devscripts packaging-dev debian-keyring
    

    Get the source from the debian repos

    $ dget http://cdn.debian.net/debian/pool/main/g/grep/grep_2.21-1.dsc
    

    After that you should see these files under the current directory:

    $ ls
    grep_2.21-1.debian.tar.bz2  grep_2.21-1.dsc  grep_2.21.orig.tar.xz
    
    • grep_2.21-1.dsc: Package’s debian source control file.
    • grep_2.21.orig.tar.xz: Original source.
    • grep_2.21-1.debian.tar.bz2: Debian files and patches.

    Unpack the source

    The dpkg-source file will unpack the original and debian tarballs and apply the debian patches, if available.

    $ dpkg-source -x grep_2.21-1.dsc
    

    Download build dependencies

    $ sudo apt-get -y build-dep grep
    

    Build the unmodified package

    Change into the source tree and run debuild. We are using the -us and -uc flags to suppress warnings about signing.

    $ cd grep-2.21/
    $ debuild -us -uc
    

    It doesn’t take long before this package is build. In the parent directory you will find three more files:

    • grep_2.21-1_amd64.deb: the newly created deb package file.
    • grep_2.21-1_amd64.build: the build log. You can use this to check for errors or warnings.
    • grep_2.21-1_amd64.changes: Debian applied changes.

    Modify the package

    I suggest you do not touch the source, unless you fancy spending your day debugging. We will simply make some modifications on the debian/changelog and debian/rules files.

    1. Add the --with-gnu-ld flag in the DEB_CONFIGURE_EXTRA_FLAGS line of the debian/rules file.

      The --with-gnu-ld is a trivial change and shouldn’t break anything during build. Find this line:

      DEB_CONFIGURE_EXTRA_FLAGS += --without-included-regex
      

      And change it to this:

      DEB_CONFIGURE_EXTRA_FLAGS += --without-included-regex --with-gnu-ld
      
    2. Update the changelog file.

      We are going to use the dch utility which is simply a wrapper around your default editor, with changelog syntax checking. Run the following command in the source tree:

      $ DEBFULLNAME="John Doe" DEBEMAIL="john.doe@example.net" dch -i
      

      Make the following changes (in bold) the debian/changelog file:

      grep (2.21-1ubuntu1) experimental; urgency=low
      
       * Support for GNU ld linker.
      
      -- John Doe <john .doe@example.net>  Fri, 19 Jun 2015 13:01:37 +0300
      

      Note how the name and email of the patcher have been automatically inserted in the changelog entry. That’s because we defined the DEBFULLNAME and DEBEMAIL variables in the shell that runs dch.

    Rebuild the modified package

    $ debuild -us -uc
    

    You will see the following error:

    dpkg-source: error: aborting due to unexpected upstream changes, see /tmp/grep_2.21-1ubuntu1.diff.eYCcPk
    dpkg-source: info: you can integrate the local changes with dpkg-source --commit
    dpkg-buildpackage: error: dpkg-source -b grep-2.21 gave error exit status 2
    debuild: fatal error at line 1376:
    

    The above error happens because the --with-gnu-ld flag makes changes to the upstream source and this is against the Debian policy.

    Apply patches for Policy compliance

    $ cp /tmp/grep_2.21-1ubuntu1.diff.eYCcPk debian/patches/90-enable-gnu-ld.patch
    $ echo 90-enable-gnu-ld.patch >> debian/patches/series
    

    It is considered a good practice to edit the patch, add a description and a short summary and fill the headers related to the [Patch Tagging Guidelines](http://dep.debian.net/deps/dep3/ "Patch Tagging Guidelines").

    This method is only a workaround. The correct way to deal with patches is [Quilt](https://pkg-perl.alioth.debian.org/howto/quilt.html "Quilt Howto").

    Final rebuild of the modified package

    $ debuild -us -uc
    

    You will see five additional files in the parent directory:

    • grep_2.21-1ubuntu1_amd64.deb: the new deb package file.
    • grep_2.21-1ubuntu1.debian.tar.bz2: the new debian directory tarball.
    • grep_2.21-1ubuntu1.dsc: the new debian source control file.
    • grep_2.21-1ubuntu1_amd64.build: the build log.
    • grep_2.21-1ubuntu1_amd64.changes: the changes file.

    Checking the differences between original and new

    The following commands will find the differences in the debian source control and changes files:

    $ diff ../*.changes
    $ diff ../*dsc
    

    Install the new package

    $ sudo debi
    

    Check if grep is the correct version:

    $ grep --version
    grep (GNU grep) <strong>2.21</strong>
    Copyright (C) 2014 Free Software Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law.
    
    Written by Mike Haertel and others, see <http://git.sv.gnu.org/cgit/grep.git/tree/AUTHORS>.
    

    All done. If you have reached so far congratulations! You have successfully rebuilt a Debian package.

    References:

    • http://www.lucas-nussbaum.net/
    • https://www.debian.org/doc/manuals/packaging-tutorial/packaging-tutorial.en.pdf
    • http://dep.debian.net/deps/dep3/</john>