Feed aggregator

Simos Xenitellis: How to install LXD containers on Ubuntu on Scaleway

Planet Ubuntu - Wed, 07/13/2016 - 14:14

Scaleway, a subsidiary of Online.net, does affordable VPSes and baremetal ARM servers. They became rather well-known when they first introduced those ARM servers.

When you install Ubuntu 16.04 on a Scaleway VPS, it requires some specific configuration (compile ZFS as DKMS module) in order to get LXD. In this post, we see those additional steps to get LXD up and running on a Scaleway VPS.

An issue with Scaleway is that they heavily modify the config of the Linux kernel and you do not get the stock Ubuntu kernel when you install Ubuntu 16.04. There is a feature request to get ZFS compiled into the kernel, at https://community.online.net/t/feature-request-zfs-support/2709/3 Most probably it will take some time to get added.

In this post I do not cover the baremetal ARM or the newer x86 dedicated servers; there is an additional error there in trying to use LXD, an error about not being able to create a sparse file.

Creating a VPS on Scaleway

Once we create an account on Scaleway (we also add our SSH public key), we click to create a VC1 server with the default settings.

There are several types of VPS, we select the VC1 which comes with 2 x86 64-bit cores, 2GB memory and 50GB disk space.

Under Security, there is a default policy to disable «SMTP». These are firewall rules drop packets destined to ports 25, 465 and 587. If you intend to use SMTP at a later date, it makes sense to disable this security policy now. Otherwise, once you get your VPS running, it takes about 30+30 minutes of downtime to archive and restart your VPS in order for this change to take effect.

Once you click Create, it takes a couple of minutes for the provisioning, for the kernel to start and then booting of the VPS.

After the creation, the administrative page shows the IP address that we need to connect to the VPS.

Initial package updates and upgrades $ ssh root@163.172.132.19 The authenticity of host '163.172.132.19 (163.172.132.19)' can't be established. ECDSA key fingerprint is SHA256:Z4LMCnXUyuvwO16HI763r4h5+mURBd8/4u2bFPLETes. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '163.172.132.19' (ECDSA) to the list of known hosts. _ ___ ___ __ _| | _____ ____ _ _ _ / __|/ __/ _` | |/ _ \ \ /\ / / _` | | | | \__ \ (_| (_| | | __/\ V V / (_| | |_| | |___/\___\__,_|_|\___| \_/\_/ \__,_|\__, | |___/ Welcome on Ubuntu Xenial (16.04 LTS) (GNU/Linux 4.5.7-std-3 x86_64 ) System information as of: Wed Jul 13 19:46:53 UTC 2016 System load: 0.02 Int IP Address: 10.2.46.19 Memory usage: 0.0% Pub IP Address: 163.172.132.19 Usage on /: 3% Swap usage: 0.0% Local Users: 0 Processes: 83 Image build: 2016-05-20 System uptime: 3 min Disk nbd0: l_ssd 50G Documentation: https://scaleway.com/docs Community: https://community.scaleway.com Image source: https://github.com/scaleway/image-ubuntu The programs included with the Ubuntu system are free software; the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright. Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. root@scw-test:~# apt update Hit:1 http://archive.ubuntu.com/ubuntu xenial InRelease Get:2 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [95.7 kB] Get:3 http://archive.ubuntu.com/ubuntu xenial-security InRelease [94.5 kB] Get:4 http://archive.ubuntu.com/ubuntu xenial/main Translation-en [568 kB] ... Reading package lists... Done Building dependency tree Reading state information... Done 51 packages can be upgraded. Run 'apt list --upgradable' to see them. root@scw-test:~# apt upgrade Reading package lists... Done Building dependency tree Reading state information... Done Calculating upgrade... Done The following NEW packages will be installed: libpython3.5 The following packages will be upgraded: apt apt-utils base-files bash bash-completion bsdutils dh-python gcc-5-base grep init init-system-helpers libapt-inst2.0 libapt-pkg5.0 libblkid1 libboost-iostreams1.58.0 libboost-random1.58.0 libboost-system1.58.0 libboost-thread1.58.0 libexpat1 libfdisk1 libgnutls-openssl27 libgnutls30 libldap-2.4-2 libmount1 libnspr4 libnss3 libnss3-nssdb libpython2.7-minimal libpython2.7-stdlib librados2 librbd1 libsmartcols1 libstdc++6 libsystemd0 libudev1 libuuid1 lsb-base lsb-release mount python2.7 python2.7-minimal systemd systemd-sysv tzdata udev util-linux uuid-runtime vim vim-common vim-runtime wget 51 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. Need to get 27.6 MB of archives. After this operation, 5,069 kB of additional disk space will be used. Do you want to continue? [Y/n] Y Get:1 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 base-files amd64 9.4ubuntu4.1 [68.4 kB] Get:2 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 bash amd64 4.3-14ubuntu1.1 [583 kB] ... Setting up librados2 (10.2.0-0ubuntu0.16.04.2) ... Setting up librbd1 (10.2.0-0ubuntu0.16.04.2) ... Processing triggers for libc-bin (2.23-0ubuntu3) ... root@scw-test:~# Installing ZFS as a DKMS module

There are instructions on how to install ZFS as a DKMS module at https://github.com/scaleway/kernel-tools#how-to-build-a-custom-kernel-module

First, we install the build-essential package,

root@scw-test:~# apt install build-essential

Second, we run the script that is provided at https://github.com/scaleway/kernel-tools#how-to-build-a-custom-kernel-module It takes about a minute for this script to run; it downloads the kernel source and prepares the modules for compilation.

Third, we install the zfsutils-linux package as usual. In this case, it takes more time to install, as it needs to recompile the ZFS modules.

root@scw-test:~# apt install zfsutils-linux

This step takes lots of time. Eight and a half minutes!

Installing the LXD package

The final step is to install the LXD package

root@scw-test:~# apt install lxd Initial configuration of LXD

A VPS at Scaleway does not have access to a separate block device (the dedicated servers do). Therefore, we are creating the ZFS filesystem in a loopback device.

root@scw-test:~# df -h / Filesystem Size Used Avail Use% Mounted on /dev/vda 46G 2.1G 42G 5% /

We have 42GB of free space, therefore let’s allocate 36GB for the ZFS filesystem.

root@scw-test:~# lxd init Name of the storage backend to use (dir or zfs): zfs Create a new ZFS pool (yes/no)? yes Name of the new ZFS pool: mylxd-pool Would you like to use an existing block device (yes/no)? no Size in GB of the new loop device (1GB minimum): 36 Would you like LXD to be available over the network (yes/no)? no Do you want to configure the LXD bridge (yes/no)? yes ...we accept the defaults in creating the LXD bridge... Warning: Stopping lxd.service, but it can still be activated by: lxd.socket LXD has been successfully configured. root@scw-test:~#

 

Create a user to manage LXD

We create a non-root user to manage LXD. It is advised to create such a user and refrain from using root for such tasks.

root@scw-test:~# adduser ubuntu Adding user `ubuntu' ... Adding new group `ubuntu' (1000) ... Adding new user `ubuntu' (1000) with group `ubuntu' ... Creating home directory `/home/ubuntu' ... Copying files from `/etc/skel' ... Enter new UNIX password: ******* Retype new UNIX password: ******* passwd: password updated successfully Changing the user information for ubuntu Enter the new value, or press ENTER for the default Full Name []: Room Number []: Work Phone []: Home Phone []: Other []: Is the information correct? [Y/n] Y root@scw-test:~#

Then, let’s add this user ubuntu to the sudo (ability to run sudo) and lxd (manage LXD containers) groups,

root@scw-test:~# adduser ubuntu sudo # For scaleway. For others, the name might be 'admin'. root@scw-test:~# adduser ubuntu lxd

Finally, let’s restart the VPS. Although it is not necessary, it is a good practice in order to make sure that lxd starts automatically even with ZFS being compiled through DKMS. A shutdown -r now would suffice to restart the VPS. After about 20 seconds, we can ssh again, as the new user ubuntu.

Let’s start up a container

We log in as this new user ubuntu (or, sudo su – ubuntu).

ubuntu@scw-test:~$ lxc launch ubuntu:x mycontainer Creating mycontainer Retrieving image: 100% Starting mycontainer ubuntu@scw-test:~$ lxc list +-------------+---------+------+------+------------+-----------+ | NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS | +-------------+---------+------+------+------------+-----------+ | mycontainer | RUNNING | | | PERSISTENT | 0 | +-------------+---------+------+------+------------+-----------+ ubuntu@scw-test:~$ lxc list +-------------+---------+----------------------+------+------------+-----------+ | NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS | +-------------+---------+----------------------+------+------------+-----------+ | mycontainer | RUNNING | 10.181.132.19 (eth0) | | PERSISTENT | 0 | +-------------+---------+----------------------+------+------------+-----------+ ubuntu@scw-test:~$

We launched an Ubuntu 16.04 LTS (Xenial: “x”) container, and then we listed the details. It takes a few moments for the container to boot up. In the second attempt, the container completed the booting up and also got the IP address.

Forums Council: Ubuntu Forums currently down – 160713

Planet Ubuntu - Wed, 07/13/2016 - 13:13

The Ubuntu Forums are currently down for maintenance. For the last several days they suffered several outages and slow performances. Canonical sysadmins have been doing basic maintenance until the database and the hardware needed intensive care. The forums will be down for some time again, please accept our apologies for the inconvenience. Many thanks to fo0bar who has been with us for over 24h now.

Edit : Forums have been back up around 21:00 UTC.


Ubuntu Insights: A new Skype for Ubuntu… Alpha available now!

Planet Ubuntu - Wed, 07/13/2016 - 07:49

Super exciting news for the Ubuntu and Linux community! News has just been announced that the Alpha version of a new Skype for Linux has been launched!

A brand new WebRTC version of Skype for Linux will be available from today which will allow continued support for the Ubuntu users for years to come. Skype for Linux Alpha is not a fully functioning Skype client yet but will be with users soon. It’s pretty different to the Skype for Linux you use today as the UI is faster and more responsive plus you can share files, photos and videos, and even send a whole new range of new emoticons! Download the app here.

The Alpha version of Skype for Linux uses the next generation of calling architecture, which allows you to call your friends and family on the latest versions of Skype on Windows, Mac, iOS and Android. However, you won’t be able to make or receive calls to and from the previous versions of Skype for Linux (4.3.0.37).

We’re thrilled by this news as Skype reaches over 300 million users around the world! Today’s news shows the strength and importance of the Linux desktop and also Ubuntu as the main player in this field, with 80% share in the Linux desktop market.

Head over to Skype’s Community page to find out more here.

Simos Xenitellis: Trying out LXD containers on Ubuntu on DigitalOcean, with block storage

Planet Ubuntu - Wed, 07/13/2016 - 06:52

We have seen how to try out LXD containers on Ubuntu on DigitalOcean. In this post, we will see how to use the new DigitalOcean block storage support (just out of beta!).

This new block storage has the benefit of being additional separate disk space that should be faster to access. Then, software such as LXD would benefit from this. Without block storage, the ZFS pool for LXD is stored as a loopback file on the ext4 root filesystem. With block storage, the ZFS pool for LXD is stored on the block device of the block storage.

When you start a new droplet, you get by default the ext4 filesystem and you cannot change it easily. Some people managed to hack around this issue, https://github.com/fxlv/docs/blob/master/freebsd/freebsd-with-zfs-digitalocean.md though there are no instructions on how to do with a Linux distribution. The new block storage allows to get ZFS on additional block devices without hacks.

Actually, this block storage feature is so new that even the DigitalOcean page still asks you to request early access.

When you create a VPS, you have now the option to specify additional block storage. The pricing is quite simple, US$0.10 per GB, and you can specify from 1 GB and upwards.

It is also possible to add block storage to an existing VPS. Finally, as shown in the screenshot, block storage is currently available at the NYC1 and SFO2 datacenters.

For our testing, we created an Ubuntu 16.04 $20/month VPS at the SFO2 datacenter. It is a dual-core VPS with 2GB of RAM.

The standard disk is

Disk /dev/vda: 40 GiB, 42949672960 bytes, 83886080 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: gpt Disk identifier: 4CF812E3-1423-1923-B28E-FDD6817901CA Device Start End Sectors Size Type /dev/vda1 2048 83886046 83883999 40G Linux filesystem

While the block device for the block storage is

Disk /dev/sda: 50 GiB, 53687091200 bytes, 104857600 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes

 

Here is how to configure LXD to use the new block device,

root@ubuntu-2gb-sfo2-01:~# lxd init Name of the storage backend to use (dir or zfs): zfs Create a new ZFS pool (yes/no)? yes Name of the new ZFS pool: mylxd-pool Would you like to use an existing block device (yes/no)? yes Path to the existing block device: /dev/sda Would you like LXD to be available over the network (yes/no)? no Do you want to configure the LXD bridge (yes/no)? yes Warning: Stopping lxd.service, but it can still be activated by: lxd.socket LXD has been successfully configured.

Let’s see some benchmarks! We run bonnie++, first on the standard storage, then on the new block storage,

# bonnie -d /tmp/ -s 4G -n 0 -m STANDARDSTORAGE -f -b -u root

Version 1.97 Sequential Output Sequential Input Random
Seeks
Sequential Create Random Create Size Per Char Block Rewrite Per Char Block Num Files Create Read Delete Create Read Delete K/sec % CPU K/sec % CPU K/sec % CPU K/sec % CPU K/sec % CPU /sec % CPU /sec % CPU /sec % CPU /sec % CPU /sec % CPU /sec % CPU /sec % CPU STANDARDSTORAGE 4G 749901 92 611116 80 1200389 76 +++++ +++ Latency 50105us 105ms 7687us 11021us Latency

# bonnie -d /media/blockstorage -s 4G -n 0 -m BLOCKSTORAGE -f -b -u root

Version 1.97 Sequential Output Sequential Input Random
Seeks
Sequential Create Random Create Size Per Char Block Rewrite Per Char Block Num Files Create Read Delete Create Read Delete K/sec % CPU K/sec % CPU K/sec % CPU K/sec % CPU K/sec % CPU /sec % CPU /sec % CPU /sec % CPU /sec % CPU /sec % CPU /sec % CPU /sec % CPU BLOCKSTORAGE 4G 193923 23 96283 14 217073 18 2729 58 Latency 546ms 165ms 8882us 35690us Latency

The immediate benefits are that the latency is much lower with the new block storage, and the CPU usage is also low.

Let’s try with dd,

root@ubuntu-2gb-sfo2-01:~# dd if=/dev/zero of=/tmp/standardstorage.img bs=4M count=1024
1024+0 records in
1024+0 records out
4294967296 bytes (4.3 GB, 4.0 GiB) copied, 4.91043 s, 875 MB/s

root@ubuntu-2gb-sfo2-01:~# dd if=/dev/zero of=/media/blockstorage/blockstorage.img bs=4M count=1024
1024+0 records in
1024+0 records out
4294967296 bytes (4.3 GB, 4.0 GiB) copied, 19.8969 s, 216 MB/s

On the other hand, the standard storage appears four times faster than the new block storage.

I am not sure how these should be interpreted. I look forward to reading other reports about this.

 

 

Ubuntu Insights: Etisalat partners with Canonical to Deploy NFV Telco Infrastructure

Planet Ubuntu - Wed, 07/13/2016 - 03:18

Etisalat, the Middle East’s leading telecoms provider announced today (download PDF) that it has built and launched its first live Network Function Virtualization telco cloud in Abu Dhabi.

The NFV-based telco infrastructure has been built with Quanta servers, Arista switches and Canonical’s Ubuntu OpenStack, a multi-vendor combination integrated for production for the first time ever globally.

Based on open-source OpenStack cloud platforms used by the likes of NASA, CERN and leading web companies, further clouds are in progress in more sites across the UAE.

Realizing the potential and benefits of cloud-based and software-defined technologies, Etisalat launched a corporate-wide program in 2016 to “cloudify the network”, dubbed Sahaab—an Arabic word that translates to ‘cloud’. The program aims to harmonize between the hardware-centric telecom services and the software-centric cloud services across the corporation.

“Etisalat are demonstrating real vision and innovation by the speed at which they are embracing network function virtualisation, “ said Anand Krishnan, EVP, Cloud, Canonical. “We are delighted that Etisalat have selected Canonical OpenStack as their NFV infrastructure and Juju as their generic VNF manager.  As the global leader in OpenStack deployments, we will work closely with Etisalat to  ensure their customers can benefit from the virtualisation and cloudification of network functions that this will deliver.”

The Fridge: Bug 1602344 opened against the CoC for more explicit condemnation of harassment

Planet Ubuntu - Tue, 07/12/2016 - 17:24

I just opened a bug to pave our way to amend the CoC so that it gets to be clearly visible we do not accept harassment (and, by consequence, bullying) in any form, for any reason.

This is the sequence from the discussions previously held in this mailing list, and a session on UOS 1605.

Our view is that a small change will suffice. Michael Hall has added the proposed branch to the bug. This, we expect, will precede additional information added around the CoC (but not directly part of the CoC), so that examples and pointers can be given.

Please comment. We need feedback from the community.

Originally posted to the ubuntu-community-team mailing list on Tue Jul 12 20:10:37 UTC 2016 by C de-Avillez

Bug 1602344 opened against the CoC for more explicit condemnation of harassment

The Fridge - Tue, 07/12/2016 - 17:24

I just opened a bug to pave our way to amend the CoC so that it gets to be clearly visible we do not accept harassment (and, by consequence, bullying) in any form, for any reason.

This is the sequence from the discussions previously held in this mailing list, and a session on UOS 1605.

Our view is that a small change will suffice. Michael Hall has added the proposed branch to the bug. This, we expect, will precede additional information added around the CoC (but not directly part of the CoC), so that examples and pointers can be given.

Please comment. We need feedback from the community.

Originally posted to the ubuntu-community-team mailing list on Tue Jul 12 20:10:37 UTC 2016 by C de-Avillez

Svetlana Belkin: What Programs Do I Use: Irssi

Planet Ubuntu - Tue, 07/12/2016 - 06:29

I use IRC like many people of the Community who uses IRC for most of their text-based business, excluding e-mailing to mailing-list threads or longer pieces of text.  From 2009 to April 2016, I used X-chat.  X-chat was one of the programs that were removed from the main repo of Ubuntu software center.  Simply because it’s not maintained anymore.  I’m aware that there is Hex Chat but it doesn’t work well with my ZNC Bouncer.  Nowadays I use the command-line based IRC client called Irssi.

I haven’t really messed around with the config of the client but I do have set it up to the point where it’s usable.

Like many Open Source programs,  Irssi has scripts to make it more usable and I use a few of them:

  • autoaway.pl: Your basic auto away script.  I have mine set for five (5) minutes.  I usually use the main window, where I can see the away log without going to the channel where I was pinged, and I use “.” then enter to see the log.
  • seen.pl: Useful when you need to know when someone was last on or said something.
  • smartfilter.pl: This script hides the join/part messages, something that I don’t like to see.

The Fridge: Ubuntu Membership Board call for nominations: Call Number Two

Planet Ubuntu - Mon, 07/11/2016 - 17:18

Svetlana Belkin, on the behalf of the Community Council, repeats her call for nominations to the Ubuntu Membership Board.

As a refresher, the following requirements exist for for nominees:

  • be an Ubuntu member (preferably for some time)
  • be confident that you can evaluate contributions to various parts of our community
  • be committed to attending the membership meetings
  • broad insight into the Ubuntu community at large is a plus

Read the email for full details about expectations for members, meeting times and how to apply.

A link to her related blog post which notes that the deadline is extended to July 25th, 2016.

Ubuntu Membership Board call for nominations: Call Number Two

The Fridge - Mon, 07/11/2016 - 17:17

Svetlana Belkin, on the behalf of the Community Council, repeats her call for nominations to the Ubuntu Membership Board.

As a refresher, the following requirements exist for for nominees:

  • be an Ubuntu member (preferably for some time)
  • be confident that you can evaluate contributions to various parts of our community
  • be committed to attending the membership meetings
  • broad insight into the Ubuntu community at large is a plus

Read the email for full details about expectations for members, meeting times and how to apply.

A link to her related blog post which notes that the deadline is extended to July 25th, 2016.

The Fridge: Ubuntu Weekly Newsletter Issue 473

Planet Ubuntu - Mon, 07/11/2016 - 17:03

Welcome to the Ubuntu Weekly Newsletter. This is issue #473 for the week July 4 – 10, 2016, and the full version is available here.

In this issue we cover:

The issue of The Ubuntu Weekly Newsletter is brought to you by:

  • Elizabeth K. Joseph
  • Walter Lapchynski
  • Leonard Viator
  • Simon Quigley
  • Chris Guiver
  • Athul Muralidhar
  • Chris Sirrs
  • And many others

If you have a story idea for the Weekly Newsletter, join the Ubuntu News Team mailing list and submit it. Ideas can also be added to the wiki!

Except where otherwise noted, content in this issue is licensed under a Creative Commons Attribution 3.0 License BY SA Creative Commons License

Ubuntu Weekly Newsletter Issue 473

The Fridge - Mon, 07/11/2016 - 17:03

Welcome to the Ubuntu Weekly Newsletter. This is issue #473 for the week July 4 – 10, 2016, and the full version is available here.

In this issue we cover:

The issue of The Ubuntu Weekly Newsletter is brought to you by:

  • Elizabeth K. Joseph
  • Walter Lapchynski
  • Leonard Viator
  • Simon Quigley
  • Chris Guiver
  • Athul Muralidhar
  • Chris Sirrs
  • And many others

If you have a story idea for the Weekly Newsletter, join the Ubuntu News Team mailing list and submit it. Ideas can also be added to the wiki!

Except where otherwise noted, content in this issue is licensed under a Creative Commons Attribution 3.0 License BY SA Creative Commons License

Svetlana Belkin: The Ubuntu Community Wants YOU to Join the Membership Board!

Planet Ubuntu - Mon, 07/11/2016 - 13:58

NOTE: I know that “join” isn’t the correct word in this sense but it makes sense for the title.  And I also know that I should of posted this when the first call was made but I was just burnt out at that time and I was get ready for a trip to Sydney, NSW.

From the call:

As you may know, Ubuntu Membership is a recognition of significant and sustained contribution to Ubuntu and the Ubuntu community. To this end, the Community Council recruits members of our current membership community for the valuable role of reviewing and evaluating the contributions of potential members to bring them on board or assist with having them achieve this goal.

We have seven members of our boards expiring from their 2 year terms within the next couple months, which means we need to do some restaffing of this Membership Board.

We’re looking for Ubuntu Members who can participate either in the 20:00 UTC meetings or 22:00 UTC (if you can make both, even better).

Both the 20:00 UTC and the 22:00 UTC meetings happen once a month, specific day may be discussed by the board upon addition of new members.

Currently we have seven (7) nominees and we would like to have at least two (2) more.  Because of this, the deadline is extended to July 25th, 2016.

We have the following requirements for nominees:

  • be an Ubuntu member (preferably for some time)
  • be confident that you can evaluate contributions to various parts of our community
  • be committed to attending the membership meetings
  • broad insight into the Ubuntu community at large is a plus

Why?

As a Board member this is why I think it’s worth it to be apart of the Board:

  • It’s a good step for higher level leadership
  • Can help on Community Council elections because you showed that you have/had a position in a  Board
  • It allows you to see the weak points of the applicants and possibly help them on strengthening them
  • It allows you to see what is happening the Community

How?

To nominate yourself or somebody else (please confirm they wish to accept the nomination and state you have done so), please send a mail to the membership boards mailing list (ubuntu-membership-boards at lists.ubuntu.com). You will want to include some information about the nominee, a launchpad profile link and which time slot (20:00 or 22:00) the nominee will be able to participate in.

I wish good luck on those who nominate themselves or others!

Thank you.

Thomas Ward: The Road Ahead for NGINX in Ubuntu

Planet Ubuntu - Mon, 07/11/2016 - 09:46

Hello, everyone! Two blog posts and a flurry of tweets in a day, what the heck has gotten into me?

Some fun things have happened in the last development cycle leading up to Xenial for nginx! Let’s recap a couple of the big things that’re ‘great’ happenings:

  • NGINX 1.9.x was accepted into Xenial during the development process.
  • Later in the dev cycle, we were given the ACK by the Security Team to enable the HTTP/2 module (yay, HTTP/2 support!)
  • Close to the end, that was also updated to 1.10.x post-release to get us onto a Stable version for the duration of the LTS! Yay, an LTS with a Stable version!

All in all, a good dev cycle for getting NGINX into the Ubuntu repositories! Now, we look ahead to the future.

First, a note about Wily. The NGINX PPAs will no longer get any Wily updates, as of today. This close to the End of Life date of Wily, I can’t guarantee there’ll be any updates beyond security-critical ones prompting such updates, given the EOL date of Wily being in a couple weeks.

This means, for the most part, that bugs which are against the Wily package in Ubuntu also get less scrutiny as we focus on the future. Any such Wily-filed bugs will need to be confirmed in another release of an equal or newer version (basically, Xenial or later) before I poke at them or another person pokes at them (this doesn’t prevent the community from submitting patches though). This also means people on Wily boxes who want to get continued NGINX support should upgrade to Xenial because I can’t guarantee they’ll get updates as they wish. And once Wily goes EOL, they get nothing.

Secondly, the road ahead. Up in Debian, they’re starting to test builds against the next OpenSSL version (1.1.0). Unfortunately, NGINX Stable 1.10.x doesn’t build. After poking upstream, I’ve learned there is a fix for this… but for NGINX Mainline… and it won’t be backported to 1.10.x. This is a little bit of a headache, for a couple reasons.

  1. NGINX Stable 1.10.x is not going to be able to be supported at some point in the future in Ubuntu, because it won’t have OpenSSL support.
  2. To get NGINX Mainline as the version in NGINX, I need to merge in the quite-evil Debian ‘dynamic modules’ support.
  3. Further, to get NGINX Mainline into Ubuntu during a development cycle, I need to go and pull in from Debian Experimental, and then build test against the older OpenSSL to make sure nothing dies off.

The big issues of this are mostly that we don’t know the full timeline of OpenSSL 1.1.0 being released in Debian. I have assurances from the Ubuntu Security Team, however, that OpenSSL 1.1.0 will not be included until packages don’t Fail to Build from Source (FTBFS) against it. Which means that I don’t have to act on this immediately.

The additional headache added to this list though is that, while I merge in Dynamic Module Support, it is not 100% ‘supported’ yet in Debian, and it won’t be totally supported in a sane way for packages which ship third-party modules. There has been discussion threads on some third-party modules packaging their modules to work as a dynamic module for Ubuntu Universe / Debian. This is a double-edged sword. Not only do I have to worry about NGINX updates, but I will have to start making sure all the dynamic modules get rebuilt for each upload. I’ll be working to try and find a better solution to this, but this will preclude updates to things getting done at times, given the signature-based approach to dynamic modules that exists currently. We’ll work through this, though, at some point, and make it more supportable in the future.

——

Just wanted to give you all some insights into the future of NGINX, and the headaches I will have to work through, for Ubuntu’s packages going forward.

Daniel Holbach: Snappy Playpen event tomorrow!

Planet Ubuntu - Mon, 07/11/2016 - 09:27

Distributing software has never been easier. snapcraft makes it easy to build any kind of app, snapd and snap-confine bring security and hassle-free updates. Maintaining the app in the store is simple and you get lots of flexibility with different release channels.

If you’re interested or curious, adding your software to the Snappy Playpen, might be a good first step. Tomorrow, Tuesday 12th July 2016, we are working together on getting more snaps landed, getting things improved, updating our docs, helping out the snapd/snapcraft people, and upstreaming snaps.

It’s easy to get in touch, we are both hanging out in

We are looking forward to seeing you there.

Ubuntu Insights: IoT Tech Expo Central Europe

Planet Ubuntu - Mon, 07/11/2016 - 07:57

At IoT Tech Expo Central Europe I was invited to join a panel exploring the future of IoT development. One of the key topics highlighted by the moderator was the need for standardisation in the IoT space.

Granted, the world of IoT is highly fragmented at this point in time… At every level of the stack, from electronic components and their variety local communication protocols, to the multiple home baked Linux used in different appliances, to the score of communication protocols (every vertical has its own) and finally the number of cloud options (from PaaS to DIY). Yes, fragmentation is an issue but where do you start? If we were to start a standardisation effort for each of these areas we would probably still be sitting on a working committee in 10 years. Even worse a lot of these areas are already standardised… and that does not seem to curb fragmentation at all… [list of industrial protocols]

All on the panel agreed though that fragmentation was just a fact of IoT today and something that needed to be embraced. For example, one could complain about the number of programming languages available to IoT developers, yet many adopt a scheme where nodeJS is used for fast prototyping and C for efficient.

So do we really need standardisation? Or do we just need a way to work that limits the number of “wheel reinventions”. A way to work that makes it easy for someone to pick up someone else’s work where it was left. A way to work that makes it possible for someone’s work to be deployed on many different machines without having to build new code all the time…

And if you want to solve this problem, as someone’s pointed out in the audience, “you do not need standardisation you need an Operating System”. Creating a more “standardised” world is exactly what an OS does, separating the complexity of hardware from the world of software. And Open Source is particularly well suited to play this role, allowing faster collaboration among developers across an entire ecosystem.

And the fact is that Linux has been especially successful at building standardisation in the IoT. It is estimated that xx % of IoT deployments today are using Linux. Do we really need more OS led standardisation? Certainly! The world of embedded Linux is far from perfect but already offering a great place to start from. On the week where snaps as universal way of deploying apps on Linux were unveiled… I’d like to tell the world of IoT look at what’s happening and join in!

Thomas Ward: NGINX Mainline PPA: 1.11.2 is being built and released; Ubuntu Wily EOL: No new NGINX versions in PPAs for Wily.

Planet Ubuntu - Mon, 07/11/2016 - 07:37

Been a while since I posted about NGINX on my blog.

Anyways, good news. NGINX 1.11.2 has been uploaded to the staging PPA, and is in the process of being built. If there’s no issues with the builds, then I’ll push the packages to the main Mainline PPA when they’re completed.

NGINX 1.11.2 includes a few new features, but also a bunch of bugfixes:

Changes with nginx 1.11.2 05 Jul 2016 *) Change: now nginx always uses internal MD5 and SHA1 implementations; the --with-md5 and --with-sha1 configure options were canceled. *) Feature: variables support in the stream module. *) Feature: the ngx_stream_map_module. *) Feature: the ngx_stream_return_module. *) Feature: a port can be specified in the "proxy_bind", "fastcgi_bind", "memcached_bind", "scgi_bind", and "uwsgi_bind" directives. *) Feature: now nginx uses the IP_BIND_ADDRESS_NO_PORT socket option when available. *) Bugfix: a segmentation fault might occur in a worker process when using HTTP/2 and the "proxy_request_buffering" directive. *) Bugfix: the "Content-Length" request header line was always added to requests passed to backends, including requests without body, when using HTTP/2. *) Bugfix: "http request count is zero" alerts might appear in logs when using HTTP/2. *) Bugfix: unnecessary buffering might occur when using the "sub_filter" directive; the issue had appeared in 1.9.4.

All in all this is a good thing.

However, for Ubuntu Wily 15.10 server users, who use the Mainline PPA, this is the last update for the Mainline PPA for Ubuntu Wily. Ubuntu Wily goes End of Life on July 28, 2016. This means it is no longer supported by Ubuntu upstream, and will receive no new security updates, bug fix updates, etc. With the EOL date being so close, this is the last upload to the Mainline PPA for Ubuntu Wily. (This also holds true for the Stable PPA – there will be no new Wily updates except for security updates that may happen between now and July 28th)

Daniel Pocock: Let's Encrypt torpedoes cost and maintenance issues for Free RTC

Planet Ubuntu - Mon, 07/11/2016 - 06:34

Many people have now heard of the EFF-backed free certificate authority Let's Encrypt. Not only is it free of charge, it has also introduced a fully automated mechanism for certificate renewals, eliminating a tedious chore that has imposed upon busy sysadmins everywhere for many years.

These two benefits - elimination of cost and elimination of annual maintenance effort - imply that server operators can now deploy certificates for far more services than they would have previously.

The TLS chapter of the RTC Quick Start Guide has been updated with details about Let's Encrypt so anybody installing SIP or XMPP can use Let's Encrypt from the outset.

For example, somebody hosting basic Drupal or Wordpress sites for family, friends and small community organizations can now offer them all full HTTPS encryption, WebRTC, SIP and XMPP without having to explain annual renewal fees or worry about losing time in their evenings and weekends renewing certificates manually.

Even people who were willing to pay for a single certificate for their main web site may have snubbed their nose at the expense and ongoing effort of having certificates for their SMTP mail server, IMAP server, VPN gateway, SIP proxy, XMPP server, WebSocket and TURN servers too. Now they can all have certificates.

Early efforts at SIP were doomed without encryption

In the early days, SIP messages would be transported across the public Internet in UDP datagrams without any encryption. SIP itself wasn't originally designed for NAT and a variety of home routers were created with "NAT helper" algorithms that would detect and modify SIP packets to try and work through NAT. Sadly, in many cases these attempts to help actually clash with each other and lead to further instability. Conversely, many rogue ISPs could easily detect and punish VoIP users by blocking their calls or even cutting their DSL line. Operating SIP over TLS, usually on the HTTPS port (TCP port 443) has been an effective way to quash all of these different issues.

While the example of SIP is one of the most extreme, it helps demonstrate the benefits of making encryption universal to ensure stability and cut out the "man-in-the-middle", regardless of whether he is trying to help or hinder the end user.

Is one certificate enough?

Modern SIP, XMPP and WebRTC require additional services, TURN servers and WebSocket servers. If they are all operated on port 443 then it is necessary to use different hostnames for each of them (e.g. turn.example.org and ws.example.org. Each different hostname requires a certificate. Let's Encrypt can provide those additional certificates too, without additional cost or effort.

The future with Let's Encrypt

The initial version of the Let's Encrypt client, certbot, fully automates the workflow for people using popular web servers such as Apache and nginx. The manual or certonly modes can be used for other services but hopefully certbot will evolve to integrate with many other popular applications too.

Currently, Let's Encrypt only issues certificates to servers running on TCP port 443. This is considered to be a privileged port whereas any port over 1023, including the default ports used by applications such as SIP (5061), XMPP (5222, 5269) and TURN (5349), are not privileged ports. As long as Let's Encrypt maintains this policy, it is necessary to either run a web server for the domain associated with each certificate or run the services themselves on port 443. Running the services themselves on port 443 turns out to be a good idea anyway as it ensures that RTC services can be reached through HTTP proxy servers who fail to let the HTTP CONNECT method access any other ports.

Many configuration tasks are already scripted during the installation of packages on a GNU/Linux distribution (such as Debian or Fedora) or when setting up services using cloud images (for example, in Docker or OpenStack). Due to the heavily standardized nature of Let's Encrypt and the widespread availability of the tools, many of these package installation scripts can be easily adapted to find or create Let's Encrypt certificates on the target system, ensuring every service is running with TLS protection from the minute it goes live.

If you have questions about Let's Encrypt for RTC or want to share your experiences, please come and discuss it on the Free-RTC mailing list.

Ubuntu App Developer Blog: Snapd 2.0.10: new media interfaces, channel switching

Planet Ubuntu - Mon, 07/11/2016 - 05:30

If it hasn't already, snapd 2.0.10 should be making its way to your 16.04 systems. Here is what’s new!

The 2.0.10 release contains a number of improvements and fixes over the 2.0.9 release that was available before. The highlights:

Channels

Channels (stable, candidate, beta, edge) usage has been streamlined on the client.

As a shorthand to --channel=<channel>, you can now use --<channel> with the refresh and install commands.

For example:

Interfaces

New interfaces have landed with this release, giving you more freedom to interact with the OS, while keeping your app into the bounds of the existing confinement. This allows, for example, for improvements in the VLC snap’s user experience.

mpris (new)

  • Allows snaps such as music players to connect to D-Bus as an MPRIS server.
  • You can see an usage example in the VLC snapcraft.yaml.

camera (new)

optical-drive (new)

  • Grants read access to optical drives.

home

  • Allow gvfs shares in home.
General
  • Snaps can be launched under KDE Neon

  • SNAP_COMMON and SNAP_USER_COMMON are paths to unversioned data directories

  • Better handling of removed `snap try` directories

  • Fixes towards running snapd inside LXC

  • `snap change <taskid>` shows task progress

  • Auto-connect the home interface only if running on classic

The changelog is available here and the full details can be found here: https://github.com/snapcore/snapd/tree/2.0.10

Let us know what you think!

We’d like to hear your feedback about snapd and snap technologies. Is there an interface you would need for your app to be working better? Can we do better with integrating with a particular distro? Here’s how we can talk:

Paul Tagliamonte: SNIff

Planet Ubuntu - Sun, 07/10/2016 - 06:34

A while back, I found myself in need of two webservers that would terminate TLS (with different rules). I wanted to run some custom code I’d written (which uses TLS peer authentication), and also nginx on port 443.

The best way I figured out how to do this was to write a tool to sit on port 443, and parse TLS Client Hello packets, and dispatch to the correct backend depending on the SNI name.

SNI, or Server Name Indication allows the client to announce (yes over cleartext!) what server it’s looking for, similar to the HTTP Host header. Sometimes, like in the case above, the Host header won’t work, since you’ve already done a TLS handshake by the time you figure out who they’re looking for.

I also spun the Client Hello parser out into its own importable package, just in case someone else finds themselves in this same boat.

The code’s up on github.com/paultag/sniff!

Pages

Subscribe to Ubuntu Arizona LoCo Team aggregator