Category Archives: Linux

Using Letsencrypt with Wowza Media Server

As part of a work project, I needed to set up Wowza Media Server to do video streaming. As the webapp (which I wrote using the excellent ionic 3 framework) is running under https, it won’t accept video traffic coming from non-encrypted sources. Wowza has some pricey solutions for automatically installing SSL certificates for you, you can also purchase ones however these days I don’t see why everyone doesn’t just use the free and easily automated letsencrypt system. Unfortunately however, letsencrypt doesn’t let you run servers on different ports particularly easily, although it does have some hooks to stop/start services that may already be listening on port 443 (ssl). I happen to be using a redhat/centos distro, although I’m pretty sure the exact same instructions will work on ubuntu and other distros.

Firstly, you need to download the wowza-letsencrypt-converter java program which will convert letsencrypt certificates to the Java format that Wowza can use. Install that prebuild jar under /usr/bin.

Now, create a directory under the Wowza conf directory called ssl and create a file called jksmap.txt (so for example full path is /usr/local/WowzaStreamingEngine/conf/ssl/jksmap.txt) which lists all the domains the Wowza server will be listening on like:

‘secret’ is not actually a placeholder; it’s the password that the wowza-letsencrypt-converter program sets up automatically so keep it as it is.

Configure SSL on the Wowza server by editing the VHost.xml configuration file (find out more about this process in the wowza documentation). Find the 443/SSL section which is commented out by default and change the following sections:

Note the <KeyStorePath>foo</KeyStorePath> line – the value foo is ignored when using jksmap.txt, however if this is empty the server refuses to start or crashes.

Next, install letsencrypt using the instructions on the certbot website.

Once you’ve done all this, run the following command to temporarily stop the server, fetch the certificate, convert it and start the server again:

Then, in order to ensure that the certificate continues to be valid you need to set up a cron entry to run this command daily which will automatically renew the cert when it gets close to its default 3 month expiry time. Simply create /etc/cron.d/wowza-cert-renewal with the following content:

Easily setup a secure FTP server with vsftpd and letsencrypt

I recently had to set up a FTP server for some designers to upload their work (unfortunately they couldn’t use SFTP otherwise it would have been much simpler!). I’ve not had to set up vsftpd for a while, and when I last did it I didn’t much worry about needing to use encryption. So here are some notes on how to set up vsftpd with letsencrypt on ubuntu 14.04 / 16.04 so that only a specific user or two are permitted access.

First, install vsftpd:

Next, you need to make sure you have installed letsencrypt. If not, you can do so using the instructions here – fortunately letsencrypt installation has got a lot easier since my last blog post about letsencrypt almost 2 years ago.

I’m assuming you are running this on the same server as the website, and you’re wanting to set it up as ftp on the same domain or similar subdomain as the website (eg ftp access direct to example.org, or via something like ftp.example.org). If not, you can do a manual install of the certificate but then you will need to redo this every 3 months.

Assuming you’re running the site on apache get the certificate like:

You should now have the necessary certificates in the /etc/letsencrypt/live/example.org/ folder, and your site should be accessible nicely via https.

Now, create a user for FTP using the useradd command. If you want to just create a user that only has access to the server via FTP but not a regular account you can modify the PAM configuration file /etc/pam.d/vsftpd and comment out the following line:

This lets you keep nologin as the shell so the user cannot login normally but can log in via vsftpd’s PAM layer.

Now open up

Because we’re running behind a firewall we want to specify which port range to open up for the connections (as well as port 21 for FTP of course):

If you want to make it even more secure by only allowing users listed in /etc/vsftpd.userlist to be able to log in, add some usernames in that file and then add the following to the /etc/vsftpd.conf configuration file:

You can test using the excellent lftp command:

If the cert is giving errors or is self-signed, you can do the following to connect ignoring them:

Fixing Ubuntu 16.04 massive internal microphone distortion

A while ago I upgrade from Ubuntu 14.10 to 16.04. Afterwards, my laptop’s internal microphone started to become massively distorted to the point that people on the other end of skype or hangouts calls couldn’t understand me at all.

Looking in the ALSA settings I noticed that the “Internal Mic Boost” was constantly being set to 100% and when I dropped this down to 0% everything went well. It seems on my laptop at least to be coupled with the “Mic Boost” which boosts both but without quite so much distortion, ie the “Internal Mic Boost” is a boost on top of the “Mic Boost” which is obviously a problem.

I couldn’t find much detail about how to configure this properly, so after some hacking around I was able to come up with the following solution. Go through every file in /usr/share/pulseaudio/alsa-mixer/paths, look for the section “[Element Internal Mic Boost]” if it is there. You should see a setting under that section like “volume = merge“. Turn that into “volume = off“. To prevent it being changed later when ALSA is updated, you can run:

I’d love to hear if there is a simpler way to work around this issue, but it works for me at least!

Successfully downloading big files from Dropbox via Linux command-line

Recently, someone was trying to send me a 20Gb virtual machine image over dropbox. I tried a couple of times to download using chrome, however it got to 6-8Gb and then came up with a connection error. Clicking on the resume button failed and then removed the file (!). Very strange as I didn’t have any connection issues, but perhaps a route changed somewhere. I saw a number of dropbox users complaining about this on the internet. Obviously there are other approaches such as adding to your own dropbox account and using their local program to do the sync, however because I’m just on a standard free account I couldn’t add in such a large file.

Because I was using btrfs and snapper I still had a version of the half-completed download around, and so I tried seeing if standard linux tools would be able to continue the download where it left off. It turns out that simply using wget -c enables you to resume the download (it dropped a couple of times during the download but just restarting it with the same command let the whole file download just fine. So, to download a large dropbox file even if your internet connection is a bit flakey, simply go to the dropbox download link and then paste it into the terminal (may require the ?dl=1 parameter after it) like:

Apache configuration for WkWebView API service (CORS)

Switching from UIWebView to WKWebView is great, but as it performs stricter CORS checks than standard Cordova/Phonegap it can seem at first that remote API calls are broken in your app.

Basically, before WKWebView does any AJAX request, it first sends a HTTP OPTIONS query and looks at the Access-Control-* headers that are returned to determine if it is able to access the service. Most browsers can be made to allow all AJAX requests via a simple “Access-Control-Allow-Origin: *” header, however WKWebView is more picky. It requires that you expose which methods (GET, POST, etc); and which headers are allowed (eg if you are using JSON AJAX requests you probably need to use a “Content-Type: application/json” header in your main request).

Rather than having to update your API service, you can work around this in a general way using the following Apache config:

Note the last line answers any HTTP OPTIONS request with blank content and returns it straight away. Most API services would cause a lot of CPU processing just to handle a single request whether it is a true request or an OPTIONS query, so we just answer this straight from Apache without bothering to send it through to the API. The R=204 is a trick to specify that we don’t return any content (HTTP 204 code means “Success, but no content”). Otherwise if we used something like R=200 it would return a page talking about internal server error, but with a 200 response which is more bandwidth, more processing and more confusing for any users.

Programming ESP8266 from the CHIP

The CHIP is a powerful $9 computer. I saw them online and ordered 5 of them some time ago as part of a potential home automation project, and because it’s always useful to have some small linux devices around with GPIO ability. I’ve recently been playing a lot with ESP8266 devices (more on this in some future blog posts), and I’ve been using the CHIP to program them via a breadboard and the serial port header connectors (exposed as ttyS0) and esptool.py. So far so good.

However, I want to put the CHIP devices into small boxes around the house and use something like find-lf for internal location tracking based on Wifi signals emitted from phones and other devices to figure out who’s in which room. Whilst the CHIP has 2 wifi devices (wlan0, wlan1) it doesn’t allow one to run in monitor mode while the other is connected to an AP. This means we need an extra Wifi card to be in monitor mode, and as I had a number of ESP8266’s lying around, I thought I’d write a small program to just print MAC and RSSI (signal strength) via the serial port.

As these devices will be in sealed boxes I don’t want to have to go fiddling around with connectors on a breadboard to update the ESP8266 firmware, so I came up with a minimal design to allow reprogramming ESP8266 on-the-fly from CHIP devices (should work on anything with a few GPIO ports). Obviously the ESP8266 does have OTA update functionality, however as these devices will be in monitor mode I can’t use that. As the CHIP works at 3.3v, the same as ESP8266 chips this was pretty straight forwards involving 6 cables and 2 resistors, there were a few steps and gotchas to be aware of first though.

The main issue preventing this from working is that when the CHIP first boots up, the uBoot software listens for input for 2 seconds via ttyS0 (the serial port exposed on the header, not the USB one). When power first comes on, the ESP8266 will always output some bootloader messages via the serial port which means that the CHIP would never boot. Fortunately the processor has a number of different UARTs, a second one that is optionally exposed via the headers. You can read all about the technical details on this thread. In short, to expose the second serial port you need to download this dtb from dropbox and use it to replace /boot/sun5i-r8-chip.dtb. You then need to download this small program to enable the port and run it every boot up. This worked fine for me on the 4.4.13-ntc-mlc kernel. You can then use the pins found listed here to connect to the tx/rx of the ESP8266 serial and it won’t affect the boot-up of the CHIP.

The other nice thing about using ttyS2 rather than ttyS0 is that there are hardware flow control ports exposed (RTS, CTS) which I had hoped could be integrated into esptool to automatically handle the reset. Unfortunately it looks like esptool uses different hardware flow control ports to signal the ESP8266 bootloader mode/reboot so I had to connect these ports to GPIOs and trigger from there.

After doing this, wire the ESP8266 (I’m using the ESP-12 board, but should be the same for any other boards) to the CHIP in the following manner:

ESP8266 pin CHIP connector
VCC 3.3v
Gnd
CH_PD / EN XIO-P6
GPIO0 XIO-P7 via a resistor (eg 3.3k)
GPIO15 – via resistor (eg 3.3k)
TX LCD-D3
RX LCD-D2

Note that on some ESP boards TX/RX are the wrong way round so if you don’t see anything try flipping the cables around.


I then wrote a small program (called restart_esp.py) to trigger different mode reboots of the ESP8266 from the CHIP:

Then you can easily flash your ESP8266 from the CHIP using a command like:

Percent signs in crontab

As this little-known ‘feature’ of cron has now bitten me several times I thought I should write a note about it both so I’m more likely to remember in future, but also so that other people can learn about it. I remember a few years ago when I was working for Webfusion we had some cronjobs to maintain the databases and had some error message that kept popping up that we wanted to remove periodically. We set up a command looking something like:

but it was not executing. Following on from that, today I had some code to automatically create snapshots of a certain btrfs filesystem (however I recommend that for serious snapshotting you use the excellent (if a bit hard to use) snapper tool):

But it was not executing… Looking at the syslog output we see that cron is running a truncated version of it:

Looking in the crontab manual we see:

D’oh. Fortunately the fix is simple:

I’m yet to meet anyone who is using this feature to pipe data into a process run from crontab. I’m also yet to meet even very experienced sysadmins who have noticed this behaviour making this a pretty good interview question for a know-it-all sysadmin candidate!