Introduction

When you want to be sure that the person accessing your website is the right one, the common solution is to implement inside the web application itself an authentication method to let user access to a restricted area. As a result, you have to code a whole authentication and user management module, which can be very complicated (fortunately, many frameworks can do it for you).

As we will see in this article, another way is to delegate the authentication process to the webserver or the reverse proxy. Furthermore, it could also be the opportunity to use a stronger authentication method.

What we want to achieve

Let say that we have a Fedora server on the Internet with NGINX installed. We want to expose the Fedora Webserver Test Page at the address https://nginx.test.

We won’t go into detail to set up such a webserver as this is a very common configuration. Many websites can help for that. We just give below the NGINX configuration file test-page.conf placed in /etc/nginx/conf.d/ :

server {
    listen 443 ssl;
    server_name nginx.test;

    ssl_certificate /etc/pki/tls/certs/nginx.test.crt;
    ssl_certificate_key /etc/pki/tls/private/nginx.test.key;

    location / {
        root /usr/share/nginx/html;
    }
}

What we want to achieve now is to configure NGINX in order to restrict the access of this page to authorized users only.

Certificates verification

When you perform a SSL/TLS connection to a website, your connection is considered secured because your browser has verified that the certificate of the website is legitimate. To keep it simple, the website has proved to a Certificate Authority (CA) that it was the legitimate owner of the domain name. In return, the CA has signed a certificate that the owner can store on the server alongside a protected private key and send it to each client trying to perform an SSL/TLS connection. As the client browser trusts the CA, it also trusts the certificate sent by the website. In other words, the client authenticates the server : we talk about client-side authentication. This is basicaly what you (your browser) do every day while surfing the Internet. This is also what we have implemented for the moment with our Fedora Webserver Test Page.

What we want to achieve is the exact opposite : we want the server to authenticate the user. We call this process server-side authentication. Let see how we can do that with NGINX.

NGINX certificates verification

Create the CA for client certificates

The first thing we have to do is to create a CA in order to be able afterward to create client certificates.

Some articles on the Internet say that this CA must be the same than the one used to sign the SSL/TLS server certificate. Fortunately it is not the case. Indeed, CA are sometimes only used to sign server certificates and can’t be used to sign client certificates. It is for instance the case with Let’s Encrypt, one of the most used CA in the world for SSL/TLS server certificates. Otherwise, any client certificates signed by Let’s Encrypt would be recongnized as legitimate ones, which would not be helpfull to control access to our server. It is then more relevent to decorelate both CA and manage client certificates signature with a private CA.

Precisely, let’s create our private CA :

openssl genrsa -des3 -out root_ca.key 4096
openssl req -x509 -new -sha256 -key root_ca.key -subj "/C=US/ST=NY/O=nginx-test/CN=ca.nginx.test" -days 365 -out root_ca.crt

Then, you can copy the newly created certificate in /etc/pki/tls/certs/ directory.

Register the CA in NGINX

This certificate must now be given to NGINX in order for it to verify that certificates sent by clients belong to our private CA.

You simply have to add the ssl_verify_client and ssl_trusted_certificate directives to the NGINX configuration file test-page.conf placed in /etc/nginx/conf.d/ :

server {
    listen 443 ssl;
    server_name nginx.test;

    ssl_certificate /etc/pki/tls/certs/nginx.test.crt;
    ssl_certificate_key /etc/pki/tls/private/nginx.test.key;

    ssl_verify_client on;
    ssl_trusted_certificate /etc/pki/tls/certs/root_ca.crt;

    location / {
        root /usr/share/nginx/html;
    }
}

Restart NGINX service to take into account this new configuration.

sudo systemctl restart nginx

As explained in NGINX documentation, we have used here ssl_trusted_certificate directive because we don’t want the server to send the CA certificate to the client. If it doesn’t matter for you or if you want to help the client to select the right certificate, you can use instead ssl_client_certificate directive which will at first send the CA certificate to the client.

Creating the client certificates

Last but not least, we have to create the client certificate which will be used to authenticate against the server.

openssl genrsa -out client.key 2048
openssl req -new -sha256 -key client.key -subj "/C=US/ST=NY/O=nginx-test/CN=client.nginx.test" -out client.csr
openssl x509 -req -in client.csr -CA root_ca.crt -CAkey root_ca.key -CAcreateserial -out client.crt -days 365 -sha256

You must now export the client key and certificate in PKCS12 format in order to import them in your browser.

openssl pkcs12 -export -out client.p12 -inkey client.key -in client.crt

You are now able to add the certificate and its key to your browser. For example in Firefox, go to Settings > Privacy and Security > Security > Certificates > View Certificates and click on Import in Your Certificates tab.

Connection to the server

When you try to connect to https://nginx.test with Firefox, a pop-up will now ask you to select a client certificate. Select the right one if you already have imported other certificates.

You will then be able to access the Fedora Webserver Test Page. In case you have not selected the right certificate or if you refuse to give one, a 400 “Bad Request” error will be returned by the webserver.

The benefits of such a configuration

You would tell me why such a configuration is interesting whereas it is so easy and common nowadays to implement an authentication method ?

You would be right if your website is built over a framework like Symfony or Django. However, this is not always the case, especially when you serve a static website like the one in our example. This method let you secure the web access very quickly.

Moreover, when you are building your website, your authentication method could be not implemented yet or not enough configured or secured. This method let you choose who can access the website (for example another developer) while the work is still in progress.

Even if you have implemented and correctly configured an authentication method, this solution can be very helpfull in some case. Let say that a critical vulnerability has been discovered on your website but this one has to be kept online for some users. This method has two benifits in such a scenario :

• the website can still be reached by some user ;
• the website is not anymore directly accessible to potential attackers as the authentication process is delegated to the webserver which is probably not concerned by the vulnerability and which is anyway easier to update.

While it is more and more recommended to implement Multi-Factor Authentication (MFA) to reduce illegitimate accesses via usual and weak authentication method, server-side authentication is a way to do that. Indeed, MFA is the combination of two factors (what you know, what you have, what you are, …) and the method presented in this article can be considered as a second factor (what you have : a certificate) if a password authentication (what you know : a password) has already been implemented on the website.

Finally, the benefit of such a configuration can simply be the fact that the server verifies the client legitimacy to access the website via a strong authentication method, as it is already done nowadays by clients on almost every websites. This is why we also talk about mutual TLS authentication (mTLS). This can be usefull for entity not able to protect the access of one or several web applications behind a VPN but which is able to deliver client certificates.

Conclusion

We have seen in this article that we can easily implement a strong authentication method with a simple webserver configuration and some certificates handling. We have also seen that, even if this method is not so much used on the Internet, server-side authentication has many benefits that improves the security of your website.

Sources

• NGINX does not prompt for client ssl certificate
• Client Side Certificate Auth in Nginx
• Nginx Configuration Tips for Secure Communication: Enabling mTLS and checking client fingerprint

Introduction

When prototyping or developing stuffs, it is often a good idea to use Virtual Machines (VMs) in order to isolate everything and avoid messing up your computer. You are then able to test something and recreate the machines when everything is broken.

However, creating VMs can be painful and repeating this process several times can be very long. Using tools and concepts close to Infrastructure as Code (IaC) to automatically deploy the machines is the right idea and Vagrant is here to help you.

What is Vagrant basically ?

Let’s take an example with a Linux Operating System (OS) and KVM/libvirt as hypervisor. Let say that you simply want to test the new major version of your preferred OS and be sure that every tools you are using are still working. You probably won’t take the risk to completely erase your computer and install directly the OS. The best option could be to use a VM. You will then deploy the machine (via CLI with virsh/virt-install commands or via GUI with Virtual Machine Manager or Cockpit), eventuelly follow the installation process, reboot, install your tools and reconfigure your environment before (finally !) being able to test if everything is working as expected… Very long workflow that you have to reiterate for each major version release !

Another solution could be to describe once for all what you want to do and take this recipe each time you need it. This is basically what Vagrant is made for : you describe almost evrything in one file (called Vagrantfile) and you are done ! It is finally a simple abstraction layer above your hypervisor which takes over VM management and configuration.

We are talking about KVM hypervisor in our example but we could have choosen whichever one. Vagrant has a concept of providers which is basically a plugin that Vagrant can use to communicate with an hypervisor. VirtualBox is supported out of the box but you will have to install a custom provider to let Vagrant manage a KVM hypervisor via libvirt.

First step : install Vagrant

Depending on your OS, there are multiple ways to install Vagrant. You can follow the official documentation or an OS specific documentation. We will show how to do this with Fedora.

If libvirt is not already installed, there is a package group for that (called Virtualization). Vagrant and the libvirt provider is also shipped in another package group (called Vagrant). As a rusult, you can install both with :

sudo dnf group install virtualization vagrant

You should then enable livirtd to automatically start and add your current user to libvirt group in order to avoid password prompting each time you start a VM. These can be done with the following commands :

sudo systemctl enable --now libvirtd
sudo usermod -aG libvirt ${USER}
newgrp libvirt

Quick start : your first Vagrantfile

As already explained, the power of Vagrant is to simply describe in a single file the whole infrastructure you want to create. The syntax uses Ruby language but don’t worry if you know nothing about this language, you basically just have to declare variables. Let’s have a look with these lines thant you can copy and past in a file named Vagrantfile :

Vagrant.configure("2") do |config|
    config.vm.box = "fedora/41-cloud-base"
end

This Vagrantfile simply declares a VM which uses a Fedora 41 Cloud Base image. Juste note that OS images used by Vagrant are called boxes and are basically packages embedding metadata, configuration files and a VM image specifically configured for Vagrant. Hashicorp (the company behind Vagrant) hosts a repository with many boxes publicly available. As each box has a specific version for each provider (KVM/libvirt, VirtualBox, …), be sure that the box you want is compatbile with the provider(s) you want to use.

To launch the VM, the only thing you have to do is to run the following command in the directory where the Vagrantfile is :

vagrant up

Note : For those already running Fedora, the Vagrant version shipped with the distribution uses libvirt daemon session mode. As a result, Vagrant is running in an unprivileged mode and only has the same rights than the current user which is more safer. However, some functionalities (as network operations) require root permissions and Vagrant needs to run in system mode. This can be done by adding a libvirt option as described below :

Vagrant.configure("2") do |config|
    config.vm.box = "fedora/41-cloud-base"
  
    config.vm.provider :libvirt do |libvirt|
        libvirt.qemu_use_session = false
    end
end

Provisioning the machine

Launching VM is interesting but in our example above we talked about making some tests in our new machine. Automatically do stuffs is what provisioners are made for. Once again, you can observe that Vagrant is very modular as there are many different provisioners available as detailed in Vagrant documentation.

We are going to use Ansible provisioner to do stuffs in our VM. We won’t go too much into details on how Ansible works. The only important thing to know about Ansible is that it allows you to discribe in a file (called a playbook) a sequence of tasks that will be run inside the VM.

We first have to install Ansible on the host alongside Vagrant. On Fedora, this can be done with the following command :

sudo dnf install ansible

Just copy and past in a file called playbook.yml the lines below. This file must be in the same directory as the Vagrantfile. This Ansible playbook simply updates the machine.

---
- hosts: all
  gather_facts: true
  tasks:
  - name: Upgrade all packages
    ansible.builtin.dnf:
      name: "*"
      state: latest
    register: update
    become: true
  - name: Reboot to apply updates
    ansible.builtin.reboot:
      post_reboot_delay: 60
      reboot_command: reboot
    when: update.changed
    become: true

You can then update you Vagrantfile to call the playbook once the VM is started for the first time :

Vagrant.configure("2") do |config|
    config.vm.box = "fedora/41-cloud-base"
  
    config.vm.provider :libvirt do |libvirt|
        libvirt.qemu_use_session = false
    end

    config.vm.provision "ansible" do |ansible|
        ansible.playbook = "playbook.yml"
    end
end

If it is not the first time you launch the VM, you can force provisioning with the following command :

vagrant up --provision

Multi-machine infrastructure

The examples above were only launching a single machine. Obviously, you can declare multiple machines that will be launched at the same time. The syntax is slightly different but the idea is the same : you just have to include each machine configuration variables inside a config.vm.define method.

Vagrant.configure("2") do |config|
  
  config.vm.define "vm1" do |hostconfig|
    hostconfig.vm.box = "fedora/41-cloud-base"

    hostconfig.vm.provider :libvirt do |libvirt|
      libvirt.qemu_use_session = false
    end
  end

  config.vm.define "vm2" do |hostconfig|
    hostconfig.vm.box = "fedora/41-cloud-base"

    hostconfig.vm.provider :libvirt do |libvirt|
      libvirt.qemu_use_session = false
    end
  end
end

You can even factor configuration variables that are the same accross machines. This way, the Vagrantfile just above would become :

Vagrant.configure("2") do |config|

  config.vm.box = "fedora/41-cloud-base"

  config.vm.provider :libvirt do |libvirt|
    libvirt.qemu_use_session = false
  end
  
  config.vm.define "vm1" do |hostconfig|
  end

  config.vm.define "vm2" do |hostconfig|
  end
end

Naturaly, it is still possible to provision the VMs. In case we want to use the same playbook to provision each VM, the first approach would be to declare the playbook at config level :

Vagrant.configure("2") do |config|

  config.vm.box = "fedora/41-cloud-base"

  config.vm.provider :libvirt do |libvirt|
    libvirt.qemu_use_session = false
  end

  # Provisionning
  config.vm.provision "ansible" do |ansible|
    ansible.playbook = "playbook.yml"
  end
  
  config.vm.define "vm1" do |hostconfig|
  end

  config.vm.define "vm2" do |hostconfig|
  end
end

This approach is not wrong but, as explained in the Vagrant documentation, Vagrant will provision VM in sequence so that two different playbooks will be launch. As the power of Ansible is to provision multiple VMs using parallelism (this means lauching a single playbook will provision as many VMs as you want), if you want to keep this behavior, you will have to modify a little bit the syntax. The Vagrantfile just above would thus become :

Vagrant.configure("2") do |config|

  config.vm.box = "fedora/41-cloud-base"

  config.vm.provider :libvirt do |libvirt|
    libvirt.qemu_use_session = false
  end

  N = 2
  (1..N).each do |machine_id|
    config.vm.define "vm#{machine_id}" do |hostconfig|

      # Only execute once the Ansible provisioner,
      # when all the machines are up and ready.
      if machine_id == N
        hostconfig.vm.provision :ansible do |ansible|
          # Disable default limit to connect to all the machines
          ansible.limit = "all"
          ansible.playbook = "playbook.yml"
        end
      end
    end
  end
end

Multi-provider compatibility

Another good thing with Vagrant is that you can share your Vagrantfile with others. If you are developing stuffs and using Vagrant to make tests in one or multiple VMs, you can easily share your test infrastructure with other developers.

In case these others are using Vagrant with VirtualBox provider whereas you are using it with libvirt provider, it is still possible to make your Vagrantfile compatible with both. As described in the documentation, Vagrant has a default provider that will be used to launch VMs. It is generally VirtualBox but it can be another one depending on the distribution (as Fedora is not providing a VirtualBox package, libvirt is the default provider).

As a result, the minimum Vagrantfile below should work exactly the same on different environments.

Vagrant.configure("2") do |config|
    config.vm.box = "fedora/41-cloud-base"
end

As a box is specific to a provider and an architecture, Vagrant will automatically get the box compatible with your own environment (if such a box exists of course).

However, it is sometimes necessary to explicitly write provider specific parameters. Some values are indeed managed by the provider and need to be re-written for all providers that will be used with the Vagrantfile. Let say that we discribe a VM with an amount of 8 Gb of memory and 4 CPUs. If this Vangrantfile will be used with VirtualBox and libvirt providers, it should then looks like :

Vagrant.configure("2") do |config|
    config.vm.box = "fedora/41-cloud-base"

    # VirtualBox specific parameters
    config.vm.provider :virtualbox do |vbox|
      vbox.memory = 8192
      vbox.cpus = 4
    end

    # libvirt specific parameters
    config.vm.provider :libvirt do |libvirt|
      libvirt.memory = 8192
      libvirt.cpus = 4
    end
end

Conclusion

Vagrant is the perfect tool to easily describe, configure and eventually share a VM infrastructure. As shown in this post, you can quickly start to write a Vagrantfile. The Ruby syntax let you create more complicated stuffs. Don’t hesitate to also read Vagrant or associated plugins documentation to see the power of this tool.

Unfortunately, since v3.4.0, Vagrant does not have an open source license anymore as the new Business Source License restricts the use of the software to certain users. Such a change has impacts on how the product is integrated in open-source distribution like Fedora which only supports open-source softwares in their official repositories. The package is then distributed in version before v3.4.0 which does not allow user to easily get an updated version.

Sources

• Vagrant official documentation
• Vagrant documentation on Fedora Developer Portal
• Multi-provider Vagrantfile

Introduction

The “official” way to get applications on your Android device is to use Google Play. You can easily find a large amount of apps and manage those you want to install or remove directly from the plateform.

There are however many reasons why you would prefer to avoid using Google Play and directly install apps on your device without using this plateform. An Android application is after all a package (Android Package Kit (APK) format) that can be installed directly on the device and Google Play is only a package manager (others exist on Android (e.g. F-Droid)). But there are some risks to do so and this is why there are safeguards implemented by default on the operating system. Beyond those, there are also checks that can be directly done on the APK file you want to install in order to be sure that you are installing the one produced by the developer you trust. How to proceed to these verifications ? This what we are going to discuss in this article.

Default safeguards

Install permission

By default, Android doesn’t let you install a package that you have downloaded on the device or transfered from another device. As soon as you want to launch the app installation by selecting the APK file from the file manager, Android will complain that the file manager application is not allowed to install packages.

To bypass this safeguard, go to Settings > Apps, select the file manager app and go to Install unknown apps to allow package installation. No need to keep this permission permanently, so you can remove it just after you have finished the package installation.

The same situation could occur when launching the APK file directly from your web browser. In this case, the latter should be allowed to install packages.

Signing check

If you want to manually update an application that has already been installed on the device with Google Play for exemple, Android will probably complain, when launching the APK file, that the application is conflicting with an onther one on the device. In such a case, this is due to the signing check done by Android : both packages are signed with different keys and the operating system prevents the installation of a potentially malicious update that has not been built by an entity or a developer you trust.

However, there are many situations where there is no security problem. Indeed, it is simply possible that the package installed with a package manager like Google Play or F-Droid is signed with a different key than the one directly downloaded for a manual installation. As described in Android documentation, apps created after August 2021 must be signed with a key provided and managed by Google (app signing key). As a result, if a developer also want to provide its app outside Google Play pipeline, there is a significant chance that the APK will be signed with a different key.

In this situation, if you are sure that the APK file is signed by a trusted entity, the only way to bypass this safguard is to remove the previously installed package before installing manually the new one.

But how to manually verify a package signature ? This is in particular what we are going to discuss in the next section.

Signing key comparison

The second safeguard described above only applies when a similar package is already installed. But when you want to manually install a new app on your device, the signature is a way to verify that the package has been built by a developer you trust. It reduces the risk that it contains milicious code which could be harmful for your device. Unfortunately, this verification needs some steps to be done.

Let say that you want to manually install Signal App on your smartphone without using Google Play. Even if it is risky and dedicated to “advanced user”, Signal let you download the APK file directly from their website and incite users to verify the SHA-256 fingerprint of the signing certificate. In other words, the safest way to manually install Signal is to compare the fingerprint of the signing certificate which belongs to the file you’ve downloaded with the one provided on Signal website. Doing so, if the verification succeeds, you can be 100% sure that the APK file has been built by Signal developers and can be safely installed on your device. (Except if the signing key has been stolen… Risk management is always a question of decision : at some point, you will have to trust enough the entity (organisation or developer) somehow.)

Get apksigner tool

This verification needs a tool called apksigner which is provided by the Android SDK build tools. You can download Android Studio command line tools to get it. As the tools are written with Java, a prerequisite is the installation of a Java JDK. For example, it can be easily installed with the command below :

sudo dnf install java-latest-openjdk # Fedora
sudo apt-get update && sudo apt-get install default-jdk # Debian

If you have several Java versions installed on your system and if you need to switch to the latest, you can do so with the following command :

sudo alternatives --config java # Fedora
sudo update-alternatives --config javac # Debian

Once the command line tools have been downloaded, you can verify the SHA-256 checksum of the zip file with the one provided on Android website before extracting the compressed files :

sha256sum commandlinetools-*.zip
# If everything is OK :
unzip commandlinetools-*.zip
cd cmdline-tools
./bin/sdkmanager --sdk_root=./ "build-tools;34.0.0"

If the last command above returns with no error, you can then use apksigner tool.

APK signature verification

From cmdline-tools folder, you can run apksigner with the following command where <PATH_TO_APK> is the path to the APK file :

./build-tools/34.0.0/apksigner verify --print-certs <PATH_TO_APK> 

With Signal example, you should obtain the following result which is consistent with the information provided by Signal’s website.

./build-tools/34.0.0/apksigner verify --print-certs ../Signal*.apk

Signer (minSdkVersion=33, maxSdkVersion=2147483647) certificate DN: CN="Signal Messenger, LLC", SERIALNUMBER=6703101, OID.2.5.4.15=Private Organization, O="Signal Messenger, LLC", OID.1.3.6.1.4.1.311.60.2.1.2=Delaware, OID.1.3.6.1.4.1.311.60.2.1.3=US, L=Mountain View, ST=California, C=US
Signer (minSdkVersion=33, maxSdkVersion=2147483647) certificate SHA-256 digest: 4be4f6cd5be844083e900279dc822af65a547fecc26aba7ff1f5203a45518cd8
Signer (minSdkVersion=33, maxSdkVersion=2147483647) certificate SHA-1 digest: 5c6740091301285db5409fdcd1b90f1ac3ba2dcf
Signer (minSdkVersion=33, maxSdkVersion=2147483647) certificate MD5 digest: 34ac0b4e5d2c5c08b704fc05874f9a10
Signer (minSdkVersion=24, maxSdkVersion=32) certificate DN: CN=Whisper Systems, OU=Research and Development, O=Whisper Systems, L=Pittsburgh, ST=PA, C=US
Signer (minSdkVersion=24, maxSdkVersion=32) certificate SHA-256 digest: 29f34e5f27f211b424bc5bf9d67162c0eafba2da35af35c16416fc446276ba26
Signer (minSdkVersion=24, maxSdkVersion=32) certificate SHA-1 digest: 45989dc9ad8728c2aa9a82fa55503e34a8879374
Signer (minSdkVersion=24, maxSdkVersion=32) certificate MD5 digest: d90db364e32fa3a7bda4c290fb65e310

Source of truth

Unfortunately, many developers don’t advertise the fingerprint of their signing certificate as explicitly as Signal. It is sometime possible to find it somewhere, like GitHub/GitLab issues or discussions on social networks.

If you cannot retrieve easily the information on Internet, an option is to get it from the package obtained via the “official” way (i.e. Google Play) if this one is signed with a developer key and not with one managed by Google as described above (only possible for apps created before August 2021).

In such a case, you can install first the package with Google Play, transfer the APK file on your computer and get the fingerprint with apksigner.

However, how to transfer the APK file from your device to your computer ? This can be done with Android Debug Bridge (ADB) that can be installed with the following command :

sudo dnf install android-tools # Fedora
sudo apt-get update && sudo apt-get install adb # Debian

Before using ADB, you have to enable developer options and USB debugging on your device as described here.

You can then check that everything is working plugging in your Android device to your computer. A prompt on the device should ask you to trust the computer. Then the following command on the computer should show the connected device :

adb devices -l

The list of installed packages can be obtained with the following command :

adb shell pm list packages

If you have multiple users on your Android device, you can get the list of installed package for a specific user with this command where <USER_ID> is the ID of this user obtained with adb shell pm list users :

adb shell pm list packages --user <USER_ID>

The package names are displayed with their reversed Internet domain name which is a Java convention for package naming. Find the one you are looking for and get the path of the APK file on the device with this command where <APK_NAME> is the reversed Internet domain name of your package :

adb shell pm path <APK_NAME>

Finally, you can transfer your file on your computer with the following command where <REMOTE_APK_PATH> is the path previously obtained and <LOCAL_APK_PATH> is the path on your computer where you want to store the APK file :

adb pull <REMOTE_APK_PATH> <LOCAL_APK_PATH>

Once the APK file is on your computer, you can use apksigner to get the fingerprint and compare it with the one from the APK file manually downloaded. Once a package is manually installed, the second safeguard still applies and protects your device from installing a malicious package while updating the app.

Conclusion

Signing packages is one of the most powerfull method to ensure that files provided are built from an entity you trust. This is exactly what is done for years with Linux packages and Android has just implemented similar mechanisms. We have seen that these checks are processed in the background as package managers like Google Play are abstracting everything.

By-passing packages managers is not a reason to decrease security level and this article has shown methods to check that a package is the right one, provided by an entity you trust, before installing it manually on your Android device.

Sources

• Verifying APK Fingerprints
• Another way to verify APK Fingerprints
• ADB Documentation

Introduction

If you care about security and privacy, Qubes OS is a pretty good choice as it implements securely-isolated compartments called “Qubes” and integrates notably Whonix as a Tor gateway. More precisely, Qubes are virtual machines that takes advantage of all virtualization features to create those isolated compartments.

It is still possible to increase your privacy, espacially from your ISP, in using a VPN service like Mullvad within Qubes OS. There are plenty of VPN providers but Mullvad and some others have made big efforts on transparency and are providing open-source clients (cf. Mullvad VPN application). Mullvad already provides guides like this one to help you configure the service on Qubes OS. It however only creates tunnel with a specifically choosen server and thus doesn’t use all the benefits of Mullvad VPN application like switching server, location or protocol.

We will explain how to install Mullvad VPN application in a similar configuration : a proxy Qube which provides to any other Qubes an Internet access through a Mullvad VPN.

The procedure bellow has been tested with Qubes OS 4.1.

Mullvad VPN application installation inside a proxy Qube

Create a new template

Qubes OS project’s security model give a great importance to base templates like Fedora or Debian. This is why any software should be installed from the default repositories of these distributions and, when it is not possible, the security of the template should be considered as downgraded.

As Mullvad VPN application is not in the default repositories, we will clone the Fedora template and use this new one to install the software as discribed in the Qubes OS documentation :

• Open the Qube Manager (Qubes Tools / Qube Manager)
• Click on the default Fedora template and open the Settings window
• You should be able to click on Clone qube button at the bottom right of the window
• Choose a non-black color for this template (red for example) indicating that it is potentially less secure that the default template
• Call your new template fedora-XX-mullvad where XX is the Fedora release number

Give Internet access to the template

This is the risky part of the process. Qubes OS project’s security model considers that templates are secured as it has no access to Internet and can only download packages from default repositories through a proxy. We will however have to give Internet access to our new template in order to download the necessary files :

• In the Qube Manager, click on the fedora-XX-mullvad template and open the Settings window
• For Net qube option, select sys-firewall
• In Firewall rules tab, select Limit outgoing connections to ... and click on Allow full access for 15 min in order to get an Internet access during 15 minutes (which should be enough to install Mullvad VPN application) and block it again when time has elapsed.

NOTE : Adding and removing Internet access to the template is the prescribed way in Qubes OS documentation. You can however bypass these steps without added risks in reaching directly the Qube proxy :

• Set https_proxy environment variable

export https_proxy=127.0.0.1:8082

• Install Mullvad VPN App as explained below
• Once everything is installed, unset https_proxy environment variable

unset https_proxy

Install Mullvad VPN application

We will simply follow the Mullvad VPN application installation instructions on Fedora with a signature check :

• Start the template fedora-XX-mullvad
• Open a terminal for this template
• Download the signing key provided by Mullvad which should has been used to sign the RPM package :

curl https://mullvad.net/media/mullvad-code-signing.asc -o mullvad-code-signing.asc

• Import the key and check that its fingerprint is A119 8702 FC3E 0A09 A9AE 5B75 D5A1 D4F2 66DE 8DDF

gpg --import mullvad-code-signing.asc

(If you want to verify the fingerprint before importing the key, you can do so with gpg --show-keys --with-fingerprint mullvad-code-signing.asc)

• Edit the key with the command gpg --edit-key A1198702FC3E0A09A9AE5B75D5A1D4F266DE8DDF. The output should be something like this :

gpg (GnuPG) 2.1.13; Copyright (C) 2016 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

pub  rsa4096/D5A1D4F266DE8DDF
     created: 2016-10-27  expires: never       usage: SC 
     trust: unknown       validity: unknown
sub  rsa4096/C187D22C089EF64E
     created: 2016-10-27  expires: never       usage: E  
sub  rsa4096/A26581F219C8314C
     created: 2016-10-27  expires: never       usage: S  
[ unknown] (1). Mullvad (code signing) <admin@mullvad.net>

• If everything seems correct, you can trust the key via the gpg prompt : gpg> trust. You will be asked to set the trust level. Set it to 5 in order to trust the key ultimately
• Leave the gpg prompt : gpg> q
• Download the app and its signature :

curl https://mullvad.net/download/app/rpm/latest -o mullvad.rpm
curl https://mullvad.net/download/app/rpm/latest/signature -o mullvad.asc

• In the same folder where the files have been downloaded, run gpg --verify mullvad.asc mullvad.rpm to proceed to the signature check. You should see in the output of this command something like this :

gpg: Good signature from "Mullvad (code signing) <admin@mullvad.net>" [ultimate]

• If every verifications above have seceeded, you can then install the app with the folowing command :

sudo dnf install -y ./mullvad.rpm

• When the installation is finished, enable mullvad-daemon service so that the app launches automatically when the Qube starts :

sudo systemctl enable mullvad-daemon

• Finally shutdown the template

Remove Internet access to the template

As we have given access to Internet to our new template, we will remove this one for consistency with Qubes OS project recommendations.

• In the Qube Manager, click on the fedora-XX-mullvad template and open the Settings window
• For Net qube option, select back default (none)

Create the proxy Qube

We will use our new template to create the proxy Qube which will privide an Internet access to any other Qube.

• Open the Qube Manager (Qubes Tools / Qube Manager)
• Click on New qube to create a new Qube
• In Basic tab, you can call this Qube sys-vpn and use the color red. Select AppVM as Type, fedora-XX-mullvad as Template and default (sys-firewall) as Networking.
• In Advanced tab, enable Provides network access to other qubes

Configure Mullvad VPN

Everything should be ready to use Mullvad VPN. Start sys-vpn and use the following commands to configure your Mullvad account from CLI :

• Configure your acount :

mullvad account login

You will be asked to enter your Mullvad account number.

• Verify that your device is registered :

mullvad account get

You should see the device name that has been assigned by Mullvad to your Qube.

• Configure the server you want to use (e.g. se-mma-wg-001 which is a Wireguard server located in Sweden, in the city of Malmö) :

mullvad relay set location se-mma-wg-001

• Connect to the server :

mullvad connect

• Check that you are connected :

mullvad status

You should see something like Connected to se-mma-wg-001 in Malmö, Sweden.

• Finally, auto-connect Mullvad VPN each time the Qube starts :

mullvad auto-connect set on

Fix DNS problem

As Mullvad VPN application will change the DNS server in /etc/resolv.conf, it is necessary to run again the /usr/lib/qubes/qubes-setup-dnat-to-ns script which is normally executed when the Qube starts. It will update the DNAT rules in iptables so that to reach the right DNS server. You can verify the consistency of /etc/resolv.conf file and the DNAT rules with the command sudo iptables -t nat -L. This is a minimal workaround and this DNS update should be directly managed by Mullvad VPN application.

To do so, simply run the following command in sys-vpn Qube :

sudo /usr/lib/qubes/qubes-setup-dnat-to-ns

Configure persistency for Mullvad VPN application configuration

Most of data in AppVM are volatile, especially the ones in /etc folder and subfolders. Each AppVM Qube has a persistent folder in /rw. This is where you can keep configurations files across Qube reboots (e.g. in /rw/config). We will use this folder in order to be sure that configurations made in Mullvad VPN application are kept after each reboot. To do so, we will use the /rw/config/rc.local script which is executed everytime the Qube starts.

• Create a persistent folder for Mullvad VPN application configuration files :

sudo mkdir /rw/config/mullvad-vpn

• Copy the already existing configuration files :

sudo cp /etc/mullvad-vpn/* /rw/config/mullvad-vpn/

• Customise the rc.local file to reconfigure Mullvad VPN application at every startup. The file should looks like that :

#!/bin/sh

# Configure Mullvad VPN application
rm -rf /etc/mullvad-vpn/
ln -s /rw/config/mullvad-vpn /etc/mullvad-vpn
systemctl restart mullvad-daemon

# fix DNS problem (https://github.com/mullvad/mullvadvpn-app/issues/3803)
sleep 10 # Waiting a bit that Mullvad succeeds to establish connection
/usr/lib/qubes/qubes-setup-dnat-to-ns

Provide Internet access to other Qubes through Mullvad VPN

We now have our proxy Qube called sys-vpn configured to build a tunnel to a Mullvad server using Mullvad VPN application. The last thing to do is to configure your Qubes to reach Internet through Mullvad VPN. For those Qubes, the only thing to do is to open the Settings and select sys-vpn for Net qube option instead of default (sys-firewall).

You can verify that you are accessing Internet through a Mullvad VPN :

• Open a terminal from the Qube connected to sys-vpn
• Run the following command : curl https://am.i.mullvad.net/connected. The output should starts with You are connected to Mullvad ...

(Optional) Manage configuration via Mullvad VPN application GUI

With all the steps above, you should have a working configuration which let you access Internet through a Mullvad VPN. However we interacted with Mullvad VPN application through the CLI.

But you can also access to the GUI to interact with Mullvad VPN application. In sys-vpn qube settings, open Applications tab and add Mullvad VPN to the selected application.

You can then access the App GUI via Service : sys-vpn / sys-vpn : Mullvad VPN.

Conclusion

We have set up a proxy VM which is able to provide to other Qubes an Internet access through a Mullvad VPN. This kind of configuration is based on what is done by default in Qubes OS with sys-whonix service which provides an Internet access through Tor to other Qubes.

As it has already been mentionned, the DNS update in rc.local file is a minimal workaround and has many ceveats. For example, if Mullvad VPN application connects to another server and changes the DNS, the DNAT rules won’t be updated automatically and the /usr/lib/qubes/qubes-setup-dnat-to-ns script should be run again manually. This is why the DNS issue should clearly be managed directly by Mullvad VPN application and the is the goal of this Github issue.

Sources

• Qubes OS Documentation : How to install software
• Qubes OS Documentation : Networking
• Mullvad Documentation : Wireguard on Qubes OS
• VPN providers respecting privacy