Introduction to APT

After the basics of Linux, it is time to learn the package management system of a Debian-based distribution. In such distributions, including Kali, the Debian package is the canonical way to make software available to end-users. Understanding the package management system will give you a great deal of insight into how Kali is structured, enable you to more effectively troubleshoot issues, and help you quickly locate help and documentation for the wide array of tools and utilities included in Kali Linux.

In this chapter, we will introduce the Debian package management system and introduce dpkg and the APT suite of tools. One of the primary strengths of Kali Linux lies in the flexibility of its package management system, which leverages these tools to provide near-seamless installation, upgrades, removal, and manipulation of application software, and even of the base operating system itself. It is critical that you understand how this system works to get the most out of Kali and streamline your efforts. The days of painful compilations, disastrous upgrades, debugging gcc, make, and configure problems are long gone, however, the number of available applications has exploded and you need to understand the tools designed to take advantage of them. This is also a critical skill because there are a number of security tools that, due to licensing or other issues, cannot be included in Kali but have Debian packages available for download. It is important that you know how to process and install these packages and how they impact the system, especially when things do not go as expected.

We will begin with some basic overviews of APT, describe the structure and contents of binary and source packages, take a look at some basic tools and scenarios, and then dig deeper to help you wring every ounce of utility from this spectacular package system and suite of tools.

8.1. Introduction to APT

Let's begin with some basic definitions, an overview, and some history about Debian packages, starting with dpkg and APT.

8.1.1. Relationship between APT and dpkg

A Debian package is a compressed archive of a software application. A binary package (a .deb file) contains files that can be directly used (such as programs or documentation), while a source package contains the source code for the software and the instructions required for building a binary package. A Debian package contains the application's files as well as other metadata including the names of the dependencies the application needs, as well as scripts that enable the execution of commands at different stages in the package's lifecycle (installation, removal, and upgrades).

The dpkg tool was designed to process and install .deb packages, but if it encountered an unsatisfied dependency (like a missing library) that would prevent the package from installing, dpkg would simply list the missing dependency, because it had no awareness or built-in logic to find or process the packages that might satisfy those dependencies. The Advanced Package Tool (APT), including apt and apt-get, were designed to address these shortcomings and could automatically resolve these issues. We will talk about both dpkg and the APT tools in this chapter.

The base command for handling Debian packages on the system is dpkg, which performs installation or analysis of .deb packages and their contents. However, dpkg has only a partial view of the Debian universe: it knows what is installed on the system and whatever you provide on the command line, but knows nothing of the other available packages. As such, it will fail if a dependency is not met. APT addresses the limitations.

APT is a set of tools that help manage Debian packages, or applications on your Debian system. You can use APT to install and remove applications, update packages, and even upgrade your entire system. The magic of APT lies in the fact that it is a complete package management system that will not only install or remove a package, but will consider the requirements and dependencies of the packaged application (and even their requirements and dependencies) and attempt to satisfy them automatically. APT relies on dpkg but APT differs from dpkg, as the former installs the latest package from an online source and works to resolve dependencies while dpkginstalls a package located on your local system and does not automatically resolve dependencies.

If you have been around long enough to remember compiling programs with gcc (even with the help of utilities such as make and configure), you likely remember that it was a painful process, especially if the application had several dependencies. By deciphering the various warnings and error messages, you may be able to determine which part of the code was failing and most often that failure was due to a missing library or other dependency. You would then track down that missing library or dependency, correct it, and try again. Then, if you were lucky, the compile would complete, but often the build would fail again, complaining about another broken dependency.

APT was designed to help alleviate that problem, collate program requirements and dependencies, and resolve them. This functionality works out-of-the-box on Kali Linux, but it isn't foolproof. It is important that you understand how Debian and Kali's packaging system works because you will need to install packages, update software, or troubleshoot problems with packages. You will use APT in your day-to-day work with Kali Linux and in this chapter, we will introduce you to APT and show you how to install, remove, upgrade, and manage packages, and even show you how to move packages between different Linux distributions. We will also talk about graphical tools that leverage APT, show you how to validate the authenticity of packages, and delve into the concept of a rolling distribution, a technique that brings daily updates to your Kali system.

Before we dig in and show you how to use dpkg and APT to install and manage packages, it is important that we delve into some of the inner workings of APT and discuss some terminology surrounding it.

Package Source and Source Package


The word source can be ambiguous. A source package—a package containing the source code of a program—should not be confused with a package source—a repository (website, FTP server, CD-ROM, local directory, etc.) that contains packages.

APT retrieves its packages from a repository, a package storage system or simply, "package source". The /etc/apt/sources.list file lists the different repositories (or sources) that publish Debian packages.

8.1.2. Understanding the sources.list File

The sources.list file is the key configuration file for defining package sources, and it is important to understand how it is laid out and how to configure it since APT will not function without a properly defined list of package sources. Let's discuss its syntax, take a look at the various repositories that are used by Kali Linux, and discuss mirrors and mirror redirection, then you will be ready to put APT to use.

Each active line of the /etc/apt/sources.list file (and of the /etc/apt/sources.list.d/*.list files) contains the description of a source, made of three parts separated by spaces. Commented lines begin with a #character:

Let's take a look at the syntax of this file. The first field indicates the source type:

  • deb for binary packages,
  • deb-src for source packages.

The second field gives the base URL of the source: this can consist of a Debian mirror or any other package archive set up by a third party. The URL can start with file:// to indicate a local source installed in the system's file hierarchy, with http:// to indicate a source accessible from a web server, or with ftp:// for a source available on an FTP server. The URL can also start with cdrom: for CD-ROM/DVD-ROM/Blu-ray disc-based installations, although this is less frequent since network-based installation methods are more and more common.

The cdrom entries describe the CD/DVD-ROMs you have. Contrary to other entries, a CD-ROM is not always available, since it has to be inserted into the drive and usually only one disc can be read at a time. For those reasons, these sources are managed in a slightly different way and need to be added with the apt-cdromprogram, usually executed with the add parameter. The latter will then request the disc to be inserted in the drive and will browse its contents looking for Packages files. It will use these files to update its database of available packages (this operation is usually done by the apt update command). After that, APT will request the disc if it needs a package stored on it.

The syntax of the last field depends on the structure of the repository. In the simplest cases, you can simply indicate a subdirectory (with a required trailing slash) of the desired source (this is often a simple "./", which refers to the absence of a subdirectory—the packages are then directly at the specified URL). But in the most common case, the repositories will be structured like a Debian mirror, with multiple distributions each having multiple components. In those cases, name the chosen distribution, then the components (or sections) to enable. Let's take a moment to introduce these sections.

Debian and Kali use three sections to differentiate packages according to the licenses chosen by the authors of each work.

Main contains all packages that fully comply with the Debian Free Software Guidelines.

The non-free archive is different because it contains software that does not (entirely) conform to these principles but which can nevertheless be distributed without restrictions.

Contrib (contributions) is a set of open source software that cannot function without some non-free elements. These elements may include software from the non-free section or non-free files such as game ROMs, BIOS of consoles, etc. Contrib also includes free software whose compilation requires proprietary elements, such as VirtualBox, which requires a non-free compiler to build some of its files.

Now, let's take a look at the standard Kali Linux package sources, or repositories.

8.1.3. Kali Repositories

A standard sources.list file for a system running Kali Linux refers to one repository (kali-rolling) and the three previously mentioned components: main, contrib, and non-free:

Let's take a look at the various Kali repositories.

8.1.3.1. The Kali-Rolling Repository

This is the main repository for end-users. It should always contain installable and recent packages. It is managed by a tool that merges Debian Testing and the Kali-specific packages in a way that ensures that each package's dependencies can be satisfied within kali-rolling. In other words, barring any bug in maintainer scripts, all packages should be installable.

Since Debian Testing evolves daily, so does Kali Rolling. The Kali-specific packages are also regularly updated as we monitor the upstream releases of the most important packages.

8.1.3.2. The Kali-Dev Repository

This repository is not for public use. It is a space where Kali developers resolve dependency problems arising from the merge of the Kali-specific packages into Debian Testing.

It is also the place where updated packages land first, so if you need an update that was released recently and that has not yet reached kali-rolling, you might be able to grab it from this repository. This is not recommended for regular users.

8.1.3.3. The Kali-Bleeding-Edge Repository

This repository contains packages automatically built out of the upstream Git (or Subversion) repository. The upside is that you immediately have access to the latest features and bug fixes less than 24 hours after they have been committed. This is an ideal way to verify if a bug that you reported upstream has been fixed.

The downside is that these packages have not been tested or vetted: if the upstream changes impacted the packaging (adding a new dependency), then that package might not work. Because of this, the repository is marked in such a way that APT does not automatically install packages from it, particularly during an upgrade.

You can register the repository either by editing /etc/apt/sources.list or by creating a new file under the /etc/apt/sources.list.d directory, which has the benefit of leaving the original system sources.list file un-altered. In this example, we opt to create a separate /etc/apt/sources.list.d/kali-bleeding-edge.list file like this:

8.1.3.4. The Kali Linux Mirrors

The sources.list extracts above refer to http.kali.org: this is a server running MirrorBrain, which will redirect your HTTP requests to an official mirror close to you. MirrorBrain monitors each mirror to ensure that they are working and up-to-date; it will always redirect you to a good mirror.

Debugging a Mirror Redirection


If you have a problem with the mirror (for instance because apt update fails), you can use curl -sI to see where you are being redirected:

If the problem persists, you can edit /etc/apt/sources.list and hardcode the name of another known working mirror in place of (or before) the http.kali.org entry.

We also have a second MirrorBrain instance: where http.kali.org hosts the package repositories, cdimage.kali.org hosts the released ISO images.

http://cdimage.kali.org

If you want to request a list of official Kali Linux Mirrors, you can add .mirrorlist to any valid URL pointing to http.kali.org or cdimage.kali.org.

http://http.kali.org/README.mirrorlist
http://cdimage.kali.org/README.mirrorlist

These lists are not exhaustive due to some MirrorBrain limitations (most notably mirrors restricted to some countries do not appear in the list unless you are in the given country). But they contain the best mirrors: they are well maintained and have large amounts of bandwidth available.