Let's begin with some basic definitions, an overview, and some history about Debian packages, starting with dpkg and APT.
A Debian package is a compressed archive of a software application. A binary package (a .deb file) contains files that can be directly used (such as programs or documentation), while a source package contains the source b for the software and the instructions required for building a binary package. A Debian package contains the application's files as well as other metadata including the names of the dependencies the application needs, as well as scripts that enable the execution of commands at different stages in the package's lifecycle (installation, upgrades, and removal).
The dpkg tool was designed to process and install .deb packages, but if it encountered an unsatisfied dependency (like a missing library) that would prevent the package from installing, dpkg would simply list the missing dependency, because it had no awareness or built-in logic to find or process the packages that might satisfy those dependencies. The Advanced Package Tool (APT), including apt and apt-get, were designed to address these shortcomings and could automatically resolve these issues. We will talk about both dpkg and the APT tools in this chapter.
The base command for handling Debian packages on the system is dpkg, which performs installation or analysis of .deb packages and their contents. However, dpkg has only a partial view of the Debian universe: it knows what is installed on the system and whatever you provide on the command line, but knows nothing of the other available packages. As such, it will fail if a dependency is not met. APT addresses the limitations.
APT is a set of tools that help manage Debian packages, or applications on your Debian system. You can use APT to install and remove applications, update packages, and even upgrade your entire system. The magic of APT lies in the fact that it is a complete package management system that will not only install or remove a package, but will consider the requirements and dependencies of the packaged application (and even their requirements and dependencies) and attempt to satisfy them automatically. APT relies on dpkg but APT differs from dpkg, as the former installs the latest package from an online source and works to resolve dependencies while dpkg installs a package located on your local system and does not automatically resolve dependencies.
If you have been around long enough to remember compiling programs with gcc (even with the help of utilities such as make and configure), you likely remember that it was a painful process, especially if the application had several dependencies. By deciphering the various warnings and error messages, you may have been able to determine which part of the b was failing and most often that failure was due to a missing library or other dependency. You would then track down that missing library or dependency, correct it, and try again. Then, if you were lucky, the compile would complete, but often the build would fail again, complaining about another broken dependency.
APT was designed to help alleviate that problem, collate program requirements and dependencies, and resolve them. This functionality works out-of-the-box on Kali Linux, but it isn't foolproof. It is important that you understand how Debian and Kali's packaging system works because you will need to install packages, update software, or troubleshoot problems with packages. You will use APT in your day-to-day work with Kali Linux and in this chapter, we will introduce you to APT and show you how to install, remove, upgrade, and manage packages, and even show you how to move packages between different Linux distributions. We will also talk about graphical tools that leverage APT, show you how to validate the authenticity of packages, and delve into the concept of a rolling distribution, a technique that brings daily updates to your Kali system.
Before we dig in and show you how to use dpkg and APT to install and manage packages, it is important that we delve into some of the inner workings of APT and discuss some terminology surrounding it.
APT retrieves its packages from a repository, a package storage system or simply, "package source". The /etc/apt/sources.list file lists the different repositories (or sources) that publish Debian packages.
The sources.list file is the key configuration file for defining package sources, and it is important to understand how it is laid out and how to configure it since APT will not function without a properly defined list of package sources. Let's discuss its syntax, take a look at the various repositories that are used by Kali Linux, and discuss mirrors and mirror redirection, then you will be ready to put APT to use.
Each active line of the /etc/apt/sources.list file (and of the /etc/apt/sources.list.d/*.list files) contains the description of a source, made of three parts separated by spaces. Commented lines begin with a # character:
#deb cdrom:[Debian GNU/Linux 2020.3 _Kali-rolling_ - Official Snapshot amd64 LIVE/INSTALL Binary 20200728-20:25]/ kali-last-snapshot contrib main non-free deb http://http.kali.org/kali kali-rolling main non-free contrib
Let's take a look at the syntax of this file. The first field indicates the source type:
- deb for binary packages,
- deb-src for source packages.
The second field gives the base URL of the source: this can consist of a Kali mirror or any other package archive set up by a third party. The URL can start with file:// to indicate a local source installed in the system's file hierarchy, with http:// to indicate a source accessible from a web server, or with ftp:// for a source available on an FTP server. The URL can also start with cdrom: for CD/DVD-ROM/Blu-ray disc-based installations, although this is less frequent since network-based installation methods are more and more common.
The cdrom entries describe the CD/DVD-ROM you have. Contrary to other entries, a CD/DVD-ROM is not always available, since it has to be inserted into the drive and usually only one disc can be read at a time. For those reasons, these sources are managed in a slightly different way and need to be added with the apt-cdrom program, usually executed with the add parameter. The latter will then request the disc to be inserted in the drive and will browse its contents looking for Packages files. It will use these files to update its database of available packages (this operation is usually done by the apt update command). After that, APT will request the disc if it needs a package stored on it.
The syntax of the last field depends on the structure of the repository. In the simplest cases, you can easily indicate a subdirectory (with a required trailing slash) of the desired source (this is often a simple “./”, which refers to the absence of a subdirectory—the packages are then directly at the specified URL). But in the most common case, the repositories will be structured like a Kali mirror, with multiple distributions each having multiple components. In those cases, name the chosen distribution, then the components (or sections) to enable. Let's take a moment to introduce these sections.
Debian and Kali use three sections to differentiate packages according to the licenses chosen by the authors of each work. A difference between Debian and Kali is that, Debian only has main enabled by default, whereas Kali has all three enabled by default.
main contains all packages that fully comply with the Debian Free Software Guidelines.
The non-free archive is different because it contains software that does not (entirely) conform to these principles but which can nevertheless be distributed without restrictions.
contrib (contributions) is a set of open source software that cannot function without some non-free elements. These elements may include software from the non-free section or non-free files such as game ROMs, BIOS of consoles, etc. contrib also includes free software whose compilation requires proprietary elements, such as VirtualBox, which requires a non-free compiler to build some of its files.
Now, let's take a look at the standard Kali Linux package sources, or repositories.
A standard sources.list file for a system running Kali Linux refers to one repository (kali-rolling) and the three previously mentioned components: main, contrib, and non-free:
# Main Kali repository deb http://http.kali.org/kali kali-rolling main contrib non-free
Let's take a look at the various Kali repositories.
This is the main repository for end-users. It should always contain installable and recent packages. It is managed by a tool that merges Debian Testing and the Kali-specific packages in a way that ensures that each package's dependencies can be satisfied within kali-rolling. In other words, barring any bug in maintainer scripts, all packages should be installable.
Since Debian Testing evolves daily, so does Kali Rolling. The Kali-specific packages are also regularly updated as we monitor the upstream releases of the most important packages.
This repository is not for public use. It is a space where Kali developers resolve dependency problems arising from the merge of the Kali-specific packages into Debian Testing.
It is also the place where updated packages land first, so if you need an update that was released recently and that has not yet reached kali-rolling, you might be able to grab it from this repository. This is not recommended for regular users.
The sources.list extracts above refer to http.kali.org: this is a server running MirrorBrain, which will redirect your HTTP requests to an official mirror close to you. MirrorBrain monitors each mirror to ensure that they are working and up-to-date; it will always redirect you to a good mirror.
We also have a second MirrorBrain instance: where http.kali.org hosts the package repositories, cdimage.kali.org hosts the released ISO images.
If you want to request a list of official Kali Linux Mirrors, you can add .mirrorlist to any valid URL pointing to http.kali.org or cdimage.kali.org.
These lists are not exhaustive due to some MirrorBrain limitations (most notably mirrors restricted to some countries do not appear in the list unless you are in the given country). But they contain the best mirrors: they are well maintained and have large amounts of bandwidth available.