Modifying Kali Packages

Comments are off

Topic Progress:

9.1. Modifying Kali Packages

Modifying Kali packages is usually a task for Kali contributors and developers: they update packages with new upstream versions, they tweak the default configuration for a better integration in the distribution, or they fix bugs reported by users. But you might have specific needs not fulfilled by the official packages and knowing how to build a modified package can thus be very valuable.

You might wonder why you need to bother with the package at all. After all, if you have to modify a piece of software, you can always grab its source code (usually with git) and run the modified version directly from the source checkout. This is fine when it is possible and when you use your home directory for this purpose, but if your application requires a system-wide setup (for example, with a make install step) then it will pollute your file system with files unknown to dpkg and will soon create problems that cannot be caught by package dependencies. Furthermore, with proper packages you will be able to share your changes and deploy them on multiple computers much more easily or revert the changes after having discovered that they were not working as well as you hoped.

So when would you want to modify a package? Let's take a look at a few examples. First, we will assume that you are a heavy user of Social-Engineer Toolkit (SET) and you noticed a new upstream release but the Kali developers are all busy for a conference and you want to try it out immediately. You want to update the package yourself. In another case, we will assume that you are struggling to get your MIFARE NFC card working and you want to rebuild "libfreefare" to enable debug messages in order to have actionable data to provide in a bug report that you are currently preparing. In a last case, we will assume that the "pyrit" program fails with a cryptic error message. After a web search, you find a commit that you expect to fix your problem in the upstream GitHub repository and you want to rebuild the package with this fix applied.

We will go through all of those samples in the following sections. We will try to generalize the explanations so that you can better apply the instructions to other cases but it is impossible to cover all situations that you might encounter. If you hit problems, apply your best judgment to find a solution or go seek help on the most appropriate forums (see Chapter 6, Helping Yourself and Getting Help).

Whatever change you want to make, the general process is always the same: grab the source package, extract it, make your changes, then build the package. But for each step, there are often multiple tools that can handle the task. We picked the most relevant and most popular tools, but our review is not exhaustive.

9.1.1. Getting the Sources

Rebuilding a Kali package starts with getting its source code. A source package is composed of multiple files: the main file is the *.dsc (Debian Source Control) file as it lists the other accompanying files, which can be *.tar.{gz,bz2,xz}, sometimes *.diff.gz, or *.debian.tar.{gz,bz2,xz} files.

The source packages are stored on Kali mirrors that are available over HTTP. You could use your web browser to download all the required files but the easiest way to accomplish this is to use the apt source source_package_name command. This command requires a deb-src line in the /etc/apt/sources.list file and up-to-date index files (accomplished by running apt update). By default, Kali doesn't add the required line as few Kali users actually need to retrieve source packages but you can easily add it (see sample file in Section 8.1.3, "Kali Repositories" and the associated explanations in Section 8.1.2, "Understanding the sources.list File").

$ apt source libfreefare
Reading package lists... Done
NOTICE: 'libfreefare' packaging is maintained in the 'Git' version control system at:
git://anonscm.debian.org/collab-maint/libnfc.git
Please use:
git clone git://anonscm.debian.org/collab-maint/libnfc.git
to retrieve the latest (possibly unreleased) updates to the package.
Need to get 119 kB of source archives.
Get:1 http://kali.download/kali kali-rolling/main libfreefare 0.4.0-2.1 (dsc) [2,144 B]
Get:2 http://kali.download/kali kali-rolling/main libfreefare 0.4.0-2.1 (tar) [113 kB]
Get:3 http://kali.download/kali kali-rolling/main libfreefare 0.4.0-2.1 (diff) [3,732 B]
Fetched 119 kB in 1s (100 kB/s)
dpkg-source: info: extracting libfreefare in libfreefare-0.4.0
dpkg-source: info: unpacking libfreefare_0.4.0.orig.tar.gz
dpkg-source: info: unpacking libfreefare_0.4.0-2.1.debian.tar.xz
$ cd libfreefare-0.4.0
$ ls
AUTHORS    cmake           configure.ac  COPYING  examples  libfreefare        m4           NEWS    test
ChangeLog  CMakeLists.txt  contrib       debian   HACKING   libfreefare.pc.in  Makefile.am  README  TODO
$ ls debian
changelog  compat  control  copyright  libfreefare0.install  libfreefare-bin.install  libfreefare-dev.install  libfreefare-doc.install  README.Source  rules  source  watch

In this example, while we received the source package from a Kali mirror, the package is the same as in Debian since the version string doesn't contain "kali." This means that no kali-specific changes have been applied.

If you need a specific version of the source package, which is currently not available in the repositories listed in /etc/apt/sources.list, then the easiest way to download it is to find out the URL of its .dsc file by looking it up on http://pkg.kali.org and then handing that URL over to dget (from the devscripts package).

After having looked up the URL of the libreefare source package available in kali-dev, you can download it with dget. It will first download the .dsc file, then parse it to know what other files are referenced, and then download those from the same location:

$ dget http://http.kali.org/pool/main/libf/libfreefare/libfreefare_0.4.0+0~git1439352548.ffde4d-1.dsc
dget: retrieving http://http.kali.org/pool/main/libf/libfreefare/libfreefare_0.4.0+0~git1439352548.ffde4d-1.dsc
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   362  100   362    0     0   1117      0 --:--:-- --:--:-- --:--:--  1120
100  1935  100  1935    0     0   3252      0 --:--:-- --:--:-- --:--:--  3252
dget: retrieving http://http.kali.org/pool/main/libf/libfreefare/libfreefare_0.4.0+0~git1439352548.ffde4d.orig.tar.gz
[...]
libfreefare_0.4.0+0~git1439352548.ffde4d-1.dsc:
dscverify: libfreefare_0.4.0+0~git1439352548.ffde4d-1.dsc failed signature check:
gpg: WARNING: no command supplied.  Trying to guess what you mean ...
gpg: Signature made Wed 12 Aug 2015 12:14:03 AM EDT
gpg:                using RSA key 43EF73F4BD8096DA
gpg: Can't check signature: No public key
Validation FAILED!!
$ dpkg-source -x libfreefare_0.4.0+0~git1439352548.ffde4d-1.dsc
gpgv: Signature made Wed 12 Aug 2015 12:14:03 AM EDT
gpgv:                using RSA key 43EF73F4BD8096DA
gpgv: Can't check signature: No public key
dpkg-source: warning: failed to verify signature on ./libfreefare_0.4.0+0~git1439352548.ffde4d-1.dsc
dpkg-source: info: extracting libfreefare in libfreefare-0.4.0+0~git1439352548.ffde4d
dpkg-source: info: unpacking libfreefare_0.4.0+0~git1439352548.ffde4d.orig.tar.gz
dpkg-source: info: unpacking libfreefare_0.4.0+0~git1439352548.ffde4d-1.debian.tar.xz

It is worth noting that dget did not automatically extract the source package because it could not verify the PGP signature on the source package. Thus we did that step manually with dpkg-source -x dsc-file. You can also force the source package extraction by passing the --allow-unauthenticated or -u option. Inversely, you can use --download-only to skip the source package extraction step.

Retrieving Sources from Git

You might have noticed that the apt source invocation tells you about a possible Git repository used to maintain the package. It might point to a Debian Git repository or to a Kali Git repository.

All Kali-specific packages are maintained in Git repositories hosted on gitlab.com/kalilinux/packages. You can retrieve the sources from those repositories with git clone https://gitlab.com/kalilinux/packages/source-package.git.

Contrary to what you get with apt source, the obtained tree will not have patches automatically applied. Have a look at debian/patches/ to learn about the possible changes made by Kali.

$ git clone https://gitlab.com/kalilinux/packages/kali-meta.git
Cloning into 'kali-meta'...
remote: Counting objects: 760, done.
remote: Compressing objects: 100% (614/614), done.
remote: Total 760 (delta 279), reused 0 (delta 0)
Receiving objects: 100% (760/760), 141.01 KiB | 0 bytes/s, done.
Resolving deltas: 100% (279/279), done.
Checking connectivity... done.
$ cd kali-meta
$ ls
debian
$ ls debian
changelog  compat  control  copyright  rules  source

You can use the git repositories as another way to retrieve the sources and thus (mostly) follow the other instructions from this section. But when Kali developers work with those repositories, they use another packaging workflow and use tools from the git-buildpackage package that we will not cover here. You can learn more about those tools here:

https://honk.sigxcpu.org/piki/projects/git-buildpackage/

9.1.2. Installing Build Dependencies

Now that you have the sources, you still need to install build dependencies. They will be necessary to build the desired binary packages but are also likely required for partial builds that you might want to run to test the changes while you make them.

Each source package declares its build dependencies in the Build-Depends field of the debian/control file. Let's instruct apt to install those (assuming that you are in a directory containing an unpacked source package):

$ sudo apt build-dep ./
Note, using directory './' to get the build dependencies
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  autoconf automake autopoint autotools-dev debhelper dh-autoreconf
  dh-strip-nondeterminism gettext intltool-debian libarchive-zip-perl
  libfile-stripnondeterminism-perl libtool po-debconf
0 upgraded, 13 newly installed, 0 to remove and 0 not upgraded.
Need to get 4 456 kB of archives.
After this operation, 14,6 MB of additional disk space will be used.
Do you want to continue? [Y/n]
[...]

In this sample, all build dependencies can be satisfied with packages available to APT. This might not always be the case as the tool building kali-rolling does not ensure installability of build dependencies (only dependencies of binary packages are taken into account). In practice, binary dependencies and build dependencies are often tightly coupled and most packages will have their build dependencies satisfiable.

9.1.3. Making Changes

We can't cover all the possible changes that you might want to make to a given package in this section. This would amount to teaching you all the nitty gritty details of Debian packaging. However, we will cover the three common use cases presented earlier and we will explain some of the unavoidable parts (like maintaining the changelog file).

The first thing to do is to change the package version number so that the rebuilt packages can be distinguished from the original packages provided by Debian or Kali. To achieve this, we usually add a suffix identifying the entity (person or company) applying the changes. Since buxy is my IRC nickname, I will use it as a suffix. Such a change is best effected with the dch command (Debian CHangelog) from the devscripts package, with a command such as dch --local buxy. This invokes a text editor (sensible-editor, which runs the editor assigned in the VISUAL or EDITOR environment variables, or /usr/bin/editor otherwise), which allows you to document the differences introduced by this rebuild. This editor shows that dch really did change the debian/changelog file:

$ head -n 1 debian/changelog
libfreefare (0.4.0-2) unstable; urgency=low
$ dch --local buxy
[...]
$ head debian/changelog
libfreefare (0.4.0-2buxy1) UNRELEASED; urgency=medium

  * Enable --with-debug configure option.

 -- Raphael Hertzog   Fri, 22 Jan 2021 10:36:00 -0400

libfreefare (0.4.0-2) unstable; urgency=low

  *  Update debian/copyrtight.
     Fix license to LGPL3+.

If you do such changes regularly, you might want to set the DEBFULLNAME and DEBEMAIL environment variables to your full name and your email, respectively. Their values will be used by many packaging tools, including dch, which will embed them on the trailer line shown above (starting with " -- ").

9.1.3.1. Applying a Patch

In one of our use cases, we have downloaded the pyrit source package and we want to apply a patch that we found in the upstream git repository. This is a common operation and it should always be simple. Unfortunately, patches can be handled in different ways depending on the source package format and on the Git packaging workflow in use (when Git is used to maintain the package).

9.1.3.1.1. With an Unpacked Source Package

You have run apt source pyrit and you have a pyrit-0.4.0 directory. You can apply your patch directly with patch -p1< patch-file:

You have run apt source pyrit and you have a pyrit-0.4.0 directory. You can apply your patch directly with patch -p1 < patch-file:

$ apt source pyrit
[...]
$ cd pyrit-0.4.0
$ wget https://github.com/JPaulMora/Pyrit/commit/14ec997174b8e8fd20d22b6a97c57e19633f12a0.patch -O /tmp/pyrit-patch
[...]
$ patch -p1

At this point, you have manually patched the source code and you can already build binary packages of your modified version (see Section 9.1.4, "Starting the Build"). But if you try to build an updated source package, it will fail, complaining about "unexpected upstream changes." This is because pyrit (like a majority of the source packages) uses the source format (see debian/source/format file) known as 3.0 (quilt), where changes to the upstream code must be recorded in separate patches stored in debian/patches/ and where the debian/patches/series file indicates the order in which patches must be applied. You can register your changes in a new patch by running dpkg-source --commit:

$ dpkg-source --commit
dpkg-source: info: local changes detected, the modified files are:
 pyrit-0.4.0/cpyrit/pckttools.py
Enter the desired patch name: fix-for-scapy-2.3.patch
dpkg-source: info: local changes have been recorded in a new patch: pyrit-0.4.0/debian/patches/fix-for-scapy-2.3.patch
$ tail -n 1 debian/patches/series
fix-for-scapy-2.3.patch

Quilt Patch Series

This patch management convention has been popularized by a tool named quilt and the "3.0 (quilt)" source package format is thus compatible with this tool—with the small deviation that it uses debian/patches instead of patches. This tool is available in the package of the same name and you can find a nice tutorial here:

https://raphaelhertzog.com/2012/08/08/how-to-use-quilt-to-manage-patches-in-debian-packages/

If the source package uses the 1.0 or 3.0 (native) source format, then there is no requirement to register your upstream changes in a patch. They are automatically bundled in the resulting source package.

9.1.3.1.2. With a Git Repository

If you have used Git to retrieve the source package, the situation is even more complicated. There are multiple Git workflows and associated tools, and obviously not all Debian packages are using the same workflows and tools. The distinction already explained about source format is still relevant but you must also check whether patches are pre-applied in the source tree or whether they are only stored in debian/patches (in this case, they are then applied at build time).

The most popular tool is git-buildpackage. It is what we use to manage all repositories on gitlab.com/kalilinux/packages. When you use it, patches are not pre-applied in the source tree but they are stored in debian/patches. You can manually add patches in that directory and list them in debian/patches/series but users of git-buildpackage tend to use gbp pq to edit the entire patch series as a single branch that you can extend or rebase to your liking. Check the manual pages for gbp-pq(1) to learn how to invoke it.

git-dpm (with associated command of the same name) is another git packaging tool that you can find in use. It records metadata in debian/.git-dpm and keeps patches applied in the source tree by merging a constantly-rebased branch that it builds out of the content of debian/patches.

9.1.3.2. Tweaking Build Options

You usually have to tweak build options when you want to enable an optional feature or behavior that is not activated in the official package, or when you want to customize parameters that are set at build time through a ./configure option or through variables set in the build environment.

In those cases, the changes are usually limited to debian/rules, which drives the steps in the package build process. In the simplest cases, the lines concerning the initial configuration (./configure ...) or the actual build ($(MAKE) ... or make ...) are easy to spot. If these commands are not explicitly called, they are probably a side effect of another explicit command, in which case, please refer to their documentation to learn more about how to change the default behavior. With packages using dh, you might need to add an override for the dh_auto_configure or dh_auto_build commands (see their respective manual pages for explanations on how to achieve this).

To make those explanations more concrete, let's apply them to our sample use case. You decided to modify libfreefare to pass the --enable-debug option to the ./configure script so that you could get a more verbose output from your near field communication (NFC) tools and file a better bug report about your non-recognized Mifare NFC card. Since the package uses dh to drive the build process, you add (or in this case modify) the override_dh_auto_configure target. Here is the corresponding extract of libfreefare's debian/rules file:

override_dh_auto_configure:
  dh_auto_configure -- --without-cutter --disable-silent-rules --enable-debug

9.1.3.3. Packaging a New Upstream Version

Let's take a look at an example at this point, as we discuss packaging upstream versions. Let's say you are a SET power-user and you noticed a new upstream release (7.4.5) that is not yet available in Kali (which only has version 7.4.4). You want to build an updated package and try it out. This is a minor version bump and you thus don't expect the update to require any change at the packaging level.

To update the source package, you extract the new source tarball next to the current source package and you copy the debian directory from the current source package to the new one. Then you bump the version in debian/changelog.

$ apt source set
Reading package lists... Done
NOTICE: 'set' packaging is maintained in the 'Git' version control system at:
https://gitlab.com/kalilinux/packages/set.git
Please use:
git clone https://gitlab.com/kalilinux/packages/set.git
to retrieve the latest (possibly unreleased) updates to the package.
Need to get 42.3 MB of source archives.
[...]
dpkg-source: warning: failed to verify signature on ./set_7.4.4-0kali1.dsc
dpkg-source: info: extracting set in set-7.4.4
dpkg-source: info: unpacking set_7.4.4.orig.tar.gz
dpkg-source: info: unpacking set_7.4.4-0kali1.debian.tar.xz
dpkg-source: info: applying edit-config-file
dpkg-source: info: applying fix-path-interpreter.patch
$ wget https://github.com/trustedsec/social-engineer-toolkit/archive/7.4.5.tar.gz -O set_7.4.5.orig.tar.gz
[...]
$ tar xvf set_7.4.5.orig.tar.gz
[...]
social-engineer-toolkit-7.4.5/src/wireless/wifiattack.py
$ cp -a set-7.4.4/debian social-engineer-toolkit-7.4.5/debian
$ cd social-engineer-toolkit-7.4.5
$ dch -v 7.4.5-0buxy1 "New upstream release"

That's it. You can now build the updated package.

Depending on the kind of changes that the new upstream version introduces, you may also need to change build dependencies and run-time dependencies, and install new files. Those are much more involved operations that are not covered by this book.

9.1.4. Starting the Build

When all the needed changes have been applied to the sources, you can start generating the actual binary package or .deb file. The whole process is managed by the dpkg-buildpackage command and it looks like this:

$ dpkg-buildpackage -us -uc -b
dpkg-buildpackage: source package libfreefare
dpkg-buildpackage: source version 0.4.0-2buxy1
dpkg-buildpackage: source distribution UNRELEASED
dpkg-buildpackage: source changed by Raphael Hertzog 
dpkg-buildpackage: host architecture amd64
[...]
   dh_builddeb
dpkg-deb: building package 'libfreefare0-dbgsym' in '../libfreefare0-dbgsym_0.4.0-2buxy1_amd64.deb'.
dpkg-deb: building package 'libfreefare0' in '../libfreefare0_0.4.0-2buxy1_amd64.deb'.
dpkg-deb: building package 'libfreefare-dev' in '../libfreefare-dev_0.4.0-2buxy1_amd64.deb'.
dpkg-deb: building package 'libfreefare-bin-dbgsym' in '../libfreefare-bin-dbgsym_0.4.0-2buxy1_amd64.deb'.
dpkg-deb: building package 'libfreefare-bin' in '../libfreefare-bin_0.4.0-2buxy1_amd64.deb'.
dpkg-deb: building package 'libfreefare-doc' in '../libfreefare-doc_0.4.0-2buxy1_all.deb'.
 dpkg-genchanges -b >../libfreefare_0.4.0-2buxy1_amd64.changes
dpkg-genchanges: binary-only upload (no source code included)
 dpkg-source --after-build libfreefare-0.4.0
dpkg-buildpackage: binary-only upload (no source included)

The -us -uc options disable signatures on some of the generated files (.dsc, .changes) because this operation will fail if you do not have a GnuPG key associated with the identity you have put in the changelog file. The -b option asks for a “binary-only build.” In this case, the source package (.dsc) will not be created, only the binary (.deb) packages will. Use this option to avoid failures during the source package build: if you haven't properly recorded your changes in the patch management system, it might complain and interrupt the build process.

As suggested by dpkg-deb's messages, the generated binary packages are now available in the parent directory (the one that hosts the directory of the source package). You can install them with dpkg -i or apt install.

$ sudo apt install ../libfreefare0_0.4.0-2buxy1_amd64.deb \
  ../libfreefare-bin_0.4.0-2buxy1_amd64.deb
Reading package lists... Done
Building dependency tree
Reading state information... Done
Note, selecting 'libfreefare0' instead of '../libfreefare0_0.4.0-2buxy1_amd64.deb'
Note, selecting 'libfreefare-bin' instead of '../libfreefare-bin_0.4.0-2buxy1_amd64.deb'
The following packages will be upgraded:
  libfreefare-bin libfreefare0
2 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/69,4 kB of archives.
After this operation, 2 048 B of additional disk space will be used.
[...]

We prefer apt install over dpkg -i as it will deal with missing dependencies gracefully. But not so long ago, you had to use dpkg as apt was not able to deal with .deb files outside of any repository.

dpkg-buildpackage wrappers

More often than not, Debian developers use a higher-level program such as debuild; this runs dpkg-buildpackage as usual, but it also adds an invocation of a program (lintian) that runs many checks to validate the generated package against the Debian policy. This script also cleans up the environment so that local environment variables do not pollute the package build. The debuild command is one of the tools in the devscripts suite, which share some consistency and configuration to make the maintainers' task easier.