Filing a good bug report

Topic Progress:

6.3. Filing a Good Bug Report

If all of your efforts to resolve a problem fail, it is possible that the problem is due to a bug in the program. In this case, the problem may have resulted in a bug report. You can search for bug reports to find a solution to your problem but let's take a look at the procedure of reporting a bug to Kali, Debian, or directly to the upstream developers so you understand the process should you need to submit your own report.

The goal of a bug report is to provide enough information so that the developers or maintainers of the (supposedly) faulty program can reproduce the problem, debug its behavior, and develop a fix. This means that your bug report must contain appropriate information and must be directed to the correct person or project team. The report must also be well-written and thorough, ensuring a faster response.

The exact procedure for the bug report will vary depending on where you will submit the report (Kali, Debian, or upstream developers) but there are some generic recommendations that apply to all cases. In this chapter we will discuss those recommendations.

6.3.1. Generic Recommendations

Let's discuss some general recommendations and guidelines that will help you submit a bug report that is clear, comprehensive, and improves the chances that the bug will be addressed by the developers in a timely fashion.

6.3.1.1. How to Communicate

Write Your Report in English

The Free Software community is international and unless you know your interlocutor, you should be using plain English. If you are a native speaker of English, use simple sentences and avoid constructions that might be hard to understand for people with limited English skills. Even though most developers are highly intelligent, not all of them have strong English language skills. It is best never to assume.

Be Respectful of the Developers' Work

Remember that most Free Software developers (including those behind Kali Linux) are benevolent and are spending their limited free time to work on the software that you are freely using. Many are doing this out of altruism. Thus, when you file a bug report, be respectful (even if the bug looks like an obvious mistake by the developer) and don't assume that they owe you a fix. Thank them for their contribution instead.

If you know how to modify and recompile the software, offer to assist the developers in testing any patches that they submit to you. This will show them that you are willing to invest your own time as well.

Be Reactive and Ready to Provide More Information

In some cases, the developer will come back to you with requests for more information or requests for you to try to re-create the problem perhaps by using different options or using an updated package. You should try to respond to those queries as quickly as possible. The quicker you submit your response, the higher the chance that they will be able to solve it quickly while the initial analysis is still fresh in their mind.

While you should aim to respond quickly, you should also not go too fast: the data submitted must be correct and it must contain everything that the developers requested. They may be annoyed if they have to request something a second time.

6.3.1.2. What to Put in the Bug Report

Instructions to Reproduce the Problem

To be able to reproduce the issue, the developers need to know what you are using, where you got it from, and how you installed it.

You should provide precise, step-by-step instructions describing how to reproduce the problem. If you need to use some data to reproduce the problem, attach the corresponding file to the bug report. Try to come up with the minimal set of instructions needed to reproduce the bug.

Give Some Context and Set Your Expectations

Explain what you were trying to do and how you expected the program to behave.

In some cases, the bug is only triggered because you were using the program in a way that it was not designed to operate by the developers. By explaining what you were trying to achieve, you will allow the developers to clearly see when this is the case.

In some other cases, the behavior that you describe as a bug might actually be the normal behavior. Be explicit about what you expected the program to do. This will clarify the situation for the developers. They may either improve the behavior or improve the documentation, but at least they know that the behavior of their program is confusing some users!

Be Specific

Include the versions numbers of the software that you use, possibly with the version numbers of their dependencies. When you refer to something that you downloaded, include its complete URL.

When you get an error message, quote it exactly as you saw it. If possible, include a copy of your screen output or a screenshot. Include a copy of any relevant log file, ensuring that you remove any sensitive data first.

Mention Possible Fixes or Workarounds

Before filing the bug report, you probably tried to resolve the problem. Explain what you tried and what results you received. Be very clear about what is a fact and what was just a hypothesis on your part.

If you did an Internet search and found some explanations about a similar problem, you can mention them, in particular when you found other similar bug reports in bug trackers.

If you found a way of achieving the desired result without triggering the bug, please document that as well. This will help other users who are hit by the same issue.

Long Bug Reports Are Fine

A two-line bug report is insufficient; providing all the information needed usually requires several paragraphs (or sometimes pages) of text.

Supply all the information you can. Try to stick to what is relevant, but if you are uncertain, too much is better than too little.

If your bug report is really long, take some time to structure the content and provide a short summary at the start.

6.3.1.3. Miscellaneous Tips

Avoid Filing Duplicate Bug Reports

In the Free Software world, all bug trackers are public. Open issues can be browsed and they even have a search feature. Thus, before filing a new bug report, try to determine if your problem has already been reported by someone else.

If you find an existing bug report, subscribe to it and possibly add supplementary information. Do not post comments to bump, such as “Me too” or “+1”; they serve no purpose. But you can indicate that you are available for further tests if the original submitter did not offer this.

If you have not found any report of your problem, go ahead and file it. If you have found related tickets, be sure to mention them.

Ensure You Use the Latest Version

It is very frustrating for developers to receive bug reports for problems that they have already solved or problems that they can't reproduce with the version that they are using (developers almost always use the latest version of their product). Even when older versions are maintained by the developers, the support is often limited to security fixes and major problems. Are you sure that your bug is one of those?

That is why, before filing a bug report, you should make sure that you are using the latest version of the problematic system and application and that you can reproduce the problem in that situation.

If Kali Linux does not offer the latest version of the application, you have alternative solutions: you can try a manual installation of the latest version in a throw-away virtual machine, or you can review the upstream Changelog (or any history logs in their chosen version control system) to see that there hasn't been any change that could fix the problem that you are seeing (and then file the bug even though you did not try the latest version).

Do Not Mix Multiple Issues in a Single Bug Report

File one bug report per issue. That way, the subsequent discussions do not get too messy and each bug can be fixed according to its own schedule. If you don't do that, either the single bug needs to be repurposed multiple times and can only be closed when all issues have been fixed, or the developers must file the supplementary reports that you should have created in the first place.

6.3.2. Where to File a Bug Report

To be able to decide where to file the bug report, you must have a good understanding of the problem and you must have identified in which piece of software the problem lies.

Ideally, you track the problem down to a file on your system and then you can use dpkg to find out which package owns that file and where that package comes from. Let's assume that you found a bug in a graphical application. After looking at the list of running processes (the output of ps auxf), you discovered that the application was started with the /usr/bin/cherrytree executable:

$ dpkg -S /usr/bin/cherrytree
cherrytree: /usr/bin/cherrytree
$ dpkg -s cherrytree | grep ^Version:
Version: 0.38.8-0kali1

You learn that /usr/bin/cherrytree is provided by the cherrytree package, which is in version 0.38.8-0kali1. The fact that the version string contains kali indicates to you that the package comes from Kali Linux (or is modified by Kali Linux). Any package that does not have kali in its version string (or in its package name) comes straight from Debian (Debian Testing in general).

Double Check Before Filing Bugs against Debian


If you find a bug in a package imported straight from Debian, it should ideally be reported and fixed on the Debian side. However, before doing this, ensure that the problem is reproducible on a plain Debian system since Kali may have caused the problem by modifying other packages or dependencies.

The easiest way to accomplish this is to setup a virtual machine running Debian Testing. You can find an installation ISO for Debian Testing on the Debian Installer website:

https://www.debian.org/devel/debian-installer/

If you can confirm the problem in the virtual machine, then you can submit the bug to Debian by running reportbug within the virtual machine and following the instructions provided.

Most bug reports about the behavior of applications should be directed to their upstream projects except when facing an integration problem: in that case, the bug is a mistake in the way the software gets packaged and integrated into Debian or Kali. For example, if an application offers compile-time options that the package does not enable or the application does not work because of a missing library (thus putting into light a missing dependency in the package meta-information), you may be facing an integration problem. When you don't know what kind of problem you face, it is usually best to file the issue on both sides and to cross-reference them.

Identifying the upstream project and finding where to file the bug report is usually easy. You just have to browse the upstream website, which is referenced in the Homepage field of the packaging meta-data:

$ dpkg -s wpscan | grep ^Homepage:
Homepage: http://www.wpscan.org

6.3.3. How to File a Bug Report

6.3.3.1. Filing a Bug Report in Kali

Kali uses a web-based bug tracker at https://bugs.kali.org/ where you can consult all the bug reports anonymously, but if you would like to comment or file a new bug report, you will need to register an account.

6.3.3.1.1. Signing Up for a Bug Tracker Account

To begin, simply click Signup for new account on the bug tracker website, as shown in Figure 6.1, "Kali Bug Tracker Start Page".

Kali Bug Tracker Start Page
Figure 6.1. Kali Bug Tracker Start Page

Next, provide a username, e-mail address, and response to the CAPTCHA challenge. Then click the Signup button to proceed (Figure 6.2, "Signup Page").

Signup Page
Figure 6.2. Signup Page

If successful, the next page (Figure 6.3, "Signup Confirmation Page") will notify you that the account registration has been processed, and the bug tracker system will send a confirmation email to the address you provided. You will need to click the link in the email in order to activate your account.

Once your account has been activated, click Proceed to continue to the bug tracker login page.

Signup Confirmation Page
Figure 6.3. Signup Confirmation Page

6.3.3.1.2. Creating the Report

To begin your report, log into your account and click the Report Issue link on the landing page. You will be presented a form with many fields to fill, as shown in Figure 6.4, "Form to report a bug".

Form to report a bug
Figure 6.4. Form to report a bug

Here is a rundown of all the fields on the form:

Category (mandatory)

This field describes the category of the bug you are submitting. Reports that can be attributed to a specific package should be filed in the Kali Package Bug or Kali Package Improvement categories. Other reports should use the General Bug or Feature Requests categories. The remaining categories are for specific use cases: Tool Upgrade can be used to notify the Kali developers of the availability of a new version of a software packaged in Kali. New Tool Requests can be used to suggest new tools to package and integrate in the Kali distribution. Kali Websites & Docs can be used to report bugs or updates relating to the various Kali websites. Queued Tool Addition is reserved for the Kali tool team, for when a tool submission has been agreed-upon to be added into Kali.

Reproducibility

This field documents whether the problem is reproducible in a predictable way or if it happens only somewhat randomly.

Severity and Priority

Those fields are best left unmodified as they are mainly for the developers. They can use them to sort the list of issues according to the severity of the problem and to the priority at which it must be handled.

Product Version

This field should indicate what version of Kali Linux you are running (or the one which is the closest to what you are running). Think twice before reporting an issue on an old release that is no longer supported.

Assign To and Target Version

Those fields are also best left unmodified as they are again, mainly for the developers. They can use them to indicate which developer is handling the issue and when the issue should be resolved.

Summary (mandatory)

This is essentially the title of your bug report and it is the first thing that people will see. Make sure that it conveys the reason why you are filing the report. Avoid generic descriptions like "X doesn't work" and opt instead for "X fails with error Y under condition Z."

Description (mandatory)

This is the body of your report. Here you should enter all of the information you collected about the problem that you are experiencing. Don't forget all the recommendations given in the former section.

Steps to Reproduce

In this field, list all the detailed instructions explaining how to trigger the problem.

Additional Information

In this section, you can provide any additional information you believe is relevant to the issue. If you have a fix or workaround for the issue, please provide it in this section.

Attach Tags

This is a section better left unmodified. Tags are used by the developers to allow easier access to similar bug reports.

Upload File

Not everything can be explained with plain text. This field lets you attach arbitrary files to your reports: screenshots to show the error, sample documents triggering the problem, log files, etc.

View Status

Leave that field set to "public" so that everybody can see your bug report. Use "private" only for security-related reports containing information about undisclosed security vulnerabilities.

6.3.3.2. Filing a Bug Report in Debian

Debian uses a (mostly) email-based bug tracking system known as Debbugs. To open a new bug report, you will send an email (with a special syntax) to submit@bugs.debian.org. This will allocate a bug number XXXXXX and inform you that you can send additional information by mailing XXXXXX@bugs.debian.org. Each bug is associated to a Debian package. You can browse all the bugs of a given package (including the bug that you are thinking of reporting) at https://bugs.debian.org/package. You can check the history of a given bug at https://bugs.debian.org/XXXXXX.

6.3.3.2.1. Setting Up Reportbug

While you can open a new bug with a simple e-mail, we recommend using reportbug because it will help you draft a solid bug report with all the required information. Ideally, you should run it from a Debian system (for example, in the virtual machine where you reproduced the problem).

The first run of reportbug starts a configuration script. First, select a skill level. You should choose Novice or Standard; we use the latter because it offers more fine-grained control. Next, select an interface and enter your personal details. Finally, select a user interface. The configuration script will allow you to use a local mail transport agent, an SMTP server, or as a last resort, a Debian SMTP server.

Welcome to reportbug! Since it looks like this is the first time you have
used reportbug, we are configuring its behavior. These settings will be
saved to the file "/home/kali/.reportbugrc", which you will be free to edit
further.
Please choose the default operating mode for reportbug.

1 novice    Offer simple prompts, bypassing technical questions.

2 standard  Offer more extensive prompts, including asking about things
            that a moderately sophisticated user would be expected to
            know about Debian.

3 advanced  Like standard, but assumes you know a bit more about Debian,            including "incoming".

4 expert    Bypass most handholding measures and preliminary triage
            routines. This mode should not be used by people unfamiliar
            with Debian's policies and operating procedures.

Select mode: [novice] standard
Please choose the default interface for reportbug.

1 text   A text-oriented console user interface

2 gtk2   A graphical (GTK+) user interface.

3 urwid  A menu-based console user interface

Select interface: text
Will reportbug often have direct Internet access? (You should answer
yes to this question unless you know what you are doing and plan to
check whether duplicate reports have been filed via some other channel.)
[Y|n|q|?]? Y
What real name should be used for sending bug reports?
[kali]> Raphaël Hertzog
Which of your email addresses should be used when sending bug reports?
(Note that this address will be visible in the bug tracking system, so you
may want to use a webmail address or another address with good spam
filtering capabilities.)
[kali@localhost.localdomain]> buxy@kali.org
Do you have a "mail transport agent" (MTA) like Exim, Postfix or SSMTP
configured on this computer to send mail to the Internet? [y|N|q|?]? N
Please enter the name of your SMTP host. Usually it's called something
like "mail.example.org" or "smtp.example.org". If you need to use a
different port than default, use the : alternative
format. Just press ENTER if you don't have one or don't know, and so a
Debian SMTP host will be used.
>
Please enter the name of your proxy server. It should only use this
parameter if you are behind a firewall. The PROXY argument should be
formatted as a valid HTTP URL, including (if necessary) a port number; for
example, http://192.168.1.1:3128/. Just press ENTER if you don't have one
or don't know.
>
Default preferences file written. To reconfigure, re-run reportbug with
the "--configure" option.
6.3.3.2.2. Using Reportbug

With the setup phase completed, the actual bug report can begin. You will be prompted for a package name, although you can also provide the package name directly on the command line with reportbug package).

Please enter the name of the package in which you have found a problem, or
type 'other' to report a more general problem. If you don't know what
package the bug is in, please contact debian-user@lists.debian.org for
assistance.
> wireshark

Contrary to the advice given above, if you don't know against which package to file the bug, you should get in touch with a Kali support forum (described in Section 6.2, "Kali Linux Communities"). In the next step, reportbug downloads the list of bugs filed against the given package and lets you browse them to see if you can find yours.

*** Welcome to reportbug.  Use ? for help at prompts. ***
Note: bug reports are publicly archived (including the email address of the submitter).
Detected character set: UTF-8
Please change your locale if this is incorrect.

Using '"Raphaël Hertzog" ' as your from address.
Getting status for wireshark...
Verifying package integrity...
Checking for newer versions at madison...
Will send report to Debian (per lsb_release).
Querying Debian BTS for reports on wireshark (source)...
35 bug reports found:

Bugs with severity important
   1) #478200  tshark: seems to ignore read filters when writing to...
   2) #776206  mergecap: Fails to create output file > 2GB
   3) #780089  wireshark: "On gnome wireshark has not title bar. Does...
Bugs with severity normal
   4) #151017  ethereal: "Protocol Hierarchy Statistics" give misleading...
   5) #275839  doesn't correctly dissect ESMTP pipelining
[...]
  35) #815122  wireshark: add OID 1.3.6.1.4.1.11129.2.4.2
(24-35/35) Is the bug you found listed above [y|N|b|m|r|q|s|f|e|?]? ?
y - Problem already reported; optionally add extra information.
N - (default) Problem not listed above; possibly check more.
b - Open the complete bugs list in a web browser.
m - Get more information about a bug (you can also enter a number
    without selecting "m" first).
r - Redisplay the last bugs shown.
q - I'm bored; quit please.
s - Skip remaining problems; file a new report immediately.
f - Filter bug list using a pattern.
e - Open the report using an e-mail client.
? - Display this help.
(24-35/35) Is the bug you found listed above [y|N|b|m|r|q|s|f|e|?]? n
Maintainer for wireshark is 'Balint Reczey '.
Looking up dependencies of wireshark...

If you find your bug already filed, you can choose to send supplementary information, otherwise, you are invited to file a new bug report:

Briefly describe the problem (max. 100 characters allowed). This will be the bug email subject, so keep the summary as concise as possible, for example: "fails to send email"
or "does not start with -q option specified" (enter Ctrl+c to exit reportbug without reporting a bug).
> does not dissect protocol foobar
Rewriting subject to 'wireshark: does not dissect protocol foobar'

After providing a one-line summary of your problem, you must rate its severity along an extended scale:

How would you rate the severity of this problem or report?

1 critical        makes unrelated software on the system (or the whole system) break, or causes serious data loss, or introduces a security hole on systems where you install
                  the package.
2 grave           makes the package in question unusable by most or all users, or causes data loss, or introduces a security hole allowing access to the accounts of users who
                  use the package.
3 serious         is a severe violation of Debian policy (that is, the problem is a violation of a 'must' or 'required' directive); may or may not affect the usability of the
                  package. Note that non-severe policy violations may be 'normal,' 'minor,' or 'wishlist' bugs. (Package maintainers may also designate other bugs as
                  'serious' and thus release-critical; however, end users should not do so.). For the canonical list of issues deserving a serious severity you can refer to
                  this webpage: http://release.debian.org/testing/rc_policy.txt .
4 important       a bug which has a major effect on the usability of a package, without rendering it completely unusable to everyone.
5 does-not-build  a bug that stops the package from being built from source. (This is a 'virtual severity'.)
6 normal          a bug that does not undermine the usability of the whole package; for example, a problem with a particular option or menu item.
7 minor           things like spelling mistakes and other minor cosmetic errors that do not affect the core functionality of the package.
8 wishlist        suggestions and requests for new features.

Please select a severity level: [normal]

If you are unsure, just keep the default severity of normal.

You can also tag your report with a few keywords:

Do any of the following apply to this report?

 1 a11y      This bug is relevant to the accessibility of the package.
 2 d-i       This bug is relevant to the development of debian-installer.
 3 ftbfs     The package fails to build from source.
 4 ipv6      This bug affects support for Internet Protocol version 6.
 5 l10n      This bug reports a localization/internationalization issue.
 6 lfs       This bug affects support for large files (over 2 gigabytes).
 7 newcomer  This bug has a known solution but the maintainer requests someone else implement it.
 8 patch     You are including a patch to fix this problem.
 9 upstream  This bug applies to the upstream part of the package.
10 none

Please select tags: (one at a time) [none]

Most tags are rather esoteric, but if your report includes a fix, you should select the patch tag.

Once this is completed, reportbug opens a text editor with a template that you should edit (Example 6.2, "Template generated by reportbug"). It contains a few questions that you should delete and answer, as well as some information about your system that has been automatically collected. Notice how the first few lines are structured. They should not be modified as they will be parsed by the bug tracker to assign the report to the correct package.

Example 6.2. Template generated by reportbug

Subject: wireshark: does not dissect protocol foobar

Package: wireshark
Version: 3.2.5-1
Severity: normal

Dear Maintainer,

*** Reporter, please consider answering these questions, where appropriate ***

   * What led up to the situation?
   * What exactly did you do (or not do) that was effective (or
     ineffective)?
   * What was the outcome of this action?
   * What outcome did you expect instead?

*** End of the template - remove these template lines ***


-- System Information:
Distributor ID: Kali
Description:    Kali GNU/Linux Rolling
Release:        2020.3
Codename:       kali-rolling
Architecture: x86_64

Kernel: Linux 5.7.0-kali1-amd64 (SMP w/4 CPU threads)
Kernel taint flags: TAINT_UNSIGNED_MODULE
Locale: LANG=en_US.utf8, LC_CTYPE=en_US.utf8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages wireshark depends on:
ii  wireshark-qt  3.2.5-1

wireshark recommends no packages.

wireshark suggests no packages.

-- no debconf information

Once you save the report and close the text editor, you return to reportbug, which provides many other options and offers to send the resulting report.

Spawning sensible-editor...
Report will be sent to "Debian Bug Tracking System" 
Submit this report on wireshark (e to edit) [Y|n|a|c|e|i|l|m|p|q|d|t|s|?]? ?
Y - (default) Submit the bug report via email.
n - Don't submit the bug report; instead, save it in a temporary file (exits reportbug).
a - Attach a file.
c - Change editor and re-edit.
e - Re-edit the bug report.
i - Include a text file.
l - Pipe the message through the pager.
m - Choose a mailer to edit the report.
p - print message to stdout.
q - Save it in a temporary file and quit.
d - Detach an attachment file.
t - Add tags.
s - Add a X-Debbugs-CC recipient (a CC but after BTS processing).
? - Display this help.
Submit this report on wireshark (e to edit) [Y|n|a|c|e|i|l|m|p|q|d|t|s|?]? Y
Saving a backup of the report at /tmp/reportbug-wireshark-backup-20210328-19073-87oJWJ
Connecting to reportbug.debian.org via SMTP...

Bug report submitted to: "Debian Bug Tracking System" 
Copies will be sent after processing to:
  buxy@kali.org

If you want to provide additional information, please wait to receive the
bug tracking number via email; you may then send any extra information to
n@bugs.debian.org (e.g. 999999@bugs.debian.org), where n is the bug
number. Normally you will receive an acknowledgement via email including
the bug report number within an hour; if you haven't received a
confirmation, then the bug reporting process failed at some point
(reportbug or MTA failure, BTS maintenance, etc.).

6.3.3.3. Filing a Bug Report in Another Free Software Project

There is a large diversity of free software projects, using different workflows and tools. This diversity also applies to the bug trackers in use. While many projects are hosted on GitHub and use GitHub Issues to track their bugs, there are also many others hosting their own trackers, based on Bugzilla, Trac, Redmine, Flyspray, and others. Most of them are web-based and require you to register an account to submit a new ticket.

We will not cover all the trackers here. It is up to you to learn the specifics of various trackers for other free software projects, but since GitHub is relatively popular, we will take a brief look at it here. As with other trackers, you must first create an account and sign in. Next, click the Issues tab, as shown in Figure 6.5, "Main page of a GitHub project".

Main page of a GitHub project
Figure 6.5. Main page of a GitHub project

You can then browse (and search) the list of open issues. Once you are confident that your bug is not yet filed, you can click on the New issue button (Figure 6.6, "Issues page of a GitHub project").

Issues page of a GitHub project
Figure 6.6. Issues page of a GitHub project

You are now on a page where you must describe your problem (Figure 6.7, “GitHub form to file a new issue”). GitHub has a template feature allowing the owner of the repository to define their own custom issue template, however it appears that this repository does not have one setup. However, the bug reporting mechanism is fairly straight-forward, allowing you to attach files, apply formatting to text, and much more. Of course, for best results, be sure to follow our guidelines for creating a detailed and well-described report.

GitHub form to file a new issue
Figure 6.7. GitHub form to file a new issue