Filing a good bug report

6.3. Filing a Good Bug Report

If all of your efforts to resolve a problem fail, it is possible that the problem is due to a bug in the program. In this case, the problem may have resulted in a bug report. You can search for bug reports to find a solution to your problem but let's take a look at the procedure of reporting a bug to Kali, Debian, or directly to the upstream developers so you understand the process should you need to submit your own report.

The goal of a bug report is to provide enough information so that the developers or maintainers of the (supposedly) faulty program can reproduce the problem, debug its behavior, and develop a fix. This means that your bug report must contain appropriate information and must be directed to the correct person or project team. The report must also be well-written and thorough, ensuring a faster response.

The exact procedure for the bug report will vary depending on where you will submit the report (Kali, Debian, upstream developers) but there are some generic recommendations that apply to all cases. In this chapter we will discuss those recommendations.

6.3.1. Generic Recommendations

Let's discuss some general recommendations and guidelines that will help you submit a bug report that is clear, comprehensive, and improves the chances that the bug will be addressed by the developers in a timely fashion.

6.3.1.1. How to Communicate

Write Your Report in English

The Free Software community is international and unless you know your interlocutor, you should be using plain English. If you are a native speaker of English, use simple sentences and avoid constructions that might be hard to understand for people with limited English skills. Even though most developers are highly intelligent, not all of them have strong English language skills. It is best never to assume.

Be Respectful of the Developers' Work

Remember that most Free Software developers (including those behind Kali Linux) are benevolent and are spending their limited free time to work on the software that you are freely using. Many are doing this out of altruism. Thus, when you file a bug report, be respectful (even if the bug looks like an obvious mistake by the developer) and don't assume that they owe you a fix. Thank them for their contribution instead.

If you know how to modify and recompile the software, offer to assist the developers in testing any patches that they submit to you. This will show them that you are willing to invest your own time as well.

Be Reactive and Ready to Provide More Information

In some cases, the developer will come back to you with requests for more information or requests for you to try to re-create the problem perhaps by using different options or using an updated package. You should try to respond to those queries as quickly as possible. The quicker you submit your response, the higher the chance that they will be able to solve it quickly while the initial analysis is still fresh in their mind.

While you should aim to respond quickly, you should also not go too fast: the data submitted must be correct and it must contain everything that the developers requested. They will be annoyed if they have to request something a second time.

6.3.1.2. What to Put in the Bug Report

Instructions to Reproduce the Problem

To be able to reproduce the issue, the developers need to know what you are using, where you got it from, and how you installed it.

You should provide precise, step-by-step instructions describing how to reproduce the problem. If you need to use some data to reproduce the problem, attach the corresponding file to the bug report. Try to come up with the minimal set of instructions needed to reproduce the bug.

Give Some Context and Set Your Expectations

Explain what you were trying to do and how you expected the program to behave.

In some cases, the bug is only triggered because you were using the program in a way that it was not designed to operate by the developers. By explaining what you were trying to achieve, you will allow the developers to clearly see when this is the case.

In some other cases, the behavior that you describe as a bug might actually be the normal behavior. Be explicit about what you expected the program to do. This will clarify the situation for the developers. They may either improve the behavior or improve the documentation, but at least they know that the behavior of their program is confusing some users!

Be Specific

Include the versions numbers of the software that you use, possibly with the version numbers of their dependencies. When you refer to something that you downloaded, include its complete URL.

When you get an error message, quote it exactly as you saw it. If possible, include a copy of your screen output or a screenshot. Include a copy of any relevant log file, ensuring that you remove any sensitive data first.

Mention Possible Fixes or Workarounds

Before filing the bug report, you probably tried to resolve the problem. Explain what you tried and what results you received. Be very clear about what is a fact and what was just a hypothesis on your part.

If you did an Internet search and found some explanations about a similar problem, you can mention them, in particular when you found other similar bug reports in the Debian bug tracker or in the upstream bug tracker.

If you found a way of achieving the desired result without triggering the bug, please document that as well. This will help other users who are hit by the same issue.

Long Bug Reports Are Fine

A two-line bug report is insufficient; providing all the information needed usually requires several paragraphs (or sometimes pages) of text.

Supply all the information you can. Try to stick to what is relevant, but if you are uncertain, too much is better than too little.

If your bug report is really long, take some time to structure the content and provide a short summary at the start.

6.3.1.3. Miscellaneous Tips

Avoid Filing Duplicate Bug Reports

In the Free Software world, all bug trackers are public. Open issues can be browsed and they even have a search feature. Thus, before filing a new bug report, try to determine if your problem has already been reported by someone else.

If you find an existing bug report, subscribe to it and possibly add supplementary information. Do not post comments such as "Me too" or "+1"; they serve no purpose. But you can indicate that you are available for further tests if the original submitter did not offer this.

If you have not found any report of your problem, go ahead and file it. If you have found related tickets, be sure to mention them.

Ensure You Use the Latest Version

It is very frustrating for developers to receive bug reports for problems that they have already solved or problems that they can't reproduce with the version that they are using (developers almost always use the latest version of their product). Even when older versions are maintained by the developers, the support is often limited to security fixes and major problems. Are you sure that your bug is one of those?

That is why, before filing a bug report, you should make sure that you are using the latest version of the problematic system and application and that you can reproduce the problem in that situation.

If Kali Linux does not offer the latest version of the application (neither in kali-rolling nor in kali-bleeding-edge, see Section 8.1.3.3, "The Kali-Bleeding-Edge Repository"), you have alternative solutions: you can try a manual installation of the latest version in a throw-away virtual machine, or you can review the upstream ChangeLog (or Git commit history) to see that there hasn't been any change that could fix the problem that you are seeing (and then file the bug even though you did not try the latest version).

Do Not Mix Multiple Issues in a Single Bug Report

File one bug report per issue. That way, the subsequent discussions do not get too messy and each bug can be fixed according to its own schedule. If you don't do that, either the single bug needs to be repurposed multiple times and can only be closed when all issues have been fixed, or the developers must file the supplementary reports that you should have created in the first place.

6.3.2. Where to File a Bug Report

To be able to decide where to file the bug report, you must have a good understanding of the problem and you must have identified in which piece of software the problem lies.

Ideally, you track the problem down to a file on your system and then you can use dpkg to find out which package owns that file and where that package comes from. Let's assume that you found a bug in a graphical application. After looking at the list of running processes (the output of ps auxf), you discovered that the application was started with the /usr/bin/sparta executable:

You learn that /usr/bin/sparta is provided by the sparta package, which is in version 1.0.1+git20150729-0kali1. The fact that the version string contains kali indicates to you that the package comes from Kali Linux (or is modified by Kali Linux). Any package that does not have kali in its version string (or in its package name) comes straight from Debian (Debian Testing in general).

Double Check Before Filing Bugs against Debian


If you find a bug in a package imported straight from Debian, it should ideally be reported and fixed on the Debian side. However, before doing this, ensure that the problem is reproducible on a plain Debian system since Kali may have caused the problem by modifying other packages or dependencies.

The easiest way to accomplish this is to setup a virtual machine running Debian Testing. You can find an installation ISO for Debian Testing on the Debian Installer website:

https://www.debian.org/devel/debian-installer/

If you can confirm the problem in the virtual machine, then you can submit the bug to Debian by running reportbug within the virtual machine and following the instructions provided.

Most bug reports about the behavior of applications should be directed to their upstream projects except when facing an integration problem: in that case, the bug is a mistake in the way the software gets packaged and integrated into Debian or Kali. For example, if an application offers compile-time options that the package does not enable or the application does not work because of a missing library (thus putting into light a missing dependency in the package meta-information), you may be facing an integration problem. When you don't know what kind of problem you face, it is usually best to file the issue on both sides and to cross-reference them.

Identifying the upstream project and finding where to file the bug report is usually easy. You just have to browse the upstream website, which is referenced in the Homepage field of the packaging meta-data:

6.3.3. How to File a Bug Report

6.3.3.1. Filing a Bug Report in Kali

Kali uses a web-based bug tracker at http://bugs.kali.org where you can consult all the bug reports anonymously, but if you would like to comment or file a new bug report, you will need to register an account.

6.3.3.1.1. Signing Up for a Bug Tracker Account

To begin, simply click Signup for new account on the bug tracker website, as shown in Figure 6.1, "Kali Bug Tracker Start Page".

Kali Bug Tracker Example

Figure 6.1. Kali Bug Tracker Start Page

Next, provide a username, e-mail address, and response to the CAPTCHA challenge. Then click the Signup button to proceed (Figure 6.2, "Signup Page").

Figure 6.2. Signup Page

If successful, the next page (Figure 6.3, "Signup Confirmation Page") will notify you that the account registration has been processed, and the bug tracker system will send a confirmation email to the address you provided. You will need to click the link in the email in order to activate your account.

Once your account has been activated, click Proceed to continue to the bug tracker login page.

Bug Tracker Sign-up Response

Figure 6.3. Signup Confirmation Page

6.3.3.1.2. Creating the Report

To begin your report, log into your account and click the Report Issue link on the landing page. You will be presented a form with many fields to fill, as shown in Figure 6.4, "Form to report a bug".

Figure 6.4. Form to report a bug

Here is a rundown of all the fields on the form:

Category (mandatory)

This field describes the category of the bug you are submitting. Reports that can be attributed to a specific package should be filed in the Kali Package Bug or Kali Package Improvement categories. Other reports should use the General Bug or Feature Requests categories. The remaining categories are for specific use cases: Tool Upgrade can be used to notify the Kali developers of the availability of a new version of a software packaged in Kali. New Tool Requests can be used to suggest new tools to package and integrate in the Kali distribution.

Reproducibility

This field documents whether the problem is reproducible in a predictable way or if it happens only somewhat randomly.

Severity and Priority

Those fields are best left unmodified as they are mainly for the developers. They can use them to sort the list of issues according to the severity of the problem and to the priority at which it must be handled.

Product Version

This field should indicate what version of Kali Linux you are running (or the one which is the closest to what you are running). Think twice before reporting an issue on an old release that is no longer supported.

Summary (mandatory)

This is essentially the title of your bug report and it is the first thing that people will see. Make sure that it conveys the reason why you are filing the report. Avoid generic descriptions like "X doesn't work" and opt instead for "X fails with error Y under condition Z."

Description (mandatory)

This is the body of your report. Here you should enter all of the information you collected about the problem that you are experiencing. Don't forget all the recommendations given in the former section.

Steps to Reproduce

In this field, list all the detailed instructions explaining how to trigger the problem.

Additional Information

In this section, you can provide any additional information you believe is relevant to the issue. If you have a fix or workaround for the issue, please provide it in this section.

Upload File

Not everything can be explained with plain text. This field lets you attach arbitrary files to your reports: screenshots to show the error, sample documents triggering the problem, log files, etc.

View Status

Leave that field set to "public" so that everybody can see your bug report. Use "private" only for security-related reports containing information about undisclosed security vulnerabilities.

6.3.3.2. Filing a Bug Report in Debian

Debian uses a (mostly) email-based bug tracking system known as Debbugs. To open a new bug report, you will send an email (with a special syntax) to submit@bugs.debian.org. This will allocate a bug number XXXXXX and inform you that you can send additional information by mailing XXXXXX@bugs.debian.org. Each bug is associated to a Debian package. You can browse all the bugs of a given package (including the bug that you are thinking of reporting) at https://bugs.debian.org/package. You can check the history of a given bug at https://bugs.debian.org/XXXXXX.

6.3.3.2.1. Setting Up Reportbug

While you can open a new bug with a simple e-mail, we recommend using reportbug because it will help you draft a solid bug report with all the required information. Ideally, you should run it from a Debian system (for example, in the virtual machine where you reproduced the problem).

The first run of reportbug starts a configuration script. First, select a skill level. You should choose Novice or Standard; we use the latter because it offers more fine-grained control. Next, select an interface and enter your personal details. Finally, select a user interface. The configuration script will allow you to use a local mail transport agent, an SMTP server, or as a last resort, a Debian SMTP server.

6.3.3.2.2. Using Reportbug

With the setup phase completed, the actual bug report can begin. You will be prompted for a package name, although you can also provide the package name directly on the command line with reportbug package).

Contrary to the advice given above, if you don't know against which package to file the bug, you should get in touch with a Kali support forum (described in Section 6.2, "Kali Linux Communities"). In the next step, reportbug downloads the list of bugs filed against the given package and lets you browse them to see if you can find yours.

If you find your bug already filed, you can choose to send supplementary information, otherwise, you are invited to file a new bug report:

After providing a one-line summary of your problem, you must rate its severity along an extended scale:

If you are unsure, just keep the default severity of normal.

You can also tag your report with a few keywords:

Most tags are rather esoteric, but if your report includes a fix, you should select the patch tag.

Once this is completed, reportbug opens a text editor with a template that you should edit (Example 6.2, "Template generated by reportbug"). It contains a few questions that you should delete and answer, as well as some information about your system that has been automatically collected. Notice how the first few lines are structured. They should not be modified as they will be parsed by the bug tracker to assign the report to the correct package.

Example 6.2. Template generated by reportbug

Once you save the report and close the text editor, you return to reportbug, which provides many other options and offers to send the resulting report.

6.3.3.3. Filing a Bug Report in Another Free Software Project

There is a large diversity of free software projects, using different workflows and tools. This diversity also applies to the bug trackers in use. While many projects are hosted on GitHub and use GitHub Issues to track their bugs, there are also many others hosting their own trackers, based on Bugzilla, Trac, Redmine, Flyspray, and others. Most of them are web-based and require you to register an account to submit a new ticket.

We will not cover all the trackers here. It is up to you to learn the specifics of various trackers for other free software projects, but since GitHub is relatively popular, we will take a brief look at it here. As with other trackers, you must first create an account and sign in. Next, click the Issues tab, as shown in Figure 6.5, "Main page of a GitHub project".

Figure 6.5. Main page of a GitHub project

You can then browse (and search) the list of open issues. Once you are confident that your bug is not yet filed, you can click on the New issue button (Figure 6.6, "Issues page of a GitHub project").

Figure 6.6. Issues page of a GitHub project

You are now on a page where you must describe your problem (Figure 6.7, "GitHub form to file a new issue"). Although there is no template like the one found in reportbug, the bug reporting mechanism is fairly straight-forward, allowing you to attach files, apply formatting to text, and much more. Of course, for best results, be sure to follow our guidelines for creating a detailed and well-described report.

Figure 6.7. GitHub form to file a new issue