Testing

For the testing section, discuss the approach developed to testing the system and describe the outcome of that testing.

During testing, a set of Test Cases are developed, where each one considers one specific part of the system that has been developed. If you have used automated tests, which you will include with the Technical Submission. If manual tests are used, it is better to include the detailed Test Case descriptions in an appendix.

This section focuses on automated testing and using test tables to document manual testing.

Automated testing

During the degree, Unit Testing frameworks such as JUnit will have been discussed. Each method is a Test Case, which describes in code how to setup the test and what the expected result is. If a tool like JUnit has been used, describe that in the report, considering how you approached the testing and where the reader will find the details of the testing in the Technical Submission.

Test frameworks like JUnit are designed for automating test cases. Despite the name, a unit testing framework can be used to automate any test cases for other types of tests such as integration tests and system tests.

Test tables

As an alternative to automated testing, it is possible to write out a test table in a document such as a Word document or a LaTeX document. A table provides details for a number of Test Cases. The table helps to understand the test situations that were considered and what results were observed.

Table 1: Example test table shows one possible structure for the table. It has the following columns:

  • Test ID – this is an identifier that is used for the test case. The example table has TC1 for the first test case. Other formats can be used for the Test ID. Whatever format is chosen, be consistent in the numbering and keep it relatively simply.

  • Reference – This is a place which refers to the requirements that are being tested.

  • Description – Provide a short description of the purpose for the test.

  • Steps – Describe what steps are necessary to perform the test. Is any setup necessary before the test can be performed? If so, the test case should describe the setup that is necessary so that someone carrying out the test knows what to do.

  • Inputs – Describe any inputs that need to be entered. This will have something for most test cases, but there will be situations where this is empty for a test case.

  • Expected Result – Describe what the expected result is. How will someone know if the test passed or failed?

  • Pass/Fail – Add a column that notes if the test case passed or failed. Try a test case and report the result. If it failed, add some comments about what failed.

A table that shows information for a test case.

Table 1: Example test table

For more complex systems, it might be useful to break the testing into separate sections. For example, a table that describes tests for the logic that manages the application and another table that describes tests for the user interface. Other tables might be relevant depending on your project.

Writing tests in a table might seem restrictive if there is a lot to say for each item. In that situation, it might be easier to list each test as a sub-section such as that shown in Figure 4: Example of displaying a test case in a section.

An alternative listing for a test case, showing the information set out in a section.

Figure 4: Example of displaying a test case in a section

Even if you are writing automated tests, there are probably some issues that are hard to test automatically. Using a mixture of automated and test tables could be appropriate.

User testing

As noted in the Report Templates, user testing is possible type of testing. However, beware of relying on it as the only type of testing. Users can give you feedback of the usability of the application, but they are unlikely to test each aspect of the system that you have developed. It is not the user’s responsibility to test how the application responds in normal situations and exceptional situations.

User testing is relevant for some projects, but not others. If you do include user testing for your project, use it as part of your strategy, but make sure you demonstrate your own testing of the code and system that you have produced.