Software Testing: The Basics
This article is aimed at undergraduates who are learning to program. Whilst it is aimed at complete beginners, it also provides a useful reminder for more experienced programmers.
This article covers the following basic questions about software testing:
- What is testing?
- Why do we test programs?
- Who does the testing?
- How do we test programs?
- When do we test programs?
What is testing?
The general idea of testing a software program is simple: we run the program in order to try and find out whether there is anything wrong with it.
What sorts of things could be wrong with a program? All kinds of things. And there are several different types of software testing, to test for different things that can be wrong with a program. For example,
- Functionality testing explores whether the software has the correct functionality or not. That is, does the software do what it is supposed to do?
- Performance testing measures how well the software performs, for example how fast it runs, or what workload it can handle without crashing.
- Usability testing explores whether the software is easy to use ("user-friendly") or not.
- Accessibility testing explores whether people with disabilities can successfully use the software.
The remainder of this article concentrates on functionality testing, which is the most common kind of testing carried out when learning to program.
Why do we test programs?
The overall aim of software testing is to try and improve the quality of the software. Testing aims to finds defects in the program code (these defects are known as bugs), so that the bugs can be fixed (this fixing is known as debugging).
Bugs are very common indeed in software. If a program is sufficiently long to do something interesting, the chances are good that it has a bug in it somewhere! It is not just learner programmers that make mistakes in their coding; even very experienced programmers make mistakes too. If this seems surprising, think of it this way: whilst experienced programmers may make fewer elementary coding mistakes, they can still make subtle hard-to-spot mistakes! Also, experienced programmers typically write code forming part of large programs, and large programs have plenty of places for bugs to hide, especially bugs of the subtle sneaky kind.
So a good rule-of-thumb is that all software contains bugs, and software testing does its best to find them!
Be careful, though: just because you test a program doesn't mean that it is 100% bug-free. Software testing does help you to find bugs, but it does not give you any guarantees about the quality of a program. Instead, you should view testing in this way: testing code is definitely better than not testing it at all, and testing can give you a limited measure of confidence in the code, depending on how good the testing was.
Testing can only prove the presence of bugs, not their absence.
-- Edsger W. Dijkstra
Who does the testing?
Testing is a big part of software development. In the world of professional programming, software is typically developed by teams of programmers, and testing is done on a large-scale. The testers may not be the same people as the developers; indeed the testers may have been hired specially to do software testing.
Whilst you are learning to program as an undergraduate at a university, however, testing is rather different. As a beginning programmer, you are usually encouraged to practise testing on small programs that you yourself have coded. This is not ideal in some respects, because it is more difficult to find bugs in your own code rather than someone else's. However, testing is an important part of your programming skills, to help you get better at finding and fixing problems in program code.
How do we test programs?
A detailed explanation of how to test programs, especially in the context of undergraduate programming assignments, would require a whole other article. This article will appear at some stage, but for now, here's an overview of how testing is carried out:
Recall that functionality testing tries to test whether the software does what it is supposed to do. So how do we know what the program is supposed to do? We need something called a specification. What is a specification? You can think of a specification as "a precise clear description of what a program should do".
If you are given a specification for your programming coursework, then the specification will probably be written in English. But in general a specification can be written in many ways, and may include mathematics or diagrams.
So testing consists of running the program code and see if it behaves as it should do, according to the specification. Usually the program is run several times, with different input data used for each run. Testers try and run lots and lots of tests, using a wide variety of different data, to try and spot as many bugs as possible. The aim is to be as thorough as possible in the testing. A good test is one that discovers a bug!
Another aspect of testing is documentation. When you are learning to program at university, if you are asked for testing documentation, this is so that in the assessment of your work, you can get marks for how well you carried out the testing. In the professional programming world, however, testing documentation is important to record what tests have been done and what the results were. Why is this important? Well imagine that you are part of a team of testers: how can you know what has been tested so far and what hasn't, unless there are records of what testing has been carried out? Or, for example, if a bug gets found in a test, and the programmers then go away and come back later saying that they fixed it, how can you repeat the original test that found the bug, if that test wasn't documented sufficiently for you to know what it was?
When do we test programs?
Programs are tested at all stages of development. Typically, as each programmer works on the code, he or she will test in an incremental way, doing small tests for each small piece of code written, to check that it seems to be working ok. In addition, there are also large-scale tests carried out when substantial parts of the code have been assembled.
In general, it is better to find bugs as early in the development as possible. The longer bugs remain hidden in the code, the greater the risk of writing more wrong code as a result.
