Consider a function called openCDTray( ) which ejects a CD from the drive.
This function should be operated only when the CD tray is closed. The function also has the following constraint(for effect). Attempting to open the tray when it is already open could result in the tray falling off and reattaching the tray is a cumbersome activity! :-)
The system maintains the status of the tray in a global variable/object called gCDStatus. openCDTray( ) should check the status and then only attempt to eject; otherwise all hell would break loose. But does the function implementation take care of this? Maybe the developer thought that nobody in their right mind would do such a thing and omitted the check. It's so obvious!
A unit test-case to check the response of the function in such a scenario could simply do the following:
1. Set gCDStatus to TRAY_IS_OPEN.
2. Call the function.
3. Check the result. The function should not have succeeded.
But our developers would never skip such basic checks! But even in such cases, unit-test can catch errors that could be missed in a cursory inspection of the code.
1. Typos in the assertion-check.
if ( gCDStatus = TRAY_IS_OPEN) throw ExceptionAlreadyOpen;
or if you consider that to be improbable too
if( gCDStatus = CD_EJECTED) throw ExceptionAlreadyOpen, where CD_EJECTED is a similar-looking, but different valid value for gCDStatus
2. The openCDTray( ) function might have been modified (copy-paste!) and the programmer inadvertently does something that causes a change of the gCDStatus value.
The unit-test also helps to find out whether the assertion-check has in fact been skipped. This becomes crucial during integration and has to be guaranteed before functional testing starts.
1. The user of the function may not be aware of all the preconditions and hence may not ensure all of them before calling the function.
2. The definition of another part of the system may have changed.
In the example of openCDTray( ) , we would be saved a trip to the CD repair shop by the unit-test!
Wednesday, August 29, 2007
Sunday, August 19, 2007
Unit Test Scripts
A unit-test function correspoding to each unit-test case broadly consists of the following sections.
1) Precondition Tweaking (Prologue)
Prepare the conditions necessary for the function to execute. Do this for all preconditions not mandated by the test-case; not supplying precondition(s) to see if the function fails may be the test-case.
2) Invocation (Test)
Call the function.
3) Post-condition Verification (Epilogue)
Check whether all the post-conditions have been enforced by the function. Check whether the result tallies with the expected result of the unit-test case.
4) Logging
This may be part of the unit-testing framework itself, if you are using one.
Link the test-function with the test-harness and the unit to be tested. Call these test-functions from a driver program to see how your unit copes!
1) Precondition Tweaking (Prologue)
Prepare the conditions necessary for the function to execute. Do this for all preconditions not mandated by the test-case; not supplying precondition(s) to see if the function fails may be the test-case.
2) Invocation (Test)
Call the function.
3) Post-condition Verification (Epilogue)
Check whether all the post-conditions have been enforced by the function. Check whether the result tallies with the expected result of the unit-test case.
4) Logging
This may be part of the unit-testing framework itself, if you are using one.
Link the test-function with the test-harness and the unit to be tested. Call these test-functions from a driver program to see how your unit copes!
Friday, August 17, 2007
Unit Test Environment
A Unit Test Suite consists of code which verifies and validates the unit within the unit-testing environment. It consists of -
1) Test Script - Code which calls different functions of the unit under different conditions.
2) Test Harness - In order to achieve its functionality, the unit under test might need the help of other modules. Substitute all such external functions with dummy versions, stripped to the bare minimum.
The unit under test should be run under this unit-testing environment. For the unit of a complex, heterogeneous system, it is more practical and useful to have a unit-testing environment which is much simpler than the actual deployment environment.
Aside: It is tempting to make the unit-test environment "more real, just to see if it works too; anyway I am testing, so why not?". So if you do this, you will end up doing something that's neither unit- nor functional-testing, and you won't get the possible benefits of either!
More about test-scripts and test-harnesses later.
1) Test Script - Code which calls different functions of the unit under different conditions.
2) Test Harness - In order to achieve its functionality, the unit under test might need the help of other modules. Substitute all such external functions with dummy versions, stripped to the bare minimum.
The unit under test should be run under this unit-testing environment. For the unit of a complex, heterogeneous system, it is more practical and useful to have a unit-testing environment which is much simpler than the actual deployment environment.
Aside: It is tempting to make the unit-test environment "more real, just to see if it works too; anyway I am testing, so why not?". So if you do this, you will end up doing something that's neither unit- nor functional-testing, and you won't get the possible benefits of either!
More about test-scripts and test-harnesses later.
Wednesday, August 15, 2007
Limits Of Unit Diagnosis
Unit-testing can uncover many kinds of deficiencies, gaps and faults. It is not the responsibility of a unit test to verify whether the function actually works or not, that is the domain of functionality-tests. Indeed, a unit may be successfully tested by the unit-test suite and still not work!
1) Inaccurate definitions - If the function succeeds even if some preconditions deemed necessary are not fulfilled, then probably the designer has made an error in the specification.
2) Unaddressed requirements - That lengthy switch-case might be missing a few cases!
3) Certain kinds of bugs - Typos in parts of the code that affect the post-condition enforcement can be caught.
4) Programmer indiscipline - Perhaps that developer might be missing a few basic but essential checks?
If nothing, it provides a way of ensuring complete coverage and execution of the developed code, an objective that would be difficult or time-consuming to achieve otherwise.
1) Inaccurate definitions - If the function succeeds even if some preconditions deemed necessary are not fulfilled, then probably the designer has made an error in the specification.
2) Unaddressed requirements - That lengthy switch-case might be missing a few cases!
3) Certain kinds of bugs - Typos in parts of the code that affect the post-condition enforcement can be caught.
4) Programmer indiscipline - Perhaps that developer might be missing a few basic but essential checks?
If nothing, it provides a way of ensuring complete coverage and execution of the developed code, an objective that would be difficult or time-consuming to achieve otherwise.
Monday, August 13, 2007
Unit Testing As Claim Validation
"Give me a lever long enough and a fulcrum on which to place it, and I shall move the world." - Archimedes (Mathematician and inventor of ancient Greece, 280-211BC)
A function is much like Archimedes's lever, a block of code that claims that it will do something if some things have already been done. A unit test should ensure that the function can actually do exactly what it claims to, when all its conditions are met.
The definition of a function, no matter how complex it is, can be decomposed as
1) Preconditions (Conditions/Environment)- Things which must be done or conditions which must prevail when the function is called. These might include constraints on the input parameters and global state variables.
2) Body (Action) - The main block of code which achieves the functionality.
3) Postconditions (Side-effects)- Things that the function will ensure by the time the body of the function finishes executing. These can be the function's return value or the modification of the state of the system.
Unit-tests should determine whether all the postconditions are fulfilled when all the preconditions are fulfilled by the caller of the function. It is important to remember that unit-tests are more like mathematical verification of the function-model; they check whether the code conforms to the definition of the functions.
A function is much like Archimedes's lever, a block of code that claims that it will do something if some things have already been done. A unit test should ensure that the function can actually do exactly what it claims to, when all its conditions are met.
The definition of a function, no matter how complex it is, can be decomposed as
1) Preconditions (Conditions/Environment)- Things which must be done or conditions which must prevail when the function is called. These might include constraints on the input parameters and global state variables.
2) Body (Action) - The main block of code which achieves the functionality.
3) Postconditions (Side-effects)- Things that the function will ensure by the time the body of the function finishes executing. These can be the function's return value or the modification of the state of the system.
Unit-tests should determine whether all the postconditions are fulfilled when all the preconditions are fulfilled by the caller of the function. It is important to remember that unit-tests are more like mathematical verification of the function-model; they check whether the code conforms to the definition of the functions.
Saturday, August 11, 2007
Unit Testing Primer
(Breaking this article into smaller, more readable fragments. Blog-style!)
Unit-testing deals with the verification of what an unit can achieve on its own, without help from other units. In other words, assuming all the other modules required by the unit are available, can the unit achieve what it claims to do? This is the question that unit-testing seeks to answer.
I started off a skeptic, regarding the usefulness of having unit tests, seemed to be such a waste of time. What won me over? Simply put, it helps us to avoid stupid, elementary mistakes. But god knows (as do we!) how common those are, especially during bug-fixes!
My primary motivation to write this series of articles is to-
1) define the scope of a unit-test - There seems to be misconceptions about what unit-testing is all about, what a unit-test should and should not do,
2) list the benefits of unit-testing - there's more to it than meets the eye at first sight. All love is not at first-sight!
3) share my understanding(s) - I had great fun exploring unit-testing, especially regarding what could be achieved with dummy or mock versions of external functions (Harnesses)
Here goes...
Unit testing = Claim Validation
Diagnosis Limits
Unit Testing Environment
Test Scripts
Test Harnesses
Example (Read this first if you will!)
Unit-Testing Frameworks
Run the unit-test suite frequently and regularly, especially at integration-points to ensure that development is proceeding in the right direction. Unit-testing can go a long way in -
1) making your team's code bullet-proof and,
2) importantly, avoiding last-minute-release-time-chaos-induced bloopers.
I must also add here that I am not a fan (yet?) of test-driven development. I would recommend unit-testing for finding errors that could easily escape visual checks or code walk-throughs..
- Thomas Jay Cubb
Please also read Learning To Love Unit Testing.
TODO - Diagrams
Unit-testing deals with the verification of what an unit can achieve on its own, without help from other units. In other words, assuming all the other modules required by the unit are available, can the unit achieve what it claims to do? This is the question that unit-testing seeks to answer.
I started off a skeptic, regarding the usefulness of having unit tests, seemed to be such a waste of time. What won me over? Simply put, it helps us to avoid stupid, elementary mistakes. But god knows (as do we!) how common those are, especially during bug-fixes!
My primary motivation to write this series of articles is to-
1) define the scope of a unit-test - There seems to be misconceptions about what unit-testing is all about, what a unit-test should and should not do,
2) list the benefits of unit-testing - there's more to it than meets the eye at first sight. All love is not at first-sight!
3) share my understanding(s) - I had great fun exploring unit-testing, especially regarding what could be achieved with dummy or mock versions of external functions (Harnesses)
Here goes...
Unit testing = Claim Validation
Diagnosis Limits
Unit Testing Environment
Test Scripts
Test Harnesses
Example (Read this first if you will!)
Unit-Testing Frameworks
Run the unit-test suite frequently and regularly, especially at integration-points to ensure that development is proceeding in the right direction. Unit-testing can go a long way in -
1) making your team's code bullet-proof and,
2) importantly, avoiding last-minute-release-time-chaos-induced bloopers.
I must also add here that I am not a fan (yet?) of test-driven development. I would recommend unit-testing for finding errors that could easily escape visual checks or code walk-throughs..
- Thomas Jay Cubb
Please also read Learning To Love Unit Testing.
TODO - Diagrams
Subscribe to:
Posts (Atom)