T 2.29 Software testing with production data

It can be frequently observed that software tests are performed with production data. The main reasons given for this are that the only way to make a definitive assessment of the functions and performance of the product is to compare it directly with existing work results. Additional reasons for doing this are inadequate security awareness, exaggerated confidence in the software under test and ignorance of potential damage.

Testing with production data may result in the following problems:

• Software is tested with copies of production data in an isolated test environment:
  
  If new software is tested with data which has not been made anonymous, unauthorised employees or third parties who have been charged with testing the software may gain access to files carrying confidential information.
• Software is tested with production data in actual operation:
  
  Software which malfunctions during testing may, as in the before-mentioned case, lead not only to impaired confidentiality but also to a loss of integrity and availability of production data.
  
  Because different programs may be incompatible, side effects can arise which may lead to significant impairments in other system components. In the case of networks, this may range from loss of performance through to a crashing of the network.
  
  If software under test performs incorrectly or if operating errors are made, production data may be inadvertently modified. It is possible that such a modification may not be identified. To avoid redundancy, databases are increasingly shared by different programs so that these errors potentially have an effect on other IT applications as well. When damage occurs, there are not only costs involved in reconstructing the data, but existing work results must also be checked for integrity.