Test automation can be much more than just automating what a manual tester does. This workshop is about deciding about test automation – whether to do it, what to automate, and how to use it to extend our reach and do testing that cannot be done manually. The workshop is not about any specific tools, but presents overall views and approaches to test automation.
The primary focus in this workshop is high power automated testing – generating and evaluating massive volumes of data, monitoring program activities not observable by humans, and exploring ever-changing areas of the program. This workshop describes these types of tests and mechanisms to productively implement them. It focuses on exploratory automated tests and automated test oracles for evaluating software behavior for the tests.
Exploratory automated testing is a testing approach that uses the power of the computer to look for bugs that functional testing misses. Unlike regression tests that do the same thing each time they run, exploratory tests do different things each time. Through the power of the computer, these tests may run and check millions or billions of functions – volumes unthinkable with manual tests, automated scripts, or even table-driven automation. These techniques go after bugs that are virtually impossible to expose or isolate using manual testing, uncovering gnarly bugs from unexpected special values and long convoluted sequences of events.
Automated test oracle mechanisms are required for these tests. They can also increase the power of all tests by looking for and potentially exposing various types of errors, often without modifying the tests themselves. A common example is memory leak checking, but there are scores of other types of errors we can check for.
Topics:
- Return on investment (ROI) from test automation
- Selecting tools and frameworks
- Regression and non-regression testing
- Exploratory tests
- Automating exploratory tests
- How to employ massive numbers of tests
- Applying pseudo-random number generators in tests
- A dozen mechanisms for automated test oracles
