Automate everything or don't do any automation, this is the question?!
Laaaaaaadiiiiies aaaaaaaand Gentlemeeeeeeeen,
We are happy to announce the first ever Test Clash Keynote on Test Automation.
We invite you to a historical clash on two very different opinions:
"Automate everything or don't do any automation"
Don't miss this phenomenal 6 rounds clash between the Testautomation Engineer Noah Sussman and Senior Test Engineer Paul Holland!
Are youuuu readyyyyy toooo rummmmbleeeeeee?
Paul Holland says:
As a tester - and an ex-automation developer - I have come to realize that automation will not find “new” bugs in the code. Once an automated script is working the script will only find newly introduced bugs that are placed along the thin paths through the code that are being exercised by the automation. So, one might expect that once a script goes green it will remain green unless the developers make mistakes that happen to interfere with the execution of these paths. What about all the other paths through the code, or the same paths with different values? Our customers do not follow scripts when they use our code, so they will be attempting combinations and paths through the code that don’t make sense to automate - especially at the UI level. UI level automation is brittle, needs a lot of hand holding and investigation of failures, plus it is difficult and expensive to maintain. Automation will only find bugs that it is programmed to identify. It will be blind to almost all other bugs. Testing by a human can exercise different paths each time they test, a vigilant tester will be able to notice many different aspects of the program that appear to be behaving incorrectly, they can adapt to the situation, they can experiment, they can find bugs.
Noah Sussman says:
As a developer I have always tested my own designs and implementations. Before I was introduced to unit testing, I was already writing my code in a testable manner: small units, loosely coupled interfaces and easily inspectable internals. Because of this I can see the state of my application in production in real time, and I can also see the recent history of both behavior, and changes made to production. I worked very hard to perfect a continuous delivery workflow that enables me to push fixes for broken functionality within minutes of the solution being written. Part of CD is embracing a database architecture that only allows soft deletes, and in which schema changes are non-destructive so that it is always possible to recover data that might be lost forever due to an incident in a typical Web application database. Given all this, I find it difficult to say what specific value a tester could add to my team. I can detect problems quickly, recover quickly, and my data integrity is assured because of the aforementioned soft deletes and non-destructive schema changes. Given that I can detect, recover and restore data with such rapidity, I see very little risk if the occasional bug reaches my customers. I'll just fix it fast!
WHO WILL WIN???