One of the most beneficial features of Test Design Studio is its ability to statically analyze your code and warn you of potential pitfalls. We want to highlight two of the new rules that are coming to the next release of Test Design Studio.
The error handling capabilities in VBScript are pretty limited, but still necessary. Using the statement ‘On Error Resume Next’ within your code is often unavoidable, and this statement is easy to abuse. We have added two rules to assist with the use of this statement:
The “Use error handling with caution” rule is very simple. Any use of ‘On Error Resume Next’ will be flagged as a warning by code analysis and you will need to manually suppress it. Ideally, your suppression comment will also indicate why you used the statement. This rule will help raise awareness to the dangers of not using the statement properly.
The “Close error handlers in routines” rule attempts to make sure that if you turn on error handling in a routine (using “On Error Resume Next”), you also turn off error handling before exiting the routine (using “On Error Goto 0”). It is not uncommon for a code author to turn on error handling within their routine to deal with a specific usage scenario, and then forget to turn it off before leaving the routine. Any calls to this routine will inadvertently turn on error handling without the callers knowledge. This can lead to unexpected code execution.
Already a Success Story
I was recently working with a client to help determine why a certain test was running in an infinite loop after a replay error. Not only was the script not failing gracefully, it was delaying the entire test suite by not freeing the lab machine for other tests to execute. I loaded the test and it’s relevant libraries into Test Design Studio with these new rules enabled, and Test Design Studio quickly located the needle within a haystack of over 100,000 lines of code.
A custom user function registered on a test object had set “On Error Resume Next” without turning it off. Later in the test, a “While” loop was processing that was encountering an error with the condition statement of the loop. Instead of failing the loop, “Resume Next” meant it continued with the next line of execution… the first line within the body of the loop! The errors continued through the loop until the “Wend” token instructed execution to return to the top of the loop and reevaluate the condition.
This process repeated until remote execution of the test was forced to terminate, but not until after more than 30,000 steps had been sent back to Quality Center/ALM for the last run. Since QC/ALM likes to show you the steps of the last run when you select a test instance in Test Lab, it also locked up QC/ALM while it attempted to process all those steps and eventually failed due to an out of memory condition. The steps had to be manually deleted from the database to restore functionality.
All this work because someone forgot to restore error handling functionality! With these new rules, hopefully that will never happen again!