I just about had a nightmare of sorts regarding my paycheck. Apparently the 401k management company that my company works with has a slight problem communicating the amount of contribution made by an individual.
Here’s what I got from payroll:
Justin,
On your 401k enrollment, it came across as “53″.
Would you please confirm if you want to contribute 53 percent of your wages or 53 dollars to the 401k plan each paycheck
Thanks!
Just a bit scary. Considering I’d entered $53 for every paycheck and it says it clearly on the site. This email made me glad that the person that handles payroll thought it best to verify what exactly was meant.
This is a very good time to make the point that no matter how smartly testing plans and automated test systems are put together there is always the need to have an actual person behind the scenes checking out the details.
Even though it’d be nice to be saving 53% of what I make for my retirement (me at age <insert retirement age here> would be gracious) I can’t afford that right now. So, thank you to payroll for verifying the information received and what was actually wanted.
<insert sigh here>
–dez
Albert Gareev
September 11th, 2009
Hi Justin,
Would you agree with the point that human beings can also sometimes be careless or inaccurate? And who will be checking after them?
I’d more rely on the auditable and traceable process with automated reconciliation rather than purely manual one.
Thanks.
dez
September 11th, 2009
Yes, I definitely agree that humans can be careless and inaccurate which is why a lot of application bugs come out when that application gets released to the general public. On that same token, an automated test is just as good as it’s maker. If you make the test to check against the wrong thing, you’ll get a false positive.
Michael at DeveloperSense has a great post about the differences between testing and checking. Checking being described as confirming a known quantity where testing is exploratory and investigative. A machine can’t ask the question “What happens if I do this…” in the middle of a test.
I think both kinds of testing are necessary, but neither should be relied on too much more than the other. Automation lacks creativity and exploration while manual testing is time consuming and full of forgetfulness and inaccuracies.
Albert Gareev
October 13th, 2009
Hi Justin,
(sorry didn’t check for your reply earlier)
The actual statement of yours that originally caught my attention is this one, particularly the very end of it.
“This is a very good time to make the point that no matter how smartly testing plans and automated test systems are put together there is always the need to have an actual person behind the scenes _checking_ _out_ _the_ _details_.”
The main point I express is that if you have to check details after automated test you sunk on it (because there will be 1000’s of atomic checks), you lost your time, and you lost your investment in automation because it simply failed to prove its reliability.
I admit, vast number of automated tests are no more than UI data entry that is definitely not testing and is not even checking. But that is not the only possible approach.
If you’d like to find out about my approach on how atomic test steps should be implemented in automation:
http://automationbeyond.wordpress.com/2009/06/05/qa-automation-gui-function-wrapping/
The mentioned atomic GUI and logical steps should have PASS/FAIL status produced, and execution should be Test Case oriented, with a major PASS/FAIL condition. Then it all could be reconciled, and, combined with the human-friendly reporting, will give a _big_ picture view that you can still drill down if you want to check details.
Only being at the top of the pyramid testers can benefit from the automated test execution.
With regards to the original problem, I see it more as a “machine-person communication issue”. The program you entered your numbers in obviously “knew” that you meant dollar amount and not percents, but it failed to express it clearly to a person processing the enrollments. So it’s a miscommunication. Hardly testable, isn’t it?
Thank you,
Albert Gareev
Michael Bollton
September 11th, 2009
That sounds to me, Albert, like you’re saying, “Machines are better than people.”
In fact, machines can greatly extend human capabilities, but they have serious cognitive limitations too. This is at the heart of my current thinking about testing vs. checking, as Dez cites in his reply
A more nuanced way of looking at the issue would be to ask some questions from McLuhan: “In what ways can media extend or enhance or accelerate or intensify some human capability?” “In what can media reverse into the opposite of their original or intended effect?” You might be interested in http://www.developsense.com/articles/2007-09-McLuhanForTesters.pdf
Cheers,
—Michael B.
Albert Gareev
October 14th, 2009
Hi Michael,
>> That sounds to me, Albert, like you’re saying, “Machines are better than people.”
- That sounds to me that you are too ambiguous here. Better than people in what? Where? How?
First of all, getting back to the original point: built-in, constant automatic validation and reconciliation is better than an occasional/casual human checking. This is what I stated. Please do not morph it.
I enjoy reading your “Testing vs. Checking” series as well as other articles.
However, you may not be aware of academic research in A.I. methodology. There are Artificial Intelligence methods that have already been successfully utilized in software development for a variety of needs: from image/text/voice recognition to automated/automatic control systems.
Having Master’s degree in automated systems I used elements of A.I. methods implementing my test automation framework, to name a couple: Associative Networks and Fuzzy Logic.
So, quoting your article…
“Machines can recognize inconsistencies and problems that they have been programmed to recognize, but not new ones”. – In fact, even _new_ _ones_ could be recognized and reported.
Yes, with limitations. Yes, automated testing must be integrated into the cycle first, and, yes, that requires investment. But don’t say they’re just incapable.
Thank you,
Albert Gareev
Why Human Validation is always needed (in discussion) « Automation Beyond
October 15th, 2009
[...] Albert Gareev on 2009/10/15 This is the discussion slowly going on Justin Dessonville’s blog post. With the arguments taken from famous “Testing vs. Checking” article it once again [...]