Planed to post an article on SST-MM soon..
Cheers,
-jerry and Team SST
Monday, October 22, 2007
Friday, October 19, 2007
30 Usability Issues to be aware of
Today i found a wonderful article about usability issues of web.
Hope this will be very helpful in designing a good web page and in testing too...
http://www.smashingmagazine.com/2007/10/09/30-usability-issues-to-be-aware-of/
The next one is about the same usability in web but some nightmares to come out from
http://www.smashingmagazine.com/2007/09/27/10-usability-nightmares-you-should-be-aware-of/
Cheers,
-jerry
Hope this will be very helpful in designing a good web page and in testing too...
http://www.smashingmagazine.com/2007/10/09/30-usability-issues-to-be-aware-of/
The next one is about the same usability in web but some nightmares to come out from
http://www.smashingmagazine.com/2007/09/27/10-usability-nightmares-you-should-be-aware-of/
Cheers,
-jerry
Friday, October 12, 2007
The final question to KAREN N. JOHNSON by QA Zone
Q8- Thank you very much for your time, Karen. Let’s end with a quick recommendation…For QAZone members that are looking to grow their team, what do you think are the most important skills or personality characteristics that they should be looking for in new team members during their hiring process?
KNJ- Curiosity, I can’t stress this enough!. Testers need to be curious people, constantly wondering things such as what if I do this, or what if fill-in-the-blank here. Technical skills can be taught if someone has the aptitude and motivation. But behind all good testers, there is a burning curiosity. I think this is why testers enjoy TV shows such as the CSI series. We like to solve mysteries, we like the investigations, the fact-finding and the details. Coincidentally I think having a curious mind keeps us young and engaged with life as well.
KNJ- Curiosity, I can’t stress this enough!. Testers need to be curious people, constantly wondering things such as what if I do this, or what if fill-in-the-blank here. Technical skills can be taught if someone has the aptitude and motivation. But behind all good testers, there is a burning curiosity. I think this is why testers enjoy TV shows such as the CSI series. We like to solve mysteries, we like the investigations, the fact-finding and the details. Coincidentally I think having a curious mind keeps us young and engaged with life as well.
Five Important things to work with in before Test Plan
Its so simple...
1. Your Project.
2. Your Resources.
3. Your Budget.
4. Time Available.
Keep all these things in your mind and prepare Test Plan.
Don't do test plan with a mind setup of satisfying ur customer or to do 100% testing.
That's not possible. So work with the possible things..
-jerry .. :-)
1. Your Project.
2. Your Resources.
3. Your Budget.
4. Time Available.
Keep all these things in your mind and prepare Test Plan.
Don't do test plan with a mind setup of satisfying ur customer or to do 100% testing.
That's not possible. So work with the possible things..
-jerry .. :-)
Friday, October 5, 2007
Good explanation on test-driven development
There seems to be a lot of confusion and misinformation out there about what exactly we mean by test-driven development.
I sometimes use this little analogy to help explain it:When I moved in to my first house, me and my girlfriend of the time (who was a total cow, I should hasten to add - not that I'm bitter or anything) went shopping for kitchenware.
Our method was simply to sit down and draw up a list of "things that a kitchen should have" - like crockery, cutlery, mugs, glasses, scales, egg whisk, bowls, and so on.We spent hours in IKEA, British Home Stores and various other purveyors of culinary paraphernalia, and then we dragged everything back to the house and unpacked it into our new kitchen.
When we'd finished we sat at the breakfast bar with a cup of tea and surveyed our handiwork. It wasn't until the next day when I tried to make a mushroom omelette that I realised we'd forgotten to buy a frying pan. And when my girlfriend tried to make herself a Jack Daniels & Coke on ice that we realised we'd forgotten to buy any ice trays for the freezer. Or suitably sized glasses for a whiskey and coke, come to that. So I had cornflakes for breakfast, and and in the evening she enjoyed a warm whiskey and coke served in a small wine glass.
What we should have done, of course, was make a list of examples of things we'd be using our kitchen to do. And then figuring out what we'd need in our kitchen in order to successfully execute each of these scenarios.That way, I would have most certainly remembered the frying pan, and Sarah would have had a nice, cold drink our of a more appropriate glass.
TDD is specification by example. Each test case describes a way in which the end product will be used - whether that end product is a kitchen, a software applictaion, or just a single method in a single class in a software application. It's not really a testing discipline at all. It's a design discipline. I think maybe it's the "t" word that confuses people, though. Some teams write seven billion lines of code, and then write a handful of unit tests to cover it, and then come to me and say "we're doing test-driven development". No. Not if you write the tests after you've written the code, you're not. In TDD, tests drive development - the clue's in the name. You only write code needed to pass tests. So your tests are a specification for the code you need to write.
The fact that afterwards you end up with a suite of executable regression tests is a positive side effect of doing TDD. It is not the primary goal of TDD, though.
The primary goal of TDD is to use tests to drive the design and implementation of the software.And that, my fine feathered friends, is test-driven development.
-Thanks to Jason Gorman....wonderfully explained.. :-)
I sometimes use this little analogy to help explain it:When I moved in to my first house, me and my girlfriend of the time (who was a total cow, I should hasten to add - not that I'm bitter or anything) went shopping for kitchenware.
Our method was simply to sit down and draw up a list of "things that a kitchen should have" - like crockery, cutlery, mugs, glasses, scales, egg whisk, bowls, and so on.We spent hours in IKEA, British Home Stores and various other purveyors of culinary paraphernalia, and then we dragged everything back to the house and unpacked it into our new kitchen.
When we'd finished we sat at the breakfast bar with a cup of tea and surveyed our handiwork. It wasn't until the next day when I tried to make a mushroom omelette that I realised we'd forgotten to buy a frying pan. And when my girlfriend tried to make herself a Jack Daniels & Coke on ice that we realised we'd forgotten to buy any ice trays for the freezer. Or suitably sized glasses for a whiskey and coke, come to that. So I had cornflakes for breakfast, and and in the evening she enjoyed a warm whiskey and coke served in a small wine glass.
What we should have done, of course, was make a list of examples of things we'd be using our kitchen to do. And then figuring out what we'd need in our kitchen in order to successfully execute each of these scenarios.That way, I would have most certainly remembered the frying pan, and Sarah would have had a nice, cold drink our of a more appropriate glass.
TDD is specification by example. Each test case describes a way in which the end product will be used - whether that end product is a kitchen, a software applictaion, or just a single method in a single class in a software application. It's not really a testing discipline at all. It's a design discipline. I think maybe it's the "t" word that confuses people, though. Some teams write seven billion lines of code, and then write a handful of unit tests to cover it, and then come to me and say "we're doing test-driven development". No. Not if you write the tests after you've written the code, you're not. In TDD, tests drive development - the clue's in the name. You only write code needed to pass tests. So your tests are a specification for the code you need to write.
The fact that afterwards you end up with a suite of executable regression tests is a positive side effect of doing TDD. It is not the primary goal of TDD, though.
The primary goal of TDD is to use tests to drive the design and implementation of the software.And that, my fine feathered friends, is test-driven development.
-Thanks to Jason Gorman....wonderfully explained.. :-)
Tuesday, September 4, 2007
10 Software Development No-Brainers
Here is a cool post by jason Gorman....
I spend a considerable amount of my life locked in meeting rooms listening to protracted discussions about stuff that I could have sworn we'd sorted out years ago.
I'm thinking maybe, to save time and heartache, I should make a list of these no-brainers and their screamingly obvious solutions so we can all get on and apply our feeble minds to more important problems - like where to get the cheapest liquor.
1. What should be our Coding Standards? - Google "coding standards" for your chosen programming language, find a set that makes sense to you and then use them.
2. Should we do iterative development? - No. You should stick your head up an elephant's arse instead. It has about the same effectiveness as waterfall delivery, but is slightly less unpleasant for the parties involved.
3. Model-View-Controller or Model-View-Presenter? - seriously, who gives a sh*t? Pick one and get on with it.
4. .NET or Java? - ditto.
5. Are we doing too much unit testing? - No.
6. Do we need a business/systems analyst? - nobody needs a business/systems analyst. They're like nipples for men - they serve no practical purpose, but they can still be used to hurt you.
7. Should we adopt a Service-Oriented Architecture? - Sure. Just let the rest of us know when you've figured out what the hell that means, okay?
8. Are we doing too much refactoring? - No.
9. The guy we want is just outside our budget. What do we do? - Increase your budget. Find a way.
10. How can we make more time for quality? - Just do it. The extra time will make itself.
--Thanks Jason...this is cool
Friday, August 24, 2007
How did we miss THAT?
Here is another intresting blog i visited....
How did we miss THAT?
Filed under: Thinking Like a Tester, Ruminations — Elisabeth Hendrickson @ 11:12 am
“Oh goodness. How did I miss THAT bug?”
Over the years, I’ve asked myself that question numerous times.
I asked that question when another tester found a blazingly obvious, critical bug that I completely missed. (The answer: I spent too much time tinkering with an ineffective automated script I’d written, and too little time observing the actual behavior of the system. That’s the project where I learned a lot about how NOT to do test automation.)
I asked myself that question, repeatedly, when I participated on a project some years ago now where we shipped software that crashed left and right. (I’m still sorting out the answers to that last one. Catastrophic failures are almost never the result of a single, simple error. And this particular catastrophic failure represented failures at all levels in an organization that had, um, issues. But I digress.)
I asked the question again when I learned that a web site I tested had back-button problems. After all, I was sure I’d tested for that. And I had. But I hadn’t re-tested for it after a particular set of code changes that changed some operations from HTTP GETs to HTTP POSTs. Oops.
And I asked myself that question more recently when I learned that a system I worked on earlier this year failed to save a change, and also failed to report an error, when it encountered data misformatted in a particular way in one specific field. Badly formatted data is one of my specialties, and I couldn’t believe I forgot to test the particular case that resulted in the problem. But it turned out that I did, indeed, fail to test what would happen if you entered “www.testobsessed.com” into a URL field instead of “http://www.testobsessed.com”. In hindsight, it’s an obvious test. Another lesson learned.
I reflected on those missed bugs when a colleague, Sandeep, a test manager, recently wrote to say that he’s been asking himself “how did my testing team miss that?”
He decided to seek out patterns of testing problems by categorizing escaped bugs according to the hole(s) in testing that allowed the bug to slip through. The idea is to improve the test effort by figuring out the common causes behind escaped bugs.
My initial reaction was, “that makes sense.” If you can identify the top 20% of testing holes that let 80% of the bugs through, and you can make some serious improvements to the test effort.
And my next reaction was, “but be careful.” Sandeep’s intent is good: use lessons learned from escaped bugs to improve testing. However, asking “How did we miss that?” is perilously close to heading down the slippery slope to “How did JoeBob miss that?” to “It’s JoeBob’s fault.” Having talked to Sandeep, I know he’s not trying to play “pin the blame on the tester.”
So I suggested a small reframe.
Instead of categorizing escaped bugs by asking the question, “How did testing miss that?”, categorize them by asking the question, “How can we improve the probability that testing will find bugs like that in the future?”
It’s a subtle difference.
But the result of reframing the question is that instead of identifying categories as noun phrases like “insufficient test data,” we end up with imperative statements like, “add test data.” Those two categories may look almost identical, but only one is actionable. I can “add test data.” The statement prompts me to do something different next time. But “insufficient test data” only gives me something to regret. And regret won’t help me ship better software.
So how can you categorize escaped bugs to improve the test effort without falling into the blaming trap? Try an Affinity Exercise with the question, “What could we do differently next time to increase the probability that if we have another bug like this we’ll catch it in test?”
To prepare:
Choose a team to participate in the activity. Affinity exercises can work with any number of people, but for this particular activity, I find a smaller group - say 3 to 5 people - works best. It’s a good idea to include people with diverse roles and skill sets.
Set up a meeting time and place. Plan for the whole activity to take 2 hours. And arrange to meet in a place with plenty of table and/or wall space.
Gather (or shop for) office supplies. You’ll need:
Index cards or sticky notes. Bigger is better. I like 5×8 cards or the SuperSticky 5×8 Post Its.
Felt-tip markers. I like Sharpies because they make consistently dark, readable marks. (Beware: Sharpies are permanent. Do NOT confuse your Sharpies and your White Board markers in the conference rooms. Facilities people get tetchy about such mix-ups.)
Gather a list of escaped bugs you want to analyze. If you have a lot of escaped bugs, prioritize them and time box the exercise. (You probably won’t be able to analyze more than 50 in an hour, possibly less, so don’t print out a list of 500.)
In the Meeting:
Review each bug with the team, asking: “What could we do differently next time to increase the probability that if we have another bug like this we’ll catch it in test?”
Have participants write their suggestions on the cards/stickies, one idea per card, in the form of an actionable statement. The suggestions should complete the sentence, “In the next release/iteration/sprint, we can ______.” Tips:
Also ask the participants to make their suggestions as concrete and specific as possible. For example, instead of writing “add test data,” write “add titles with ampersands (&) to the test data.”
And ask the participants to stick to test-related actions, and avoid blaming individuals. “Revoke NancySue’s checkin priviledges” is not an acceptable suggestion.
When you’ve reviewed all the bugs, or when an hour has passed, stop reviewing bugs. (If you still have lots of bugs to go and want to continue analyzing after an hour, stop anyway. Finish the rest of the exercise - the grouping. When you’ve worked through the whole process, if you still think more analysis would help, you can always do the exercise again.)
Gather all the cards/stickies, and lay them out on a large work surface: a table, the walls, or even the floor can all work well.
Sort through the cards/stickies as a team. The cards are now owned by the team, and everyone should take a hand in organizing them. Encourage participants to move cards that seem alike together so they are stacked together. Continue until the team agrees that it’s satisfied with the stacks of cards/stickies.
Ask the team to give meaningful names to the stacks of cards/stickies. This is the part of the activity where will generate the more abstract categories like “add test data.”
The result of this exercise is a list of categories for improving the testing effort that we can then use to determine which kinds of improvements will have the biggest bang for the buck. And the best part is that the list emerged from the actual problems your software has had in the field rather than being some arbitrary list of theoretical “improvements” based on someone else’s unrelated experience.
But wait; you’re not done. Now that you’ve created a first draft list of categories, test it. (Did I mention I’m Test Obsessed?) Choose a different set of escaped bugs, and assign each to one or more categories from the list. Notice how easy or hard it seems to find a category for each bug. This will give you a lot of feedback about how well the category list will work in practice. You may find the team needs to spend some additional time iterating on the list.
Once you’re satisfied with your list of categories, you can run the numbers to see how many bugs are in each category. Then you can create a Pareto diagram of the results to see what 20% of the improvement opportunities on your list will result in an 80% improvement. Now you can truly leverage escaped bug information into concrete actions that will make the test effort more effective.
Over time, as you try to use that original list to categorize reports of new bugs in the field, you will probably find that the list becomes less and less relevant. I hope so, anyway. It indicates that the improvement efforts are working, that the team has improved the test effort.
That’s when you know it’s time to do the process all over again to classify the next generation of escaped bugs.
How did we miss THAT?
Filed under: Thinking Like a Tester, Ruminations — Elisabeth Hendrickson @ 11:12 am
“Oh goodness. How did I miss THAT bug?”
Over the years, I’ve asked myself that question numerous times.
I asked that question when another tester found a blazingly obvious, critical bug that I completely missed. (The answer: I spent too much time tinkering with an ineffective automated script I’d written, and too little time observing the actual behavior of the system. That’s the project where I learned a lot about how NOT to do test automation.)
I asked myself that question, repeatedly, when I participated on a project some years ago now where we shipped software that crashed left and right. (I’m still sorting out the answers to that last one. Catastrophic failures are almost never the result of a single, simple error. And this particular catastrophic failure represented failures at all levels in an organization that had, um, issues. But I digress.)
I asked the question again when I learned that a web site I tested had back-button problems. After all, I was sure I’d tested for that. And I had. But I hadn’t re-tested for it after a particular set of code changes that changed some operations from HTTP GETs to HTTP POSTs. Oops.
And I asked myself that question more recently when I learned that a system I worked on earlier this year failed to save a change, and also failed to report an error, when it encountered data misformatted in a particular way in one specific field. Badly formatted data is one of my specialties, and I couldn’t believe I forgot to test the particular case that resulted in the problem. But it turned out that I did, indeed, fail to test what would happen if you entered “www.testobsessed.com” into a URL field instead of “http://www.testobsessed.com”. In hindsight, it’s an obvious test. Another lesson learned.
I reflected on those missed bugs when a colleague, Sandeep, a test manager, recently wrote to say that he’s been asking himself “how did my testing team miss that?”
He decided to seek out patterns of testing problems by categorizing escaped bugs according to the hole(s) in testing that allowed the bug to slip through. The idea is to improve the test effort by figuring out the common causes behind escaped bugs.
My initial reaction was, “that makes sense.” If you can identify the top 20% of testing holes that let 80% of the bugs through, and you can make some serious improvements to the test effort.
And my next reaction was, “but be careful.” Sandeep’s intent is good: use lessons learned from escaped bugs to improve testing. However, asking “How did we miss that?” is perilously close to heading down the slippery slope to “How did JoeBob miss that?” to “It’s JoeBob’s fault.” Having talked to Sandeep, I know he’s not trying to play “pin the blame on the tester.”
So I suggested a small reframe.
Instead of categorizing escaped bugs by asking the question, “How did testing miss that?”, categorize them by asking the question, “How can we improve the probability that testing will find bugs like that in the future?”
It’s a subtle difference.
But the result of reframing the question is that instead of identifying categories as noun phrases like “insufficient test data,” we end up with imperative statements like, “add test data.” Those two categories may look almost identical, but only one is actionable. I can “add test data.” The statement prompts me to do something different next time. But “insufficient test data” only gives me something to regret. And regret won’t help me ship better software.
So how can you categorize escaped bugs to improve the test effort without falling into the blaming trap? Try an Affinity Exercise with the question, “What could we do differently next time to increase the probability that if we have another bug like this we’ll catch it in test?”
To prepare:
Choose a team to participate in the activity. Affinity exercises can work with any number of people, but for this particular activity, I find a smaller group - say 3 to 5 people - works best. It’s a good idea to include people with diverse roles and skill sets.
Set up a meeting time and place. Plan for the whole activity to take 2 hours. And arrange to meet in a place with plenty of table and/or wall space.
Gather (or shop for) office supplies. You’ll need:
Index cards or sticky notes. Bigger is better. I like 5×8 cards or the SuperSticky 5×8 Post Its.
Felt-tip markers. I like Sharpies because they make consistently dark, readable marks. (Beware: Sharpies are permanent. Do NOT confuse your Sharpies and your White Board markers in the conference rooms. Facilities people get tetchy about such mix-ups.)
Gather a list of escaped bugs you want to analyze. If you have a lot of escaped bugs, prioritize them and time box the exercise. (You probably won’t be able to analyze more than 50 in an hour, possibly less, so don’t print out a list of 500.)
In the Meeting:
Review each bug with the team, asking: “What could we do differently next time to increase the probability that if we have another bug like this we’ll catch it in test?”
Have participants write their suggestions on the cards/stickies, one idea per card, in the form of an actionable statement. The suggestions should complete the sentence, “In the next release/iteration/sprint, we can ______.” Tips:
Also ask the participants to make their suggestions as concrete and specific as possible. For example, instead of writing “add test data,” write “add titles with ampersands (&) to the test data.”
And ask the participants to stick to test-related actions, and avoid blaming individuals. “Revoke NancySue’s checkin priviledges” is not an acceptable suggestion.
When you’ve reviewed all the bugs, or when an hour has passed, stop reviewing bugs. (If you still have lots of bugs to go and want to continue analyzing after an hour, stop anyway. Finish the rest of the exercise - the grouping. When you’ve worked through the whole process, if you still think more analysis would help, you can always do the exercise again.)
Gather all the cards/stickies, and lay them out on a large work surface: a table, the walls, or even the floor can all work well.
Sort through the cards/stickies as a team. The cards are now owned by the team, and everyone should take a hand in organizing them. Encourage participants to move cards that seem alike together so they are stacked together. Continue until the team agrees that it’s satisfied with the stacks of cards/stickies.
Ask the team to give meaningful names to the stacks of cards/stickies. This is the part of the activity where will generate the more abstract categories like “add test data.”
The result of this exercise is a list of categories for improving the testing effort that we can then use to determine which kinds of improvements will have the biggest bang for the buck. And the best part is that the list emerged from the actual problems your software has had in the field rather than being some arbitrary list of theoretical “improvements” based on someone else’s unrelated experience.
But wait; you’re not done. Now that you’ve created a first draft list of categories, test it. (Did I mention I’m Test Obsessed?) Choose a different set of escaped bugs, and assign each to one or more categories from the list. Notice how easy or hard it seems to find a category for each bug. This will give you a lot of feedback about how well the category list will work in practice. You may find the team needs to spend some additional time iterating on the list.
Once you’re satisfied with your list of categories, you can run the numbers to see how many bugs are in each category. Then you can create a Pareto diagram of the results to see what 20% of the improvement opportunities on your list will result in an 80% improvement. Now you can truly leverage escaped bug information into concrete actions that will make the test effort more effective.
Over time, as you try to use that original list to categorize reports of new bugs in the field, you will probably find that the list becomes less and less relevant. I hope so, anyway. It indicates that the improvement efforts are working, that the team has improved the test effort.
That’s when you know it’s time to do the process all over again to classify the next generation of escaped bugs.
Subscribe to:
Posts (Atom)
