Tuesday, December 22, 2009

What is Fuzz testing

On weekend, one of my friends talked about Fuzzy testing, which pushed me to do some "googling" about fuzzy testing. I am sharing this information however, you can read about fuzzy testing more on wikipedia.
Fuzz testing or fuzzing is a software testing technique that provides invalid, unexpected, or random data to the inputs of a program. If the program fails (for example, by crashing or failing built-in code assertions), the defects can be noted.
File formats and network protocols are the most common targets of fuzz testing, but any type of program input can be fuzzed. Interesting inputs include environment variables, keyboard and mouse events, and sequences of API calls. Even items not normally considered "input" can be fuzzed, such as the contents of databases, and shared memory.
However, fuzz testing is not a substitute for exhaustive testing or formal methods: it can only provide a random sample of the system's behavior, and in many cases passing a fuzz test may only demonstrate that a piece of software handles exceptions without crashing, rather than behaving correctly. Thus, fuzz testing can only be regarded as a bug-finding tool rather than an assurance of quality.The oldest and simplest form of fuzzing, sending a stream of random bits to software, continues to find bugs in command-line applications. Another common technique that is easy to implement is mutating existing input (e.g. files from a test suite) by flipping bits at random or moving blocks of the file around. But the most successful fuzzers have detailed understanding of the format or protocol being tested.
Types of bugs found: Straight-up failures such as crashes, assertion failures, and memory leaks are easy to detect. The use of a memory debugger can help find bugs too subtle to always crash.
Fuzz testing is especially useful against large C or C++ applications, where any bug affecting memory safety is likely to be a severe vulnerability. It is these security concerns that motivate the development of most fuzzers.
Since fuzzing often generates invalid input, it is especially good at testing error-handling routines, which are important for software that does not control its input. As such, simple fuzzing can be thought of as a way to automate negative testing. More sophisticated fuzzing tests more "main-line" code, along with error paths deep within it.

Reproduction and isolation: As a practical matter, developers need to reproduce errors in order to fix them. For this reason, almost all fuzz testing makes a record of the data it manufactures, usually before applying it to the software, so that if the computer fails dramatically, the test data is preserved. If the fuzz stream is pseudo-random number generated it may be easier to store the seed value to reproduce the fuzz attempt.
Once a bug found through fuzzing is reproduced, it is often desirable to produce a simple test case to make the issue easier to understand and debug. A simple testcase may also be faster and therefore more suitable for inclusion in a test suite that is run frequently
Enjoy testing :)

Tuesday, November 17, 2009

Defect Removal effectiveness

Defect Removal Effectiveness (or efficiency as used by some writers) is calculated as:

DRE = Defects removed during a development phase/ Defects latent in the product at that phase x 100%

Since the latent defects in a software product is unknown at any point in time, it is Approximated by adding the number of defects removed during the phase to the number of defects found later (but that existed during that phase).

Defect Removal Effectiveness can also be calculated for the entire development cycle to examine defect detection efforts before the product is released to the field. According to Capers Jones, world class organizations have Development DRE greater than 95%.
Development DRE = (Pre-release Defect) / (Total Defects) x 100% = X%
The longer a defect exists in a product before it is detected, the more expensive it is to fix knowing the DRE for each phase can help an organization target its process improvement efforts to improve defect detection methods where they can be most effective. Future DRE measures can then be used to monitor the impact of those improvement efforts.

More details can be read from here


Monday, November 16, 2009

Microsoft open sources .Net Micro development framework

Microsoft announced on Monday the release and open-sourcing of its .Net Micro Framework 4.0, which provides a .Net-based development and execution environment for small devices. But the TCP/IP stack and cryptography stack were omitted from the open source effort.
Version 4.0 is available under an Apache 2.0 license. Microsoft still intends to remain actively involved in development of the framework, said Microsoft's Peter Galli, senior open source community manager, in a blog post.
[ Recently, the Microsoft-backed CodePlex Foundation for open source projects outlined project submission guidelines. ]
"While the license will allow customers to take the code and make specialized versions to fit their needs, customers told us they wanted Microsoft to stay involved to avoid any possible fragmentation of the platform," Galli said. The Net Micro Framework 4.0 effort was revealed at the Microsoft Professional Developers Conference in Los Angeles.
As part of the open source move, developers get access to source code for almost all of the product, including base class libraries and CLR (Common Language Runtime) code. The libraries contain implementations of primary functionality that managed applications call into for communications, system functions like threading, UI and more, according to Microsoft's Colin Miller, program manager for .Net Micro Framework.
"However, both the TCP/IP stack and cryptography libraries are not included in the source code," Galli said. "Program Manager Colin Miller told me this was because the TCP/IP stack is third-party software that Microsoft licenses from EBSNet, so we do not have the rights to distribute that source code. If someone needs to access the source code for the TCP/IP stack, they can contact EBSNet directly."
"As for the cryptography libraries, they are not included in source code because they are used outside of the scope of the .Net Micro Framework. Customers who need to have access to the code in the cryptography functions will find that these libraries can be replaced," Galli quoted Miller as saying.
An analyst applauded the open source move but cited the TCP/IP omission as a drawback.
"It's an interesting decision on Microsoft's part and one that seems to acknowledge the benefits of open source in terms of visibility and distribution," said analyst Stephen O'Grady, of RedMonk. "That said, the lack of an included TCP/IP library is likely to throttle interest, because even the "resource-constrained devices" that are the [framework's] targets must function in a network context."
Microsoft plans to establish a core technology team for the framework made up of Microsoft and non-Microsoft contributors. A community also is being formed to help shape direction of the product.

Thursday, November 12, 2009

Important Usuability Concepts

User-centered design (UCD)
User-centered design is a design philosophy in which users, their needs, interests and behavior define the foundation of web-site in terms of site structure, navigation and obtaining the information. UCD is considered as a standard approach for modern web-applications, particularly due to the rise of user generated content.
Vigilance (sustained attention)
Vigilance is the ability to sustain attention during prolonged, monotonous tasks such as proofreading a text looking for spelling errors, reminding of appointments, auto-saving word processor documents etc. In modern web-applications vigilance tasks are performed in background, automatically and thus improve the usability of the service.
Walk-Up-And-Use Design
A Walk-up-and-use design is self-explanatory and intuitive, so that first-time or one-time users can use it effectively without any prior introduction or training.

I found these concepts quite interesting regrading ususability and website designing, you can read deatiled material from here

Wednesday, November 11, 2009

Google released 'Go' as open source

Google has invented a new programming language designed to reduce the complexity of coding without compromising the performance of applications.
Called Go, the language has been tested internally at Google but is still at an experimental stage, so the company is releasing it Tuesday as open-source code in the hope that it will get help with its future development.

Go aims to improve on the way existing programming languages manage dependencies, which are the software components that applications re-use, such as libraries,. The language also is designed to handle multiprocessor work particularly well, thanks to its concurrent programming model.
Google started working on Go about two years ago and devoted a team to work on it full time about a year ago. It was conceived as a language for systems programming, such as Web servers, storage systems and databases. However, Google is open to seeing it branch out into other areas.

At this point, Go isn't used in any user-facing Google service or application, since it still has to mature, which is where external programmers come into the picture. "We need better libraries and tools, and the open-source community is great at helping you with things like that.
By creating a new programming language, Google continues making inroads into the realm of computing building blocks, prompted by a sense of urgency at making them better. This motivation has also led Google to embark on developing the Android mobile operating system, the Chrome PC browser and the still-unreleased Chrome operating system.

Monday, November 9, 2009

Appearing in an interview

Almost three months back, I got interview call from the one of reputable software house in the town, XYZ Ltd. This organization is quite known for its high salary package and odd timingsJ. The thing which I want to share with you guys is interview process and the way XYZ conducted interview for QA/Tester candidate. I had to go from 5 interview sessions.
  1. Written GRE based test
  2. Technical interview (For Software Test Engineer)
  3. HR Analytical interview
  4. 2nd Technical Interview
  5. Final HR interview


Fortunately I cleared all theses interview sessions and got the offer eventually, which I did not accept due to some reasons, but in technical interview, manager asked me to solve two Programming problems, and asked questions regarding DB, OOP, Web architecture and programming concepts. Solving Programming problems was quite challenging for me, and I was not expecting such questions especially for QA/Tester position, but that was great learning experience and I enjoyed a lot.

What is your opinion, what is your point of view about programming questions asked in an interview for a post of QA/tester?

Monday, November 2, 2009

Writting Blog is fun

When I started my blog few months back, I was quite excited and had lots of plans, people around me appreciated my effort but most of the people said that its difficult to mange, writing posts on every day, seems quite committed task to do, In the start, I posted almost 3 to 4 posts per week, then 2 to 3 per week, and at last month I could manage only one post for whole month, :)
But I will must say that writing blog is always great experience, you learn a lot while writing, for good writing you have to be good reader, so guys I am back now, trying to be more regular in my posts.

Keep Testing.

Friday, October 2, 2009

Involve Testers from the Beginning

Testers need to be involved from the beginning of a project's life cycle so they can understand exactly what they are testing and can work with other stakeholders to create testable requirements.
Defect prevention is the use of techniques and processes that can help detect and avoid errors before they propagate to later development phases. Defect prevention is most effective during the requirements phase, when the impact of a change required to fix a defect is low: The only modifications will be to requirements documentation and possibly to the testing plan, also being developed during this phase. If testers (along with other stakeholders) are involved from the beginning of the development life cycle, they can help recognize, discrepancies, ambiguities, and other problems that may affect the project requirements Testability, correctness, and other qualities.

A requirement can be considered testable if it is possible to design a procedure in which the functionality being tested can be executed, the expected output is known, and the output can be programmatically or visually verified. Testers need a solid understanding of the product so they can devise better and more complete test plans, designs, procedures, and cases.
Early test-team involvement can eliminate confusion about functional behavior later in the project life cycle. In addition, early involvement allows the test team to learn over time which aspects of the application are the most critical to the end user and which are the highest-risk elements. This knowledge enables testers to focus on the most important parts of the application first, avoiding over-testing rarely used areas and under-testing the more important ones.

I have taken this topic from the book” Effective Software testing” by Elfriede Dustin.

Monday, September 28, 2009

Exclusive interview with Mark Crowther

I spoke to Mark Crowther a few months ago about his book An Introduction to Web Test Automation with Selenium & Ruby and agreed to do a review of the first few chapters. When I followed up with comments and feedback we chatted about the project and what follows is some of the main questions and answers that came up during that conversation. I really appreciate Mark's kind behavior and quite encouraging response he responded to all my questions quite patiently.

How long have you been in software testing and what’s your current role?
Just over ten years in software testing and about five in manufacturing QA and testing. My first Test Manager job was with AOL (UK) in London and I was there about 5 years or so. Since then I’ve worked at Electronic Arts and lastminute.com as their Head of QA. My current role is with NMQA a test solutions provider based in London where I’m Head of Testing Practice. That means I’m responsible for anyone in the business who does, thinks, talks or breaths testing.

What’s your testing philosophy?
A take a very context driven, holistic approach to the way I do testing. I rarely deliver a testing engagement or testing to a project in quite the same way. My perspective is that Waterfall, V-Model, Agile, Scrum, etc, and even artefacts such as Test Strategy’s or Bug Reports along with practices such as bug advocacy and reporting are all useful and have a place in the testing world. The way I see it a test professional has to understand all of these so they can participate fully in the test community and respond to project needs in a way that is appropriate and correct. Just saying you’re agile or only manual tester isn’t professional. You face a testing problem you need to pick the right tools.

Who are the testing professionals you admire and why?
Well, James Bach is obviously one because he was the earliest influence on me in terms of challenging me to think about what I was thinking about. Michael Bolton partly because he’s been around forever and partly because I learn something off this guy every single time I read something he’s written. James Lyndsay who teaches Rapid Testing with Michael Bolton, the material he teaches can transform a tester if they’re willing to take the material on board. Elizabeth Hendrickson as she’s a deeply technical, talks constant sense and really seems to contribute a lot to the testing community. Matt Heusser, Bret Pettichord,

What are you currently reading?

I have time for reading? Just kidding! Book wise I’m reading ‘The RSpec Book’ to learn more about that and related items such as BDD and Cucumber. Most of the time I do my reading on the forums to be honest.

In a few sentences how would you describe the book and who is the book written for?
The book is very much a hands-on user’s guide to getting set-up and working with the Selenium tools and using Ruby as the language of choice. The idea is that in a week of sitting with the book at your computer you’ve got a fully open source Selenium-Ruby based automation framework, with a set of template test examples, some neat ideas on how to manage your automation effort and you’re off using it.

The book is a kind of idiots guide or a Selenium-Ruby for Dummies?
Right, think of the book as a manual you’d expect to get if Selenium-Ruby was some kind of software package you bought off a vendor and are now sitting down to install and configure. It’s meant to be step by step with some insights and techniques that’ll help you learn to use the framework. The book assumes you know what testing is but doesn’t assume knowledge of Selenium tools or Ruby.

Can you give me some detail about the book structure and specific examples of what’s covered?
Sure, the book is currently ten chapters and just two chapters are about setting up the Selenium tools of IDE, Remote Control and Grid. We’ve got a chapter dedicated to installing the Ruby language and Ruby Gems we need, getting SciTE and EasyEclipse set-up and making sure Ant and Java are on the system correctly.

There’s a walkthrough of how to use the IDE for rapid prototyping of tests and over two chapters a detailed look at using Grid in various ways. In another chapter we do a code walkthrough of a Selenium-Ruby template script looking at each piece of the script and why it’s there. The book also has a chapter dedicated to creating ten template scripts that cover common testing challenges such as doing cross browser and data driven tests, testing Flash, dynamic content, handling pop-ups and Ajax and also capturing screen shots.

OK, so what would you say the book isn’t? What will a reader not find in there?
It’s not a file by file technical dissection of the tools themselves, I don’t go through scripts and files parsing out the lines of code and looking at how and why it’s programmed in a certain way. There are small amounts of this where relevant to setting up the framework, for example where we look at configurations appearing in the Grid Hub page, but I’m not cracking open jar files just for the technical fun of it.

What new things are in the book that I can’t find elsewhere?
There are three sections within various chapters that are about practices I’ve either developed or employed. They’re related to Test Case Trees, Environment Management and using Pivotal Tracker. I’ve added these to give readers some extra support around their new automation framework. Given there’s no book focused on Selenium-Ruby the whole book is kind of unique and add in it contains ‘fixes’ for common configuration issues there’s a lot there for the tester trying to quickly get the framework in place and move on to doing testing.

Why did you choose to combine Selenium with Ruby? Why not choose Java for example?
I wanted to create a framework based entirely on open source technology and with components that are easy to learn and use. As a tester I don’t consider Java that easy to use, Ruby is easy to read, learn and use and it’s far prettier and efficient, at least in my view!

How much Ruby is there in the book? Does a reader need to be a Ruby guru?
In terms of complexity of coding and use of the language there’s not that much. There’s just enough to complete the templates plus a few neat tricks to show how Ruby can help us in our testing. This isn’t a Ruby programming book, remember the book is a intended to get all the components in place and the reader delivering testing it’s not a Selenium architecture or Ruby course.

Isn’t there already a book or website about Selenium automation?
You’d think so wouldn’t you? In fact there’s not the expected raft of Selenium books out there that you’d expect. If you look over Barnes & Noble, Amazon, etc. there appears to be exactly one about Selenium with Twill. There’s nothing like the book I’ve written as far as I can tell.

What’s been the most difficult for you when writing and researching the book material?
Writing’s been fairly easy, sure it takes forever, but it’s not hard. I’ve also implemented everything I’ve written on a number of client sites and in my test team at NMQA which while sometimes tricky was the real fun part. It helped prove out everything I’ve written too. I’d say the hard or tiring bit has been the research and the disappointment of not getting answers to questions I’ve asked on forums, etc. It just disappointing to put the effort in to work bits out yourself, search for answers online or with colleagues and post questions thinking it’ll be fun to get talking to folks and end up with no replies.

Final question, when can we get a copy and from where?
I’m hunting for a publisher now, the manuscript is ready subject to review and no doubt some rewrites that will be required. I’ll have to ask folks to wait a few months for the publisher to give the book the nod and start making it available. If I get enough knock-backs, as in no interest, I’ll just self-publish via Lulu ready for November or December latest.

Final thoughts before we finish?

You know, this book is really about helping other testers like myself avoid the issues I encountered that might stop them adopting Selenium tools, that might mean they never get to learn about them and use them. I hope folks will enjoy the book and get excited about Selenium and Ruby, I also hope that the Java, C#, Python, etc. Selenium community out there will ‘translate’ the book into examples that apply to their chosen languages.


This interview is reproduced with permission of the author and is copyright Kashif Ali (C) 2009. It is prohibited to reproduce this interview in any form without written permission from the copyright holder. All rights reserved.

Tuesday, September 15, 2009

Is it possible to find all error in the application?

Software testing is an integral phase of software testing, testers are the people who always question the reliability and stability of the software application, considering the role of software testing, it is also claimed that it is not possible to find all errors in the application. Let’s use basic testing types in order to start discussion about this statement.

Black Box Testing
Lets explore Black testing type, this testing type is also called Data Driven testing, Input/Output testing. As it quite known in this type of testing, testers are only concerned with the functionality of testing, and treat system as a black box, they derive their test cases from the functional behavior and least bothered about the internal structure of the application. So if we want to find all errors in the application using black box testing , then we have to use to technique of exhaustive input. That is we have to cater each possible input to the application.
Testing software with each possible input, sounds impossible because it is impossible to measure all possible valid or invalid input condition for example consider a program, which takes any two numbers for addition, now if we analyze the input to that program, it can be from 0 to infinity, in the same way negatives infinity to zero. All of us is agreed that it is the most simple computer program, but still in order to test program with each possible input condition is not possible. Consider a program like Compilers, operation system, or flight control system in which you have to derive test case for each possible input, what do you think it is possible?
So testing software with all exhaustive input is not possible, so we can test the program with every input to the program, thus claiming that software is error free for all possible input can not be feasible and practically impossible task to do,
But yes we can use different techniques to build our confidence about the stability of the program by using test case design technique that is (Equivalence partition, boundary value analysis etc and the objective should be to maximize the yield on software testing investment by finding maximum number of errors in definite number of test cases.

White Box testing
Another testing technique, white box testing also know as logic driven testing, in this type of testing test cases are derived from the internal code logic of the program, that by considering all logical paths, testers design test cases, in this type of testing all logical path are tested to find maximum errors, so as we discussed in black box testing we test program with exhaustive input, in this scenario we will say it as exhaustive logic path testing.
Testing software by designing test cases that cover all logic paths to ensure that software is error free, is quite impossible because we have to go through all possible paths of control flow in order to test software 100 % accurately, when we talk about testing each logical control path, consider a program which runs almost 50 line of code within loop, and for every iteration of loop, it passes through different branching conditions, so writing test case for every nested statement, seems impractical, if not impossible.
Considering if we are somehow able to test each control flow path of the program, but still it is not sure that software will be error free, due to following reasons:
1-If we test all logical code paths, it will not ensure that code has been written according to specification provided. For example if programmer writes ascending order routine instead of descending order, by testing all logical paths, we can not guarantee that there will be no error in the application.
2-There can be error due to missing input paths, because exhaustive input can not determine important logical paths.
3-Exhaustive input testing does not cover data sensitivity errors.
Looking for your feedback :)

Tuesday, September 8, 2009

Interview with Mark Crowther

I spoke to Mark Crowther a few months ago about his book An Introduction to Web Test Automation with Selenium & Ruby and agreed to do a review of the first few chapters. we had a dicussion on different topics of the book, i compiled the conversation in the form of an interview, it contains interesting information about mark's book. and mark's passion about software testing so be ready for "Mark Crowther exclusive interview :)
'

Friday, September 4, 2009

Simplify your bug report

Communication between developers and testers plays quite important rule, often we see a debate going on about the bugs between developer and tester, there are few Things that makes programmers to resist spending their time on the bug that are:

• The programmer can’t replicate the defect.
• Strange and complex set of steps required to induce the failure.
• Not enough information to know what steps are required and it will take a lot of work to figure them out.
• The programmer doesn’t understand the bug report.

The point of testing is to find bugs. Bug reports are your primary work product. This is what people outside of the testing group will most notice and most remember of your work. Programmers operate under time constraints and competing priorities. A bug report is a tool that you use to sell the programmer on the idea of spending her time and energy to fix a bug.

How to simplify your bug report
1-Write each step in detail.
2-If you see two failures, write two reports combining failures on one report creates problems: 3-You’ll often see one bug get fixed but not the other.
The summary description should not be vagued. The words like “fails” or “doesn’t work” should not be used instead of describing the failure more vividly. This weakens the Impact of the summary.
4-Put Variations after the Main Report Suppose that the failure looks different under slightly different circumstances. For example, the timing changes if you do a additional two sub-tasks before hitting the final reproduction step. This is all useful information for the programmer and you should include it. But to make the report clear.
5-Write suggested solution if possible.
6-Write expected Benefits to the customers and vendors

you can read the detailed article on bug advocacy from here

Wednesday, August 26, 2009

Who is good tester?

We can have various points eloborating the traits of good testers, rather we can have a good debate on this topic, but Cem kaner have described beautfully,i liked his statement a lot , according to him

"The best tester isn’t the one who finds the most bugs or who embarrasses the most Programmers. The best tester is the one who gets the most bugs fixed."

Enough for today's post :) , looking for our feedback

Saturday, August 22, 2009

Voice of software testers

Few days back, there was a discussion on STC, what testers hate about testing, Rosie Shery concluded different comments, I liked following comments

  • Finding the same silly bugs year after year whatever program you are testing - leaving an input box blank, finding that 99999999999 overflows. etc
  • Calling testers as QA (Quality Assurance), QA is process but not a designation title.
  • Unreliable test environments where most of my time and effort is expended identifying
    environment issues rather than product issues.
  • Code that isn't architected in a testable manner, causing testers to have to write ntegration tests when simple unit tests could find the same bugs.
  • Testers who get comfortable with what they already know and stop pushing themselves to learn more.
  • Developers believing they know HOW to test, WHEN to test and WHAT to test. (Note: Not aimed at ALL devs. Just some of them.) (Note: If they do know testing - then why do we find so many bugs?)
  • Delivery date stays the same when development time spills over (way over).
  • No/Bad/Clueless responses from project managers.
  • Sometimes we view ourselves as second citizens.
  • The perception that a career in testing is somehow of less value.
  • Successful testing is about passing tests.
  • Testers testing on a developer's PC, and bad lab environment for testing.
  • Generate and share meaningless reports/metrics.
  • Waiting for the release to happen from development team.
  • Testing team not informed about the changes.
  • Testing time being crunched because developers ran over


You can read the detailed thread here

Tuesday, August 18, 2009

Famous Five Why Analysis

In my last post, I described about “kaizen”, a continuous improvement process, but now the question is how we can introduce “kaizen”, in our lifes and partcularly in software development. Its always a great practice to review your work , either you are working in agile mode or some other other incremental approach, in some organizations, there are Post Project Report,or when we talk about agile , we have sprints and product owners often call for Sprint review meeting.

Five Why analysis is form of kaizen, it’s a technique of root cause anlaysis, it is a meeting of all stake holders , discussing critical issue faced during sprint or in project.

Lets suppose we faced a critical issue during sprint end while deploying our application to the production , the problem was that one of the key component was throwing exception on the server , but working fine in our environment with same code, team suffered a lot due to this issue ,suppose we solved the problem on priority basis, in order to complete our sprint, but now is the time to analyze about this issue using five why analysis

So


Why the component was not working in production environment?

The component was dependent upon the utility , whose files (Dll, EXE etc ) was not installed on the server.


Why it was not installed on the server?

During sever set up, team missed this utility, because it was not mentioned in the check list prepared for server set up.

Why it was not in the check list?

Because the respective code(utility) was not checked-in the code repository by the development team, and resource who preapred check-list, listed all componenets that were available on the Repository.

Why it was not checked in to the code repository?

Because developer did not push the utiliy to the code repository, and conducted unit tests on his machine,because he was not aware of that utility as well.

How testing team missed that dependency?

Because, the utility was already deployed to the QA server long time ago, so testing team was unaware of that utility.

So after these question,. We concluded that :


  • There should be properly reviewed installation checklist for server set up.
  • Before production, there should be a Pre-Production server, where Deployment team set envoirnment by taking with the code repository and executing checklist.

Its just an example, there are many scenarios, so just conducting a five why analysis at the end of each miles stones, we can reduce our gaps,if this practice continues, we can have our improvement process continue.

you can read more about five why form here

Thursday, August 13, 2009

Kaizen(Continuous Improvement Process)

Kaizen is a Japanese philosophy that focuses on continuous improvement throughout all aspects of life. When applied to the workplace, Kaizen activities continually improve all functions of a business, from manufacturing to management and from the CEO to the assembly line workers.By improving standardized activities and processes, Kaizen aims to eliminate waste. Kaizen was first implemented in several Japanese businesses during the country's recovery after World War II and has since spread to businesses throughout the world
Kaizen is a daily activity, the purpose of which goes beyond simple productivity improvement. It is also a process that, when done correctly, humanizes the workplace, eliminates overly hard work ,and teaches people how to perform experiments on their work.

In software development, people talk a lot about incremental approach, specially in agile mode, where we plan sprints and release product with minimal features, and then we keep on improving the product/application in each sprint, this is where we implement “kaizen” in software development.

There are ways by which we can continue our improvement process, that I will write in my coming posts :)

Tuesday, August 11, 2009

Collect Stats that impact our software business

James whittakar is a quite respectable name in the software testing world; I came across his article about agile testing, in this article, he emphasized on some valuable data.i am writting brief description of his point of view.

We used to compile different status report during our testing phases like, number of test cases executed, number of test cases failed or passes etc. theses status reports are quite important for QA managers or project managers, to decide various thing like release time etc but these status reports do not reflect the impact on business that we are facing due to bugs found during testing cycle.
So one type of data that would be of value is direct business impact statistics like:

1. Sales missed due to bugs,
2. Customer renewal rate and relative cost of defect
3. Defect Density
4. Containment

This type of information is like gold to decision-makers. Brief description of the defect density and containment is given below

Defect Density: Information that shows number of bugs in the context of the size of the particular project module or phase. For example, if a requirements spec for a particular project is 120 pages, and 52 defects were collected relating to this spec, then the Defect Density is 0.43/pg. Similarly, you can measure Defect Density per kilo line of code or per object or module design.

Containment: This measures the success of preventing errors from propagating. As we all know, the earlier a bug is identified, the lower the cost of fixing it. Measuring containment, therefore, can tell us where we can save money and time.

You can read the detailed article from http//utest.com

Friday, August 7, 2009

Be an Agile Tester

In a business of software, every organization has to deal with multiple challenges like, Delivering quality Product, Satisfying customer needs, On-time Release within allocated resources an so on, on the other hand, organizations have to meet the coming customer ‘s requirements , and changing market trends. So software product has to go through multiple releases with additional features in every release, agile model helps organizations to work with idea of “functional feature in the release, delivered to the market” as soon s possible.
In this model, testing plays quite important role, because we always performing testing in agile model, As Agile projects do not necessarily have comprehensive documentation. Product requirements in agile projects are often captured in the form of user stories. The testers should be experienced and smart in analyzing and defining the testing requirements in the absence of any product documentation. The ability to work side-by-side with the developers, analysts and customers is an important part in quickly understanding the test requirements. The testers should also be equipped with alternative ways of deriving test requirements.
  • Getting access to all the material that can provide input for the intended product behavior. This can include input provided to the developer to build the application, feature list, brief write-up on application capabilities and partially written use cases etc.
  • Knowledge transfer from product owners and subject matter experts (SME).
  • Gaining insights into the capabilities needed from the product under test by exploration of comparable products, reading user manuals and user documentation, exploring help text via user interface and taking an application tour through the GUI.
  • The testing team should be well trained to scope the work, ask the right questions, and deliver a valuable output within a few hours. Exploratory test engineers should be able to analyze a product, think critically to evaluate risks and craft test cases that systematically explore the product.
You can read detailed article about agile testing from http://applabs.com.
Looking for your feedback :)

Saturday, August 1, 2009

Philosophy of testing

Often software testing is discussed from different technical aspects, but it is quite important to discuss psychology and economics of software testing, as software testing is quite integral part of software development life cycle, so it is quite important to adopt the appropriate frame of mind towards testing. So lets starts our discussion.

Definition Of Testing
It is quite adequate that purpose and objective should be cleared before starting any activity, the same is true with software testing, people use different definitions of testing, e-g we come across statements like

“Testing is the process to make sure that errors are not present in the software.”
Or
“ The purpose of testing is to show that program is performing its intended functionality.”
Or
“ Testing is the process of establishing the confidence that a developer does what it is supposed to do.”


If we analyze these statements, we will notice that these are reflecting exactly the opposite to the objective of software testing because testing is an activity which requires human resources and effort, so one can think what should be the outcome of software testing?

Problematic Definitions
If we analyze the definition such as “Testing is the process to make sure that errors are not present in the software.” this definition has psychology effect, because it is not possible that any software is error free and this goal is quite impossible to achieve so when any testers start testing with this mindset, it will not end up in good results because psychology studies show that human perform poorly in a job which they know it is not feasible or impossible. So defining software testing as a process of uncovering errors makes it a feasible task, and thus overcoming the psychology problem.

In the same way, we have a definition “The purpose of testing is to show that program is performing its intended functionality”. It is quite evident that error would be there if the software is not doing what it is supposed to do, but errors can be there if the software is doing what it is not supposed to do.

Correct Definition of Testing
The outcome of software testing should be that software application should be reliable, it can only be reliable when it is error free, it can only be error free when testing is started with assumption that it contains errors in it. If we start testing with this frame of mind, then testing can add lot of value in software/application development. So the correct definition of software testing can be stated as:

“ Testing is the process of executing the program with the intent of find errors”.

Here human psychology comes into play because humans are highly goal oriented, and establishing a proper goal has an important psychological effect. If the testing is done with a goal that program has no error then testers will subconsciously steered towards this goal, and he will un consciously select the test data that has low probability of finding errors, on the other hand if testing is done with the goal that errors are there in the application, then software testing will add a great value to the application, and testers unconsciously steered towards test cases and test data that has high probability of finding errors.

Conclusion
To summarize this vital discussion, we can say that software testing is a destructive and sadistic process of finding the uncovered errors in the program Its human nature that most of us are geared toward building objects rather ripping them apart, that is why testing is a difficult job to do.
A successful test case is one who progresses in this direction and finds error in the software, because purpose of software testing is to establish a confidence that software does what it is supposed to do, and does not do what it is not supposed to do.



Wednesday, July 29, 2009

Testers Vs Managers

Yesterday, I came across a presentation from Matthew Heusser, I found this joke, si I though it would be great if I share this and post on my blog.

Man Reducing altitude, he spotted a man on the ground and descended to shouting range.

"Excuse me," he shouted. "Can you help me? I promised my friend I would meet him a half hour ago, but I don't know where I am."

The man below responded: "Yes. You are in a hot air balloon, hovering approximately 30 feet above this field. You are between 40 and 42 degrees North Latitude, and between 58 and 60 degrees West Longitude."

"You must be a tester," responded the balloonist.
"I am," the man replied. "How did you know?"

"Well," said the balloonist, "everything you have told me is technically correct, but I have no idea what to make of your information, and the fact is I am still lost."

Whereupon the man on the ground responded, "You must be a manager."

"That I am" replied the balloonist, "but how did you know?"

"Well," said the man, "you don't know where you are, or where you're going. You have made a promise which you have no idea how to keep, and you expect me to solve your problem. The fact is, you are in the exact same position you were before we met, but now it is somehow my fault. “

Sunday, July 26, 2009

Rubber Duck Principle



Code inspections and code walkthrough are two important techniques of human testing, along with formal testing methodologies, these techniques help in finding errors early in the development cycle and reduce the bug count. Theses inspections and walkthroughs help the developers to improve their quality code and reducing bug count. But there is some thing you can do as well.

The simple act of reading aloud your program/Solution/Scenario to yourself is a good technique of finding errors. This technique is also called “Rubber Duck Principle”.

It is applicable to all, whenever you are trying to solve any problem, or as a tester creating scenarios, take any thing on your desk and consider it as “Rubber Duck” :). And explain the whole problem or whole solution to him; you will feel while narrating, you will find other missing parts in your solution or problem.

It will not only help you to identify your problem but also act as a good Peer review as well.

So put something on your desk as “Rubber Duck” as your team member :)

Friday, July 17, 2009

Do not throw test cases

As designing test cases is quite important activity of testing phase , does not matter what approach we are using that is traditional waterfall or agile.
Do not throw test cases until the program life has ended, when we create test cases to test a specific functionality of a module, effort is there to create test cases to cater all scenarios. But after testing if the test cases are thrown away, it would create difficulty when we have to re-test the program it mean the effort we put for creating test cases will be lost so maintaining the repository of test cases is a good practice.

Having test case repositor helps a lot when we are going to automate regression testing as well.

Tuesday, July 14, 2009

What is a successful test case?

Successful test case is one who finds an un-discovered error in the application; test cases are designed having different methodologies like (boundary value analysis, Cause effecting graph, or equivalence partition). Writing test cases which are un-able to cover maximum part of code and obviously have the probability of finding very minimum errors puts question mark in the effectiveness of test cases.
In an organization, development managers use the term “successful test case “ in a different way, it is observed if after executing 50 test cases, 35 test cases find errors in the application while execution, the managers used to say the 35 test cases are not successful and 15 are successful J.
But as we said the test case that finds error in the application is a successful has proven to be a valuable investment and test cases which do not find any error in the application, because designing test cases is a costly activity.

Thursday, July 9, 2009

How to create software model for testing



The reality is that modeling does not represent a new skill. Test engineers always build models. The only question is whether or not the models are in a persistent form. The ‘model’ may only exist for a short time and live on a napkin or in the mind of the engineer.
In order to write a test script or test plan, an engineer must understand the basic steps required to use the system. Modeling at the behavioral level is very similar to flowcharting; the major transactions in the use of the product are defined in a graphical format. The sequence of actions that could occur during the use of the system are defined. The actions that “could” occur also imply that that there may be more than one possible action at a specific point in the process.
There are several approaches that can be used to develop tests from a model; most of these is the concept of a path. A path is a sequence of events or actions that traverse through the model defining an actual use scenario of the system. Each element in a path, a transition or state, can have some test primitives associated with it. The primitives will define what test actions are required to move the system from its current state to the next state, verify that the state is reached or check that the system has responded properly to previous inputs. Once a path through the model has been defined, a test script can be created for that path. When this script is applied to the actual system, the actual system will follow the same sequence (or path) as defined by the model path from which the test script was extracted. This process can then be repeated for another path, which defines another use scenario, and verifies another sequence of actions. Many methods can be used to select paths, each with its own distinct objectives and advantages.
Just for example, I have shown a sample login flow chart, here we have build model for login functionality, and we mapped the paths, both successful and un successful. Now it is same thing that testers usually have in his mind, by building model, we translated our application flow in the form of model, now considering each path, we can design our test cases.

This short description is taken from the article from Larry Apfelbaum and John Doyle and login flow chart is just for sample from downloaded from google.

looking for your feedback :)

Tuesday, July 7, 2009

Help your project manager to decide about shipment


Project Manger: can we ship our new release on this weekend?
QA: hmm, well, there are few critical bugs that are show stopper. if we ship the code to the client, we will suffer a lot in the production support.
Project Manager: ok, let me talk to Dev TL

This conversation is a part of our professional life and we often come across such statements. The job of software testers/Quality engineer is to find bugs but there are some other important things that can add value to over all products like

Block premature product releases.
This tester stops premature shipment by finding bugs so serious that no one would ship the product until they are fixed. For every release decision meeting, the tester’s goal is to have new showstopper bugs.
Help managers make ship / no-ship decisions.
Managers are typically concerned with risk in the field. They want to know about coverage (maybe not the simplistic code coverage statistics, but some indicators of how much of the product has been addressed and how much is left), and how important the known problems are. Problems that appear significant on paper but will not lead to customer dissatisfaction are probably not relevant to the ship decision.
Minimize technical support costs.
Working in conjunction with a technical support or help desk group, the test team identifies the issues that lead to calls for support. These are often peripherally related to the product under test.

I think it is quite important that quality engineer should know about the overall status of the release, about its stability, and helping project mangers and other stake holders in making decision about the shipment of release

I have taken these testing objectives from Cem kaner article “What is a good test case”.You can consult in case of further reading

Sunday, July 5, 2009

Model Based Testing (Testing using flow Diagrams)

What is Model Based Testing
Few days back, my dearest teacher and good friend of mine Zahid Irfan, replied to my post and asked me about my view on model based testing, his questions inclined me to read some more material about model based testing, so that I can answer his question, After studying lot of material I am summarizing my views about model based testing.

Models are used to understand, specify and develop systems in many disciplines. , models are used to establish understanding and provide a reusable framework for product development. In the software engineering process, models are now accepted as part of a modern object oriented analysis and design approach by all of the major OO methodologies. Model based testing has given a innovative way of test case designing, in most of the software development projects, test cases are derived from RS (Requirement specification) or FS (functional specification).

Designing test cases by using model of the software helps the tester in finding and exploring different paths or scenarios for the end user, modeling technique also helps in determining the correct output at given path

Problems in Textual Description
The traditional means of specifying the correct system behavior is with English prose in the form of a Requirement Specification or Functional specification. The specification, , is often incomplete - only the typical or ideal use of the feature(s) is defined, not all of the possible actions or use scenarios. This incomplete description forces the test engineer to wait until the system is delivered so that the entire context of the feature is known. When the complete context is understood, tests can be developed that will verify all of the possible remaining scenarios. Another problem with textual descriptions is that they are ambiguous, (i.e., “if an invalid digit is entered, it shall be handled appropriately.”) The ‘appropriate’ action is never defined; rather, it is left to the reader’s interpretation. For example, you think we should allow the user to retry the entry, but I think we should abort the command. If you wrote the code and I wrote the tests, they would fail and we will waste time resolving an issue that could have been done before we even started. At a recent Requirements Engineering Conference, papers and presentations described the root cause of 60-80% of all defects as incorrect requirements specifications.

Advantages of Model Based Testing
Modeling is a very economical means of capturing knowledge about a system and then reusing this knowledge as the system grows. For a testing team, this information is gold; what percentage of a test engineer's task is spent trying to understand what the System under Test (SUT) should be doing? Once this information is understood, how is it preserved for the next engineer, the next release, or change order? If you are lucky it is in the test plan, but more typically buried in a test script or just lost, waiting to be rediscovered. By constructing a model of a system that defines the systems
Desired behavior for specified inputs to it, a team now has a mechanism for a structured analysis of the system.
Scenarios are described as a sequence of actions to the system [as it is defined in the model], with the correct responses of the system also being specified.
The largest benefit is in reuse; all of this work is not lost. The next test cycle can start where this one left off. If the product has new features, they can be incrementally added to the model;
if the quality must be improved, the model can be improved and the tests expanded; if there are new people on the team, they can quickly come up to speed by reviewing the model.

It is just a brief description; I would like to continue to post some more material about model based testing.

Articles from Larry Apfelbaum and John Doyle helped me a lot in writing this post.


Thursday, June 25, 2009

Benefits of Inbound marketing

I used to spend some time on my weekend at my buddy’s place Waqas Younas, we used to discuss a lot about “Business of software” and “”innovations in software testing”. Last week, we discussed the benefits of inbound marketing in order to grow any business. I would like to share our discussion with you guys, Waqas presented some great explanation and ideas regarding inbound marketing . It is bit different topic because I used to write more about software testing topics, but I am sure this post will open new ideas in your mind.
Mostly companies rely on outbound marketing. Outbound marketing is a phenomenon in which companies reach out to customers by cold calling them, spamming them and etc. While inbound marketing is a phenomenon in which a company creates good content and people reach out to them. A total opposite of outbound marketing. There are other benefits to Inbound marketing is well. By creating good content you improve your organic/natural rankings in Google. Also, by applying basic SEO principles you can become a good Internet citizen.
However, there is more to inbound marketing. When people are searching for something on a search engine then normally they are directly looking for some information & there is even a good chance that they are a potential customer. So when they find your quality content odds of converting people (looking for information on search engines) into customer increases by an order of magnitude. Not only that, you get a lot of incoming links to your website & that helps immensely in terms of your search engine rank. More importantly, when you write good content you're considered as an authority and this surpasses every thing.
Just for the sake of example: as per Google almost 72,000 people are searching monthly on the keyword "Offshore QA". Now let's just suppose that out of those 72,000 only 2000 land to your website. Those 2000 see good content on your website and there is a good chance that they will refer it to their friends. Even if nothing happens we can assume that out of those 2000 probably 100 people will send us queries about our product. And if we can make 1 customer out of those 100 then in my humble opinion it's a big win. Because this customer was taken on board without any third party marketing services.
Putting good contents on your website improves your traffic a lot.

Hitler as a Project Manager

This is great video, you will must enjoy it :) Enjoy

Thursday, June 18, 2009

what is Rapid software Testing

Few days back, I read an article about Rapid software testing written by James Bach. I am summarizing this article into few points here in this blog. You can read the full article from link given above.
Definition:
Rapid Software Testing, the skill of testing any software, any time, under any conditions, such that your work stands up to scrutiny. Based on the ideas of James Bach, Michael Bolton, and Cem Kaner, with substantial contributions by other members of the Context-Driven School of software testing, this is the closest thing in the business to a martial art of software testing.

Why rapid testing?
Rapid testing is a way to scale thorough testing methods to fit compressed schedule.
Rapid testing doesn't mean "not thorough", it means "as thorough as is reasonable and required, given the constraints on your time."
A good rapid tester is a skilled practitioner who can test productively under a wider variety of conditions than conventionally trained (or untrained) testers.
Most of the value of testing comes from how testers think, but even excellent testers struggle to articulate or justify their ideas about how to test. Rapid testing is a personal discipline, teachable and learnable, that allows you to think and talk about testing with confidence. By contrast, a conventionally trained tester generally is limited to complaining about how the requirements aren't fully documented.
Rapid testing is vital when you are asked to test something on short notice, off the top of your head, early in the development process, or when you're in the process of developing test cases and procedures for future use.
This approach is also useful even when you're called upon to test more thoroughly, and given the time and resources to do so.

How does rapid testing work?
A core skill in rapid testing is the ability to think critically and a good tester thinks like a scientist or a detective.
Rapid test design is an organized process, driven by a set of concise heuristics (think of them as guidelines) designed to assure that you don't forget to test anything important.
For new testers, the heuristics provide basic guidance. For experienced testers, the heuristics help you organize and access your experience, so that even under pressure, you perform like an expert and feel like one, too. With practice, you get better and better at testing rapidly while still being fully accountable for your work.

Where did the rapid testing ideas come from?
From 1987 to 1995, James Bach worked mostly alone to develop a systematic heuristic-based test methodology that applied to commercial mass-market software projects. Traditional test methodology didn't work well for market-driven test project. Starting in 1995,
He began to collaborate with other thinkers and writers in the field, who helped find and fix errors in his work, and helped extend it beyond the scope of market-driven projects.
Some of his colleagues eventually came together to identify themselves as the Context-Driven School of software test methodology.
Others helped create the Agile Alliance, which published the Agile Manifesto.
Cem Kaner, James Bach, and Brett Pettichord published the first context-driven testing textbook Lessons Learned in Software Testing. James and his colleagues speak and write regularly about testing, doing their best to advance the state of the art.

How to Validate Requirements

It is quite true that requirement engineering is quite integral part of Software development, even we do Verification and validation against our RS (Requirement specification),change request etc, there is enough possibility that we can miss understand some requirements of the client.

Studies have shown that almost 60% of errors/bugs are due to un-cleared requirements because extracting requirement is quite difficult job to do, requires great deal of experience and focus.

In agile, we work along with clients, and client is quite important stakeholder as we all know Specially in agile, so testers should be part of each communication with the client. so that can perform gap analysis along with the software activities.

According to Model based testing we developed the model of the system by analyzing requirements. Its model is just like flow chart or state diagram, then for each branch testers derive the test cases, this model grows with the passage of time because it is difficult to develop model at once. I think testers can "reverse think" by following this technique.

Testers can get an idea while executing flows that some application flow can be differ from client expectations.

I will write more materials regarding Model based testing in my coming posts.


Looking for your comment

Wednesday, June 10, 2009

How to measure Tester/QE Performance?

I read an article from Cem Kaner, in this article he talked about measuring performance of Testers. I am sharing my thoughts about this topic.

“We can not measure performance of Quality engineer/Test engineer”. We often used to encounter such statements from project managers or QA managers. So the question arises that how we can measure performance of QE.

We can consider “Bug Count” as a measure of QE performance but consider a situation where we assign two different modules, Module A and Module B ,to testers, consider following points regarding assigned modules.

· Module A is quite mature that it is gone through stable release; certainly it is thoroughly tested already.
· Developer who worked on this module is quite competent, and he covered all major scenarios, and spent a good time to unit test his module.

On the other hand, Module B has following points to consider.
· This module is pushed first time into to QA department.
· Module is developed with lots of assumptions and requirements are not cleared to the developers.
· Due to strict deadline, development team could not run unit tests.

There can be different aspects of these modules as well, but we can see obviously Module A will show lesser bug count as compared to Module B, so can we make any judgment regarding performance of testers working on these modules?, I think certainly not.
I think we can measure tester’s performance, efficiency and ability by considering following points.


1. Ability to find bug as early as possible in SDLC.

2. Ability to write effective, clear bug report.

3. Ability to convince developers and management about particular bug.

4. Working with developer without any ego (find bugs in program, not in programmer).

5. Ability to write good test plan and test strategies.

6. Ability to lead a QA team.

7. Good knowledge of different concepts like Database optimization, application security, Web application concepts and most importantly the framework on which product is based on.

8. Ability to find out of the box test scenarios.

9. Innovative and willing to improve quality of a product.

10. Good code breaking attitude.

11. Ability to write automation scripts.

We can judge QE performance by analyzing these points, this thing is quite sure that it is not possible to measure QE performance by bug count.

Looking for your comments :)

Tuesday, June 9, 2009

12 Common bugs of web application

We all know that development is quite creative and innovative work; there is always opportunity for improvement and performance optimization in web/Desktop application. Developers used to follow sometimes strict deadlines for completing their tasks, so I observed common bugs while working on different applications, which can be avoided easily by executing units tests on the behalf of development team

1.When screen is opened focus should be set on required field.

2.After displaying error message or information message, focus should be set on specific field.

3.Focus should be set on controls like Date control and Dropdown

4.Shortcut keys should be verified it should be working.

5.Field length should be fixed according to DB columns.

6.Validation should be applied on fields like Date, Password, Email etc.

7.Mandatory fields should be validated and make sure that mandatory fields are according to DB columns.

8.Error summary should be displayed properly; sometimes error summary triggers more than once in page life cycle.

9.Execute all page links; make sure that each link works fine, sometime we found that page path is not valid behind link so displays “alarming Crashing messages”.

10.Make sure that there should be no dead links on the pages.

11.If the screen is resizable, the page design (CSS+html) should be there to handle this feature. Sometimes if we resize the window, all page contents overlap each other and showing quite nasty look

12.If the window is resizable, horizontal and vertical scroll bar should be properly used.

By adding these points in your unit test check list, Development lead can reduce the bug count effectively.

Monday, June 8, 2009

Why companies need CMM

Developing reliable and usable software that is delivered on time and within budget is a difficult endeavor for many organizations. Products that are late, over budget, or that don't work as expected also cause problems for the organization's customers. As software projects continue to increase in size and importance, these problems become magnified. These problems can be overcome through a focused and sustained effort at building a process infrastructure of effective software engineering and management practices.

The Capability Maturity Model for Software (CMM) is a framework that describes the key elements of an effective software process. The CMM describes an evolutionary improvement path from an ad hoc, immature process to a mature, disciplined process.
The CMM covers practices for planning, engineering, and managing software development and maintenance. When followed, these key practices improve the ability of organizations to meet goals for cost,schedule, functionality, and product quality.
The CMM is composed of five maturity levels.

With the exception of Level 1, each maturity level is composed of several key process areas. Each key process area is organized into five sections called common features. The common features specify the key practices that, when collectively addressed,accomplish the goals of the key process area.

Saturday, June 6, 2009

My First Blog

I used to write blog in my school time, but i couldnt continue this activity due to some reason :). but i still have that diary in my book shelf, that reminds me my old days , when i used to play a a lot sports like hockey, cricket ,table tennis for my school, certainly "those were the best days of my life".

Now i am IT profeessional, and have been in this field for 3 years after completing my bachelors in computer science. now a days i am thinking to establisg a offshore testing company. so brainstorming is contine and at its peak now a days :)