Testing Methodologies

Acceptance Testing
Testing the system with the intent of confirming readiness of the product and customer acceptance. Acceptance testing, which is a black box testing, will give the client the opportunity to verify the system functionality and usability prior to the system being moved to production. The acceptance test will be the responsibility of the client; however, it will be conducted with full support from the project team. The Test Team will work with the client to develop the acceptance criteria.

Ad Hoc Testing
Testing without a formal test plan or outside of a test plan. With some projects this type of testing is carried out as an adjunct to formal testing. If carried out by a skilled tester, it can often find problems that are not caught in regular testing. Sometimes, if testing occurs very late in the development cycle, this will be the only kind of testing that can be performed. Sometimes ad hoc testing is referred to as exploratory testing.

Alpha Testing
Testing after code is mostly complete or contains most of the functionality and prior to users being involved. Sometimes a select group of users are involved. More often this testing will be performed in-house or by an outside testing firm in close cooperation with the software engineering department.

Automated Testing
Software testing that utilizes a variety of tools to automate the testing process and when the importance of having a person manually testing is diminished. Automated testing still requires a skilled quality assurance professional with knowledge of the automation tool and the software being tested to set up the tests.

Beta Testing
Testing after the product is code complete. Betas are often widely distributed or even distributed to the public at large in hopes that they will buy the final product when it is released.

Black Box Testing
Testing software without any knowledge of the inner workings, structure or language of the module being tested. Black box tests, as most other kinds of tests, must be written from a definitive source document, such as a specification or requirements document.

Compatibility Testing
Testing used to determine whether other system software components such as browsers, utilities, and competing software will conflict with the software being tested.

Configuration Testing
Testing to determine how well the product works with a broad range of hardware/peripheral equipment configurations as well as on different operating systems and software.

End-to-End Testing
Similar to system testing, the 'macro' end of the test scale involves testing of a complete application environment in a situation that mimics real-world use, such as interacting with a database, using network communications, or interacting with other hardware, applications, or systems if appropriate.

Functional Testing
Testing two or more modules together with the intent of finding defects, demonstrating that defects are not present, verifying that the module performs its intended functions as stated in the specification and establishing confidence that a program does what it is supposed to do.

Independent Verification and Validation (IV&V)
The process of exercising software with the intent of ensuring that the software system meets its requirements and user expectations and doesn't fail in an unacceptable manner. The individual or group doing this work is not part of the group or organization that developed the software. A term often applied to government work or where the government regulates the products, as in medical devices.

Installation Testing
Testing with the intent of determining if the product will install on a variety of platforms and how easily it installs. Testing full, partial, or upgrade install/uninstall processes. The installation test for a release will be conducted with the objective of demonstrating production readiness. This test is conducted after the application has been migrated to the client's site. It will encompass the inventory of configuration items (performed by the application's System Administration) and evaluation of data readiness, as well as dynamic tests focused on basic system functionality. When necessary, a sanity test will be performed following the installation testing.

Integration Testing
Testing two or more modules or functions together with the intent of finding interface defects between the modules or functions. Testing completed at as a part of unit or functional testing, and sometimes, becomes its own standalone test phase. On a larger level, integration testing can involve a putting together of groups of modules and functions with the goal of completing and verifying that the system meets the system requirements. (see system testing)

Load Testing
Testing with the intent of determining how well the product handles competition for system resources. The competition may come in the form of network traffic, CPU utilization or memory allocation.

Parallel/Audit Testing
Testing where the user reconciles the output of the new system to the output of the current system to verify the new system performs the operations correctly.

Performance Testing
Testing with the intent of determining how quickly a product handles a variety of events. Automated test tools geared specifically to test and fine-tune performance are used most often for this type of testing.

Pilot Testing
Testing that involves the users just before actual release to ensure that users become familiar with the release contents and ultimately accept it. Often is considered a Move-to-Production activity for ERP releases or a beta test for commercial products. Typically involves many users, is conducted over a short period of time and is tightly controlled. (see beta testing)

Recovery/Error Testing
Testing how well a system recovers from crashes, hardware failures, or other catastrophic problems.

Regression Testing
Testing with the intent of determining if bug fixes have been successful and have not created any new problems. Also, this type of testing is done to ensure that no degradation of baseline functionality has occurred.

Sanity Testing
Sanity testing will be performed whenever cursory testing is sufficient to prove the application is functioning according to specifications. This level of testing is a subset of regression testing. It will normally include a set of core tests of basic GUI functionality to demonstrate connectivity to the database, application servers, printers, etc.

Security Testing
Testing of database and network software in order to keep company data and resources secure from mistaken/accidental users, hackers, and other malevolent attackers.

Software Testing
The process of exercising software with the intent of ensuring that the software system meets its requirements and user expectations and doesn't fail in an unacceptable manner. The organization and management of individuals or groups doing this work is not relevant. This term is often applied to commercial products such as internet applications. (contrast with independent verification and validation)

Stress Testing
Testing with the intent of determining how well a product performs when a load is placed on the system resources that nears and then exceeds capacity.

System Integration Testing
Testing a specific hardware/software installation. This is typically performed on a COTS (commercial off the shelf) system or any other system comprised of disparent parts where custom configurations and/or unique installations are the norm.

Unit Testing
Unit Testing is the first level of dynamic testing and is first the responsibility of the developers and then of the testers. Unit testing is performed after the expected test results are met or differences are explainable / acceptable.

Usability Testing
Testing for 'user-friendliness'. Clearly this is subjective and will depend on the targeted end-user or customer. User interviews, surveys, video recording of user sessions, and other techniques can be used. Programmers and testers are usually not appropriate as usability testers.

White Box Testing
Testing in which the software tester has knowledge of the inner workings, structure and language of the software, or at least its purpose.

What is Testing?

Testing is the process of identifying defects, where a defect is any variance between actual and expected results.

Defect can be caused by a flaw in the application software or by a flaw in the application specification. For example, unexpected (incorrect) results can be from errors made during the construction phase, or from an algorithm incorrectly defined in the specification. Testing is commonly assumed to mean executing software and finding errors. This type of testing is known as dynamic testing, and while valid, it is not the most effective way of testing. Static testing, the review, inspection and validation of development requirements, is the most effective and cost efficient way of testing. A structured approach to testing should use both dynamic and static testing techniques.

What is SEI? CMM? CMMI? ISO? IEEE? ANSI?

SEI = 'Software Engineering Institute' at Carnegie-Mellon University; initiated by the U.S. Defense Department to help improve software development processes.

CMM = 'Capability Maturity Model', now called the CMMI ('Capability Maturity Model Integration'), developed by the SEI. It's a model of 5 levels of process 'maturity' that determine effectiveness in delivering quality software. It is geared to large organizations such as large U.S. Defense Department contractors. However, many of the QA processes involved are appropriate to any organization, and if reasonably applied can be helpful. Organizations can receive CMMI ratings by undergoing assessments by qualified auditors.

Level 1
Characterized by chaos, periodic panics, and heroic efforts required by individuals to successfully complete projects. Few if any processes in place; successes may not be repeatable.

Level 2
Software project tracking, requirements management, realistic planning, and configuration management
processes are in place; successful practices can be repeated.

Level 3
Standard software development and maintenance processes are integrated throughout an organization; a Software Engineering Process Group is is in place to oversee software processes, and training programs are used to ensure understanding and compliance.

Level 4
Metrics are used to track productivity, processes, and products. Project performance is predictable, and quality is consistently high.

Level 5
The focus is on continouous process improvement. The impact of new processes and technologies can be
predicted and effectively implemented when required.

Perspective on CMM ratings: During 1997-2001, 1018 organizations were assessed. Of those, 27% were rated at Level 1, 39% at 2, 23% at 3, 6% at 4, and 5% at 5. (For ratings during the period 1992-96, 62% were at Level 1, 23% at 2, 13% at 3, 2% at 4, and 0.4% at 5.)

The median size of organizations was 100 software engineering/maintenance personnel; 32% of organizations were U.S. federal contractors or agencies. For those rated at Level 1, the most problematical key process area was in Software Quality Assurance.

ISO
'International Organisation for Standardization' - The ISO 9001:2000 standard (which replaces the previous standard of 1994) concerns quality systems that are assessed by outside auditors, and it applies to many kinds of production and manufacturing organizations, not just software.
It covers documentation, design, development, production, testing, installation, servicing, and other processes.
The full set of standards consists of:
(a)Q9001-2000 - Quality Management Systems: Requirements;
(b)Q9000-2000 - Quality Management Systems: Fundamentals and Vocabulary;
(c)Q9004-2000 - Quality Management Systems: Guidelines for Performance Improvements.

To be ISO 9001 certified, a third-party auditor assesses an organization, and certification is typically good for about 3 years, after which a complete reassessment is required. Note that ISO certification does not necessarily indicate quality products - it indicates only that documented processes are followed. Also see http://www.iso.ch/ for the latest information.

IEEE
'Institute of Electrical and Electronics Engineers' - among other things, creates standards such as 'IEEE Standard for Software Test Documentation' (IEEE/ANSI Standard 829), 'IEEE Standard of Software Unit Testing (IEEE/ANSI Standard 1008), 'IEEE Standard for Software Quality Assurance Plans' (IEEE/ANSI Standard 730), and others.

ANSI
'American National Standards Institute', the primary industrial standards body in the U.S.; publishes some software-related standards in conjunction with the IEEE and ASQ (American Society for Quality).
Other software development/IT management process assessment methods besides CMMI and ISO 9000 include SPICE, Trillium, TickIT, Bootstrap, ITIL, MOF, and CobiT.

Advantages in a process driven organization

Imagine a situation that you have to start working in a project from the middle, in spite of being new to this project, you are bound to deliver the best anytime and every time andthat is expected this time too. There is no documentation available in that project, there are no processes how to carry out the tasks and there is noting in place. Your PM is too busy to devote full time to you to solve your difficulties...... How many times you have been in such a situation? Not many times, then you are lucky. But what if you get in such a situation? How to handle this????

To overcome this trap and to save you from this ambush there is a very systematic and robust way-out. Documenting everything in the project and defining Processes for carrying out all the tasks in a Project. If the processes are set then documentation automatically follows it.

Defining processes helps us in more than one way. A few advantages in a process driven organization are:

1. Processes set a disciplined way of doing the tasks that leads to Standard and Consistent output every time.
2. In processes, everything is clearly documented in step by step manner, so anybody can start doing the tasks just by reading the process, irrespective of the stage of the person's involvement in the project.
3. In the Process there is a greater visibility in the activities done for accomplishing the tasks, so a greater control over everything can be achieved.
4. Roles and responsibilities are mentioned explicitly in a process, so there is no chance of any ignorance or negligence or assumptions.
5. Processes make the organizational activities person independent.
6. Process Improvement is also possible over a period of time that ensures greater efficiency in executing the projects.
7. With formalized processes in place for all the activities, organizations can estimate better and can achieve smoother execution of projects.

After seeing some of the benefits like this, would you not go ahead for a disciplined process structure for your project?

Difference between QA and QC

What is Software Quality?

Though there are a number of definitions propounded by the gurus in the field - each having its own adherents, two definitions that are widely accepted and that are complementary to each other are:

- Conformance to explicitly stated functional and performance requirements, explicitly documented development standards, and implicit characteristics that are expected of all professionally developed software.

- The degree to which a system, component, or process meets specified requirements and customer or user needs or expectations.

Quality assurance and quality control both contribute in delivering a high quality software product though the way they go about it is different. This can be illustrated by looking at the definitions of the two.

What is Software Quality Assurance?

Software QA involves the entire software development PROCESS - monitoring and improving the process, making sure that any agreed-upon standards and procedures are followed, and ensuring that problems are found and dealt with. It is oriented to ’prevention’.

This is a ’staff’ function, and is responsible for establishing standards and procedures to prevent defects and breakdowns in the SDLC. The focus of QA is prevention, processes, and continuous improvement of these processes.

What is Software Quality Control?

This is a department function, which compares the standards to the product, and takes action when non-conformance is detected for example testing.

This involves operation of a system or application under controlled conditions and evaluating the results (e.g., ’if the user is in interface A of the application while using hardware B, and does C, then D should happen’). The controlled conditions should include both normal and abnormal conditions. Testing should intentionally attempt to make things go wrong to determine if things happen when they shouldn’t or things don’t happen when they should. It is oriented to ’detection’.

Relationship between QC and QA

An application that meets its requirements totally can be said to exhibit quality. Quality is not based on a subjective assessment but rather on a clearly demonstrable, and measurable, basis. Quality.

- Quality Control is a process directed at validating that a specific deliverable meets standards, is error free, and is the best deliverable that can be produced. It is a responsibility internal to the team.

- QA, on the other hand, is a review with a goal of improving the process as well as the deliverable. QA is often an external process. QA is an effective approach to producing a high quality product.

One aspect is the process of objectively reviewing project deliverables and the processes that produce them (including testing), to identify defects, and then making recommendations for improvement based on the reviews. The end result is the assurance that the system and application is of high quality, and that the process is working. The achievement of quality goals is well within reach when organizational strategies are used in the testing process. From the client's perspective, an application's quality is high if it meets their expectations.

Software Testing Glossary

Acceptance Testing
Testing conducted to enable a user/customer to determine whether to accept a software product. Normally performed to validate the software meets a set of agreed acceptance criteria.

Accessibility Testing
Verifying a product is accessible to the people having disabilities (deaf, blind, mentally disabled etc.).

Ad Hoc Testing
A testing phase where the tester tries to 'break' the system by randomly trying the system's functionality. Can include negative testing as well. See also Monkey Testing.

Agile Testing
Testing practice for projects using agile methodologies, treating development as the customer of testing and emphasizing a test-first design paradigm. See also Test Driven Development.

Application Binary Interface (ABI)
A specification defining requirements for portability of applications in binary forms across defferent system platforms and environments.

Application Programming Interface (API)
A formalized set of software calls and routines that can be referenced by an application program in order to access supporting system or network services.

Automated Software Quality (ASQ)
The use of software tools, such as automated testing tools, to improve software quality.

Automated Testing:
· Testing employing software tools which execute tests without manual intervention. Can be applied in GUI, performance, API, etc. testing.
· The use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions.

Backus-Naur Form
A metalanguage used to formally describe the syntax of a language.

Basic Block
A sequence of one or more consecutive, executable statements containing no branches.

Basis Path Testing
A white box test case design technique that uses the algorithmic flow of the program to design tests.

Basis Set
The set of tests derived using basis path testing.

Baseline
The point at which some deliverable produced during the software engineering process is put under formal change control.

Beta Testing
Testing of a re-release of a software product conducted by customers.

Binary Portability Testing
Testing an executable application for portability across system platforms and environments, usually for conformation to an ABI specification.

Black Box Testing
Testing based on an analysis of the specification of a piece of software without reference to its internal workings. The goal is to test how well the component conforms to the published requirements for the component.

Bottom Up Testing
An approach to integration testing where the lowest level components are tested first, then used to facilitate the testing of higher level components. The process is repeated until the component at the top of the hierarchy is tested.

Boundary Testing
Test which focus on the boundary or limit conditions of the software being tested. (Some of these tests are stress tests).

Bug
A fault in a program which causes the program to perform in an unintended or unanticipated manner.

Boundary Value Analysis
BVA is similar to Equivalence Partitioning but focuses on "corner cases" or values that are usually out of range as defined by the specification. his means that if a function expects all values in range of negative 100 to positive 1000, test inputs would include negative 101 and positive 1001.

Branch Testing
Testing in which all branches in the program source code are tested at least once.

Breadth Testing
A test suite that exercises the full functionality of a product but does not test features in detail.

CAST
Computer Aided Software Testing.

Capture/Replay Tool
A test tool that records test input as it is sent to the software under test. The input cases stored can then be used to reproduce the test at a later time. Most commonly applied to GUI test tools.

CMM
The Capability Maturity Model for Software (CMM or SW-CMM) is a model for judging the maturity of the software processes of an organization and for identifying the key practices that are required to increase the maturity of these processes.

Cause Effect Graph
A graphical representation of inputs and the associated outputs effects which can be used to design test cases.

Code Complete
Phase of development where functionality is implemented in entirety; bug fixes are all that are left. All functions found in the Functional Specifications have been implemented.

Code Coverage
An analysis method that determines which parts of the software have been executed (covered) by the test case suite and which parts have not been executed and therefore may require additional attention.

Code Inspection
A formal testing technique where the programmer reviews source code with a group who ask questions analyzing the program logic, analyzing the code with respect to a checklist of historically common programming errors, and analyzing its compliance with coding standards.

Code Walkthrough
A formal testing technique where source code is traced by a group with a small set of test cases, while the state of program variables is manually monitored, to analyze the programmer's logic and assumptions.

Coding
The generation of source code.

Compatibility Testing
Testing whether software is compatible with other elements of a system with which it should operate, e.g. browsers, Operating Systems, or hardware.

Component
A minimal software item for which a separate specification is available.

Component Testing
See Unit Testing.

Concurrency Testing
Multi-user testing geared towards determining the effects of accessing the same application code, module or database records. Identifies and measures the level of locking, deadlocking and use of single-threaded code and locking semaphores.

Conformance Testing
The process of testing that an implementation conforms to the specification on which it is based. Usually applied to testing conformance to a formal standard.

Context Driven Testing
The context-driven school of software testing is flavor of Agile Testing that advocates continuous and creative evaluation of testing opportunities in light of the potential information revealed and the value of that information to the organization right now.

Conversion Testing
Testing of programs or procedures used to convert data from existing systems for use in replacement systems.

Cyclomatic Complexity
A measure of the logical complexity of an algorithm, used in white-box testing.

Data Dictionary
A database that contains definitions of all data items defined during analysis.

Data Flow Diagram
A modeling notation that represents a functional decomposition of a system.

Data Driven Testing
Testing in which the action of a test case is parameterized by externally defined data values, maintained as a file or spreadsheet. A common technique in Automated Testing.

Debugging
The process of finding and removing the causes of software failures.

Defect
Nonconformance to requirements or functional / program specification

Dependency Testing
Examines an application's requirements for pre-existing software, initial states and configuration in order to maintain proper functionality.

Depth Testing
A test that exercises a feature of a product in full detail.

Dynamic Testing
Testing software through executing it. See also Static Testing.

Emulator
A device, computer program, or system that accepts the same inputs and produces the same outputs as a given system.

Endurance Testing
Checks for memory leaks or other problems that may occur with prolonged execution.

End-to-End testing
Testing a complete application environment in a situation that mimics real-world use, such as interacting with a database, using network communications, or interacting with other hardware, applications, or systems if appropriate.

Equivalence Class
A portion of a component's input or output domains for which the component's behavior is assumed to be the same from the component's specification.

Equivalence Partitioning
A test case design technique for a component in which test cases are designed to execute representatives from equivalence classes.

Exhaustive Testing
Testing which covers all combinations of input values and preconditions for an element of the software under test.

Functional Decomposition
A technique used during planning, analysis and design; creates a functional hierarchy for the software.

Functional Specification
A document that describes in detail the characteristics of the product with regard to its intended features.

Functional Testing
See also Black Box Testing.
· Testing the features and operational behavior of a product to ensure they correspond to its specifications.
· Testing that ignores the internal mechanism of a system or component and focuses solely on the outputs generated in response to selected inputs and execution conditions.

Glass Box Testing
A synonym for White Box Testing.

Gorilla Testing
Testing one particular module,functionality heavily.

Gray Box Testing
A combination of Black Box and White Box testing methodologies: testing a piece of software against its specification but using some knowledge of its internal workings.

High Order Tests
Black-box tests conducted once the software has been integrated.

Independent Test Group (ITG)
A group of people whose primary responsibility is software testing,

Inspection
A group review quality improvement process for written material. It consists of two aspects; product (document itself) improvement and process improvement (of both document production and inspection).

Integration Testing
Testing of combined parts of an application to determine if they function together correctly. Usually performed after unit and functional testing. This type of testing is especially relevant to client/server and distributed systems.

Installation Testing
Confirms that the application under test recovers from expected or unexpected events without loss of data or functionality. Events can include shortage of disk space, unexpected loss of communication, or power out conditions.

Load Testing
See Performance Testing.

Localization Testing
This term refers to making software specifically designed for a specific locality.

Loop Testing
A white box testing technique that exercises program loops.

Metric
A standard of measurement. Software metrics are the statistics describing the structure or content of a program. A metric should be a real objective measurement of something such as number of bugs per lines of code.

Monkey Testing
Testing a system or an Application on the fly, i.e just few tests here and there to ensure the system or an application does not crash out.

Negative Testing
Testing aimed at showing software does not work. Also known as "test to fail". See also Positive Testing.

Path Testing
Testing in which all paths in the program source code are tested at least once.

Performance Testing
Testing conducted to evaluate the compliance of a system or component with specified performance requirements. Often this is performed using an automated test tool to simulate large number of users. Also know as "Load Testing".

Positive Testing
Testing aimed at showing software works. Also known as "test to pass". See also Negative Testing.

Quality Assurance
All those planned or systematic actions necessary to provide adequate confidence that a product or service is of the type and quality needed and expected by the customer.

Quality Audit
A systematic and independent examination to determine whether quality activities and related results comply with planned arrangements and whether these arrangements are implemented effectively and are suitable to achieve objectives.

Quality Circle
A group of individuals with related interests that meet at regular intervals to consider problems or other matters related to the quality of outputs of a process and to the correction of problems or to the improvement of quality.

Quality Control
The operational techniques and the activities used to fulfill and verify requirements of quality.

Quality Management
That aspect of the overall management function that determines and implements the quality policy.

Quality Policy
The overall intentions and direction of an organization as regards quality as formally expressed by top management.

Quality System
The organizational structure, responsibilities, procedures, processes, and resources for implementing quality management.

Race Condition
A cause of concurrency problems. Multiple accesses to a shared resource, at least one of which is a write, with no mechanism used by either to moderate simultaneous access.

Ramp Testing
Continuously raising an input signal until the system breaks down.

Recovery Testing
Confirms that the program recovers from expected or unexpected events without loss of data or functionality. Events can include shortage of disk space, unexpected loss of communication, or power out conditions.

Regression Testing
Retesting a previously tested program following modification to ensure that faults have not been introduced or uncovered as a result of the changes made.

Release Candidate
A pre-release version, which contains the desired functionality of the final version, but which needs to be tested for bugs (which ideally should be removed before the final version is released).

Sanity Testing
Brief test of major functional elements of a piece of software to determine if its basically operational. See also Smoke Testing.

Scalability Testing
Performance testing focused on ensuring the application under test gracefully handles increases in work load.

Security Testing
Testing which confirms that the program can restrict access to authorized personnel and that the authorized personnel can access the functions available to their security level.

Smoke Testing
A quick-and-dirty test that the major functions of a piece of software work. Originated in the hardware testing practice of turning on a new piece of hardware for the first time and considering it a success if it does not catch on fire.

Soak Testing
Running a system at high load for a prolonged period of time. For example, running several times more transactions in an entire day (or night) than would be expected in a busy day, to identify and performance problems that appear after a large number of transactions have been executed.

Software Requirements Specification
A deliverable that describes all data, functional and behavioral requirements, all constraints, and all validation requirements for software/

Software Testing
A set of activities conducted with the intent of finding errors in software.

Static Analysis
Analysis of a program carried out without executing the program.

Static Analyzer
A tool that carries out static analysis.

Static Testing
Analysis of a program carried out without executing the program.

Storage Testing
Testing that verifies the program under test stores data files in the correct directories and that it reserves sufficient space to prevent unexpected termination resulting from lack of space. This is external storage as opposed to internal storage.

Stress Testing
Testing conducted to evaluate a system or component at or beyond the limits of its specified requirements to determine the load under which it fails and how. Often this is performance testing using a very high level of simulated load.

Structural Testing
Testing based on an analysis of internal workings and structure of a piece of software. See also White Box Testing.

System Testing
Testing that attempts to discover defects that are properties of the entire system rather than of its individual components.

Testability
The degree to which a system or component facilitates the establishment of test criteria and the performance of tests to determine whether those criteria have been met.

Testing
· The process of exercising software to verify that it satisfies specified requirements and to detect errors.
· The process of analyzing a software item to detect the differences between existing and required conditions (that is, bugs), and to evaluate the features of the software item (Ref. IEEE Std 829).
· The process of operating a system or component under specified conditions, observing or recording the results, and making an evaluation of some aspect of the system or component.

Test Automation
See Automated Testing.

Test Bed
An execution environment configured for testing. May consist of specific hardware, OS, network topology, configuration of the product under test, other application or system software, etc. The Test Plan for a project should enumerated the test beds(s) to be used.

Test Case
· Test Case is a commonly used term for a specific test. This is usually the smallest unit of testing. A Test Case will consist of information such as requirements testing, test steps, verification steps, prerequisites, outputs, test environment, etc.
· A set of inputs, execution preconditions, and expected outcomes developed for a particular objective, such as to exercise a particular program path or to verify compliance with a specific requirement.

Test Driven Development
Testing methodology associated with Agile Programming in which every chunk of code is covered by unit tests, which must all pass all the time, in an effort to eliminate unit-level and regression bugs during development. Practitioners of TDD write a lot of tests, i.e. an equal number of lines of test code to the size of the production code.

Test Driver
A program or test tool used to execute a tests. Also known as a Test Harness.

Test Environment
The hardware and software environment in which tests will be run, and any other software with which the software under test interacts when under test including stubs and test drivers.

Test First Design
Test-first design is one of the mandatory practices of Extreme Programming (XP).It requires that programmers do not write any production code until they have first written a unit test.

Test Harness
A program or test tool used to execute a tests. Also known as a Test Driver.

Test Plan
A document describing the scope, approach, resources, and schedule of intended testing activities. It identifies test items, the features to be tested, the testing tasks, who will do each task, and any risks requiring contingency planning. Ref IEEE Std 829.

Test Procedure
A document providing detailed instructions for the execution of one or more test cases.

Test Script
Commonly used to refer to the instructions for a particular test that will be carried out by an automated test tool.

Test Specification
A document specifying the test approach for a software feature or combination or features and the inputs, predicted results and execution conditions for the associated tests.

Test Suite
A collection of tests used to validate the behavior of a product. The scope of a Test Suite varies from organization to organization. There may be several Test Suites for a particular product for example. In most cases however a Test Suite is a high level concept, grouping together hundreds or thousands of tests related by what they are intended to test.

Test Tools
Computer programs used in the testing of a system, a component of the system, or its documentation.

Thread Testing
A variation of top-down testing where the progressive integration of components follows the implementation of subsets of the requirements, as opposed to the integration of components by successively lower levels.

Top Down Testing
An approach to integration testing where the component at the top of the component hierarchy is tested first, with lower level components being simulated by stubs. Tested components are then used to test lower level components. The process is repeated until the lowest level components have been tested.

Total Quality Management
A company commitment to develop a process that achieves high quality product and customer satisfaction.

Traceability Matrix
A document showing the relationship between Test Requirements and Test Cases.

Usability Testing
Testing the ease with which users can learn and use a product.

Use Case
The specification of tests that are conducted from the end-user perspective. Use cases tend to focus on operating software as an end-user would conduct their day-to-day activities.

Unit Testing
Testing of individual software components.

Validation
The process of evaluating software at the end of the software development process to ensure compliance with software requirements. The techniques for validation is testing, inspection and reviewing.

Verification
The process of determining whether of not the products of a given phase of the software development cycle meet the implementation steps and can be traced to the incoming objectives established during the previous phase. The techniques for verification are testing, inspection and reviewing.

Volume Testing
Testing which confirms that any values that may become large over time (such as accumulated counts, logs, and data files), can be accommodated by the program and will not cause the program to stop working or degrade its operation in any manner.

Walkthrough
A review of requirements, designs or code characterized by the author of the material under review guiding the progression of the review.

White Box Testing
Testing based on an analysis of internal workings and structure of a piece of software. Includes techniques such as Branch Testing and Path Testing. Also known as Structural Testing and Glass Box Testing. Contrast with Black Box Testing.

Workflow Testing
Scripted end-to-end testing which duplicates specific workflows which are expected to be utilized by the end-user.

Software Testing books - Amazon.com

The best thing about "Testing Computer Software" is its practical point-by-point guide to everyday software testing, from creating a test plan, to writing effective bug reports, to working with programming staff and management to fix bugs.

Topics covered: test case design, test planning, project lifecycle overview, software errors, boundary conditions, bug reports, regression testing, black box testing, software quality and reliability, managing test teams, printer testing, internationalization, and managing legal risk.

-----------------------------------------------------------



While this book does cover some rules of thumb for testing techniques, it goes further. It provides insight into the human element of testing and gives practical advice on how to communicate defects, trust in the doers to make estimates, understand the importance of a diverse culture and other areas that have a significant impact on the development process. This is all done well within the context of the impact to software testing.

-----------------------------------------------------------




For a practical guide to software testing, readers can look to Rex Black's "Managing the Testing Process", a compendium of real-world advice on managing software testing successfully.
Early sections examine the design of test plans, along with strategies for assessing and prioritizing risk as well as catching bugs through effective testing. Sample case studies include a network hardware device and a Java word processor.

Throughout this book, a variety of documents (including Excel spreadsheets and Access databases) are presented to get you started on your own testing projects. (all sample documents are included on the accompanying CD-ROM.) The book also looks at metrics for measuring the performance of your testing operation.

"Managing the Testing Process" shows how a bug-tracking database is the most effective model for managing the testing cycle. This book is chock-full of advice on testing management. The author also presents dozens of tips for succeeding in the software Q/A job market.

-----------------------------------------------------------




"Software Testing and Continuous Quality Improvement" illustrates a quality framework for software testing in traditional structured and unstructured environments. Section I reviews modern QA principles and best practices. Section II examines the waterfall development methodology. The next section contrasts waterfall development methodology with the rapid application spiral environment. Section IV discusses fundamental challenges of maintaining and improving existing systems. Section V contains a history of software testing, previews future testing tools, and guides the choosing of proper tools for various environments. It provides examples of some of the most popular products, and offers a detailed methodology for evaluating them.

-----------------------------------------------------------




This book shows the latest technologies and expert tools, tips, and techniques for testing Web applications that run across many platforms and networks, including desktop and wired/wireless mobile clients. You'll quickly be able to maximize productivity, minimize quality risks, and optimize success in the Internet economy.
With numerous real-world examples at hand, you'll learn how to:
- Analyze the complex Web-application model and gain a better understanding of the effective practices for deploying a successful testing strategy
- Design smart test cases, utilize best test practices, and perform effective bug analysis
- Identify similarities and differences between testing desktop and mobile Web applications, and devise test plans for each platform
- Apply advanced and specialized testing techniques including security testing, performance testing, database testing, and server-side testing.

-----------------------------------------------------------




"How to Break Software" is a departure from conventional testing in which testers prepare a written test plan and then use it as a script when testing the software. The testing techniques in this book are as flexible as conventional testing is rigid. And flexibility is needed in software projects in which requirements can change, bugs can become features and schedule pressures often force plans to be reassessed. Software testing is not such an exact science that one can determine what to test in advance and then execute the plan and be done with it. Instead of a plan, intelligence, insight, experience and a "nose for where the bugs are hiding" should guide testers. This book helps testers develop this insight.

The techniques presented in this book not only allow testers to go off-script, they encourage them to do so. Don't blindly follow a document that may be out of date and that was written before the product was even testable. Instead, use your head! Open your eyes! Think a little, test a little and then think a little more. This book does teach planning, but in an "on- the-fly while you are testing" way. It also encourages automation with many repetitive and complex tasks that require good tools (one such tool is shipped with this book on the companion CD). However, tools are never used as a replacement for intelligence. Testers do the thinking and use tools to collect data and help them explore applications more efficiently and effectively.

-----------------------------------------------------------




If you're a software tester or in the software quality assurance field, especially if you're interested in security, you need to read this book as it will likely be an eye-opener. It's not full of shocking anecdotes to scare developers into writing better software, it's a handbook of what to look for when testing software after you think you've done all your testing, and at the same time gives developers and project managers good information on how to design, code, and state requirements better.

If you're a security person, especially the burgeoning field of application security, you might also find this book pretty enlightining. Everyone's heard of penetration testing and vulnerability assessment, but typically only in the context of attacking remotely over a network. This book shows you how to attack the software on your local machine.

-----------------------------------------------------------



In this book, you'll find the latest methodologies for the design of effective test cases, including information on psychological and economic principles, managerial aspects, test tools, high-order testing, code inspections, and debugging. Accessible, comprehensive, and always practical, this edition provides the key information you need to test successfully, whether a novice or a working programmer.

-----------------------------------------------------------




This book is a distillation of a broad array of concepts, from requirements analysis to automated testing tools to evaluating how effective a tester is. This wide scope gives the book a broad appeal and it serves to make it clear that there is quite a bit going on in the world of testing.
This is an excellent reference book for the average test group within an organization and is certainly worth having in your library if you in any way deal with the world of quality assurance or quality testing.

-----------------------------------------------------------




This text describes MITs (Most Important Tests), a risk based test methodology. The MITs are determined using various methods, including path analysis, boundary value analysis, expert interviews, and test ranking. MITs makes use of prioritized test cases, which collectively are referred to as a test inventory. The book is organized in 3 sections. Chapters 1 through 5 focus on background concepts. Chapters 6 through 8 focus on the test inventory and how to create it. Chapters 9 through 14 discuss risk analysis, test techniques , and test planning and estimation.
In addition to being aligned to product line development, this book's approach can also be easily tailored to rapid, iterative development approaches such as agile methods. If you are working in an internal development environment that uses 'heavier' development lifecycles this book is not going to fit; however, if you work in a product-oriented environment this book will not only change your thinking, but will provide the basis for an integrated development-marketing approach that could make a real difference in competitive advantage.

-----------------------------------------------------------




"Automated Software Testing" presents a methodology for test managers called Automated Testing Lifecycle Management (ATLM). This isncludes initial planning, budgeting, staffing, building a test plan and choosing test tools to executing tests and even improving your testing process the next time around. Though somewhat thickly written--with plenty of software engineering terminology--this book can also be useful to more practically minded readers because of its many sample test documents. (Besides numerous lists and charts outlining the steps in the ATLM process, the book presents a sample test plan, budget estimates, and staffing guides.)
A truly standout feature is the book's survey of currently available automated tools that can be used throughout the testing cycle, as well as how to choose the right ones for your next project.

-----------------------------------------------------------




This book provides the latest in standards for measuring how good your organization's commitment to software testing is and many ways to improve it. In all, with its numerous lists and practical step-by-step guide to testing, this book points the way toward more economical and effective software testing.

This book's major strength is its meticulous 11-step guide to all aspects of today's software testing process--from initial analysis and test planning to testing software installation and looking at ways to improve the testing cycle the next time around. The book is filled with to-do lists that enumerate the resources and tasks required for each step with helpful hints for what to do, how to work with management, and how to staff and execute a test plan from start to finish. (There is a chapter devoted to each of the 11 steps.)

-----------------------------------------------------------