TMMi-P_Syll2020 TMMi - Test Maturity Model Integration Professional Questions and Answers

In an Agile context, theSCRUM teammodel is a typical format for a test organisation. Testing is integrated into the cross-functional SCRUM team, with testers working alongside developers and product owners throughout the sprint. This ensures that testing is a continuous activity, rather than a separate phase. The test organisation in Agile environments focuses on collaboration and team-based activities, rather than a separate or independent test group​​.

The specific practice "Define a coordinated approach that coordinates both static and dynamic testing" is part of theAdvanced Reviewsprocess area at TMMi Level 4. The goal of this process area is to integrate static and dynamic testing, particularly through peer reviews, which helps optimise both defect detection and product quality management throughout the lifecycle​​​.

This statement is not true regarding a staged model. In astaged model, such as TMMi, organisations must follow a pre-defined path through maturity levels, and the process areas are organised in a fixed order that must be achieved sequentially. Each maturity level forms the foundation for the next, meaning that an organisation cannot achieve process areas at different capability levels independently, as is possible in acontinuousmodel like CMMI. The staged model is designed to ensure that each process area is fully implemented and institutionalised before progressing to the next maturity level.

TheDebugging-oriented phasefrom Gelperin and Hetzel's evolutionary testing model is associated withMaturity Level 1 "Initial"of TMMi. At this maturity level, testing is not yet a distinct and well-defined process but is often interleaved with debugging. Organizations at this level have chaotic and ad-hoc testing activities, with no separation between testing and debugging.

Maturity Level 1organizations lack formal processes, and testing is often reactive, performed alongside fixing bugs in the code. This is in line with the debugging-oriented phase, where testing and debugging are seen as part of the same activity.

TMMi References:

TheTMMi frameworkdraws directly from the evolutionary model of Gelperin and Hetzel, associating theDebugging-oriented phasewith Level 1, which describes a chaotic, unmanaged testing environment​​.

In theDefect Preventionprocess area at TMMi Level 5, one of the key specific goals (SG2) is to prioritise and define actions to systematically eliminate the root causes of defects. This involves proposing solutions to address common defect causes that have been identified during causal analysis. These solutions aim to prevent defects from reoccurring across the development lifecycle.

In theActing phaseof the IDEAL model, the focus is on the implementation of the proposed solutions. Activities such as refining the solution and implementing the solution are key components of this phase. These actions are based on the plan developed in the earlier phases, and they are designed to apply the solutions to achieve the desired improvements. Specifically:

Refine solutions (i)refers to improving or adjusting the solution based on feedback from pilots or initial tests.

Implement Solution (iii)refers to the full implementation of the refined solution across the test process or organisation​​.

Planning pokeris an Agile technique commonly used to estimate effort, and it can be adapted for use in theTest Planningprocess area to help estimate test efforts (SG3: Establish Test Estimates). This collaborative technique brings stakeholders together to estimate the effort required for tasks by assigning numerical values in a consensus-driven manner. It is particularly effective in Agile environments where estimation is iterative and frequently updated based on evolving project needs​.

Defect Preventionis aTMMi Level 5process area. This process area focuses on identifying the root causes of defects and implementing preventive measures to avoid their recurrence. At TMMi Level 5, the goal is to optimize processes and move from defect detection to defect prevention.

Advanced ReviewsandTest Measurementare Level 4 process areas, whileOrganizational Innovation and Deploymentis part of CMMI, not TMMi.

TMMi References:

Defect Preventionis a key process area inTMMi Level 5, focusing on continuous process improvement and defect avoidance​​.

ISO/IEC 15504, which is known for its process assessment model, was not one of the sources used during the development of the TMMi model. The TMMi model was developed using inputs from sources such as the Capability Maturity Model Integration (CMMI), international testing standards (like those from ISO and IEEE), and industry testing best practices surveys. While the ISO/IEC 15504 model has relevance to general process assessments, it was not specifically referenced in the creation of the TMMi.

In the context of TMMi and test improvement programs, anindirect benefitrefers to outcomes that are not directly tied to the technical improvement of the testing process but affect the overall success of the organization in less measurable ways.

Increased personnel motivationis considered anindirect benefitof a test improvement program because it boosts team morale and engagement, leading to better productivity in the long run. This is different from direct benefits such asimprovements in defect detectionortest execution lead-time, which are quantifiable metrics directly related to the testing process.

TMMi References:

Direct benefitssuch as defect detection rates and test execution speed are frequently mentioned in TMMi as measurable outcomes from process improvement efforts.

Indirect benefits, like improved motivation, are acknowledged as part of the cultural and organizational improvements that can come from a well-executed test improvement strategy​​.

A direct benefit of a test improvement program ishigher productivity. When test processes are streamlined, better test planning, design, and execution can lead to reduced testing times, fewer defects, and quicker turnaround of quality products. This has a direct impact on organisational efficiency, allowing for more effective use of resources and time.

As part of theProduct Quality Evaluationprocess area (4.2), the specific goal"Project Goals for Product Quality and their Priorities are Established"includes the practice ofidentifying product quality needs. This involves understanding and defining what quality means for the product, determining quality characteristics (such as reliability, usability, etc.), and establishing priorities for these characteristics based on project goals.

Other practices such asestablishing test process measuresandmeasuring product quality quantitativelycome later in the lifecycle or are part of other specific goals.

TMMi References:

InProduct Quality EvaluationatTMMi Level 4, identifying product quality needs is critical for setting up measurable and prioritized goals for product quality​​.

In the scenario described, where an organisation suffers from poor test estimations despite many practices being in place, the process area that requires attention is Test Planning. Specifically,SG 3 (Establish Test Estimates)focuses on developing well-founded estimates for testing effort and cost. This includes creating a top-level work breakdown structure, defining the test lifecycle, and determining estimates for the test effort and costs associated with different test activities. By addressing these practices, the organisation can improve its ability to generate accurate test estimations, leading to better project planning and resource allocation.

Regularcode reviewsandarchitectural reviewsare part of theAdvanced Reviewsprocess area, which belongs toTMMi Level 4. This process area emphasizes conducting static reviews (e.g., code and design reviews) early in the lifecycle to measure product quality and adjust the testing strategy based on the insights gained from these reviews.

TMMi References:

TheAdvanced Reviewsprocess area in TMMi Level 4 aims to ensure that static testing activities such as code and architectural reviews are used to measure product quality early in the lifecycle​​.

TheActing phaseof a test improvement cycle focuses on executing the planned improvements. Key activities during this phase includecreating solutionsto address identified problems,planning actionsto implement those solutions, andpiloting solutionsbefore full-scale deployment. Thesesteps ensure that the improvements are well-defined, tested in a controlled environment, and refined before being fully implemented across the organisation.

The specific goal during the implementation of the "Test Planning" process area at TMMi Level 2 is to establish well-founded plans for managing and performing the testing activities. One of the crucial objectives is to"Establish a Master Test Plan"at TMMi Level 3, particularly under the "Test Lifecycle and Integration" process area. This is designed to ensure that a coherent, high-level plan integrates the activities across all levels of testing.

The process includes defining a test approach based on identified risks and involves the creation of the test estimates, schedules, and plans. Test plans should align with organisational strategies and cover test activities for all relevant phases of the development cycle​​.

In an Agile context, the emphasis is typically placed on lightweight, flexible documentation. Test documentation should be "just enough" to support the test process, and excessive documentation may be seen as unnecessary overhead. Therefore, calling formore rigorous and thorough documentationisnotin line with Agile principles. Instead, Agile favours working software and collaboration over comprehensive documentation, supported by mentoring and just-in-time guidance rather than formalised documentation​.

The process area that addresses the definition of test functions and the establishment of test career paths is the Test Organization process area,found at TMMi Level 3. This process area focuses on creating a well-structured test organisation, defining various test functions, and establishing career paths for testers. Specific practices include defining roles for test specialists, assigning staff to those roles, and establishing career development frameworks to support the professional growth of testers within the organisation.

The generic practice"Establish a defined process"addresses the need to tailor a managed process from the organization’s set of standard processes. This practice is part ofGeneric Goal 3 (GG 3), which aims to ensure that processes are standardized and defined across the organization. By tailoring processes based on established guidelines, organizations can create specific implementations that fit their projects while maintaining consistency and control across the organization.

TMMi References:

In TMMi, establishing a defined process involves tailoring standard processes to suit specific project needs, ensuring the institutionalization of best practices across projects​​.

Sprint retrospectives are held at the end of every sprint (not every 2-3 iterations) to evaluate the team's performance and identify opportunities for improvement. Retrospectives are a core Agile practice, but they are not considered peer reviews, which typically involve reviewing work products such as code or design documents. Peer reviews focus on product quality, while retrospectives focus more on team performance and process improvements.

Formal assessments, in contrast to informal ones, are conducted with more rigor, and the results can lead to an official certification of maturity level. A formal assessment report includes a validated maturity level rating, making it distinct from informal assessments, which are used only for indicative purposes and do not result in an official rating. Thus, only formal assessments provide an assessment report with a valid maturity level rating​​.

Expected components in the TMMi model guide organisations on how to implement improvements or conduct assessments. These components, such as Specific Practices and Generic Practices, are necessary for satisfying goals. Expected components must be implemented, either as described or through acceptable alternatives, for an organisation to achieve compliance with the model and reach maturity in test processes.

This statement isFALSEbecauseTMMiis flexible and does not require the imposition of a predefined set of practices in Agile organizations. Instead, it encourages the adaptation of practices to fit the Agile context. TMMi allows for alternative practices that achieve the same goals as the expected practices, making it possible to integrate testing improvements in a way that aligns with Agile principles. The focus is on continuous improvement, not compliance with rigid practices.

TMMi References:

TMMiemphasizes flexibility in adapting practices, particularly in Agile environments where imposed processes would contradict Agile values of adaptability and continuous improvement​​.

A typical business reason for starting a test improvement program is to achievehigher productivity. Test process improvements can lead to better efficiency in test execution, reduction of defects, and optimised resource usage, all of which contribute to the overall productivity of the organisation. By improving testing processes, organisations can reduce time-to-market and minimise costs associated with fixing defects post-production, which in turn boosts productivity.

Peer ReviewsisNOTaTMMi Level 2process area. Peer Reviews is a process area atTMMi Level 3. At Level 3, organizations formalize their testing processes, including peer reviews, which involve static testing activities like inspections and walkthroughs to identify defects early.

Test Policy and Strategy,Test Monitoring and Control, andTest Environmentare all process areas atTMMi Level 2.

TMMi References:

Peer Reviews are introduced atTMMi Level 3, whereasTest Policy and Strategy,Test Monitoring and Control, andTest Environmentare process areas at Level 2​​.

In Agile environments, TMMi is often used to complement the Agile methodology by providing structured guidance on critical testing practices that may be overlooked. While Agile focuses on flexibility and responsiveness, TMMi can remind organisations of essential practices that ensure thorough testing, such as formal reviews or defect management. The TMMi framework does not impose strict processes but offers structured support for critical test process improvements that align with Agile principles.

The specific practice"Establish the Organization’s Test Process Database"is found within theTest Lifecycle and Integrationprocess area, which is part ofTMMi Level 3. This process area aims to establish standardized test processes across the organization, including maintaining atest process databasethat holds process-related information and data for reuse and continuous improvement.

TMMi References:

TheTest Lifecycle and Integrationprocess area is responsible for establishing and maintaining organizational test process assets, including the test process database​​.

Advanced Reviewsis aTMMi Level 4process area. AtTMMi Level 4 (Measured), organizations focus on making testing aquantifiable process, using metrics and measurements to evaluate and improve product quality and the testing process.Advanced Reviewsinvolve more structured andformalized review processes that are integrated with the dynamic testing approach to measure and improve product quality earlier in the lifecycle.

TheTest Policy and Strategyis part ofLevel 2.

Defect Preventionis part ofLevel 5.

Test Lifecycle and Integrationis part ofLevel 3.

TMMi References:

TMMi definesAdvanced Reviewsas a Level 4 process area aimed at improving product quality through more rigorous and measurable review processes​​.

To improve the management and control of test environments, one of the specific practices under SG 3 (Manage and Control Test Environments) of the Test Environment process area is to coordinate the availability and usage of test environments. This involves managing reservations, scheduling, and ensuring the efficient use of test environments across different projects. This practice is essential for ensuring that the test environments are available when needed, reducing delays, and improving the overall efficiency of the testing process.

TheTMMi process area Test Planningis indeed supported by severalCMMI process areas, includingProject Planning,Requirements Management, andRisk Management. These CMMI areas provide foundational support for ensuring that the test planning process incorporates adequate project management, requirement traceability, and risk identification, all of which are critical for effective testing.

Specific Goal 2 of the CMMI Validation process arearelates to validation activities, not peer reviews, and therefore does not support the TMMi Peer Reviews process area.

Causal Analysis and Resolutionis linked toDefect Preventionin TMMi, not Quality Control.

Performance Managementdoes not directly support the Non-Functional Testing process area in TMMi.

TMMi References:

TheTMMi Test Planningprocess area is supported by relevant CMMI process areas, such asProject Planning, which helps in aligning project goals with test planning activities​​.