How to Test with Rapid Software Testing (RST) - Michael Bolton

The podcast delves into the philosophy and practice of software testing, emphasizing its role as a human-driven process rooted in exploration, experimentation, and critical thinking. It highlights Michael Bartons career and contributions to Rapid Software Testing (RST), a context-driven approach that prioritizes efficiency, shallow testing (to identify surface-level issues quickly), and the integration of human judgment over rigid methodologies. RST challenges misconceptions that testing is about proving a product works, instead focusing on identifying critical flaws early by leveraging experience, domain knowledge, and an investigative mindset. The discussion underscores the importance of testers as skilled investigators rather than passive executors of tasks, advocating for a balance between speed and depth in testing, especially in complex or AI-driven systems where automated checks alone cannot capture nuanced risks or user needs.

Key themes include the necessity of deep testing for hidden, subtle bugs in complex systems, the erosion of reflective testing due to industry pressures for speed, and the risks of over-reliance on automation or certifications that prioritize process over practical skills. The conversation critiques the mischaracterization of testing as a universal "best practice" and stresses the value of human-centric approaches that acknowledge fallibility and prioritize communication, ethical issue reporting, and contextual understanding. It also addresses the limitations of traditional metrics and KPIs, advocating instead for metrics that reflect meaningful risk assessment. Ultimately, the dialogue promotes RST as a flexible framework that empowers testers to adapt their methods, prioritize critical issues, and engage with users and developers to uncover problems that scripted or automated approaches might miss.

• What if you replaced your automated regression tests with a series of exploratory testing sessions focused on high-risk user flows?

  • Concrete move: Allocate 2 hours weekly to manually test 35 critical user paths, documenting unexpected behaviors and edge cases.
  • Why now: The text emphasizes that automation alone cannot identify meaningful issues, and RST prioritizes efficiency by focusing on critical risks early. Manual exploration uncovers subtle bugs automation might miss.
  • Expected upside: Faster identification of high-impact flaws, reduced reliance on brittle automation, and deeper understanding of user experience risks.

• What if you applied RSTs shallow testing principles to your next feature release, prioritizing speed over exhaustive test coverage?

  • Concrete move: Design a 1-hour test session targeting 3 key assumptions about the feature, using heuristics (e.g., "What if users input invalid data?").
  • Why now: RST advocates for shallow testing to quickly validate assumptions, aligning with the need for speed in modern development. This avoids wasting time on low-value tests.
  • Expected upside: Rapid feedback on critical risks, reduced testing overhead, and faster iteration cycles without sacrificing quality.

• What if you documented your testing process as an investigator, not just a task executor, using sketches or diagrams to explain risks to stakeholders?

  • Concrete move: After each test session, create a 1-page summary with a diagram showing the systems behavior under test, annotated with risks and assumptions.
  • Why now: The text highlights the need for testers to act as investigators and communicate complex ideas effectively. This approach aligns with RSTs focus on critical thinking and stakeholder alignment.
  • Expected upside: Improved stakeholder trust, clearer prioritization of fixes, and a stronger foundation for collaborative decision-making.

• Adopt Rapid Software Testing (RST) principles: Prioritize efficiency by focusing on shallow, context-driven testing to identify critical issues early, rather than exhaustive test case execution. Use experience and domain knowledge to guide investigation, not rigid methodologies.
• Integrate human judgment with automation: Use automated tools for checking (e.g., rule-based validation), but avoid equating automation with testing. Always evaluate discrepancies manually to determine if they represent meaningful issues, not just pass/fail outcomes.
• Engage with users and stakeholders: Identify context-specific risks by understanding user needs, interactions, and pain points. Avoid "malnourished" testing by actively gathering insights from users and developers to align testing with real-world scenarios.
• Re-evaluate KPIs and metrics: Replace traditional metrics (e.g., test case completion rates) with risk-based indicators that focus on uncovering critical issues. Prioritize metrics that reflect product understanding and risk coverage, not just volume or speed.
• Invest in critical thinking and communication skills: Develop skills in analytical systems thinking, user-centric empathy, and clear articulation of complex ideas. Focus on becoming an investigator with curiosity and skepticism, rather than relying on automation tools alone.

Here are some key insights and takeaways from the provided text, along with their relevance and usefulness to readers:

Key Insights:

1. Testing is a human behavior: Testing is inherent to human learning, evaluation, and exploration. This emphasizes the importance of human involvement in the testing process.
2. Efficiency in testing: Rapid Software Testing (RST) prioritizes the most efficient testing methods that quickly identify critical issues, focusing on speed and minimal resource usage.
3. Context-driven testing: RST emphasizes the importance of understanding the product's context, user needs, and system interactions to effectively identify problems.
4. Human judgment in testing: Automated checks can highlight discrepancies, but human judgment is necessary to evaluate the significance of these issues.
5. Testing beyond GUI automation: Automation should not be limited to graphical user interface (GUI) checks; it can be used for data collection, analysis, or generation.
6. Avoiding certifications: Certifications based on multiple-choice exams are not representative of practical skills, while skills-based certifications are more reliable.
7. Testers' role and pressure: Testers should not sign off on product releases; that responsibility lies with managers. Testers should focus on validating their testing work and identifying product risks.
8. Critical thinking in testing: Testing, especially RST, is rooted in critical thinking to avoid self-deception and ensure objective evaluation of the product.
9. Human qualities for effective testing: Testers must develop skills as investigators, focusing on critical thinking, self-reflection, and understanding user needs.
10. Applying Rapid Software Testing (RST): To apply RST, assess your testing practices, learn from resources like books, videos, and communities, and focus on critical thinking and user-centric skills.

Key Takeaways:

1. Focus on critical issues: Prioritize identifying the most critical issues in the product, rather than exhaustive testing.
2. Leverage experience and domain knowledge: Use your experience and domain knowledge to effectively test products.
3. View testers as skilled investigators: Recognize test developers as skilled investigators, not just task executors.
4. Test beyond GUI automation: Use automation for data collection, analysis, or generation, not just GUI checks.
5. Develop soft skills: Prioritize critical thinking, problem analysis, and communication skills over mastering tools like scripting frameworks.
6. Engage with the community: Network, learn from others, and gain practical experience to break into the field.
7. Tool agnosticism: Focus on the problem to solve, not the tools to use.

Why these insights and takeaways are relevant and useful:

1. Real-world application: These insights and takeaways are based on real-world experiences and case studies, making them relevant to readers who want to apply them in their own work.
2. Practical advice: The text provides practical advice and guidance on how to approach testing, making it useful for readers who want to improve their skills and knowledge.
3. Contextual understanding: The insights and takeaways emphasize the importance of understanding the product's context, user needs, and system interactions, which is essential for effective testing.
4. Human-centered approach: The text highlights the human-centered approach to testing, recognizing the importance of human judgment, empathy, and creativity in the testing process.
5. Adaptability: The insights and takeaways encourage adaptability and critical thinking, which are essential skills for testers in today's fast-changing technological landscape.

Overall, the text provides valuable insights and takeaways for readers who want to improve their testing skills and knowledge. It emphasizes the importance of human-centered approach, critical thinking, and practical application, making it relevant and useful for readers in various industries and roles.