Skip to main content

What to Test?

When you are inside the code for an application, often this question doesn't get pondered.

 It seems like an obvious question, "What to test?" "Duh, everything!" or "What I just implemented." 

 But really, testing everything is often out of the question for a reasonably complex application, The permutations of states and settings grows quickly. If you have 4 settings which can each be one of two values, that's 16 (2 x 2 x 2 x 2) different combinations to be tested. Add another setting of only 2 choices and you have 32 combinations. If each setting is one of three values, there are 81 (3 x 3 x 3 x 3) combinations.  I think you can see the problem. If you have a good unit testing framework that will let you automate generation of numerous test cases, that can help a lot. But you still need to consider the question and the answer and decide if you are testing the right parts of the application , whether it's unit tests, manual tests, system tests, whatever.

 There are a number of different approaches you can think through to analyze your testing, which can help answer the question of what to test and put together a plan for effective testing. The first idea is to think about where risks are. Often this is in the newest code, the most complicated or the areas with the most demanding performance requirements.  Look for long routines and count the conditional statements if you want to make it a quantitative analysis. 

 Looking at the interfaces is a different angle for analysis. Not just application programming interfaces, but also network communications, shared memory, library calls, database queries and data acquisition are all types of interfaces. Interfaces are often the areas where there are more edge cases to test and where there is more chance for input values to wrong. 

 Often using probabilities can help focus testing efforts. What are the most likely code paths? Here you can make use of logging and historical information from your software.  Looking at your application from the outside-in or top-down plays well in terms of looking at features for testing.

There's always something to test. Of course, these analyses can also help inform your decisions on what not to test as well. This is equally important, as you want to avoid diminis hinges returns on your testing efforts. 

Good ideas on what to do instead of test is here: https://users.ece.cmu.edu/~koopman/des_s99/sw_testing/#alternative

Labels:

Comments

Post a Comment

Popular posts from this blog

Let's Not Mess Around with Security on our Personal Systems Either!

Essential Security Practices for Your Personal Systems Ensuring a minimal level of cybersecurity, privacy, and availability on your personal systems means you need to manage the following essential practices. This is a brief overview of recommendations from sources like CISA, NSA, etc., focused on personal laptop, phone, and other systems' security. Anti-virus  I've found you'll get the best anti-virus protection and usability from a paid product - I've always had good luck with Norton labeled products. If you are looking for current vendor offerings see:  https://www.pcmag.com/picks/the-best-antivirus-protection Regardless of whether you choose to use a commercial product or open-source anti-virus tool, it is absolutely something you need to use. This is the minimally needed level of system security. Once installed, ideally, it should be invisible until there's a security problem it can't prevent or solve.   Backups You need to have at least a minimal level of ...
Post a Comment
Read more

RACI, Cybersecurity and NICE Framework

The NICE framework from a RACI point of view The NICE framework ( NIST SP 800-181 rev. 1) established a standard approach for describing cybersecurity work, in order to help stakeholders share a common language and ideally improve how to identify, recruit, develop and retain talent. It breaks down cybersecurity work role categories into: Oversight and Governance; Design and Development; Implementation and Operation; Protection and Defense; Investigation.  Which is very cybersecurity-centric and not related to common tools for project management within companies. Especially smaller enterprises that do not have dedicated people to mange and coordinate cybersecurity needs. A  RACI chart  is   a project management tool used to define and clarify roles and responsibilities within a project team.   It stands for Responsible, Accountable, Consulted, and Informed, and visually represents who is responsible for what, who is accountable for the outcome, who needs to be c...
Post a Comment
Read more

Typescript - It might not be easier, but but it's surely different

Typescript is a statically typed language, that is a superset of JavaScript. I've had the discussions and debates about that aspect of the language. I am all for static typing. Any way my tools can help me be better is alright by me. So I avoid the ' any ' type designation and make sure I have guards on ' unknown ' types, as much as I can.  Any  does not carry any useful type information, while unknown does, and allows it to enforce type checking.  Anything can be assigned to a variable of type unknown , but an unknown value cannot be assigned to variables of other types without explicit type assertion or narrowing. Similarly, no operations are permitted on an unknown value until its type is refined. This behavior ensures type safety and prevents runtime errors. (Refined with help from google). I bring this up because I was arguing with the compiler recently because I'd assumed both made no use on any type information in any circumstance - because I haven't ...
Post a Comment
Read more