Is it possible to cover all scenarios ? - Part II

So lets say we have the following code:

def several_scenarios_method(val):
       if eval_condition1(val) or eval_condition2(val:
           do_something()
           return calculation1()
       return calculation2()

So how many scenarios do we have in this method ?

Lets imagine that a new condition has been requested that should satisfy condition 1 and 2 and execute a special logic.

def several_scenarios_method(val):
       if eval_condition1(val) or eval_condition2(val:
           if eval_condition3():
                do_something_special()
           do_something()
           return calculation1()
       return calculation2()

Then what is the new conditions tree ?

Ok nothing unusual. But our Product Owner comes to our desk and tell us to add a quick condition. "it is just an if" he says smiling. You nod gracefully and as it is almost time for lunch you add a new condition 4 !.

def several_scenarios_method(val):
       if eval_condition1(val) or eval_condition2(val:
           if eval_condition3():
                if eval_condition4():
                   do_something_even_more_special()
                do_something_special()
           do_something()
           return calculation1()
       return calculation2()

Ok lets see our conditions tree again !

Now the question is what will happen if we continue adding conditions that way ?

As we do a quick math we will get exponential growth of our conditions. In our scenario 2^N where N is the level of conditions that we continually keep adding.

So at this point i think it is clear that conditionals are evil when creating new scenarios but also we learn that manual testing is futile at this point. As it is not possible that a person can keep all the scenarios in his/her head.

Lets be a bit optimistic and lets say we are good engineers and we do unit testing !. So as we add new conditions we are covering all new scenarios. So we have a new code:

def code_that_only_admin_can_access(name):
      if name != 'admin' and name.islower() :
         return False
     return True

So we create our tests:

def test_code_that_only_admin_can_access_FAILURE():
      self.assertEqual(self.accessClass.code_that_only_admin_can_access('ADMIN'), False)

def test_code_that_only_admin_can_access_SUCCESS():
      self.assertEqual(self.accessClass.code_that_only_admin_can_access('admin'), True)

Obviously nobody will do that horrondeous code ever(don't we ?) but lets imagine that someone was very sleepy and changed the code to:

def code_that_only_admin_can_access(name):
      if name != 'admin' or name.islower() :
         return False
     return True

Our test will fail ? Or we will seen 100% coverage. Answer is YES. Tests wont fail.

Everybody will be happy until our security department told us that all users now have admin role and we better learn magic and make dissapear ourselves from the company.

That's it for now. Next blog post i will try to answer the question or maybe try to give a solution who knows ;).

/*------------------------- BTW the technique used above is called mutation testing. Cool name for a great technique that can show us that our tests are garbage if any change to our code does not have any impact in our tests.

testing.googleblog.com/2021/04/mutation-tes.. ---------------------------*/