The Lost Art of Experimentation in Software Engineering

How do you know when developing a new feature or a new app if is going to work ?

And im not talking about A/B testing or UX researching techniques. Im talking about a technical decision.

Answer is Experimentation but in my years of experience what I have noticed in most developers is the following:

What is the problem with this ?

• It is a waste of time as we can fail minutes, hours or even days.
• It is a brute-force algorithm that we are applying in our own developments.
• We might just choose the first option that works ignoring better options.
• Creativity is blocked as we are not consciously designing the solution but just trying options without learning.

Even senior developers use this try-error technique but with a difference:

Yes, senior developers now have a cache where previous errors in their years of experience are saved so they ignore these approaches and try new ones. However, same problems as before continue to be present. It is not optimal and even it is not doing engineering.

Then what is being Experimental ? Or doing Experiments ?

From the awesome book: Modern Software Engineering.

Being Experimental Experimentation is defined as “a procedure carried out to support, refute, or validate a hypothesis. Experiments provide insight into cause-and-effect by demonstrating what outcome occurs when a particular factor is manipulated.”1

Taking an experimental approach to solving problems is profoundly important. I would argue that science, and the experimental practice at its heart, is what differentiates our modern, high-tech society from the agrarian societies that preceded us, more than anything else.

For example, let's pretend that we need to choose a new library in our project ! What can we do ?

So let's say that we trust our colleagues and choose a library based on their recommendations. However after a month we noticed that the library does not support a deserialization of a type that we use in our project !

Now we are in trouble and our colleague just told us that for their project they don't have that problem. Solution ? Well you just extend the library and override some methods and Voila now you have a tight coupled dependency. If that open source library is deprecated or an update is done. You will need to change your code too !.

How could this be avoided ? Yep you guess right => Experimentation.

The experiment would look something like this:

Most of these experiments in engineering are called POC or Proof of Concept. POC are a tool where you just try a new library, technique, optimization in a controlled environment or section of code.

A proof of concept (POC) is an exercise in which work is focused on determining whether an idea can be turned into a reality. A proof of concept is meant to determine the feasibility of the idea or to verify that the idea will function as envisioned. source: techtarget.com/searchcio/definition/proof-o..

Oh for new projects this is fine but what about legacy code ? How can I do experiments safely ?

Well Feature Flags should always be in your projects toolkits. A feature flag allow us to turn on or off a feature, an optimization or even a big code change.

In summary a feature flag will look like:

if (feature_flag_is_ON) { // run new code } else { // run old code }

However one of the disadvantages is how intrusive can be in the code. As you will need to add flags in all the code that might change based on your experiment. Happily feature flags are removed once the new feature/library/change has proven to be ok and optimal.

So just don't run as a hamster in a wheel with try/error approaches but be smart and choose where and when to run.

EOF