Is Test-Driven Development effective?

During some interesting discussion, one very respectful gentleman said “there are plenty of studies demonstrating the effectiveness of TDD”. But is it so?

Test-driven development (TDD) has been a hot topic for a long time. There’re proponents and opponents of TDD. There seems to be no middle ground for those opposing groups — TDD works well for those who claim that TDD is beneficial and spectacularly fail to work for those who think otherwise.

Is there any scientific evidence?

There’re many pieces of research about TDD effectiveness, and many papers conclude that TDD brings good results, however, their findings are not consistent.

For example, in the paper called “An Experimental Evaluation of the Effectiveness and Efficiency of the Test Driven Development”, published in 2007 it was concluded that while TDD increases efficiency there was no evidence of quality improvement [1].

In a separate paper, which reviewed and summarized several related pieces of research, “Evaluating the Effectiveness of Test Driven Development: Advantages and Pitfalls”, published in 2017 it was found that test-first approach can improve quality, while decreasing productivity. Besides, it was claimed that most developers feel test-last approach as more natural [2].

During the “A Replicated Experiment on the Effectiveness of Test-First Development”, 2013 they could not find any effect of using Test-First approach on quality or productivity with comparison to Test-Last approach [3].

Finally, in paper “Overview of the Test Driven Development Research Projects and Experiments”, 2012, which tried to find a decisive conclusion based on several pieces of research and papers, it was concluded that there’s no persuasive evidence of the positive impact of TDD neither on quality nor productivity [4].

So, why so many people would still insist that TDD is so effective technique?

As a huge proponent of TDD, I can offer only one possible answer — the effectiveness of TDD hugely depends on the developer. It was not possible to get a consistent result on the TDD because it was not [only] TDD impacting quality and productivity, but developers themselves. It seems that there’re developers for whom TDD brings very good results, there’re developers for whom it does not have any positive (or negative) effect and there’re developers whose productivity and effectiveness is deteriorated by TDD.

I can offer only one possible answer — the effectiveness of TDD hugely depends on the developer.

The only thing now would be to consider if we can design an experiment to prove or disprove this hypothesis. In my view, this hypothesis is a natural advancement of the idea that development is a social activity. The only problem with social phenomenons is that they are not deterministic, which would explain so different results.

References

[1] A. Gupta, P. Jalote “An Experimental Evaluation of the Effectiveness and Efficiency of the Test Driven Development” presented at First International Symposium on Empirical Software Engineering and Measurement, Madrid, Spain, 2007
[2] Z. Khanam, M. Ahsan, “Evaluating the Effectiveness of Test Driven Development: Advantages and Pitfalls” International Journal of Applied Engineering Research vol. 12, no 18, p. 7705, 2017. Available: https://www.ripublication.com/ijaer17/ijaerv12n18_81.pdf. [Accessed May 13, 2020].
[3] D. Fucci, B. Turhan, “A Replicated Experiment on the Effectiveness of Test-first Development” presented at ACM / IEEE International Symposium on Empirical Software Engineering and Measurement, Baltimore, MD, USA, 2013
[4] A. Bulajic, S. Sambasivam and R. Stojic “Overview of the Test Driven Development Research Projects and Experiments” presented at Proceedings of Informing Science & IT Education Conference (InSITE), 2012. Available: https://pdfs.semanticscholar.org/3278/971bb53a25822171df127d47a57243dd3bbd.pdf. [Accessed May 13, 2020].

Originally published at https://senpay.github.io.

Russian translation: https://habr.com/ru/post/501900/