Towards a philosophical understanding of agile software methodologies : the case of Kuhn versus Popper

Abstract

This dissertation is original in using the contrasting ideas of two leading 20th century philosophers of science, Karl Popper and Thomas Kuhn, to provide a philosophical understanding, firstly, of the shift from traditional software methodologies to the so-called Agile methodologies, and, secondly, of the values, principles and practices underlying the most prominent of the Agile methodologies, Extreme Programming (XP). This dissertation will take a revisionist approach, following Fuller—the founder of social epistemology—in reading Popper against Kuhn's epistemological hegemony. The investigations in this dissertation relate to two main branches of philosophy— epistemology and ethics. The epistemological part of this dissertation compares both Kuhn and Popper's alternative ideas of the development of scientific knowledge to the Agile methodologists' ideas of the development of software, in order to assess the extent to which Agile software development resembles a scientific discipline. The investigations relating to ethics in this dissertation transfer concepts from social engineering—in particular, Popper's distinction between piecemeal and utopian social engineering—to software engineering, in order to assess both the democratic and authoritarian aspects of Agile software development and management. The use of Kuhn's ideas of scientific revolutions and paradigm shift by several leading figures of the Agile software methodologies—most notably, Kent Beck, the leader of the most prominent Agile software methodology, Extreme Programming (XP)—to predict a fundamental shift from traditional to Agile software methodologies, is critically assessed in this dissertation. A systematic investigation into whether Kuhn's theory as a whole, can provide an adequate account of the day-to-day practice of Agile software development is also provided. As an alternative to the use of Kuhn's ideas, the critical rationalist philosophy of Karl Popper is investigated. On the one hand, this dissertation assesses whether the epistemological aspects of Popper's philosophy—especially his notions of falsificationism, evolutionary epistemology, and three worlds metaphysics—provide a suitable framework for understanding the philosophical basis of everyday Agile software development. On the other hand, the aspects of Popper's philosophy relating to ethics, which provide an ideal for scientific practice in an open society, are investigated in order to determine whether they coincide with the avowedly democratic values of Agile software methodologies. The investigations in this dissertation led to the following conclusions. Firstly, Kuhn's ideas are useful in predicting the effects of the full-scale adoption of Agile methodologies, and they describe the way in which several leaders of the Agile methodologies promote their methodologies; they do not, however, account for the detailed methodological practice of Agile software development. Secondly, several aspects of Popper's philosophy, were found to be aligned with several aspects of Agile software development. In relation to epistemology, Popper's principle of falsificationism provides a criterion for understanding the rational and scientific basis of several Agile principles and practices, his evolutionary epistemology resembles the iterative-incremental design approach of Agile methodologies, and his three worlds metaphysical model provides an understanding of both the nature of software, and the approach advocated by the Agile methodologists' of creating and sharing knowledge. In relation to ethics, Popper's notion of an open society provides an understanding of the rational and ethical basis of the values underlying Agile software development and management, as well as the piecemeal adoption of Agile software methodologies.