Great leaps are being made in the development of Machine Learning (Artificial Intelligence) models that are capable of reading, identifying, and interpreting patterns in data; be they images, videos, words, or numbers. The systems that achieve this are complex by nature, and so are the teams that build them; typically comprising the blended skills of Data Engineers, Mathematicians and Data Scientists. Together these people are tasked with developing systems that consume data, process it and generate valuable insights that are then presented to their human operators to assist them in their work.
By definition these projects are complex and often experimental by nature, which means that Agile Practices should be ideally suited. But which ones to use and why? Why do Data Scientists, sometimes, rebel against being Agile? What do we need to do differently when working with experimental Data Science models, as opposed to established ones? Where does model training fit in? And how do we estimate tasks that are both simple and time consuming?