Software engineering has a crucial role in managing complex scientific facilities
The importance of software in scientific research and facilities like the European XFEL cannot be overstated. This presentation highlights the significance of software in handling massive amounts of data and managing complex systems, emphasizing the need for sustainable software engineering practices.
Firstly, the motivation behind the importance of software in scientific research is introduced. The exponential growth of computing power (Moore’s Law) and the increasing volume of data generated by modern light sources like XFEL is illustrated. The facility has produced around 100 petabytes of data, emphasizing the role of software in processing and managing this massive dataset.
Sustainable software engineering
Software at XFEL includes control software, data acquisition software, data processing tools, and coordination procedures. XFEL has over 400,000 lines of code, and around 50 out of 150 staff members are specifically tasked with writing software.
The concept of sustainable software engineering is introduced, emphasizing the efficiency, longevity, and network/storage efficiency of software. Sustainable software should also be sustainable for humans to use, develop, and maintain.
Interdisciplinary software development
Examples of software applications at XFEL are provided, such as the Karabo distributed control system, which enables remote control and monitoring of equipment. Its event-driven nature reduces unnecessary data transmission. Also highlighted is the importance of testing and code reviews to ensure reliability.
The interdisciplinary nature of software development at XFEL is discussed: experts in software engineering, physics, and other fields are involved. The need for gender diversity in software teams is pointed out.
Agile vs waterfall
The benefits of agile development methodologies, which empower teams to think creatively, iterate rapidly, and receive frequent feedback, is highlighted. Agile is contrasted with the traditional waterfall model, which can lead to less adaptability and longer development cycles.
Lastly, the Data Operations Center (DOC) at XFEL is described. It provides support for various aspects of data management and analysis. An incident involving a broken fiber optic cable is used to illustrate how interdisciplinary teams and agile methods help solve complex problems efficiently.
Software is critical
In conclusion, this presentation underscores the critical role of software in managing the vast amounts of data and complex systems at XFEL. Sustainable software engineering practices, interdisciplinary teams, and agile methodologies contribute to the success of XFEL’s software development efforts. Moreover, the knowledge and skills acquired in this process can be applied in various scientific facilities and industries, making XFEL’s software development efforts valuable beyond its specific use case
