Making a Pipeline Production-Ready: Challenges and Lessons Learned in the Healthcare Domain
Lawand,Daniel Angelo Esteves Lam,Lucas Quaresma Medina Bolgheroni,Roberto Oliveira Ferreira,Renato Cordeiro Goldman,Alfredo Finger,Marcelo
Lawand,Daniel Angelo Esteves
Lam,Lucas Quaresma Medina
Bolgheroni,Roberto Oliveira
Ferreira,Renato Cordeiro
Goldman,Alfredo
Finger,Marcelo
Abstract
Deploying a Machine Learning (ML) training pipeline into production requires good software engineering practices. Unfortunately, the typical data science workflow often leads to code that lacks critical software quality attributes. This experience report investigates this problem in SPIRA, a project whose goal is to create an ML-Enabled System (MLES) to pre-diagnose insufficiency respiratory via speech analysis. This paper presents an overview of the architecture of the MLES, then compares three versions of its Continuous Training subsystem: from a proof of concept Big Ball of Mud (v1), to a design pattern-based Modular Monolith (v2), to a test-driven set of Microservices (v3). Each version improved its overall extensibility, maintainability, robustness, and resiliency. The paper shares challenges and lessons learned in this process, offering insights for researchers and practitioners seeking to productionize their pipelines.
Description
Date
2025-05-09
Journal Title
Journal ISSN
Volume Title
Publisher
Research Projects
Organizational Units
Journal Issue
Keywords
Code Quality, MLOps, Software Architecture, Machine Learning Enabled Systems, Healthcare Domain, Experience Report
Citation
Lawand, D A E, Lam, L Q M, Bolgheroni, R O, Ferreira, R C, Goldman, A & Finger, M 2025, Making a Pipeline Production-Ready : Challenges and Lessons Learned in the Healthcare Domain. in Software Architecture : ECSA 2025 Tracks and Workshops: Limassol, Cyprus, September 15–19, 2025, Proceedings. vol. 15982, Lecture Notes in Computer Science, vol. 15982, pp. 354-362. https://doi.org/10.1007/978-3-032-04403-7_30