![](https://csdnimg.cn/release/download_crawler_static/88021265/bg3.jpg)
Kreuzberger, Kühl, and Hirschl
Strauss [5, p.61], this stage is called “theoretical saturation.” All
interviews are conducted between June and August 2021.
With regard to the interview design, we prepare a semi-
structured guide with several questions, documented as an
interview script [33]. During the interviews, “soft laddering” is
used with “how” and “why” questions to probe the interviewees’
means-end chain [39]. This methodical approach allowed us to gain
additional insight into the experiences of the interviewees when
required. All interviews are recorded and then transcribed. To
evaluate the interview transcripts, we use an open coding scheme
[8].
4 Results
We apply the described methodology and structure our resulting
insights into a presentation of important principles, their resulting
instantiation as components, the description of necessary roles, as
well as a suggestion for the architecture and workflow resulting
from the combination of these aspects. Finally, we derive the
conceptualization of the term and provide a definition of MLOps.
4.1 Principles
A principle is viewed as a general or basic truth, a value, or a
guide for behavior. In the context of MLOps, a principle is a guide
to how things should be realized in MLOps and is closely related
to the term “best practices” from the professional sector. Based on
the outlined methodology, we identified nine principles required to
realize MLOps. Figure 2 provides an illustration of these principles
and links them to the components with which they are associated.
P1 CI/CD automation. CI/CD automation provides continuous
integration, continuous delivery, and continuous deployment. It
carries out the build, test, delivery, and deploy steps. It provides
fast feedback to developers regarding the success or failure of
certain steps, thus increasing the overall productivity
[15,17,26,27,35,42,46] [α, β, θ].
P2 Workflow orchestration. Workflow orchestration
coordinates the tasks of an ML workflow pipeline according to
directed acyclic graphs (DAGs). DAGs define the task execution
order by considering relationships and dependencies
[14,17,26,32,40,41] [α, β, γ, δ, ζ, η].
P3 Reproducibility. Reproducibility is the ability to reproduce
an ML experiment and obtain the exact same results [14,32,40,46]
[α, β, δ, ε, η].
P4 Versioning. Versioning ensures the versioning of data,
model, and code to enable not only reproducibility, but also
traceability (for compliance and auditing reasons) [14,32,40,46] [α,
β, δ, ε, η].
P5 Collaboration. Collaboration ensures the possibility to
work collaboratively on data, model, and code. Besides the
technical aspect, this principle emphasizes a collaborative and
communicative work culture aiming to reduce domain silos
between different roles [14,26,40] [α, δ, θ].
P6 Continuous ML training & evaluation. Continuous
training means periodic retraining of the ML model based on new
feature data. Continuous training is enabled through the support of
a monitoring component, a feedback loop, and an automated ML
workflow pipeline. Continuous training always includes an
evaluation run to assess the change in model quality [10,17,19,46]
[β, δ, η, θ].
P7 ML metadata tracking/logging. Metadata is tracked and
logged for each orchestrated ML workflow task. Metadata tracking
and logging is required for each training job iteration (e.g., training
date and time, duration, etc.), including the model specific
metadata—e.g., used parameters and the resulting performance
metrics, model lineage: data and code used—to ensure the full
traceability of experiment runs [26,27,29,32,35] [α, β, δ, ε, ζ, η, θ].
P8 Continuous monitoring. Continuous monitoring implies
the periodic assessment of data, model, code, infrastructure
resources, and model serving performance (e.g., prediction
accuracy) to detect potential errors or changes that influence the
product quality [4,7,10,27,29,42,46] [α, β, γ, δ, ε, ζ, η].
P9 Feedback loops. Multiple feedback loops are required to
integrate insights from the quality assessment step into the
development or engineering process (e.g., a feedback loop from the
experimental model engineering stage to the previous feature
engineering stage). Another feedback loop is required from the
monitoring component (e.g., observing the model serving
performance) to the scheduler to enable the retraining
[4,6,7,17,27,46] [α, β, δ, ζ, η, θ].
4.2 Technical Components
After identifying the principles that need to be incorporated into
MLOps, we now elaborate on the precise components and
implement them in the ML systems design. In the following, the
components are listed and described in a generic way with their
essential functionalities. The references in brackets refer to the
respective principles that the technical components are
implementing.
C1 CI/CD Component (P1, P6, P9). The CI/CD component
ensures continuous integration, continuous delivery, and
continuous deployment. It takes care of the build, test, delivery, and
deploy steps. It provides rapid feedback to developers regarding the
success or failure of certain steps, thus increasing the overall
productivity [10,15,17,26,35,46] [α, β, γ, ε, ζ, η]. Examples are
Jenkins [17,26] and GitHub actions (η).
Source Code
Repository
CI/CD
Component
Workflow
Orchestration
Component
Feature
Stores
Model Training
Infrastructure
Model
Registry
ML Metadata
Stores
Monitoring
Component
Model Serving
Component
PRINCIPLES
P1 CI/CD automation
P2 Workflow orchestration
P3 Reproducibility
P4 Versioning of data, code, model
P5 Collaboration
P6 Continuous ML training & evaluation
P7 ML metadata tracking
P8 Continuous monitoring
P9 Feedback loops
P1
P6
P9
P4
P5
P2
P3
P6
P6
P3
P4
P3
P4
P8
P9
P4
P7
P1
COMPONENT
Figure 2. Implementation of principles within technical
components