![](https://csdnimg.cn/release/download_crawler_static/73453997/bg6.jpg)
CSCW ’21, , Liu et al.
Compared with synchronous collaboration (through shared workplaces or real-time networked
displays) [
37
,
39
,
40
,
49
], remote asynchronous collaboration [
28
,
30
,
75
] remains relatively unex-
plored, particularly in the context of time series data. Compared with text or images, time series are
intrinsically dicult for humans to interpret without sucient contextual information. This makes
it even harder for annotators to create accurate externalizations (external representations of a per-
son’s internal thoughts) in order to communicate. How to encode and display these externalizations,
and how to maintain awareness between team members in a remote asynchronous setting, are
still open research questions. In this work, we summarize a streamlined human-AI collaboration
workow that supports ecient collaborative anomaly annotations. More importantly, we propose
a set of novel visualizations and interactions to support in-situ annotation and communication, as
well as to enhance team members’ awareness of each other’s ndings.
3 METHOD
3.1 Iterative Design
We followed an iterative user-centered design process to develop MTV. We collaborated with
industrial domain experts, gathering design requirements and collecting feedback from them. We
began with proof-of-concept mockup designs made using Figma (an online collaborative interface
design tool). We then moved on to an interactive system prototype, a high-delity prototype, and
eventually to a deployable system.
3.1.1 Participants: The entire design process involved 9 participants in two groups.
The primary group consisted of 6 domain experts from a world-leading communication satellite
company: one spacecraft program manager (P1) and ve senior satellite engineers (P2-P6). The
team analyzes spacecraft telemetry data for signs of hazards that may result in system failure. Each
expert has between 5 and 17 years of spacecraft telemetry data analysis experience, and between 0
and 3 years of machine learning experience. This group participated in our regular design meetings,
which occurred once or twice per month for three years, and provided feedback on prototypes.
The external group included 3 senior engineers (E1-E3) from an energy company with expertise
in analyzing time series from wind turbines and energy pipelines. They took surveys about the
system design and usability, and tested MTV with their own data. This group provided assurance
that the design requirements and features we developed with the primary group are generalizable
in certain contexts (dened in Fig. 2), as well as applicable to similar elds.
3.1.2 Design stages: The design process occurred in three stages.
We began the rst stage by conducting interviews with the experts from the primary group
(P1-P6) to understand in detail the methods they use to analyze spacecraft telemetry data for
anomalies as part of their routine workload, along with the associated challenges. We detail this
process in Section 3.2. Knowing all these challenges, we had additional conversions with E1-E3 and
conrmed that the same challenges also exist in their domain.
In this rst stage, we gathered initial requirements and designed mockups using Figma by
choosing dierent combinations of visualization techniques in accordance with design principles.
We rened the requirements iteratively, and nally derived a set of requirements to guide our
development of the early version of MTV.
In the second stage, we focused on implementing MTV and iteratively improving it by engaging
domain experts from the primary group. We then ran our high-delity prototype on public data.
This process involved many informal discussions and interviews. From time to time, we also
presented the prototype to E1-E3 and collected feedback from them. The output of this stage was a
high-delity software prototype.
6