116 S.M. Falconer and M.-A. Storey
own term matches by interacting with the schema trees. Hovering over a potential map-
ping displays a confidence level about the match as a value between zero and one.
P
ROMPT, developed by the Stanford Medical Informatics group, was designed as a
plugin for the popular ontology editor Prot´eg´e. The plugin supports managing multiple
ontologies including ontology differencing, extraction, merging, and mapping. The user
begins the mapping procedure by specifying a source and target ontology. P
ROMPT then
computes an initial set of candidate mappings based largelyon lexical similarity between
the ontologies. The user works with this list to verify the recommendations or create
custom mappings missed by the algorithm. Once a user verifies a mapping, P
ROMPT’s
algorithm uses this to perform analysis based on the graph structure of the ontologies.
This usually results in further mapping suggestions and the process is repeated until the
user deems the mapping complete. Similarly to P
ROMPT, AlViz is a plugin for Prot´eg´e
to do ontology mapping. However, the tool is in an early research phase.
OLA (OWL Lite Alignment) provides automated alignment and an environment for
manipulating alignments [11]. OLA supports parsing and visualization of ontologies,
automated computing of similarities between entities, manual construction of align-
ments, visualization of alignments, and comparison of alignments. The mapping algo-
rithm finds matches by analyzing the structural similarity between the ontologies using
graph-based similarity techniques. This information is combined with label similarity
measures to produce mapping correspondences.
Evaluations of these tools have mostly focused on comparing mappings produced
with known mappings. P
ROMPT is an exception in that the authors performed user eval-
uation experiments [20]. The experiment evaluated tool-generated mapping suggestions
by having several users merge two ontologies. The number of steps required, sugges-
tions followed and not followed, and resulting ontologies were all recorded. Precision
and recall was used to evaluate the quality of the suggestions. Similarly, Lambrix and
Edberg [16] performed a user evaluation of P
ROMPT and Chimaera [19] for the specific
use case of merging ontologies in bioinformatics. The participants were given a number
of tasks to perform, a user manual on paper, and the software’s help system for support.
They were also instructed to “think aloud” during the experimentwhile an evaluator took
notes. Afterwards, the users completed a questionnaire about their experience. The tools
were evaluated with the same precision and recall measurements used in the previously
described P
ROMPT experiment,while theuser interfaces were evaluated using the REAL
(Relevance, Efficiency, Attitude, and Learnability) [18] approach. Under both criteria,
P
ROMPT outperformed Chimaera, but the participants found learning how to merge on-
tologies in either tool was equally difficult. The participants found it particularly difficult
to perform non-automatedprocedures in P
ROMPT, such as creating user-definedmerges.
Other than these examples, little research has looked at the user side of mapping.
We propose that more comprehensive experiments that focus on how people perform
mappings will lead to productivity gains in schema matching [2].
3 Cognitive Support and Decision Making
Cognitive support refers to the assistance that tools provide to humans in their think-
ing and problem solving [30]. We often rely on external artifacts (tools) to support