REGULAR PAPER ACCEPTED BY IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING 2
those in natural images. Furthermore, sketches can undergo
severe elastic deformations while remaining perceptually
similar [6]. In this paper, by unifying both 3D CAD models
and users’ input by a sketch representation, we propose a
user-profile-adaptive statistical modeling approach for sketch
similarity measurement. Different users may have different
sketching habits. One major advantage of the proposed
statistical modeling approach is that a system using it can
adapt itself to different users based on their individual
characteristic styles.
Recent neuroscience research [31] reveals that sketching
using the line drawing form may exploit the underlying neural
codes of human vision. As an effective way to communicate
message between designers, sketches may find a wide range
of applications in design automation in manufacturing industry
[17]. By mapping a user’s sketch directly to the engineering
models in the database, the wealth of model semantics across
product families and lifecycles can be used as early as in the
conceptual design stage [21], [24]. Since user’ sketches are
usually fuzzy and inaccurate, this mapping through retrieval
to the detailed product semantics may also offer a new way for
knowledge recovery and reuse in product design and assembly
[7], [35]. By taking user’s behavior into consideration, the
user-adaptive sketching model developed in this paper also
offers a preliminary study on cognition in production systems
[3]. To summarize, in this paper we propose a new 3D
CAD model retrieval method that uses a 2D pen-based sketch
input to reflect the user’s design intent and is suitable in a
conceptual design process in which no prior 3D CAD models
or detailed engineering drawings are available. In particular,
two contributions are made:
• A statistical modeling approach is proposed for sketch
similarity measurement, which can be tailored to any
individual sketching style.
• A sketch generation pipeline is proposed that converts
every 3D CAD model in a database into a small
yet sufficient set of representative sketches that are
perceptually similar to a person’s drawings.
II. RELATED WORK
Most companies in the manufacturing industry now
have massive libraries of 3D CAD models that are readily
available for use by designers. However, model retrieval by
matching some particular shape or form from such a library is
challenging. We can categorize different approaches of model
retrieval based on the types of inputs such a system requires.
First CAD models can be represented by different feature
types [24], including design features, machining features and
assembly features, etc. In this study, we mainly consider the
shape based retrieval, i.e., 3D CAD models are characterized
by their geometric and topological information such as holes,
pockets, fillets, chamfers and the adjacency relations between
them. By concentrating our work on shape based retrieval in
a design process, the retrieval methods can be grouped into
two categories, depending on whether the input is 2D data or
3D data.
For a 3D-input and 3D-output retrieval, how to encode the
3D design features with effective shape signatures is critical.
Many 3D shape signatures have been proposed and notably
they can be classified into global and local ones. A typical 3D
global shape signature that effectively depicts design features
is the attributed graph representation [13]. The nodes in an
attributed graph represent part surfaces and the edges are
part edges. The attributed graph itself encodes the model
topology and attributes encoding parts’ geometry are typically
attached to the nodes and edges in the graph. The local shape
signatures utilize the local model structure by segmenting
3D models based on design and assembly features. Several
representative local shape descriptors obtained from a scale-
space decomposition had been proposed in [5], [23]. One
advantage of using the local shape descriptor is its support
of partial matching.
A few works exist in a 2D-input and 3D-output retrieval
style. One of the earliest 3D model retrieval methods with 2D
input was proposed in [15], in which every stored 3D model is
preprocessed into 13 2D orthographic views. Then 3D models
are retrieved by matching the user’s sketches to the 2D images
of 13 views. Using a fixed number of representative views
regardless of the model complexity, however, is obviously not
optimal. Pu et al. [28] proposed a sketch-based retrieval that
requires a user to draw three orthogonal 2D views of a 3D
model as input. However, it is difficult for a user to sketch three
orthogonal drawings consistently as in a formal engineering
drawing. Wang et al. [32] proposed a method that retrieves
3D CAD models using both geometric (called 2D outline) and
topological (called 2D skeleton) information. This approach
requires users to provide a skeleton sketch and three 3D outline
sketches, and thus suffers from the same inflexibility as in [28].
In this paper, we propose a 3D CAD model retrieval method
in which a user can freely sketch any shapes in a single line
drawing form (i.e., it does not need three consistent sketches
from orthogonal views as input) and thus offer a more flexible
and useful way for retrieval.
Since we use pen-based free-form sketches for 3D CAD
model retrieval, feature extraction and representation for
sketched shape are important. Here we draw attention to
other retrieval applications which also use sketch input. As a
natural and concise visual form of 2D shape, sketches have
been used in image and video retrieval for many years. A
representative work in [6] models users’ sketches as closed
silhouettes and matches them to the edges in the image using
a similarity measure defined by the degree of matching and
the elastic deformation energy. Chalechale et al. [8] proposed
to use an angular-spatial distribution of pixels in the abstract
images (akin to the sketches used in this paper) as a compact
and effective feature for a sketch-based image matching.
Similar to the angular partitioning of abstract images, the
shape context descriptor proposed in [4] utilized a histogram
in log-polar space that actually leads to a histogram with
partitioning along the angular and radial directions in the
image space. Inspired by the work in [4], [8], [26], in this
paper we propose to extract features in sketches for shape
matching using a radial-partitioning-based histogram.
The rest of the paper is organized as follows. The feature
representation of pen-based sketches and corresponding shape
matching mechanism is proposed in Section III. Given the