Res Eng Design (2009) 20:7996 DOI 10.

1007/s00163-008-0054-1

ORIGINAL PAPER

Reverse Engineering and restyling of aesthetic products based on sketches interpretation

Maura Mengoni Michele Germani

Received: 4 May 2008 / Accepted: 11 August 2008 / Published online: 19 November 2008 Springer-Verlag London Limited 2008

Abstract In the conceptual design stage, outcomes of industrial designers work are generally represented by set of sketches where curves, notes, shadows, and colors implicitly represent creative ideas. Signs and annotations are used to synthesize and concretize the design intent that, nally, will be transformed into the styling product visual appearance. The loss of the original design intent may be due to the complexity of the design process, and the involvement of different actors. Our aim is to provide a method and relative tools in order to interpret signs on sketches for eliciting the design intent. The analysis result is a set of aesthetic features that can be used for driving CAD modeling, in the case both of Reverse Engineering applications and of product modeling for restyling purposes. Sketches analysis is based on a semiotic interpretation driven by the formalization of the cognitive models used in the conceptual design phase. The approach showed promising results on different styling products test cases. Keywords Styling products 2D sketches Semiotic analysis Cognitive models Reverse Engineering

Product design, in broad sense, is associated with a set of human activities aimed at developing an artifact; from product shape ideation to product manufacture till maintenance and other after-sale services. Restricting the interest to the phases strictly linked to styling design, they typically include (but not are limited to): analysis of customers taste and needs. It includes the study of socio-cultural trends in order to identify the target market; translation of the customers needs into marketing specications; creation of the design concept (denition of the aesthetic and functional product features); virtual and physical prototyping of alternatives solutions; analysis of technical and manufacturing feasibility implying the choice of materials, colors, technological solutions and, simultaneously, the identication of possible manufacturing processes; embodiment design and detailed design.

1 Introduction The word design has at least three important meanings: as a process, as an object, and a discipline (Lawson 2006).

M. Mengoni M. Germani (&) Department of Mechanics, Faculty of Engineering, Polytechnic University of Marche, Via Brecce Bianche, 60131 Ancona, Italy e-mail: m.germani@univpm.it URL: http://www.dipmec.univpm.it

The growing complexity of modern products requires an increasing degree of specialization to manage the whole design process. The product design has to be realized and evaluated involving several viewpoints and disciplines. Industrial designers generally interact with marketing departments, product engineers, and manufacturing engineers for achieving the nal design solution. In this collaborative process, communication problems deriving from different technical backgrounds and individuals experience can emerge. This is particularly evident when industrial designers and product engineers interact. The difcult to effectively transmit the designers creative ideas to other process actors results in a conict

123

80

Res Eng Design (2009) 20:7996

that generates a high number of design iterations and aesthetic errors. In order to overcome the above-mentioned problems, we state that it is necessary to formalize and, hence, preserve designers intentions from the conceptual phases to the following engineering developments. We observed that communication problems are particularly critical in two different design situations: when the designer realizes a physical prototype and Reverse Engineering (RE) techniques are applied to obtain its digital representation, and when a product is subjected to restyling (RS) in order to satisfy new emerging additional requirements. In both situations, designers generally conceive the ideal product shape by realizing sketches or physical models and CAD (Computer Aided Design) operators belonging to the engineering department interpret the designers intentions in order to develop the digital model. Being forced to save time, they make autonomous 3D modeling choices taking into account only the easier CAD modeling strategy, and neglecting which shape features must remain unchanged and must be modied according to designers constraints. Errors are emphasized if the product shape is characterized by a complex freeform surface. The goal of our research is the development of a computational tool for improving communication, avoiding design intent misrepresentations, and, preserving it during the whole design process. This goal can be achieved by two main activities: 1. the denition of a method for the formalization of the design intent based on the recognition and extraction of the aesthetic features on the cloud of points (RE process; the cloud of points is the set of points acquired by a 3D scanning system) or on the CAD model (RS process). The method aims at dening the optimal design intent oriented CAD modeling strategies; the study and development of rules and algorithms that allow the future implementation of proposed method within a CAD system.

2.

In this work, we describe the approach and its experimentation. The current process is not automatic as human steering remains essential to interpret signs on sketches (signs can be considered as lines, curves, words, etc., representing ideas on paper and they are collected into sketches) and to identify the CAD modeling strategy. The future method implementation will result into software tool for automatically obtaining a 3D skeleton from 2D sketches by a limited humans interaction and for automatically realizing 3D CAD models based on design intent analysis. We dene the design intent as the set of the design values represented by aesthetic features, and that must remain unchanged throughout the whole product development. Aesthetic features are conceived by the designer in

the rst conceptual phase. From an operational point of view, we dene an aesthetic feature as a parametric description of a shape, containing the styling curves, the set of parameters and attributes that dene them, and the aesthetic constraints that allow modeling the shape according to the idea generation process. Styling curves are the curves that characterize the aesthetic identity of product. Aesthetic features and their use in surface modeling can enable users to coherently and easily modify the model in the following product development stages. As sketches are the favorite channels of design intent communication through the design process, we state that it is important to study the creative process by analyzing sketches evolution from a cognitive and a semiotic perspective in order to identify the coding/decoding rules to link the design intent with the product shape attributes, to highlight signs representing the styling curves, to formalize the design intent and to translate it in the aesthetic features. The proposed approach is illustrated by the restyling of the Intervista chair designed by Lella and Massimo Vignelli and produced by Poltrona Frau. This particular case study is chosen, because it requires rst the application of RE technique to obtain a digital model of the existing product, and then the RS in order to answer to new market requirements. The paper is structured as follows: rst, we analyze the RE and the RS processes to identify the common problems running into the validation phase, and how the proposed method can overcome them. The denition of aesthetic features and how CAD systems can manage them are provided in the following section. Then, in order to track the styling design process, we study the cognitive models of design and the interrelation between the act of creative designing and sketching. In Sect. 6, we describe the general approach and the related method to support aesthetic features recognition on the product digital model, CAD model surfaces reconstruction and the denition of the geometrical constraints to manage shape modications. Finally, test cases and the experimental results are presented in order to evaluate and discuss the potential timesaving and product quality improvement obtained by the proposed method application.

2 Reverse Engineering and restyling processes Despite the large diffusion of CAD systems in all productive elds, industrial designers still prefer to follow the practice that the creative idea, conceived during the conceptual design phase, can be represented with the necessary freedom only by hand-made sketches (Van Dijk and Mayer 1997). The sketched idea, then, is usually converted into a

123

Res Eng Design (2009) 20:7996

81

physical prototype. Designers prefer to realize full-scale hand-made physical mock-ups that match the impressions and emotions indicated by the corresponding sketches on paper. Depending on the size of object, they sometimes realize reduced scale prototypes (e.g., automotive or architectural design). Prototypes are made to concretize the ideas represented by drawings, to evaluate the spatial effects of complex topological shapes and to assess the nal product shape. The RE process becomes fundamental to obtain usable 3D digital models for embodiment and detail design. The RE process can be subdivided in the following phases: physical prototype acquisition by using a 3D digitizing system, elaboration of the cloud of points in order to eliminate noisy data and prepare it for the following operations, surfaces reconstruction to dene the exact geometry of the digital model, product engineering to meet production and functional requirements, and, nally, virtual product model validation performed by the whole design team. Surfaces reconstruction errors, mainly aesthetic errors, are due to the poor quality of initial styling information and to the large amount of data that have to be managed, often noisy, sparse or incomplete. The high density of data and/ or the incompleteness can hide aesthetic and functional features to the CAD operator who is forced to make autonomous modeling choices, pursuing the less timeconsuming approach. Furthermore, the CAD model is subsequently modied in order to satisfy technical, functional and manufacturing requirements. This generally causes several process iterations, in order to preserve the initial designers intentions, during the validation phase of product model. Therefore, only if the reconstructed model is coherent with the design intent, the validation phase can be successful. We argue that the process iterations can be avoided if the meaningful aesthetic properties of physical prototype can be extracted from the cloud of points. It is worth to notice that in the design development another critical process is the restyling of existing products. When a product, which gained success in the market and represents the company brand, needs to be innovated to meet new emerging market tastes, it can improved in terms of additional functions, less expensive manufacturing processes or innovative materials, of eco-sustainability, of compliance with new normative standards or with a more attractive style. As current research (Norman 2004) has demonstrated, emotions play a decisive role in the customer decisionmaking process. The preservation of the design intent depends on the maintenance of those aesthetic properties that have previously appealed to consumers emotions. RS can be performed by the designer or by the engineering staff of the company manufacturing the product itself. In

the second case, the company staff makes individual choices in the RS; the design intent may be lost due to the difculty to identify the design intent, and modify the product shape according to it. As a consequence the success of restyled product fails. Restyling generally starts by the elaboration of a 3D CAD model. When the company does not have the original CAD model, or when the model lacks the necessary information to allow aesthetic features recognition (e.g., the model has been stored in neutral formats, such as IGES, STEP, STL, etc.), the RS process is not a trivial task. In the rst case, RE process is applied to acquire the original product shape, in the second the CAD model has to be elaborated in order to extract styling curves, modeling strategies and design parameters. We argue that the analogy between RE and RS consists in the common properties of the cloud of points, and of the CAD model that must be elaborated in RS. Both of them implicitly contain the aesthetic information that must be extracted in order to manage design modications, while the design intent is preserved. As a consequence, in this paper we do not distinguish between the rst and the second type of virtual models; we generally speak of 3D data sets.

3 CAD modeling based on aesthetic features Nowadays, several design activities, from conceptual design to detail design, are supported by CAD systems that enable the creation of digital models from 3D datasets by using strategies based on features. The research on features can be roughly subdivided into two main topics: Feature Recognition and Design by Feature. This distinction can be applied also to the freeform features (Fontana et al. 2000; Langerak and Vergeest 2006). If this denition is associated to aesthetic products with complex freeform shapes, they are called aesthetic features highlighting the stylistic elements they embody (Germani and Mandorli 2004). Design by Feature approach can support the creation of aesthetic shapes by providing designers with features reecting the way of product shape generation. Available computational tools are not yet adequate to model free form shapes and intuitively control them in the rst conceptual phase (Sequin 2005). Feature Recognition approach is more useful to manage RE and RS problems, because it allows the extraction of those features that must be preserved during the whole design process. In this case, it is possible to use available CAD modeling rules to recognize the styling curves on the cloud of points, t surfaces and set proper design parameters (Ke et al. 2006).

123

82

Res Eng Design (2009) 20:7996

Methods for aesthetic features recognition are a widely explored topic in the CAS/CAID (Computer Aided Styling/ Computer Aided Industrial Design) research community. Thompson et al. (1999) proposed and implemented a method to recognize features from digitized data, but their application was limited to regular features. Recently, Langerak and Vergeest (2007) studied an approach for recognizing free form features by using two different methods: template matching and feature line detection. The rst method uses shape matching to nd regions on the digitized data that resemble the shape of a feature shape template. They establish a set of independent template parameters to progressively search the shape similarity between a shape and an instance of template. The second approach is based on a slicing strategy to extract 2D proles by taking as an input a feature library that is assumed but not dened. The method automatically performs RE activities, but does not specify how the user chooses the intersection planes orientation to extract features. As mentioned before, Ke et al. (2006) proposed a possible RE approach based on a 2D sketch space and traditional surface modeling rules such as extruding, revolving and sweeping the 3D proles. In order to extract the layered sectional proles, their research method applies the slicing technique to the unorganized cloud of points, then estimates discrete circular curvatures and derivatives the points contained in the interested sectional proles. The recognition of the slicing planes orientation, and the classication of the aesthetic features are driven only by human expertise. Consequently, we argue that if the expertise is not formalized, the choice and the identication of the aesthetic proles depend only on the sensibility of CAD experts. A further study in this direction is proposed for the creation of a framework for design intent management based on precedents reuse. The method involves features recognition in the freeform domain in order to obtain a freeform shape parameterization according to the design intent (Vergeest et al. 2006). In a previous research work a 3D problem is simplied into a 2D sketch approach (Germani and Mandorli 2004). The functional denition of the product allows identifying planes and 2D curves characterizing the aesthetic regions of the model shape. The dependency from the user makes the method scarcely structured to objectify the design intent recognition on 3D datasets. On the other side we have observed that not only the styling curves extraction is useful for the reconstruction process, but also the identication of the proper modeling strategy to t surfaces and the corresponding design parameters (Mengoni et al. 2006, 2007). It is worth noticing that in order to preserve the design intent, the modeling strategy would be reect the process of idea generation as it affects the way of free from shape

modication during the following engineering phases (Cheutet et al. 2004).

4 Cognitive models of design to track sketches evolutionary process We have adopted well-known models of design, based on multimodal perceptual representation and diagrammatic reasoning, to track the creative design process in order to correlate design meanings and the geometric attributes of shapes, and to recognize the descriptive models of idea generation by which designer conceived the product shape. Design is a complex mental process that takes place in the designers mind. It may be represented by a sequence of distinct and identiable activities that does not necessarily occur in a logical order; problem and solution often emerge together. Many attempts have been made to objectify the design process. Despite the diversity of nomenclature proposed in literature (Dorst and Cross 2001; Gero and Kannengiesser 2004), certain major activities begin to crystallize. These included: Formulation implies the problem space framing and the denition of design requirements; Synthesis refers to the act of moving from the problem space to the solution space. It is the activity that mainly characterized the creative design process; Representation consists of the act of externalizing the conceived shape. Representation and Synthesis are strictly interconnected: drawing affects synthesis performance as it stimulates emergence and reinterpretation. Drawing is not the only mean of design outcomes representation, but one of the most important in the rst conceptual phase; Evaluation is the rst decision-making process in which the designer elaborates a judgment by comparing the achieved solution with design requirements and formulates possible changes.

This model is less linear that it seems; continuous interactions between all activities may lead to an acceptable solution. Cognitive science provides a novel standpoint to analyze the design process. It attempts to uncover the mechanisms through which designers achieve simple tasks for each mentioned activity by processing a large amount of information (previous case studies, internal and external constraints, articial and natural word images coming from different domains, etc.) Chandrasekaran (1999) proposed a cognitive model of design, drawing a parallel between thought and perception. He aimed at understanding the transition from formulation

123

Res Eng Design (2009) 20:7996

83

to synthesis. He proposed a computational framework made of two different levels, the perceptual and conceptual ones, containing logical structures and mental images; ideas are externalized by designers through free-hand sketches and textual notes. Multimodal perceptual representation and diagrammatic reasoning is a cognitive model that views a cognitive state as an integrated and interlinked collection of images in various modalities: the perceptual and the conceptual ones. Thinking, problem solving, reasoning, etc., are viewed as sequences of such states, where there is no an intrinsically preferred mode. The perceptual level consists of several free-hand sketches as design elaborations of the external images and previous case studies. That phase structures the design space (domain). In the conceptual level, the designers dene the design values drawing relationships between signs and concepts. By analyzing the evolutionary process of sketches, signs, and annotations contained in each sketch, it is possible to recognize which sketches are realized either in the perceptual or in the conceptual level. By investigating well-known techniques and procedures of creative design, we adopt the ve main descriptive models of idea generation proposed by Cross (1997): combination, mutation, analogy, emergence and rst principles (Fig. 1). The identication of which strategies lash-up in the designers mind, has several implications for CAD modeling. For example, in mutation, Cross underlined the necessity of recognizing which features of existing design must be selected for modication or how the product behavior can affect the deformation process. In analogy, the difculty consists in abstracting the appropriate features of an existing design or natural shape in order to model the new product model.

5 Sketches as traditional means of design intent representation Sketches analysis provides useful data for design intent formalization in term of adopted creative strategies and styling curves. Sketches, as cited above, are a set of free-hand 2D signs, and annotations correlated to the creative mental processes of the designer. They support formal and functional reasoning, and they are the preferred media to communicate ideas (Schutze et al. 2003). Ferguson (1992) classies three kinds of sketches: the thinking sketch, the talking sketch, and the prescriptive sketch (Fig. 2). Thinking sketches refer to the designers making use of the drawing surface in support of their individual thinking

processes. They are essential, abstract, vague, and diagrammatic. The industrial designer identies clues that can be used to form and inform emerging design concepts. He/ she creates series of rapid sketches to generate images in his mind, to elaborate case studies and to adapt them to the specic design problem. Their ambiguity stimulates reinterpretation. They are characterized by monochrome line drawing without shading or colors and have a uniform thickness. Talking sketches refer to designers making use of the (shared) drawing surface in support of the design team discussion. Suwa and Tversky (1997) claim that designers draw to externalize their concepts, and that drawings provide visual cues for revision and renement of ideas. Talking sketches, in some cases, can be generated as consequent transformation of the rst ones. The designer modies the thinking sketches by adding, deleting or varying the initial curves drawn on paper. He/she generally varies the line thickness of the sketched curves, highlights the lines that better match his/her ideas, while modifying the others. These types of drawings may include brief annotations such as the main dimensions of the product parts. Prescriptive sketches are used by designers to communicate design decisions to all persons that are outside of the creative process. They are detailed and measured drawings. They contain all the necessary dimensions and annotations that allow the creation of physical prototypes and digital models of the product design. A further class, dened by Ullman et al. (1990), is the storing sketches. They refer to the designers using the drawing surface to archive design ideas for their own future reference. Storing sketches have much in common with prescriptive ones for the amount of annotations and graphic signs, drawn by the designer to x the most important features in his/her mind, but are as essential as the talking sketches. They both freeze, rather than develop, design ideas. They are intended for retaining information, whereas prescriptive sketches for communicating information. The evolution from thinking sketches to prescriptive ones is explained by Goel (1995); sketches successively undergo two different types of transformation, lateral and vertical transformation. In a lateral transformation, movement is from one idea to a slightly different idea. In a vertical transformation, movement is from one idea to a more detailed and exacting version of the same idea. An obvious change in thinking (divergence) is a lateral transformation, while if the change is instead to a more detailed version of the same idea then a vertical transformation (convergence) has occurred. (Rodgers et al. 2000). Lateral transformation stimulates emergence, while vertical transformation inspires reinterpretation. Sketches are thus central to the phenomena of emergence, and

123

84

Res Eng Design (2009) 20:7996

Fig. 1 The ve descriptive models of creative design

reinterpretation during early design activity. Emergence refers to new thoughts and ideas that could not be anticipated or planned before sketching. Reinterpretation refers to the ability to transform, develop and generate new images in the mind while sketching. There is considerable evidence to suggest that the production of design ideas appears to depend heavily on this interaction with sketches (Kavakli and Gero 2001; Tovey et al. 2003). Thus, also the evolutionary process of sketching has to be taken into account considering every phase of its path. By analyzing the design process of several test cases from both cognitive and semiotic perspectives, we have observed that thinking and storing sketches are generally used in the perceptual level as they are essential, abstract and stimulate lateral transformation while talking and prescriptive sketches are realized in the conceptual level as

they are more detailed, contain much more information (annotations, textual notes, graphic symbols) than the rst ones. In our work, we focus the attention on prescriptive sketches containing more analyzable information from a computational point of view. The other classes of sketches are arbitrary, vague, incomplete due to the nature of the mechanisms through which designers form their own ideas (Gross 1994). Prescriptive sketches resume the concepts, elaborated and externalized in previous sketches, in an organic way. Thus, thinking and talking sketches can be a very useful support to analyze prescriptive ones in order to track the creative design process, extract 2D styling curves, recognize the adopted idea generation strategies and nally, link shape attributes with design values.

123

Res Eng Design (2009) 20:7996

85

Fig. 2 Example of the three classes of sketches correlated to the increasing level of detail

6 Sketch-based RE and RS modeling: approach overview On the basis of the previous considerations, our approach adopts two different standpoints: semiotic and cognitive. (Fig. 3). The rst is used: (1) to interpret the structure of sketches, (2) to understand how product ideation can be conceived as an act of signication, how creative outcomes are affected by the interpretations of communicating
Fig. 3 Schemata of the proposed approach for feature recognition

actors, how concepts are linked to signs on paper, (3) to identify the generation/synthesis strategies by analyzing graphic signs and annotations, (4) to recognize the styling curves on prescriptive sketches and nally, (5) to study the different channels of ideas transmission. The result of the semiotic analysis consists in the 2D interpretative schemata of the prescriptive sketches that allow highlighting the meaningful styling curves successively translated into the 3D styling curves useful to create the digital model of the product shape.

123

86

Res Eng Design (2009) 20:7996

On the other side, the cognitive perspective allows the recognition of the ways of idea generation and of the link between graphics signs on paper and design attributes. The structure of the design process into perceptual and conceptual levels provides a tool to achieve these tasks. Extracted data are then elaborated to identify the proper set of CAD modeling strategies to t surfaces from 3D datasets and nally, the parameters and constraints that allow managing shape modications while preserving the design intent. From an operational point of view, our approach starts from the denition of aesthetic features as the set of shape attributes determining style, and of their role in transmitting and understanding the message implicitly contained in the product shape. The design intent is generated by applying shape generating strategies, geometrical and functional relationships between forms, and by repeatedly using a set of prominent forms. This can translated in the set of aesthetic features that remain unchanged throughout the product development process: the styling curves, the relationships between them (aesthetic constraints), and the strategies to combine them in tting surfaces. Free-hand sketches have a central role in the identication of the aesthetic information. Both thinking and prescriptive sketches are sets of implicit and explicit features. In particular, explicit features are expressed as annotations, generally contained mainly in the prescriptive sketches. They may specify functional conditions (geometrical and dimensional) and generic product attributes (color, material, number of interfaces, etc.). Implicit features are the translation of creative and functional concepts in graphic signs in the hand made sketches. These signs contain both the styling curves, meaningful for the shape denition, and the main specications to understand and computationally replicate the process of form generation.

7.1 Extraction of the styling curves from 3D datasets We briey summarize the main steps to be carried out for the extraction of the styling curves from free-hand sketches. It is worth noticing that different types of drawings are associated with different stages of the design process. As there is an intimate relationship between the process of cognition, the act of sketching and the ways of ideas representation on paper, the formalization of the creative process starts with the analysis of free-hand sketches evolution. By observing several hand-made sketches realized in different stages of the design process, we deduce that: different free hand sketches reproduce the designer ideas as they evolve in his/her mind; prescriptive sketches contain all information necessary for physical and virtual prototyping as they are used to represent the nal design solution. They can be considered the outcome of the vertical transformation of some thinking sketches where the designer has partially elaborated the design solution or he/she has achieved some meaningful form curves (styling lines) that will be strengthened in subsequent drawings. This cognitive state of transformation corresponds to a continuous growing of complexity and detail from thinking sketches to prescriptive ones; free hand sketches implicitly represent the designers intentions; form curves appear in all different types of drawings, they are modied during the design evolution. The designer generally increases the thickness of styling curves that better concretize his/her ideas; freehand sketches contain textual annotations to communicate and clarify design contents; prescriptive sketches differ from thinking and talking sketches for the increased level of details; the visual assessment of sketches evolution and their comparison show that there are some lines that remain unvaried. They are the styling lines as dened by Tovey et al. (2003). These invariant lines are all articulated in the nal sketches, while in the early drawings they do not contemporary appear. They may occur in different stages of the design process as soon as the design ideas lash-up in the designers mind; prescriptive sketches realized on orthographic/cross sectioning views are the outcomes of the design reasoning activities. Therefore, they represent the result of the bedding and joining of meaningful signs drawn in previous sketches;

7 The proposed method: steps and tools The different steps that lead to styling curves extraction and to modeling strategy denition are: extraction of the styling curves from 3D datasets; creation of a 3D skeleton of styling curves and grouping them according to the recognized ways of design idea generation; translation of the idea generation processes into a proper set of surface modeling strategies and identication of the design parameters and constraints to manage design modications coherently with the design intent.

As a result of design observation, we assume the styling lines as the invariant elements in the evolution of the free-

123

Res Eng Design (2009) 20:7996

87

hand sketches. Design intent formalization starts by searching these invariant lines, whose articulation generates the interpretative schemata of 3D datasets. From these general considerations, styling curves extraction is performed in accord with the following steps: 1. measurement of similarity between early sketches and prescriptive ones. Chalechale method (Chalechale et al. 2005), based on angular partitioning of two images, is adopted to extract the invariant curves between the rst diagrammatic sketches, and the detailed ones that are all scaled according to the 3D dataset dimensions. Articulation of the achieved invariant curves in the same reference plane generates the interpretative schemata. The number of interpretative schemata is the same of the prescriptive drawings. Textual notes and graphic symbols play a crucial role in right scaling and positioning free-hand sketches for similarity measurement. The objective is to extract invariant curves and interpretative schemata with the same dimensions of the shape in the 3D CAD modeling environment. As prescriptive drawings contain annotations about the overall dimensions of the product and the reference view to which they are related, they respectively allow sizing all sketches in accord with the 3D datasets measure, comparing early and detailed sketches and recognizing the reference planes in the 3D CAD environment (Fig. 4); replication of the interpretative schemata in the 3D CAD modeling environment. The extracted interpretative schemata are positioned in the corresponding plane in the 3D dataset and replicated as B_Spline curves. Each B_Spline curve is characterized by parameters such as the position, the curvature and tangency of the control points. styling curves extraction. Each 2D B_Spline curve is projected on the 3D dataset along perpendicular

directions to the identied reference planes. In case of RE application, the projection of the interpretative schemata detects a set of points that are imposed as the control vertices of the 3D styling curves. On the other hand, in case of RS, it detects a curve that is already the searched styling curve. The result of this step is a set of styling curves on the 3D dataset. Styling curves extraction can be also successful in the case of non-complete denition of form lines on paper. The method actually provides a tool to support CAD experts in surface reconstruction from 3D datasets by recognizing meaningful lines to realize exible digital models. During the validation phase, the extracted styling curves can be varied in order to improve surface quality: constraints and parameters guarantee the coherence of modications in respect with the design values dened in the early stages of the design process and, only subsequently formalized by aesthetic features. 7.2 Creation of the 3D skeleton Characteristic curves and character curves on 3D models are meaningful geometric elements for aesthetic features denition. The rst are tangible curves, such as boundary edges, internal edges and llets edges. The second are only visually perceived. They are the styling curves meaningful for conceiving the 3D product model. Furthermore, there is another type of curve that is important for surface modeling, but without a specic aesthetic meaning; they are the cross-sectioning curves. They can be extracted by sectioning the 3D dataset with perpendicular planes to character curves. They are low-level geometric elements that are related to the styling curves. They allow the direct control of the product shape. They are particularly useful when the projected schemata do not perfectly coincide with the visually perceived styling curves on the 3D dataset as

2.

3.

Fig. 4 Searching for invariant curves (interpretative schemata) in the Intervista product. The early sketches are compared with the detailed ones in order to recognize the styling curves on the 3D

dataset. The interpretative schemata are extracted in the same reference planes of the prescriptive sketches

123

88

Res Eng Design (2009) 20:7996

they provide a more detailed scanning of the 3D dataset surfaces. The lack of the correspondence can be due to small errors that may occur in the extraction of the interpretative schemata from prescriptive sketches. The 3D skeleton results from the extraction and elaboration of styling curves, characteristic and cross-sectioning curves. All curves are smoothed in order to avoid noisy data and reach the required degree of surfaces quality. For complex free form shapes, all extracted curves are grouped into smaller sets of curves in order to allow the easily management of surface modeling. Each group represents different aesthetic and functional parts of the product model. Free-hand sketches are also useful to perform grouping curves, for example, designers draw arrows to indicate relationships or movements of product parts, in this way the main functions and instructions for product use are dened. It is possible to identify the main functional parts of the product. Designers usually label design concepts and parts names in their drawings. Moreover, the use colors in sketches allows the identication of parts with different aesthetic properties. The extracted curves must also be grouped in accord with the identied modeling strategies; it is necessary to point out which curves belong to each strategy, and their role in it. As each modeling strategy corresponds to a different way of idea generation, every group of curves reect the aesthetic and functional properties of the product design. Therefore, the grouping of 3D skeleton curves in accord with the modeling strategies is coherent with the design intent (Fig. 5). 7.3 Translation of the shape generation process into the set of modeling strategies Our assumption is that the result of the parametric surface modeling and the coherence of CAD models modications in respect with the design intent, strictly depend on the type of strategy adopted to construct surfaces and on the
Fig. 5 Projection of the interpretative schemata on the 3D dataset (left) and the 3D skeleton resulted from the extraction of the styling curves, characteristic and crosssectioning curves from the 3D dataset (right)

hierarchical chain of operations used to the whole surface model. In the context of freeform shapes, two prescriptive models of creative design proposed by Cross (1997), are representative of organic freeform shapes: mutation and analogy. Mutation can be realized by applying different transformation actions: fusion or melting: when two or more forms are combined and their edges are strictly interconnected; it is not possible to distinguish when the rst ends and the second begins. The product form is the result of the interaction between the original shapes; global deformation: when a form is modied by applying forces along different directions and with different weights; morphing-like deformation: when a shape is generated based on weighted average of two other existing shapes.

In all mutation strategies, the problem regards with the denition of which surface entities are subjected to deformation and of which constraints drive the deformation. The choice of the rsts inuences the second. Catalano et al. (2004) classied freeform features by dening three different types of deformations and related constraints: pointdriven deformation, curve-driven deformation, surface-driven deformation. In order to apply deformation operations, it is important to translate the mutation strategy into CAD modeling operators and for each operator to identify which parameters and constraints must be used to t surfaces. In creative and conceptual design, designers often look to books, magazines, and other collection of images to nd forms they can adopt and adapt in designs. They use referencesimages of natural and articial world from rocks and owers to boats and buildings, internal images that come from their experience of realityas visual analogies and metaphors. All images are arranged in different ways according to the design values they aim at communicating through the product shape.

123

Res Eng Design (2009) 20:7996

89

Among creative strategies, similarity-based or analogy-based creativity attracts most research interest recognizing the key role of the analogical reasoning in creative design (Goel 1995). It is a key point of the idea generation as demonstrated by several works of contemporary designers where analogical reasoning involves accessing and transferring elements from familiar categories to use it in constructing a novel product design. (e.g., Karim Rashid, Future Systems, Frank OGehry, Michael Graves, Massimiliano Fuksas, Philippe Starck, etc.). The understanding of the similarity-based mechanisms claries the importance of associations for creative thinking. Creativity intensively makes use of concepts kept in memory and bound by associative relations. The nature of these relations is neither logical nor mathematical, but perceptual and experiential (Hofstadter 2001). As analogy emerges as a relation of similarity between two existing shapes, features or entities stored in long-term memory, or rather between source and target (Goldschmidt 2001), a possible classication of analogical reasoning strategies starts from the identication of the terms of similarity: analogy between forms or features; analogy between opened or closed sections (entities).

In the rst case, analogy can be created by morphing two forms: the rst can be retrieved by a 3D library where the designer collects interesting forms, case studies and previous works; the second can be a primitive shape or a low elaborated shape. The problem deals with the denition of the relationships between geometrical properties of the initial and the target forms, key reference points or curves. This generation process is similar to the morphinglike deformation process in mutation. In the second case, when two forms are similar in the morphology of some section curves, it is important to identify the process through surfaces have been generated from these sections: process of sliding of one section along another, or process of translation, rotation, extrusion along a direction; process of joining and connecting multiple sections.

In the context of freeform shapes, the similarity is set between natural forms (i.e., organic shapes) or by applying form generation processes deriving from nature. Existing feature-based CAD tools difculty support freeform features design by providing functionalities able to apply the above-mentioned generation processes, because they allow only the transformation of existing surfaces by arbitrary changing the position of their control points. It is difcult to identify proper CAD modeling strategies to t surfaces reecting creative processes and the

hierarchical chain of operations for whole product modeling, entities and parameters that can be modied and the constraints that can be used to express the aesthetic properties of shape. Tables 1 and 2 try to map the ways of idea generation with available CAD modeling strategies. The rst map identies creative strategies within available CAD modeling functions, while the second one provides a list of geometric entities, parameters and constraints that can be set for each CAD modeling strategy encompassing the corresponding creative technique. The proposed CAD modeling strategies can be found in different feature-based CAD systems, and is not limited to the adopted technology for method validation. Free-hand sketches evolution analyses may be useful to understand designers intentions and the ways of free-form shapes generation. For example, designers draw arrows and lines to indicate the main directions of the orthographic and sectional views, the main movements or relationships between product parts. They also draw lines characterized by conventional styles to indicate symmetry axes, parallel planes, perpendicular lines, etc., but also trajectories of sliding, of translation, etc. They use to underline key reference points or curves. Designers, who prefer aesthetic proportions and balance, usually sketch lines to form grids that may constitute geometrical constraints for the following engineering developments. The use of big arrows generally means the weight of forces applied for shape deformation. Designers usually write words, stick images, make collages, in order to recall precedents and meaningful reference images that can be useful for similarity entities recovery. Finally, designers write numbers to prescribe the overall dimensions and to identify product parts interfaces; this data is necessary not only for scaling sketches according to the 3D datasets dimensions in order to extract the interpretative schemata, but also to identify parameters and constraints for coherent shape modications, such as position parameters, dimensions, angles of revolution, etc. Once the styling curves have been extracted and grouped, modeling strategies choice starts from the analysis of sketches evolution in order to search for which geometric entities should be engaged in tting surfaces. For example, if a revolution axis and a styling curve nearby is identied in the free-hand sketches, the revolution sweep strategy should be adopted, or if the product shape results from the deformation of a grid of curves, the grid strategy should be applied. Possible parameters and constraints have been dened for managing modications (Table 2), so the whole parametric model is achieved by identifying the proper chain of CAD operators (Fig. 6). While the extraction of styling, characteristic, and crosssectioning curves is quite automatic, the identication of the CAD modeling strategies from sketches evolution is still manual. The proposed method provides a framework

123

90

Res Eng Design (2009) 20:7996

Table 1 Classication of the creative process of free form generation. Mapping with 3D CAD modeling strategies

to support CAD experts to analyze the design intent in order to identify the proper chain of operations to t surfaces and manage the following engineering modications on digital prototypes. Tables 1 and 2 allow mapping strategies with available modeling techniques.

8 Experimental validation In RE context, the validation of the proposed method has been preliminarily carried out using two test cases: a mouse prototype for left-hand users and a telephone design. Each product is characterized by free-form shapes. Small groups of engineering students (groups of 45 members) have been chosen to design each prototype. Students have a practice of RE methods akin to that of designers. This particular experimental condition has been chosen, because it allows the analysis of the whole ideation process in a fully monitored context where the adopted cognitive models of design, the classied strategies of creative design have been tested. During the conceptual design phase, students explored ideas by free-hand sketching on paper searching for a meaningful shape that is

able to represent their personal product concept. After continuous design reviews, the product shape seemed to reect the design values expressed by students both in a verbal manner and in the annotations and graphic signs drawn on paper. Physical models have been realized by sculpturing malleable clay both to actualize design ideas in concrete terms, and to evaluate the spatial effects of the conceived products shape. In order to obtain digital models, students used commercial technologies for digitizing both the mouse and the telephone physical mock-ups acquisitions were done by a contact system (Modela MDX15 by Roland). A 3D CAD system (CATIA v.5.14 by Dassault Systemes) has been chosen to perform the following main steps of the proposed method. The system has been chosen, because it supports simultaneously the feature-based modeling and the parametric surface modeling and it provides tools both for points cloud data elaboration (post-processing data points), for surface modeling and for structural, ergonomic and manufacturing simulations. This CAD system allows furthermore managing attributes to link the semantic contents with the graphical entities. Similarities measure between preliminary sketches and prescriptive ones has been performed. The result is a set of

123

Res Eng Design (2009) 20:7996

91

Table 2 CAD modeling strategies using commercial CAD/CAID systems. Denition of the parameters and constraints to control shape modications for each strategy

interpretative schemata of sketches that consists of 2D styling curves on the same reference planes of the prescriptive sketches. In order to extract the necessary curves for 3D skeleton creation, students followed the following steps: 1. analysis of textual notes contained in the prescriptive sketches to gather the position of reference planes;

2.

3.

identication of the corresponding planes in the digitized data CAD environment in order to position the interpretative schemata and replicated them as B_Spline curves; projection of B_Spline curves on the clouds of points along perpendicular directions to the reference planes. The result is the set of the styling curves;

123

92

Res Eng Design (2009) 20:7996

Fig. 6 Grouping the extracted curves in accord both with the modeling strategies and with the aesthetical and functional properties of the product parts. The parameterization of Intervista surface model. As balance among the product part represent one of the

designers intentions, shape deformation is constrained to a grid that was drawn on the early sketches. The restyling process is due to the need for an improvement of the ergonomic characteristics: enlargement of the dimensions of the seat, lowering of the backrest

4.

extraction of cross-sectioning curves and characteristic curves to obtain a 3D skeleton of each product model to t surfaces.

The analysis of free-hand sketches evolution and the structuring of the whole design process according to the perceptual and conceptual levels proposed by the adopted cognitive standpoint showed that: 1. in the case of the mouse design, analogy strategy has been adopted, creating a relation between different natural forms. Analogical reasoning leads to the identication of some meaningful curves on natural shapes, such as owers and leaves, that are replicated on the particular design context: in nature, visible curves correspond to the structure of the natural element. in the case of the telephone design, mutation has been adopted to conceive the shape. A primitive form (oval shape) was progressively hollowed according to the specied ergonomic requirements: students attempted to facilitate the phones hold. In the product brief was specically explained that key numbers need to be clear visible: the reasoning relationship between the adjective visible and the image of an eye appear in the rst sketches. The hollow followed the shape of a big eye.

2.

By mapping the idea generation techniques with the CAD modeling strategies (Tables 1, 2), surface reconstruction has been performed and aesthetic constraints have been imposed (Figs. 7, 8). On the other side, method validation has been performed on real industrial design cases. The restyling of two

existing products (Intervista chair by Lella e Massimo Vignelli and BIBI seat for Distillerie Nardini theatre by Massimiliano Fuksas) have been developed in collaboration with Poltrona Frau, a worldwide leader in the eld of furniture design and manufacture. Its products are characterized by high aesthetic values. The rst test case is well explained across the paper in order to illustrate the proposed method. The BIBI restyling started from the companys needs to redesign an ofce chair that would preserve the original BIBI design intent; some aesthetic features must remain unchanged while new additional functions appear. The company had the CAD model of the BIBI but in a neutral format (e.g. IGES). The RS was carried out by CATIA v 5.14 that is used by Poltrona Frau technical staff for everyday works. In order to recognize aesthetic features, sketches and preliminary prototypes have been analyzed. The way of idea generation resulted in the combination of three different techniques: analogy with the drop shape, mutation as the melting of the drop shape when it comes in contact with the seating, and rst principles in terms of the structure that determines shape. Some meaningful curves have been simultaneously extracted by similarity measurement. The translation of interpretative schema into 3D styling curves over the BIBI digital model and the application of the identied CAD modeling strategies mapped by using Tables 1 and 2, allowed the creation of a parameterized 3D model that was modied according to new emerging ideas for the ofce chair (Fig. 9). In order to qualify the styling design process improvements by means of the proposed method application,

123

Res Eng Design (2009) 20:7996

93

Fig. 7 The application of the method for the RE of the mouse prototype

Fig. 8 The application of the method for the RE of the telephone prototype

123

94

Res Eng Design (2009) 20:7996

Fig. 9 The BIBI restyling: from the interpretation of sketches and physical prototypes of the existing product to the generation of new design concepts, to the extraction of the aesthetic features and the redesign of the new ofce chair

performance measurement is essential. A list of process metrics has been established to evaluate the method efciency in terms of: 1. time needed for surfaces reconstruction and product validation as we have observed that design iterations and aesthetic errors are mainly due to the difculty of design intent transmission from the designers to CAD experts; number of hours needed for prototyping new physical mock-ups when validation fails and for re-engineering the product model. The design process generally needs both raw prototypes to dene the product aesthetic shape and well-nished prototypes (with real colors, materials, additional functions, etc.) for nal decisionmaking activities. If validation fails, the number of physical prototypes increases. The number of hours for prototyping demonstrates the time to market increase or decrease; hours for CAD model modications after product engineering. During the engineering development, the product digital model is generally modied according to structural, technical, manufacturing requirements.

Modications are carried out by CAD experts with the support of the whole technical staff. If shape variations are not coherent with the design intent, product validation fails. This metric measures how coherent is the nal product shape with the initial designers intentions; In order to measure the adopted metrics, the RE and RS are performed both with the traditional and the proposed method (Table 3). By analyzing the rst validation results we can infer the following considerations: CAD modeling time increases due to the addition of more steps such as the recovery of the sketches, the extraction of the interpretative schemata and the styling curves and the analysis of the creative design process, method application eliminates the need to construct additional physical prototypes when validation is not successful design iterations are reduced.

2.

3.

Globally the new process allowed saving nearby 30% of time for restyling, and 40% for RS.

123

Res Eng Design (2009) 20:7996 Table 3 The results of the validation phase Surfaces reconstruction or CAD modeling time Armchair Intervista Traditional RE/RS process RE/RS performed by the proposed method Mouse prototype Traditional RE process RE performed by the proposed method Telephone prototype Traditional RE process RE performed by the proposed method BIBI chair Traditional RS process 24 12 (2) 6 (1) 12 (6 h for each prototype) 6 2 1 5 2 57 35 6 8 2 (1) 0 4 (2 h for each prototype) 0 0.3 0.15 1 0.5 13.3 8.65 6 12 4 (2) 0 12 (6 h for each prototype) 0 1 0.6 5 2 28 14.6 6 9 10 (2) 5 (1) 12 (6 h for each prototype) 0 1.25 0.5 3 1 32.25 15.5

95

Time for physical Time for surface Validation Number of modeling Total hours prototyping for the reconstruction of time (h) hours for CAD model for product nal approval (number the new mock-ups modications design of physical mock-ups)

RS performed by the proposed 20 method

9 Conclusions The proposed method uses sketches analysis from a cognitive and a semiotic standpoint for interpreting the creative design process and the outcomes of the synthesis activity in order to extract aesthetic features from 3D datasets. The method improves the performance of two different critical design situations: the transformation of the product physical mock-up into a CAD model by applying RE techniques and the RS of existing products when new additional requirements emerge. The main research objective is to guarantee the preservation of the design intent from the rst conceptual design phases to the following engineering developments. The design intent is formalized in terms of styling curves and CAD modeling strategies. The hand-made sketches evolution represents the starting point for aesthetic features identication. A detailed analysis of design models and of sketches role in developing the product design concept is carried out in order to theoretically establish solid basis for the proposed method. Different application examples in the industrial design eld have been described. They have been used to demonstrate the approach applicability in case of free-form shape products. The preliminary validation has shown positive results in terms of increased timesaving, improvements of models surfaces quality, and of the

accuracy and exibility of the obtained CAD models, coherently with the design intent. On the other hand, the approach needs of further methodological and technical improvements. From the methodological point of view, further research work will be concentrated on: a better and robust link between some descriptive models and aesthetic features, between 3D skeleton attributes and CAD modeling strategies, the study of further methods for semi-automatically identifying aesthetic features from sketches analysis, rationalizing the rules to develop algorithms useful for the software implementation into a commercial design system, and, nally, making an extended protocol analysis to validate, and eventually improve the CAD modeling strategies deduced by sketches analysis. From the technical side it is necessary to study and develop the integration of the procedure within a CAD system in order to support CAD operators with a tool for automatic sketch analysis, for aesthetic features recognition and, nally, for automating the right modeling strategies.

References
Catalano CE, Falcidieno B, Giannini F (2004) Introduce sweep features in modeling with subdivision surfaces. J WSCG 12(1):8188

123

96 Chalechale A, Naghdy G, Mertins A (2005) Sketches-based image matching using angular partitioning. IEEE Transactions on Systems, Man and cybernetics: part a: systems and humans 35(1):5766 Chandrasekaran B (1999) Multimodal perceptual representation and design problem solving. In: Proceeding of visual and spatial reasoning in design: computational and cognitive approaches, Cambridge, MA Cheutet V, Catalano CE, Pernot JP, Falcidieno B, Giannini F, Leon JC (2004) 3D Sketching with fully free form deformation features (d-F4) for aesthetic design. In: Hughes JF, Jorge JA (eds) Proceedings of EUROGRAPHICS workshop on sketchbased interfaces and modelling, Grenoble, France, pp 918 Cross N (1997) Descriptive models of creative design: application to an example. Des Stud 18:427455 Dorst K, Cross N (2001) Creativity in the design process: coevolution of problem-solution. Des Stud 22(5):425437 Ferguson ES (1992) Engineering and the minds eye. MIT Press, Cambridge Fontana M, Giannini F, Meirana M (2000) Free form features for aesthetic design. Int J Shape Model 6(2):273302 Germani M, Mandorli F (2004) Aesthetic design: a methodology to preserve the stylist intent using digitised models. In: Horvath I, Xirouchakis P (eds) Proceedings of international symposium on tools and methods of competitive engineering, Lausanne, Switzerland, pp 131141 Gero JS, Kannengiesser U (2004) The situated function-behaviour structure framework. Des Stud 25:372391 Goel V (1995) Sketches of thought. MIT Press, Cambridge Goldschmidt G (2001) Visual analogy: a strategy for design reasoning and learning. In: Eastman CM, McCracken WM, Newstetter WC (eds) Design knowing and learning: cognition in design education. Elsevier, Amsterdam, pp 199220 Gross MD (1994) Recognizing and interpreting diagrams in design. In: Proceedings of advanced visual interfaces, Bari, Italy, pp 88 94 Hofstadter DR (2001) Analogy as the core of cognition. In: Gentner D, Holyoak K, Kokinov B (eds) The analogical mind: perspectives from cognitive science. The MIT Press/Bradford Book, Cambridge, pp 499538 Kavakli M, Gero J (2001) Sketching as mental imagery processing. Des Stud 22(4):347364

Res Eng Design (2009) 20:7996 Ke Y, Fan S, Zhu W, Li A, Liu F, Xiquan S (2006) Feature-based reverse modeling strategies. Comput Aided Des 38:485506 Langerak TR, Vergeest JSM (2006) A new framework for the denition and recognition of free form features. In: Horvath I, Rusak Z (Eds) Proceedings of the International Symposium on Tools and Methods of Competitive Engineering, Ljubljana, Slovenia pp 359370 Langerak TR, Vergeest JSM (2007) Feature recognition of userdened freeform features. Comput Aided Des Appl 4(14):529 538 Lawson B (2006) How designers think. Architectural Press, Oxford Mengoni M, Mandorli F, Germani M (2006) Surface reconstruction method for reverse engineering based on aesthetic knowledge. In: Horvath I, Rusak Z (Eds) Proceedings of the international symposium on tools and methods of competitive engineering, Ljubljana, Slovenia, pp 251262 Mengoni M, Germani M, Mandorli F (2007) Reverse engineering of aesthetic products: use of hand-made sketches for the design intent formalization. J Eng Des 18(5):413435 Norman DA (2004) Emotional design: why we love (or hate) everyday thing. Basic Books, New York Rodgers PA, Green G, McGown A (2000) Using concept sketches to track design process. Des Stud 21:451464 Schutze M, Sachse P, Romer A (2003) Support value of sketching in the design process. Res Eng Des 14(2):8997 Sequin CH (2005) CAD tools for aesthetic engineering. Comput Aided Des 37:737750 Suwa M, Tversky B (1997) What architects and students perceive in their sketches: a protocol analysis. Des Stud 18:385403 Thompson WB, Owen JC, De St.Germain HJ, Stark SR, Henderson TC (1999) Feature-based reverse engineering of mechanical parts. IEEE Trans Rob Autom 15(1):5766 Tovey M, Porter S, Newman R (2003) Sketching, concept development and automotive design. Des Stud 24(2):135153 Ullman DG, Wood S, Craig D (1990) The importance of drawing in the mechanical design process. Comput Graph 14(2):263274 Van Dijk CGC, Mayer AAC (1997) Sketch input for conceptual design. Comp Ind 34:125137 Vergeest JSM, Song Y, Langerak TR (2006) Design intent management for design reuse. In: Rohatynski R, Poslednik P (Eds.) Proceedings of the Design Methods for Practice. Zielona Gora, Polonia pp 163170

123