Вы находитесь на странице: 1из 6

Available online at www.sciencedirect.

com
Available online at www.sciencedirect.com
ScienceDirect
Available ScienceDirect
online
Procedia
Available atonline
CIRPwww.sciencedirect.com
00 (2019) 000–000
at www.sciencedirect.com
www.elsevier.com/locate/procedia
ScienceDirect
Procedia CIRP 00 (2019) 000–000
ScienceDirect www.elsevier.com/locate/procedia

Procedia CIRP 00 (2017)


Procedia 000–000
CIRP 81 (2019) 346–351
www.elsevier.com/locate/procedia
52nd CIRP Conference on Manufacturing Systems
52nd CIRP Conference on Manufacturing Systems
An Application of 3D Model Reconstruction and Augmented Reality for
An Application of 3D
Real-Time
28th CIRP Model
Monitoring Reconstruction
Design Conference, May 2018,and
of Additive Augmented
Manufacturing
Nantes, France Reality for
Real-Time Monitoring of Additive Manufacturing
A newAmmar
methodology
Malika*, Hugo to Lhachemi
analyzea,the Joernfunctional
Ploennigsb, Amadou and physical architecture
Bab, Robert Shortena of
existing products
Ammar for an
Malika*,Electrical
Hugo and assembly
Lhachemi
Electronic
a
a
, Joernoriented
Engineering, Ploennigs
University product
b
College ,Dublin,
Amadou family
Dublin, Ba b
identification
, Robert
Ireland Shortena
b
Ireland Research Lab, IBM, Dublin, Ireland
Electrical and Electronic Engineering, University College Dublin, Dublin, Ireland
a

Paul Stief *, Jean-Yves Dantan, Alain Etienne, Ali Siadat


* Corresponding author. E-mail address: ammar.malik@ucdconnect.ie
b
Ireland Research Lab, IBM, Dublin, Ireland

* Corresponding author. E-mail address: ammar.malik@ucdconnect.ie


École Nationale Supérieure d’Arts et Métiers, Arts et Métiers ParisTech, LCFC EA 4495, 4 Rue Augustin Fresnel, Metz 57078, France

Abstract
* Corresponding author. Tel.: +33 3 87 37 54 30; E-mail address: paul.stief@ensam.eu
Abstract
This paper presents a novel scan-based method for the real-time monitoring of additive manufacturing processes. Most traditional scanning
techniques used for generating 3D models capture the only outer shape of the object after completion of the printing process. The method proposed
This paper presents a novel scan-based method for the real-time monitoring of additive manufacturing processes. Most traditional scanning
in this paper differs as it relies on a layer-by-layer scanning of the 3D object directly during the printing process. This strategy has been
Abstract
techniques used for generating 3D models capture the only outer shape of the object after completion of the printing process. The method proposed
successfully implemented with a fused filament 3D printer (PRUSA i3 MK3). Furthermore, in order to offer an increased interaction between the
in this paper differs as it relies on a layer-by-layer scanning of the 3D object directly during the printing process. This strategy has been
Inobtained
today’s 3D
successfully
modelenvironment,
business and the user,the
implemented
a virtual
with a fused trend environment
towards
filament more has
3D printer
beenvariety
product
(PRUSA
developed for the augmented
and customization
i3 MK3). Furthermore, inisorder
reality glasses
unbroken. Due
to offer toHoloLens. The novelty
this development,
an increased
of this
the need
interaction between of
the
method
agile and lies in the layer-by-layer
reconfigurable production3Dsystems
model reconstruction
emerged to of with
cope both various
the outerproducts
shape and and the inner layers
product of the
families. To printedand
design part.optimize
It enables the user,
production
obtained 3D model and the user, a virtual environment has been developed for the augmented reality glasses HoloLens. The novelty of this
directly as
systems during the printing
chooseprocess, to view and detect potential defects, not only at theare surface but Indeed,
also in the inner layers of themethods
printed aim
object.
method lieswell as to
in the the optimal
layer-by-layer 3D model product matches,
reconstruction product
of analysis
both the outer methods
shape and theneeded.
inner layers of most of the
the printed known
part. It enables the to
user,
Therefore,
analyze it can provide detailed information about the build quality and can be used as the basis of a decision-making tool.
directlyaduring
productthe orprinting
one product family
process, on theand
to view physical
detect level. Different
potential defects,product
not onlyfamilies, however,
at the surface butmay
alsodiffer
in thelargely in terms
inner layers of of
thethe number
printed and
object.
nature of components.
Therefore, it can provide This fact impedes
detailed informationan efficient
about thecomparison
build quality andandchoice
can beofused
appropriate product
as the basis family combinations
of a decision-making tool. for the production
© 2019AThe
system. new Authors. Published
methodology by Elsevier
is proposed Ltd. This
to analyze is an open
existing access
products in article
view ofunder
theirthe CC BY-NC-ND
functional license
and physical architecture. The aim is to cluster
© 2019 The Authors. Published by Elsevier Ltd.
(http://creativecommons.org/licenses/by-nc-nd/3.0/)
these
© products
2019 The in new assembly
Authors. Published oriented product
by Elsevier Ltd.families foropen
This is license
an the optimization
access articleofunder
existing
the assembly
CC BY-NC-NDlines and the creation of future reconfigurable
license
This is an
Peer-review open access
under article under
responsibility the scientific
of Flow
the CC BY-NC-ND committee (http://creativecommons.org/licenses/by-nc-nd/3.0/)
ofstructure
the 52ndofCIRP Conference on Manufacturing Systems.
assembly systems. Based on Datum Chain,
(http://creativecommons.org/licenses/by-nc-nd/3.0/) the physical the products is analyzed.
Peer-review under responsibility of the scientific committee of the 52nd CIRP Conference on Manufacturing Systems. Functional subassemblies are identified, and
aPeer-review
functional analysis is performed.
under responsibility ofMoreover, a hybrid
the scientific committeefunctional and physical
of the 52nd architecture
CIRP Conference on graph (HyFPAG)
Manufacturing is the output which depicts the
Systems.
Keywords: Additive Manufacturing; Augmented Reality; Real-Time 3D Model Reconstruction; Real-Time Monitoring.
similarity between product families by providing design support to both, production system planners and product designers. An illustrative
example
Keywords:of Additive
a nail-clipper is used toAugmented
Manufacturing; explain the proposed
Reality; methodology.
Real-Time 3D Model An industrial case
Reconstruction; study Monitoring.
Real-Time on two product families of steering columns of
thyssenkrupp Presta France is then carried out to give a first industrial evaluation of the proposed approach.
© 2017 The Authors. Published by Elsevier B.V.
1. Introduction parameters, such as process parameters (heat source, trajectory
Peer-review under responsibility of the scientific committee of the 28th CIRP Design Conference 2018.
1. Introduction generation,
parameters, such etc.), ambientparameters
as process parameters (heat (temperature
source, trajectory or
Additive manufacturing (AM), also known
Keywords: Assembly; Design method; Family identification as 3D printing, humidity), and intrinsic properties of
generation, etc.), ambient parameters (temperature or the employed materials,
refers to the manufacturing
Additive various processes (AM),of adding
also knowntogether materials
as 3D to
printing, have a strong
humidity), andimpact on the
intrinsic quality of
properties ofthethefinal productmaterials,
employed [7]. Any
create three-dimensional objects based on 3D model
refers to the various processes of adding together materials to files under uncertainty in even one of these parameters
have a strong impact on the quality of the final product [7]. Any might result in
computer control. Originally used for fast prototyping
create three-dimensional objects based on 3D model files under [1-2], defects. For instance, due to the off-line
uncertainty in even one of these parameters might result in generation and
1.AM
Introduction
has emerged of the product range and characteristics manufactured and/or
computer control. as a disruptive
Originally usedtechnology poised to deeply
for fast prototyping [1-2], optimization
defects. For of the printing
instance, due trajectories,
to the off-line the accuracy
generationof andthe
transform manufacturing [3]. However, despite its huge assembled
geometry in
maythis not
system.
reachIn this
the context,
required thelevel
main for
challenge
a in
given
AM has emerged as a disruptive technology poised to deeply optimization of the printing trajectories, the accuracy of the
Due toAMthe
potential, still fast
does development
not meet the in theof conventional
standards domain of modelling
application, andand analysis
because is ofnowthe not only to cope
layer-based withprocess,
printing single
transform manufacturing [3]. However, despite its huge geometry may not reach the required level for a given
communication
manufacturing. andparticular,
In an ongoing trendfrom
it suffers of digitization
low productivityand products, a limited can
inhomogeneities product range yielding
appear, or existingproblems
product families,
in the
potential, AM still does not meet the standards of conventional application, and because of the layer-based printing process,
digitalization,
rates, poor manufacturing
quality and enterprises
repeatability of are facing important
manufacture, uncertain but also to beproperties
mechanical able to analyze
[8]. and to compare products to define
manufacturing. In particular, it suffers from low productivity inhomogeneities can appear, yielding problems in the
challenges
properties inthetoday’s market parts,
environments: a continuing newItproduct families. It can that
be observed that classical existing
rates, poorofquality manufactured
and repeatability etc.manufacture,
of [4]. These limitations
uncertain is widely
mechanical recognized
properties [8]. the development of adequate
tendency
are some towards
of the reduction
reasons that of product
prevent the development
widespread times and
adoption of product
monitoringfamilies
methodsare regrouped
combined inwith
function of clients
efficient or features.
decision-making
properties of the manufactured parts, etc. [4]. These limitations It is widely recognized that the development of adequate
shortened
AM product lifecycles.
technologies with stringentIn addition,
process there is an increasing
requirements (such ofas However,
tools is aassembly
key steporiented
toward product
thewith families are
widespread hardly of
to find.
are some of the reasons that prevent the widespread adoption monitoring methods combined efficient adoption
decision-making AM
demand
aerospace of customization,
[5] and being industries
biomedical at the same timeThe
[6]). in aessential
global On the product
technologies in family
industriallevel,sectors
productspresenting
differ mainly in two
stringent
AM technologies with stringent process requirements (such as tools is a key step toward the widespread adoption of AM
competition
cause of these withproblems
competitors all inherent
is the over the world. This trend, main characteristics: (i) the number of components and (ii) the
aerospace [5] and biomedical industries difficulty
[6]). Thetoessential
model, requirements
technologies [9]. in Recent years
industrial have seen
sectors many applications
presenting stringentin
which
monitor, is and
inducing
control thethedevelopment
underlying fromprocess
AM macro [4]. to micro
Many type
which of components
augmented (e.g.
realitymechanical,
is used electrical,
to enhance electronical).
3D printing
cause of these problems is the inherent difficulty to model, requirements [9]. Recent years have seen many applications in
markets, results in diminished lot sizes due to augmenting Classical methodologies considering mainly single products
monitor, and control the underlying AM process [4]. Many which augmented reality is used to enhance 3D printing
product varieties (high-volume to low-volume production) [1]. or solitary, already existing product families analyze the
2212-8271 © 2019 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license
To cope with this augmenting variety as well as to be able to
(http://creativecommons.org/licenses/by-nc-nd/3.0/)
product structure on a physical level (components level) which
identify
2212-8271 possible
© 2019 The optimization
Authors. Published potentials
by Elsevier in
Ltd. the
This is existing
an open access
Peer-review under responsibility of the scientific committee of the 52nd CIRP Conference causes on difficulties
article under the CC BY-NC-ND
Manufacturing regarding
Systems. license an efficient definition and
(http://creativecommons.org/licenses/by-nc-nd/3.0/)
production system, it is important to have a precise knowledge comparison of different product families. Addressing this
Peer-review under responsibility of the scientific committee of the 52nd CIRP Conference on Manufacturing Systems.
2212-8271 © 2019 The Authors. Published by Elsevier Ltd.
This is an©open
2212-8271 2017access article Published
The Authors. under theby CC BY-NC-ND
Elsevier B.V. license (http://creativecommons.org/licenses/by-nc-nd/3.0/)
Peer-review
Peer-review under
under responsibility
responsibility of scientific
of the the scientific committee
committee of the of theCIRP
28th 52ndDesign
CIRPConference
Conference2018.
on Manufacturing Systems.
10.1016/j.procir.2019.03.060
Ammar Malik et al. / Procedia CIRP 81 (2019) 346–351 347
2 Ammar Malik et al. / Procedia CIRP 00 (2019) 000–000

technology. This paper presents a novel scan-based method for assisted manufacturing systems to achieve reduced cost and
the real-time monitoring of AM processes using 3D model improved quality. For example, in the automotive industry, the
reconstruction and Augmented Reality (AR). The principal 3D models of car interiors have been overlaid on real body
difference between our work and that of other authors is that mock-up cars during the initial phases of development using
image processing, and component models are used to realize a AR [18]. Similarly, a gesture-based AR design environment,
layer-by-layer reconstruction of the printed part. It consists of GARDE, has been developed [19]. In which, using gestures,
a 3D model reconstruction of the printed part based on images the designer can visualize, design and make modifications in a
taken layer-by-layer during the printing process. The resulting 3D model in an AR environment. AR in manufacturing has
3D model can be used either for the monitoring of the printing been mainly focused on the design phase of the manufacturing
process or the detection of internal defects of the final printed process [20,21,22]. However, AR can also play a significant
object. In order to improve the interaction between the user and role in the improvement of process monitoring. For instance, it
the resulting 3D model, an AR environment has been was proposed in [23] to bridge the gap between digital and
developed for the mixed-reality glasses HoloLens*. Note that physical works by projecting the 3D model directly on top of
in this paper, AR is used as a technology brick to allow the user the currently printed part in the printer workspace. In this
to interact with the 3D model, either to visualize the full part or context, the user can control both the displayed 3D model and
to perform layer-by-layer interrogation of the final printed part. the printer actuations toward an integrated design. The idea of
superimposing the 3D model to the currently printed part was
The remainder of the paper is organized as follows. Related also developed in [24] as a comparison tool for fast
works are presented in Section 2. An overview of our prototyping. An automatic image comparison procedure was
experimental equipment is given in Section 3. The proposed developed for the detection of the printing failures by
methodology is detailed in Section 4 and obtained results for a comparing a picture of the part being printed and the original
case study are related in Section 5. Finally, concluding remarks 3D model.
are provided in Section 6. Proposed system: The 3D reconstruction technique
proposed in this paper requires only one camera mounted on
2. Related works the top of the build plate; thus, avoiding the need for multiple
static cameras or moving a single camera around the object.
Geometry reconstruction is a well-studied area of research Also, the proposed scanning technique suggests an automated
in computer graphics and vision. In the past, the 3D geometries cropping of the background from the acquired images, thus
have been reconstructed using passive cameras [10,11], active generating a 3D model of the object only. Since the proposed
sensors [12], online images [13] or from unordered 3D points technique reconstructs the 3D model layer by layer, it can be
to generate watertight surfaces [14]. Traditional 3D model used for real-time monitoring of the build to detect the defects
reconstruction techniques can be generally classified as on the surface as well as within the object. Finally, we use
photogrammetry and 3D scanning. In Photogrammetry (or AR to monitor the process of 3D printing using the
Structure from Motion [15]), the three-dimensional coordinates reconstructed 3D model of a part.
of surface points of a 3D printed object are estimated using
pictures taken from different angles. In non-contact active 3D 3. System overview
scanning, the scanner estimates the surface points of the object
by detecting the reflection of radiation or light emitted by the The setup of the system used to test the proposed technique
scanner towards the object. The emitted radiations could be is shown in Fig. 1. It includes a Fused Filament Fabrication
infra-red, ultrasound or X-ray. This scanning technique either (FFF) technology based, Prusa i3 MK3, 3D printer with 13
use time of flight [16] or triangulation method [17] to estimate Mega Pixels camera mounted at its gantry and HoloLens. The
surface points of the object. Both photogrammetry and 3D camera lens has been oriented parallel to the printer’s print bed
scanning either require multiple cameras/scanners to be to acquire top view image of each printed layer. An app has
installed or a single camera/scanner to be moved around the also been developed in Unity Platform which enables the user
object to acquire images/data from different angles. The 3D to interact with the digital model in a mixed reality environment
surface points generated from these images/data also contain using HoloLens.
surface points of the environment around the object, and hence a
a manual removal (3D cropping) of these surface points is
required to retain surface points of the object only. These Camera
techniques capture only the surface information of the object
and do not provide any information about the internal condition Filament
Extruder
of the product. Print Bed
Augmented Reality (AR) is a human-computer
interaction tool that augments computer-generated perceptual b
information on the real-world environment. Extensive research
is being carried out to design and implement integrated AR-

*
https://www.microsoft.com/en-us/hololens Fig. 1. (a) Camera Mounted on 3D Printer; (b) HoloLens.
348 Ammar Malik et al. / Procedia CIRP 81 (2019) 346–351
Ammar Malik et al. / Procedia CIRP 00 (2019) 000–000 3

4. Methodology SPECT. In image registration, one of the images (referred to as


the moving or source image) is spatially transformed to align
This section presents the methodology for layer-by-layer with the other image (referred to as the target, fixed or sensed
reconstruction of the 3D model of the printed product and image). In our particular case, we have set mask as a moving
development of the app to interact with the digital model for image, and the camera-acquired image as a target image. While
real-time monitoring of the build process using HoloLens. image registration algorithms can be intensity-based or feature-
based, we have used MATLAB’s inbuilt function ‘imregister’
4.1. Image acquisition and processing for intensity-based automatic image registration to align the
images. In intensity-based registration, the misaligned image
To acquire images after the deposition of each layer, the (moving image) is spatially transformed to align with the target
extruder and print bed must be positioned in such a way that image by comparing intensity patterns in images [25].
the work is in front of the camera. Therefore, the G-code has
been modified by adding the following command before the 4.2. 3D Model Reconstruction
commands for the next layer (i.e. before the change in z-
coordinate). The reconstruction of the 3D model involves converting the
processed image to a 3D file format. A 3D file format is used
G1 X0 Y200; for storing information concerning the 3D models. The
common 3D file formats used for 3D printing are STL, OBJ,
This returns the extruder to the left and pushes the build plate PLY, AMF, 3MF, and FBX. Since HoloLens supports OBJ and
forward so that top view is clearly visible by the camera after FBX 3D model file formats, therefore, we have used OBJ file
deposition of each layer. The camera-acquired images are then format to estimate the geometry of the 3D model from
processed in MATLAB to generate the 3D model. Since the processed images.
image acquired after deposition of each layer also contains
information about the previous layers (and/or about the print 4.2.1. 3D model surface encoding
bed) therefore, the image must be processed to extract the
The processed image is converted into a binary image. The
information concerning the current layer.
area of the pixels of the previous layers is assigned a black
color, whereas the current layer area is assigned a gray color.
4.1.1. Segregation of the Current Layer
These pixel values are then used as depth values to convert 2D
processed image to 3D image. Fig. 2 (a) shows an example of
During the acquisition of the top view image, depth
a down-sampled triangular estimation of a processed 2D image
information is lost. Therefore, it is difficult to differentiate
and Fig. 2 (b) shows 3D geometry estimation from the
between the current layer-area and the previous layer-area. One
processed image. The processed binary image is then converted
possible solution to this problem is to use color-based image
to OBJ file to encode the geometry of the current layer based
segmentation. Since the majority of the layers in a 3D printed
on the processed image. In this encoding technique, the binary
product are of the same color therefore, we need some other
processed image is covered with the mesh of non-overlapping
strategy for segregation. One such strategy, as proposed in this
triangles. The pixels of the binary processed image are assigned
paper, is to use the information about the current layer through
as vertices of these triangles. The row number and column
the G-code of the 3D model being used for printing. The idea
number of the image pixels are assigned as x and y coordinates
is to use the image obtained by the G-code simulator of the
whereas pixel values are assigned as z-coordinates of the
current layer as a mask, and then to keep the information from
vertices of these triangles. Since we do not want the triangles
the camera acquired image which lies under the area delineated
lying on the previous layer to be encoded in the OBJ file of the
by this mask. Then, we assign the color black to the area which
current layer therefore, we use a condition to encode only the
does not lie under the mask. This image, containing
triangles having non-zero z-coordinates.
information about the current layer, is termed as the processed
image in the remainder of the paper, and we refer to the overall
technique as automated cropping. Since the camera-acquired a
image and the image acquired from G-Code simulator are from
different sources, we need to align both images before using
the image acquired from the G-Code simulator as a mask.

4.1.2. Image Registration


b
To use the image acquired from G-Code simulator as mask, the
mask must be aligned with the camera-acquired image to
overcome issues such as image rotation, scale, and skew. This
problem of aligning multiple scenes into a single integrated
image is solved by image registration. Image registration is an
image processing technique usually used to align satellite Fig. 2. (a) Triangle mesh over Processed 2D image; (b)
images or medical images captured with different imaging Estimated 3D Geometry from Processed Image.
sources, such as Magnetic resonance imaging (MRI) and
Ammar Malik et al. / Procedia CIRP 81 (2019) 346–351 349
4 Ammar Malik et al. / Procedia CIRP 00 (2019) 000–000

4.3. Interaction with the reconstructed model via HoloLens 5.1. Current Layer area Segregation

The Microsoft HoloLens is a holographic computer, Fig. 4 (b) shows the camera-acquired image after deposition of
enabling the user to interact with holograms in a mixed reality layer 2. The white area in Fig. 4 (b) is layer 1, whereas the blue
environment. An application has been developed in Unity area is layer 2. The mask (image obtained from G-Code
Platform which allows the user to interact with 3D models Simulator) to segregate layer 2 area from layer 1 area in Fig. 4
using HoloLens. The user can move and rotate to inspect the (b) is shown in Fig. 5. The misalignment between the camera
reconstructed model of each layer using hand gestures. Two acquired image (Fig. 4 (b)) and G-Code simulator acquired
voice commands, expand model and reset model, have also image (Fig. 5) is shown in Fig. 6. To use the layer 2 image from
been added. The expand model command shows the digital G-code simulator as a mask, we need to align Fig. 5 with Fig.
model layer-by-layer and let the user inspect each layer using 4 (b). Fig. 7 (a) shows the result of alignment using Image
hand gestures whereas reset model command reconstructs and registration. The segregated area of the layer 2 obtained after
joins all the layers back together. applying the aligned mask (Fig. 7 (b)) is shown in Fig. 8.

5. Results

The methodology described in Section 4 has been tested on


a 3D printed object shown in Fig. 3. Although the printed object
consists of 35 layers, for illustration purposes, we assume this
object consists only the of 4 layers shown in Fig. 4. The 3D
model of this product has been designed to contain a hidden I-
Form logo in the second layer and an intentional defect in the
third layer.
Fig.5. Layer 2 Image acquired from G-Code Simulator.

Fig. 3. 3D Printed Object.

a
Fig. 6. Misalignment between camera acquired and G-Code Simulator
acquired images.

c
Defect
b

Fig.4. Camera acquired images after deposition of (a) Layer 1; (b) Layer Fig. 7. (a) Aligned Images; (b) Mask Image after alignment.
2; (c) Layer 3; (d) Layer 4.
350 Ammar Malik et al. / Procedia CIRP 81 (2019) 346–351
Ammar Malik et al. / Procedia CIRP 00 (2019) 000–000 5

6. Concluding remarks

In this research, a novel scan-based method to reconstruct


the 3D model of the product during the manufacturing process
has been proposed and tested. The proposed method updates
the 3D model of the work in progress, layer after layer, hence
enabling real-time monitoring of the process. The proposed
scanning technique also allows the user to inspect the build
quality of the product by visualizing the reconstructed 3D
model in a mixed reality environment using HoloLens. The app
Fig. 8. Segregated Layer 2 (Processed) Image.
developed for HoloLens also allows the user to interact with
the digital model using hand gestures and voice commands.
Thus, the proposed scanning technique not only speeds up the
5.2. 3D Model Reconstruction iterative process development by enabling real-time
monitoring but also provides detailed information about the
After extraction of the layer 2 area from camera acquired build quality. The proposed scanning technique has the
image (Fig. 4 (b)), the current layer (Layer 2) geometry is potential to facilitate the development of a decision support
encoded in the OBJ file. The generated OBJ file for the current system using techniques from machine learning. This decision
layer viewed in 3D builder app is shown in Fig. 9. This process support system will not only be responsible for real-time defect
is repeated after deposition of each layer to generate OBJ file detection but will also provide support for the operator’s
for each layer and these files are then merged in the developed decision-making capabilities in order to mitigate the occurred
app. defects. This approach will be the direction of future work.

5.3. Interaction with the reconstructed 3D model Acknowledgments

Fig. 10 shows the reconstructed 3D Model of the four layers This publication has emanated from research supported in
viewed through HoloLens. The defect occurred during the part by a research grant from Science Foundation Ireland (SFI)
deposition of the third layer can be seen in Fig. 10. Hence, under grant number 16/RC/3872 and is co-funded under the
using the proposed scanning technique, the user can visualize European Regional Development Fund and by I-Form industry
the hidden I-Form logo in the second layer as well as the defect partners.
in the third layer.
References

[1] Ramakers R, Anderson F, Grossman T, Fitzmaurice G. Retrofab: A design


tool for retrofitting physical interfaces using actuators, sensors and 3d
printing. In Proceedings of the 2016 CHI Conference on Human Factors in
Computing Systems 2016 May 7 (pp. 409-419). ACM.
[2] Mueller S, Im S, Gurevich S, Teibrich A, Pfisterer L, Guimbretière F,
Baudisch P. WirePrint: 3D printed previews for fast prototyping.
InProceedings of the 27th annual ACM symposium on User interface
software and technology 2014 Oct 5 (pp. 273-280). ACM.
[3] Cotteleer M, Joyce J. 3D opportunity: Additive manufacturing paths to
performance, innovation, and growth. Deloitte Review. 2014 Jan 17;14:5-
Fig. 9. Generated OBJ file for layer 2. 19.
[4] Tapia G, Elwany A. A review on process monitoring and control in metal-
based additive manufacturing. Journal of Manufacturing Science and
Engineering. 2014 Dec 1;136(6):060801.
Layer 1 Layer 2 [5] Uriondo A, Esperon-Miguez M, Perinpanayagam S. The present and future
of additive manufacturing in the aerospace sector: A review of important
aspects. Proceedings of the Institution of Mechanical Engineers, Part G:
Journal of Aerospace Engineering. 2015 Sep;229(11):2132-47.
[6] Gu DD, Meiners W, Wissenbach K, Poprawe R. Laser additive
manufacturing of metallic components: materials, processes and
mechanisms. International materials reviews. 2012 May 1;57(3):133-64.
Layer 3 Layer 4
[7] Zeng K, Pal D, Stucker B. A review of thermal analysis methods in laser
Defect sintering and selective laser melting. In Proceedings of Solid Freeform
Fabrication Symposium Austin, TX 2012 Aug 6 (Vol. 60, pp. 796-814).
[8] Abeykoon C, Martin PJ, Li K, Kelly AL. Dynamic modelling of die melt
temperature profile in polymer extrusion: Effects of process settings, screw
geometry and material. Applied Mathematical Modelling. 2014 Feb
15;38(4):1224-36.
[9] Oropallo W, Piegl LA. Ten challenges in 3D printing. Engineering with
Computers. 2016 Jan 1;32(1):135-48.
Fig. 10. Reconstructed 3D Model of the layers viewed through HoloLens. [10] Hartley, Richard, and Andrew Zisserman. "Multiple view geometry in
computer vision." Robotica 23.2 (2005): 271-272.
Ammar Malik et al. / Procedia CIRP 81 (2019) 346–351 351
6 Ammar Malik et al. / Procedia CIRP 00 (2019) 000–000

[11] Merrell P, Akbarzadeh A, Wang L, Mordohai P, Frahm JM, Yang R,


Nistér D, Pollefeys M. Real-time visibility-based fusion of depth maps.
In2007 IEEE 11th International Conference on Computer Vision 2007 Oct
14 (pp. 1-8). IEEE.
[12] Levoy M, Pulli K, Curless B, Rusinkiewicz S, Koller D, Pereira L, Ginzton
M, Anderson S, Davis J, Ginsberg J, Shade J. The digital Michelangelo
project: 3D scanning of large statues. InProceedings of the 27th annual
conference on Computer graphics and interactive techniques 2000 Jul 1
(pp. 131-144). ACM Press/Addison-Wesley Publishing Co..
[13] Frahm JM, Fite-Georgel P, Gallup D, Johnson T, Raguram R, Wu C, Jen
YH, Dunn E, Clipp B, Lazebnik S, Pollefeys M. Building rome on a
cloudless day. In European Conference on Computer Vision 2010 Sep 5
(pp. 368-381). Springer, Berlin, Heidelberg.
[14] Kazhdan M, Hoppe H. Screened poisson surface reconstruction. ACM
Transactions on Graphics (ToG). 2013 Jun 1;32(3):29.
[15]Schonberger JL, Frahm JM. Structure-from-motion revisited.
InProceedings of the IEEE Conference on Computer Vision and Pattern
Recognition 2016 (pp. 4104-4113).
[16] Cui Y, Schuon S, Chan D, Thrun S, Theobalt C. 3D shape scanning with
a time-of-flight camera. In s2010 IEEE Computer Society Conference on
Computer Vision and Pattern Recognition 2010 Jun 13 (pp. 1173-1180).
IEEE.
[17] Chow J, Ang K, Lichti D, Teskey W. Performance analysis of a low-cost
triangulation-based 3D camera: Microsoft Kinect system. InInt. Soc. for
Photogrammetry and Remote Sensing Congress (ISPRS) 2012 Aug 25
(Vol. 39, p. B5).
[18] Fründ J, Gausemeier J, Matysczok C, Radkowski R. Using augmented
reality technology to support the automobile development. InInternational
Conference on Computer Supported Cooperative Work in Design 2004
May 26 (pp. 289-298). Springer, Berlin, Heidelberg.
[19] Ng LX, Oon SW, Ong SK, Nee AY. GARDE: a gesture-based augmented
reality design evaluation system. International Journal on Interactive
Design and Manufacturing (IJIDeM). 2011 Jun 1;5(2):85.
[20] Ong S, Pang Y, Nee AY. Augmented reality aided assembly design and
planning. CIRP annals. 2007 Jan 1;56(1):49-52.
[21] Pang Y, Nee AY, Khim Ong S, Yuan M, Youcef-Toumi K. Assembly
feature design in an augmented reality environment. Assembly
Automation. 2006 Jan 1;26(1):34-43.
[22] Shen Y, Ong SK, Nee AY. Augmented reality for collaborative product
design and development. Design Studies. 2010 Mar 1;31(2):118-45.
[23] Yamaoka J, Kakehi Y. MiragePrinter: interactive fabrication on a 3D
printer with a mid-air display. In ACM SIGGRAPH 2016 Studio 2016 Jul
24 (p. 6). ACM.
[24] Ceruti A, Liverani A, Bombardi T. Augmented vision and interactive
monitoring in 3D printing process. International Journal on Interactive
Design and Manufacturing (IJIDeM). 2017 May 1;11(2):385-95.
[25] Oliveira FP, Tavares JM. Medical image registration: a review. Computer
methods in biomechanics and biomedical engineering. 2014 Jan
25;17(2):73-93.