Вы находитесь на странице: 1из 7

See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/326822437

Automated Process Monitoring in 3D Printing Using Supervised Machine


Learning

Article · August 2018


DOI: 10.1016/j.promfg.2018.07.111

CITATIONS READS

6 201

2 authors:

Ugandhar Delli Shing I. Chang


Kansas State University Kansas State University
2 PUBLICATIONS   7 CITATIONS    54 PUBLICATIONS   867 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Bulk Dispatch Optimization View project

All content following this page was uploaded by Ugandhar Delli on 04 September 2018.

The user has requested enhancement of the downloaded file.


Available online at www.sciencedirect.com
Available online at www.sciencedirect.com
ScienceDirect
ScienceDirect
Procedia Manufacturing
Available 00 (2018) 000–000
Availableonline
onlineatatwww.sciencedirect.com
www.sciencedirect.com
Procedia Manufacturing 00 (2018) 000–000 www.elsevier.com/locate/procedia
www.elsevier.com/locate/procedia
ScienceDirect
ScienceDirect 
Procedia Manufacturing 26 (2018) 865–870
Procedia Manufacturing 00 (2017) 000–000
www.elsevier.com/locate/procedia
46th SME North American Manufacturing Research Conference, NAMRC 46, Texas, USA
46th SME North American Manufacturing Research Conference, NAMRC 46, Texas, USA
Automated Process Monitoring in 3D Printing Using Supervised
Automated Process Monitoring in 3D Printing Using Supervised
Machine Learning
Machine
Manufacturing Engineering Society Learning
International Conference 2017, MESIC 2017, 28-30 June
2017, Vigo (Pontevedra), Spain

Costing models for capacity optimization in Industry 4.0: Trade-off


Ugandhar Delli* and Shing Chang
between used capacity
Ugandhar and
Delli* andoperational
Shing Chang efficiency
Department of Industrial and Manufactruing Systems Engineering, Kansas State University, 2061 Rathbone Hall, 1701B Platt St, Manhattan KS
Department of Industrial and Manufactruing Systems a Engineering, Kansas
66502, United State
States University,
of America b 2061 Rathbone Hall,
b 1701B Platt St, Manhattan KS
A. Santana , P. Afonso , A. Zanin , R. Wernke
a,*
66502, United States of America
* Corresponding author. Tel.: +1-785-851-9541; fax: +0-000-000-0000 .
a
University of Minho, 4800-058 Guimarães, Portugal
E-mail address:author.
* Corresponding Tel.: +1-785-851-9541; bfax: +0-000-000-0000 .
ugandhardelli@ksu.edu
Unochapecó, 89809-000 Chapecó, SC, Brazil
E-mail address: ugandhardelli@ksu.edu

Abstract
Abstract
Abstract
Quality monitoring is still a big challenge in additive manufacturing, popularly known as 3D printing. Detection of
Quality
defectsthe
Under monitoring
during is
ofstill
the printing
concept aprocess
big challenge
"Industry will help
4.0", ineliminate
additiveprocesses
production manufacturing,
waste ofwill bepopularly
material and time.
pushed known
to Defect asdetection
3D printing.
be increasingly Detection
during of
the initial
interconnected,
defects
stages during
of the
printing printing
may process
generate an will
alert help
to eliminate
either pause waste
or stopof material
the printing and time.
process Defect
so that detection
corrective
information based on a real time basis and, necessarily, much more efficient. In this context, capacity optimization during
measuresthe initial
can be
stages ofprevent
takenbeyond
to printingthe may generate
need toaim an alert
reprint to either
the parts. pause
This paperorproposes
stop the printing
a method process so that corrective
to organization’s
automatically assess measures
the quality can be
of 3D
goes the traditional of capacity maximization, contributing also for profitability and value.
taken
printedtoparts
prevent
with thetheneed to reprintofthe
integration a parts. This
camera, paper
image proposes and
processing, a method to automatically
supervised machine assess the
learning. quality
Images of of 3D
semi-
Indeed, lean management and continuous improvement approaches suggest capacity optimization instead of
printed
finishedparts
partswith
are the integration
taken atofseveral ofcritical
a camera,
stagesimage
ofand processing,
the and supervised
printingmodels
process machine learning. Images machine
of semi-
maximization.
finished
Thetaken
studyat capacity optimization costing isaccording
an importantto the part
research geometry.
topic thatA deserves
learning parts are support
method, severalmachine
vector critical (SVM),
stages ofisthe printingtoprocess
proposed classifyaccording
the parts to theeither
into part geometry.
‘good’ or A machine
‘defective’
contributions
learning fromsupport
both thevector
practical and theoretical isperspectives. This paper presents andeither
discusses
‘good’a mathematical
orA‘defective’
category.method,
Parts using ABS and PLA machine (SVM),
materials were proposed
printed totodemonstrate
classify thethe
parts into
proposed framework. numerical
model for
category. capacity
Parts usingmanagement
ABS and based
PLA on different
materials werecosting
printed models
to (ABC
demonstrate and TDABC).
the proposed A generic
framework.model has been
A numerical
example is provided to demonstrate how the proposed method works.
developed and it was used to analyze idle capacity and
example is provided to demonstrate how the proposed method works. to design strategies towards the maximization of organization’s
value.
© 2018TheThe trade-off capacitybymaximization
Authors. Published Elsevier B.V. vs operational efficiency is highlighted and it is shown that capacity
© 2018 The Authors.
optimization might Published
hide by Elsevier
operational B.V.
inefficiency.
© 2018 The under
Peer-review Authors. Published by
responsibility of Elsevier B.V. committee of NAMRI/SME.
the scientific
Peer-review
© under responsibility
2017 The Authors. of Elsevier
Published by the scientific
B.V.committee of the 4th International Conference on System-Integrated Intelligence.
Peer-review under responsibility of the scientific committee of NAMRI/SME.
Peer-review
Keywords: 3Dunder responsibility
printing, of theSupervised
Image processing, scientific machine
committee of theSVM
learning, Manufacturing Engineering Society International Conference
2017.
Keywords: 3D printing, Image processing, Supervised machine learning, SVM

1. Introduction
Keywords: Cost Models; ABC; TDABC; Capacity Management; Idle Capacity;as
Operational Efficiency
3D printing allows for more manufacturing options
1. Introduction as 3D printing
because allows
it has theforability
more manufacturing options
to create complex
Additive manufacturing (AM) has become a viable because it shapes
geometrical has thewithability
the helptoof create complex
design software.
1. Introduction
Additive
option formanufacturing (AM)years.
production in recent has become
AM oftena knownviable geometrical shapesofwith
Other advantages AMthe help ofrapid
include design software.
prototyping,
option for production in recent years. AM often known Other advantages of AM include rapid prototyping,
The cost of idle capacity is a fundamental information for companies and their management of extreme importance
in modern©production
2351-9789 systems.
2018 The Authors. In general,
Published it isB.V.
by Elsevier defined as unused capacity or production potential and can be measured
in several©under
2351-9789
Peer-review ways: tons of production,
2018responsibility
The Authors. Published
of available
by Elsevier
the scientific B.V.hours
committee of manufacturing, etc. The management of the idle capacity
of NAMRI/SME.
Peer-review underTel.:
* Paulo Afonso. responsibility
+351 253 of the761;
510 scientific committee
fax: +351 253 604of741
NAMRI/SME.
E-mail address: psafonso@dps.uminho.pt

2351-9789 © 2017 The Authors. Published by Elsevier B.V.


Peer-review under responsibility of the scientific committee of the Manufacturing Engineering Society International Conference 2017.
2351-9789 © 2018 The Authors. Published by Elsevier B.V.
Peer-review under responsibility of the scientific committee of the 4th International Conference on System-Integrated Intelligence.
10.1016/j.promfg.2018.07.111
866 Ugandhar Delli et al. / Procedia Manufacturing 26 (2018) 865–870
2 Delli, Chang/ Procedia Manufacturing 00 (2018) 000–000

inventory reduction, and decentralized production. 3D Wuest et al. [3] proposed a method to deal with the
printing has already contributed significantly to large amounts of complex and high-dimensional data
several technological advances in medicine and is in conventional manufacturing. First, they described
emerging in many other fields like fashion design, and the concept of individual product states along the
architecture and even Nike recently launched the first entire manufacturing line including all sorts of
athletic shoe including 3D printed components [1]. information which could be utilized during the
process. Then, application of a combination of Cluster
Many 3D printing machines do not have a Analysis and SVM on product state data has been
designated system to track and monitor the progress of presented as a potential way for improved quality
printing process. 3D printers may continue to print the monitoring. Fastowicz and Okarma [4] proposed a
method for live monitoring of the progress of 3D
part until all the layers have been completed even if
printing process based on texture analysis using the
the filament ran out or there are any potential defects
Gray-Level Co-occurrence Matrix (GLCM) and
in the print. Detecting defects by performing quality
chosen Haralick features. Images obtained by
check at various (critical) stages of the printing scanning the 3D printed plates have been used to
process not only help in assuring corrective measures verify the approach. Also, taking the influence of
but also eliminate waste of printing bad parts. lighting conditions into consideration, several
Automatic process monitoring is especially important experiments have been performed by taking pictures
for production scale 3D printing operation in which with the help of a camera mounted on a 3D printer.
hundreds of 3D printing machines are used in mass Due to the influence of lighting conditions on the
production of the same part. images obtained when compared to the scanned
images, several modifications have been made to the
A method has been proposed to monitor the 3D proposed method to achieve better results.
printing process by pausing the printer at various
checkpoints to detect any possible defects and take Vaezi and Chua [5] studied the effects of two
proper corrective actions such as stopping the print if parameters of layer thickness and binder saturation
a potential defect is detected. level on the mechanical strength, surface quality and
dimensional accuracy in the 3D printing process.
2. Literature review Experimental results show that under the same layer
thickness, increasing the binder saturation level from
Given the fact that 3D printing is still a relatively 90% to 125% would result in an increase of tensile and
modern technology in the manufacturing field, the flexural strengths of the specimens and decrease of
amount of literature available addressing the quality dimensional accuracy and surface uniformity of the
aspects in 3D printing is limited. Straub [2] proposed specimens. On the other hand, under the same binder
a method to detect completion failure defects during saturation level, increase in layer thickness from 0.087
the 3D printing process. A multi-camera system to 0.1 mm would decrease tensile strength and increase
comprised of five camera units (each unit consisted of flexural strength. Also, it yields better surface
a Raspberry Pi and a Raspberry Pi camera) and image uniformity. Other work related to 3D printing quality
processing software created using C# and Dot Net can be found in [7] and [8]. Besides the efficient use
framework was used to assess the quality of the parts of supervised machine learning to monitor quality in
produced. Images (of the in-process object) were taken 3D printing which is first of its kind, our research
at eight positions by each of five cameras placed at five focuses on some of the key issues in 3D printing that
different angles (to make a total of 40 images) and they were not addressed in the previous research like
were compared against a final object image. The detection of structural defects.
comparison was done on a pixel-by-pixel basis. The
author affirmed that the comparison in his method
could detect two types of errors: when a part has been 3. Implementation strategy
stopped during the progress of printing which results
in an incomplete part and when there are issues with Conventionally, the quality of 3D printed parts is
the printer which results in a failure of injection of being checked after the printing is done. Detecting
filament into the nozzle. However, this method cannot defects during the printing process not only help to
detect geometrical or structural defects which are eliminate waste of material and time but also prevent
common in any manufacturing process.
Ugandhar Delli et al. / Procedia Manufacturing 26 (2018) 865–870 867
Delli, Chang/ Procedia Manufacturing 00 (2018) 000–000 3

the need to reprint the whole part. In this study, quality


is being checked at various checkpoints during the
printing process with the help of images taken by a
camera. A checkpoint is defined as an
important/critical stage in the printing process where
LED light
there is a significant change in the geometry of the
part. For example, consider a complex part which
involves different stages of printing like skirt/base,
body and the top. These stages could be considered as
the desired checkpoints to inspect quality at. The
following steps implement the proposed quality
monitoring of a 3D printed part during production.

Step 1. Identify proper check points for 3D printing


part according to its geometry.
Step 2. Take images of the semi-finished part at each
check point.
Step 3. Perform image processing and analysis. Camera

Conventionally, quality is represented in terms of an


integer or a numerical value. But here, an image is
used to represent quality of the 3D printed material. A
Bed
python code has been developed to automate the
process of taking pictures at checkpoints, then image
processing and finally classifying the parts as either
‘good’ or ‘bad’ using SVM. The proposed method is Figure 1. Experimental setup
designed to detect both completion failure defects as
well as geometrical or structural defects. Considering the effect of lighting conditions, several
experiments have been performed initially by taking
images using the camera for different lighting
4. Experimental setup and research methods conditions. Changes in the lighting conditions have
significantly changed the results of pixel analysis. So,
The 3D printer used for this research is Lulzbot Mini an assumption has been made to maintain constant
and the 3D printing process applied is Fused lighting (using the LED light shown in Figure 1)
Deposition Modeling (FDM). It should be noted that, throughout the course of experiments to facilitate
at this point, our proposed method is specific to the image comparison.
mentioned printer and printing process only. The parts
being 3D printed are monitored by taking pictures at At first, the part desired to be printed is designed by
various checkpoints with the help of a camera and the a design software such as SolidWorks. After that, the
setup is shown in Figure 1. The printing materials used STL file of the designed model is obtained. Then,
during this research are ABS and PLA. Both materials checkpoints are identified depending on the geometry
have different mechanical properties which have a of the part. The next step is to start the printing process
significant impact on the quality of the printed and it continues until it reaches a specified checkpoint
products using these two materials. Programming for the given print. When it reaches a checkpoint, the
printing process is paused for a while and the nozzle is
language python has been used to build a code to
raised up to the specified height. Then the bed moves
automate real-time monitoring of the printing process.
all the way to the front end of the printer where camera
We maintained the same 3D printer configuration
is positioned to take pictures. Then the camera takes a
throughout the duration of this research. picture after a small pause. After that, the bed and the
nozzle move back to their original position (where
they were before the printer paused) and starts printing
again.
868 Ugandhar Delli et al. / Procedia Manufacturing 26 (2018) 865–870
4 Delli, Chang/ Procedia Manufacturing 00 (2018) 000–000

4.1 Image processing of parts is SVM classification to identify the parts as


either ‘good’ or ‘defective’. Detection of defective
The dimensions of the images taken at each parts may need the printing process to be stopped to
checkpoint is 640x480. A sample image taken at a take corrective measures. If both the SVM results and
checkpoint (of the part printed with PLA material) is the results of initial quality check using experimental
shown in Figure 2. error data go hand in hand, it indicates that the
proposed method is working as expected.

4.2 Supervised machine learning

The machine learning method used in this research is


Support Vector Machine (SVM). Applying SVM
classification is the final stage in the quality assurance
process. Before going into the research methodology
applied, let us take a brief look into SVM. Support
Vector Machine is a supervised machine learning
model which analyses data used for both regression
Figure 2. Image taken at a checkpoint analysis/classification. Given a set of training models,
each of them belonging to one or the other category,
To facilitate image analysis, every image taken at the the SVM training algorithm builds a model that
checkpoints is divided into 16 identical sections in a assigns any new models to one or the other category.
4x4 matrix. For each section, average of the RGB A simple example of SVM is shown in Figure 3.
values of all pixels is calculated. That means, for an Several applications of SVM are found in
image taken at a checkpoint, 16 average (RGB) values bioinformatics, image and text recognition etc.
have been calculated for all the 16 sections. Let si be
the average (RGB) values for sections where i = 1 to
16. The same procedure is applied for all the images
taken at various checkpoints during the printing
process. For proposed method to work, an ideal print
should be available at each check point. Any new
image at a checkpoint is then compared to the ideal
print image. At each checkpoint, experimental error Ei
(i = 1 to 16) is calculated for each section’s average
RGB values. Experimental error is calculated using
the formula:

Ei = ((Ai – Ii)/Ii) * 100 (1)

where Ei = experimental error


Ai = actual value
Ii = ideal value
i = 1 to 16.

One way to initially detect potential errors or defects


is by generating an alert if the experimental error value
of at least 4 sections are > 10% (metric chosen after
performing several experiments taking environmental Figure 3. Basic concept of SVM
factors into consideration) as it signals a significant
variation from the ideal print. After the initial quality
check, the main method applied to assess the quality
Ugandhar Delli et al. / Procedia Manufacturing 26 (2018) 865–870 869
Delli, Chang/ Procedia Manufacturing 00 (2018) 000–000 5

SVM classification can be either linear or non-linear. 5. A numerical example


As it is not possible to generate linear fits for most of
the real word problems, non-linear classifiers are A 3D model of a DNA has been used to print parts for
widely used for pattern analysis. The most commonly this example. Five checkpoints are defined according
used approach to create a non-linear classifier is the to the geometry of the DNA structure to perform
kernel trick. Kernel methods are fundamentally based quality check at five critical stages of the printing
on calculating the similarity between the unlabeled process. ABS material is used to print the parts. Once
current inputs and the training inputs. Prediction for the printing process starts, it continues until it reaches
the unlabeled inputs, i.e. for those which are not in the a specific checkpoint. After reaching a checkpoint, the
training datasets, is done by the application of a printing process pauses for a while and the camera
similarity function k, called a kernel, between the mounted on the printer takes an image of the part. The
unlabeled current input x’ and each of the training same procedure goes on until pictures are taken at
inputs xi. For example, a binary classifier using kernel every checkpoint. A training set containing both
trick typically calculates a weighted sum of ‘good’ and ‘defective’ parts should be used to train the
similarities which is given by: proposed SVM model to classify any new incoming
inputs. A python package scikit-learn is used to both
ŷ = sgn ∑𝑖𝑖=1,𝑛𝑛 wiyik(xi,x’) (2) train and execute the proposed SVM algorithm.
where: 𝑛𝑛
• ŷ ∈ {-1, +1} is the predicted label of Then the model is validated by another dataset which
kernelized binary classifier for the unlabeled is not used in the training dataset. The outcome of the
input x’ whose hidden true label y is to be trained SVM model will predict the quality of each
found part as either ‘good’ or ‘defective’. A sample image of
• k is the kernel function that measures a part taken at a checkpoint is shown in Figure 4.
similarity between any pairs of inputs xi, x’
• yi ∈ {-1, +1} is the true label for the given
training set of inputs xi
• wi ∈ R are the weights for the training
examples determined by the learning
algorithm
• sgn is the sign function that determines
whether the predicted classification ŷ results
as positive or negative [6].

Now coming to this research, the section averages


(of RGB values) of the images calculated at
checkpoints are loaded as input to the vectors of
training models. Two categories of training models
namely ‘good’ (identical to the ideal parts) and ‘bad’
(defective parts) have been loaded into the system.
Then the SVM training algorithm has built a model Figure 4. Image at a checkpoint for an unlabeled
which classifies any new models loaded into the current input
system as either ‘good’ or ‘bad’ print. If a part is
identified as ‘bad’ during the initial stages of printing, The average RGB values of the 16 sections of the
then a corrective measure to stop the printing process image at the (first) checkpoint are as follows: (148.18,
will be taken to prevent further wastage of material 120.62, 106.28, ..., 87.60), which is a vector having 16
and time and then reprint the part again. The SVM average elements. It is then fed into the SVM shown
used for this research is a 5 or 6 stage process in equation (2). Then the kernelized binary classifier
depending upon how many checkpoints (5 or 6) are labels the input into either ‘good’ or ‘defective’
defined for a specific print. category. In this case, the result is that it is a ‘good
part’. This numerical example demonstrates how the
870 Ugandhar Delli et al. / Procedia Manufacturing 26 (2018) 865–870
6 Delli, Chang/ Procedia Manufacturing 00 (2018) 000–000

proposed method can be applied to perform real-time


monitoring of a 3D printing process.

6. Results and conclusions

The proposed method which integrates the efficacy of


image processing and supervised machine learning
successfully serves the purpose of real-time
monitoring of a 3D printing process. The method is
capable of detecting both completion failure defects
such as filament running out or printing stopped in the
mid-progress and structural or geometrical defects.
The main drawback of the proposed method is that the
printing process needs to be paused while the images
of a semi-finished part are taken. Another drawback is
that since only top view images are taken, the proposed
method might not be able to detect the defects on the
vertical plane which cannot be seen in the top view
image. This gives us a direction for future research to
incorporate cameras on the sides of the printer as well
to detect defects on both the horizontal and vertical
planes. A possible improvement can be made by in situ
camera mounting on print head. Future work will also
focus on the study of impacting factors for choosing
proper check points.

References

[1] G. White, Industry analysis: The pros and cons of 3D printing


(2015). Retrieved from http://www.manufacturingglobal.com/
[2] J. Straub, Initial work on the characterization of additive
manufacturing (3D printing) using software image analysis.
Journal of Machines, 3 (2015) 55-71.
[3] T. Wuest, C. Irgens, K. D. Thoben, An approach to monitoring
quality in manufacturing using supervised machine learning on
product state data. Journal of Intelligent Manufacturing, 25
(2014) 1167-1180.
[4] J. Fastowicz, K. Okarma, Texture based quality assessment of
3D prints for different lighting conditions. In Proceedings of the
International Conference on Computer Vision and Graphics,
ICCVG (2016) 17-28.
[5] M. Vaezi, C. K. Chua, Effects of layer thickness and binder
saturation level parameters on 3D printing process.
International Journal of Advanced Manufacturing Technology,
53 (2011) 275-284.
[6] J. S. Taylor, N. Cristianni, Kernel methods for pattern analysis.
Cambridge University Press, 2004.
[7] D.A. Roberson, D. Espalin, R.B. Wicker, 3D printer selection:
A decision-making evaluation and ranking model. Virtual and
Physical Prototyping, 2015, 8:3, 201-212.
[8] W. M. Wang, C. Zanni, L. Kobbelt, Improved surface quality in
3D printing by optimizing the printing direction. Eurographics,
2016.

View publication stats