Elsevier Editorial System(tm) for Analytica Chimica Acta

Manuscript Draft

Manuscript Number:

Title: Hyperspectral Image Analysis. A feasible example of Hierarchical Classification of plastics

containing flame retardants.

Article Type: Tutorial Paper (ONLY)

Keywords: Hyperspectral; Near Infrared; class-modelling; plastic sorting; hierarchical modelling; flame
retardant

Corresponding Author: Dr. Jos Manuel Amigo, Ph.D.

Corresponding Author's Institution: University of Copenhagen

First Author: Jos Manuel Amigo, Ph.D.

Order of Authors: Jos Manuel Amigo, Ph.D.; Hamid Babamoradi; Saioa Elcoroaristizabal

Manuscript Region of Origin: DENMARK

Suggested Reviewers: Alessandro Ulrici

alessandro.ulrici@unimore.it
Expert in hyperspectral analysis

Aoife Gowen
aoife.gowen@ucd.ie
Expert in Hyperspectral Analysis

Davide Ballabio
davide.ballabio@unimib.it
Expert in class-modelling

Marina Cocchi
marina.cocchi@unimore.it
Expert in class-modelling
Cover Letter(including Novelty Statement)

FACULTY OF SCIENCE
UNIVERSITY OF COPENHAGEN

Copenhagen, 27th of May, 2015

Dear Editor,

Please find enclosed a copy of the following manuscript to be considered for publication in
Analytica Chimica Acta:

Hyperspectral Image Analysis. A feasible example of Hierarchical Classification of plastics

containing flame retardants

The article is original, unpublished and not being considered for publication elsewhere. We believe
that the manuscript meets the requirements for its publication in Analytica Chimica Acta.

Important to note is that this is a tutorial paper that Prof. Manuel Mir Llad personally
invited us to write.

Looking forward to hearing from you,

Sincerely,

Jos Manuel Amigo Rubio

Associate Professor
Departament of Food Science. University of Copenhagen
Denmark
*Graphical Abstract
*Highlights

Highlights:
- Comprehensive tutorial of Hyperspectral Image analysis
- Hierarchical discrimination of six classes of plastics containing flame retardant
- Step by step guidelines to perform class-modeling on hyperspectral images
- Fusion of multivariate data analysis and digital image processing methods
- Promising methodology for real-time detection of plastics containing flame retardant
*Manuscript (including figures, tables, text graphics and associated captions)
Click here to view linked References

1 Hyperspectral Image Analysis. A feasible example of Hierarchical Classification of plastics

2
3
4 containing flame retardants.
5
6
7
8
Jos Manuel Amigo1*, Hamid Babamoradi1, Saioa Elcoroaristizabal2
9
10
11
12
13 1
Spectroscopy and Chemometrics group, Department of Food Sciences, Faculty of Science,
14
15
16 University of Copenhagen. Rolighedsvej 30, Frederiksberg C, DK1958, Denmark.
17
18 2
Chemical and environmental engineering department, School of Engineering, University of the
19
20
21 Basque Country, Alameda de Urquijo s/n, E-48013, Bilbao, Spain.
22
23
24
25
26
*Corresponding author: jmar@food.ku.dk
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62 1
63
64
65
 1 Abstract
2
3
4 Undoubtedly, hyperspectral imaging is becoming a major tool in many research laboratories. The
5
6 non-destructive ability to obtain both spectral and spatial information of surfaces is attracting many
7
8 researchers and generating new applications in different fields. Technology improvements have
9
10
11 consequently advanced in the direction of the development of more robust, reliable and cheaper
12
13 devices which offer high quality hyperspectral images in a wide range of modalities and spectral
14
15
16 intervals. This blooming of hyperspectral images also demands more and more reliable data
17
18 analysis methodologies to manage and extract the desired information from the sometimes
19
20
21 overwhelming information that a single sample can produce. This manuscript aims at providing
22
23 guidelines and practical tools to assist with the analysis of hyperspectral images. Considering the
24
25
26
broad character of current applications, this paper has focused on an industrial chemical framework
27
28 to explain, in a step-wise manner, how to develop a classification methodology to differentiate
29
30 between several types of plastics by using Near infrared hyperspectral imaging and Partial Least
31
32
33 Squares Discriminant Analysis. Thus, the reader is guided through every single step and oriented
34
35 in order to adapt those strategies to the users case.
36
37
38
39
40
41
42
43
Keywords: Hyperspectral; Near Infrared; class-modelling; plastic sorting; hierarchical modelling;
44
45 flame retardant;
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62 2
63
64
65
 1 Contents
2
3
4
5
6
7 1 Hyperspectral Imaging.................................................................................................................. 4
8
9 2 Materials and methods .................................................................................................................. 6
10
11
12 2.1 Materials ................................................................................................................................ 6
13
14
15 2.2 Measurements ........................................................................................................................ 7
16
17 2.3 Software................................................................................................................................. 8
18
19
20 3 Structure of hyperspectral images ................................................................................................ 9
21
22 4 General scheme to perform hyperspectral images analysis ........................................................ 11
23
24
25 5 Classification model development for the five major categories of plastics .............................. 13
26
27
28 5.1 Hypercube pre-processing ................................................................................................... 13
29
30 5.2 PLS-DA model performance ............................................................................................... 15
31
32
33 5.3 Statistical assessment of the model ..................................................................................... 18
34
35
6 Classification models for each individual category .................................................................... 20
36
37
38 7 From pixel-wise to object-wise projection ................................................................................. 24
39
40
41 7.1 Suppression of the pixels within the pellets ........................................................................ 25
42
43 7.2 PLS-DA results projected to the individual pellets ............................................................. 27
44
45
46 8 Remarks and final comments...................................................................................................... 29
47
48
49
9 Acknowledgements..................................................................................................................... 31
50
51 10 References ............................................................................................................................... 32
52
53
54
55
56
57
58
59
60
61
62 3
63
64
65
 1 1 Hyperspectral Imaging
2
3
4
5
6 Hyperspectral Imaging started in the early 70s with application on remote sensing [1-4]. After
7
8 becoming a popular technique in remote sensing, it started blooming in many different areas:
9
10
11 pharmaceutical research [5-11], pharmaceutical production [12-15], food sciences [16-19], food
12
13 quality assurance [20-26], forensic sciences [27-32], biochemistry and biomedicine [33-35], or
14
15
16 cultural heritage [36, 37], among others. These are just some examples found in the bibliography,
17
18 since making an exhaustive review of all the scientific fields using hyperspectral imaging is out of
19
20
21 the scope of this manuscript. But clearly, the feasibility of a technique that is able to measure a
22
23 whole spectrum (in different spectral modalities e.g. Near Infrared, Middle Infrared, Raman,
24
25
26
Ultraviolet-Visible, Fluorescence, Terahertz [38] or even Magnetic Resonance [39]) [40] for every
27
28 single pixel in which the sample is divided, makes possible this blooming in the applications.
29
30
31
32
33 This breakthrough is due to two main reasons: the tremendous evolution of faster, more reliable and
34
35 more robust optical devices and the implementation of powerful algorithms for processing
36
37
38 hyperspectral images. The big success of hyperspectral imaging could not be understood without its
39
40 close synergy with the, sometimes overwhelming, implementation of different algorithms to handle
41
42
43
all the data generated for a single image. The final aim of hyperspectral image analysis is to obtain a
44
45 picture containing selective and specific information (quantitative, groups or spatial distribution) of
46
47 the conforming compounds of the sample.
48
49
50
51
52 Leaving the applications where this selective picture can be extracted by depicting the picture of
53
54
55 one single wavelength (sometimes found in Raman hyperspectral images), hyperspectral analysis is
56
57 inherently linked to multivariate data analysis (a.k.a. chemometrics). The main issue on this is,
58
59
60
61
62 4
63
64
65
 1 precisely, how to extract the selective information. Or, better said, which multivariate/chemometric
2
3
4 method should be used. In this sense, many reviews have also been written pointing out the main
5
6 multivariate or statistical methods to be applied in the scientific field for different purposes [5-9, 16,
7
8 41-47]. Every multivariate method has an aim and its own particularities. It is very common to find
9
10
11 several algorithms for the same analytical purpose, and it would be nonsense to revise and describe
12
13 all of them in a precise manner. This manuscript will highlight the main points to be considered in
14
15
16 every single step in the analysis of hyperspectral images. Besides, we will expose some major
17
18 benefits and drawbacks of the most common multivariate methods generally applied in multivariate
19
20
21 data analysis on hyperspectral images.
22
23
24
25
26
In order to follow a conducting thread, this tutorial will be more focused on the development of a
27
28 hierarchical classification model for plastic sorting. This will help us to put a chemical framework.
29
30 We have chosen classification of plastics for two reasons: 1) Classification is one of the main
31
32
33 application domains of hyperspectral imaging. 2) It is more and more common to see hyperspectral
34
35 cameras installed in recycling plants for plastic sorting [48-51].
36
37
38
39
40 This manuscript has an additional original research value. Flame retardants (FRs) are organic
41
42
43
compounds added to the plastics to increase the resistance to ignition, reduce flame spreading,
44
45 suppress smoke formation, and prevent a polymer from dripping [52]. Among all, Brominated FRs
46
47 (BFRs) constitute a diverse group of FRs for minimizing fire hazards. BFRs are cost-effective and
48
49
50 offer a high degree of processability, making them the most commonly used FRs in plastics.
51
52 However, BFRs cause major environmental problems. Waste objects that contain BFRs must be
53
54
55 automatically identified and removed from the waste stream. Identification of BFRs in plastics has
56
57 been successfully accomplished using many methods such as Infrared Spectroscopy [53], Raman
58
59
60
61
62 5
63
64
65
 1 spectroscopy [54], Laser-Induced Breakdown Spectrometry (LIBS) [55], Sliding spark
2
3
4 spectroscopy [56], X-Ray Fluorescence spectroscopy (XRF) [57], and chromatography [58].
5
6 However, many of these methods are slow, expensive, and difficult to implement in a real-time
7
8 framework, which make them unsuitable for automated sorting systems. In contrast, near infrared
9
10
11 spectroscopy (NIR) is extensively used for automatic sorting due to fast scanning abilities, relative
12
13 low cost and non-destructive nature. Nevertheless, successful identification of BFRs using NIR
14
15
16 spectroscopy has not been reported yet. Considering all this, this paper aims at showing how
17
18 successful classification models can be created using Hyperspectral-NIR cameras to discriminate
19
20
21 between different plastics containing variable amounts of BFRs. This will allow us to create
22
23 different models using NIR for automatic sorting of plastics with critical additives at a rate of
24
25
26
several parts per second in order to implement an economically feasible recycling process that
27
28 meets the major requirement of the industry.
29
30
31
32
33
34
35 2 Materials and methods
36
37
38 2.1 Materials
39
40
41 The plastics used in this study were kindly provided by the INNOSORT consortium
42
43 (http://innosort.teknologisk.dk/). Five different plastics were supplied (Figure 1, left): acrylonitrile
44
45
46
butadiene styrene (ABS), polyamide 6 (PA6), polybutylene terephthalate (PBT), polypropylene
47
48 (PP) and polystyrene (PS). These plastics were produced in the shape of small pellets (of around 5
49
50 millimeters of diameter). The reference plastic for each category (i.e. without BFRs addition) was
51
52
53 labeled as REF. All plastics were also doped in the manufacturing process with variable amounts of
54
55 brominated flame retardants (BFRs) (Figure 1, right) (due to confidentiality reasons and different
56
57
58 research ongoing, the type of BFR will not be provided). Despite this fact, we can say that the
59
60
61
62 6
63
64
65
 1 doped amount of BFR was between the boundaries of the corresponding legislation). In total, they
2
3
4 comprised 14 different categories of plastics (5 main categories and 8 sub-categories).
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
Figure 1. Left, pellets of the different plastics measured. Right, the identification of each group and sub-groups of
35
36
plastics. The numbers are only for indication of groups and sub-groups should not be taken as any quantity.
37
38
39
40
41
42 2.2 Measurements
43
44
The NIR-Hyperspectral images of this work were obtained with a spectrometer (Headwall
45
46
47 photonics model 1002A-00371) working in the wavelength range of 1000 - 1700 nm with a spectral
48
49 resolution of 7 nm (a total of 142 bands were recorded for each spectrum). The spectrometer was
50
51
52 adapted to a line mapping configuration with a line of 320 pixels. This hyperspectral camera was
53
54 kindly provided by FOSS (FOSS A/S, Denmark).
55
56
57
58
59
60
61
62 7
63
64
65
 1 Depending of the type of spectral radiation and the setup of the detectors, they must be calibrated in
2
3
4 a different way. Always make sure of performing a proper calibration of the camera. In this case,
5
6 the camera was calibrated using a spectralon plate. If the correction of the hypercube with the
7
8 reference and dark current is performed automatically by the software, be sure that is performed
9
10
11 correctly [59-61]. In this work, spectra were recorded in the reflectance mode giving a final pixel
12
13 resolution of 300 m.
14
15
16
17
18
19 2.3 Software
20
21
22
Use the most adequate software for your needs. If you have programming skills, we highly
23
24 recommend open source software [62, 63] in which is very common to find libraries for
25
26 hyperspectral analysis. The same fact occurs with Matlab [64]. Despite the fact that Matlab is not
27
28
29 open source, it allows programming and using external toolboxes like, e.g. MIA toolbox [65]. If, on
30
31 the contrary, you want to keep yourself away from programming, software like ENVI [66], for
32
33
34 example, will help you. There are also some others software available that are free. But no matter
35
36 which software you use, be always aware of how the different multivariate methods are
37
38
39 implemented. When you use open code software, this is not much problem. You use your own
40
41 libraries or you can check what other researchers have programmed. This, though, is more difficult
42
43 to check in commercial software (see ref [67] for an example).
44
45
46
47
48 In this case, the image data processing was performed by using HYPER-Tools, an in-house library
49
50
51 working under Matlab [64] and freely available upon request (jmar@life.ku.dk for last version).
52
53 Classification models were performed by using Partial Least Squares-Discriminant Analysis (PLS-
54
55
56 DA) implemented in the PLS_Toolbox [68]. Moreover, the image processing methods used in the
57
58 last section were applied using the Image Processing toolbox [69]. This seems a very complicated
59
60
61
62 8
63
64
65
 1 combination of software. Nevertheless, the three of them (HYPER-Tools, PLS_Toolbox and Image
2
3
4 processing toolbox) work under Matlab environment and, moreover, all their utilities can be used in
5
6 an automatic manner by means of in house-generated scripts.
7
8
9
10
11 3 Structure of hyperspectral images
12
13 The final result of measuring a hyperspectral image is a 3-D dataset, known as a spectral hypercube
14
15 (Figure 2, top left). In the structure of the 3-D dataset X (MN), the M and N axes represent
16
17
18 spatial information, while represents the spectral dimension.
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51 Figure 2. Structure of a hypercube. Top left: Depiction of a hypercube obtained for the plastics. Several pixels
52
53
belonging to different plastics plus the background are selected and their corresponding spectral profile shown in the top
54
55
right figure. The bottom figure represents the hyperspectral image obtained for two different wavelengths
56
57
(intensity/false color images) without the influence of the background.
58
59
60
61
62 9
63
64
65
 1
2
3
4 This data structure usually comprises thousands of spectra (spectral signature) distributed over the
5
6 measured area (spatial signature). Therefore, understanding the structure of your hyperspectral
7
8 image is essential to select the proper tools for dealing with the final aim. It has not to forget that
9
10
11 both spectral and spatial signatures contain the interesting analytical information and, at the same
12
13 time, different issues that must be handled in a particular and tailored manner. Moreover, both
14
15
16 signatures are inherently linked to each other. That is, the spatial distribution of a specific
17
18 compound will be fully assessed if the spectral signature of the same compound is selective enough
19
20
21 to be independently extracted from the rest, and vice versa. Assessing the spatial distribution of
22
23 elements in one surface (e.g. studying homogeneity or distribution of elements) is not an easy task
24
25
26
and is far out from the scope of this manuscript. We will be more focused on the characteristics of
27
28 the spectral signature.
29
30
31
32
33 Each pixel represents a spectrum that is the sum of the absorbances/influences/counts of all the
34
35 compounds (k = 1K) present in that particular pixel plus the unavoidable spectral noise. Taking
36
37
38 NIR as example, the absorbance for a single pixel mn at one wavelength can be postulated as in
39
40 equation 1:
41
42
43
44
45 amn = 1*C1mn + 2*C2mn + .. + k*Ckmn + emn (1)
46
47
48
49
50 where is a constant that depends on the respective wavelength, C is the concentration of the k
51
52 absorbing compounds in the specific sample, and e is the noise. This equation can be generalized
53
54
55 for every single pixel in all wavelengths (equation 2):
56
57
58
59
60
61
62 10
63
64
65
 1 amn = 1*C1mn + 2*C2mn + .. + k*Ckmn + emn (2)
2
3
4
5
6 Nevertheless, we should note that a single pixel is not a single sample. It is, indeed, strongly
7
8 influenced by different issues:
9
10
11 - Irregularities and shape of the surface: the surfaces measured with hyperspectral cameras are
12
13 seldom totally flat. This, of course, depends on the final resolution of the image. As a simple
14
15
16 example, we could mention the roughness of a tablet, the shape of an orange or the shades that a
17
18 mountain creates. This affects the light intensity that one pixel receives and, therefore, its
19
20
21 reflectance spectrum will be highly influenced by this.
22
23 - Penetration depth of the electromagnetic radiation: mostly, we tend to think that the measurement
24
25
26
only concerns to the surface of one sample. Nevertheless, vibrational spectroscopy has certain
27
28 penetration depth [70]. This makes that a pixel usually contains not only information of the surface
29
30 layer of the sample, but also of the most adjacent internal layers, as well as the adjacent pixels.
31
32
33 - Resolution: It is seldom found that a pixel contains selective information of one component,
34
35 especially in food samples. For example, tablets are usually measured with a pixel resolution that is
36
37
38 bigger than the particles sizes. Therefore, the pixel will contain mixed information of all the
39
40 components present on it.
41
42
43
44
45
46
47 4 General scheme to perform hyperspectral images analysis
48
49
50
51
52 Acknowledging that chemometrics is currently a very well-known discipline, it has been used for
53
54 many years on hyperspectral images as well. This has generated that, depending on the purpose,
55
56
57 different procedures can be chosen. This is depicted as a general flow-chart in Figure 3.
58
59
60
61
62 11
63
64
65
 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26 Figure 3. Comprehensive flow-chart for the analysis of hyperspectral images. Partially extracted and modified from [16]
27
28 with permission of Elsevier.
29
30
31
32
33
34 The main building blocks of the flow-chart on Figure 3 are: 1) Spatial and spectral pre-processing,
35
36 2) exploration, 3) resolution and/or regression, 4) segmentation and 5) final image processing.
37
38
39 Every step has been widely defined in several reviews [5-8, 16], giving a comprehensive
40
41 explanation of every step and several applications. This manuscript will focus on classification
42
43
44 methods. Specifically, class-modelling methods in a hierarchical manner (Figure 4):
45
46 1) In a first step, the plastics were separated according to their different composition of the main
47
48
49
class.
50
51 2) The second step involved the performance of individual models for each plastic to discriminate
52
53 between plastics containing flame retardant.
54
55
56
57
58
59
60
61
62 12
63
64
65
 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28 Figure 4. Two steps strategy for plastic sorting. The different categories are defined in material and methods section.
29
30
31
32
33
34 To accomplish the final aim, there are several steps that involve the usage of different algorithms.
35
36 Therefore, we will expose the main features of every single step in further sections.
37
38
39
40
41
42
43 5 Classification model development for the five major categories of plastics
44
45
46
47 5.1 Hypercube pre-processing
48
49 Following the scheme on Figure 3, the first step, in our case, is to remove the background in order
50
51
52
to analyze only the pixels that belong to plastics.
53
54 - Spatial pre-processing: Usually, this part is also linked to the selection of the Regions Of Interest
55
56 (ROIs) in the image, also referred as masking the non-interesting areas. Since eliminating the
57
58
59 unwanted pixels is not physically possible (otherwise the 3D structure would be lost), the most
60
61
62 13
63
64
65
 1 common procedure is creating a mask, with the same spatial dimensions than the image, but
2
3
4 containing information for every single pixel concerning its selection for subsequent analysis. In
5
6 Matlabs language, this mask is a logical matrix containing 0s and 1s, denoting no-selection and
7
8 selection, respectively. There are many different methods to perform masking that strongly depend
9
10
11 on the signal of both the background and the elements in the image. In our case, the black
12
13 background was removed using K-means algorithm applied on the derivative spectra. Note to be
14
15
16 clear that this does not mean that for further analysis we will be obliged to use the derivative
17
18 spectra.
19
20
21 - Spectral pre-processing: As said before, this operation depends on the subsequent analysis. In
22
23 this case, the spectra were pre-processed by using Standard Normal Variate (SNV) [71] and
24
25
26
smoothing (Savitzky -Golay filter with window size of 7 points and second degree polynomia). The
27
28 final pre-processed spectra are shown in Figure 5. Obviously, after de-noising and/or highlighting
29
30 the spectral information, they must be normalized to leverage the importance of all the spectral
31
32
33 variables. Therefore, the final (and almost mandatory when continuous spectra are measured)
34
35 operation is the application of mean centering of the spectra, previous to the analysis. Be extremely
36
37
38 careful with over pre-processing and also with the type of pre-processing that is applied [47].
39
40
41
42
43
Observing carefully the selected ROIs in the image generated at 1207 nm (Figure 5, bottom) it can
44
45 be noticed that there are still pixels that belong to the background within several pellets. These
46
47 pixels are within pellets and, therefore, they are sometimes hard to eliminate. This is the problem
48
49
50 when objects are close each other or even touching. The different spectra of the objects affect the
51
52 spectra of the background within the objects, making the task of background removal quite complex
53
54
55 if an automated method is applied. For now, we will keep those pixels in the analysis, since it is
56
57
58
59
60
61
62 14
63
64
65
 1 interesting to see how the classification model handles them. In the last section of the manuscript
2
3
4 we will remove them and evaluate the difference.
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35 Figure 5. Top, 20 random spectra of the pre-processed image. Bottom, normalized false color image taken at 1207 nm.
36
37
38
39
40
41
42 5.2 PLS-DA model performance
43
44 Once the spectra are properly pre-processed and normalized, the first classification model was
45
46
47 developed only considering the 5 major categories: ABS, PS, PA6, PP and PBT. As mentioned
48
49 before, the classification models were developed using PLS-DA [72]. It is wrongly thought that
50
51
PLS-DA models can be performed when only two classes need to be separated. In a multi-class
52
53
54 task, PLS-DA models are usually performed by using PLS2 algorithm. Due to the high amount of
55
56 pixels, 75% of them, randomly selected for each class, were used to build the model. The model
57
58
59 was cross-validated using random subsets cross-validation approach. Moreover, and considering the
60
61
62 15
63
64
65
 1 high amount of pixels for each class (an inherent advantage of images), an external test set of pixels
2
3
4 was also created by using 25% of the pixels randomly selected for each class. As a matter of fact, it
5
6 never hurts to remember that all multivariate models developed for classification and regression
7
8 must be validated. Validation is the main tool that tells us how our model predicts, since a good
9
10
11 model is not the one that better calibrates, but better predicts. Usually, cross-validation is the correct
12
13 choice. But profiting the great amount of pixels in hyperspectral images, it is worth implementing a
14
15
16 validation with an external set of pixels that have not been used for model development.
17
18
19
20
21 The results, as expected, were of high quality in terms of classification performance. In Figure 6a it
22
23 is very easy to recognize that the 5 plastics have been perfectly separated in the score space spanned
24
25
26
by the first 3 LVs space of the scores.
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62 16
63
64
65
 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35 Figure 6. a) Scores, b) loadings and c) VIP vectors plots of the general PLS-DA model
36
37
38
39
40 One aspect that usually is not thoughtfully investigated is the extremely rich information that
41
42
43 loadings offer (Figure 6b). Indeed, they allow us to understand the real reason of the separation of
44
45 the plastics. Observing them, together with the score plot, can give an idea of which variables are
46
47
48 the most important ones in the separation of the classes. Moreover, every time we develop a
49
50 targeted multivariate model, there will be the option of performing variable selection for improving
51
52
53 the models. Although this is not the scope of this manuscript, it is worth exploring some methods
54
55 which assess the variables responsible for the separation of the groups in a more statistical manner.
56
57 In this case, the software that we have used contains the Variable Importance in Projection (VIP)
58
59
60
61
62 17
63
64
65
 1 score for the loadings [73], denoting that variables with a VIP value higher than one are the ones
2
3
4 (Figure 6c) to be chosen in further variable selection.
5
6
7
8
9 5.3 Statistical assessment of the model
10
11
12 The statistical assessment of the classification performance can be done by using different
13
14 classifiers [74-76]. In our case, the model was statistically validated by using the sensitivity,
15
16
17 specificity and class error for the calibration (CAL), cross-validation (CV) and external (PRED) sets
18
19 (Table 1), as well as the Receiver Operator Characteristics (ROC) curves.
20
21
22 Table 1. Sensitivity, specificity ad classification error for the five main categories of plastics
23
24
25 GLOBAL MODEL
26
27
28 PA6 PBT PP PS ABS
29
30 Sensitivity (CAL) 0,945 0,993 0,964 0,991 0,985
31
32
33 Sensitivity (CV) 0,946 0,993 0,963 0,991 0,983
34
35 Sensitivity (PRED) 0,864 0,996 0,967 0,983 0,972
36
37
38
39
40 Specificity (CAL) 0,964 0,997 0,976 0,998 0,979
41
42 Specificity (CV) 0,965 0,997 0,976 0,998 0,979
43
44
Specificity (PRED) 0,957 0,995 0,96 0,993 0,962
45
46
47
48
49 Class error (CAL) 0,044 0,004 0,03 0,005 0,018
50
51
52 Class error (CV) 0,044 0,005 0,03 0,005 0,019
53
54 Class error (PRED) 0,089 0,004 0,036 0,011 0,032
55
56
57
58
59
60
61
62 18
63
64
65
 1 The statistical results are, indeed, very impressive. The first point to observe is the high similarity
2
3
4 between the values obtained for calibration, cross-validation and external prediction. This indicates
5
6 that the models are not under or over-fitted. Apart from that, the sensitivity and specificity values
7
8 obtained for every single plastic in external prediction were greater than 0.95 (except the sensitivity
9
10
11 of PA6). That means that there is a probability higher than 95% of classifying correctly every single
12
13 pixel in its corresponding category. This is also confirmed by the ROC curves obtained for the
14
15
16 prediction of the external set of pixels (Figure 7).
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56 Figure 7. ROC curves obtained for the external prediction set of the first model.
57
58
59
60
61
62 19
63
64
65
 1 Regarding the pixels within pellets, it is remarkable to see how most of them have been assigned to
2
3
4 belong to the corresponding plastic. This may lead to problems in terms of accuracy of the model,
5
6 since we know that those pixels belong to the background. It is also important to note how the
7
8 classification was performed in the calibration model (Figure 8). As observed in this figure, there
9
10
11 are a small proportion of unassigned pixels (deep blue pixels in Figure 8). Those pixels are mostly
12
13 located in the edges of the pellets. This is due to the different surface characteristics of the plastic
14
15
16 pellets, which are not completely flat. Despite the pre-processing steps, total suppression of artifacts
17
18 coming from the roughness and the shape of the surfaces is not always possible.
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
Figure 8. External prediction of the standard sample to verify the usefulness of the global PLS-DA model.
49
50
51
52
53 6 Classification models for each individual category
54
55 Once every plastic class has been separated from the rest, the second step is the development of
56
57
58 PLS-DA models for every category to classify the plastics according to their content of flame
59
60
61
62 20
63
64
65
 1 retardant. For modelling every sub-category of each plastic, different number of LVs and different
2
3
4 spectral pre-processing methods were needed in order to obtain optimal results. This was tested
5
6 with the same validation procedure as before: cross-validation based on random subsets selection
7
8 and external test prediction. The parameters for each plastic and the optimal number of LVs are
9
10
11 displayed in Table 2.
12
13
14 Table 2. PLS-DA model parameters for every plastic category.
15
16
17 Plastic LVs Pre-treatment
18
19
20
ABS 7 SNV + Second derivative
21
22 PS 8 SNV + smoothing
23
24 PBT 6 SNV + smoothing
25
26
27 PA6 6 SNV + First derivative
28
29 PP 8 SNV + smoothing
30
31
32
33
34 The results for the classification parameters are shown in
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62 21
63
64
65
 1 Table 3. This table clearly shows that PA6 and PP can be successfully separated from the
2
3
4 corresponding plastics containing flame retardant. Acceptable results were also obtained for PBT,
5
6 ABS and PS, but not as good as with PA6 and PP.
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62 22
63
64
65
 1 Table 3. Sensitivity, specificity and error rate for the PLS-DA models developed for each plastic
2
3
4 ABS PS
5
6
ABS10 ABS15 ABS02 ABS PS00 PS07 PS09 PS
7
8
9 Sensitivity (CAL) 0,892 0,87 0,831 0,755 0,931 0,878 0,698 0,81
10
11 Sensitivity (CV) 0,891 0,87 0,826 0,752 0,924 0,873 0,693 0,812
12
13
14
15
16 Specificity (CAL) 0,57 0,837 0,737 0,635 0,83 0,763 0,65 0,842
17
18 Specificity (CV) 0,569 0,837 0,737 0,631 0,83 0,762 0,648 0,841
19
20
21
22
23 Class error (CAL) 0,269 0,146 0,215 0,304 0,119 0,179 0,325 0,174
24
25
26 Class error (CV) 0,27 0,146 0,218 0,308 0,122 0,182 0,329 0,173
27
28
29
PBT PA6 PP
30
31
32 PBT010 PBT PA6010 PA6 PP202 PP
33
34 Sensitivity (CAL) 0,807 0,599 0,856 0,893 0,945 0,944
35
36
37 Sensitivity (CV) 0,798 0,603 0,852 0,895 0,944 0,945
38
39
40
41
Specificity (CAL) 0,599 0,807 0,893 0,856 0,944 0,945
42
43
44 Specificity (CV) 0,603 0,798 0,895 0,852 0,944 0,944
45
46
47
48
49 Class error (CAL) 0,296 0,296 0,125 0,125 0,055 0,055
50
51 Class error (CV) 0,299 0,299 0,126 0,126 0,055 0,055
52
53
54
55
56 Looking at the prediction results on the image (Figure 9), it can be observed that the amount of
57
58 pixels not classified in their corresponding class for ABS and PS is extremely high. Also, focusing
59
60
61
62 23
63
64
65
 1 on the ROC curves obtained for ABS (Figure 10), there is a strong confounding effect within the
2
3
4 subclasses.
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35 Figure 9. Cross-validated prediction for every plastic and their corresponding version containing flame-retardant.
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62 24
63
64
65
 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38 Figure 10. ROC curves obtained for the external prediction set of the first model.
39
40
41
42 The reasons for the not totally satisfactory performance of the models for ABS and PS could be the
43
44
45 high similarity between the spectral signature of the sub-classes, the roughness of the pellets and the
46
47 presence of pixels belonging to the background.
48
49
50
51
52
53
54 7 From pixel-wise to object-wise projection
55
56
57 As we have mentioned before, there are still several background pixels that were not totally
58
59 eliminated by the automatic mask procedure applied. This is sometimes difficult, since the spectral
60
61
62 25
63
64
65
 1 influence of the pellets around the background pixels make their spectra closely similar to those
2
3
4 ones of the pellets.
5
6
7
8 One of the main advantages of using hyperspectral images is that - classical methods of digital
9
10
11 image processing can be implemented to improve the classification performance in our final
12
13 outcomes. In our case, there are still two problems concerning the classification models: the pixels
14
15
16 that are within pellets, and the pixels belonging to pellets that have not been assigned to any class.
17
18 Therefore, we would aim at assigning an identification mark for each pellet, in such a way that all
19
20
21 the pixels belonging to those pellets are classified in the same class.
22
23
24
25
26
For solving the first issue, several manual procedures of digital image analysis can be applied. The
27
28 second issue is more related to the philosophy behind the aim of the separation. In our case, the
29
30 pellets used are of homogeneous composition. One solution might be assigning the whole pellet to
31
32
33 the most contributing class (the class that has the largest number of pixels in that pellet). It should
34
35 be noted that the solution makes sense only when the number of pixels assigned to the most
36
37
38 contributing class is high compared to the total number of pixels in the pellet. This condition helps
39
40 to avoid random assignments of the pellet to a class.
41
42
43
44
45
46 7.1 Suppression of the pixels within the pellets
47
48 There are as many methods for digital image processing as you can imagine. Those looking for a
49
50
51 good reference to start learning about some of the most important methods will find a good tool in
52
53 this book [77].
54
55
56
57
58
59
60
61
62 26
63
64
65
 1 In our case, the first step consists of using a picture that represents the shape of the pellets and the
2
3
4 contrast with the inner background pixels as good as possible. This can be done by selecting a
5
6 specific wavelength or the combination of usually three different wavelengths to create a false RGB
7
8 picture. For example, the false RGB picture shown in Figure 1 was constructed by equally dividing
9
10
11 the whole spectral interval into three equivalent intervals and averaging the absorbance values in
12
13 each pixel. This picture, thought, is not very clear. Therefore, three different wavelengths were
14
15
16 chosen from the hyperspectral image with the spectra in first derivative. The chosen wavelengths
17
18 were 1237, 1295 and 1441 nm, since they were the ones giving the highest contrast between the
19
20
21 background and the pellets. The final false RGB picture can be seen in Figure 11.
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45 Figure 11. a) False RGB image created by the combination of the images obtained at three different wavelengths and b)
46
47 the segmentation of the pellets with the contour in white.
48
49
50
51
52 We have chosen this region of the original image because almost all the pellets are touching each
53
54
55 other, and this represents a problem. There are many ways to tackle this problem. To perform it, we
56
57
58
59
60
61
62 27
63
64
65
 1 have used the image processing toolbox [69] working under Matlab environment. The one
2
3
4 represented here is one of the options, focused in three steps:
5
6 1) Transform the RGB picture into a grayscale picture.
7
8 2) Image adjustment of the color maps in such a way that the values between 0.4 and 1 (in intensity
9
10
11 of the corresponding RGB color map) map to values between 1 and 0.4, respectively.
12
13 3) Edge detection by using the Laplacian of Gaussian method [78].
14
15
16
17
18
19 7.2 PLS-DA results projected to the individual pellets
20
21
22
Once the edge detection was performed to the entire picture, the located pellets were individually
23
24 assessed by using the results of the PLS-DA model previously performed. The final result is shown
25
26 in Figure 11b. As it can be observed, the contour obtained for each pellet allows us to identify each
27
28
29 individual pellet in most of the cases. Moreover, the pixels belonging to the background have been
30
31 totally removed. Nevertheless, there are still some pellets that cannot be separated and, what is
32
33
34 more, one pellet has been split into two. Of course, this can be solved by a manual intervention on
35
36 the image. However, the point is to show that even when robust methods exist, they must be
37
38
39 optimized for every case and sometimes it is difficult to find a perfect solution.
40
41
42
43 This methodology was applied to the whole image and, as defined before, every pellet obtained was
44
45
46 treated as an individual for classification. The results of the application of the hierarchical model
47
48 can be seen in Figure 12.
49
50
51
52
53
54
55
56
57
58
59
60
61
62 28
63
64
65
 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34 Figure 12. PLS-DA classification results for the image considering the pellets as samples.
35
36
37
38
39
40
The first point of this result that catches the attention is that there are still some pellets that are
41
42 touching. Hence, there might be some miss-classification issues. Apart from this point, and with the
43
44 aim of showing another very practical tool in classification, Table 4 shows the confusion matrix
45
46
47 obtained in cross-validation (in this case, no external set was prepared, since the number of pellets
48
49 for each class is not very large). The confusion matrix gives information of which pellets have been
50
51
52 correctly classified (numbers in green in the table) and which ones have been misclassified
53
54 (numbers in red in the table). In this case, and analogously of what happens previously, there are
55
56
57
some pellets in the ABS and in the PS classes that have been misclassified within the corresponding
58
59 class. Moreover, there are also some pellets in PBT010 that have been classified as pure PBT.
60
61
62 29
63
64
65
 1
2
3
4 Table 4. Confusion matrix obtained for the PLS-DA model applied to the single pellets
5
6
7
8
Predicted class
9
10 ABS010 ABS015 ABS020 ABS PA6010 PA6 PBT010 PBT PP202 PP PS00 PS07 PS09 PS
11
ABS010 2 0 7 1 0 0 0 0 0 0 0 0 0 0
12
13 ABS015 0 11 0 9 0 0 0 0 0 0 0 0 0 0
14
15 ABS020 0 0 12 0 0 0 0 0 0 0 0 0 0 0
16
17 ABS 0 0 0 21 0 0 0 0 0 0 0 0 0 0
18 PA6010 0 0 0 0 8 0 0 0 0 0 0 0 0 0
19
20 PA6 0 0 0 0 0 22 0 0 0 0 0 0 0 0
21
22 PBT010 0 0 0 0 0 0 12 5 0 0 0 0 0 0
23
PBT 0 0 0 0 0 0 0 13 0 0 0 0 0 0
24
25 PP202 0 0 0 0 0 0 0 0 11 0 0 0 0 0
26
27 PP 0 0 0 0 0 0 0 0 0 16 0 0 0 0
28
29 PS00 0 0 0 0 0 0 0 0 0 0 8 0 0 4
30 PS07 0 0 0 0 0 0 0 0 0 0 0 8 0 4
Real class

31
32 PS09 0 0 0 0 0 0 0 0 0 0 0 0 16 0
33
34 PS 0 0 0 0 0 0 0 0 0 0 0 0 0 16
35
36
37
38
39
40
41
42
43 8 Remarks and final comments
44
45
46
47
48 This tutorial gives some ideas of how to deal with several issues concerning the development of
49
50 classification models in hyperspectral imaging in a practical and visual manner. In addition, we
51
52
53
have provided with key references and some hints of very fundamental aspects dealing with
54
55 hyperspectral image analysis. As stated at the beginning, every single case in hyperspectral
56
57
58
59
60
61
62 30
63
64
65
 1 modelling needs a tailored solution. Therefore, the methods employed in every step must be taken
2
3
4 as an optimal solution for this particular case.
5
6
7
8 Going through the results obtained, all the pellets were correctly classified between the different
9
10
11 main categories of plastics. Nevertheless, there are still some issues regarding the classification of
12
13 the pellets within the classes with and without flame retardant. This is mainly due to the lack of
14
15
16 difference between the pellets of the same plastic, but with different amount of flame retardant.
17
18 From a more applied point of view, the methodology proposed can be easily implemented in a
19
20
21 company for plastic sorting, since all the methods used herein can be easily automatized. Of course,
22
23 there is always room for improvement. To improve the classification results, the first action would
24
25
26
comprise a better separation of the pellets in a, for instance, a conveyor belt. The second action
27
28 would concern looking for increasing the wavelength range. Nowadays is very common to find NIR
29
30 cameras that work between 1000 and 2700 nm, for instance.
31
32
33
34
35 Finally, but not least important, we would like to remark that everyone should look for the
36
37
38 hyperspectral application and multivariate/digital image techniques that better suit to their purposes.
39
40 There are many commercial software and libraries that can be used, as well as many types of
41
42
43
cameras. Therefore, first, think in the application, aim of the analysis and the scientific problem and
44
45 answer yourself these questions:
46
47 - Do we really need hyperspectral imaging? To our knowledge, there are numerous applications
48
49
50 where the use of hyperspectral imaging is not totally necessary. Of course, the capability of fast
51
52 scanning is very attractive to obtain many spectra of a sample in short time. Nevertheless, this does
53
54
55 not justify the use of hyperspectral. Hyperspectral imaging should be used when the
56
57 characterization of the surface of a sample is required.
58
59
60
61
62 31
63
64
65
 1 - Which kind of spectral radiation do I need? Obviously, the NIR radiation is the workhorse in
2
3
4 many applications. Nevertheless, there are many cases in which visible imaging can be more than
5
6 enough.
7
8 - Which spatial resolution do I need? This depends on the type of sample to be measured. It is clear
9
10
11 that to characterize the distribution of an active pharmaceutical ingredient in a tablet, a spatial
12
13 resolution in micrometers is needed. But if the samples are big, we might only need spatial
14
15
16 resolutions of millimeters.
17
18 - Which spectral resolution do I need? There are many cases where the target can be achieved by
19
20
21 using two or three selective wavelengths. In this case, filter-based approaches might be wiser than
22
23 scanning a whole spectrum.
24
25
26
- Which models should I use? This is extremely important to assure the selection of the proper data
27
28 analysis tool. Moreover, in recent publications we have seen a trend of using non-linear methods by
29
30 default (e.g. support vector machines SVM). Nevertheless, as personal opinion, we prefer to keep
31
32
33 the analysis as simpler as possible. That is, if your problem can be solved by linear models, there is
34
35 no need to go for non-linear modelling.
36
37
38 - Which kind of software do I need? As explained before, this strongly depends on our ability with
39
40 handling big data. If punctual analysis wants to be performed, we might think in using commercial
41
42
43
software. Nevertheless, if our aim is to master in hyperspectral analysis we strongly advice of using
44
45 software that allows us to program our own routines.
46
47
48
49
50 9 Acknowledgements
51
52 This manuscript has been possible thanks to the financial support received from INNOSORT
53
54 project. INNOSORT is a consortium that brings together technology suppliers, research and
55
56
57
58
59
60
61
62 32
63
64
65
 1 knowledge institutions and recipients of waste separation techniques and separated materials. See
2
3
4 further information about INNOSORT and partners in http://innosort.teknologisk.dk/.
5
6
7
8 10 References
9
10
11
[1] R.A. Schowengerdt, Remote Sensing, Third Edition: Models and Methods for Image
12
13 Processing, Elsevier2006.
14
15 [2] E. Adam, O. Mutanga, D. Rugege, Multispectral and hyperspectral remote sensing for
16
17
18 identification and mapping of wetland vegetation: a review, Wetlands Ecology and Management, 18
19
20 (2010) 281-296.
21
22
23 [3] C.A. Lee, S.D. Gasster, A. Plaza, C.I. Chang, B. Huang, Recent Developments in High
24
25 Performance Computing for Remote Sensing: A Review, Ieee Journal of Selected Topics in Applied
26
27
28 Earth Observations and Remote Sensing, 4 (2011) 508-527.
29
30 [4] F.D. van der Meer, H.M.A. van der Werff, F.J.A. van Ruitenbeek, C.A. Hecker, W.H. Bakker,
31
32
33
M.F. Noomen, M. van der Meijde, E.J.M. Carranza, J.B. de Smeth, T. Woldai, Multi- and
34
35 hyperspectral geologic remote sensing: A review, International Journal of Applied Earth
36
37 Observation and Geoinformation, 14 (2012) 112-128.
38
39
40 [5] J.M. Amigo, J. Cruz, M. Bautista, S. Maspoch, J. Coello, M. Blanco, Study of pharmaceutical
41
42 samples by NIR chemical-image and multivariate analysis, Trac-Trend Anal Chem, 27 (2008) 696-
43
44
45 713.
46
47 [6] C. Gendrin, Y. Roggo, C. Collet, Pharmaceutical applications of vibrational chemical imaging
48
49
50
and chemometrics: a review, J Pharm Biomed Anal, 48 (2008) 533-553.
51
52 [7] A.A. Gowen, J.M. Amigo, Applications of Spectroscopy and Chemical Imaging in
53
54 Pharmaceutics, in: J. Popp, V.V. Tuchin, A. Chiou, S.H. Heinemann (Eds.) Handbook of
55
56
57
58
59
60
61
62 33
63
64
65
 1 Biophotonics, Volume 3: Photonics in Pharmaceutics, Bioanalysis and Environmental Research,
2
3
4 Wiley2012.
5
6 [8] P.Y. Sacr, C. De Bleye, P.F. Chavez, L. Netchacovitch, P. Hubert, E. Ziemons, Data
7
8 processing of vibrational chemical imaging for pharmaceutical applications, Journal of
9
10
11 Pharmaceutical and Biomedical Analysis, 101 (2014) 123-140.
12
13 [9] J.M. Amigo, Practical issues of hyperspectral imaging analysis of solid dosage forms, Anal
14
15
16 Bioanal Chem, 398 (2010) 93-109.
17
18 [10] Y. Roggo, A. Edmond, P. Chalus, M. Ulmschneider, Infrared hyperspectral imaging for
19
20
21 qualitative analysis of pharmaceutical solid forms, Anal Chim Acta, 535 (2005) 79-87.
22
23 [11] C. Cairs, J.M. Amigo, R. Watt, J. Coello, S. Maspoch, Implementation of enhanced
24
25
26
correlation maps in near infrared chemical images: Application in pharmaceutical research, Talanta,
27
28 79 (2009) 657-664.
29
30 [12] S. Sasic, D.A. Clark, J.C. Mitchell, M.J. Snowden, Raman line mapping as a fast method for
31
32
33 analyzing pharmaceutical bead formulations, Analyst, 130 (2005) 1530-1536.
34
35 [13] M. Khorasani, J.M. Amigo, J. Sonnergaard, P. Olsen, P. Bertelsen, J. Rantanen, Visualization
36
37
38 and prediction of porosity in roller compacted ribbons with near-infrared chemical imaging (NIR-
39
40 CI), Journal of Pharmaceutical and Biomedical Analysis, 109 (2015) 11-17.
41
42
43
[14] C. Ravn, E. Skibsted, R. Bro, Near-infrared chemical imaging (NIR-CI) on pharmaceutical
44
45 solid dosage forms-comparing common calibration approaches, J Pharm Biomed Anal, 48 (2008)
46
47 554-561.
48
49
50 [15] J. Cruz, M. Bautista, J.M. Amigo, M. Blanco, Nir-chemical imaging study of acetylsalicylic
51
52 acid in commercial tablets, Talanta, 80 (2009) 473-478.
53
54
55
56
57
58
59
60
61
62 34
63
64
65
 1 [16] J.M. Amigo, I. Mart, A. Gowen, Hyperspectral Imaging and Chemometrics. A Perfect
2
3
4 Combination for the Analysis of Food Structure, Composition and Quality, Data Handling in
5
6 Science and Technology, 2013, pp. 343-370.
7
8 [17] G. ElMasry, M. Kamruzzaman, D.W. Sun, P. Allen, Principles and Applications of
9
10
11 Hyperspectral Imaging in Quality Evaluation of Agro-Food Products: A Review, Critical Reviews
12
13 in Food Science and Nutrition, 52 (2012) 999-1023.
14
15
16 [18] G. Elmasry, N. Wang, A. ElSayed, M. Ngadi, Hyperspectral imaging for nondestructive
17
18 determination of some quality attributes for strawberry, J Food Eng, 81 (2007) 98-107.
19
20
21 [19] P.M. Mehl, Y.-R. Chen, M.S. Kim, D.E. Chan, Development of hyperspectral imaging
22
23 technique for the detection of apple surface defects and contaminations, J Food Eng, 61 (2004) 67-
24
25
26
81.
27
28 [20] D. Lorente, N. Aleixos, J. Gomez-Sanchis, S. Cubero, O.L. Garcia-Navarrete, J. Blasco,
29
30 Recent Advances and Applications of Hyperspectral Imaging for Fruit and Vegetable Quality
31
32
33 Assessment, Food and Bioprocess Technology, 5 (2012) 1121-1142.
34
35 [21] D. Sun, Hyperspectral Imaging for Food Quality Analysis and Control, Elsevier Inc.2010.
36
37
38 [22] A.A. Gowen, C.P. O'Donnell, P.J. Cullen, G. Downey, J.M. Frias, Hyperspectral imaging - an
39
40 emerging process analytical tool for food quality and safety control, Trends in Food Science &
41
42
43
Technology, 18 (2007) 590-598.
44
45 [23] J.H. Cheng, D.W. Sun, Hyperspectral imaging as an effective tool for quality analysis and
46
47 control of fish and other seafoods: Current research and potential applications, Trends in Food
48
49
50 Science and Technology, 37 (2014) 78-91.
51
52 [24] S. Cubero, N. Aleixos, E. Molt, J. Gmez-Sanchis, J. Blasco, Advances in Machine Vision
53
54
55 Applications for Automatic Inspection and Quality Evaluation of Fruits and Vegetables, Food and
56
57 Bioprocess Technology, 4 (2011) 487-504.
58
59
60
61
62 35
63
64
65
 1 [25] I. Kim, M.S. Kim, Y.R. Chen, S.G. Kong, Detection of skin tumors on chicken carcasses using
2
3
4 hyperspectral fluorescence imaging, Transactions of the Asae, 47 (2004) 1785-1792.
5
6 [26] Y.Z. Feng, D.W. Sun, Application of Hyperspectral Imaging in Food Safety Inspection and
7
8 Control: A Review, Critical Reviews in Food Science and Nutrition, 52 (2012) 1039-1058.
9
10
11 [27] K. Flynn, R. O'Leary, C. Roux, B.J. Reedy, Forensic analysis of bicomponent fibers using
12
13 infrared chemical imaging, Journal of forensic sciences, 51 (2006) 586-596.
14
15
16 [28] G.J. Edelman, E. Gaston, T.G. van Leeuwen, P.J. Cullen, M.C.G. Aalders, Hyperspectral
17
18 imaging for non-contact analysis of forensic traces, Forensic Sci Int, 223 (2012) 28-39.
19
20
21 [29] M.A.F. De La Ossa, C. Garca-Ruiz, J.M. Amigo, Near infrared spectral imaging for the
22
23 analysis of dynamite residues on human handprints, Talanta, 130 (2014) 315-321.
24
25
26
[30] M.T. Fernndez de la Ossa, J.M. Amigo, C. Garca-Ruiz, Detection of residues from explosive
27
28 manipulation by near infrared hyperspectral imaging: A promising forensic tool, Forensic Sci Int,
29
30 242 (2014) 228-235.
31
32
33 [31] C.S. Silva, M.F. Pimentel, R.S. Honorato, C. Pasquini, J.M. Prats-Montalban, A. Ferrer, Near
34
35 infrared hyperspectral imaging for forensic analysis of document forgery, Analyst, 139 (2014)
36
37
38 5176-5184.
39
40 [32] K. Flynn, R. O'Leary, C. Lennard, C. Roux, B.J. Reedy, Forensic applications of infrared
41
42
43
chemical imaging: Multi-layered paint chips, Journal of forensic sciences, 50 (2005) 832-841.
44
45 [33] G.L. Lu, B.W. Fei, Medical hyperspectral imaging: a review, J Biomed Opt, 19 (2014).
46
47 [34] R.F. Kokaly, G.P. Asner, S.V. Ollinger, M.E. Martin, C.A. Wessman, Characterizing canopy
48
49
50 biochemistry from imaging spectroscopy and its application to ecosystem studies, Remote Sens
51
52 Environ, 113 (2009) S78-S91.
53
54
55
56
57
58
59
60
61
62 36
63
64
65
 1 [35] C.G. Zhang, H. Wang, J.F. Huang, Q. Gao, The visible to the near infrared narrow band
2
3
4 acousto-optic tunable filter and the hyperspectral microscopic imaging on biomedicine study, J
5
6 Optics-Uk, 16 (2014).
7
8 [36] M.E. Klein, B.J. Aalderink, R. Padoan, G. de Bruin, T.A.G. Steemers, Quantitative
9
10
11 hyperspectral reflectance imaging, Sensors, 8 (2008) 5576-5618.
12
13 [37] F. Rosi, C. Miliani, R. Braun, R. Harig, D. Sali, B.G. Brunetti, A. Sgamellotti, Noninvasive
14
15
16 Analysis of Paintings by Mid-infrared Hyperspectral Imaging, Angew Chem Int Edit, 52 (2013)
17
18 5258-5261.
19
20
21 [38] Z.M. Xu, E.Y. Lam, Image reconstruction using spectroscopic and hyperspectral information
22
23 for compressive terahertz imaging, J Opt Soc Am A, 27 (2010) 1638-1646.
24
25
26
[39] C.M. Wang, S.C. Yang, P.C. Chung, C.I. Chang, C.S. Lo, C.C. Chen, C.W. Yang, C.H. Wen,
27
28 Orthogonal subspace projection-based approaches to classification of MR image sequences,
29
30 Comput Med Imag Grap, 25 (2001) 465-476.
31
32
33 [40] S. Sasic, Y. Ozaki, Raman, Infrared, and Near-Infrared Chemical Imaging, Wiley2010.
34
35 [41] J.A.F. Pierna, P. Vermeulen, O. Amand, A. Tossens, P. Dardenne, V. Baeten, NIR
36
37
38 hyperspectral imaging spectroscopy and chemometrics for the detection of undesirable substances
39
40 in food and feed, Chemometr Intell Lab, 117 (2012) 233-239.
41
42
43
[42] P. Geladi, H. Grahn, M. Manley, Data Analysis and Chemometrics for Hyperspectral Imaging,
44
45 in: S. Sasic, Y. Ozaki (Eds.) Raman, Infrared, and Near-Infrared Chemical Imaging, Wiley2010.
46
47 [43] C.I. Chang, X. Jiao, C.C. Wu, Y. Du, M.L. Chang, A review of unsupervised spectral target
48
49
50 analysis for hyperspectral imagery, Eurasip Journal on Advances in Signal Processing, 2010 (2010).
51
52 [44] D.M. Haaland, H.D.T. Jones, M.H. Van Benthem, M.B. Sinclair, D.K. Melgaard, C.L. Stork,
53
54
55 M.C. Pedroso, P. Liu, A.R. Brasier, N.L. Andrews, D.S. Lidke, Hyperspectral Confocal
56
57
58
59
60
61
62 37
63
64
65
 1 Fluorescence Imaging: Exploring Alternative Multivariate Curve Resolution Approaches, Applied
2
3
4 Spectroscopy, 63 (2009) 271-279.
5
6 [45] J.M. Prats-Montalbn, A. de Juan, A. Ferrer, Multivariate image analysis: A review with
7
8 applications, Chemometr Intell Lab, 107 (2011) 1-23.
9
10
11 [46] B. Boldrini, W. Kessler, K. Rebnera, R.W. Kessler, Hyperspectral imaging: A review of best
12
13 practice, performance and pitfalls for in-line and on-line applications, Journal of Near Infrared
14
15
16 Spectroscopy, 20 (2012) 483-508.
17
18 [47] M. Vidal, J.M. Amigo, Pre-processing of hyperspectral images. Essential steps before image
19
20
21 analysis, Chemometr Intell Lab, 117 (2012) 138-148.
22
23 [48] G. Bonifazi, L. Damiani, S. Serranti, E.J. Bakker, P.C. Rem, Innovative Sensing Technologies
24
25
26
Applied to Post-Consumer Polyolefins Recovery, Metal Int, 14 (2009) 5-10.
27
28 [49] S. Serranti, A. Gargiulo, G. Bonifazi, Characterization of post-consumer polyolefin wastes by
29
30 hyperspectral imaging for quality control in recycling processes, Waste Manage, 31 (2011) 2217-
31
32
33 2227.
34
35 [50] S. Serranti, A. Gargiulo, G. Bonifazi, Classification of polyolefins from building and
36
37
38 construction waste using NIR hyperspectral imaging system, Resour Conserv Recy, 61 (2012) 52-
39
40 58.
41
42
43
[51] A. Ulrici, S. Serranti, C. Ferrari, D. Cesare, G. Foca, G. Bonifazi, Efficient chemometric
44
45 strategies for PET-PLA discrimination in recycling plants using hyperspectral imaging, Chemometr
46
47 Intell Lab, 122 (2013) 31-39.
48
49
50 [52] C.A. de Wit, An overview of brominated flame retardants in the environment, Chemosphere,
51
52 46 (2002) 583-624.
53
54
55 [53] Shimadtzu, Identification of brominated flame retardants in polymers, Shimadtzu News,
56
57 Shimatzu, 2005, pp. 10-11.
58
59
60
61
62 38
63
64
65
 1 [54] S. Kikuchi, K. Kawauchi, S. Ooki, M. Kurosawa, H. Honjho, T. Yagishita, Non-destructive
2
3
4 rapid analysis of brominated flame retardants in electrical and electronic equipment using Raman
5
6 spectroscopy, Anal Sci, 20 (2004) 1111-1112.
7
8 [55] M. Stepputat, R. Noll, On-line detection of heavy metals and brominated flame retardants in
9
10
11 technical polymers with laser-induced breakdown spectrometry, Applied Optics, 42 (2003) 6210-
12
13 6220.
14
15
16 [56] A. Golloch, D. Siegmund, Sliding spark spectroscopy - rapid survey analysis of flame
17
18 retardants and other additives in polymers, Fresen J Anal Chem, 358 (1997) 804-811.
19
20
21 [57] C. Gallen, A. Banks, S. Brandsma, C. Baduel, P. Thai, G. Eaglesham, A. Heffernan, P.
22
23 Leonards, P. Bainton, J.F. Mueller, Towards development of a rapid and effective non-destructive
24
25
26
testing strategy to identify brominated flame retardants in the plastics of consumer products,
27
28 Science of the Total Environment, 491 (2014) 255-265.
29
30 [58] M. Schlummer, F. Brandl, A. Maurer, R. van Eldik, Analysis of flame retardant additives in
31
32
33 polymer fractions of waste of electric and electronic equipment (WEEE) by means of HPLC-
34
35 UV/MS and GPC-HPLC-UV, J Chromatogr A, 1064 (2005) 39-51.
36
37
38 [59] J. Burger, P. Geladi, Hyperspectral NIR image regression part II: Dataset preprocessing
39
40 diagnostics, J Chemometr, 20 (2006) 106-119.
41
42
43
[60] J. Burger, P. Geladi, Hyperspectral NIR image regression part I: calibration and correction, J
44
45 Chemometr, 19 (2005) 355-363.
46
47 [61] P. Geladi, J. Burger, T. Lestander, Hyperspectral imaging: calibration problems and solutions,
48
49
50 Chemometr Intell Lab, 72 (2004) 209-217.
51
52 [62] Phyton, https://www.python.org/.
53
54
55 [63] RStudio, http://www.rstudio.com/.
56
57 [64] I. The MathWorks, http://se.mathworks.com/.
58
59
60
61
62 39
63
64
65
 1 [65] M.T.b.E.R. Inc., http://www.eigenvector.com/software/mia_toolbox.htm.
2
3
4 [66] ENVI, http://www.exelisvis.com/ProductsServices/ENVIProducts/ENVI.aspx.
5
6 [67] J.M. Amigo, C. Ravn, N.B. Gallagher, R. Bro, A comparison of a common approach to partial
7
8 least squares-discriminant analysis and classical least squares in hyperspectral imaging, Int J
9
10
11 Pharmaceut, 373 (2009) 179-182.
12
13 [68] E.R. Inc., http://eigenvector.com/.
14
15
16 [69] I. The MathWorks, Image Processing Toolbox, 2015.
17
18 [70] F.M. Mirabella, Internal-Reflection Spectroscopy, Appl Spectrosc Rev, 21 (1985) 45-178.
19
20
21 [71] A. Rinnan, F. van den Berg, S.B. Engelsen, Review of the most common pre-processing
22
23 techniques for near-infrared spectra, Trac-Trend Anal Chem, 28 (2009) 1201-1222.
24
25
26
[72] M. Barker, W. Rayens, Partial least squares for discrimination, J Chemometr, 17 (2003) 166-
27
28 173.
29
30 [73] I.G. Chong, C.H. Jun, Performance of some variable selection methods when multicollinearity
31
32
33 is present, Chemometr Intell Lab, 78 (2005) 103-112.
34
35 [74] D. Lorente, J. Blasco, A.J. Serrano, E. Soria-Olivas, N. Aleixos, J. Gomez-Sanchis,
36
37
38 Comparison of ROC Feature Selection Method for the Detection of Decay in Citrus Fruit Using
39
40 Hyperspectral Images, Food and Bioprocess Technology, 6 (2013) 3613-3619.
41
42
43
[75] J. Demsar, Statistical comparisons of classifiers over multiple data sets, J Mach Learn Res, 7
44
45 (2006) 1-30.
46
47 [76] D.J. Hand, Assessing the Performance of Classification Methods, Int Stat Rev, 80 (2012) 400-
48
49
50 414.
51
52 [77] R.C. Gonzalez, R.E. Woods, E.L. Eddins, Digital Image Processing Using MATLAB,
53
54
55 Gatesmark Publishing2009.
56
57
58
59
60
61
62 40
63
64
65
 1 [78] J. Canny, A Computational Approach to Edge-Detection, Ieee T Pattern Anal, 8 (1986) 679-
2
3
4 698.
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62 41
63
64
65
Tables

Table 1. Sensitivity, specificity ad classification error for the five main categories of plastics

GLOBAL MODEL
PA6 PBT PP PS ABS
Sensitivity (CAL) 0,945 0,993 0,964 0,991 0,985
Sensitivity (CV) 0,946 0,993 0,963 0,991 0,983
Sensitivity (PRED) 0,864 0,996 0,967 0,983 0,972

Specificity (CAL) 0,964 0,997 0,976 0,998 0,979

Specificity (CV) 0,965 0,997 0,976 0,998 0,979
Specificity (PRED) 0,957 0,995 0,96 0,993 0,962

Class error (CAL) 0,044 0,004 0,03 0,005 0,018

Class error (CV) 0,044 0,005 0,03 0,005 0,019
Class error (PRED) 0,089 0,004 0,036 0,011 0,032
Table 2. PLS-DA model parameters for every plastic category.

Plastic LVs Pre-treatment

ABS 7 SNV + Second derivative
PS 8 SNV + smoothing
PBT 6 SNV + smoothing
PA6 6 SNV + First derivative
PP 8 SNV + smoothing
Table 3. Sensitivity, specificity and error rate for the PLS-DA models developed for each plastic

ABS PS
ABS10 ABS15 ABS02 ABS PS00 PS07 PS09 PS
Sensitivity (CAL) 0,892 0,87 0,831 0,755 0,931 0,878 0,698 0,81
Sensitivity (CV) 0,891 0,87 0,826 0,752 0,924 0,873 0,693 0,812

Specificity (CAL) 0,57 0,837 0,737 0,635 0,83 0,763 0,65 0,842
Specificity (CV) 0,569 0,837 0,737 0,631 0,83 0,762 0,648 0,841

Class error (CAL) 0,269 0,146 0,215 0,304 0,119 0,179 0,325 0,174
Class error (CV) 0,27 0,146 0,218 0,308 0,122 0,182 0,329 0,173

PBT PA6 PP
PBT010 PBT PA6010 PA6 PP202 PP
Sensitivity (CAL) 0,807 0,599 0,856 0,893 0,945 0,944
Sensitivity (CV) 0,798 0,603 0,852 0,895 0,944 0,945

Specificity (CAL) 0,599 0,807 0,893 0,856 0,944 0,945

Specificity (CV) 0,603 0,798 0,895 0,852 0,944 0,944

Class error (CAL) 0,296 0,296 0,125 0,125 0,055 0,055

Class error (CV) 0,299 0,299 0,126 0,126 0,055 0,055
Table 4. Confusion matrix obtained for the PLS-DA model applied to the single pellets

Predicted class
ABS010 ABS015 ABS020 ABS PA6010 PA6 PBT010 PBT PP202 PP PS00 PS07 PS09 PS

ABS010 2 0 7 1 0 0 0 0 0 0 0 0 0 0

ABS015 0 11 0 9 0 0 0 0 0 0 0 0 0 0

ABS020 0 0 12 0 0 0 0 0 0 0 0 0 0 0

ABS 0 0 0 21 0 0 0 0 0 0 0 0 0 0

PA6010 0 0 0 0 8 0 0 0 0 0 0 0 0 0

PA6 0 0 0 0 0 22 0 0 0 0 0 0 0 0
Real class

PBT010 0 0 0 0 0 0 12 5 0 0 0 0 0 0

PBT 0 0 0 0 0 0 0 13 0 0 0 0 0 0

PP202 0 0 0 0 0 0 0 0 11 0 0 0 0 0

PP 0 0 0 0 0 0 0 0 0 16 0 0 0 0

PS00 0 0 0 0 0 0 0 0 0 0 8 0 0 4

PS07 0 0 0 0 0 0 0 0 0 0 0 8 0 4

PS09 0 0 0 0 0 0 0 0 0 0 0 0 16 0

PS 0 0 0 0 0 0 0 0 0 0 0 0 0 16
Figure 01
Click here to download high resolution image
Figure 02
Click here to download high resolution image
Figure 03
Click here to download high resolution image
Figure 04
Click here to download high resolution image
Figure 05
Click here to download high resolution image
Figure 06
Click here to download high resolution image
Figure 07
Click here to download high resolution image
Figure 08
Click here to download high resolution image
Figure 09
Click here to download high resolution image
Figure 10
Click here to download high resolution image
Figure 11
Click here to download high resolution image
Figure 12
Click here to download high resolution image
JMAmigo

Dr. Jos Manuel Amigo

Born in 1978. He obtained his PhD (Cum Laude) in Chemistry from the Autonomous University of
Barcelona, Spain. Since 2007 he has been employed at the Department of Food Science,
Spectroscopy and Chemometrics group of the University of Copenhagen, Denmark, as associate
professor. Current research interests include hyperspectral and digital image analysis,
chromatographic data analysis, Food Sciences, environmental modelling, curve resolution and
teaching chemometrics. He has authored more than 85 publications (65+ peer-reviewed papers,
books, book chapters, proceedings, etc.) and given more than 40 conferences at international
meetings. Jose has supervised or is currently supervising several Masters, Post Docs and PhD
students and he is an editorial board member of four scientific journals within chemometrics,
pharmaceutical sciences and analytical chemistry. Moreover, he has recently received the 2014
Chemometrics and Intelligent Laboratory Systems Award.
HBabamoradi

Dr. Hamid Babamoradi

Born in 1982, he obtained his PhD in Chemometrics and Statistical Process Control from University
of Copenhagen, Denmark. Since end of 2012, he has been employed as postdoctoral scholar at
Spectroscopy and Chemometrics section, Department of Food Science, University of Copenhagen,
Denmark. Current research interests include hyperspectral and digital image analysis, multivariate
statistical process control (MSPC), dairy process control. He has authored multiple peer-reviewed
papers and been working on multiple industrial projects (such as automation of waste sorting and
dairy process control). He has also been involved in teaching MSPC, PAT, and Chemometric
courses.
SElcoroaristizabal

Dr. Saioa Elcoroaristizabal

Born in 1983, she obtained her International Ph.D. in Environmnetal Engineering from the
University of the Basque Country, Spain, in 2015. She obtained her Chemical Engineering degree
in 2007 and developed her research skills through different projects related to environmental
engineering carried out for private and public entities. In 2010, she obtained her Ms.C in
Environmental Engineering, acquiring additional theoretical and practical background on
chemometrics during several research stays. Since then, she has authored and co-authored several
peer-review papers and presented her research findings in several national and international
conferences. She has also been involved in teaching in the degree of Environmental Engineering at
the University of the Basque Country, Spain.
Image_JMAmigo
Image_HBabamoradi
Image_SElcoroaristizabal