You are on page 1of 12

Measuring classication consistency in the context of temporal evolution

Jonas Henrique Mendona, Isabel Escada, Sandra Sandri

Instituto Nacional de Pesquisas Espaciais 12201-970, So Jos dos Campos, SP E-mail:,,,

Abstract. In this work we propose some measures to verify the quality of classication in the context of temporal evolution. We address this problem in a classic setting and then briey examine it in the fuzzy classication framework. We classify each pair formed by an object and a moment in time, but verify the overall accuracy for sequences of time steps for each object before aggregating the obtained accuracies into a global one. This work has been motivated by the need to verify the temporal evolution of deforestation patterns in the Brazilian rainforest region, in an automated manner.


The goal of this work is to propose accuracy indices to measure the quality of classiers in the context of temporal evolution. Here we consider classication functions that map X T to C , where C is set of classes, X is a set of objects relevant in a given application (cells in a grid on a map, pixels in an image, prices for a set of goods, etc) and T is nite a time scale. In a such a context, the most straightforward procedure is to assess how good is the classication of the state of an object at each moment of time, independently of any other moment. In other words, the accuracy of the classication of an object x0 X at time t T is done without considering any other time step ti = t . In this case, global accuracy is basically an aggregation of the individual accuracy of the pairs (x, t) X T , considering that each pair is associated to a class in C . In other words, we aggregate the (individual) accuracy of each pair in set {(x1 , t1 ), (x1 , t2 ), ..., (x1 , tm ), ..., (xn , t1 ), (xn , t2 ), ..., (xn , tm )}, (xi , tj ) X T , n =| X |, m =| T |. However, in some applications, it is interesting to measure the quality of classication not as an aggregation of the quality of the individual classications, but as the overall quality of the classication of a given object considering its evolution in time. Here we propose to obtain the global accuracy considering n objects and m time steps as an aggregation of the (overall) accuracies of the sequences in the set {< (x1 , t1 ), (x1 , t2 ), ..., (x1 , tm ) >, < (x2 , t1 ), (x2 , t2 ), ..., (x2 , tm ) >, ..., < (xn , t1 ), (xn , t2 ), ..., (xn , tm ) >}, each of which associated to an object in X . As in the former scheme above, each pair (x, t) X T is associated to a class in C . The schemes above are not the same as the one described in [6], in the context of temporal evolution classication. In that work, the classier determines the kind of land

use that took place in a given area during a single year (different crops, deforestation, etc), using a signicant amount of measurements (e.g. every 16 days). Each temporal sequence is classied as a single entity, i.e. each < (xi , t1 ), (xi , t2 ), ..., (xi , tm ) > is associated to a class in C , whereas in both schemes above, each individual pair (xi , tj ) is associated to a class. Moreover, here we are particularly interested in classifying the type of deforestation suffered by a given area in a long period of time (minimal three years) but with a small amount of measurements (e.g. one per year). In this work, we study some measures to verify the quality of classication in our particular context of temporal evolution, addressing two main cases: i) the valid temporal behaviour of a variable is unrestricted and ii) the valid temporal behaviour of a variable is monotonic. The unrestricted temporal trajectory occurs, for instance, when we have a set of crops that can grow in a given piece of land: what is a corn eld today may become a potato eld next year and possibly again a corn eld some years later. In the general case, any ordering of the classes would be arbitrary and the trajectories could be modeled by graphs allowing cycles. In the monotonic case, the classes can be put in at least a partial order and the trajectory can be modeled using a directed acyclic graph. An example of monotonic trajectories is the temporal evolution of deforestation patterns [1]; a completely deforested area is not expected to become a full-edged forest again in decades or even centuries. The motivation for this work is the need to verify the temporal evolution of deforestation patterns in the Brazilian rainforest. The visual classication of patterns is a painstaking job, so there exists a great interest in using the known classication of patterns in certain cells as the input to an automated system that would infer the classication of patterns for other cells. Automatic classication of deforestation patterns for parts of the Brazilian Amazon region has so far been done using two approaches: decision trees [3] and a fuzzy classier [4], both considering individual cells (the rst scheme above). In both works, after the classication task is concluded for each pair (xi , tj ) X T , the temporal trajectories obtained by the individual classications are then analysed. Here we rst address the problem considering that the classication of the patterns are given as a (crisp) sequence of classes, to then consider the case in which there is a fuzzy set associated with the classication of each time step. We use a small set of examples to assess the quality of the proposed indices. The present work is the rst step towards a long term goal of creating automatic classiers for deforestation patterns that takes into account the quality of temporal evolution of these patterns. In the same way that the classication of a cell in an image is usually related to the classication of the neighboring cell, we here assume that the classication of a cell in a given moment of time is related to the classication of that cell in the next few moments of time. The goal is to use this hypothesis to obtain coherent sequences of individual classications. This work is organized as follows. In Section 2 we rst briey describe the problem of temporal evolution of deforestation patterns in the Brazilian rainforest. We propose a general framework for classication of temporal trajectories in Section 3. In Section 4 we study indices of temporal inconsistency for cases i) and ii) described above. In Section 5 we discuss how fuzzy classication can be explored in our context and Section 6 nally brings the conclusions.

A temporal evolution problem: deforestation patterns in the Brazilian Amazon region

Since 1988, the Brazilian National Institute for Space Research (INPE) has been producing the annual rates of deforestation in the legal Amazon region. From the year 2002, these estimates are being produced by digital classication of images following the PRODES methodology [2], stored in the eponymous database. The main advantage of this procedure is the accuracy of the geo-referencing of deforestation polygons in order to produce a multi-temporal geographic database. The PRODES database has been used, among other types of application, to detect different types of land occupation and use. In the Amazon region, the deforestation patterns fall into 5 main classes [1]: forest (f ), diffuse (d), geometric (g ), multi-dimensional (m), and consolidated (c), as depicted in Figure 1.

Fig. 1. Deforestation patterns (Source: [3])

Terra do Meio (Middle Land) is a region located in the Brazilian State of Par, that covers the municipalities of So Felix do Xingu, Altamira and Tucum. The deforestation patterns in the region have been classied using a grid of 10km by 10km cells in [3][5] (see Figure 1). Figure 2 compares the deforestation patterns in this region for a period of 3 years. Considering temporal resolution of one year, the temporal evolution of deforestation patterns in the Amazon region, with few exceptions, have been found to follow a general scheme: 1. At the beginning of the trajectory, a cell can display any pattern. 2. The forest pattern may evolve to either the diffuse or geometric patterns. 3. The diffuse pattern may evolve to either the geometric or the multidirectional patterns. 4. The geometric pattern may evolve to either the multidirectional or consolidated patterns. 5. The multidirectional pattern may evolve to the consolidated pattern.

Fig. 2. Classication results for Terra do Meio cells for years 1997, 2003 and 2006 (Source: [3])

6. The consolidated pattern does not evolve to any other pattern. 7. A cell may maintain the same pattern indenitely. Note that the diffuse pattern may evolve to the geometric pattern but not vice-versa. Figure 3 illustrates a graph and automaton constructed according to these temporal restrictions (more details about the formalism are given in the next section). For example, in a period of 5 years, temporal trajectory < f, f, d, g, c > can be considered valid whereas < f, g, d, d, c > cannot.



Fig. 3. Deforestation patterns temporal consistency in Terra do Meio: a) graph and b) automaton

Classication of temporal trajectories

First of all, let us establish the notation used throughout this work: X is a set of variables that refer to elements of interest in an application (cells in a grid, for example). T is a nite temporal scale. C is an ordered set of classes; each (x, t) X T is associated to a class in C . E is a set of classiers (experts), be they human or automated systems. nX , nT , nC and nE denote the cardinality of sets X , T , C and E , respectively.

nT Bi = (b1 i , ..., bi ) is a nT -tuple modeling the actual temporal trajectory of xi in X over period of time | T |. nT Be,i = (b1 e,i , ..., be,i ) is the estimated temporal trajectory of xi over T , as furnished by expert e E , i.e. the sequence of classications for an object xi for each moment tj T , according to e.

Subscript i and superscript j will be dropped when no confusion is possible. In an application where time does not play a role, the accuracy of a classier is the number of times it predicted the correct class of the objects relevant to that application. When a given application has a temporal component, as in our case, we propose to use the error rate function GlobalErr : E R as GlobalErr(e) =

Err(e, xi )

where Err(e, xi ) 0 measures the error of the estimated temporal trajectory with respect to a single object. In other words, for each object xi X , function Err considers the whole temporal sequence of classications Be,i furnished by an expert e E . Function GlobalErr has a compensatory effect: a bad evaluation in a time step can be balanced by a good evaluation in another one. Other functions could be used instead of the summation to avoid this behaviour, but this one is adopted here for its simplicity. In the remaining of this text we discuss what Err could be, considering the restrictions on temporal behaviour. The most straightforward choice for function Err is the Hamming distance, which in our notation is given as ErrHamm(Bi , Be,i ) =
j =1,nT j equal(bj i , be,i )

equal(a, b) =

1, if a = b 0, otherwise

ErrHamm considers that all errors are equal and is thus not adequate for our framework. The goal of this work is to study better choices for function Err. Before discussing these choices, we give a formal presentation of valid trajectories and present a penalty function, which has a similar role to function equal above. 3.1 Valid temporal trajectories

The classication of patterns in cells from Terra do Meio were obtained by visual inspection and not by ground truth. Moreover, the classication of patterns in cells at different time steps were considered individually. The available knowledge about the patterns classication is thus not error-free. In applications such as this, to measure the consistency of temporal evolution estimates in relation to the actual temporal trajectory of a variable, it is rst necessary to ensure that the actual trajectory is itself valid. The invalid trajectories should not used as a reference, i.e. they should not belong to the training set of a classier, as this lack of monotonicity is a strong evidence that the estimate should not be trusted.

In our context, the validity of temporal trajectories can be veried using a graph GC = (C, EC ) with the set of nodes (vertices) given by the set of classes C and a set of edges EC that consists of pairs of C (see Figure 3.a). Here we classify the trajectories in two categories: monotonic, when graph GC is directed and acyclic, except for edges (a, a) EC , and unrestricted, otherwise. In a non-directed graph, a, b C, (a, b) EC implies (b, a) EC , which does not necessarily hold in a directed graph. To verify whether a temporal trajectory is valid, it sufces to construct a nite state automaton whose recognized language consists of all the possible paths on graph GC . In the deforestation problem, the automaton MC = {, Q, q0 , F, } is constructed from GC with an input alphabet = {f, d, g, m, c}, a set of states Q = {1, 2, 3, 4, 5, 6}, an initial state q0 = 1, a set of nal states F = {2, 3, 4, 5, 6} and a transition function : Q Q as shown in Figure 3.b. It is easy to see that L(MC ), the language recognized by automaton MC , is the set of paths derivable from graph GC . In the monotonic case, the trajectory of a variable can be considered invalid when evolution is not monotonically increasing. The graph in Figure 3.b is acyclic, except for the edges from a node to itself, so for our purposes it can be considered a DAG (directed acyclic graph). In such graphs, the trajectories can be represented by monotonically increasing sequences, because it would be possible to assign numbers to the nodes in such a way that the sequence of such numbers in the valid temporal trajectories would never be decreasing. Let us suppose that the actual temporal evolution of given cell is B = (m, m, m) and that we have estimates from ve experts, for a given three years period: B = (m, m, g ), B = (c, m, g ), B = (m, m, c), B = (g, m, c) and B = (d, m, c). It is easy to verify that B belongs to L(MC ) as well as B , B and B . Trajectories B and B do not belong to L(MC ) as they are monotonically decreasing (see Figure 4).

Fig. 4. Actual trajectory B with 5 estimates for a 3 years period


Cost of confusion between classes in trajectories

To measure the error in the estimate of an expert in relation to temporal trajectory of a single object xi X , we make use of a penalty function pen : C 2 R0 ,

that measures the error between bj e,i , the estimation of the value of object xi at time tj given by an expert e, and the actual value bj i . A reasonable requirement for function pen is that, if there exist a path between a and b, and a path between b and c, then we should have pen(a/c) min(pen(a/b), pen(b/c)), a, b, c C . The most straightforward penalty function is penequal (a/b) = 1 when a = b and penequal (a/a) = 0, which corresponds to function equal used in the Hamming distance, as seen previously. Simple penalty functions can be constructed when each class is renamed by a natural number. One such penalty function is penabs (a/b) =| a b |. This simple function is commutative, in the same way as penequal , and may fulll the requirement above depending on the class renaming.


Measuring temporal accuracy

Accuracy in the unrestricted framework

In the unrestricted trajectory framework, the simplest choice for a temporal error rate function between the real trajectory Bi and an estimate Be,i is ErrP e(e, xi ) =
j =1,nT j pen(bj e,i /bi ).

This measure is equivalent to the Hamming distance (ErrHamm) when pen = penequal is used and to the Manhattan distance when pen = penabs . Let B B indicate that estimate B is better than B and B B indicate that B and B are equally good. Let us consider the non-directed version of the graph depicted in Figure 3. In this case, the graph becomes cyclic and the semantics attributed to the labels are not valid anymore, of course, but the graph structure is helpful to illustrate the unrestricted framework. Let us consider a given cell x X with actual and estimated trajectories depicted in Figure 4. Taking penequal , we obtain ErrP e(e , x) = ErrP e(e , x) = 1 and ErrP e(e , x) = ErrP e(e , x) = ErrP e(e , x) = 2, from which we get the ordering B B B B B . Renaming classes {f, d, g, m, c} as {1, 2, 3, 4, 5}, we obtain a version of function penabs such that, for instance, penabs (f /c) =| 1 5 |= 4. Using this penabs , we obtain ErrP e(e , x) = ErrP e(e , x) = 1, ErrP e(e , x) = ErrP e(e , x) = 2 and ErrP e(e , x) = 3, from which we then get the ordering B B B B B . We see that penabs , with the class renaming given above, makes a distinction between B and B , because the renaming makes g and m be considered closer together than d and m. In both cases, the error calculated for an estimated trajectory does not depend of its monotonicity, but eventually between how far two classes are in the graph, which makes sense in the unrestricted framework, in which cycles are allowed.


Accuracy in the monotonic framework

In the monotonic framework, we have to take into account the information in the graph modeling the valid trajectories. For example, in the problem of deforestation patterns addressed in Section 2, the classes can be arranged on a scale, ranging from forest" to consolidated". The temporal evolution of the deforestation pattern of a cell is expected to be monotonically increasing, with no return to a previous stage. Note that it is not strictly monotonic though, as a cell may present the same pattern for an indenite number of years. When commutative operators are used for pen, function ErrP e is intuitively not adequate in the monotonic framework. One alternative would be to use more complex non-commutative functions for pen that somehow model the information in the graph. But that may be too burdensome to the user. In the following, we propose the use of alternative functions for Err in the monotonic framework, that are able to deal with simple commutative pen functions. Let us consider the deforestation example, with the trajectories from Figure 4. Considering the graph in Figure 3.a and the actual trajectory B , intuitively B should be considered as the best estimate, because it is valid and only misses the correct value once, and B should be considered the worst one, because it is invalid and misses the correct value twice. We see that considering both penalty functions penequal and penabs , function ErrP e is not able to distinguish between the monotonically increasing trajectory estimate B and the monotonically decreasing B . The same happens in relation to estimates B and B . We also see that using penabs , B is considered worse than the monotonically decreasing estimate B , contrary to what happens with penequal . We need thus to nd a function that is capable of measuring monotonicity. A basic function able to measure how much a n-uple p of real numbers is monotonically increasing is given by k=1,n1 max(0, p(k ) p(k + 1)). This function goes to its minimal value, 0, when p is monotonically increasing. One way to adapt this function to our framework is to take M o(Be,i ) =
j =1,nT 1 j +1 M (bj e,i , be,i ),

M (cl , cu ) =

0, if {(c1 , c2 ), ..., (ck1 , ck )} EC , such that c1 = cl and cn = cu M (cl , cu ) = max(pen(cl /cu ), pen(cu /cl )), otherwise

For the trajectories above, we have M o(B ) = M o(B ) = M o(B ) = 0, M o(B ) = 1 and M o(B ) = 2. We see that this measure is capable of distinguishing between the monotonically increasing trajectories from the monotonically decreasing ones. The product of functions M o and P e could be taken to derive a new monotonic inconsistency index. However, this index would be equal to 0 for both B and B , even though B is clearly closer to B than B . We create a new measure ErrM i combining functions P e and M o as: ErrM o(e, xi ) = ErrP e(Bi , Be,i ) (1 + M o(Be,i )).

When an estimate is equal to the actual trajectory, we obtain 0 as the result, as expected. Applying this function on our example with penabs as given above, we obtain ErrM o(e , x) = 1, ErrM o(e , x) = ErrM o(e , x) = 2, ErrM o(e , x) = 3 and ErrM o(e , x) = 6. Using penequal instead of penabs , the only difference is that then we have ErrM o(e , x) = 2. Thus, using penequal , we obtain B B B B B , whereas using penabs with the class renaming given above, we obtain B B B B B . We see that the penalties derived from both penequal and penabs , as expected, B and B are considered the best and the worst estimates, respectively. Also in both cases, B and B are considered equivalent, thus exhibiting a compensatory effect; B errs twice but is monotonic, whereas B errs only once but is not monotonic. Note that it is not evident which of these estimates should be considered better. A suitable function pen should be used according to the choice from the experts in cases such as this. Finally, the penalties derived from penabs consider B inferior to B , whereas they are viewed as equivalent when penequal is used, as in the unrestricted framework.

Measuring temporal inconsistency with fuzzy classication

When a fuzzy classier is used in the context of temporal evolution, the fuzzy set (or possibility distribution) [10] associated with each object of the set to be classied (cells in a grid, in our case) can be used to verify the quality of the temporal evolution of that element. Let j e,i : C [0, 1] be the fuzzy set obtained by a fuzzy classier e for element xi at moment tj . In the following we assume that every j e,i is normalized, i.e. xi j X, tj T, maxcC e,i (c) = 1. Subscript i will be omitted whenever possible. 5.1 Unrestricted framework

A general fuzzy temporal trajectory error measure for a variable xi X furnished by an expert e is given by
j F (e, xi ) = j =1,nT {1 j e,i (bi )}, j where : [0, 1]nT [0, 1] is an appropriate fuzzy aggregation operator. Each j e,i (bi ) can be seen as how much the expert believes the value of xi at moment of time tj is bj i , which is the actual value of that variable for that moment. Aggregation operator should be such that F goes to 0 in the best case (the expert gives maximal degrees of belief for the actual trajectory values) and nT when in the worst case (the expert gives minimal degrees of belief for the actual trajectory values). This is the case of means, t-norms, t-conorms, the most used fuzzy operators [10]. Let us suppose that the actual temporal trajectory of a given cell x X is given by B = (f, d, g ) and that the estimated trajectory furnished by expert e for the same cell is B = (c, m, g ). In this case, we obtain ErrP e(e ) = 2. Moreover, let us suppose the original assessment furnished by e for cell x at tj included 1 (f ) = 0, 2 (d) = .5 and 3 (g ) = 1. In other words, expert e does not

believe at all that the pattern at cell x was a forest at time t1 , believes completely that x presented a geometrical pattern at time t3 , and neither believes nor disbelieves that the deforestation pattern of x at time t2 is diffuse. The expert is thus completely wrong with respect to t1 , completely right with respect to t3 , and partially right/partially wrong with respect to t2 . So the error for each individual moment of time tj for variable x with respect to e is thus 1, .5 and 0. Using as the T-norm min, the T-conorm max and the arithmetic means, we obtain Fmin (e , x) = 0, Fmax (e , x) = 1 and Fam (e , x) = .5, respectively. These three functions model different attitudes for aggregating the individual errors: using max, the nal error equals the biggest individual error, and thus, being right about t3 is disregarded. Using min, the nal error equals the smallest of the individual errors, and the fact of being completely wrong about t1 is disregarded. The arithmetic means allows for compensation. Let us now suppose that expert e furnishes the same prediction for the temporal trajectory as expert e . However, let us suppose that the original assessment of expert 2 3 e included 1 (f ) = .9, (d) = .5 and (g ) = 1, which produce the individual errors .1, .5 and 0. Note that, contrary to expert e , e strongly believes that f is a good class for the actual value of x at time t1 . Therefore, the expected order between these estimates is B B . Aggregating the individual errors from e we obtain Fmin (e , x) = 0, Fmax (e , x) = .5 and Fam (e , x) = .2. Therefore, expert e is considered better than e when the arithmetic means and max are used, and equally good when min is used. However, the use of min does not produce the inverse order, which would not be acceptable. We propose to combine ErrP e and F as ErrP eF (e, xi ) = ErrP e(e, xi ) (1 + F (e, xi )). In our example, using penequal for cell x we have ErrP e(e , x) = ErrP e(e , x) = 2. Calculating, for e we obtain ErrP eFmin (e , x) = 2, ErrP eFmax (e , x) = 3 and ErrP eFam (e , x) = 2.4. For e , we obtain ErrP eFmin (e , x) = 2, and ErrP eFmax (e , x) = 4 and ErrP eFam (e , x) = 3. The orders obtained using ErrP eF and F are obviously the same, since ErrP e is a constant in this case. Using penabs produce different values but the same orderings. 5.2 Monotonic framework

Let us suppose that we are in the monotonic case, with the graph given in Figure 3. Estimates B and B above are thus both very inconsistent with the actual trajectory of x, since B increases whereas B and B decrease (see the previous section). Let now suppose that expert e+ estimates the temporal trajectory of cell x as B+ = (c, f, g ). Using penequal , we obtain ErrP E (e+ , x) = 2. Let us now suppose that the fuzzy 2 3 assessments furnished by e+ are 1 + (f ) = .9, + (d) = .5 and + (g ) = 1. B+ is thus similar to B , but presents less problems with monotonicity, so it is reasonable to expect the order B B B+ . Calculating, we obtain ErrP e(e+ , x) = 2, Fm (e+ , x) = .2 and ErrP eFm (e+ , x) = 2.4. Therefore, considering the functions above, B+ is undistinguishable from B .

In the monotonic framwork, one way to measure of trajectory quality would consist in combining measures ErrM i and F . We propose to to use ErrM oF (e, xi ) = ErrM o(e, xi ) (1 + F (e, xi )). We obtain ErrM o(e , xi ) = ErrM o(e , xi ) = 6, ErrM o(e+ , xi ) = 4. We thus obtain ErrM oFam (e , xi ) = 9, ErrM oFam (e , xi ) = 7.2, ErrM oFam (e+ , xi ) = 4.8. Using ErrM oFam , we obtain the expected order; the same happens when max is used. However, with min we obtain B B B+ . When penabs is used, we obtain different values but get the same orderings as those obtained using penequal .

Concluding Remarks

We proposed here a series of quality measures to guide the classication process in the context of temporal evolution. We have examined two cases, unrestricted and monotonic, and briey discussed the framework when a fuzzy classier is used. We have used a small set of examples to assess the quality of the proposed indices. Experiments are under way to assess the quality of the proposed measures, in a realworld application that involves the temporal trajectories of deforestation patterns in the Brazilian Amazon region. The proposals for the non-fuzzy framework produced very good results [5]. Our initial study on the proposed fuzzy indices indicates that both the unrestricted and monotonic frameworks may benet from taking the results of fuzzy classication into account. However, further work is needed to assess the real usefulness of the fuzzy approach, possibly incorporating information about fuzzy estimate precision [11]. As future work, we intend to address what kind of restrictions should be used for penalty functions in order to make them produce acceptable behavior. We are also interested to study temporal evolution problems in which there exists a full cycle of events. That happens, for instance, when a culture planted on a piece of land, such as sugarcane, corn, etc, undergoes various stages of growth. In this case, a good measure of quality, apart from the distance between the actual and the estimated trajectories is the smoothness of the estimated trajectory, as measured, for instance, by the Tikhonov second order regularization [8]. We now discuss the differences between our approach and Hidden Markov models (HMM) [9], a framework that is also related to temporal trajectories and use graphs to deal with them. HMMs are primarily used to predict in which state an object will be after a xed amount of time, or how long it will take for an object to reach a particular state. In these models, probabilities are attached to arcs in the graph, that assess how likely it is for an object to move from one state to another. To use HMMs, a large amount of data is needed to create the probabilistic assessments, otherwise the results cannot be considered reliable. HMMs and systems performing classication of elements of a temporal trajectory have in common the use of recorded data to estimate the unknown value of a variable. But in HMMs, the past behavior of an object, such as a cell in an image, is used to estimate the future behavior of that same object. In classication problems involving temporal trajectories such as ours, however, the data regarding the past behavior of

an object is used to estimate the past behavior of other similar objects, a completely different problem. Moreover, we do not have enough data to derive the probabilities on the arcs, as the use of HMMS would require. Interestingly enough, for a similar application in which no data is available about actual trajectories, the use of a graph weighted with probabilities is potentially useful to create a set of possible trajectories. This set could then be used to create, test or choose between classiers. For instance, a sufciently large and representative set of those trajectories could be used to tune a fuzzy rule-based system or to train a neural network.

The authors are grateful to Solon V. de Carvalho for kindly sharing his knowledge on Markov Hidden models.

1. Escada, M. I. S., Monteiro, A. M., Aguiar, A. P., Carneiro, T. and Camara, G., Anlise de padres e processos de ocupao para a construo de modelos na Amaznia (Analysis of land use patterns and processes for the construction of models in Amazonia: Experiments in Rondonia). In XII Brazilian Symposium on Remote Sensing, Goinia (Br), pp. 2973-2983. In Portuguese (2005) 2. INPE, 2005, Monitoramento da Floresta Amaznica Brasileira por Satlite (Monitoring the Brazilian Amazon Forest by Satellite). Available online at (accessed 01/03/2013). In Portuguese (2013) 3. Lobo, F. L., Escada, M. I.: Anlise da dinmica da paisagem na regio do centro de endemismo Xingu: estudo de caso na regio da Terra do Meio, Par. In Portuguese, submitted (2013) 4. Mendona, J.H., Sandri, S., Martins-Bed, F., Guimares, R., Carvalho, O.: Estudo de relaes de similaridade difusas e sistemas neuro-fuzzy aplicados ao raciocnio baseado em casos. In Congresso Brasileiro de Sistemas Fuzzy, Natal (Br). In Portuguese (2012) 5. Mendona, J.H.: Raciocnio baseado em casos: uma abordagem utilizando relaes de proximidade difusa. CAP MSc Thesis, INPE, SJCampos (Br). In Portuguese (2013) 6. Pimentel, T., Ramos, F.M., Sandri, S.: Uso de Perceptrons Multicamadas Difusos para a Classicao de Padres em Sries Temporais. In Proc. ENIAC13, Fortaleza (Br). In Portuguese (2013) 7. Sandri, S., Mendona, J. H., Martins-Bed, F.T.: Weighted Fuzzy Similarity Relations CaseBased Reasoning: a case study in classication. In Proc. WCCI12, Brisbane (Au) (2012) 8. Tikhonov, A.N. and Arsenin, V.S.: Solutions of Ill-Posed Problems. Winston and Sons (1977) 9. Baum, L. E., Petrie, T.: Statistical Inference for Probabilistic Functions of Finite State Markov Chains. The Annals of Mathematical Statistics, 37(6), pp. 1554-1563 (1966). 10. Dubois, D., Prade, H.: Possibility Theory: An Approach to Computerized Processing of Uncertainty. Plenum Press (1988) 11. Sandri, S., Dubois, D., Kalfsbeek, H.W. Elicitation, assessment, and pooling of expert judgments using possibility theory. IEEE Transactions on Fuzzy Systems, 3(3), pp. 313-335 (1995)