You are on page 1of 9

Electrochimica Acta 51 (2006) 14531461

Benets of evolutionary strategy in modeling of impedance spectra


O. Kanoun , U. Tr ltzsch, H.-R. Tr nkler o a
University of the Bundeswehr Munich, Institute for Measurement and Automation, 85579 Neubiberg, Germany Received 23 June 2004; received in revised form 25 October 2004; accepted 25 February 2005 Available online 2 September 2005

Abstract The more effects and mechanisms are represented in an impedance spectrum, the more unknown parameters are needed for accurate modeling. The inverse identication problem corresponding to this parameter extraction process becomes thereby more difcult to solve and needs generally a lot of a priori knowledge and trials. The use of evolutionary strategies (EV) can signicantly contribute to increase efciency during this process. With selected examples, many benets of evolutionary strategy are shown by means of simulations with more than 100 trials. For simple models, the main advantage of evolution strategy is to be nearly insensitive to the chosen search region. For models with a big number of unknown parameters, the combination with the LevenbergMarquardt (LevMq) method reaches very good results compared with each method alone. The combined method proted of positive properties of both methods. The evolution strategy is, also for an ill-posed inverse problem, able to calculate a parameters vector near the optimum. The LevenbergMarquardt method prots from it and is then able to calculate the optimum more exactly, because it is already in its next neighborhood. The resulting process is robust and reaches good modeling results even, if only a few a priori knowledge is available concerning expected parameter values. 2005 Elsevier Ltd. All rights reserved.
Keywords: Impedance spectroscopy; Evolutionary strategy; Genetic algorithm; Inverse problem; Sensitivity analysis

1. Introduction Modeling of impedance spectra needs suitable methods for parameter extraction, in order to facilitate investigation of models and calculation of corresponding parameter values. Especially in this eld, convergence difculties and strong dependencies on the start values are observed even with relative simple impedance models. This is due to their typical strong parametric non-linearity [1]. The stronger the non-linear behavior of a model, the weaker is the convergence of the corresponding regression and the more is the probability to trap into local minima of the quality function. Particularly at investigation stage, knowledge about the suitability of models and their typical parameter values is restricted. Local non-linear regression algorithms, such as GaussNewton (GN) or LevenbergMarquardt (LevMq)

Corresponding author. Tel.: +49 89 6004 3740; fax: +49 89 6004 2557. E-mail address: kanoun@ieee.org (O. Kanoun).

method, need therefore a lot of trials in order to reach meaningful results. This is due to the fact, that these gradient-based methods are suitable only for models with low parametric non-linearity. In the eld of impedance spectroscopy, many quantities should be simultaneously modeled, such as electrode dimensions, diffusion coefcients, reaction rates, charge transfer coefcients and concentrations. Impedance models have therefore generally many unknown parameters and a higher probability of parametric non-linearity. Local algorithms nd generally the closest optimum to the starting point. The parameter extraction procedure should be therefore tailored for every case. Thereby, strategies are developed, e.g. holding some parameters constant at the beginning, choosing good initial estimates of the parameters or using some suitable weighting or penalty functions. Under these conditions, the investigation of a model suitability becomes time consuming and unreliable. It depends strongly on the number of unknown parameters, the available time for modeling and the experience of the model developer.

0013-4686/$ see front matter 2005 Elsevier Ltd. All rights reserved. doi:10.1016/j.electacta.2005.02.123

1454

O. Kanoun et al. / Electrochimica Acta 51 (2006) 14531461

The parameter extraction process represents mathematically an inverse identication problem and is therefore associated with certain difculties. Depending on the application considered, it can be evaluated with many features, such as the quality of t, the calculation time and robustness. This last is inalienable for the application considered in our case, in which model parameters should be automatically calculated online without any interventions [2]. In order to nd out a robust parameter extraction method, we propose to make use of global methods like evolutionary strategy. Evolutionary and genetic algorithms have been already applied in many elds for optimization problems [3,4], mainly due to their ability to avoid trapping into local optima of the quality function. They make use of a population and use therefore a search region instead of start parameters. Genetic algorithms imitate directly biological genetic variation mechanisms with more cross over mechanisms than mutation processes [5]. The necessary binary coding of the variable vector makes them laborious for continuous problems [4,6]. Evolutionary strategy (EV), which is considered in this paper, was in contrast specially developed for parameter extraction. It uses the parameter values in their original format and allows a targeted deterministic selection of best individuals. In some literature, evolutionary strategies are termed as genetic algorithms with real numbers. Although evolution strategy and genetic algorithms are already well established in a lot of application elds [3,4,7], their use in modeling of impedance spectra was not widely used until now. Even in literature, only a few contributions [6,810] were dedicated to the application of these methods. In [6], a comparison of genetic algorithms with Quasi-Newton method (QN) was carried out with emphasis on calculation time, convergence behavior and sensitivity to measurement noise. The results will be considered all through this paper. In [8], a hybrid method was developed using alternately one generation of the genetic algorithm followed by one attempt with the GaussNewton method in order to improve convergence. This method benets only partly from the advantages of evolutionary strategy as will be explained in Section 4. The main aim of this paper is to investigate benets of evolution strategy for impedance spectroscopy, to give a general overview of them and to explain them based on consistent simulations for different classes of models.

Fig. 1. Investigated models.

two Randles models in series, which can represent the two electrodes of an electrochemical battery [12] and has altogether seven unknown parameters. A sensitivity analysis was carried out in order to characterize the investigated models for non-linear parameter extraction. This method can provide considerable insight into an estimation problem [13,14]. Although it is a common method in parameter estimation, it was rarely used in electrochemical impedance spectroscopy as criticized in [15]. We can generally distinguish global and local methods of sensitivity analysis [15]. Global methods, such as Fourier amplitude sensitivity test and stochastic sensitivity analysis [15], produce measures of parametric importance over entire regions in parameter space. In this case, the sensitivity information is averaged on the parameter region considered. Local methods, such as partial derivates or sensitivity densities, determine the effect of parameter variation of the solution at a single point of the parametric domain. These are more suitable for our investigation, because we know the original parameter values, which were used for generation of the simulated data subjected to t processes. In this paper, we use partial derivates of the model function as sensitivity coefcient [13]. SCP = ZMod k P (1)

where P is the parameter, ZMod the impedance model and k is the angular frequency. A scaling is performed by multiplying the sensitivity coefcients by the parameters values. The so scaled sensitivity coefcients have the same unit and allow their comparison to each other. Scaled SCP = P ZMod k P (2)

2. Investigated models In order to investigate different methods of parameter extraction, we dened two impedance models with different levels of complexity (Fig. 1). The investigations are carried out using typical models in impedance spectroscopy, especially the fundamental Randles model [11] with a constant phase element for representation of the Warburg impedance. Model1 has four unknown parameters. Model2 consists of

In general, the scaled sensitivity coefcients are desired to be large for each parameter and uncorrelated (linearly independent) for different parameters. A correlation (linear dependency) between the sensitivity coefcients is available, if there exists a set of parameters a1 , a2 , . . ., an so that the relationship (3) is fullled for the whole observed frequency range. In this case, there is no unique minimum and all the

O. Kanoun et al. / Electrochimica Acta 51 (2006) 14531461

1455

Fig. 2. Scaled sensitivity coefcients for Model1 .

parameters cannot be simultaneously and uniquely estimated [13]. a1 SCP1 + a2 SCP2 + + an SCPn = 0 (3)

The differences between Model1 and Model2 represent a suitable basis for evaluation and comparison issues as will be seen in the next sections.

where SCPi is the sensitivity coefcient corresponding to the parameter Pi and n is the number of unknown parameters. In [1], a method for assessing non-linearity in impedance models was developed and tested. The results show, that impedance spectra have generally, and also the Randles models, a strong parametric non-linearity [1]. This is one reason why, we can expect some difculties, also with this relative simple model. The scaled sensitivity coefcients for Model1 (Fig. 2) are not totally correlated but the sensitivity coefcient for the parameter is very low. Both facts let us characterize the process of parameter extraction to be solved as illposed inverse identication problem. The stronger is the ill-posedness of an inverse identication problem, the more dependency on start parameters, the more convergence difculties, the more probability to trap into local minima of the quality function and the higher sensitivity to measurement noise. In the case of Model2 , not only the bigger number of unknown parameters and the expected higher parametric nonlinearity [1] pose problems. The sensitivity analysis illustrates also a low information content in the measurement data concerning the parameters 1 and 2 (Fig. 3).

3. Evolutionary strategy Evolutionary strategy (EV) is classied in contrast to tailored methods to stochastic methods which are more suitable for general-purpose problems in which the a priori knowledge about parameters is limited [7]. Their advantages are the easy implementation even of different tting criteria, easy consideration of constraints through rejection of undesired individuals and generally robustness. Their main disadvantage is the calculation expense due to the big number of function evaluations to be carried out [6]. EV makes use of a population with a certain number of individuals. The genes of each individual correspond to the values of the unknown parameters. Every individual represents a trial solution to the tting problem and is evaluated by the corresponding tting error. Beginning with a randomly generated population, the best individuals are selected, propagated to the original population size and than mutated by changing the gene of the individuals. This process is very important for exploration, which consists in nding trial solutions in new regions of the search space. Selection and mutation processes are repeated until a certain termination condition is reached.

Fig. 3. Scaled sensitivity coefcients for Model2 .

1456

O. Kanoun et al. / Electrochimica Acta 51 (2006) 14531461

Many methods were proposed to design evolutionary strategy for a certain optimization problem. The aims thereby were, e.g. to minimize calculation time [6], to introduce some constraints or to adjust selection criterion to improve certain properties [9]. For the comparison carried out in this paper, we use conscious a standard evolutionary strategy without any adjustments. The search region, which is the only adjustment in order to consider the available knowledge about model parameters, was used in the generation of the initial population and in the mutation procedure. The population size was chosen by 400 individuals. A smaller population can lead to a more rapid convergence, but thereby the risk of converging to a local optimum increases, because no sufcient thorough search of the search space will be done. From every generation the best 40 individuals were selected by choosing the minimal values of the quality function: Quality function (P) =
k

ZMod (P) ZMeas

Fig. 5. Quality function for selected individuals (e.g. Model2 ).

(4) 4. Comparison of the evolutionary strategy with the LevenbergMarquardt method For comparison with evolutionary strategy, we chose to use the LevenbergMarquadt method, because it is nowadays more often implemented in parameter estimation tools due to of some specic advantages relative to GaussNewton method, for example. For evaluation of both methods more than 100 tting trials were carried out for every investigation. The frequency range was selected between 10 mHz and 5 kHz. The generated impedance spectra using the parameter values from Fig. 1 consist of 200 data points constantly distributed on the logarithmic axis. For comparison purpose, the options of each method, such as number of individuals for EV and termination conditions for LevMq, were set to typical values and maintained constant independent of the investigated aspect and of the used model. Therefore, the results should be not considered as the best reachable in every case. For LevMq, we used randomly varied start values with a given relative variance around the original parameter, with which the simulated spectrum were generated (Fig. 6). For EV, the search region was incrementally increased with a certain percentage from the original parameter values (Fig. 6). For graphical representation of results, we will make use of the network representation, which allows the simultaneous display of results in more than two dimensions in a plane. In this representation, axes have an intersection on a point, which is not corresponding to zero. In order to reach a better differentiation, the relative deviations of parameter from the original value are represented in percent relative deviation (Eq. (6)). 100 Pcalculated Porginal Porginal (6)

where P is the parameter vector, ZMod the impedance model, ZMeas the measurement data and k is the angular frequency. The termination condition was a given maximal number of 50 generations. During mutation processes, the parameter values (genes of individuals) are changed. The mutation was implemented under consideration of the search region, variable within the same generation and exponential declining with progress in order to increment convergence probability (Fig. 4). gi+1 = gi + g et/ randomnormal (5)

where gi is the genes of the ith generation, g the search region, the mutation variation factor within a generation, the mutation declination with processing time and randomnormal is a normal distributed random number. The quality function corresponding to selected best individuals in every generation are shown in Fig. 5. We can see that increasing the generation number leads to a continuous decrease of the quality function for the best individuals. The norms corresponding to selected individuals have a higher density towards the lowest quality function value. We see also that through the mutation process even individuals with lower performance are taken into consideration.

Fig. 4. Mutation variation factor in dependence of the number of the individual Pi within one generation.

O. Kanoun et al. / Electrochimica Acta 51 (2006) 14531461

1457

Fig. 6. Comparison of evolution strategy (EV) with LevenbergMarquardt Method (LevMq).

For example, a value of 0.1 on one axis corresponds in reality to a 0.0001 relative deviation from the original parameter value. The simulation results for Model1 are represented in Fig. 7. The small deviations between parameter values and their estimates observed for this model are not always remarked during modeling processes. If we compare the simulation results for different parameters relative to each other, we will see that conform to the results of the sensitivity analysis in Section 1, the parameter was calculated with more difculties. This problem was observed in the results obtained with both meth-

ods, because it is a problem more related to a low information content in measurement data, than to the used optimization algorithm. A main advantage which was observed by EV is the big search region allowed (Fig. 7). In the case of Model1 , the global optimum was found even by a search region of 1500% of the original parameter values and beyond. Enlarging the search region by maintaining the same population size, as carried out in this paper, leads to a less precise calculation of the optimum. The results obtained by the LevenbergMarquardt method show a signicantly higher dependence on start parameters. Already by a relative variance of 0.1 (can be compared to approximately 10% search range by the evolution) some cases of non-convergence were observed (Fig. 7d). By a relative variance of 2 non-convergences were observed in 44% of the 100 trials, which should be critically considered as a result because of the model simplicity. The main advantage of LevMq was the more precise calculation of the optimum, if it found it. Similar results were reported in [6] with the Quasi-Newton method. It was observed that QN method converged within a smaller number of function evaluations than genetic algorithms and more precise, but it did not always converge to the global optimum. The simulation results for Model2 are represented in Fig. 8. This model has shown, as it was expected, more difculties with parameter extraction. Allowed search region and

Fig. 7. Results for Model1 : (a) EV for search region 10%; (b) LevMq with start parameters within a relative variance of 0.05; (c) EV for search region 600%; (d) LevMq with start parameters within a relative variance of 0.1.

1458

O. Kanoun et al. / Electrochimica Acta 51 (2006) 14531461

Fig. 8. Results for Model2 : (a) LevMq with start parameter within a relative variance of 0.1; (b) LevMq with start parameters within a relative variance of 0.3; (c) EV for search region 20%.

relative variance were signicantly smaller than in case of Model1 . Besides, It was observed that meaningful results were obtained for a much larger search region in the case of EV (Fig. 8c) as, through the corresponding relative variance (Fig. 8a) in the case of LevMq. This means, if we have a certain knowledge concerning the expected range of a parameter, we obtain with EV a better estimation. We only noticed that the quality of parameter estimates was not uniform in case of EV. Some identication problems were observed for the parameters 1 , 2 , Cd1 and Cd2 (Fig. 8c). We can conclude that LevMq was more sensitive to the difculties related with ill-posed optimization problem. A higher sensitivity on start parameters and higher rate of nonconvergence and trapping in secondary optima were observed in every case. Already at a relative variance of 0.1 (Fig. 8a) the results were in reality unacceptable at all. We also notice that only EV has difculties by the calculation of the parameters Cd1 and Cd2 . This was also the case by the results of Model1 , but the problems to calculate the parameter were more serious. This can be explained by the relative at behavior of the quality function in dependence of these parameters (Fig. 9). In this case, the selection process is not working well related to these parameter. The sensitivity to measurement noise was comparatively investigated in [6]. The results show that evolutionary strategy is less sensitive to noise than a gradient-based method (QN). This is mainly due to noise sensitivity of the gradients calculation. With a similar model to Model1 and at a noise level of 1% both methods converged to the right optimum. At a noise level of 2%, the gradient-based method converged frequently to a local optimum corresponding to a 5% tting

error. The evolution strategy was able to nd a signicantly better t. The longer calculation time of EV can be some times critically considered. Our results can weaken this expression. They have shown, that using a today standard computer with a processor frequency of 1500 MHz the average calculation time was in case of evolution strategy approximately constant and amounted to 17 s (Model1 ). This is due to the deterministic calculation procedure. In the case of LevMq method, the calculation time was variable depending on the start parameters and was at average approximately 7 s (Model1 ). A comparative study of calculation time was carried out in [6] between genetic algorithms with real numbers, which are similar to evolutionary strategy, and Quasi-Newton method using a slight different model to Model1 . The comparison was made on the basis of the number of function evaluations and show, that tting error decreases quadratic with the number of function evaluations. For genetic algorithms the convergence

Fig. 9. Dependence of the quality function on the space charge capacity (Model2 ).

O. Kanoun et al. / Electrochimica Acta 51 (2006) 14531461

1459

results were slower at the beginning of the t algorithm and at the end. They need 20000 function evaluations. The tting error of the best individual improves in discrete jumps. The QN method needs approximately 2000 function evaluations before it reached convergence. These include the function evaluations required for calculation of gradient at each point. The counter-argument of high calculation time should be perhaps critically considered by modeling dynamically changing processes, but not in every case. At investigation stages, the differences in calculation time are no really relevant because of the high calculation power of nowadays processors. Besides, if we consider the number of trials needed for guessing start values for local optimization methods, we will see that evolutionary strategy has even benets concerning the whole modeling and parameter extraction time.

Fig. 11. Combination of evolution strategy and LevenbergMarquardt method with a search region 700% for Model1 .

5. Combination of evolutionary and LevenbergMarquardt method The main idea in this section to combine a stochastic global method with a local method [7]. The stochastic method should increase the probability of landing in the neighborhood of the global optimum and the local method can meet precisely the right value of the optimum. The proposed combination of both evolution strategy and LevenbergMarquardt method is shown in Fig. 10. The results of the EV are taken as start parameter vector for the LevMq method. In reality, this procedure can be repeated as a loop with few iterations in cases, where we should be really sure of meeting the global optimum [2]. In this paper, we will only evaluate one direct process. There are some main differences to the method proposed by Yang et al. [8] using genetic algorithms and GaussNewton method. The hybrid method by Yang et al. uses alternately one generation of the genetic algorithm followed by one attempt with the GN method. On one hand, this method is better than local optimization methods and can realize a good solution if the corresponding inverse problem to be solved is not very ill-posed and the calculation time is critical. On the other hand, this method does not prot from real advantages of global search in evolutionary strategy and is therefore less sure against trapping in local optima. The results for Model1 are shown in Fig. 11. They are better than for EV alone because of the advantages of LevMq in meeting the real value of the optimum. But the difference

Fig. 10. Combination of evolution strategy with LevenbergMarquardt method.

observed can be in reality not signicant, because it will lie under the measurement noise. The results for Model2 (Fig. 12) are better than the results of EV or LevMq alone. Relative to EV we can say, that no problems concerning Cd1 and Cd2 were observed at all (Fig. 12b and c). The main improvement relative to LevMq was the better handling of the problem related with the calculation of 1 and 2 (Fig. 12a and c). The explanation of these results is that evolution brought the parameters near the optimum and the LevenbergMarquardt method was therefore able to nd the optimum easier and more precisely. The difculties of Evolution with at quality functions could be corrected using the gradient-based method. The difculties of LevMq method by ill-posed optimization problem could be overcome with good start parameters. This hybrid solution proted from different sensitivities of both methods and allowed a better handling of difcult inverse identication problems. The quality function consists of the sum of all contributions of tting errors. Although it does not give a differentiate consideration of the calculated parameter values, it gives an insight into an optimization solution. The values of the quality function are represented in Fig. 13 for all attempts in case of Model2 . The results for evolution are the worst, because of the combination of difculties in calculation the parameters 1 , 2 , Cd1 and Cd2 . This can be surely avoided, if a bigger population number is chosen, but this kind of adjustments was not subject of investigation in this paper. The results for LevMq are generally better. The only problem thereby is the very unstable behavior, which leaded in practice to the need of several iterations with this method. The results for the combination show a denitively better result, which is reliable over the whole number of trials and makes only some non-signicant uctuations. These results demonstrate a signicant reduction of time expenditure during model selection and parameter calculation processes for a given model/data set. The proposed parameter extraction method was tested for several models with 11 and more unknown parameters [2].

1460

O. Kanoun et al. / Electrochimica Acta 51 (2006) 14531461

Fig. 12. Results for Model2 : (a) LevMq with start parameters within a relative variance of 0.1; (b) EV for a search region 10%; (c) combination of EV and LevMq with a search region 10%.

Fig. 13. Quality function for Model2 and different methods (corresponds to Fig. 12).

6. Conclusion and outlook Evolution strategy has a lot of benets for parameter estimation in impedance spectroscopy. With its stochastic properties, it is more able to nd out global minima. A comparison between evolutionary strategy and LevenbergMarquardt method has been carried out basing on simulations with a statistically signicant number of trials (>100). Comparing models with different complexity levels, we found out, that evolution alone can be a protable alternative in case of models with lower complexity. The main advantages thereby are the large search region allowed, the low sensitivity to noise and the robust convergence to the right optimum of the quality function. Difculties were only in

case of at behavior of the quality function in dependence of some parameter values. In this case the selection process was not working well considering these parameters. For models with higher complexity level and more unknown parameters, the inverse parameter extraction problem to be solved becomes more difcult. In this case, we found out, that using the results of evolution strategy as start parameter vector for LevenbergMarquardt method reached better and more reliable results compared to both methods alone over the whole number of attempts. The evolution brought the parameters near the optimum and the LevenbergMarquardt method proted from it and was therefore able to nd the optimum easier and more precisely. This hybrid solution proted also from different sensitivities of both methods and allowed a better handling of difcult modeling problems. An easily realizable implementation of evolutionary strategy in standard impedance evaluation software involves many advantages for users. As an alternative or in combination with other parameter extraction methods, it can signicantly contribute to a shortening of time expenditure for model selection and mechanisms investigations. References
[1] X. Wu, W. Zhang, J. Electroanal. Chem. 383 (1995) 1. [2] U. Tr ltzsch, O. Kanoun, H.-R. Tr nkler, Technisches Messen 71 (9) o a (2004) 509518.

O. Kanoun et al. / Electrochimica Acta 51 (2006) 14531461 [3] K.S. Tang, K.F. Man, S. Kwong, Q. He, IEEE Signal Proc. Mag. 13 (6) (June 1996) 2237. [4] B. Kost, Optimierung mit Evolutionsstrategien, Harry Deutsch Verlag, Frankfurt, 2003. [5] D.E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning, Addison-Wesley, New York, 1989. [6] T.J. Van der Noot, I. Abrahams, J. Electroanal. Chem. 448 (1998) 17. [7] G.C. Onwubolu, B.V. Babu, New Optimization Techniques in Engineering, Springer, 2004. [8] M. Yang, X. Zhang, X. Li, X. Wu, J. Electroanal. Chem. 519 (2002) 1.

1461

[9] J.X. Yu, J. Electroanal. Chem. 474 (1999) 69. [10] G.P. Kasperzyk, M. Jaskula, J. Electroanal. Chem. 567 (2004) 39. [11] D. Linden, Handbook of Batteries, McGraw-Hill, New York, 2002. [12] U. Tr ltzsch, O. Kanoun, H.-R. Tr nkler, Chemnitzer Fachtagung o a Mikrosystemtechnik, vol. 6, Chemnitz, Germany, October 2930, 2003. [13] J.V. Beck, K. Arnold, Parameter Estimation in Engineering and Science, Wiley, New York, 1977. [14] D.M. Bates, D.G. Watts, Non Linear Regression and Its Applications, Wiley, 1988. [15] L.K. Bieniasz, B. Speiser, J. Electroanal. Chem. 441 (1998) 271.