Hybrid ¢ltering to rescue stable oscillations from noise-induced chaos in continuous cultures of budding yeast

In large-scale fermentations with oscillating microbial cultures, noise is commonly present in the feed stream(s). As this can destabilize the oscillations and even generate chaotic behavior, noise ﬁlters are employed. Here three types of ﬁlters were compared by applying them to a noise-affected continuous culture of Saccharomyces cerevisiae with chaotic oscillations. The aim was to restore the original noise-free stable oscillations. An extended Kalman ﬁlter was found to be the least efﬁcient, a neural ﬁlter was better and a combined hybrid ﬁlter was the best. In addition, better ﬁltering of noise was achieved in the dilution rate than in the oxygen mass transfer coefﬁcient. These results suggest the use of hybrid ﬁlters with the dilution rate as the manipulated variable for bioreactor control.


Nomenclature
C intracellular storage carbohydrate concentration (g L À1 ) D dilution rate (h À1 ) e i key enzyme concentration for ith pathway (g g À1 biomass) E ethanol concentration in the bioreactor (g L À1 ) G glucose concentration in the bioreactor (g L À1 ) G 0 glucose concentration in the feed stream (g L À1 ) k L a oxygen mass transfer coefficient (h À1 ) K i Michaelis constant for ith pathway (g L À1 ) K O 2 ; K O 3 oxidative pathway oxygen saturation constants (mg L À1 ) O dissolved-oxygen concentration in the bioreactor (mg L À1 ) O Ã dissolved-oxygen solubility limit (mg L À1 ) r i biomass growth rate on ith pathway (h À1 ) S i carbon substrate concentration for ith pathway (g L À1 ) T elapsed time (h) u i cybernetic variable controlling key enzyme synthesis for ith pathway (-) v i cybernetic variable controlling key enzyme activity for ith pathway (-) X biomass concentration in the bioreactor (g L À1 ) Y i yield coefficient for ith pathway (g biomass g À1 substrate) Greek letters a specific enzyme synthesis rate (h À1 ) a Ã constitutive enzyme synthesis rate (g h À1 ) b specific enzyme degradation rate (h À1 ) j i stoichiometric coefficient for ith carbon substrate (-) g i stoichiometric coefficients for storage carbohydrate synthesis and degradation (-) m i specific growth rate of biomass on ith substrate (h À1 ) m i, max maximum specific growth rate on ith substrate (h À1 )

Introduction
Biochemical and metabolic processes within cells, and transport of nutrients and products across cell walls, are closely linked with observations of sustained oscillations in continuous cultures of the budding yeast Saccharomyces cerevisiae (Beuse et al., 1993;Duboc et al., 1996;Wolf et al., 2001). The occurrence and type of oscillations depend on the operating conditions, mainly the dilution rate and the rate of transport of oxygen into the culture broth (Beuse et al., 1993;Jones & Kompala, 1999).
Although it is possible to maintain prolonged oscillations of a particular type in well-controlled disturbance-free laboratory-scale bioreactors, under more realistic conditions the infiltration of noise from the environment distorts the oscillations. Oscillatory behavior in such situations then shows fluctuations around the (unobservable) deterministic (noise-free) profiles. The intrinsic stable oscillations are not just camouflaged by the fluctuations but may even be driven to chaotic behavior if the noise becomes sufficiently intense. Noise carried by feed streams is common in large-scale continuous fermentations (Rohner & Meyer, 1995). The recovery of stable, observable oscillations from chaotic data is therefore important in understanding and controlling the process, and there are continuing efforts to achieve this (Sinha, 1997).
Noise filters of different kinds have been employed. They are broadly of two kinds: algorithmic and non-algorithmic. The former are more common, and many of these have been described by Nelles (2000). The performances of those applicable to bioreactors have been studied recently (Patnaik, 2003a). Algorithmic filters require a reliable model of the process, have limited adaptability to time-dependent noise and, for the latter reason, can be difficult to optimize on-line for complex biological processes. Non-algorithmic filters, mainly based on neural networks and fuzzy logic, are more flexible, do not require a model and can be programmed for automatic on-line tuning. However, algorithmic filters reflect more faithfully the key features of a (fermentation) process, whereas neural networks are 'black box' devices that can sometimes be difficult to train before the real application (Nelles, 2000).
As neural networks are more effective than algorithmic filters in retrieving stable oscillations from noise-distorted behavior (Patnaik, 2003a), it is reasonable to expect a combination of the two to contain the effectiveness of a neural filter and the mechanistic fidelity of an algorithmic filter. This concept is also motivated by its success in simulating and controlling bioreactors with imperfect mixing and inflow of noise (Patnaik, 2003b). In a hybrid model, an algorithmic and a non-algorithmic (neural) filter operate in tandem, either independently or interactively. Details about this filter and the cultivation process are provided in later sections.
In this study, a hybrid neural filter is compared with a pure neural filter and the most common algorithmic filter, the extended Kalman filter (EKF), in order to compare their abilities to rescue steady noise-free oscillations from chaotic oscillations induced by noise in continuous cultures of S. cerevisiae. As explained here, clear oscillatory behavior provides metabolic information and is of significance for the bioprocess.

Fermentation description and data generation
Many experimental studies (reviewed by Patnaik, 2003c) have reported sustained oscillations of different types in continuous fermentations with Saccharomyces cerevisiae. However, all of them have used small laboratory-scale bioreactors that are operated under well-controlled and sanitized conditions which are free from the disturbances that inevitably occur on an industrial scale (Rohner & Meyer, 1995). As noise-affected realistic data were required for this study, and commercial and proprietary considerations restrict the availability and public disclosure of industrial data, computer-generated data simulating industrial operation were generated by 'corrupting' a mathematical model validated with laboratory data by adding noise to the substrate feed stream. The rationale and usefulness of this approach have been described and justified in many previous studies (Simutis & Lubbert, 1997;Chen & Rollins, 2000;Patnaik, 2003a, b).
As in previous studies (Patnaik, 2003a(Patnaik, , 2004a(Patnaik, , 2005, a model developed by Jones & Kompala (1999) was used to generate noise-free and noise-affected performance data. These studies have shown that Gaussian noise with even 5% variance in the inflow rate of the substrate feed stream can generate chaotic oscillations, except at large dilution rates and small mass transfer coefficients of oxygen. This variance is typical of the noise present in production processes (DiMassimo et al., 1992;Rohner & Meyer, 1995), and its chaotic effect may be observed both through the concentration profiles (Patnaik, 2005) and their Lyapunov coefficients (Patnaik, 2003a(Patnaik, , 2004a. The Jones-Kompala model was solved without noise and by adding Gaussian noise with 5% variance to the inflow rate of the substrate feed stream. The Jones-Kompala model was chosen because it expresses in a simple and adequate manner most of the key features of the oscillations observed in continuous cultures of S. cerevisiae. It also departs from most other models in a fundamental way. Whereas most other models are mechanistic, that of Jones & Kompala (1999) adopts a cybernetic perspective. The cybernetic approach (Ramkrishna et al., 1987) attributes to microorganisms the ability to decide and utilize optimally the available resources so as to maximize their own survival. The optimality is usually expressed mathematically by maximization of the growth rate. In a sense, cybernetic modeling is a formalization of a well-established evolutionary concept, and it incorporates regulatory processes within the cells, which mechanistic models do not.
With glucose as the carbon source, S. cerevisiae may follow one or more of three pathways in continuous cultures (Satroutdinov et al., 1992;Duboc et al., 1996): glucose fermentation, ethanol oxidation and glucose oxidation. When sufficient glucose is present, the organism grows on glucose and produces ethanol. However, in a glucosedepleted medium, S. cerevisiae utilizes ethanol as the carbon source. The fermentative pathway is then not followed and purely respiratory oscillations occur (Keulers et al., 1996), but their time periods are of a few minutes, whereas those with glucose cover hours or days, as the feed concentration of glucose increases (Satroutdinov et al., 1992;Beuse et al., 1993;Bai et al., 2004). These short-cycle ultradian oscillations are also mechanistically different from the longer circadian oscillations observed with large bioreactors. Moreover, growth on ethanol is not of industrial interest because the main objective is to produce ethanol.
Although ethanol is produced predominantly under anaerobic conditions in batch cultures, oscillating continuous fermentations generate ethanol in certain ranges of the dissolved oxygen concentration and the gas-liquid mass transfer coefficient (Satroutdinov et al., 1992;Patnaik, 2003c;Bai et al., 2004). Jones & Kompala (1999) postulated that dynamic competition among the pathways, according to the culture conditions, was the main cause of oscillations. They formulated cybernetic equations for each pathway and conditions for switching from one pathway to another, and showed that by manipulating the dilution rate and the mass transfer coefficient it is possible to change the occurrence and the type of oscillations.
Both smoothly oscillating noise-free profiles and chaotic profiles generated by a noisy feed stream were employed in this study. Although the model includes eight component concentrations, four key measurable ones were studied: biomass, glucose, dissolved oxygen and ethanol. These variables provide sufficient insight into the nature and the mechanism of oscillations (Duboc et al., 1996;Wolf et al., 2001).

Evaluation of filter performance
In recent studies (Patnaik, 2003a(Patnaik, , 2004a, it has been shown that the Lyapunov exponent is a compact and reliable measure of the ability of a filter to remove noise and restore nearly noise-free performance. A full description of the Lyapunov exponent is available elsewhere (Elert, 2000), so only a brief introduction sufficient for the present purpose is provided here.
Consider two trajectories in time. In our application these are a pair of time-domain concentrations of any variable, one trajectory that of a noise-free culture and the other that of the corresponding noise-distorted chaotic oscillation. Let x 0 be the value of a concentration just prior to the start (initial time t = 0) of a disturbance or noise signal, and let this value be displaced by Dx(x 0 , t) as time progresses. The initial displacement is obviously Dx(x 0 , 0). The mean exponential rate of divergence of the two trajectories is then calculated as The number l is called the Lyapunov exponent, and it applies to both continuous and discrete processes.
If l o 0, the disturbed trajectory is attracted eventually to a stable periodic orbit. For oscillating cultures of the kind analyzed here, this means the concentration profiles return to their original stable oscillations after the effect of the noise has decayed or has been removed [by methods such as the use of filters (Patnaik, 2003b(Patnaik, , 2004a]; in the limit l ! À 1, the system is said to be super-stable, i.e. no disturbance of any magnitude can permanently displace the oscillations. By contrast, l 4 0 denotes an unstable and chaotic trajectory, which is the subject of the present investigation.
The intermediate situation of l = 0 signifies a neutrally stable orbit. In the present context this means the disturbed oscillations and the original deterministic oscillations stay apart by a constant mean distance for an indefinite duration until perturbed again. Such a system is said to be Lyapunov-stable.
The extended Kalman filter (EKF) is a widely preferred algorithmic filter for bioprocess data analysis and monitoring under noise-affected conditions (Karjala & Himmelblau, 1994;Zorzetto & Wilson, 1996;Simutis & Lubbert, 1997) and it was therefore used for comparisons in the present work. The EKF performs well for Saccharomyces cerevisiae oscillations (Patnaik, 2004b), but under limited conditions and with computational rigidity. It has been shown previously (Patnaik, 2003a) that neural networks can overcome some of these weaknesses. Among different configurations, an autoassociative (AA) neural filter was selected as the best. This choice is also physically reasonable because a noise filter receives and generates the same variables after suitable processing, and an AA network has generic compatibility with this kind of processing. However, a neural filter, being essentially a 'black box' input-output mapping device, may be limited by difficulties in training, computational costs and extrapolation capability. So, to combine the advantages and reduce the weaknesses of the two kinds of filters, a hybrid filter was created by combining a neural filter and an EKF as shown in Fig. 1. Variables that have weak noise or weak influences on the fermentation can be processed by the  Figure 1 also allows information flow between the filters; the hatches across the arrows are intended to indicate that information transfer in either direction is optional. This configuration is among those recommended by Schubert et al., 1994, but the two-way internal flow of information has been added to accommodate the complexities of the intracellular biochemical reactions (Wolf et al., 2001) and to enhance flexibility.

Brief description of the EKFand the AA filter
The Kalman filter is a set of mathematical equations that provides an efficient recursive solution of the least-squares type. The filter can provide estimations of past, present and future states of a system even when a precise model is not known. This feature is useful for microbial processes under non-ideal (realistic) conditions because models developed with laboratory data may become inapplicable or imprecise under the influence of disturbances and spatial gradients (Gillard & Tragardh, 1999;Shuler & Kargi, 2002).
The Kalman filter addresses the problem of trying to estimate the state x of a discrete-time controlled process that is governed by the linear difference equation: with a measurement vector that follows: In principle, the EKF determines the current estimates of a set of variables by linearization, using the partial derivatives of the process and measurement functions evaluated at the (known) previous instant of time. The detailed theory and equations are given in the literature (Stephanopoulos & Park, 1992;Grewal & Andrews, 1993;Welch & Bishop, 2004). Although eqns (2) and (3) are in discrete forms, whereas most biological processes are described by continuous models, this is not an impediment because, in practice, data are sampled at discrete points in time.
An autoassociative neural network receives a set of inputs, processes them and generates transformed outputs of the same variables. The nature of processing or transformation depends on the application. In this study, processing involved reduction of the noise in the feed stream. Although the noise is present directly in the flow rate, it also affects other variables, as eqns (A1)-(A6) show, because they are mechanistically connected to the feed stream. Moreover, a neural filter is normally used in conjunction with a neural controller (Patnaik, 2003b), which uses output information to adjust the input variables continually. The liquid feed stream (of glucose) is characterized by its concentration and flow rate, and only the flow rate of air may be adjusted as its composition is fixed. Thus, the AA filter has three neurons each in the input and output layers, and the number of neurons in the hidden layer was adjusted until the output profiles were within 2% of the input profiles. As shown in a previous study with a purely neural filter (Patnaik, 2003a), the optimum number turned out to be two, thus generating the topology shown in Fig. 2.
As both the EKF and the AA neural network allow any arbitrary variation in the sampling interval, this may be varied according to the nature of the process. For instance, the interval may be made inversely proportional to the current concentration gradient, generating closely spaced data when the variations are steep and more widely separated points during mild variations (Patnaik, 1997).
Earlier studies Rohner & Meyer, 1995;Patnaik, 1997) have suggested that the feed stream is a major carrier of noise in continuous and fedbatch fermentations, and white noise is the principal component of the observed fluctuations. To generate data simulating a noise-influenced oscillating culture, therefore, the equations in the Appendix were solved with the parameter values used by Jones & Kompala (1999) (see Table 1) and white noise in the flow rate of the substrate. Data from the simulated profiles were sampled at intervals inversely proportional to the local concentration gradients.

Application and discussion
To maintain consistency with earlier work (Patnaik, 2004a, b), the same case studies were chosen from Jones & Kompala (1999). They considered the effects of changes in the dilution rate and the gas-liquid mass transfer coefficient of oxygen on the occurrence and the nature of oscillations. These two variables are commonly used in control policies for continuous fermentations, with the dilution rate being preferred (Henson & Seborg, 1992;Dochain & Perrier, 1997).
Studies by Jones & Kompala (1999) and others (Beuse et al., 1993;Duboc et al., 1996) have shown that oscillations decay in both amplitude and frequency as the dilution rate is increased, whereas the mass transfer coefficient has the opposite effect. These studies in laboratory-scale bioreactors were not influenced by the inflow of noise and did not show any chaotic oscillations. In the present context this observation is significant. As there was no deterministic chaos, any chaos observed in the simulated data was due to noise alone. Thus, the Lyapunov exponents indicate purely noise-induced chaos and provide reliable comparisons of filtering devices for their abilities to rescue noise-free oscillations.
The progress of Lyapunov exponents for four variables normally monitored for fermentation performance is compared in Figs 3 and 4, the former for the dilution rate and the latter for the mass transfer coefficient. Just as the deterministic oscillations decay with increasing dilution rate and amplify with increasing oxygen mass transfer coefficient, so do the corresponding Lyapunov exponents. This implies that strongly oscillating cultures are more likely to Fig. 3. Variations of the Lyapunov exponents with the dilution rate for noise-free (empty circles) and noise-distorted (filled circles) cultures.  (Jones & Kompala, 1999) Parameter Units Value a h À 1 1.0 a Ã g h À 1 0.1 b h À 1 0.2 g 1 g g À 1 6.0 g 2 g g À 1 6.0 g 3 g g À 1 0.3 m 1,max h À 1 0.44 m 2,max h À 1 0.32 m 3,max h À 1 0.31 j 1 g g À 1 0.27 j 2 g g À 1 1.067 j 3 g g À 1 2.087 Stable oscillations from noise-induced chaos be destabilized and eventually driven to chaos by noise in the substrate feed stream. Both sets of figures also show that the Lyapunov exponents without noise are consistently smaller than zero, whereas those with noise are positive. These results corroborate the absence of chaos in noise-free experiments (Beuse et al., 1993;Duboc et al., 1996;Jones & Kompala, 1999). The effectiveness of different filtering devices is compared in Figs 5 and 6. Representative values of the dilution rate and the oxygen mass transfer coefficient were chosen from the work of Jones & Kompala (1999). All three types of filters eliminate noise significantly, but there are also equally significant improvements from an EKF to a neural filter to a hybrid filter. The EKF was chosen among the algorithmic filters because of its popularity in bioreactor applications as well as its suitability for oscillating fermentations (Patnaik, 2004b).
Although the EKF removes a substantial part of the noise in the feed stream (as evident from the large reductions in the Lyapunov exponents), it does not always restore stable oscillations. For dilution rates of 0.10 h À1 for glucose and dissolved oxygen, 0.10 and 0.13 h À1 for ethanol, and all three dilution rates for biomass, the filtered exponents still remain positive, indicating residual chaos (Fig. 5). Similar results are seen at all the three mass transfer coefficients for biomass and dissolved oxygen, whereas for ethanol the culture is just marginally stable at 275 and 325 h À1 , implying that a small perturbation can again generate chaos (Fig. 6).
Although the hybrid model, combining an EKF and a neural filter as in Fig. 1, creates the largest improvements toward noise-free stable oscillations, these improvements are generally much smaller for noise reductions in the mass transfer coefficients (Fig. 6) than in the dilution rates (Fig. 5). Even with a hybrid neural filter, oscillations in the biomass concentration remain precariously close to a relapse into the chaotic regime.
As the biomass concentration is the most severely affected by noise (Patnaik, 2005), the rates of convergence of all three filters in trying to restore stable oscillations in this variable are compared in Fig. 7. The relative performances correlate well with their transient learning abilities. This is characterized by the mean sum of squares of errors, defined as where X e j and X p j are respectively the 'experimental' (or simulated) and predicted values of a variable at the jth point  in time, and N is the total number of data. Here, too, a hybrid neural filter outperforms a pure neural filter and an EKF. The relatively inferior performances of all noise filters for biomass concentration when compared with other variables are because the oscillations in this variable are disrupted more strongly by inflow of noise than are other concentrations. Likewise, better filtering of noise is possible in the dilution rates than in the oxygen mass transfer coefficients, which enhances its suitability as a manipulated variable for bioreactor control. This further supports Henson & Seborg's (1992) recommendation to employ input -output linearizing control based on the dilution rate.

Conclusions
It is difficult to eliminate the flow of noise into large-scale fermentations. However, it is important to generate reasonably noise-free performance to identify and act upon salient features of the process and to avoid destabilization and degeneration into chaotic behavior.
The restoration of smooth oscillations from chaos induced by noise in the feed stream to continuous fermentations with Saccharomyces cerevisiae has been explored here. Three kinds of noise filter were investigated: the extended Kalman filter (EKF), an autoassociative (AA) neural filter, and a hybrid filter, which was a combination of these. The effectiveness of each filter was measured by calculating the Lyapunov exponents from the time-domain profiles of the variables of interest. Large positive exponents denote a preponderance of chaos, and large negative exponents indicate a return to stable periodic orbits. The EKF was the least efficient, the AA neural filter was better and the hybrid filter was the best. This underlines a fundamental weakness of algorithmic filters, of which the EKF is the most commonly used.
Two controllable variables determine the nature of S. cerevisiae oscillations: the dilution rate and the mass transfer coefficient of oxygen to the liquid. The performances of all three kinds of filter were better for the dilution rate (i.e. the flow rate of the substrate) than for the mass transfer coefficient. This observation and the superiority of a hybrid neural filter strengthen the preference for the dilution rate as a manipulated variable and of hybrid neural networks for non-ideal bioreactor simulation and control.
Inclusion of the term a Ã in the enzyme synthesis equations (A10) is based on Turner & Ramkrishna (1988), who have shown its importance in predicting the induction of enzymes that have been repressed for long durations. The specific growth rates thus also include a Ã in the model: Equation (A11) expresses the rate of change of internal storage carbohydrates that are an integral part of the metabolism (Satroutdinov et al., 1992;Duboc et al., 1996).
The j i are the stoichiometric coefficients for different substrates S i , and g i are similar coefficients for carbohydrate synthesis and consumption by the cells. Jones & Kompala (1999) may be consulted for a full discussion of the model. A point not clarified there is the identification of S 1 , S 2 and S 3 . Reference to eqns (A1)-(A3) shows that S 1 = G, S 2 = E and S 3 = G. This identification is needed to solve the model. The values of the parameters are listed in Table 1.