-
PDF
- Split View
-
Views
-
Cite
Cite
Andre Russowsky Brunoni, Andrew H. Kemp, Eduardo M. Dantas, Alessandra C. Goulart, Maria Angélica Nunes, Paulo S. Boggio, José Geraldo Mill, Paulo A. Lotufo, Felipe Fregni, Isabela M. Benseñor, Heart rate variability is a trait marker of major depressive disorder: evidence from the sertraline vs. electric current therapy to treat depression clinical study, International Journal of Neuropsychopharmacology, Volume 16, Issue 9, October 2013, Pages 1937–1949, https://doi.org/10.1017/S1461145713000497
- Share Icon Share
Abstract
Decreased heart rate variability (HRV) is a cardiovascular predictor of mortality. Recent debate has focused on whether reductions in HRV in major depressive disorder (MDD) are a consequence of the disorder or a consequence of pharmacotherapy. Here we report on the impact of transcranial direct current stimulation (tDCS), a non-pharmacological intervention, vs. sertraline to further investigate this issue. The employed design was a double-blind, randomized, factorial, placebo-controlled trial. One hundred and eighteen moderate-to-severe, medication-free, low-cardiovascular risk depressed patients were recruited for this study and allocated to either active/sham tDCS (10 consecutive sessions plus two extra sessions every other week) or placebo/sertraline (50 mg/d) for 6 wk. Patients were age and gender-matched to healthy controls from a concurrent cohort study [the Brazilian Longitudinal Study of Adult Health (ELSA-Brasil)]. The impact of disorder, treatment and clinical response on HRV (root mean square of successive differences and high frequency) was examined. Our findings confirmed that patients displayed decreased HRV relative to controls. Furthermore, HRV scores did not change following treatment with either a non-pharmacological (tDCS) or pharmacological (sertraline) intervention, nor did HRV increase with clinical response to treatment. Based on these findings, we discuss whether reduced HRV is a trait-marker for MDD, which may predispose patients to a host of conditions and disease even after response to treatment. Our findings have important implications for our understanding of depression pathophysiology and the relationship between MDD, cardiovascular disorders and mortality.
Introduction
Major depressive disorder (MDD) is the most prevalent psychiatric disorder and will be one of the most disabling medical conditions by 2020 (Murray and Lopez, 1997). Furthermore, MDD and cardiovascular disorders are associated; a depressive episode following an acute coronary syndrome more than doubles mortality risk over 7 yr (Glassman et al., 2009), and depression increases risk of cardiac mortality three- to fourfold (Penninx et al., 2001). Antidepressant drugs – the mainstream treatment for MDD – have also been independently associated with increased cardiovascular risk and sudden cardiac death (Whang et al., 2009). Hence, a concerning possibility is that antidepressants might have direct side-effects on heart activity that surpass the potential clinical benefits of treating the depression. However, other studies, including a large cohort of depressed patients, have reported protective cardiovascular effects for antidepressants (Garfield et al., 2011), so, it remains unclear whether these medications are advantageous or hazardous for the heart (Brunoni et al., 2012a). An outstanding question is whether different antidepressant treatments have differential effects on validated markers of cardiac health, such as heart rate variability (HRV).
HRV is an electrocardiograph-based technique developed to assess the relative influences of sympathetic and vagal branches over heart beat-to-beat activity (ESC, 1996). HRV is considered an important marker for cardiovascular events and its complications (Buccelletti et al., 2009); for instance, low HRV (i.e. decreased vagal activity) predicts death after myocardial infarction (Bigger et al., 1992). Recently, we demonstrated (Kemp et al., 2010) that unmedicated patients with MDD, free from cardiovascular disease, display reductions in HRV, and that these reductions are not ameliorated with treatment, nor with symptom resolution. By contrast, others have reported that antidepressant medications, including the tricyclics, the serotonin and noradrenaline reuptake inhibitors (SNRIs) and the selective serotonin reuptake inhibitors (SSRIs) – not depression per se – adversely impact on HRV (Licht et al., 2008; Kemp et al., 2010). Longitudinal findings further revealed that withdrawal from antidepressants was associated with an increase in HRV (Licht et al., 2010). Such findings suggest that antidepressants, acting on the relay nuclei of the autonomous nervous system, could be responsible for these findings; however, methodological issues hamper solid conclusions, as discussed previously (Kemp, 2011). The resolution of the etiology of decreased HRV in depression is paramount to understanding the relationship between depression, antidepressants, HRV and cardiovascular disorders, and has important clinical and theoretical implications for MDD treatment.
We recently proposed (Brunoni et al., 2012a) that transcranial direct current stimulation (tDCS) – a non-pharmacological, neuromodulatory technique involving the application of a direct electric current to the scalp (Brunoni et al., 2012c) – could be used to disentangle whether it is the antidepressant and/or depression that adversely impacts on HRV (Kemp et al., 2011; Brunoni et al., 2012a). There is growing evidence that tDCS is an effective treatment for MDD, including one meta-analysis and one large clinical trial (Brunoni et al., 2012b; Kalu et al., 2012). Moreover, tDCS is an intervention with minimal, systemic side-effects (Brunoni et al., 2010). In fact, acute tDCS seems to have either no or favorable effects on sympathovagal balance (Sampaio et al., 2012; Vandermeeren et al., 2010; Montenegro et al., 2011; Brunoni et al., 2013a) – the latter effects possibly due to the role of the prefrontal cortex in inhibiting cardio-acceleratory circuitry, thereafter increasing vagal activity (Thayer et al., 2010). Therefore, our hypothesis was that reductions in HRV following antidepressant drugs and increases in HRV following tDCS would demonstrate the adverse effects of pharmacological vs. non-pharmacological treatments. This would highlight a particularly important role for non-pharmacological interventions in treating depressed patients with high cardiovascular risk. Here we compare sertraline – a medication in the SSRI class of antidepressants; a first-line treatment of depression (Cipriani et al., 2009) – with tDCS to examine this possibility. Recent longitudinal research highlights the adverse cardiovascular effects (HRV reductions) of multiple classes of antidepressants including SSRIs (Licht et al., 2008; Kemp et al., 2010). By contrast, a lack of change in HRV following treatment – and the tDCS responders in particular – would provide evidence for HRV as a trait marker of MDD (and also evidence against the hypothesis that antidepressants have unfavorable effects on HRV). This finding would provide a possible explanation for the increased morbidity and mortality from a host of cardiovascular conditions that have been reported in depressed patients (Penninx et al., 2001; Glassman et al., 2009).
This study presents HRV data collected in the Sertraline vs. Electric Current Therapy for Treating Depression Clinical Study (SELECT-TDCS, Brunoni et al., 2011b), a study that enrolled 120 patients with depression, which was conducted at Hospital Universitário, University of São Paulo, Brazil. Patients were randomized to four groups: (1) sham tDCS – placebo pill, (2) sham tDCS – sertraline, (3) active tDCS – placebo pill and (4) active tDCS – sertraline. We previously reported that patients allocated to the combined treatment therapy arm displayed increased and faster antidepressant efficacy relative to other groups, and that the efficacy of tDCS-only and sertraline-only groups was similar (Brunoni et al., 2013b). Here, we compared HRV values across all groups to determine whether HRV changes would be associated with depression improvement and/or tDCS/sertraline interventions. We also compared these patients with gender- and age-matched healthy controls from the Brazilian Longitudinal Study of Adult Health (ELSA-Brasil, Richter et al., 2010), to confirm whether HRV in depressed patients is different from that in individuals without depression, as reported in our previous work on independent samples (Kemp et al., 2010, 2012). Further, this is also the first study to investigate the effects of daily-repeated tDCS on HRV, an important safety consideration regarding its role as a therapy for MDD (Brunoni et al., 2011a).
Methods and materials
The complete design of SELECT-TDCS trial and the ELSA-Brasil cohort are described elsewhere (Richter et al., 2010; Brunoni et al., 2011b). Here, we detail the aspects relevant to the present study. Both studies were approved by the Local and National Ethics Committee, and all subjects provided written, informed consent.
Participants
Patients met the following criteria: (1) acute, unipolar, non-psychotic MDD episode, (2) Hamilton Depression Rating Scale scores >17, (3) age between 18 and 65 yr and (4) absence of other Axis I, II and III disorders (although depressed patients with anxiety symptoms were allowed). Diagnoses were determined by certified psychiatrists through clinical interview and the Mini International Neuropsychiatric Interview (MINI) questionnaire (Sheehan et al., 1998).
Prior to trial onset, previously medicated participants underwent a drug washout period lasting at least five half-lives of the drug (mean washout period of 18 d). Low-dose (up to 20 mg/d diazepam-equivalents) benzodiazepines were the only drug on which participants (n = 23, 19%) were allowed to continue, since complete drug withdrawal would require specific interventions to avoid the benzodiazepine withdrawal syndrome, ultimately hindering enrolment and external validity of results of this trial. Similar use of low-dose benzodiazepines has also been employed in other larger brain stimulation trials for depression (O'Reardon et al., 2007; George et al., 2010).
Patients were matched according to age, gender, cardiovascular risk and medication status (i.e. drug-nałx00C5;x00EF;ve or low-dose benzodiazepines) with controls. Patients under 35 yr (29% of the sample) were matched with controls that were recruited by word-of-mouth; these participants included staff and students from the University of São Paulo. Patients over 35 yr (71% of the sample) were matched with the ELSA-Brasil database (Richter et al., 2010), a naturalistic cohort of 15105 subjects aged łx00C5;x025B;x2122;25CB;25A1;2A7D;2A7E;35 yr. Importantly, this large number allowed us to perform an exact match on almost all clinical and demographic variables, including age, gender, medication use and physical co-morbidities. The questionnaire employed in ELSA-Brasil was the Clinical Interview Schedule – Revised (CIS-R), which was specifically developed for assessing common mental disorders (Lewis et al., 1992; Nunes et al., 2011). Specifically for depression, the CIS-R assesses the domains of appetite, somatic symptoms, fatigue, concentration, sleep, irritability, depressive mood, depressive ideas, worry and anxiety. We selected controls that did not display any depressive symptom across all domains.
Procedures
Randomization was performed using a 1:1:1:1 permuted block randomization, and the allocation was concealed using a central randomization method. Participants were, therefore, randomized into four treatment groups. The complete trial duration was 6 wk; comprising an acute treatment period, in which 10 daily, consecutive tDCS sessions were performed, and two follow-up tDCS sessions given every other week. Sertraline 50 mg/d was started simultaneously with tDCS. TDCS intervention comprised placement of the anode over the (EEG 10–20) F3 area (corresponding to the left dorsolateral prefrontal cortex) and the cathode over the F4 area (right dorsolateral prefrontal cortex). We applied a direct current of 2 mA (current density = 0.80 A/m2) for 30 min/d for 10 d, followed by two extra tDCS sessions every other week until the study endpoint (total charge density of 1728 C/m2). For sham conditions, the device was turned off after 1 min of active stimulation, a blinding method that has been previously described as reliable (Brunoni et al., 2012c).
Depression severity was measured by the Montgomery–Asberg Depression Rating Scale (MADRS) score. Responders were defined as having at least a 50% reduction of MADRS scores relative to baseline scores. We also assessed other common clinical characteristics, including age, gender, duration of index episode, refractoriness, subtype of depression and so on. In addition, patients were asked about risk factors for cardiovascular disease, such as previous diagnosis of hypertension, diabetes, myocardial infarction and smoking status. Body mass index (BMI) was determined according to self-reported height and weight.
A digital electrocardiograph (Micromed, sampling rate of 250 Hz) was used to acquire the records. The electrodes were placed on the limbs, and the signals were recorded for 15 min from the higher R-wave amplitude lead (usually D2).
Electrocardiograms were collected on the first and last day of treatment. The possible confounding effect of acute tDCS effects on HRV were avoided by always collecting ECGs before the tDCS session. Data from both depressed subjects and control participants were always collected in the afternoon (between 13:00 and 16:00 hours. Participants were at rest at least 1 h prior to ECG acquisition, which was conducted in a quiet, temperature-controlled room set at 22°C. Importantly, controls were recruited from the same center where our clinical trial was conducted, using the same devices and procedures for HRV collection (in fact, the same personnel collected and analyzed the HRV data from both studies).
Data analysis
Researchers (E.M.D. and J.G.M.) who analyzed the HRV were blinded to treatment allocation and were not involved in any other aspect of the trial. Wincardio 4.4 was used to generate beat-to-beat R-R interval series; artifacts and ectopic beats were corrected by the spline cubic interpolation method. Almost 30 HRV variables can be extracted from the electrocardiogram. Here we focused on two of these – the most extensively evaluated in studies with depression (Kemp et al., 2010) – to avoid bias related to multiple analyses (such as false positive results). These included: Higher values on each of these variables reflect higher variability.
in the time-domain, the root mean square of successive differences (RMSSD): RMSSD is the most appropriate time-domain measure for short-duration ECG recordings (ESC, 1996).
in the frequency-domain, high frequency (HF, 0.15–0.4 Hz): a Fast Fourier Transform (FFT) was employed to extract the HF index – an index of activity within the parasympathetic branch (ESC, 1996) – from the R-R time-series.
We also performed exploratory analyses in other five HRV indices: approximate entropy and the two Poincaré plots, nonlinear measures that might be sensitive in detecting HRV differences between groups (Kemp et al., 2010) and decreased during stress (Melillo et al., 2011), and the symbolic measures 0 and 2 V, since it was experimentally shown that sympathetic activation increases 0 V and decreases 2 V, and parasympathetic activation decreases 0 V and increases 2 V (Guzzetti et al., 2005). We did not analyze low frequency (LF) activity and LF/HF because recent data questioned whether LF reflects sympathetic and/or parasympathetic activity (Reyes Del Paso et al., 2013).
Due to technical reasons and dropouts, HRV data were not collected in all participants and we were therefore able to analyze HRV measures from 118 patients at baseline and 93 at endpoint.
Statistical analysis
We used Stata 12 for Mac OS (Statacorp, USA) for statistical analyses. Data normality was assessed by the Shapiro–Wilk test and homogeneity of variances by the Levene test. We also assessed outliers by examining observations beyond mean ± 3 s.d. and also using the boxplot method, using ‘fences’ of three interquartile ranges above and below the third and the first quartile, respectively, as limits for non-outlier observations (Seo, 2006). Baseline clinical and demographic characteristics among groups were assessed using one-way ANOVA or the χ2 statistics. We also performed Pearson's correlations to assess whether depression severity and HRV variables were correlated and also to explore the relationship between the changes of depression scores and the changes of RMSSD and HF. RMSSD and HF variables underwent natural logarithmic transformation for normalization. Statistical significance level was set at p < 0.05 for all comparisons performed, after correcting post-hoc pairwise comparisons using the Bonferroni method. Cohen's d was used to estimate the effect size of between-group differences; values of 0.2, 0.5 and 0.8 represent small, medium and large effect sizes, respectively (Cohen, 1988).
Our first step was to compare healthy controls and depressed subjects on HRV (i.e. HF and RMSSD). Between-group t tests were conducted to confirm that patients displayed reduced HRV, consistent with our prior studies (Kemp et al., 2010, 2012) and the broader literature.
After this, we performed two repeated-measures ANOVAs, using each HRV variable as the dependent variable separately. The between-subject factors were group (placebo/sham tDCS, sertraline/sham tDCS, placebo/active tDCS and sertraline/active tDCS) and response (non-response, response). Time (pre- and post-scores) was the within-subject factor. Therefore, we analyzed the effects of treatment by verifying whether or not the main factor of time and its interactions with group, response and group × response were significant.
Finally, as benzodiazepines influence HRV (Galletly et al., 1996) – such that this medication decreases HRV – sensitivity analyses were performed to assess whether removal of patients on this drug class would affect the observed results.
Results
Participant characteristics
Patients and healthy controls were similar regarding age, gender and other clinical characteristics (Table 1). No significant differences between patient groups were observed on clinical, cardiovascular and HRV measurements at baseline assessment; nor did HRV correlate with MADRS depression scores at baseline (r's < 0.07 and p's > 0.15). Patients presented an overall low cardiovascular risk – no patients with prior myocardial infarction or other cardiovascular/cerebrovascular conditions were included in the study. (Table 2)
Clinical and demographic characteristics of depressed patients and matched controls
. | Patients with depression . | Healthy controls . | p . |
---|---|---|---|
Gender (male/female) | 38/82 | 38/82 | 1 |
Age (s.d.) | 42.27 (12.6) | 42.9 (13.17) | 0.7 |
BMI (s.d.) | 25.86 (5.15) | 24.7 (8.1) | 0.19 |
Hypertension, n (%) | 27 (22.5%) | 23 (19.1%) | 0.51 |
Diabetes mellitus, n (%) | 6 (5%) | 6 (5%) | 1 |
Current smokers, n (%) | 21 (17.5%) | 18 (15%) | 0.59 |
Use of benzodiazepines, n (%) | 23 (19%) | 23 (19%) | 1 |
. | Patients with depression . | Healthy controls . | p . |
---|---|---|---|
Gender (male/female) | 38/82 | 38/82 | 1 |
Age (s.d.) | 42.27 (12.6) | 42.9 (13.17) | 0.7 |
BMI (s.d.) | 25.86 (5.15) | 24.7 (8.1) | 0.19 |
Hypertension, n (%) | 27 (22.5%) | 23 (19.1%) | 0.51 |
Diabetes mellitus, n (%) | 6 (5%) | 6 (5%) | 1 |
Current smokers, n (%) | 21 (17.5%) | 18 (15%) | 0.59 |
Use of benzodiazepines, n (%) | 23 (19%) | 23 (19%) | 1 |
BMI, Body mass index (kg/m2); s.d., standard deviation.
The table displays clinical data from patients (from the SELECT-TDCS clinical trial) and controls (from the ELSA-Brasil cohort). The groups were compared with t tests, χ2 tests or Fisher's exact test, when appropriate.
Clinical and demographic characteristics of depressed patients and matched controls
. | Patients with depression . | Healthy controls . | p . |
---|---|---|---|
Gender (male/female) | 38/82 | 38/82 | 1 |
Age (s.d.) | 42.27 (12.6) | 42.9 (13.17) | 0.7 |
BMI (s.d.) | 25.86 (5.15) | 24.7 (8.1) | 0.19 |
Hypertension, n (%) | 27 (22.5%) | 23 (19.1%) | 0.51 |
Diabetes mellitus, n (%) | 6 (5%) | 6 (5%) | 1 |
Current smokers, n (%) | 21 (17.5%) | 18 (15%) | 0.59 |
Use of benzodiazepines, n (%) | 23 (19%) | 23 (19%) | 1 |
. | Patients with depression . | Healthy controls . | p . |
---|---|---|---|
Gender (male/female) | 38/82 | 38/82 | 1 |
Age (s.d.) | 42.27 (12.6) | 42.9 (13.17) | 0.7 |
BMI (s.d.) | 25.86 (5.15) | 24.7 (8.1) | 0.19 |
Hypertension, n (%) | 27 (22.5%) | 23 (19.1%) | 0.51 |
Diabetes mellitus, n (%) | 6 (5%) | 6 (5%) | 1 |
Current smokers, n (%) | 21 (17.5%) | 18 (15%) | 0.59 |
Use of benzodiazepines, n (%) | 23 (19%) | 23 (19%) | 1 |
BMI, Body mass index (kg/m2); s.d., standard deviation.
The table displays clinical data from patients (from the SELECT-TDCS clinical trial) and controls (from the ELSA-Brasil cohort). The groups were compared with t tests, χ2 tests or Fisher's exact test, when appropriate.
. | Sham tDCS . | Active tDCS . | . | ||
---|---|---|---|---|---|
. | Placebo . | Sertraline . | Placebo . | Sertraline . | |
Clinical characteristics | |||||
Female/male (n) | 14/9 | 12/11 | 18/5 | 19/6 | 0.28 |
Age (s.d.) | 46.4 (13.7) | 40.8 (11.9) | 41.2 (12) | 40.7 (12.7) | 0.24 |
Baseline MADRS (s.d.) | 30.8 (5.3) | 30.5 (6.8) | 30.8 (5.8) | 30.7 (6.7) | 0.99 |
Cardiovascular characteristics | |||||
BMI (s.d.) | 25.4 (5.8) | 25.7 (4.3) | 26.3 (5.4) | 25.9 (5.2) | 0.92 |
Hypertension (%) | 8 (27%) | 6 (20%) | 9 (30%) | 4 (13%) | 0.42 |
Diabetes mellitus (%) | 1 (3%) | 3 (10%) | 1 (3%) | 1 (3%) | 0.55 |
Current smokers (%) | 7 (23%) | 7 (23%) | 3 (10%) | 4 (13%) | 0.4 |
HRV baseline characteristics | |||||
ln RMSSD mean (s.d.) | 3.6 (0.9) | 3.2 (0.6) | 3.6 (0.8) | 3.3 (0.8) | 0.21 |
ln HF mean (s.d.) | 6 (1.8) | 5.3 (1.2) | 6.1 (1.6) | 5.6 (1.5) | 0.24 |
Depression endpoint scores | |||||
Endpoint MADRS (s.d.) | 24.7 (8.6) | 21.7 (13.1) | 19.1 (12.2) | 13.1 (8.5) | <0.01 |
Response rate (%) | 5 (17%) | 10 (33%) | 13 (43%) | 19 (63%) | <0.01 |
Remission rate (%) | 4 (13%) | 9 (30%) | 12 (40%) | 14 (46.7%) | 0.03 |
. | Sham tDCS . | Active tDCS . | . | ||
---|---|---|---|---|---|
. | Placebo . | Sertraline . | Placebo . | Sertraline . | |
Clinical characteristics | |||||
Female/male (n) | 14/9 | 12/11 | 18/5 | 19/6 | 0.28 |
Age (s.d.) | 46.4 (13.7) | 40.8 (11.9) | 41.2 (12) | 40.7 (12.7) | 0.24 |
Baseline MADRS (s.d.) | 30.8 (5.3) | 30.5 (6.8) | 30.8 (5.8) | 30.7 (6.7) | 0.99 |
Cardiovascular characteristics | |||||
BMI (s.d.) | 25.4 (5.8) | 25.7 (4.3) | 26.3 (5.4) | 25.9 (5.2) | 0.92 |
Hypertension (%) | 8 (27%) | 6 (20%) | 9 (30%) | 4 (13%) | 0.42 |
Diabetes mellitus (%) | 1 (3%) | 3 (10%) | 1 (3%) | 1 (3%) | 0.55 |
Current smokers (%) | 7 (23%) | 7 (23%) | 3 (10%) | 4 (13%) | 0.4 |
HRV baseline characteristics | |||||
ln RMSSD mean (s.d.) | 3.6 (0.9) | 3.2 (0.6) | 3.6 (0.8) | 3.3 (0.8) | 0.21 |
ln HF mean (s.d.) | 6 (1.8) | 5.3 (1.2) | 6.1 (1.6) | 5.6 (1.5) | 0.24 |
Depression endpoint scores | |||||
Endpoint MADRS (s.d.) | 24.7 (8.6) | 21.7 (13.1) | 19.1 (12.2) | 13.1 (8.5) | <0.01 |
Response rate (%) | 5 (17%) | 10 (33%) | 13 (43%) | 19 (63%) | <0.01 |
Remission rate (%) | 4 (13%) | 9 (30%) | 12 (40%) | 14 (46.7%) | 0.03 |
tDCS, Transcranial direct current stimulation; MADRS, Montgomery–Asberg Depression Rating Scale; BMI, body mass index (kg/m2); RMSSD (ms), the square root of the mean squared difference of successive normal to normal intervals; HF, high-frequency (ms2).
RMSSD and HF are transformed in the natural logarithmic base (ln).
. | Sham tDCS . | Active tDCS . | . | ||
---|---|---|---|---|---|
. | Placebo . | Sertraline . | Placebo . | Sertraline . | |
Clinical characteristics | |||||
Female/male (n) | 14/9 | 12/11 | 18/5 | 19/6 | 0.28 |
Age (s.d.) | 46.4 (13.7) | 40.8 (11.9) | 41.2 (12) | 40.7 (12.7) | 0.24 |
Baseline MADRS (s.d.) | 30.8 (5.3) | 30.5 (6.8) | 30.8 (5.8) | 30.7 (6.7) | 0.99 |
Cardiovascular characteristics | |||||
BMI (s.d.) | 25.4 (5.8) | 25.7 (4.3) | 26.3 (5.4) | 25.9 (5.2) | 0.92 |
Hypertension (%) | 8 (27%) | 6 (20%) | 9 (30%) | 4 (13%) | 0.42 |
Diabetes mellitus (%) | 1 (3%) | 3 (10%) | 1 (3%) | 1 (3%) | 0.55 |
Current smokers (%) | 7 (23%) | 7 (23%) | 3 (10%) | 4 (13%) | 0.4 |
HRV baseline characteristics | |||||
ln RMSSD mean (s.d.) | 3.6 (0.9) | 3.2 (0.6) | 3.6 (0.8) | 3.3 (0.8) | 0.21 |
ln HF mean (s.d.) | 6 (1.8) | 5.3 (1.2) | 6.1 (1.6) | 5.6 (1.5) | 0.24 |
Depression endpoint scores | |||||
Endpoint MADRS (s.d.) | 24.7 (8.6) | 21.7 (13.1) | 19.1 (12.2) | 13.1 (8.5) | <0.01 |
Response rate (%) | 5 (17%) | 10 (33%) | 13 (43%) | 19 (63%) | <0.01 |
Remission rate (%) | 4 (13%) | 9 (30%) | 12 (40%) | 14 (46.7%) | 0.03 |
. | Sham tDCS . | Active tDCS . | . | ||
---|---|---|---|---|---|
. | Placebo . | Sertraline . | Placebo . | Sertraline . | |
Clinical characteristics | |||||
Female/male (n) | 14/9 | 12/11 | 18/5 | 19/6 | 0.28 |
Age (s.d.) | 46.4 (13.7) | 40.8 (11.9) | 41.2 (12) | 40.7 (12.7) | 0.24 |
Baseline MADRS (s.d.) | 30.8 (5.3) | 30.5 (6.8) | 30.8 (5.8) | 30.7 (6.7) | 0.99 |
Cardiovascular characteristics | |||||
BMI (s.d.) | 25.4 (5.8) | 25.7 (4.3) | 26.3 (5.4) | 25.9 (5.2) | 0.92 |
Hypertension (%) | 8 (27%) | 6 (20%) | 9 (30%) | 4 (13%) | 0.42 |
Diabetes mellitus (%) | 1 (3%) | 3 (10%) | 1 (3%) | 1 (3%) | 0.55 |
Current smokers (%) | 7 (23%) | 7 (23%) | 3 (10%) | 4 (13%) | 0.4 |
HRV baseline characteristics | |||||
ln RMSSD mean (s.d.) | 3.6 (0.9) | 3.2 (0.6) | 3.6 (0.8) | 3.3 (0.8) | 0.21 |
ln HF mean (s.d.) | 6 (1.8) | 5.3 (1.2) | 6.1 (1.6) | 5.6 (1.5) | 0.24 |
Depression endpoint scores | |||||
Endpoint MADRS (s.d.) | 24.7 (8.6) | 21.7 (13.1) | 19.1 (12.2) | 13.1 (8.5) | <0.01 |
Response rate (%) | 5 (17%) | 10 (33%) | 13 (43%) | 19 (63%) | <0.01 |
Remission rate (%) | 4 (13%) | 9 (30%) | 12 (40%) | 14 (46.7%) | 0.03 |
tDCS, Transcranial direct current stimulation; MADRS, Montgomery–Asberg Depression Rating Scale; BMI, body mass index (kg/m2); RMSSD (ms), the square root of the mean squared difference of successive normal to normal intervals; HF, high-frequency (ms2).
RMSSD and HF are transformed in the natural logarithmic base (ln).
Comparison with controls
HRV values between controls and subjects at baseline were significantly different both for RMSSD (t117 = 2.86, p = 0.005, Cohen's d = 0.35) and HF (t117 = 3.77, p < 0.001, Cohen's d = 0.45) (Fig. 1, Table 3). This confirms that our depressed patients present decreased HRV.

Heart rate variability scores in healthy controls and depressed subjects before and after treatment. RMSSD, Root mean square of successive differences; HF, high frequency.
Heart rate variability values for healthy controls and depressed subjects (according to group and treatment response)
. | . | RMSSD . | HF . | ||
---|---|---|---|---|---|
. | n . | Mean . | s.d. . | Mean . | s.d. . |
Healthy controls | 120 | 3.73 | 0.81 | 6.47 | 1.5 |
MDD patients | |||||
Before treatment | 118 | 3.44 | 0.8 | 5.76 | 1.6 |
After treatment | 93 | 3.22 | 0.78 | 5.44 | 1.54 |
MDD – responders (endpoint) | |||||
Total | 40 | 3.2 | 0.81 | 5.48 | 1.56 |
Placebo | 4 | 3.4 | 1.21 | 5.42 | 2.72 |
tDCS | 10 | 3.16 | 0.54 | 5.42 | 0.95 |
Sertraline | 7 | 3.3 | 0.66 | 5.79 | 1.42 |
Combined treatment | 19 | 3.15 | 0.92 | 5.42 | 1.69 |
MDD – non-responders (endpoint) | |||||
Total | 53 | 3.24 | 0.75 | 5.4 | 1.54 |
Placebo | 18 | 3.21 | 0.73 | 5.21 | 1.6 |
tDCS | 13 | 3.17 | 0.97 | 5.37 | 1.83 |
Sertraline | 14 | 3.01 | 0.33 | 4.96 | 0.84 |
Combined treatment | 8 | 3.81 | 0.77 | 6.6 | 1.4 |
. | . | RMSSD . | HF . | ||
---|---|---|---|---|---|
. | n . | Mean . | s.d. . | Mean . | s.d. . |
Healthy controls | 120 | 3.73 | 0.81 | 6.47 | 1.5 |
MDD patients | |||||
Before treatment | 118 | 3.44 | 0.8 | 5.76 | 1.6 |
After treatment | 93 | 3.22 | 0.78 | 5.44 | 1.54 |
MDD – responders (endpoint) | |||||
Total | 40 | 3.2 | 0.81 | 5.48 | 1.56 |
Placebo | 4 | 3.4 | 1.21 | 5.42 | 2.72 |
tDCS | 10 | 3.16 | 0.54 | 5.42 | 0.95 |
Sertraline | 7 | 3.3 | 0.66 | 5.79 | 1.42 |
Combined treatment | 19 | 3.15 | 0.92 | 5.42 | 1.69 |
MDD – non-responders (endpoint) | |||||
Total | 53 | 3.24 | 0.75 | 5.4 | 1.54 |
Placebo | 18 | 3.21 | 0.73 | 5.21 | 1.6 |
tDCS | 13 | 3.17 | 0.97 | 5.37 | 1.83 |
Sertraline | 14 | 3.01 | 0.33 | 4.96 | 0.84 |
Combined treatment | 8 | 3.81 | 0.77 | 6.6 | 1.4 |
tDCS, Transcranial direct current stimulation; RMSSD(ms), square root of the mean squared difference of successive normal to normal intervals; HF (ms2), high-frequency.
RMSSD and HF are transformed in the natural logarithmic base (ln).
Heart rate variability values for healthy controls and depressed subjects (according to group and treatment response)
. | . | RMSSD . | HF . | ||
---|---|---|---|---|---|
. | n . | Mean . | s.d. . | Mean . | s.d. . |
Healthy controls | 120 | 3.73 | 0.81 | 6.47 | 1.5 |
MDD patients | |||||
Before treatment | 118 | 3.44 | 0.8 | 5.76 | 1.6 |
After treatment | 93 | 3.22 | 0.78 | 5.44 | 1.54 |
MDD – responders (endpoint) | |||||
Total | 40 | 3.2 | 0.81 | 5.48 | 1.56 |
Placebo | 4 | 3.4 | 1.21 | 5.42 | 2.72 |
tDCS | 10 | 3.16 | 0.54 | 5.42 | 0.95 |
Sertraline | 7 | 3.3 | 0.66 | 5.79 | 1.42 |
Combined treatment | 19 | 3.15 | 0.92 | 5.42 | 1.69 |
MDD – non-responders (endpoint) | |||||
Total | 53 | 3.24 | 0.75 | 5.4 | 1.54 |
Placebo | 18 | 3.21 | 0.73 | 5.21 | 1.6 |
tDCS | 13 | 3.17 | 0.97 | 5.37 | 1.83 |
Sertraline | 14 | 3.01 | 0.33 | 4.96 | 0.84 |
Combined treatment | 8 | 3.81 | 0.77 | 6.6 | 1.4 |
. | . | RMSSD . | HF . | ||
---|---|---|---|---|---|
. | n . | Mean . | s.d. . | Mean . | s.d. . |
Healthy controls | 120 | 3.73 | 0.81 | 6.47 | 1.5 |
MDD patients | |||||
Before treatment | 118 | 3.44 | 0.8 | 5.76 | 1.6 |
After treatment | 93 | 3.22 | 0.78 | 5.44 | 1.54 |
MDD – responders (endpoint) | |||||
Total | 40 | 3.2 | 0.81 | 5.48 | 1.56 |
Placebo | 4 | 3.4 | 1.21 | 5.42 | 2.72 |
tDCS | 10 | 3.16 | 0.54 | 5.42 | 0.95 |
Sertraline | 7 | 3.3 | 0.66 | 5.79 | 1.42 |
Combined treatment | 19 | 3.15 | 0.92 | 5.42 | 1.69 |
MDD – non-responders (endpoint) | |||||
Total | 53 | 3.24 | 0.75 | 5.4 | 1.54 |
Placebo | 18 | 3.21 | 0.73 | 5.21 | 1.6 |
tDCS | 13 | 3.17 | 0.97 | 5.37 | 1.83 |
Sertraline | 14 | 3.01 | 0.33 | 4.96 | 0.84 |
Combined treatment | 8 | 3.81 | 0.77 | 6.6 | 1.4 |
tDCS, Transcranial direct current stimulation; RMSSD(ms), square root of the mean squared difference of successive normal to normal intervals; HF (ms2), high-frequency.
RMSSD and HF are transformed in the natural logarithmic base (ln).
Impact of treatment on RMSSD and HF
For the HF ANOVA, we found no main effects of time (F195,1 = 2.16, p = 0.14). Also, we found no significant interactions between time and group (F195,3 = 1.49, p = 0.22), time and response (F195,1 = 1.96, p = 0.16) and time, group and response (F195,3 = 0.13, p = 0.94).
For the RMSSD ANOVA, we also found no main effects of time (F195,1 = 3.38, p = 0.07). Also, we found no significant interactions between time and group (F195,3 = 1.35, p = 0.26), time and response (F195,1 = 2.58, p = 0.11) and time, group and response (F195,3 = 0.29, p = 0.83).
Therefore, we observed no global effects of treatment over HRV, and also not when considering specific effects according to clinical response, group of treatment and the interaction between response and group of treatment. (Table 3) (Fig. 2)

Heart rate variability changes according to group and clinical response. RMSSD, Root mean square of successive differences; HF, high frequency; tDCS, transcranial direct current stimulation.
Pearson's correlations revealed no significant association between changes in depression scores with changes in RMSSD and HF (r = 0.15, p = 0.19 and r = −0.09, p = 0.38). We also found no association either in the group of responders (r = 0.25, p = 0.12 and r = −0.2, p = 0.21 for RMSSD and HF, respectively) or non-responders (r = −0.12, p = 0.38; r = 0.07, p = 0.58); and also no association was found for sham tDCS–placebo (r = 0.12, p = 0.58; r = −0.14, p = 0.51), sham tDCS–sertraline (r = 0.32, p = 0.14; r = −0.23, p = 0.3), active tDCS–placebo (r = 0.23, p = 0.27; r = −0.22, p = 0.30) and active tDCS–sertraline (r = 0.28, p = 0.14; r=−0.2, p = 0.30). Finally, no association was found considering responders and non-responders within each group (p = ns for all correlations).
Thus, both the results from the ANOVA and Pearson's correlations are in agreement, revealing that HRV changes are not associated with either treatment response or type of intervention.
Sensitivity analysis for benzodiazepines
To assess the influence of benzodiazepines on our findings, we performed the same analyses after 23 patients on benzodiazepines were excluded. As observed, the comparison with controls revealed that MDD, benzodiazepine-free patients (as compared to benzodiazepine-free controls) presented decreased HF (t95 = 4.68, p < 0.001, Cohen's d = 0.59) and decreased RMSSD (t95 = 3.78, p < 0.001, Cohen's d = 0.49).
Results were also replicated when evaluating the impact of treatment – i.e. for RMSSD we found no significant effect of time (F157,1 = 3.5, p = 0.07), time × group (F157,3 = 0.4, p = 0.75), time × responders (F157,1 = 2.54, p = 0.11) and time × group × responders (F157,3 = 0.4, p = 0.75), as well as for HF (F157,1 = 2.36, p = 0.12; F157,3 = 0.4, p = 0.7; F157,1 = 2.11, p = 0.14 and F157,3 = 0.83, p = 0.28 for the effects of time, time × group, time × responders and time × group × responders, respectively).
Influence of outliers
We did not identify outliers using the Tukey's boxplot method. With the standard deviation method, we found two values slightly above (<1 and 2%) the upper limit for HF, and one value slightly above (1%) of the upper margin for RMSSD. The exclusion of these outliers did not statistically influence the results.
Other HRV indices
The other HRV measures (approximate entropy, Poincaré plots 1 and 2, symbolic measures 0 and 2 V) confirmed the findings obtained for HF and RMSSD, i.e. these measures did not change over time, in spite of group and treatment response (Supplementary material).
Discussion
Main findings
Our main findings confirmed that depression is associated with lower HRV (with a small-to-moderate effect size vs. controls and moderate effect size when examining only the benzodiazepine-free participants) and further demonstrated that HRV does not change following treatment, regardless of clinical response or type of intervention. We now discuss these findings and propose that previously reported reductions in HRV are a result of pathophysiological mechanisms of major depression per se – rather than the effects of the SSRI antidepressant class of medication – and that low HRV is maintained following improvement in depression symptoms.
Remarkably, HRV did not change significantly after chronic tDCS treatment (i.e. 12 sessions applied over 6 wk) in depressed subjects. Only a few studies have investigated tDCS effects on HRV, and all of them have explored the immediate effects of a single-session of tDCS on HRV in healthy subjects. Whereas Vandermeeren et al. (2010) found no specific effects on HRV after tDCS; Montenegro et al. (2011) observed an increase in HF HRV in a subgroup of highly fit athletes. Recently, our group (Brunoni et al., 2013a) observed HF HRV to increase during anodal tDCS while participants viewed negative-valence images, indicating that tDCS may counterbalance the physiological stress associated with viewing of negative images. In this regard we recently proposed – in a systematic review (Sampaio et al., 2012) – that acute tDCS effects on cardiovascular activity occur during stress-induced paradigms.
In the present study, on the other hand, we assessed chronic tDCS antidepressant effects on HRV in patients with MDD. Therefore, and also considering that we collected endpoint HRV 4 wk after the 10-d stimulation period (and 2 wk after the 4th-wk stimulation) we were not particularly expecting acute modulatory effects of tDCS on HRV (via, for instance, prefrontal cortex activation), but rather, that tDCS could have influenced HRV through depression treatment. In this sense, our results should be compared with the study of Udupa et al. (2007) which investigated another non-pharmacological therapy – repetitive transcranial magnetic stimulation (rTMS) – that is also used on consecutive weekdays for MDD treatment and acts through direct modulation of DLPFC activity. These authors compared HRV changes after escitalopram vs. rTMS and, although they reported HRV levels to increase after rTMS and decrease following escitalopram, in fact only the standard deviation of normal to normal endpoint scores of rTMS were higher than escitalopram (i.e. for all other HRV measures, patients groupings did not differ). Furthermore, their study presented some methodological limitations, including its open-label design and non-standardized HRV endpoint assessments (2 wk for rTMS and 4 wk for escitalopram). Conversely, our factorial design allowed direct HRV comparisons between treatments at the same endpoint.
Like tDCS, the SSRI sertraline did not impact on HRV either. This finding makes an important contribution to the literature given the current debate (Kemp, 2011; Kemp et al., 2010; Brunoni et al., 2012a). Our meta-analysis (Kemp et al., 2010) of 11 studies found that HRV did not decrease after short-term antidepressant treatment, with the exception of tricyclic antidepressants. Conversely, the longitudinal study of Licht et al. (2008) found that all antidepressant classes were associated with lower HRV. They also found that antidepressant discontinuation led to the opposite effect, i.e. HRV increased to values similar to those not on antidepressants. Studies with cardiovascular samples have also shown mixed results: for instance, the trials of Glassman et al. (2007) and McFarlane et al. (2001) reported HRV to decrease and increase, respectively, after sertraline treatment in depressed patients after acute coronary syndrome. Considering the dispute over whether depression or antidepressants adversely impact on HRV (Brunoni et al., 2012a), our study indicates that MDD pathophysiology, rather than the pharmacological properties of the SSRIs, are responsible for lower HRV values relative to healthy controls. Even though one cannot ignore the direct effects of antidepressants on the heartbeat – particularly for tricyclic antidepressants – we found that HRV levels were no different to those treated with tDCS, a non-pharmacological intervention. In fact, we found that the SSRI treatment (sertraline) does not decrease HRV in otherwise healthy depressed patients, indicating that this class of medications does not have additional hazardous effects on the cardiovascular profile of the MDD patient, at least in the short-term.
We also observed no overall differences between responders vs. non-responders to either pharmacological or non-pharmacological antidepressant treatment, consistent with our meta-analysis (Kemp et al., 2010) that reported no short-term changes in HRV on MDD patients on pharmacotherapy with SSRIs. In fact, our findings demonstrate that, despite resolution of depressive symptoms, there is a general lack of improvement in HRV, irrespective of type of treatment, suggesting that HRV may be a trait marker for depression. ‘State’ and ‘trait’ are definitions used in psychopathology to differentiate certain biological and psychological features that are only or mainly present during the acute depressive episode (‘state’) from others that can be observed even after remission or prior to the first depressive episode (‘trait’), indicating increased vulnerability to the disorder. In fact, sustained low HRV in depression favors a ‘common soil’ theory relating to the relationship between depression and cardiovascular diseases, since patients with depression have an increased risk for myocardial infarction and other heart disorders (Szczepanska-Sadowska et al., 2010; Scherrer et al., 2011) and vice versa (Glassman, 2007). In fact, the risk of having a myocardial infarction is increased even when the depressive episode occurred decades before (Ford et al., 1998). This hypothesis is further corroborated by our observation that depressed patients display decreased HRV values relative to matched controls, considering that low HRV is predictive of future adverse cardiovascular events even in individuals without a prior history of cardiovascular disease (Sajadieh et al., 2004; Thayer et al., 2010).
Finally, it is important to emphasize that we focused on two of the main HRV measures – RMSSD and HF; RMSSD is a time-domain index commonly used for assessing HRV and the most appropriate measure for short-ECG record intervals (Rottenberg, 2007; Lotufo et al., 2012) while HF is a frequency-domain variable related to parasympathetic activity – in contrast with other frequency-domain variables (e.g. LF and LF/HF) on which there is more uncertainty about what exactly is being measured (Heathers, 2012). In fact, there are about 30 HRV indexes described in the literature, highlighting a need to opt for those measures more extensively studied in the literature (Kemp et al., 2010), thus increasing the generalizability of our findings. We also performed exploratory analyses in other five potential HRV measures of interest (supplementary material); obtaining results similar to those we found for HF and RMSSD measures (no changes in HRV measure after time, group and response). Therefore, it is unlikely that the use of other HRV measures would have led to different results.
Limitations
We performed the second HRV measurement 6 wk after the first one. Although there is no consensus regarding the optimal time period for assessing HRV changes in MDD, HRV recovery may be delayed after depression improvement, as this may involve long-term, neuroplastic changes in the central autonomic network (Thayer and Siegle, 2002). For instance, changes in HRV after antidepressant onset/withdrawal appear to be more evident in an extended follow-up (Licht et al., 2010) and in the elderly (Kop et al., 2010; Vasudev et al., 2011).
Further, we assessed the effects of only one pharmacological (the SSRI sertraline) and one non-pharmacological (tDCS) intervention. Therefore, other treatments might have a different impact on HRV, especially the SNRIs and tricyclic antidepressants.
Finally, although we identified two and one outliers for HF and RMSSD, respectively, the removal of these observations did not influence the results.
Theoretical and clinical implications
These findings suggest the hypothesis that low HRV may reflect a trait marker of depression that could be linked to the depressive phenotype, including social impairment and lack of facial expressiveness, that may persist with resolution of depressive symptoms (Thayer and Siegle, 2002; Thayer et al., 2010). In this regard, we have recently demonstrated a relationship between HRV and emotion recognition (Quintana et al., 2012), observing a positive association with performance on the Reading the Mind in the Eyes Test, suggesting that HRV may provide a novel marker of ability to recognize emotions in humans. Major depression is associated with a dysfunctional top-down regulation, in which cortical areas such as the prefrontal cortex are hypoactive and subcortical areas such as the amygdala are overactive (Pizzagalli, 2011; Kupfer et al., 2012; Nemeroff and Goldschmidt-Clermont, 2012). This can be considered the biological counterpart of Beck's diathesis-stress model (Beck, 2008) that proposes there is a negative cognitive style (characterized by pessimism, lack of hope and despair) facilitating depression development. Moreover, according to the polyvagal theory, proposed by Porges (2009), subcortical structures coordinate three distinct, phylogenetically ordered, autonomic systems, namely the myelinated vagus (the parasympathetic system associated with social engagement), the sympathetic system and the unmyelinated vagus (parasympathetic system associated with vegetative responses). These systems are involved with depression phenotypes and endophenotypes: the myelinated vagus, regulating the facial expression of emotion and social interaction, is impaired in depression (Porges, 2009). In fact, impaired response to emotional faces seems to be a depression trait marker (Kerestes et al., 2012). Likewise, hypersympathetic activity triggers, for instance, the hypothalamic–pituitary–adrenal (HPA) axis. Recently, it was shown that hypercortisolemia can also be considered a depression trait marker (Lok et al., 2012), particularly for patients with melancholia. Finally, we suggest that the reduced parasympathetic activity, reflected by decreased HF and RMSSD values, is another trait marker for depression, which may be underpinned by dysfunction in the subcortical structures regulating the polyvagal system.
From a clinical perspective, our findings do not suggest that the issue of reduced vagal activity in MDD should be overlooked by psychiatrists and physicians, since there are many other pathophysiological pathways (e.g. hypercortisolism, increased activity of inflammatory cytokines and so forth) linking depression to stress and cardiovascular disorders. On the contrary, the finding that HRV might be a trait marker for depression emphasizes the need for treating not only the depressive symptoms but also modifiable cardiovascular risk factors (smoking, obesity, physical activity) and environment (work stress) (Thayer et al., 2010) associated with MDD. Therefore, our findings highlight the important relationship between depression and decreased HRV, regardless of depression improvement and type (pharmacological vs. non-pharmacological) of antidepressant therapy, which is clinically relevant in primary care, since depression and cardiovascular disorders are highly prevalent and related conditions. Therefore, patients with current or previous depressive episodes should be screened and treated for other potential cardiovascular risk factors, as recommended, for instance, by the American Heart Association, American Psychiatry Association and other groups (Lichtman et al., 2008).
Moreover, our study is the first to assess long-term changes in HRV associated with repeated, daily tDCS in patients with MDD. Although HRV did not increase after tDCS, it did not decrease either, suggesting that tDCS is a safe technique from a cardiovascular perspective in comparison to, for instance, the tricyclic antidepressants (Jakobsen et al., 1984). Non-invasive brain stimulation therapies should also be considered as an alternative non-pharmacological therapy for MDD, considering other cardiovascular risk factors. Since these techniques act primarily on the central and peripheral nervous systems, it is unlikely that they lead to hazardous metabolic effects (e.g. weight gain, glucose intolerance, dyslipidemia), as is observed with some antidepressants (including the SSRIs) and antipsychotics. Non-invasive neuromodulation might also decrease the activity of the HPA axis (Baeken et al., 2011; Brunoni et al., 2013a), an important link between MDD and cardiovascular illness (Lett et al., 2004; Maes et al., 2011). Thus, the global cardiovascular impact of tDCS and rTMS warrants further investigation.
Conclusion
We found that HRV values were lower in depressed subjects compared to healthy, matched controls and that HRV does not improve with successful treatment by pharmacological (sertraline) or a non-pharmacological (tDCS) means. Taken together, these findings suggest that the pathophysiological features of MDD, rather than pharmacotherapy – at least for SSRI medication – drive reported HRV reductions in patients. We hypothesize that chronic vagal underactivity can be related to stable psychological features, and, therefore, that decreased HRV is trait marker for depression. Further, we suggest that such a trait links depression with cardiovascular disorders, as both sustained, low HRV and depression are associated with increased cardiovascular risk. Finally, we conclude that daily, repeated tDCS is a safe technique, which is important in clinical settings, as tDCS is a promising therapy for depression treatment.
Supplementary material
Supplementary material accompanies this paper on the Journal's website.
Supplementary information supplied by authors.
Acknowledgments
This study was partially funded by a grant from FAPESP (São Paulo Research Foundation, Grant number: 2009/05728-7). A.H.K. is supported by a National Health and Medical Research Council (NHMRC) Career Development Fellowship (571101). The sponsors played no role in the design and conduct of the study, collection, management, analysis and interpretation of the data, and preparation, review or approval of the manuscript. Trial registration: NCT01033084, http://clinicaltrials.gov/ct2/show/NCT01033084.
Statement of Interest
None.
References