The European Values Study 2017: On the Way to the Future Using Mixed-Modes

The European Values Study (EVS) was ﬁrst conducted in 1981 and then repeated in 1990, 1999, 2008, and 2017, with the aim of providing researchers with data to investigate whether European individual and social values are changing and to what degree. The EVS is traditionally carried out as a probability-based face-to-face survey that takes around 1 hour to complete. In recent years, large-scale population surveys such as the EVS have been challenged by decreasing response rates and increasing survey costs. In the light of these challenges, six countries that participated in the last wave of the EVS tested the application of self-administered mixed-modes


Introduction
The European Values Study (EVS) was first conducted in 1981 by the European Values Systems Study Group (EVSSG). The EVSSG was a group of academics who, at the time of the first European parliamentary elections, sought to analyse the moral and social values underlying the social and political institutions, asking questions such as do Europeans share common values; are values changing and, if so, in what direction. The first survey conducted in 1981 collected data from 16 countries. Since then, EVS was carried out every 9 years (1990,1999,2008,2017), with the highest number of countries/regions (47) in 2008. In the most recent survey, at the time of writing this data brief, 34 countries across Europe participated. The EVS is primarily funded by its participating members' institutions, such as universities, research institutes, national science foundations, and private sponsors.
The EVS was founded to respond to questions emerging in political and social debates at the end of the 1970s, such as whether Europe was one cultural entity or converging to one, whether Christian values were central in the lives of Europeans, whether new belief systems were emerging that might ultimately replace Christian belief, and what the implications of all these changes were for the unity of Europe (see www.euro peanvaluesstudy.eu for more details). To adequately address these research questions, the EVS incorporated questions measuring values and attitudes in several domains of life: family, work, religion and morale, politics and society, environment, as well as national identity, tolerance, and social solidarity. Over the waves, the substantive scope of EVS expanded, but in the last wave, the questionnaire has been shortened not to overload interview time. Since the early 1980s, EVS data sets have been used in at least 2,100 publications in the social sciences and related fields (EVS, 2020a).
With its emphasis on surveying values and attitudes on several domains of life, the EVS inspired the development of the World Values Survey (WVS, see www. worldvaluessurvey.org) also committed to research questions on values change, e.g. the shift from material to post-materialist values (Inglehart, 1997) and more recently the idea of a 'Cultural Backlash' (Norris and Inglehart, 2019). Also in its dissemination, there is a close relation with WVS, which is in existence since the early 1990s. Since the third wave of EVS, EVS and WVS released longitudinal EVS and WVS files with common dictionaries that facilitated easy merging of the two data sets and so makes it possible to undertake global analyses.
The EVS is traditionally carried out as a probabilitybased interviewer-administered face-to-face (F2F) survey that takes around 1 hour to complete. In recent years, large-scale population surveys such as the EVS have been challenged by decreasing response rates and increasing survey costs (de Leeuw and De Heer, 2002;Groves and Peytcheva, 2008;Brick and Williams, 2013;de Leeuw, Hox and Luiten, 2018). Non-response is most likely not random and might eventually correlate with opinions that are at the core of the EVS, such as trust (Billiet et al., 2007), making that traditional modes of data collection potentially jeopardize the response to substantial research questions. In the light of these developments, the EVS Methodology Group, which oversees data collection standards and quality, allowed the use of self-administered modes in EVS 2017 under the condition that they were based on the same kind of probability sample, according to the same standards as for the F2F survey. Six countries decided to (experimentally) test the implementation of self-administered mixedmode designs as part of EVS 2017 with respect to their country-specific demands and contexts: Denmark, Finland, Germany, Iceland, the Netherlands, and Switzerland.
With our data brief, we want to familiarize readers with the latest EVS data release, the recent methodological innovations, and promote the use of EVS for social science research. In our view, this brief should prove helpful to researchers who aim to use the data for substantive research as well as for methodologists who are interested in getting insights into self-administered mixed-modes in a large-scale cross-cultural survey programme.
In the following, we describe the methodological challenges that motivated the EVS Methodology Group to allow for the use of self-administered modes. We will look at the implications of this decision for the (countryspecific) survey designs and illustrate how each country has (experimentally) implemented the EVS. We then describe how it performed in each instance. We continue by providing information on the data release, structure of data files, documentation, and data access to facilitate the use of the most recent EVS wave that includes several experiments and different designs. We close with discussing limitations and opportunities for research with the EVS data.

Methodological Challenges
In many countries, survey researchers are facing declining response rates (de Leeuw and De Heer, 2002;Brick and Williams, 2013). Neither is this trend new (e.g., Steeh, 1981) nor is it limited to individual countries as declining response rates have been reported in various countries (e.g., Stoop et al., 2010). Albeit low response rates are not necessarily linked to non-response bias (Groves and Peytcheva, 2008) non-participation may lead to non-response bias. As a consequence, research has focused on meeting the non-response challenge in recent decades by advancing survey methodology in this regard (Kreuter, 2013) with incentives likely being the most prominent and well-researched tool in the field (for an overview see, e.g., Pforr et al., 2015) and the adaptive and responsive design paradigms (Groves and Heeringa, 2006;Wagner, 2008;Tourangeau et al., 2017), which advocate the use of more flexible data collection protocols, being the newest approach for tackling non-response.
The downward trend in response rates appears to be continuing and is leading researchers either to invest more effort in fieldwork or to work with larger gross samples. If outcome rates are not stabilized, the resulting sample size may be too small and the statistical power of the data may be too low, or larger gross samples must be used. As a consequence, increasing survey costs have been reported (e.g., Massey and Tourangeau, 2013).
While this development is a formidable challenge for most surveys, it becomes an even more pressing issue for large cross-national surveys such as the EVS, European Social Survey (ESS), International Social Survey Programme (ISSP), WVS, and others. These surveys' added value depends on their ability to cover a large and diverse number of countries over time. Thus, enabling cross-national comparison (over time) requires a tradeoff between the need to ensure high data quality and avoiding that countries are unable to participate because they cannot cover the costs of a survey complying with the methodological requirements of the study (e.g., using a probability-based sample, achieving a given target net sample size, or reaching specific outcome rates). Consequently, survey researchers-and especially those in charge of cross-national studiesmust cope with the rising costs in F2F surveying to maintain the availability of high-quality survey data for social science research.

Design of the Mixed-Mode Experiments
For the purpose of testing methodological innovations, especially with regard to increasing response rates and lowering survey costs, in EVS 2017 countries had the possibility to implement self-administered surveys in parallel to the traditional F2F survey. Specific guidelines were developed by the EVS Central Team in cooperation with those country teams that expressed interest in employing self-administered modes.
The suggested main alternative to fielding a F2F survey was a self-administered web-based survey (Computer Assisted Web Interview, or CAWI) of a probability-based sample. Countries were encouraged to find suitable strategies (e.g., for contact modes or incentives) to make sure that the response rate of the survey reached at least 40 per cent. For this reason, some of the countries also decided to offer a postal self-administered paper-and-pencil survey, which allowed them to reach 'offline' segments of the population and accommodate respondents who did not want to answer a CAWI survey.
Other general guidelines referred to the preferred layout of the web survey and were designed with the purpose of maintaining comparability with the F2F mode. For instance, the item-by-item presentation of questions was recommended for the CAWI mode; progress bars were not allowed; the design had to be adaptive to allow for completion on mobile devices; scrolling had to be avoided. For a complete overview of the selfadministered mixed-mode guidelines, please refer to the EVS 2017 Methodological Guidelines (EVS, 2020e).

Matrix Questionnaire
The EVS 2017 was based on a 1-hour questionnaire designed for F2F mode. Compared to F2F surveys, where interviewers can motivate and guide respondents, self-administered surveys are assumed to require a larger effort from the respondent (Klausch, Hox and Schouten, 2013). For this reason, it is common practice to reduce the length of surveys when moving from an intervieweradministered to a self-administered mode (cf. Olson et al., 2020), and the recommended threshold is usually around 20 minutes for a web-based survey (e.g., Revilla and Ochoa, 2017). The EVS not only encouraged testing the use of a 1-hour questionnaire, but also proposed the use of matrix questionnaires to reduce the overall length of the questionnaire.
A matrix questionnaire design (or split questionnaire design, Raghunathan and Grizzle, 1995) consists of splitting the questionnaire into different, shorter versions, each version including a selection of questions from the full questionnaire. Respondents are then randomly assigned to the different versions of the questionnaire, ensuring that-over the entire sample-all questions have been answered by a sufficient large share of respondents.
In the EVS, the survey was divided into four thematic blocks (A, B, C, and D) and one 'core' block with background variables. Each respondent had to answer two of the thematic blocks and the core. Each block contained no more than 60 ticks. Table 1 provides a summary of thematic blocks created in this process.
Splitting the F2F questionnaire resulted in six versions of the self-completion questionnaire, representing all possible combinations of blocks. Table 2 represents an example of allocating respondents to matrix groups for a sample size of 1,800 respondents. With this design, each block is answered by 900 respondents and at least 300 answers can be used for analyses between blocks. In the final data set, answers are missing by design, because each respondent is answering only parts of the questionnaire depending on the respective matrix group.
A set of decision rules was applied to split the questionnaire into blocks. These decision rules aimed at providing respondents with a questionnaire that made sense in its thematic order and resulted in a positive survey experience. In addition, the selection of questions for the core and the thematic blocks was intended to enable the later use of imputation methods to handle the substantial share of values that were missing by design (see Table 2). Accordingly, the core included key items from each thematic block. The following decision rules were applied: 1. Each questionnaire had to be meaningful to the respondents in its question order. 2. The question order from the F2F questionnaire had to be retained. 3. If variables were known to be often analysed together, they were placed in the same block. 4. The block of background variables (core) included not only the major socio-demographic information but also some widely used substantial variables. Less used socio-demographic variables were placed in the thematic blocks.
The assignment of EVS items to blocks is reported in the ZA7500 Matrix Design Blocks (EVS, 2020b).

Follow-up Survey
The application of a matrix design resulted in many missing values by design. A potential solution to this issue that required testing-apart from fielding the whole 1-hour questionnaire-was to conduct a follow-up survey in which each respondent was asked the remaining blocks. Table 3 illustrates such a follow-up design, where circles symbolize blocks answered in a second survey that were missing by design in the initial survey.
The EVS Methodology Group proposed the use of a follow-up survey in countries that employed the matrix   Notes: 1,800 respondents overall, 900 respondents for each substantive question; at least 300 respondents for each binary combination of questions.
questionnaire. However, it needs to be noted that respondents were free to refuse to participate in that survey or to not participate due to other reasons (e.g., noncontact). Using a follow-up survey, thus, introduced a further form of missingness in the data: panel attrition.

Country-Specific Designs
Six countries (Denmark, Finland, Germany, Iceland, the Netherlands, and Switzerland) chose to field the EVS 2017 in self-administered mode(s) in parallel to the F2F mode. Due to the various ways of fielding, the selfadministered survey outlined above and lacking research in this regard, the countries explored different implementations. Four of them (Germany, Iceland, the Netherlands, and Switzerland) administered matrix questionnaires. Three of these countries (Iceland, the Netherlands, and Switzerland) further implemented a follow-up survey to complement the matrix design. Three countries (Germany, Iceland, and Switzerland) fielded full-length questionnaires in parallel to a matrix questionnaire. Two countries (Denmark and Finland) decided to only field the full-length questionnaire.
While the design of the matrix was coordinated, some decisions on survey characteristics such as contact procedures, mode choice sequence, and incentives were left to the countries, based on their resources and previous field experience to reflect the country-specific context and survey climate. Table 4 summarizes the design per country.

Denmark
In Denmark, the sampling and fieldwork of EVS were carried out by Statistics Denmark. A stratified simple random sample was drawn from the national population register, and sampled individuals were then randomly assigned to be invited to participate either in the F2F or self-administered survey.
While the fieldwork for the F2F mode took place between 27 September 2017 and 31 January 2018, the fieldwork of the self-administered survey started later (11 December 2017) and ended alongside the F2F one. No matrix design was implemented. Thus, respondents only received the full-length EVS questionnaire. Respondents assigned to the self-administered mode received an advance letter and leaflet with the link to the web survey. The first postal reminder included also the paper-based questionnaire. The second reminder, as a final contact attempt, was made by telephone. A monetary incentive conditional upon the completion of the survey was offered to respondents. Among the 4,004 respondents selected for the self-administered mode, 1,255 completed the survey via web, and 411 respondents completed the paper-based questionnaire.

Finland
In Finland, the F2F survey was complemented by a self-administered CAWI survey that featured the whole 1-hour questionnaire. The sampling frame in both cases was the Finnish population register. For the F2F survey, a two-staged stratified random sampling was used. The strata were living area and degrees of urbanization. The CAWI survey relied on one-stage stratified random sampling with the same strata. All respondents received an invitation letter to participate in the survey. The F2F participants were informed that they will be contacted by interviewers. The CAWI participants were asked to use the provided web link and fill in the survey. As an incentive, a lottery was organized so that 11 gift cards (10 Â 100e; 1 Â 500e) were drawn among all who completed the interview.
The gross sample for the CAPI was 1,037 and the fieldwork took place between 3 January and 10 July 2018. With a minimum of four contact attempts, 388 interviews were achieved. The gross sample for the CAWI was 4,209 and the fieldwork period was 24 November 2017 to 10 July 2018. There were three rounds of invitation letters and two rounds of phone reminders. An additional option to respond by paperbased questionnaire, sent by mail, was provided for those who requested it. In the end, 811 self-administered interviews were achieved (CAWI N ¼ 668 and paperbased N ¼ 143).

Germany
In addition to the F2F survey, Germany conducted two surveys in self-administered mixed-mode (CAWI and paper-based): the first survey was performed using the matrix design and carried out using a responsive survey design with experiments on mode choice sequence (simultaneous vs. sequential) and incentives (10e conditional vs. 5e unconditional). The second survey was administered using the full-length questionnaire. In this latter case, a simultaneous mode choice sequence was offered to respondents along with a 5e unconditional incentive; this approach turned out to reach the highest response rate in the first survey. Respondents in the F2F survey received a 10e conditional incentive what was later changed to 5e unconditional to increase outcome rates.
In both surveys, respondents in the simultaneous mode choice sequence were offered paper-based and CAWI questionnaires right from the start. Respondents in the sequential mode choice sequence ('push to web') were offered CAWI questionnaires first, and paperbased questionnaires later with the second reminder.
All three surveys used the same probability-based sample that was drawn from German municipalities' population registers and then randomly split up into separate samples for each survey. The gross sample sizes for each survey were 5,833 (CAPI), 9,369 (mixed-mode matrix), and 2,106 (mixed-mode full length) cases.

Iceland
In Iceland, a stratified random sample of 2,322 individuals was drawn from the national population register for the F2F survey and another simple random sample of 3,500 individuals for the CAWI survey. The sample for the CAWI survey was randomly split into seven groups of 500 individuals each. One group was asked to finish the fulllength questionnaire, the remaining six groups received different versions of the matrix design questionnaire. Those who completed a matrix design questionnaire were then invited to participate in a follow-up survey.
Advance letters were sent to all 5,822 individuals in the samples with a web link and invitation to the CAWI sample to answer online. Following a given time allowing participants to respond, contact attempts via phone were made to those who had not at that time answered the questionnaire. After four unsuccessful contact attempts made to non-respondents, a final reminder was sent to their address along with a full-length paper-based questionnaire.
In Iceland, there was no unconditional incentive, however, 11 participants were randomly drawn from those that completed the questionnaire: 10 receiving a lottery prize of 10,000 ISK ($63e) each and one receiving 100,000 ISK ($635e).
F2F interviews were administered between 19 June 2017 and 28 March 2018 (N ¼ 915), whereas the selfadministered interviews were collected between 27 September 2017 and 4 April 2018 (N ¼ 1,379 matrix design; N ¼ 217 full length). The follow-up survey was completed by 472 respondents.

The Netherlands
In the Netherlands, the F2F survey was complemented by a self-administered CAWI survey that featured matrix design questionnaires. Those respondents who completed the matrix questionnaire were invited to a follow-up survey.
The Central Bureau of Statistics (CBS) provided a stratified random sample of 1,500 respondents, extracted from the national population register, for the F2F survey. For the CAWI survey, the country team relied on the Longitudinal Internet Studies for the Social Sciences (LISS) online panel, managed by CentERdata. A sample of 2,515 respondents for the CAWI survey was drawn from the members of the LISS panel (which consisted of 5,000 households and approximately 7,000 individuals as of July 2017). The LISS panel relies on probability samples of households which were drawn, similarly to the F2F EVS sample, from the national population register and provided by CBS. For households which are not online at the time of recruitment, an Internet connection and a computer are provided. For more information on sampling and recruitment strategies, see: www.lissdata.nl.
The F2F interviews were conducted between 31 August 2017 and 28 February 2018. The incentive strategy changed during the fieldwork: initially, all persons within the sample received an unconditional 5e gift card together with the advance letter, and a 10e gift card was sent to them after successful completion of the interview. At about three-quarters of fieldwork, interviewers were granted the possibility to award respondents with a 20e gift card if they deemed it necessary to enhance cooperation. In total, 686 respondents participated in the F2F survey.
Between 11 September and 31 October 2017, 2,053 respondents took part in the CAWI data collection. Contact attempts (invitation and up to two reminders) were only made via email, no advance letters or brochures were used. In accordance with the LISS default implementation, conditional monetary incentives were offered to respondents. More specifically, panel members were rewarded with 15e per hour of completed survey time.
Respondents who participated in the CAWI survey were later invited to participate in the follow-up survey to complete the matrix, which was carried out between 1 January and 30 January 2018. The contact design was the same as in the first round of data collection. Out of the 2,053 respondents who completed the CAWI survey, 1,722 respondents completed also the follow-up survey.

Switzerland
In Switzerland, additionally to the traditional F2F survey, respondents who were selected to participate via self-administered modes (CAWI and paper-based) received a matrix questionnaire or the full-length questionnaire. Those who received a questionnaire in matrix design then received a follow-up survey. Moreover, there were two versions of the full-length questionnaire: one with the original question order, and one with a modified question order.
Separate simple random samples were drawn from the individual-based register frame of the national statistical office for each survey. The F2F survey was based on a gross sample of 1,400 units, and the selfadministered surveys on independent gross samples of overall 6,800 units (4,800 for the matrix and 2,000 for the full-length questionnaire).
The contact procedure was held as similar as possible among the different groups: all sample units received an unconditional 10 CHF ($9e) value incentive in form of a postal check along with the invitation letter. This letter announced the visit of an interviewer or gave the credentials for the web participation. In case of non-response, at least four contact attempts were being made or reminders sent. The paper-based questionnaire was sent to all CAWI non-respondents along with the second reminder. In order to motivate respondents to also answer the follow-up questionnaire, three iPads were drawn among the respondents. The CAWI data collection spanned between 15 September and 28 December 2017, while the paper-based questionnaires were distributed and collected between 30 September 2017 and 22 February 2018 (N ¼ 2,129 matrix design; N ¼ 858 full length; 1,661 cases participated in the follow-up survey). F2F interviews were conducted between 11 September 2017 and 22 February 2018 (N ¼ 673).

Results
In this section, we provide readers with a brief introduction on how the different modes of the EVS performed in the countries that participated in the mixed-mode experiments with respect to outcome rates, representation of socio-demographic variables, and data quality.

Outcome Rates
We report the AAPOR Response Rate 6 (AAPOR, 2016) as an indicator of how well the different surveys performed with respect to achieving the required number of cases (Table 5). The EVS has its own categories for (non)response, the interested reader finds the correspondence between EVS and AAPOR categories in the Appendix, Table A1.
The F2F survey achieved different response rates across countries: While Finland and Germany showed the lowest response rates (below 30 per cent), Denmark, the Netherlands, and Switzerland report response rates of around 50 per cent, and Iceland lies in-between with 40 per cent response rate for F2F. Comparing response rate outcomes between F2F and self-administered modes, we found that some countries achieved a higher response rate for self-administered modes than for F2F (DE, IS, NL), while others reported a lower response rate for self-completion (DK, FI, CH). The full-length mixed-mode survey achieved lower response rates than the matrix mixed-mode in all countries that fielded both designs; however, the differences were very small.
Two differences stand out: In the Netherlands, the matrix mixed-mode achieved a much higher response rate than the F2F survey. This outcome was the result of using panelists from the LISS panel and is not strictly comparable to the other countries. In Finland, the response rate of the F2F survey was already low but participation in the full-length mixed-mode survey was even lower, resulting in the lowest response rate across countries and designs.
All self-completion designs except for the Netherlands included both web and paper-based versions of the questionnaire. The way of offering paperbased questionnaires to respondents differed between countries (see Table 4). Comparing the share of paper in those mixed-mode designs, we found a considerable difference between countries. Depending on when the paper-based questionnaire was offered led to different shares of participations via paper-based questionnaires. The earlier it was offered and the easier it was to get the paper-based questionnaire, the more respondents participated this way (note that based on our study, we cannot rule out country-differences in general preference for the paper mode by country).

Representation of Socio-Demographics
Another aspect of data quality is non-response bias, which can differ between modes. In the following tables, we compare the distributions of basic sociodemographic information of the achieved samples across countries and designs to the same distribution in the population. Pearson's Chi-square tests were performed in order to detect significant differences between the samples and the population for each category of the socio-demographic characteristic (sources for population data are reported in the Appendix, Table A2). With this analysis, we aimed at identifying under-or overrepresented groups.

Age
Overall, there seemed to be an over-representation of older age groups and an under-representation of younger generations (see Table 6). In Denmark, Germany, and the Netherlands, the F2F sample performed better in terms of representation of age than the mixed-mode samples, as there were fewer significant differences from the population distribution. In Iceland, the mixed-mode full-length sample provided the best representation of the population in terms of age categories.

Sex
Overall, samples appeared to be balanced when it comes to the distribution of sex. Only in the Netherlands and  Note: v 2 Test of difference: *P < 0.05, **P < 0.01, ***P < 0.001, see Appendix for the sources of population data.
in Switzerland, the mixed-mode matrix design samples slightly over-represented women (see Table 7).

Higher educational level
We found a representation bias regarding education in all countries and almost all samples in the selfadministered modes. Persons with a high education level (tertiary education) 1 were over-represented while persons with a low and/or persons with a middle level of education are under-represented. Noticeably, this also applied to the F2F sample, albeit less pronounced compared to self-administered modes (see Table 8). The Icelandic mixed-mode full-length sample and the Swiss F2F sample performed well in terms of representation.

Attrition in Follow-up Surveys
Panel surveys are challenged with attrition (Lynn, 2009). Panel attrition is the non-participation of respondents who have participated in previous waves of the survey. It not only lowers the statistical power of the data due to smaller net sample sizes, but it might also introduce a bias if respondents systematically differ from non-respondents (e.g., Kreuter and Olson, 2011;Gummer and Roßmann, 2019). Panel attrition might be a challenge for the follow-up surveys conducted as part of the EVS. Therefore, we estimated logistic regression models with participation in the follow-up survey (0 ¼ no, 1 ¼ yes) as a dependent variable for those cases who had received a matrix questionnaire in self-administered modes in Iceland, the Netherlands, and Switzerland. As independent variables, we selected a set of variables, some of which we have presented in the previous section and which we assumed to relate to the likelihood of participating in the followup (descriptive statistics for these variables are provided in the Appendix, Table A3). The results of each regression are presented in Table 9.
Across all countries, we found older respondents to be more likely to participate in a follow-up survey compared to younger respondents. In both Switzerland and Iceland, we further found that respondents who held the national citizenship were more likely to participate in the follow-up survey. Apart from these effects, some country-specific effects were significant (e.g., education in Iceland, paper-based surveys in Switzerland). Generally, we observe a tendency for selective participation in the second wave of the mixed-mode samples, based on the variables we investigated. While our analyses revealed a few significant effects, the models' Pseudo R 2 hinted towards a rather low explanatory power of the selected variables. In other words, with respect to the variables we included in our models, the degree of systematic attrition should not be over interpreted.
We recommend that researchers using data from the follow-up surveys should evaluate, based on their research questions, if the attrition process relates to their selected dependent and independent variable(s) to assess whether correction methods such as weighting might be applied (cf. Kreuter and Olson, 2011).

Data Quality
F2F surveys are assumed to be producing high-quality data-partly due to a low share of item non-response.
The assumption is that interviewers can motivate respondents to answer and help when a question is ambiguously phrased or hard to answer. However, interviewer presence might also lead to response errors (i.e., interviewer effects) such as socially desirable responding, which in turn might bias the data. In self-administered surveys, no interviewer is present. Especially when answering a survey via a paperbased questionnaire, questions can easily be skipped. Moreover, no interviewer is available to record spontaneous reactions of respondents regarding the questionnaire or to answer questions of the respondent.
Previous research has used item non-response as an indicator of low data quality that is the result of a lack of motivation or ability of the respondent to answer a question (Toepoel, Das and Van Soest, 2009;Lenzner, 2012). Similar arguments have been put forward with regard to survey break-offs where respondents provide only partially completed questionnaires (Roßmann, Blumenstiel and Steinbrecher, 2015).
To investigate whether the use of self-administered modes resulted in larger amounts of missing answers and, thus, might indicate issues with data quality, we calculated average item non-response rates for each design for each country. For the purpose of this analysis, we counted 'don't know' and 'I prefer not to answer' as item non-response. The item non-response rates were calculated by dividing the amount of item non-response by the actual number of items a respondent received, varying between 144 and 271 (depending on the survey design-full length or matrix-the respective matrix questionnaire version, and filtered questions). Table 10 summarizes our findings.
Overall, item non-response was low in most instances. Across all countries, the paper-based mode had a larger share of item non-response compared to the CAWI. In Germany, where the mixed-mode had a high share of participants via paper-based questionnaire, this comparing the different questionnaire designs, we found that the full-length questionnaire generally resulted in a higher share of item non-response compared to the matrix design. The low degree of item non-response was Note: *P < 0.05, **P < 0.01, ***P < 0.001; coeff., logit; SE, standard error. replicated for the follow-up surveys conducted in Switzerland, Iceland, and the Netherlands.

EVS Data Releases
The EVS dataset is made freely available for teaching and research purposes via the data archive at GESIS-Leibniz Institute for the Social Sciences. Datasets are complemented with comprehensive documentation allowing researchers to discover and reuse the data. The documentation covers information on project, study, and variable level. With respect to the selfadministered mixed-modes in several countries, the documentation further includes scenarios and recommendations on how to select the appropriate subsamples for analyses. The de-facto anonymized EVS 2017 survey data are available as two off-site Scientific-Use File: (1) Integrated Dataset (EVS 2017): This dataset (EVS, 2020b) contains only cases who have completed the questionnaire. More specifically, the following cases are included: (i) all F2F interviews from currently 34 EVS 2017 countries; (ii) all the full-length interviews collected via self-administered mixed-mode (Denmark, Finland and full-length questionnaires from Switzerland, Iceland, and Germany); (iii) all respondents who received a matrix questionnaire and completed the follow-up survey (a selection of cases from Iceland, the Netherlands, and Switzerland who received the full questionnaire due to the combination of matrix and follow-up questionnaires). The current version of this dataset contains data from 56,491 respondents and over 450 variables, including calibration weights for each country (based on age, sex, education, and region).
(2) Integrated Dataset (EVS 2017)-Matrix Design Data: This dataset (EVS, 2020c), which can be easily merged to dataset (1), contains all data coming from the self-administered mode in countries that implemented the matrix design, that is 10,598 interviews from Germany, Iceland, the Netherlands, and Switzerland. Respondents who filled in the full-length questionnaire are included too, as well as-for methodological purposes-break-off cases (namely cases that filled in less than 50 per cent of the questions). Cases that appear in both (1) and (2) are marked with a flag variable (fduplicate). Instructions on how to merge the datasets are provided in the Guide to the Mixed-mode Approach and Matrix Design (EVS, 2020f), alongside a description of variables that can be used to navigate the design features (e.g., mixed-mode design, matrix design group, etc. . .) presented in this data brief.
Additionally, sensitive data will be made accessible under contractual regulations. The Secure Data Center (SDC) at GESIS will grant access to the Scientific-Use File via 'contracted off-site' or 'on-site' usage.
A further dataset, the European Values Study Longitudinal Data File 1981-2008 (EVS, 2020d), gathers data from all the EVS waves from 1981 to 2008; an updated version including all data from the (1) EVS 2017 Integrated Dataset will be made available in 2021. Additionally, a joint dataset containing the EVS 2017 data together with the last wave of WVS will be available at the end of 2020.

Discussion
With the present data brief, we introduced researchers to methodological innovations implemented in the latest wave of EVS. Unlike previous waves, the EVS 2017 includes the experimental test of self-administered modes that aimed at investigating possibilities to tackle the ongoing challenges of decreasing response rates and increasing survey costs. Coping with these challenges seems essential for a large-scale cross-cultural survey programme such as the EVS because these survey programmes heavily depend on the participation of a large and diverse set of countries. The increasing fieldwork and data processing costs risk limiting countries' participation in the future. Furthermore, participation in surveys is related with items central to the EVS, including trust, inspiring alternative modes of data collection. In our data brief, we outlined the methodological challenges that motivated an openness for change in the EVS and facilitated testing in several participating countries. In our view, it is pivotal to make the reasoning behind design changes as well as the country-specific implementations transparent and show whether and how they influenced survey outcomes. The recent challenges for F2F data collection imposed by the COVID-19 pandemic (e.g., Gummer et al., 2020;Sakshaug et al., 2020) demonstrate the necessity for openness to methodological changes.
Yet, our contribution is not only targeted on other survey infrastructures but especially on users of EVS data and those researchers interested in learning about new developments. Indeed, the use of mixed-mode data collection in a large, cross-national survey opened up new possibilities for research. For instance, by incorporating the EVS in online panel surveys representative of national populations, the potential arises to supplement existing repeated cross-sectional design into a panel structure. Such shift would not only allow for continuing the study of social change with great detail but also enables to assess the effect of certain events, for instance the aforementioned COVID-19 pandemic, on relevant values and attitudes.
Nevertheless, we urge EVS data users to pay careful attention to the survey design features described in this data brief when analysing EVS data. Potential mode, selection, and order effects should be considered when investigating substantive topics. As described in the survey documentation (EVS, 2020f)) a number of variables provided in the datasets can help acknowledging such effects and/or defining relevant subsamples for analyses. Our findings highlight that employing selfadministered modes was successful in most of the participating countries. Even fielding a 1-hour questionnaire worked reasonably well in those countries that tried itin Germany, Iceland, and Switzerland, similar response rates could be achieved as in a shorter version of the questionnaire. Employing a matrix design yielded a substantive share of missingness by design. It remains to be seen whether this will impair the use of the data for substantive research, and standards to deal with this challenge still need to be established. In Iceland and Germany, the cheaper self-administered modes outperformed F2F mode in terms of outcome rates. Interestingly, in those four countries in which we widely distributed paper-based questionnaires, a non-neglectable proportion of respondents chose to use them. Our analyses on representation of socio-demographic distributions and item non-response further suggested that self-administered modes show slightly higher bias on average but-given the lower cost-still may yield acceptable data quality. However, non-response bias and more generally data quality of selfadministered surveys have to be further studied. For this purpose, we invite methodologists to examine our data on the mode experiments and the matrix questionnaire that were collected as part of the EVS. Overall, we think that self-administered modes can complement the traditional F2F mode in large-scale population-wide surveys; especially if it is possible to reduce bias and further increase data quality, for example, by further improving the adaption of question design to self-administered modes.

Notes
1 The way country-specific educational categories have been translated into ISCED main levels is described in the ZA7500 Codebook Appendix A3 and A4 (cf. EVS, 2020b).