An Evaluation of Mixture Confirmatory Factor Analysis for Detecting Social Desirability Bias Alexandru Cernat and Caroline Vandenplas Journal of Survey Statistics and Methodology, Volume 9, Issue 3, June 2021, Pages 496-522, https://doi.org/10.1093/jssam/smaa032 Abstract Collecting sensitive data using surveys is one of the most challenging tasks facing survey methodologists as people may choose to answer questions untruthfully to present themselves in a positive light. In 2014, Mneimneh et al. proposed mixed Rasch models to detect socially desirable answering behaviors. This approach combines item response theory models with latent class analysis to differentiate substantive and biased answering patterns. Their results identified two latent classes, one of which was consistent with socially desirable answering. Our aim is to expand their approach to detecting social desirability by using a mixture confirmatory factor analysis (CFA) in round 7 of the European Social Survey. First, we attempt to estimate social desirability in three constructs separately (RQ1): effect of immigration on the country, allowing people to come in the country and social connection, using a mixture CFA. We then extend the analysis by (RQ2) introducing constraints between the latent classes, (RQ3) combining different constructs in one model, and (RQ4) comparing results in Belgium and the United Kingdom. In contrast with the paper published by Mneimneh et al. in 2014, the models with two latent classes do not have the best model fit. In addition, validation with the presence of a third person, the respondent's reluctance to give answers and personality traits are not systematically in line with our expectations. A small simulation shows that the method would work if the data would behave as we expect, with social desirability being the main factor influencing answering patterns. We conclude that a mixture CFA might not be able to identify social desirability in different survey contexts, especially in complex data as originating in cross-national social surveys.

An Experimental Evaluation of an Online Interview Scheduler: Effects on Fieldwork Outcomes Katherine McGonagle and Narayan Sastry Journal of Survey Statistics and Methodology, Volume 9, Issue 3, June 2021, Pages 412-428, https://doi.org/10.1093/jssam/smaa031 Abstract In recent years, household surveys have expended significant effort to counter well-documented increases in direct refusals and greater difficulty contacting survey respondents. A substantial amount of fieldwork effort in panel surveys using telephone interviewing is devoted to the task of contacting the respondent to schedule the day and time of the interview. Higher fieldwork effort leads to greater costs and is associated with lower response rates. A new approach was experimentally evaluated in the 2017 wave of the Panel Study of Income Dynamics (PSID) Transition into Adulthood Supplement (TAS) that allowed a randomly selected subset of respondents to choose their own day and time of their telephone interview through the use of an online appointment scheduler. TAS is a nationally representative study of US young adults aged 18-28 years embedded within the worlds' longest running panel study, the PSID. This paper experimentally evaluates the effect of offering the online appointment scheduler on fieldwork outcomes, including number of interviewer contact attempts and interview sessions, number of days to complete the interview, and response rates. We describe panel study members' characteristics associated with uptake of the online scheduler and examine differences in the effectiveness of the treatment across subgroups. Finally, potential cost-savings of fieldwork effort due to the online appointment scheduler are evaluated.

Combining Information from Multiple Data Sources to Assess Population Health Trivellore Raghunathan and others Journal of Survey Statistics and Methodology, Volume 9, Issue 3, June 2021, Pages 598-625, https://doi.org/10.1093/jssam/smz047 Abstract Information about an extensive set of health conditions on a well-defined sample of subjects is essential for assessing population health, gauging the impact of various policies, modeling costs, and studying health disparities. Unfortunately, there is no single data source that provides accurate information about health conditions. We combine information from several administrative and survey data sets to obtain model-based dummy variables for 107 health conditions (diseases, preventive measures, and screening for diseases) for elderly (age 65 and older) subjects in the Medicare Current Beneficiary Survey (MCBS) over the fourteen-year period, 1999-2012. The MCBS has prevalence of diseases assessed based on Medicare claims and provides detailed information on all health conditions but is prone to underestimation bias. The National Health and Nutrition Examination Survey (NHANES), on the other hand, collects self-reports and physical/laboratory measures only for a subset of the 107 health conditions. Neither source provides complete information, but we use them together to derive model-based corrected dummy variables in MCBS for the full range of existing health conditions using a missing data and measurement error model framework. We create multiply imputed dummy variables and use them to construct the prevalence rate and trend estimates. The broader goal, however, is to use these corrected or modeled dummy variables for a multitude of policy analysis, cost modeling, and analysis of other relationships either using them as predictors or as outcome variables.

Combining Multiple Imputation and Hidden Markov Modeling to Obtain Consistent Estimates of Employment Status Laura Boeschoten and others Journal of Survey Statistics and Methodology, Volume 9, Issue 3, June 2021, Pages 549-573, https://doi.org/10.1093/jssam/smz052 Abstract Recently, a method was proposed that combines multiple imputation and latent class analysis (MILC) to correct for misclassification in combined data sets. A multiply imputed data set is generated which can be used to estimate different statistics of interest in a straightforward manner and can ensure that uncertainty due to misclassification is incorporated in the estimate of the total variance. In this article, MILC is extended by using hidden Markov modeling so that it can handle longitudinal data and correspondingly create multiple imputations for multiple time points. Recently, many researchers have investigated the use of hidden Markov modeling to estimate employment status rates using a combined data set consisting of data originating from the Labor Force Survey (LFS) and register data; this combined data set is used for the setup of the simulation study performed in this article. Furthermore, the proposed method is applied to an Italian combined LFS-register data set. We demonstrate how the MILC method can be extended to create imputations of scores for multiple time points and thereby show how the method can be adapted to practical situations.

Effects of Outcome and Response Models on Single-Step Calibration Estimators Daifeng Han and Richard Valliant Journal of Survey Statistics and Methodology, Volume 9, Issue 3, June 2021, Pages 574-597, https://doi.org/10.1093/jssam/smz057 Abstract Calibration directly to population control totals, bypassing any explicit nonresponse adjustments, is one option for correcting for nonresponse bias. Poststratification, raking, and general regression estimation are alternative methods for single-step nonresponse adjustment through calibration. The models that underlie the response mechanism and the structure of outcome variables collected in a survey both affect the efficacy of these estimators. Consistent with earlier literature, we demonstrate that the model for the outcome variables is most important in determining bias, standard errors, and confidence interval coverage. However, if the predictive power of the model for an outcome variable is weak, properties of the aforementioned estimators of totals will be very similar.

Exploring Scale Direction Effects and Response Behavior across PC and Smartphone Surveys Dagmar Krebs and Jan Karem Höhne Journal of Survey Statistics and Methodology, Volume 9, Issue 3, June 2021, Pages 477-495, https://doi.org/10.1093/jssam/smz058 Abstract The effects of scale direction on response behavior are well known in the survey literature, where a variety of theoretical approaches are discussed, and mixed empirical findings are reported. In addition, different types of survey completion devices seem to vary in their susceptibility to scale direction effects. In this study, we therefore investigate the effect of scale direction and device type on response behavior in PC and smartphone surveys. To do so, we conducted a web survey experiment in a German non-probability access panel (N = 3,401) using a two-step split-ballot design with four groups that are defined by device type (PC and smartphone) and scale direction (decremental and incremental). The results reveal that both PCs and smartphones are robust against scale direction effects. The results also show that response behavior differs substantially between PCs and smartphones, indicating that the device type (PC or smartphone) matters. In particular, the findings show that the comparability of data obtained through multi-device surveys is limited.

Fit for Purpose in Action: Design, Implementation, and Evaluation of the National Internet Flu Survey Jill A Dever and others Journal of Survey Statistics and Methodology, Volume 9, Issue 3, June 2021, Pages 449-476, https://doi.org/10.1093/jssam/smz050 Abstract Researchers strive to design and implement high-quality surveys to maximize the utility of the data collected. The definitions of quality and usefulness, however, vary from survey to survey and depend on the analytic needs. Survey teams must evaluate the trade-offs of various decisions, such as when results are needed and their required level of precision, in addition to practical constraints like budget, before finalizing the design. Characteristics within the concept of fit for purpose (FfP) can provide the framework for considering the trade-offs. Furthermore, this tool can enable an evaluation of quality for the resulting estimates. Implementation of a FfP framework in this context, however, is not straightforward. In this article, we provide the reader with a glimpse of a FfP framework in action for obtaining estimates on early season influenza vaccination coverage estimates and on knowledge, attitudes, behaviors, and barriers related to influenza and influenza prevention among civilian noninstitutionalized adults aged 18 years and older in the United States. The result is the National Internet Flu Survey (NIFS), an annual, two-week internet survey sponsored by the US Centers for Disease Control and Prevention. In addition to critical design decisions, we use the established NIFS FfP framework to discuss the quality of the NIFS in meeting the intended objectives. We highlight aspects that work well and other survey traits requiring further evaluation. Differences found in comparing the NIFS to the National Flu Survey, the National Health Interview Survey, and Behavioral Risk Factor Surveillance System are discussed via their respective FfP characteristics. The findings presented here highlight the importance of the FfP framework for designing surveys, defining data quality, and providing a set a metrics used to advertise the intended use of the survey data and results.

Oversampling of Minority Populations Through Dual-Frame Surveys Sixia Chen and others Journal of Survey Statistics and Methodology, Volume 9, Issue 3, June 2021, Pages 626-649, https://doi.org/10.1093/jssam/smz054 Abstract Previous studies have shown disparities in health conditions and behaviors among different ethnic groups. Sampling designs that do not consider oversampling certain minority populations, such as American Indians or African Americans, may not produce sufficient sample sizes for estimating health parameters for minority populations. Oversampling is one of the most common approaches that researchers use to achieve required precision levels for small domain estimation. However, it has not been rigorously investigated in dual-frame survey settings. To take advantage of extra information for minority populations in the Marketing Systems Group database, we propose a novel optimal oversampling strategy that minimizes the domain variance subject to total cost restriction or vice versa. We further extend the method to oversample multiple minorities simultaneously. Empirical study using a population-based community survey shows the benefits of our proposed methods compared with traditional methods in terms of statistical efficiency and cost balance.

Synthesizing Geocodes to Facilitate Access to Detailed Geographical Information in Large-Scale Administrative Data Jörg Drechsler and Jingchen Hu Journal of Survey Statistics and Methodology, Volume 9, Issue 3, June 2021, Pages 523-548, https://doi.org/10.1093/jssam/smaa035 Abstract We investigate whether generating synthetic data can be a viable strategy for providing access to detailed geocoding information for external researchers, without compromising the confidentiality of the units included in the database. Our work was motivated by a recent project at the Institute for Employment Research in Germany that linked exact geocodes to the Integrated Employment Biographies, a large administrative database containing several million records. We evaluate the performance of three synthesizers regarding the trade-off between preserving analytical validity and limiting disclosure risks: one synthesizer employs Dirichlet Process mixtures of products of multinomials, while the other two use different versions of Classification and Regression Trees (CART). In terms of preserving analytical validity, our proposed synthesis strategy for geocodes based on categorical CART models outperforms the other two. If the risks of the synthetic data generated by the categorical CART synthesizer are deemed too high, we demonstrate that synthesizing additional variables is the preferred strategy to address the risk-utility trade-off in practice, compared to limiting the size of the regression trees or relying on the strategy of providing geographical information only on an aggregated level. We also propose strategies for making the synthesizers scalable for large files, present analytical validity measures and disclosure risk measures for the generated data, and provide general recommendations for statistical agencies considering the synthetic data approach for disseminating detailed geographical information.

Telephone Sample Surveys: Dearly Beloved or Nearly Departed? Trends in Survey Errors in the Era of Declining Response Rates David Dutwin and Trent D Buskirk Journal of Survey Statistics and Methodology, Volume 9, Issue 3, June 2021, Pages 353-380, https://doi.org/10.1093/jssam/smz044 Abstract Telephone surveys have become much maligned in the past few years, considering recent failures to correctly predict elections worldwide, response rates declining into the single digits, and the rise of low-cost, nonprobabilistic alternatives. Yet there is no study assessing the degree to which data attained via modern-day telephone interviewing has or has not significantly declined in terms of data quality. Utilizing an elemental approach, we evaluate the bias of various cross-tabulations of core demographics from a collection of surveys collected over the past two decades. Results indicate that (1) there has been a modest increase in bias over the past two decades but a downward trend in the past five years; (2) the share of cell phone interviews in samples has a significant impact on the bias; (3) traditional weighting largely mitigates the linear trend in bias; and (4), once weighted, telephone samples are nearly on par in data quality to higher response rate unweighted in-person data. Implications for the "fit for purpose" of telephone data and its general role in the future of survey research are discussed given our findings.

The Relationship Between Interviewer-Respondent Rapport and Data Quality Hanyu Sun and others Journal of Survey Statistics and Methodology, Volume 9, Issue 3, June 2021, Pages 429-448, https://doi.org/10.1093/jssam/smz043 Abstract Interviewer-respondent rapport is generally considered to be beneficial for the quality of the data collected in survey interviews; however, the relationship between rapport and data quality has rarely been directly investigated. We conducted a laboratory experiment in which eight professional interviewers interviewed 125 respondents to see how the rapport between interviewers and respondents is associated with the quality of data-primarily disclosure of sensitive information-collected in these interviews. It is possible that increased rapport between interviewers and respondents might motivate respondents to be more conscientious, increasing disclosure; alternatively, increased rapport might inhibit disclosure because presenting oneself unfavorably is more aversive if respondents have a positive relationship with the interviewer. More specifically, we examined three issues: (1) what the relationship is between rapport and the disclosure of information of varying levels of sensitivity, (2) how rapport is associated with item nonresponse, and (3) whether rapport can be similarly established in video-mediated and computer-assisted personal interviews (CAPIs). We found that (1) increased respondents' sense of rapport increased disclosure for questions that are highly sensitive compared with questions about topics of moderate sensitivity; (2) increased respondents' sense of rapport is not associated with a higher level of item nonresponse; and (3) there was no significant difference in respondents' rapport ratings between video-mediated and CAPI, suggesting that rapport is just as well established in video-mediated interviews as it is in CAPI.

Transitions from Telephone Surveys to Self-Administered and Mixed-Mode Surveys: AAPOR Task Force Report Kristen Olson and others Journal of Survey Statistics and Methodology, Volume 9, Issue 3, June 2021, Pages 381-411, https://doi.org/10.1093/jssam/smz062 Abstract Telephone surveys have been a ubiquitous method of collecting survey data, but the environment for telephone surveys is changing. Many surveys are transitioning from telephone to self-administration or combinations of modes for both recruitment and survey administration. Survey organizations are conducting these transitions from telephone to mixed modes with only limited guidance from existing empirical literature and best practices. This article summarizes findings by an AAPOR Task Force on how these transitions have occurred for surveys and research organizations in general. We find that transitions from a telephone to a self-administered or mixed-mode survey are motivated by a desire to control costs, to maintain or improve data quality, or both. The most common mode to recruit respondents when transitioning is mail, but recent mixed-mode studies use only web or mail and web together as survey administration modes. Although early studies found that telephone response rates met or exceeded response rates to the self-administered or mixed modes, after about 2013, response rates to the self-administered or mixed modes tended to exceed those for the telephone mode, largely because of a decline in the telephone mode response rates. Transitioning offers opportunities related to improved frame coverage and geographic targeting, delivery of incentives, visual design of an instrument, and cost savings, but challenges exist related to selecting a respondent within a household, length of a questionnaire, differences across modes in use of computerization to facilitate skip patterns and other questionnaire design features, and lack of an interviewer for respondent motivation and clarification. Other challenges related to surveying youth, conducting surveys in multiple languages, collecting nonsurvey data such as biomeasures or consent to link to administrative data, and estimation with multiple modes are also prominent.