How does question design regarding non-substantive response options affect survey outcomes? This research question, and specifically the effect of using non-substantive response options – the Don’t... Show moreHow does question design regarding non-substantive response options affect survey outcomes? This research question, and specifically the effect of using non-substantive response options – the Don’t Know option, filter question and follow-up question – on item nonresponse and the substantive overall distribution of opinions was examined by conducting three experiments with three Dutch internet panels (the LISS panel, the EenVandaag Opiniepanel and Team Vier’s internet panel). The effect of offering a non-substantive response option is that item nonresponse changes, but not the resulting picture of public opinion in terms of majorities or pluralities. The effect of a non-substantive response option for substantively different issues was expected to vary per the relation to a major political dimension (resulting in less item nonresponse) or to foreign policy issues (resulting in more nonresponse). The results were mixed; more research is needed to differentiate between question content. The follow-up question, which differentiates between permissive and directive opinions, resulted in about two-third of the respondents expressing a directive opinion, which was more than expected. The main recommendation is to start including the most basic non-substantive response option, the explicit Don’t Know option, as a response category for opinion questions. Show less
