Survey Best Practices

1. Survey invitations must clearly identify the group or person who is conducting the survey and provide contact information (typically name and email address) should respondents have questions about the content of the survey or about the use/publication of survey results.
2. Participants must be notified that their participation in the survey is voluntary.
3. Participants must be notified whether their responses will be anonymous or confidential.

An anonymous survey means that the researcher cannot determine who participated in the survey and who has not. Anonymous surveys typically do not collect identifying information, and cannot restrict responses to one response per person.

A confidential survey collects information that can identify participants either alone or in combination (e.g., race/ethnicity, gender, major, class year) and allows the researcher to analyze responses along a variety of demographic factors, but also means researchers have a responsibility to protect the privacy of the participant. When data are collected confidentially, it is helpful to let participants know that results will be presented in aggregate only and that any results that can potentially identify participants will not be presented.

If your survey results will include academic or contact information for HMC students, you may be subject to Federal Family Educational Rights and Privacy Act (FERPA) regulations. Non-directory student data (e.g., GPA, Major(s), race/ethnicity) can be used without a student’s consent by college employees for “legitimate educational” purposes, provided the data are not reported in such a way that individual students can be identified. Otherwise, users must receive written consent from students.

It is typically not necessary to survey all QC students or all QC employees (census surveys). A sample needs to be large enough to protect the privacy of respondents and large enough for your results to be considered representative, but that does not mean you need to survey everyone. If you wish to target all students or all employees, OIE will typically employ a random sampling method to ensure that these populations are not over-surveyed. To calculate the minimum sample size to be considered representative within a given margin of error, visit Sample Size Calculator.

It is always best to pilot your survey to a small group from the population you intend to survey. As an assesment of the survey’s face validity, this can help you understand how the questions you have written will be perceived and answered and whether the survey is too long or overly complicated.  Be sure to ask your pilot group whether any survey question was difficult to understand or difficult to answer.

If your survey project requires more rigorous testing, please consult with OIE. We can estimate the construct validity of an instrument using confirmatory factor analysis, and the internal consistency and reliability of the instrument can be estimated with Cronbach Alphas. Factor analysis and Cronbach Alphas can determine whether items in a survey should be edited or removed.

If the OIE fields a survey on your behalf, there is no need for demographic items on the survey such as a race/ethnicity question, as we can tie confidential survey responses to administrative data. Anonymous surveys that require demographic information should ask these types of questions near the end of the survey and these questions MUST be voluntary. Please also note that as per QC and CUNY policy, any survey that includes a race/ethnicity item must include “Italian decent” or “Italian American” as a response option since Italian Americans are included among protected groups at CUNY.

We’re more likely to partipate in a survey when (A) the purpose of the survey is clear and (B) we know who is asking the questions. Survey invitations should be short and concise, but explain why particpants’ reposnes are important and how the survey results will be used. Also,  be sure to sign the survey invite as the PI. If you are sending a survey to faculty or staff at QC, it can be helpful to send out a longer email to your target poulation prior to the survey invitations so that people know what to expect.

Depending on the population and the survey, you may offer incentives to survey respondents. Incentives must be positive (i.e., no negative consequences for failing to participate). They should also be relevant to the topic of the survey, and small enough so as not to be coercive (e.g. gift cards to Amazon, Starbucks, or the bookstore). Keep in mind that while offering incentives can increase a survey’s response rate, it can also invite less meaningul responses. Please consult with OIE if you are considering offering an incentive.

It is important to request assistance with a survey well in advance. Please understand that your preferred dates of administration may be delayed due to our availability to assist you with your project, or due to a pre-existing survey. Concurrent surveys that target the same population lower response rates, overburden respondents, and lead to poor quality data.