Survey Instructions
- Setting up a Survey
- Conducting a Survey
- Analysing a Survey
- Guidance for Survey Administrators
- Guidance for Unit Cdrs
Setting up a Survey.
1. To set up a survey select item ‘5 – Set Up Unit Survey' from the left-hand menu bar and follow the instructions. This should be done by the nominated survey administrator, who has been selected by their unit cdr to oversee the survey process. Further guidance is provided during the set-up process. Following submission of a survey request the administrator and unit cdr will receive an automated email providing them with a unique survey ID for each survey requested.
2. If the unit cdr has particular areas of concern it is possible to add up to 4 additional questions to each survey. Further detail on this can be found in the ‘Optional Survey Items’ section under item ‘2 – Survey Information’.
Conducting a Survey.
3. Evidence has shown that the most successful surveys were those that had a personal and visible endorsement from the sqn cdr. The process should begin with a brief to personnel, from the sqn cdr, on the importance of the survey and what is expected of them. Further guidance to sqn cdrs can be found at the bottom of this section.
4. Following this, the survey administrator should facilitate easy completion of the survey by providing as many options for doing so as appropriate to their unit. For ease of completion, SEAT can be accessed from any internet enabled device by either clicking on the link provided in the initiation email or by selecting item '1 – Take a Survey' from the main menu and entering the survey ID.
5. Administrators should aim to have all respondents complete the survey within a 30-day period or as close as practicable. Letting surveys run for too long risks significant events affecting the results of later surveys so that the culture and concerns they represent are different to those of the earlier surveys. This will skew the overall result.
6. The survey administrator should monitor the completion rate to ensure the desired level of engagement is being achieved. Once the maximum practicable level of completion has been reached the administrator should request the survey be closed. This is achieved by contacting your service administrator, the details of which can be found under item '11 - Contact Us'.
Analyzing a Survey.
7. Once the survey is complete the survey owner will receive an access code allowing them or a nominated individual to analyse their unit’s data. The Single Service Safety Centres provide the minimum level of oversight to allow for smooth administration of the system, and provide advice to users, as required. However, they do not routinely analyse individual unit surveys. This is for 3 main reasons:
- Confidentiality – Honest and frank responses are at the heart of SEAT. Sqn cdrs need to feel comfortable using the system without the worry of their Cmd ‘long screwdrivering’ and drawing unhelpful (often inaccurate) conclusions from the data.
- Context – Effective analysis is conducted with reference to the specific context in which the responses were made. It is unlikely the Safety Centre will understand the context of individual units and any conclusions drawn are likely to be generic, inaccurate and generally unhelpful.
8. A tutorial video explaining the function and interpretation of the software may be seen in this video:
Or, from an MoDnet computer, copy and paste the URL below into a separate tab of your browser to view the instructional video:
https://modgovuk.sharepoint.com/:v:/r/teams/5699/Libraries/RAFSEET%20Analysis%20Tutorial.mp4?csf=1&e=zUXS3F 9. Deriving target numbers. Within any group there will be a variation of attitudes and perceptions. However, it is often impractical to survey every member of a target group and as such it is important to have an idea of the level of confidence one has in the data obtained for any number of responses achieved vs group population size. Fortunately, academic research has provided us with a means of attributing a quantifiable value to that question.
10. Based on the population size the system will derive a ‘target’ number. This is the number of respondents required to achieve a 95% confidence level that the results obtained represent the views of the entire group. It is not always possible to achieve the target response level and for this reason the system will indicate an approximate confidence level, which will need to be taken into account when drawing any conclusions.
Population | 10 | 20 | 30 | 40 | 50 | 60 | 70 | 80 | 90 | 100 | 120 | 140 | 160 | 180 | 200 | 250 | 300 | 400 | 500 |
Target | 10 | 19 | 28 | 36 | 44 | 52 | 59 | 66 | 73 | 80 | 92 | 103 | 113 | 123 | 132 | 152 | 169 | 196 | 217 |
Guidance for Survey Administrators
11. You have been nominated by your unit cdr to act as the survey administrator. Your role is to administer the conduct of the survey on behalf of your unit/wing OC/CO (for simplicity we will use OC/CO to refer to this position). The effectiveness of this tool is directly proportional to the effort and engagement of the unit management and as the administrator it is your job to coordinate this activity. Below are a few guiding principles which will help you discharge your duty.
- Anonymity of survey respondents: The survey process will NOT succeed unless survey respondents are assured that they have complete anonymity of their responses. Please emphasise that your sqn cdr is unable to associate any demographic information with the corresponding safety item responses. Only the averages of the safety survey items are viewable by anyone with whom the access codes are shared.
- Confidentiality for the OC/CO: OCs/COs are considered the owners of the data for the purposes of distribution. Only your OC/CO will receive the access code(s) to view their data. Anyone else requiring access will only be given the access code(s) with their permission.
- How often should a unit conduct the survey process? It is recommended that units conduct the SEAT annually and within 30 days following a change of command. Additional occasions may be driven by deployment cycle, a desire to examine a particular issue, or a sense that something just isn't right and a need for feedback from unit personnel.
- How many personnel should take the survey? ALWAYS request a number of surveys equal to your total eligible strength (e.g., all aircrew, all engineers, etc.). Then strive to achieve participation from as many personnel as possible. The aim is not to request a number which is achievable, but rather to use the system generated 'target' number as an indication of the level of confidence that the survey results represent the attitudes and perceptions of the entire target population. Table 1, above, details how the target numbers are derived. Surveys that fall short of the target number are not worthless. Surveys can be closed with any level of participation, OCs/COs will simply need to consider the confidence level during the analysis process in order to quantify the validity of any conclusions they draw.
- How many personnel have taken the survey? The survey administrator can determine how many individuals have completed the survey by clicking on the administration link in the initiation email (para 3.) or by going to ‘item 8 – Survey Admin’ link in the left-hand menu bar and entering the survey ID. This will provide the number of personnel who have completed the survey so far and corresponding confidence level.
- Publicising the survey: Effective communication is key to survey success. To assist with this a simply poster has been created on which you can write your survey IDs and deadline. Multiple copies can then be pinned around the workplace. This is supported by an informational leaflet, which provides personnel with information on what SEAT is. Both can be found under item ‘10 – Resources’. Please take the time to look at them and consider their use if necessary.
- How long should the survey run for? All respondents should complete the survey within a 30-day period or as close as practicable. Letting surveys run for too long risks significant events affecting the results of later surveys so that the culture and concerns they represent are different to those of the earlier surveys. This will skew the overall result.
- Closing the Survey: Once the maximum number of available respondents have completed the survey, the survey administrator should contact their service administrator and request the survey be closed. This will generate an automated email containing the survey access code(s) and instructions on how to interpret the data, in the form of a link to a tutorial video. This email will only be sent to the OC/CO, as the survey owner. It is at the OC/CO's discretion if they wish to delegate the analysis of the results to another individual
Guidance for Unit Commanders
12. As the unit commander this survey tool is for your use to better understand the attitudes and perceptions of your personnel, and the issues within your own unit. The effectiveness of this tool is directly proportional to the effort and engagement of the unit management and as the unit commander it is for you to direct what you require and to personally shape the success of the survey. Below are a few guiding principles which will help you get the most out of your survey:
- How often should a unit conduct the survey process? It is recommended that units conduct the SEAT annually and within 30 days following a change of command. Additional occasions may be driven by deployment cycle, a desire to examine a particular issue, or a sense that something just isn't right and a need for feedback from unit personnel.
- Confidentiality for the Unit Cdr: Unit cdrs are considered the owners of the data for the purposes of distribution. Only you will receive the access code(s) to view your data. Anyone else requiring access will only be given the access code(s) either by you or with your permission.
- Anonymity of survey respondents: The survey process will NOT succeed unless survey respondents are assured that they have complete anonymity of their responses. Please emphasise that you are unable to associate any demographic information with the corresponding safety item responses. Only the averages of the safety survey items are viewable by anyone with whom the access code(s) are shared.
- Nominate an administrator: You should nominate a survey administrator who will administer the survey on your behalf. This will include setting-up the survey, facilitating its promotion and completion, monitoring progress and finally closing the survey following your direction.
- Promotion of the survey: Evidence has shown that the most successful surveys are those that had a personal and visible endorsement from the OC/CO. The process should begin with a brief to personnel from you on the importance of the survey and what is expected of them.