Jun/100
Settlement.Org User Survey Report
Like most sites Settlement.Org has tried to get users to fill out a ’site survey’ but very few people did. The information we got was not very useful. So we took it down when we changed to our new look.
However, as part of our ongoing evaluation exercises for our projects we decided to get ambitious and go for a once-a-year type of survey with a large number of respondents.
The major purposes for the evaluation of Settlement.Org were:
- Provide a better picture of the users of Settlement.Org
- Obtain feedback from users about their satisfaction with the redesigned and reorganized Settlement.Org website
- Assess the usability of the website for newcomers
- Collect feedback from users about the strengths and limitations of Settlement.Org website and their suggestions for improvement
We worked with our evaluator Dr Arnold Love to come up with about 20 questions which included several demographic questions.
A total of 608 Settlement.Org users completed the online survey. This participation rate produced a very statistically significant set of results.
You can download the full Survey Report here (2MB, PDF)
Findings Summary
- 95% of respondents indicated a high level of satisfaction with the website
- 97% of users would recommend it to others
- Nearly all (93%) of respondents said they quickly understood how to use the Settlement.Org website.
- Suggested ways to improve the site included better searches, keeping content updated and providing a call centre/chat features.
There is much more detail in the report including demographic breakdowns on gender, immigration status, age, etc.
Lessons for Doing Surveys
We learned a lot about how to do a successful survey.
1) Offer prizes: We offered an IPod and other substantial prizes and this made a big difference we feel. Over half of the respondents opted-in to the draw.
2) Reminders: We put the survey on for 3 weeks and sent out reminders through various channels at least once a week.
3) Try to restrict survey to the audience you are trying to measure: We wanted just newcomers to take the survey but we feel there were a number of “settlement workers” who took the survey too. This isn’t a bad thing but it probably skewed a couple of questions such as “time in Canada” and “immigration status”. What we learned was to do a better job communicating who should and shouldn’t take this survey and to use the technical tools of the survey programs to direct user types to different questions.
We hope to do this survey every 2 years. The next survey planned will be dedicated to finding out how our sites can better assist workers serving newcomers.
Feel free to contact us is you have any questions about this survey.