Controlling a Survey

adavis

I asked this over in the Database Forum, but didn't get a discussion started, so I thought I'd ask it over here...

I have a survey to build from my son's school. Well, actually, I already have the survey built, all except the login feature. I want each person to log in with their student's student id number and then just tick them off on the database as having taken the survey, not relating this fact to their response to the survey in any way. This would allow each family to take the survey for each child they have at the school. Some on the school's board is concerned that the parents will not be completely open with their responses because their answers may be tied to their name in some way.

One board member suggested IP addresses, but that won't work for several reasons. First off, most people have access to muliple computers, and second almost no one has a static IP addess. I can't think of any other way to do it other than what I have already suggested.

One of the board members found an online survey service and had a bunch of us take it and then try to take it again. Okay well everyone did and said that no, they couldn't take it again, except for me, who took it, deleted my cookies, and then could take it again.

My contention that I cannot control the survey on the client side. It has to be done on the server side with a log in. Am I right on this?

We've discuss the issue of trust and that the parent will just have to trust that I won't tie the log in to responses. That aside, do any of you have any other suggestions on any other way to insure that each family only takes the survey once, but maintains their anonymity?

Thanks,
Alisa

dalecosp

Yeah, when you come right down to it, you probably couldn't even have them come to your facility, take the survey, and then stamp their forehead and not have them wash it off ....

Why don't you generate a bunch of random passcodes, stick 'em in your auth routine, print them on card stock, have them cut apart and send enough of them to each classroom for each student to have one? Then, the teacher puts them in the proverbial hat, and they draw it out, take it home, etc.

Heh, heh, nevermind that most kids can't even get home with a note from teacher or a "report card" ....

Of course, why the patrons of a Christian school don't trust their administration is quite beyond me ..... :rolleyes:

Roger_Ramjet

Good solution dalecsop.

If the parents' really do think their children would be victimised if the board don't like their answers, then I'm not surprised you need this survey. By the sound of it what you really need is a new administration and school board.

adavis

I've had a child at this school since 1998 and since then, the school has changed. I liked it better when we first came, but it still runs circles around the public schools around here. Anyway, that being said, I trust me doing the survey and anyone who knows me will trust me doing it. What I don't trust is them hiring some online survey company. However, I do a lot of surveys at work and this is an issue that comes up constantly. I am a contractor for the company that my husband works for and he never would participate in surveys until they hired me. I assured him that I wasn't collecting any information that would tie him to his responses, so he could say any thing he wanted to. So, he did 🙂.

Anyway, the board wants the parents to be completely open and say what's on their mind. They know there are some problems and some concerns. That's why they're doing this survey. Even though this is a Christian school, with a Christian staff, board, and faculty, only about 50% of our student's are from Christian homes, so that's why this trust issue was raised by the board. It isn't exactly that the parents have said they won't or don't trust the survey, it's just that the board wants the parents to know that they are anonomous and if they have to log in, how will they know for sure?

I don't think anyone is concerned that their child will be victimized if someone doesn't like their responses to the survey. You should see what I'm going to write, and they will KNOW it's from me. It's going to be way to specific to be from anyone else. But, I write about 3 surveys a year at work and I know how to write the reports so that all the comments are sorted so that you can't figure out which order they came in, much less which comment goes with which survey response. See, I don't trust anyone either, so I write my surveys in such a way that the administration can't backtrack to an individual respondant. :evilgrin: If they only knew.... 😉

Roger_Ramjet

Sound like a cultural problem in the US. I've worked for market research in the UK and Australia and the question of trust was not one that arose. People did not respond because they could not be bothered, not because they thought that their personal information would be sold on or used for a sales pitch. Must be better regulated over here.

You would not have been able to back-track from the results to the individual in my computer system: again a matter of regulation.

piersk

To be honest, I don't know if it would really matter. The fact that you are asking for the student number would possibly put off people from submitting more than one. I have recently done a survey for the events committee here at uni and we have asked for their student number and that put a lot of people off doing multiple surveys

adavis

Okay, but you agree with me that the only way to control the number of times a person takes a survey is on the server side with a login, right?

The way I have this survey is designed is that there are an open ended number of questions (stored in a db table) that can be asked with a "Strongly Agree" to "Strongly Disagree" response plus a short (varchar) comment area for each question. Then there's an area of "Rate the importance of these issues" highest to lowest. Issues probably isn't really the right word, it's more like a wish list. Lastely there is a Blob where they can add a long comment. Each of these will be tallied seperately, so there is no connection between what respondant 2639 answered on question 3 and what their comment was on said question and what their wish list was and what their blob was. All comment areas are optional as not everyone will feel that they need to comment on every topic. I would also like to have the survey so that it recognizes the login number as being from a pre-school parent or kindergarten parent or elementary parent or high school parent and target the survey accordingly. However, that does mean that they will have to take the survey multiple times if they have students in all 4 areas. Yikes! could you imagine! :eek: Mostly we have families with children in 2 areas. We've had families with as many as 5 kids, but those are few and far between and usually doctor's kids as the average engineer couldn't afford the tuition for that many.

This does get the wheels turning however, in that they could log in with all the numbers at once and then I could customize the survey accordingly so that they just have to take it once. Giving them just the questions they need. Something like:

How many students do you have at the school? 2

Enter Student Number 1 [...]
Enter Student Number 2 [...]

and then go from there...

Roger_Ramjet

How can you do any meaningfull analysis of the data if there is no correlation between the answers? I thought you said that you worked in this industry? Have you piloted the quesionaire to arrive at the questions that you can just ask respondants to rate on your scale? Do you know how to derive a code-frame with which to categorise and analyse the open-ended questions, let alone the 'blob'?

Methinks you're not going to get much out of this survey beyond confirmation of your, or the boards', pre-existing opinions. Or is that it's main purpose?

adavis

Maybe I need some education. I've only just done or taken the type of surveys where you rate on a scale. If your willing could you give me some direction on this??? I'm on IM right now. I also have Yahoo, but haven't added it to my new computer.

Roger_Ramjet

Don't use IM or IRC or any of them. I don't allow any of that on our LAN, so I can't very well use it myself.

To develop a survey questionaire, one sjuold begin with a pilot study using a representative sample of the eventual target population. All questions should be open ended: ie ask the question and take down the response verbatim. No pre-coding.

The full english answers are then analysed individually. Each meaningfull phrase must be written down, and when they crop up in another person's responses, counted. Once a phrase is seen to recur, it is allocated it's own code.

These should NOT include the meaningless answers like 'It's nice!' or 'I like it' . When running the pilot an experienced interviwer would probe for a meaningfull answer, eg 'What is nice about it in particular'?' or 'What is it the you like the most?' etc.

Focus groups are often used to identify the questions to ask in the full survey.

The code-frame derived from this analysis if the pilot study is what you then use to formulate the pre-coded questions: those that can be answered by ticking a box on a scale, or selecting amongst alternatives in a multi-choice.

Once you have run the full survey, you take a sample of the responses and again analyse the full text answers to derive a code-frame that is then used to code the whole survey. Every phrase in an individual response that matches a code in the frame must be recorded. The code frame should be extended when neccessary and appropriate.

All questions must be corellated in the analysis. EG

75% of the people who said Physical security at school was v. important also said that that was one of the reasons they chose this school.

See, it puts the first answer into a very different light.

You'll need a stats package for the analysis to draw out such significant correlations.

adavis

Thanks. I never had anyone explain to to me in that way before. I can do that. You've been a big help.

One thing about me is that I've never been afraid or embarassed to ask for help when I don't know the answer.

My kids, now that's a different story. :glare:

Roger_Ramjet

Glad to be of help. Reading back over my last post, I see one or two ambiguities (apart from my dyxlexic typing). I think that you have probably taken the correct meaning, but I'll just clarify them anyway.

Drawing up a code frame:
Each meaningfull phrase would be copied down. As a similar meaning is found in another response, it is counted. In a sample of 50, 5 counts would qualify the meaning for it's own code. The meanings left at the end without a code are then reviewed to see how they can be aggregated into several codes.

Coding the responses:
Similar meanings are coded from the code frame for each answer. Anything not covered by the code frame is extracted and written down. Some of these will quelify for their own code as the coding progresses.
Whatever is left is kept for the analysis of results. You will find that these odd leftovers can sometimes contain really significant insights and information. They can, for instance, give real insight into the attitudes of the population. Attitudes are based upon belief systems, and the basic shared assumptions within a culture are rarely voiced so they will not often crop up in peoples answers. But they are the most valuable information that your survey could reveal, especially as you and the board probably share those same assumptions all unknowingly.

adavis

I can read dyslexic pretty well, so didn't have any probelms with it 🙂.

When I went back to college when my oldest was about 18 months old, the first course I had was business statistic. I was so afraid that I would make a low grade, and I had made a D in the prerequisite calculus course about 5 years previous, that I studied constantly, working every problem at the ends of the chapters twice, plus the extra practice problems in the back of the book. Funny thing was that when I applied integrals to something I understood, like statistics, it all made since. I ended up with the top grade in a class of about 30. I don't think I sold that book, so I going to see if I can find it. It was all about this type of number manipulation we've been talking about.

Roger, you've been a great help and reminded me of a lot of things I learned and then forgot. I graduated from college in 1992, but didn't go to work until 2000. You forget a lot of basics in 8 years, but my babies needed me more than we needed the money.

If you think of anything else, please add it. I'm always wanting to learn. I want to go back to college again, but now, my babies need my paycheck more then they need me at home 🙂.

As far as data collection goes, the school had a "town hall meeting" a while back where they had an open floor discussion and gathered a lot of information that they are planning on using to structure their questions from. They know that there are problems and I'm not the only one who's said that things have declined over the past 6-7 years. They are trying to get back the "good stuff" while at the same time moving forward. One thing is that we've added school "uniforms" which has been hard for a lot of people. My son absolute hates them, though when he looks at other private schools around here, he has to rethink that. Ours are pretty flexible, and don't look like the Catholic school around the corner, which are typical Catholic school uniforms with navy slacks and white shirts for the boys and plaid skirts or jumpers and white blouses for the girls. Ours are polos and khaki chinos from Land's End.

Thanks again, Alisa

Roger_Ramjet

's alright. I was thinking about this last night, writing that stuff brought back fond memories of Melbourne.

In the 90s Total Quality Management (TQM) became flavour of the month in management circles. Now, every organisation has its mission statement, its 'Investor in People' motto, its 'commitment to quality, etc, etc, etc. But for the most part little has changed. What so many fail to see is that these are the form, not the substance of TQM.

Every organisation is a culture. And every culture has its set of shared attitudes and values and even its own language. Changing an organisation means changing its culture, and that is what so many companies have so signally failed to realise, let alone do. That is the essence of TQM - change your culture and the rest will follow.

The change process must begin by unfreezing the current state. Only then is movement possible. Of course, once you have made the move, things freeze up again. It is the 'unfeezing' that is the hard part - getting people to articulate and examine their unquestioned assumptions because these the very foundations of their belief system and to question them is to question their very feelings of self-worth.

That is why I laid so much stress on the extraction of attitudes from the survey responses - it is the essense of market research. Quite frankly, people give the answers that they think you want to hear, or that they think will make them look good. A good survey goes beyond that to reveal peoples motivations, their inner dreams and desires. Unfortunately, this is usually so that a company can exploit them.

Roger_Ramjet

cont.

Problems in education are happening all over. I personally think that we have done today's youth a diservice, we have not equipped them for the real world of work. No one is allowed to fail, but it is how you deal with failure that is the mark of your character. People become world champions not because they never fail, are never beaten, but because they deal with adversity better than the rest.

Everything has to be interesting for the kids, never boring. But you know as well as I do that no matter what your job, it will always involve hard work and just plain drudgery if you want to succeed. They think that pop stars and sports stars have it easy, that it just falls into their laps. They never see the years and years of hard dedicated work that went into building the skills and developing the talents.

I went to one of the best grammer school in the UK, it has been in the top 10 for the last 40 years. No one was under the illusion that shool work had to be interesting, or easy. We just got on with it. And that was what we expected we would do when we began our chosen careers. And we knew we were the lucky ones who would have the chance of a profession of our choosing.

Well, enough of my ranting. Good luck with the survey, and with turning the school around.

davidjam

Aside from rr's excellent advise on the building and structuring of the survey(s), I think the original problem comes down to:

a) Assure the parents on the login page that their "anonymity will be absolutely preserved". You can even say that their user information is only an "internal requirement for recording to the database. This will ensure that there will only be one survey completed per child" (or something like that) and "will be automatically removed when the survey is compiled" (or something like that). I expect there will only be a few who will not believe such statements are true.

or,

b) On the introduction page make it very clear (in red ink!) that parents may only fill out one survey per child. This is the "honor system". (I mean how many people are really that dishonest?) Perhaps on the compilation side responses that are very close to being identical (flagged automatically by the application) can be visually checked, and if the custom comments seem like the same person then just delete those!

MHO