The first poll was conducted on Sept 9th 2005 by CBS News/NY Times:
The Question: Do you think it is better for the country to have a President who comes from the same political party that controls Congress or do you think it is better to have a President from one political party and the Congress controlled by another?
Better when same party 32
Better when different party 41
Don't know/No answer 27
Topics: Presidency Congress Parties
The second poll was conducted in June 2006 by USA Today/Gallup:
Do you think it is better for the country--to have a President who comes from the same political party that controls Congress, does it make no difference either way, or do you think it is better to have a President from one political party and Congress controlled by another?
%
Same party 26
No difference 38
Different parties 31
No opinion 5
Topics: Presidency Congress Parties
The fourth poll (I will mention the third later) was conducted on Nov 6th 2008 by Quinippac University:
Do you think the government will work better now that the President and the majority of Congress both belong to the same political party, or don't you think it will work better?
%
Will work better 60
Will not 32
Don't know/No answer 7
Topics: Presidency Congress Parties
The first two polls indicate a general pattern. When asked in 2005 and 2006 if single party rule was preferable, the result was near enough to be explained rather plainly. In both cases, the most popular choice for respondents was "Different Parties" or "Better when Different party." In the former case, the margin was narrower. This is understood to be a result of a third, acceptable choice: "No difference", which took roughly 10 points off "Better when different parties." The narrowing gap demonstrates that respondents were less convinced that different parties where better, flocking to "don't know" when possible. This shift highlights one of the dangers of polls we mentioned in class: the options often do not reflect the best possible answers. It is also worth noting the first Poll was adjusted for a larger Black population. I do not think that accounts for the skew. This is not a hot-button racial issue.
The last, poll however, is striking. No more then 32% of people in 2006 thought highly of single party rule. How did that double in 2008? The explanation is Question #13 from 20 Questions about Polls.
13. What questions were asked?
You must find out the exact wording of the poll questions. Why? Because the very wording of questions can make major differences in the results.
Perhaps the best test of any poll question is your reaction to it. On the face of it, does the question seem fair and unbiased? Does it present a balanced set of choices? Would people you know be able to answer the question?
On sensitive questions--such as abortion--the complete wording of the question should probably be included in your story. But at the very least, you must have the exact wording as you are preparing the story.
It may well be worthwhile to compare the results of several different polls from different organizations on these sensitive questions. In that case, you should be careful to compare both the results and the exact wording of the questions.
In the case of the last poll, there was an important linguistic shift. Instead of asking "do you think it is better for the country..." to have single party rule, respondents where asked "Do you think the Government will work better." This question is weighted towards a "yes" response. "Will work better" is essentially asking the respondent: do you think the government will get along better now that everyone is ideologically similar. Hmmm..... Meanwhile, the first two polls asked respondents to weigh the merits of a divided government against a single party in charge: do you choose checks and balances(divided government) or efficiency (single party rule). This question is dependent on which value the individual thinks is more important, leading to the near even divide. Thus, the results from all three of these polls make perfect sense: the polls both asked about divided government, but the similarities in the questions ended there.
A second factor in the shift is illuminated in the third poll, conducted on October 13 2006 by the WSJ/ and NBC news:
I'm going to read you a statement. Please tell me how much you agree or disagree with this statement using a five-point scale. On this scale a five means you totally agree with this statement and a one means you totally disagree. You may use any number from one to five depending on how much you agree or disagree with the statement. In a time of economic uncertainty and potential terrorism, it makes sense to have unity and cooperation in the government with a majority in Congress and a president who are of the same party.
%
5-Totally agree 33
4 17
3 21
2 10
1-Totally disagree 18
Cannot rate 1
Topics: Congress Presidency Parties
Considering that we are now (2008, when this poll was conducted) in a situation that very much matches the description in the question, the results of the fourth poll are sensible. In turn, this change illuminates Question #8 from "20":
8. When was the poll done?
Events have a dramatic impact on poll results. Your interpretation of a poll should depend on when it was conducted relative to key events. Even the freshest poll results can be overtaken by subsequent events. The President may have given a stirring speech to the nation, the stock market may have crashed or an oil tanker may have sunk, spilling millions of gallons of crude on beautiful beaches.
Poll results that are several weeks or months old may be perfectly valid as history, but are not always newsworthy.
This series of polls highlight the fact that all polls need to be taken with a grain of salt (and these 20 questions.) Looking at the first two polls alone leads to one conclusion, that the American people favor split-party rule, that is not entirely true. In turn, focusing only on either 3 or 4 would mask America's preference in a theoretical government, for split party rule. Together, and with the variances in their questions noted, these polls paint one pollsome picture.
6 comments:
Was there any difference in WHO was polled (what kind of people, from where)? Or how the poll was conducted?
Excellent analysis. I was going to object that you missed one possible explanation -- that opinion actually changed over time -- but you get at that by discussing changed circumstances. A good instance of how question-wording can affect results. As to Other Daniel's question, it shouldn't, if the sampling method was sound. Many polls, but not all, will publish a short paragraph on methodology.
2 points:
1) Prof- What question of mine is "it shouldn't" answering?
2) I went back to these 4 polls and looked at who was polled and how they were conducted. Each was conducted by phone (again, something common but still important to note) and there were varying groups of individuals polled.
I will list them in order of how they appear in Daniel K's post.
1-"a national adult with oversample of blacks sample of 1,167. An oversample of African Americans was conducted for this poll for a total sample of 211 African Americans. Results are weighted to be representative of a national adult population."
2-"a national adult sample of 1,000"
3-"a national registered voters sample of 2,210"
4-"a national registered voters sample of 1,006."
I would like to make some observations about just the data I have listed:
a- Note that numbers 3 & 4 are registered voters and 1 & 2 are not necessarily. Does this make a difference?
b- I for one think that the fact that #1 has an african american oversample which the rest do not.
I think that although analysis is good about questions, as was pointed out many times throughout the semester we cannot forget about methodology and the differences in ethnicity and income when looking at political data.
As for the first poll with an over sample of African Americans it seems that they tried to compensate for that "Results are weighted to be representative of a national adult population." As for the difference between polling registered voters or just an adult population I wonder if there is much of a difference. I think we would hope that a sample of registered voters may know a little more about politics than a sample of adults. But we know this may not be the case. However if it were the case that a sample of registered voters have greater political knowledge maybe that could account for the difference in the polls. But thats just me being optimistic about this countries registered voters.
Could the polls also be skewed because of the political situation of the time. In 2005/6, Americans had seen that a congress in control of the Republicans along with a Republican President let to an unhappy america. Perhaps they are taking out their anger on the institution without truly understanding what it means for one party to control both congress and the executive branch
Sorry -- was unclear. The composition of the polled group and how the poll was conducted should not vary in any way that would affect results; doesn't mean they did not, but they should not.
And change over time does not bias the results, but it surely can alter them. But it's not bias, it's just a reflection of "reality" changing. Or perceptions of it. . . . .
Post a Comment