|
|
| |
About our methodology
This webpage describes the methodology we use for the AmericansElect.org quiz, which uses the OnTheIssues.org database as its underlying source information. Our core concept is "transparency", which means you, the voter, can see politicians' stances on the issues, in summary form and in detail if you drill down, and then you can see how we created the summary from the details. Our core method is to infer politician answers from public statements. We also ask each politician to answer directly, but most politicians don't -- so we infer their answers; this webpage explains how.
|
|---|
|
i. How we assign answers for the AmericansElect quiz
We assign answers for ten questions of the AmericansElect quiz based on public records. We assign these answers for each presidential candidate, each House and Senate incumbent, each sitting governor and member of the Supreme Court, past presidents and vice presidents, cabinet members, and political leaders. The same methods apply to all politicians.
For each of the first nine quiz questions, we gather a series of citations from public records that relate to that issue. For each citation, we assign an answer, which we label like A, B, C, D (we add the labels here for easy reference; they're not used on the AmericansElect.org website). Whichever answer has the most citations is assigned as the answer.
For each question, we list the citations and the corresponding answer for each citation, and then a count indicating which overall answer is assigned. The "citation" is a brief headline, a summary of an underlying statement from the public record. By clicking on the citations, you can drill down to the excerpt from the public record. You can click further there to see its context: for voting records, how other members of Congress voted; for debate excerpts, what was the wider context of the debate; and so on.
The 10th question is the "weighting" for the first nine quiz questions. We weight the questions based on the relative number of citations for each question. If a candidate talks a lot about a particular issue, we include more citations on that issue; and hence that question gets weighted more. When gathering citations, we attempt to cover politicians without duplicating statements, but adding citations in different contexts -- our goal is to be thorough but not exhaustive. The result is that we infer each candidate's relative importance of an issue by how often that candidates talks about the issue.
ii. How we associate citations to answers
We use a "framework for analysis" to associate each citation to an answer choice. For each possible quiz answer, we list keywords that we associate with that answer choice. If a politician talks about the issue using the keyword listed below, we assign that answer (assuming the rest of the citation hints at that same answer -- keywords might also be used negatively!). Our framework also include a list of what NOT to include. SOmetimes that's because those issues muddy the waters, or sometimes it's because they're in other quiz questions (or other potential quiz questions that we may add later). Following is our framework, organized by the AmericansElect quiz questions and answer choices (in italics).
- When you think about the US budget deficit, which of the following solutions is closest to your opinion?
A. Cutting existing programs
- Cut government growth
- pay off debt
- reduce taxes in context of budget
B. More spending cuts than tax increases (mix of both solutions)
- Balanced Budget Amendment
C. More tax increases than spending cuts (mix of both solutions) D. Raising Taxes
- Tax increases to reduce budget deficit
- New spending programs
NOT:
- Avoids tax plans unless they talk about budget
- Avoids specific spending items (rather than overall budget)
- When you think about America’s energy needs, which of the following solutions comes closest to your opinion?
A. Strong investment in renewable energy like wind and solar
- Support Kyoto Protocol
- Support Cap-and-trade
- 20-by-20 renewables
- green jobs
- carbon tax
B. More drilling than investment in renewables (mix of both solutions)
- "All of the Above"
- Comprehensive energy policy
- American energy
- clean coal
- domestic production (if it includes fossil fuels)
C. More investment in renewable than drilling (mix of both solutions)
- Energy efficiency
- end oil addiction (if done by non-fossil fuels)
D. Strong focus on offshore drilling and allowing drilling in federal lands including wildlife reserves
- Open ANWR (Alaska National Wildlife Reserve)
- Open OCS (Outer Contintental Shelf)
- more drilling
- more refineries
- more pipelines
- questioning global warming
- Don't support Cap-and-trade, or call it cap-and-tax
- No EPA regulation of CO2 (carbon dioxide)
- No EPA regulation of GHGs (greenhouse gases)
NOT:
- Avoids nuclear power
- avoids deregulation
- avoid windfall profit issues
- avoids ethanol/biofuel issues (if in context of agriculture).
- When you think about healthcare reform in the United States, which of the following solutions is closest to your opinion?
A. The Government should be the sole provider of healthcare insurance
- Healthcare is a right
- universal coverage
- ObamaCare as a good start
- nationalized healthcare
- single-payer
- Medicaid/Medicare for all
B. The Government should have a major role in providing healthcare insurance
- Support ObamaCare
- support HillaryCare
- support RomneyCare
- expand Medicare/Medicaid
- expand SCHIP
- focus on prevention
C. The Government should have a limited role in providing healthcare insurance
- Oppose HillaryCare
- Oppose RomneyCare
- support insurance reform
- support options & opt-outs
D. Only private companies should provide healthcare insurance
- Repeal ObamaCare
- ObamaCare is unconstitutional
- market solutions
- healthcare competition
- tort reform
- MSAs (Medical Savings Accounts)
- HSAs (Health Savings Accounts)
NOT:
- Avoid abortion issues
- avoid Medicare/Medicaid benefit issues (unless related to ObamaCare or funding)
- avoid prescription re-importation
- avoid AIDS/HIV/R&D
- avoid tobacco settlement.
- When you think about illegal immigration, which of the following solutions come closest to your opinion?
A. All illegal immigrants should be able to stay in the US legally
- Amnesty
- path to citizenship
- DREAM Act
- pro-diversity
B. Most illegal immigrants should be able to stay in the US, with some exceptions
- Comprehensive reform
- earned citizenship
C. Most illegal immigrants should be deported, with some exceptions
- Enforce existing law
- guest workers
- more H1B visas
- feds fail to enforce
- reduce benefits to aliens
D. All illegal immigrants should be deported
- No amnesty
- no path to citizenship
- seal the border
- build a fence
- Using the term "illegals"
- Official English
NOT:
- Avoid state issues unless they have federal implications.
- Avoid legal immigration unless in context of ilegal immigration
- When you think about the US pursuing its interests abroad, which of the following is closest to your opinion?
A. The US should always act in its own interest regardless of what other countries think
- Favors unilateralism
- pre-emptive war OK
- Defund or leave the UN
- Islamist jihadists hate our way of life
- Strong support of Iraq war/Kuwait war/keeping troops in Iraq
- Demand that Iran/North Korea/Pakistan give up nukes
B. The US should rarely listen to other countries
- Restrict the UN
- restrict international organizations
- Reduce foreign aid
- foreign aid contingent on meeting US rules
- restrict free trade for US interests
C. The US should listen to other countries more often than not
- Increase foreign aid
- address causes of terrorism
- free trade as foreign policy
D. The US should always listen to other countries before pursuing its own interests
- Favors multilateralism
- diplomacy first
- no pre-emptive war
- Defer to the UN
- Strong opposition to Iraq war
- immediate withdrawal of troops in Iraq
- Iran/North Korea/Pakistan have a right to nukes
NOT:
- Avoid general theory & principles (that's question 9)
- avoid free trade issues (that's question 9) except as a foreign aid issue
- Avoid Afghan war & Libyan war
- avoid Arab-Israeli conflict.
- When you think about education in the US, which of the following is closest to your opinion?
A. School curriculums should be set entirely at a local school board level
- Vouchers
- funding portability
- school choice including private schools
- home-schooling
- eliminate Dept. of Education
- federal government out of education
B. School curriculums should be set more by local school boards than at a national level
- Local priority
- decrease bureacracy
- charter schools
- block grants
- restrict teachers' unions
- support NCLB (No Child Left Behind)
C. School curriculums should be set more by national standards than at a local level
- National testing standards
- increased funding (general call for money)
- oppose NCLB (No Child Left Behind)
D. School curriculums should be set entirely at a national standardized level
- Maintain our public schools
- don't drain resources
- more resources for teachers
- ensure educational standards
NOT:
- Avoid merit pay issues
- avoid college loan issues
- avoid teachers' union issues
- avoid school prayer
- avoid evolution debate.
- When you think about the rights of same-sex couples, which of the following is closest to your personal opinion?
A. Same-sex couples should not be allowed to marry or form any kind of civil union
- DOMA (Defense of Marriage Act)
- Marriage Amendment
- traditional marriage
- one-man-one-woman
- don't let judges decide
B. Same-sex couples should be allowed to form civil unions, but not to marry in the traditional sense
- Civil unions
- full equal rights (without full equality)
C. Same-sex couples should be allowed to marry legally, with all the same rights as traditional marriages
- Right to marry
- full equality
- leave marriage to the states
NOT:
- Avoids mixed Civil Rights issues;
- avoid mixed defense issues (Don't-ASk-Don't-Tell)
- avoid mixed crime issues.
- Which of the following statements comes closest to your personal view?
A. Natural resources exist for the benefit of humanity
- "Wise Use"
- delist endangered species
- privatize parks
- oppose "takings"/eminent domain
- amend the CAA/CWA
- terminate EPA regulation
B. Natural resources exist for the benefit of humanity, but should be somewhat protected
- Balanced use
- environment vs. economy
- Reduce EPA regulation
- pro-hunting on public lands
- cost-benefit analysis
- forest road-building
- access to public lands
- focus on highways/roads
- state/local/non-federal primacy
C. Natural resources should be mostly protected, but also exist for the benefit of humanity
- Funding to improve infrastructure/waterways/mass transit
- environmental justice
- conservation efforts
- pollution control
- "Polluter Pays Principle"
- "pollution credits"
- Clean Water/Clean Air Acts
- recycling
D. Natural resources exist on their own and should be completely protected
- Open space initiatives
- support endangered species listings
- sustainable practices
- Stewardship
- "Smart Growth"
- land trusts/conservation easements
- brownfield development
- animal rights
NOT:
- Avoid energy issues (those are in question 2)
- focus on non-energy economic value or the intrinsic value of the environment.
- avoid natural disaster issues
- avoid gun rights issues unless in the context of hunting on public lands.
- Which of the following comes closest to your personal opinion?
A. To make this country great, we should return to the examples and values of our forefathers
- Founding Principles
- citing Founding Fathers
- citing original Constitution
- remove 10th amendment/14th amendment
- "no foreign entanglements" (as a constitutional principle rather than as unilateralism).
- Free trade if an issue of principle rather than a specific trade agreement
B. This country is already great, we shouldn't change a thing
- Maintain our sovereignty
- Support Our Troops
- American First
- Love it or Leave it
- America is unique/light on the hill/an idea/shining city
- American exceptionalism
- American armed forces second to none
- peace through strength
- Increase defense spending
- restrict free trade (if on principle rather than a specific trade agreement)
C. To make this country great, we should keep building and adapting for the future
- No Pax Americana
- No American empire
- end military adventurism
- don't police the world
- America's long-term future
- close all/most US bases abroad
- reduce American armed forces
- Decrease defense spending
- free trade benefits both sides (if on principle rather than a specific trade agreement)
NOT:
- Avoid specific free trade as business issues (except when they're about trade principles)
- avoid foreign aid issues (that's question 5)
- avoid Iraq war as military issue (that's question 5), unless it's about more general empire/control/non-empire issues
- avoid military spending as monetary issue but focus on it as a foreign policy issue
- avoid avoid Missile Defense/SDI/Star Wars
- avoid veterans' issues.
iii. How we gather citations
All citations appear on the OnTheIssues.org website; you can see the details of each citation by clicking on the AmericansElect links. OnTheIssues gathers citations from three categories of sources:
- Quotes
Individual excerpts that apply only to one politician at a time. Sources:
- Candidate debates
- book excerpts
- Incumbent press releases
- campaign websites
- Our database includes over 50,000 individual quotes
- Notes
Excerpts from signed documents that apply to the entire group of signatories. Sources:
- Bill sponsorships
- Letters to Congress or the President
- Group ratings (made by non-profit organizations or political groups)
- Court rulings (for Supreme Court)
- Quotes take priority over notes, since quotes are individual while notes are collective
- Our database includes over 65,000 individual note signatories from 800 different notes.
- Votes
Excerpts from voting records in Congress, where each member votes Yea or Nay. Sources:
- House votes
- Senate votes
- Abstentions, absences, "present" votes do not count
- Quotes and notes take priority over votes, since votes require no active involvement other than presence.
- Our database includes over 120,000 individual voting records from 650 different Congressional votes.
That's an overview of how it works. We've been at it since 1999. We hope you use our website often.
Sincerely,
Jesse Gordon, editor-in-chief, OnTheIssues.org
jesse@OnTheIssues.org
617-320-6989
|
