## Start here if you are making your first questionnaire Materials used in Week 7 of Communication Research Methods II, a graduate course at Hallym University Media School, on October 13, 2026. Starting with “I am good at using AI,” we change what the item asks and how respondents answer. Activities with a correct answer are distinguished from open exploration of response formats. This reading edition includes all 43 slides and the explanations and hints revealed by clicking. Open the interactive slides to try the controls. Use the left and right arrow keys to change slides, and the mouse to select options and operate response tools. These are practice items. They are neither validated scales nor tests of actual ability. Before research use, examine existing instruments and pretest respondents’ understanding. ## 01\. What score should I give this question? A methods laboratory for changing survey items and response tools Choose a format, answer it yourself, and ask what the resulting number means. A/B/C selects a tool or item. Enter the actual response using the control that appears. ## 02\. What if you like AI but lack confidence in creating materials? Fictional instructor Minseo likes AI but has little confidence in making a draft. I like generative AI and can make good drafts of teaching materials. Response exploration · No correct answer · Compare scores of 1, 3, and 5 Five-point agreement scale · 1 Strongly disagree / 2 Disagree / 3 Neither agree nor disagree / 4 Agree / 5 Strongly agree ### A. A high score (5) Response exploration · No correct answer “If you mean how much I like it, I would give it a 5.” Answering based on liking could be mistaken for high confidence in creating materials. Ask about liking and confidence separately. ### B. A low score (1) Response exploration · No correct answer “Making a draft is still difficult for me.” Answering based on confidence could be mistaken for disliking AI. First define what a single score is supposed to mean. ### C. The middle score (3) Response exploration · No correct answer “I do like it... I will choose the middle.” A 3 could mean that both judgments are moderate, or that the respondent compromised between two different judgments. The respondent is not wrong. The question combines two judgments. Conclusion: the score alone cannot separate liking from confidence. After distinguishing the concepts, we will split the item in the double-barreled-question laboratory. ## 03\. What does “I like it” measure? I enjoy using generative AI. Classification question · One correct answer · A/B/C are classroom answers Five-point agreement scale · 1 Strongly disagree / 2 Disagree / 3 Neither agree nor disagree / 4 Agree / 5 Strongly agree ### A. Liking Correct “I like using AI.” This asks for an evaluation of liking or disliking. Interpret this response as liking AI use. It is not an ability score. ### B. Frequency of use Incorrect “I like it, but I have not used it at all this week.” Liking and actual frequency of use can differ. To ask about frequency, specify the time period and unit of use. ### C. Confidence in creating materials Incorrect “I may like it without feeling confident about making materials.” Enjoyment alone does not tell us whether someone feels able to complete a task. Ask about a specific task the respondent believes they can perform. ## 04\. What does “how many of the past seven days” measure? On how many of the past seven days did you use generative AI to prepare teaching materials? Classification question · One correct answer · A/B/C are classroom answers Days of use · Answer with an integer from 0 to 7 · 0 = no use, 7 = every day; this is not an agreement scale ### A. Liking Incorrect “I used it for work even though I do not like it.” Days of use are not the same as liking. Ask about liking in a separate evaluative item. ### B. Frequency of use Correct “I used it on three of the seven days.” This measures use behavior with a defined period and counting unit. It means three days of use, not three points of agreement or ability. ### C. Confidence in creating materials Incorrect “I use it every day but still get stuck.” Frequent use alone does not establish confidence in creating materials. Separately ask how strongly the respondent currently believes they can perform a specific task. ## 05\. What does “I can make it” measure? I can ask generative AI to produce a draft explanation suited to the topic of my class. Classification question · One correct answer · A/B/C are classroom answers Confidence from 0 to 10 · Eleven integer options · 0 Cannot do it at all / 5 Moderately certain / 10 Highly certain I can do it ### A. Actual production ability Incorrect “I thought I could do it, but it was difficult when I tried.” The survey collected the respondent’s belief. It did not observe actual performance. Assessing ability requires a performance task and evaluation criteria. ### B. Plans for future use Incorrect “I can do it, but I will not use it in my next class.” Believing that one can do something differs from planning to do it. Ask about intention separately, for example, “I intend to use it in the future.” ### C. Confidence in creating materials Correct “I believe I can accomplish this task.” This asks about belief in one’s ability to perform a specific task: self-efficacy. For example, 8 means high confidence, not an actual ability score of 80. ## 06\. An item needs a response scale. The item is the question; the response scale is how it is answered. Adding numbers does not create a validated scale. * Liking — Agreement from 1 to 5 — 1 Strongly disagree ↔ 5 Strongly agree * Frequency of use — 0 to 7 days — Number of days used in the past seven days * Production confidence — 0 to 10 — 0 Cannot do it at all ↔ 10 Highly certain I can do it ## 07\. One coffee, but three drinks were ordered? Choose one drink to order now. Start by comparing input formats for selecting one category. ### A. Radio buttons Single choice · Numbers are category codes Americano / Latte / Tea A choice that fits the purpose Selecting one clears the previous selection. That matches a question asking for one drink. Useful for a short list that is immediately visible. ### B. Checkboxes Multiple choices · More than one response allowed Americano / Latte / Tea Does not fit the current purpose You can select all three at once. The setting conflicts with the instruction to choose one drink. Use single choice, or change the question to “Select all drinks you would like.” ### C. Dropdown Single choice · Open the list to select Americano / Latte / Tea A choice that fits the purpose It records the same categories as radio buttons, but respondents must open the list to see them. Useful for a long list. Radio buttons also work for a short list. ### Hints to consider one at a time * Radio buttons, checkboxes, and dropdowns are interface formats. They do not by themselves define a level of measurement. * Americano = 1 and latte = 2 does not mean that a latte is twice an Americano. * First align the format with “choose one” or “select all that apply.” ## 08\. What if I use both ChatGPT and Gemini? Select all generative AI services you used in the past seven days. Services used · Use or nonuse of each service ### A. Radio buttons Choose only one ChatGPT / Gemini / Claude / None used Does not fit the current purpose Even someone who used two services must keep only one. Some use is lost. Use checkboxes that allow all applicable services. ### B. Checkboxes Multiple choices · “None used” is exclusive ChatGPT / Gemini / Claude / None used A choice that fits the purpose Each service’s use can be recorded separately. “None used” cannot be selected with another option. In a real survey, consider an Other option with a write-in field. ### C. Usage slider 0 No use at all ↔ 10 Very heavy use Input range: 0–10 / Step 1 Does not fit the current purpose You can move a measure of amount, but it does not identify which services were used. Ask about service type and amount of use in separate items. ### Hints to consider one at a time * Did you write “all” but allow only one selection? * A 0/1 code for each service indicates use or nonuse, not preference ranking. * Selecting several services together with “None used” creates a contradiction. ## 09\. What is 1.7 between an iPhone and a Galaxy? What operating system does the phone you mainly use have? One operating-system category · Choose input format A/B/C and try answering ### A. Slider 1 iOS ↔ 2 Android ↔ 3 Other · Decimals allowed Input range: 1–3 / Step 0.1 Does not fit the current purpose Which operating system is 1.7? This creates a continuous quantity between category codes. Record different kinds as clearly defined categories. ### B. XY grid X 1–9 / Y 1–9 · Neither axis has a defined meaning X=1 X=9 Y=1 Y=9 Does not fit the current purpose You can place a point, but neither axis has been defined. The coordinates do not answer the question. Do not use XY unless the purpose is to measure two dimensions. ### C. Radio buttons Single category · Separate not-applicable option iOS / Android / Other / I do not use a mobile phone A choice that fits the purpose This records an operating-system category. The mean of its codes is not an average operating system. Define “mainly use” and include Other and nonuse categories. ### Hints to consider one at a time * Can the researcher interpret a response of 1.7? * Two coordinates do not automatically provide twice as much useful information. * C fits this purpose. That does not mean sliders are inherently bad. ## 10\. Eight days of use in the past seven days? On how many of the past seven days did you use AI to prepare for teaching? Days of use · Integer from 0 to 7 ### A. Number without an upper limit Enter a nonnegative number of days · No upper limit Input range: 0–No upper limit / Step 1 / Days Does not fit the current purpose It accepts eight days. The allowed input range conflicts with the question’s time period. Set a maximum of seven days and allow only integers. ### B. Slider from 0 to 7 days 0 to 7 days · One-day increments Input range: 0–7 / Step 1 / Days A choice that fits the purpose The period and unit match. Zero days is a valid response. Do not automatically save an untouched slider as zero days or a midpoint. ### C. Numeric input from 0 to 7 days 0 to 7 days · Integer input Input range: 0–7 / Step 1 / Days A choice that fits the purpose Typing a number can measure the same number of days. Explain out-of-range values and decimals, then request a valid entry. ### Hints to consider one at a time * Number of uses and number of days used are different variables. * Without zero days, nonusers have no valid answer. * Both sliders and numeric input can work. Purpose, range, and unit are what matter. ## 11\. Both use seven points. Are A and C opposites? How should we ask about attitudes toward the Chinese Communist Party? A: agreement with “dislike” / C: direct liking–disliking rating · Fictional response ### A. Seven-point agreement with “dislike” “I dislike the Chinese Communist Party.” · 1 Strongly disagree ↔ 7 Strongly agree 1 Strongly disagree / 2 Disagree / 3 Somewhat disagree / 4 Neither / 5 Somewhat agree / 6 Agree / 7 Strongly agree A choice whose conditions need checking A score of 7 in A means strongly agreeing with “I dislike it.” A score of 1 means disagreeing with that statement, not necessarily liking it very much. C also uses seven numbers, but the question and scoring direction differ. You cannot directly convert A’s 1 into C’s 7. ### B. Yes / No Dislike / Do not dislike · Two categories Dislike / Do not dislike A choice whose conditions need checking It is easy to choose, but the difference between slight and strong dislike disappears. Not disliking is not necessarily liking. Decide whether you need only the direction of the attitude or its strength as well. ### C. Direct dislike ↔ like rating “How much do you dislike or like the Chinese Communist Party?” · 1 Dislike very much ↔ 7 Like very much 1 Dislike very much / 2 Dislike / 3 Dislike somewhat / 4 Neither like nor dislike / 5 Like somewhat / 6 Like / 7 Like very much / Do not know / Prefer not to answer A choice whose conditions need checking In C, 1 means “dislike very much” and 7 means “like very much.” The target is the same as in A, but it asks directly about both poles instead of agreement with a negative statement. C is not always the correct choice. Match wording to the purpose, and separate not knowing or refusing from neutrality. ### Hints to consider one at a time * A’s 7 = strongly agreeing with “dislike.” C’s 7 = liking very much. The numbers match, but the directions are opposite. * Is A’s “do not dislike” the same as C’s “like”? Indifference or neutrality is also possible. Reverse coding alone is not evidence of equivalence. * You cannot assume that nobody will choose the middle. Clustering at the extremes alone does not make seven points wrong or two points correct. ## 12\. A score of 4: neutral, unsure, or unwilling to answer? How much do you agree with “Policy Z,” which you have just heard about for the first time? Answer as a fictional respondent who has no information about it. ### A. Forced seven-point choice 1 Strongly disagree ↔ 7 Strongly agree 1 Strongly disagree / 2 Disagree / 3 Somewhat disagree / 4 Neither / 5 Somewhat agree / 6 Agree / 7 Strongly agree Does not fit the current purpose Even uninformed respondents must choose a number. If uncertainty is mixed into 4, that score is hard to interpret as neutral. Provide distinct neutral and do-not-know categories. ### B. Separate do-not-know option Agreement from 1 to 7 + Do not know 1 Strongly disagree / 2 / 3 / 4 Neutral / 5 / 6 / 7 Strongly agree / Do not know A choice that fits the purpose “Do not know” is neither agreement level 4 nor 0. Define a separate-category or missing-data rule for analysis. Also consider whether a neutral description of the policy should be provided first. ### C. Remove the midpoint 1 Strongly oppose / 2 Oppose / 3 Support / 4 Strongly support 1 Strongly oppose / 2 Oppose / 3 Support / 4 Strongly support A choice whose conditions need checking Does removing the middle create an opinion? Lack of information is now recorded as support or opposition. First justify whether forced support–opposition fits the research purpose. ### Hints to consider one at a time * Neutrality is an attitudinal position; not knowing may mean insufficient information to judge. * Refusal and not applicable each have different meanings too. * Including a midpoint depends on the research purpose and respondent review. There is no rule to always include or always exclude it. ## 13\. Can one sex/gender question give us two kinds of information? What is your sex/gender? Sex recorded at birth and current gender identity · Measure separately when needed ### A. Male / Female Nominal categories · Two choices Male / Female Does not fit the current purpose It is unclear whether the question asks about current identity or the record at birth. People who do not describe themselves using those two categories may struggle to answer. First define the concept needed, then review options for the target population. ### B. Add “biological” “What is your biological sex?” · Male / Female Male / Female A choice whose conditions need checking That word alone does not clarify whether you mean the record at birth or particular physical characteristics. It also does not establish current gender identity. If the study needs the record at birth, ask specifically about that variable, such as “sex recorded at birth.” ### C. Ask about identity separately Separate item: “How do you currently describe your gender?” Man / Woman / Nonbinary / I use another term / Prefer not to answer A choice whose conditions need checking C is not a universal replacement for B. When research needs both the record at birth and current identity, add a separate item like C. Store the two answers as different variables. This does not mean collecting information that is unnecessary for the study. ### Hints to consider one at a time * Can B also reveal current gender identity? One question cannot readily stand in for two different kinds of information. * If both are needed, ask separately about (1) sex recorded at birth and (2) current gender identity. Simply choosing C does not complete the design. * Sexual orientation is another distinct variable. Collect only what the study needs, and pretest wording and options with respondents. ## 14\. A monthly income of KRW 3 million fits two boxes? What was your personal income before tax last month? Distinguishing income bands · Fictional person: KRW 3 million per month ### A. Overlapping bands 0–300 / 300–500 / 500 or more · Units of KRW 10,000 KRW 0–3 million / KRW 3–5 million / KRW 5 million or more Does not fit the current purpose KRW 3 million fits both the first and second bands. The boundary also overlaps at KRW 5 million. Clarify boundaries with “less than” on one side and “at least” on the other. ### B. Nonoverlapping bands 0 to under 300 / 300 to under 500 / 500 or more KRW 0 to under 3 million / KRW 3 million to under 5 million / KRW 5 million or more / Prefer not to answer A choice that fits the purpose KRW 3 million fits only the second band. This example assumes nonnegative income. In actual research, also define how to handle losses, no income, uncertainty, and refusal. ### C. Enter a number directly Units of KRW 10,000 · Zero or more Input range: 0–No upper limit / Step 1 / KRW 10,000 A choice whose conditions need checking This allows greater precision but may increase recall and disclosure burdens. Specify the necessary precision, definition of income, and refusal option. ### Hints to consider one at a time * Categories must not overlap, so one response cannot fit two boxes. * Do not leave gaps either. An Other option cannot solve every design problem. * State whether you mean personal or household, before or after tax, monthly or annual, and earned or total income. ## 15\. How should we repair this item? I like generative AI and can use AI to draft teaching materials. Classification question · One correct answer · A/B/C are classroom answers Five-point agreement scale · 1 Strongly disagree / 2 Disagree / 3 Neither agree nor disagree / 4 Agree / 5 Strongly agree ### A. Use it as it is Incorrect “I would give liking a 5, but being able to create a draft a 1.” This tries to collect two judgments in one response. Split liking and production confidence into separate items. ### B. Shorten it to “I am good at using AI” Incorrect “Do you mean I use it often, or that I produce good results?” It is shorter, but “good at” still means different things to different people. Specify the task the respondent should be able to perform. ### C. Separate liking from production confidence Correct “I will answer separately about liking it and feeling able to create a draft.” Separate responses let us distinguish the meaning of each score. For example: (1) I enjoy using AI. (2) I can use AI to draft teaching materials. Answer each separately. ## 16\. Same people, different score distributions. Change the basis for the five-point agreement response. The same 12 people remain. Five-point agreement scale · 1 Strongly disagree / 2 Disagree / 3 Neither agree nor disagree / 4 Agree / 5 Strongly agree Twelve fictional people: six with high liking and low confidence; three with low liking and high confidence; three moderate on both ### The same 12 fictional respondents | Basis for answering | 1 point | 2 points | Score 3 | Score 4 | Score 5 | | --- | --: | --: | --: | --: | --: | | First part: liking | 0 | 3 | 3 | 0 | 6 | | Second part: confidence | 6 | 0 | 3 | 0 | 3 | | Compromise at the midpoint | 0 | 0 | 12 | 0 | 0 | Fictional classroom responses—not predictions of real results. We keep the same 12 people and change only the response rule. The compromise scenario assumes everyone chooses 3; it is not an arithmetic-mean rule. The point is that the score alone does not reveal which judgment was answered. ## 17\. Split the question to reveal different judgments. (1) I enjoy using AI. (2) I can use AI to draft teaching materials. Five-point agreement scale · 1 Strongly disagree / 2 Disagree / 3 Neither agree nor disagree / 4 Agree / 5 Strongly agree Both items use agreement from 1 to 5 in this comparison. The final confidence items are designed separately on a 0–10 scale. The two scales are not converted. * Minseo type · 6 people — 5 — 1 * Junho type · 3 people — 2 — 5 * Yujin type · 3 people — 3 — 3 ## 18\. What can the graph alone tell us? Responses cluster at 3, the midpoint of a five-point agreement item. Can we determine why? Classification question · One correct answer · A/B/C are classroom answers Five-point agreement scale · 1 Strongly disagree / 2 Disagree / 3 Neither agree nor disagree / 4 Agree / 5 Strongly agree ### A. Everyone really is neutral Incorrect “Both judgments could be moderate.” That is one possible explanation, but it cannot be distinguished from compromising at the midpoint. Examine both the item and respondents’ reasons for answering. ### B. It must be a double-barreled question Incorrect “People can choose the middle for other reasons too.” The shape of a distribution cannot by itself establish an item defect. Check the meaning of the question and the response process. ### C. The distribution alone cannot establish the reason Correct “We should ask which part they were answering.” The same number can result from different judgments. Ask, “How did you understand this question?” ## 19\. All three matter. Must one come last? When choosing coffee, which matters more: price, taste, or accessibility? Compare each attribute’s importance with its relative priority. ### A. Ranking Rank 1 Most important → Rank 3 Less important · No ties Price / Taste / Accessibility A choice that fits the purpose A forced order reveals relative priority, but not how much more important first place is than second. Decide whether ties are allowed and whether priority is actually what you need. ### B. Rate each separately Each attribute: 1 Not at all important ↔ 5 Very important Price / Taste / Accessibility 1 / 2 / 3 / 4 / 5 A choice that fits the purpose All three can receive 5. Independent importance ratings need not differ. If priority is the goal, ask for ranking separately. Do not interpret ratings and ranks as the same values. ### C. Choose exactly one The single most important attribute Price / Taste / Accessibility A choice whose conditions need checking This identifies the top attribute but not the order of the remaining two. Use ranking for the full order, or single choice if only the top option is needed. ### Hints to consider one at a time * Ranks 1, 2, and 3 establish order. Equal intervals are not guaranteed. * With ratings, all three can matter. That alone is not an invalid response. * Ranking and rating answer different questions. ## 20\. The budget is 100, but the total is 180? Allocate 100 education-budget points among facilities, scholarships, and teaching support. Relative allocation of limited resources · Total 100 ### A. Allocate a total of 100 Each 0–100 points · Total 100 points Facilities / Scholarships / Teaching support A choice that fits the purpose Giving more to one item leaves fewer points for the others. This reveals allocation of limited resources. Check that the total is 100 and allow zero. These are not independent satisfaction ratings. ### B. Rate each from 0 to 100 Independent ratings · No restriction on the total Facilities / Scholarships / Teaching support 0 / 25 / 50 / 75 / 100 Does not fit the current purpose All can receive high scores, but the 100-point budget constraint disappears. Apply a total-sum constraint when asking about budget allocation. ### C. Rank only Ranks 1–3 Facilities / Scholarships / Teaching support Does not fit the current purpose Knowing that scholarships rank first does not tell us whether they receive 80 points or 34. Use a constant-sum format if allocation shares are needed. ### Hints to consider one at a time * “How good is it?” differs from “How much of a limited budget would you give it?” * The total constraint makes responses interdependent. Do not treat them as independent scales. * An incorrect total is an input-validation problem; deciding what to allocate is a measurement-design problem. ## 21\. Does putting items in a table make them one scale? Classroom evaluation: clean / quiet / easy to find Agreement with each attribute · One display, different content ### A. Matrix table Each row: 1 Strongly disagree ↔ 5 Strongly agree Clean / Quiet / Easy to find 1 / 2 / 3 / 4 / 5 A choice that fits the purpose One screen collects three separate judgments. Sharing a table does not automatically create one construct or justify an average score. Check what each row asks and the grounds for combining them. ### B. One slider for everything “Clean, quiet, and easy to find” · Agreement from 1 to 5 Input range: 1–5 / Step 1 Does not fit the current purpose What score fits a classroom that is clean but noisy? Three judgments are combined into one score. Separate judgments that respondents could answer differently. ### C. Check applicable attributes Select all applicable attributes Cleanliness / Quietness / Ease of finding A choice whose conditions need checking This identifies attributes considered present but loses degree of agreement. Reasons for leaving boxes unchecked may also be unclear. Use row-by-row ratings for degree, or clear binary questions if only presence is needed. ### Hints to consider one at a time * A matrix is a display arrangement. It does not guarantee scale validity. * Check whether choosing the same column in every row reflects real views or rapid completion. * Large tables create reading burden. Consider splitting rows on mobile screens. ## 22\. Does the AI’s tone feel cold or warm? How does the tone of the AI response you just saw feel? One bipolar impression · Semantic differential ### A. Seven-point bipolar scale 1 Very cold ↔ 4 Middle ↔ 7 Very warm Input range: 1–7 / Step 1 A choice that fits the purpose The endpoints let respondents locate an impression on a single axis. Check with respondents that the two adjectives actually form opposite ends for this target. ### B. Five stars 1–5 stars · No explanation of the stars’ meaning Does not fit the current purpose Stars are familiar, but it is unclear whether they indicate coldness, warmth, or satisfaction. Define the stars and endpoints, or present a semantic differential. ### C. Five-point agreement “This AI response sounds warm” · Agreement from 1 to 5 1 Strongly disagree / 2 Disagree / 3 Neither agree nor disagree / 4 Agree / 5 Strongly agree A choice that fits the purpose This measures agreement with a statement about warmth. Its wording differs from a direct comparison of bipolar adjectives. The score range and wording differ, so do not directly pool values from different formats. ### Hints to consider one at a time * A familiar star shape does not automatically communicate its meaning. * “Fast–accurate” is not a pair of opposites. Both can be high. * Both semantic differentials and agreement ratings can work, but do not assume they are the same item or score. ## 23\. One star for delivery: bad taste or late arrival? Overall, how satisfied were you with this order? Distinguish overall satisfaction from reasons for dissatisfaction. ### A. Overall satisfaction stars 1 Very dissatisfied ↔ 5 Very satisfied A choice that fits the purpose This quickly records overall satisfaction. One star alone cannot identify whether taste or delivery caused dissatisfaction. Use it for overall satisfaction, and add follow-up items to investigate reasons. ### B. Attribute satisfaction matrix Each row: 1 Very dissatisfied ↔ 5 Very satisfied Food taste / Delivery time / Price 1 / 2 / 3 / 4 / 5 A choice that fits the purpose Separate taste, delivery, and price ratings help identify the source of dissatisfaction. Whether their simple average represents overall satisfaction requires separate examination. ### C. Likelihood to recommend, 0–10 0 Not at all likely to recommend ↔ 10 Extremely likely to recommend Input range: 0–10 / Step 1 A choice whose conditions need checking Satisfaction and recommendation may be related, but they are different questions. Someone may be satisfied yet reluctant to recommend because of the price. Distinguish overall satisfaction from recommendation according to the research purpose. ### Hints to consider one at a time * One number can provide an overall evaluation without revealing specific reasons. * Stars are a display format; satisfaction is the content being measured. * Overall and attribute ratings serve different roles. ## 24\. Both are 9. Are they the same score? How likely are you to recommend this AI tool to a colleague? Likelihood to recommend · Identical numbers can represent different constructs ### A. Recommendation from 0 to 10 0 Not at all likely to recommend ↔ 10 Extremely likely to recommend Input range: 0–10 / Step 1 A choice that fits the purpose A 9 indicates a high likelihood of recommending. In the NPS system, responses of 9–10 are classified as promoters. NPS is an aggregate: the percentage of promoters minus the percentage of detractors. An individual’s score of 9 is not itself NPS. ### B. Production confidence from 0 to 10 0 Cannot create it at all ↔ 10 Highly certain I can create it Input range: 0–10 / Step 1 Does not fit the current purpose A 9 here represents production confidence. The scale does not match a question about recommendation. Align the anchors with likelihood of recommending. ### C. Satisfaction from 1 to 5 1 Very dissatisfied ↔ 5 Very satisfied Does not fit the current purpose A satisfaction scale does not directly answer a question about recommendation likelihood. Do not substitute recommendation, satisfaction, and confidence just because each produces a number. ### Hints to consider one at a time * 0–10 is a numerical range, not a construct. * Do not apply the NPS categories of 0–6, 7–8, and 9–10 to confidence items. * Today, focus on whether the question, scale, and interpretation match, rather than on calculation. ## 25\. Sleepy, but the class is interesting. Where do I click? Indicate interest and alertness in this class separately. Fictional state: high interest, low alertness · Two dimensions ### A. Single-line slider 1 Bored and sleepy ↔ 9 Interested and alert Input range: 1–9 / Step 1 Does not fit the current purpose Where does interested but sleepy belong? The endpoints combine two judgments into one axis. Split interest and alertness into two axes or two items. ### B. XY response grid X Interest 1–9 / Y Alertness 1–9 · Two numbers No interest 1 Very interested 9 Sleepy 1 Alert 9 A choice that fits the purpose The lower-right area represents high interest and low alertness. One point contains two response values. This is a custom classroom example, not a reproduction of the validated Affect Grid. ### C. Two-row ratings Interest and alertness each from 1 to 9 · Select 1, 3, 5, 7, or 9 Interest: none → high / Alertness: sleepy → alert 1 / 3 / 5 / 7 / 9 A choice that fits the purpose The dimensions can be answered separately. It may be easier to explain than XY, but this example offers only five possible values. Two axes and two items are different presentation formats. Compare comprehension burden and response resolution. ### Hints to consider one at a time * Pleasant–unpleasant and high–low alertness are also different dimensions. * The coordinate (8,2) is not a score of 82. It contains two values: X = 8 and Y = 2. * An XY response grid is different from a heat map aggregating many people’s clicks. ## 26\. “Half-price tuition!” Where did you click first? Click the one location in this fictional advertisement that you would first like to examine. Selected screen location · Not eye tracking ### A. Select an image location Horizontal X and vertical Y each from 0 to 100% · One click A choice that fits the purpose The chosen location is recorded as coordinates. Aggregating respondents’ locations can produce a density plot. The single point shown here is one selection. It does not measure actual gaze or the causes of attention. ### B. Interest slider 0 Not at all interested ↔ 10 Very interested Input range: 0–10 / Step 1 Does not fit the current purpose This records degree of interest but not the location clicked in the advertisement. Use coordinates for location and ratings for degree of interest. ### C. Radio buttons for regions Headline / Conditions / Apply button · Single choice Headline / Detailed conditions / Apply button A choice that fits the purpose This identifies the region selected but loses the exact coordinates within it. This format also works if only the region matters to the study. ### Hints to consider one at a time * Qualtrics Heat Map collects click locations on an image. The resulting heat map is an aggregate display of multiple clicks. * Not clicking somewhere does not prove it was unseen. These are not eye-tracking data. * This custom example demonstrates the principle. It is neither the actual product interface nor an actual response distribution. ## 27\. “I have never lied in my entire life.” Really? I have never told a lie, not even once in my entire life. Extreme wording and the possibility of socially desirable responses ### A. Five-point agreement 1 Strongly disagree ↔ 5 Strongly agree 1 Strongly disagree / 2 Disagree / 3 Neither agree nor disagree / 4 Agree / 5 Strongly agree A choice whose conditions need checking “Entire life” and “not even once” impose substantial recall demands and may sound like a moral judgment. Narrow the time period and behavior, and examine the response context. ### B. Yes / No Whether the event has ever occurred Never / At least once A choice whose conditions need checking Reducing the choices to two leaves the lifetime recall and evaluative burden intact. Changing the input tool alone cannot fix the question. ### C. Narrow the behavior and period In the past seven days, gave an untrue reason for being late · Yes/No Yes / No / No such situation occurred / Prefer not to answer A choice that fits the purpose The event and period are specific, but the limitations of self-reporting sensitive behavior remain. Explain confidentiality and why collection is necessary, and allow refusal. ### Hints to consider one at a time * An extreme response cannot automatically be classified as a lie. * Distinguish the possibility of impression-management responses from actual attitudes. * A good widget does not automatically eliminate recall error or social desirability. ## 28\. “The professor’s class is easy and fun, right?” The professor’s class is easy and fun, right? Find both leading wording and a double-barreled question. ### A. A single star rating 1 Not at all ↔ 5 Very much Does not fit the current purpose It could be easy but boring, or difficult but interesting. “Right?” also signals an expected agreement. Use neutral wording and separate difficulty from interest. ### B. Two rows: difficulty and interest Each row: 1 Low ↔ 5 High Difficulty: easy → difficult / Interest: none → high 1 / 2 / 3 / 4 / 5 A choice that fits the purpose A difficulty score of 5 means difficult; an interest score of 5 means very interesting. Difficulty alone does not mean a negative evaluation. State what a high score means in each row, and do not simply add the two scores. ### C. Yes / No Easy and fun · Yes/No Yes / No Does not fit the current purpose Changing the tool leaves two judgments combined in one answer. Repair the wording first, then choose a response format. ### Hints to consider one at a time * Does “easy” necessarily mean a good class? * Removing “right?” and separating the two judgments are two distinct repairs. * Reverse coding one item does not automatically turn different concepts into one scale. ## 29\. I did not touch the slider, but it says 50? How satisfied are you with this service? Distinguish the initial display from an actual response. ### A. Slider from 0 to 100 0 Very dissatisfied ↔ 100 Very satisfied · One-point increments Input range: 0–100 / Step 1 A choice whose conditions need checking The handle’s initial position is a display convenience. This demonstration also shows “Unanswered” until it is operated. Check whether the default value is automatically saved as an actual response. ### B. Five-point radio buttons 1 Very dissatisfied ↔ 5 Very satisfied 1 Very dissatisfied / 2 Dissatisfied / 3 Neither satisfied nor dissatisfied / 4 Satisfied / 5 Very satisfied A choice that fits the purpose The unanswered state is clear before selection. Also consider whether five categories capture the distinctions needed. Do not assume that more response categories always mean more accuracy. ### C. Number from 0 to 100 0 Very dissatisfied ↔ 100 Very satisfied · Integer Input range: 0–100 / Step 1 / Points A choice whose conditions need checking Requiring an exact number does not establish that respondents can meaningfully distinguish 63 from 64. Match the precision required to the precision respondents can actually judge. ### Hints to consider one at a time * Displaying 50 is different from a person choosing 50. * Someone may genuinely judge their answer to be 50 without moving the slider. Consider an explicit way to confirm the midpoint. * There is no rule that a 100-point scale is always better than a seven-point scale. ## 30\. Satisfaction with a service I have never used? How satisfied are you with this AI service? Fictional respondent: has never used this service ### A. Mandatory 1–5 response 1 Very dissatisfied ↔ 5 Very satisfied 1 Very dissatisfied / 2 Dissatisfied / 3 Neither satisfied nor dissatisfied / 4 Satisfied / 5 Very satisfied Does not fit the current purpose Which score should a nonuser choose? Forcing them into 3 mixes neutral satisfaction with no experience. Check usage experience first, or provide a not-applicable category. ### B. Add not applicable Satisfaction from 1 to 5 + No experience using it 1 Very dissatisfied / 2 Dissatisfied / 3 Neither satisfied nor dissatisfied / 4 Satisfied / 5 Very satisfied / No experience using it A choice that fits the purpose Nonuse can be distinguished from satisfaction. “No experience” is not zero satisfaction. Define separate handling rules for not applicable, do not know, and refusal. ### C. Experience screening question Have you ever used it? · Yes/No Yes / No A choice that fits the purpose If the answer is no, survey routing can skip the satisfaction question. This classroom widget demonstrates the screening response. In a survey tool, configure the conditions for displaying the next question. ### Hints to consider one at a time * Not applicable is neither the midpoint nor the lowest satisfaction. * Forcing a number from someone unable to answer creates a value for analysis but obscures its meaning. * Design the item, categories, question order, and branching together. ## 31\. What do we want to know today? Research aim: understand media instructors’ confidence in using AI to make drafts suited to their classes. Choose the appropriate measurement target. Classification question · One correct answer · A/B/C are classroom answers ### A. Every positive trait related to AI Incorrect “Liking, ethics, and ability—all of them?” The scope is too broad to decide which items to retain. Narrow the population, situation, specific task, and judgment. ### B. Confidence in using AI to draft teaching materials Correct “So it means believing I can create a draft while preparing a class.” This specifies who believes they can do what, and in which situation. Collect current production confidence from 0 to 10. Frequency of use is a separate variable. ### C. Frequency of AI use and quality of outputs Incorrect “Using it a lot and producing good work are different.” Combining judgments makes the score’s meaning ambiguous again. Treat frequency and output quality as separate research questions. ## 32\. Does ending with “I can” make every item the same? Common condition: asking generative AI to do something Which item fits today’s definition of confidence in drafting explanations and examples? Classification question · One correct answer · A/B/C are classroom answers Confidence from 0 to 10 · Eleven integer options · 0 Cannot do it at all / 5 Moderately certain / 10 Highly certain I can do it ### A. I can use AI to color-correct video Incorrect “Are we talking about video editing?” This concerns confidence, but lies outside today’s scope of drafting explanations and examples. Exclude it from this study, and treat it as a separate concept if needed. ### B. I can use AI every day Incorrect “Does that mean I have enough time?” This may evoke access or ability to sustain use, rather than confidence in drafting materials. Specify a concrete production task. ### C. I can draft classroom examples suited to students’ level Correct “You mean draft examples for my class.” This asks about belief in performing a task included in the definition. Retain it as a practice candidate, then check whether actual respondents interpret it consistently. ## 33\. Keep one judgment in one item. Before: I like AI and feel confident about drafting explanatory materials. Classification question · One correct answer · A/B/C are classroom answers Before: agreement from 1 to 5 (1 Strongly disagree ↔ 5 Strongly agree) ### A. I feel positive and confident about using AI Incorrect Agreement from 1 to 5 “Are you asking about both my positive feelings and my confidence?” Changing the words leaves the two judgments in place. Remove liking and ask only about belief in being able to create the draft. ### B. I can ask AI to draft an explanation suited to my class topic Correct Confidence from 0 to 10 “I just need to answer whether I believe I can accomplish this.” One belief about a specific task remains. Answer this item with current confidence from 0 to 10. ### C. I like AI / I use AI often Incorrect Agreement from 1 to 5 for each of two items “I will answer about liking it and frequency of use.” The items are separated, but the production confidence we intended to measure is missing. Even after splitting an item, check its fit with the measurement purpose. ## 34\. “Good at” evokes different situations for different people. Before: I can use AI well and effectively. Classification question · One correct answer · A/B/C are classroom answers Common format for candidate items · Confidence from 0 to 10 · 0 Cannot do it at all ↔ 10 Highly certain I can do it ### A. I can use AI very effectively Incorrect “What level does ‘very’ mean?” Adding emphasis does not make the task more specific. State what kind of material is being created. ### B. I can use AI better than other instructors Incorrect “Which instructors should I compare myself with?” This requires a new comparison standard and becomes relative self-evaluation. Ask about belief in one’s own task performance without comparison to others. ### C. I can ask AI to draft an explanation suited to my class topic Correct “You mean a draft explanation, not a finished product.” The target and expected completion level are specific. Pretest whether respondents interpret even this wording in the same way. ## 35\. A plausible item—but is it our question? Read the item together with its response scale, and choose the one matching today’s measurement purpose. Which item is closest to confidence in creating drafts? Classification question · One correct answer · A/B/C are classroom answers ### A. If the first result does not fit, I can revise the request to obtain the draft I need Correct Confidence from 0 to 10 “It asks whether I believe I can try again when the first result is unsatisfactory.” This addresses a concrete difficulty in a drafting situation. Keep it as a candidate. Validity of the actual item set still requires separate examination. ### B. What percentage of AI drafts did you fact-check in the past seven days? Incorrect Percentage checked in the past seven days · 0–100% / No AI drafts used = Not applicable “I am counting how many of the drafts I used in the past week I checked.” This is a proportion of fact-checking behavior, not confidence in drafting. Treat it separately when studying verification behavior. ### C. I think using AI will increase satisfaction with my classes Incorrect Agreement from 1 to 5 “I am being asked to predict an effect of using AI.” This is an outcome expectation, not belief in one’s ability to perform the task. Separate outcome expectations from performance confidence. ## 36\. Choose three candidates for the same concept. Common condition: you ask generative AI. Select three candidate items for confidence in creating drafts. Item selection · Three correct choices · Candidate numbers are not survey scores Common confidence scale: 0 Cannot do it at all / 5 Moderately certain / 10 Highly certain I can do it. Do not sum the items yet. ### Candidate 1 I can draft an explanation suited to my class topic. This is belief in being able to perform the task of drafting explanatory materials. Confidence from 0 to 10 Retain ### Candidate 2 On how many of the past seven days did you use AI to prepare for teaching? This is frequency of use over a defined period, not production confidence. 0–7 days (days of use) Exclude ### Candidate 3 I can draft classroom examples suited to students’ level. This is belief in being able to perform the task of drafting classroom examples. Confidence from 0 to 10 Retain ### Candidate 4 I enjoy using generative AI. This concerns liking AI. Agreement from 1 to 5 Exclude ### Candidate 5 If the first result does not fit my teaching purpose, I can revise the request to obtain the draft I need. This concerns confidence when production involves a difficulty. Confidence from 0 to 10 Retain ### Candidate 6 I intend to use AI for all my teaching preparation in the future. This is an intention for future use. Agreement from 1 to 5 Exclude ## 37\. The response scale must also match confidence. “I can use AI to draft an explanation suited to my class topic.” Which response scale fits? Classification question · One correct answer · A/B/C are classroom answers ### A. Days of use: 0–7 days Incorrect “This makes me answer how often I use it.” The item asks about confidence, but the response scale asks about frequency. Align the scale with degree of confidence. ### B. Confidence: 0–10 (eleven integers) Correct “I choose how strongly I believe I can accomplish it now.” Current performance confidence matches the meaning of the scale. Choose an integer from 0 through 10. Including zero makes eleven response categories. ### C. Liking: 1 Dislike very much–5 Like very much Incorrect “This is a scale for how much I like AI.” A liking scale does not match the performance confidence asked about in the item. Have respondents answer about belief in their ability, rather than liking or disliking. ## 38\. Apply what you learned to new items. ### Check 1 “AI is useful and safe.” Is one agreement response from 1 to 5 sufficient? Classification question · One correct answer · A/B/C are classroom answers Five-point agreement scale · 1 Strongly disagree / 2 Disagree / 3 Neither agree nor disagree / 4 Agree / 5 Strongly agree ### A. Yes, because useful means safe Incorrect “It is useful, but I worry about personal information.” People may judge usefulness and safety differently. Separate the two judgments. ### B. Split them because they are different judgments Correct “I will answer usefulness and safety separately.” The key is whether each judgment could receive a different answer. Select the concepts needed for the research purpose. ### C. It is short, so there is no problem Incorrect “It is short but still asks two things.” Sentence length and number of judgments are different. Check whether the parts could be answered differently. ### Check 2 A respondent selects 9 on a 0–10 confidence item. Can we conclude that their actual ability is high? Classification question · One correct answer · A/B/C are classroom answers Confidence from 0 to 10 · Eleven integer options · 0 Cannot do it at all / 5 Moderately certain / 10 Highly certain I can do it ### A. No. Assess performance separately Correct “Confidence and actual results can differ.” The confidence survey did not directly measure performance. Check with an actual task and evaluation criteria. ### B. Yes. People know themselves best Incorrect “I might have overestimated my ability.” Self-report and performance are different kinds of data. Interpret the answer only as confidence. ### C. Three items establish actual ability Incorrect “Even with more questions, I was still answering about my beliefs.” Adding items does not automatically change the measurement target. Check what is being measured rather than relying on the number of items. ### Check 3 We have created three candidate confidence items scored from 0 to 10. What comes next in actual research? Classification question · One correct answer · A/B/C are classroom answers ### A. Calculate the average and validation is complete Incorrect “We do not yet know whether these can be combined.” The grounds for combining items require separate examination. Check the definition, existing instruments, response interpretation, and measurement evidence. ### B. Ask AI to approve them and we are done Incorrect “Respondents might read them differently.” AI evaluation does not replace respondent pretesting or measurement validation. Check original sources and actual response processes. ### C. Examine the original scale and respondents’ understanding Correct “I need to check whether it fits my population and situation.” Distinguish practice items from instruments intended for research. Document the items, scale, original population, and differences from your study. ## 39\. In research, start by examining suitable existing scales. Practice candidates → Compare with existing instruments → Check respondent understanding. Summing, averaging, reliability, and validity come later. * Read the original — Items, response categories, score meanings, and study population * Compare meanings — Does it fit my concept and situation? * Ask someone to read it — “What did you think this question was asking?” ## 40\. Good numbers begin with answerable questions. Next question: if frequent AI users also feel more confident, did use create that confidence? * Define one target — What are we measuring? * Ask one judgment at a time — What judgment are we collecting? * Check the interpretation — Does it match our intention? ## 41\. Moving together does not establish causation. Two high scores occurring together in one survey cannot establish causal direction or rule out other explanations. * Use → Confidence — Experience using AI may increase confidence. * Confidence → Use — Confident people may use it more frequently. * Training → Both — Training may increase both use and confidence. ## 42\. Sources and the scope of today’s examples * Babbie · 2021 — The Practice of Social Research, p.253 (p.283 in the available PDF). Double-barreled questions and clear items. Whether this was the edition used during the Buffalo years is unconfirmed. * Pew Research Center — Writing Survey Questions. Question wording and the problem of combining different judgments in one item. * Bandura · 2006 — Guide for Constructing Self-Efficacy Scales, pp.308–312. Distinguishing efficacy beliefs from intentions, and the 0–10 response format. * Starting material — The private Chat conversation “KCI quantitative research lecture plan.” The items, people, and 12-person distributions were reconstructed for classroom use. [Pew Research Center](https://www.pewresearch.org/writing-survey-questions/) [Bandura: Guide for Constructing Self-Efficacy Scales](https://bpb-us-e1.wpmucdn.com/websites.uta.edu/dist/1/4994/files/2021/07/Guide-for-Constructing-Self-Efficacy-Scales-Bandura2006.pdf) ## 43\. Sources for response tools and demographic items * Qualtrics — Official Question Types: choice, matrix, slider, ranking, constant sum, NPS, and image Heat Map. * SurveyMonkey — Official Question Types: single and multiple choice, matrix/rating, slider, stars, ranking, numeric input, and more. * Australian Bureau of Statistics — 2020 Standard distinguishes sex, gender, sex characteristics, and sexual orientation. The demographic example is an independently written adaptation for practice. * Scope of this implementation — The widgets are custom demonstrations of principles. The XY grid does not replace a validated Affect Grid. Product features and availability can vary by version and plan. [Reference](https://www.qualtrics.com/support/survey-platform/survey-module/editing-questions/question-types-guide/question-types-overview/) [Reference](https://help.surveymonkey.com/en/surveymonkey/create/question-types/) [Reference](https://www.abs.gov.au/statistics/standards/standard-sex-gender-variations-sex-characteristics-and-sexual-orientation-variables/latest-release)