Activity A kit: question the AI answer, claim by claim

The brief, the college, both sources and the AI answer in this kit are FICTIONAL: for a classroom exercise. The research findings in the facilitator notes are real and sourced.

What this activity is for

Activity A runs for 40 minutes on Day 1, as the last block of "The world you are walking into". The aim is to tell a useful AI answer apart from reliable evidence. Each learner gets a short brief and two sources that disagree, one of them out of date. They write their own advice first, then read an AI answer, check it claim by claim, correct their advice and say what is still uncertain.

Learners use this skill for the rest of First Move: checking an AI translation in the Day 2 task trial, checking a salary figure on Day 3, and questioning any suggestion marked as AI in their Future Map.

The scenario (in learner-pack.md)

Aina, 18, wants to visit the open day at a fictional skills college on Sunday 13 December 2026. She asks: can I just turn up, how do I get there, what should I bring, and how much will it cost?

• Source 1, a college flyer printed in March 2023, says walk in, 9am to 1pm, Bus Route 12 every 30 minutes at RM1.50, and an application fee of RM50.
• Source 2, a website notice posted in October 2026, says register online by the Wednesday before, 9am to 12 noon, Route 12A every 60 minutes on Sundays, an application fee of RM80 from the 2027 intake, and bring MyKad and the SPM trial results slip.
• The AI answer repeats the stale flyer and adds a claim found in neither source: that the college guarantees a place to everyone who attends and lets them enrol on the spot.

All of these details are invented for the exercise.

Run sheet

Store the first attempt, the claim check, the corrected advice with the two source references, and the rule as evidence. A spoken or Bahasa Malaysia explanation counts the same as a written one.

Materials

• One printed learner pack per learner: brief, Source 1, Source 2, the AI answer and the worksheet.
• One facilitator answer key per pod (staff only), kept back until minute 37.
• The printable "Check an AI answer" checklist from the library guide on questioning AI output.
• A pod record sheet for the two criterion levels, with an example each.
• Optional: the approved classroom AI tool on shared devices for a second round. No personal accounts.

Facilitator notes

It runs without devices. The AI answer is printed in the pack, so no one needs a personal AI account and the activity still works if the Wi-Fi fails. If time and devices allow, a second round can use the approved classroom AI tool: learners ask it Aina's question with both sources pasted in, then check the new answer the same way. Nobody pastes real personal details into any AI tool.

Look for three moves. Spotting the stale facts (the newer, official notice wins). Flagging the invented guarantee as "in no source", not merely "wrong". Treating the bus fare as unknown rather than true or false, because only the 2023 flyer mentions it. Ask "Which source is newer?" before you tell anyone the answer. The full answer key is in facilitator-answer-key.md.

Assess the reasoning, not the polish. Record one level, emerging, demonstrated with support, or demonstrated independently, for "uses and verifies AI appropriately" and one for "explains a justified next step", each with a short example. These are formative teaching notes, not scores.

Why the unaided step comes first

A field experiment with nearly 1,000 high-school maths students, published in PNAS in 2025, compared a standard GPT-4 chat interface ("GPT Base") with a version designed to protect learning ("GPT Tutor")^[1]. During practice, grades improved by 48% with GPT Base and 127% with GPT Tutor; but when access was removed, students who had used GPT Base did 17% worse than students who never had access^[1]. The safeguards in GPT Tutor largely reduced that harm^[1].

That study was about maths in one setting, and it does not tell us the size of any effect at First Move. It does support a simple design rule: ask for the learner's own attempt first, keep the AI output separate, and check what the learner can explain without it.

How this links to wider AI literacy

UNESCO's AI competency framework for students, published in August 2024, sets out 12 competencies across four dimensions (a human-centred mindset, the ethics of AI, AI techniques and applications, and AI system design) at three levels: understand, apply and create^[2]. It stresses critical judgement of AI solutions^[2]. First Move uses the framework as a curriculum reference only; it is not an accreditation.

What to do next

1. Facilitators: print one learner pack per learner and one answer key per pod, and agree with each other what "independently" looks like before the session.
2. Curriculum lead: confirm whether there is an approved classroom AI tool for the optional second round. If you change the scenario, keep the three ingredients: one stale source, one current source, and one AI claim found in neither.
3. Learners: next time an AI tool gives you a number, a date or a deadline, find where it came from and write the source's date next to it before you rely on it.