Capabilities and uncertainty
Variation, confidence, and unsupported answers
Use evidence and repeat checks instead of treating fluency or repeated wording as proof.
Lesson 4 of 6 in the recommended order · About 25 min (estimate)
On this page
Practical AI glossary — terms and common confusions
- Model
- A learned component used to produce a result.
A drafting model is one part of a letter app; it is not the whole app.
- Application
- The software experience around components and services.
A letter app adds accounts, storage, and a Send button.
- Prompt
- Instructions and input supplied for a task.
“Summarize this notice in two bullets” specifies a task and shape, not a truth guarantee.
- Token
- A unit a model processes; it need not be a whole word.
A tokenizer can split a name into pieces; count using the actual system.
- Context
- Information available for the current request.
An earlier attachment may be absent even when its filename is visible.
- Training
- Adjusting a model using examples.
A training log differs from a conversation correction.
- Inference
- Using a trained model to produce a result.
Drafting another sentence does not by itself prove retraining.
- Retrieval
- Selecting material to supply to a task.
A retrieved notice still needs a relevance and date check.
- Embedding
- A numerical representation used to compare or organize items.
A similar search match is a candidate, not proof of the same meaning.
- Tool
- A function an application can call.
A calendar lookup can be read-only; finding an event is different from creating one.
- Agent
- Software that can choose and carry out steps toward a goal.
An agent still needs bounded permissions and checked action receipts.
- Hallucination
- Plausible generated content that is unsupported or false.
An invented opening time needs evidence or removal, even when phrased confidently.
- Evaluation
- Checking results against stated criteria.
Twenty checked examples establish a bounded result, not universal reliability.
- Multimodal
- Working with more than one kind of input or output.
A photo plus text does not prove every small label in the photo was read correctly.
Outcome
Distinguish response variation, source support, and measured performance without inventing confidence.
Why it matters
Use evidence and repeat checks instead of treating fluency or repeated wording as proof.
Concept
A generated answer can sound settled while containing an unsupported claim. It can also be correct. Evaluate the claim against evidence rather than assuming that every output is wrong or that confident language guarantees accuracy.
Several responses to the same task can differ. A second response is a comparison opportunity, not an independent source for facts. Repetition can reveal consistency but does not establish truth. When a task includes a tested accuracy rate, preserve its sample and conditions; do not turn it into a guarantee for the next case. Product controls for variation differ, so these lessons require no sampling settings.
Worked example
Three fictional summaries all say a venue is open Sundays, but the current notice says it is closed Sundays. Agreement among the summaries does not outweigh the source. Correct the hours and cite the notice. If a fourth summary preserves the notice, accept it; demanding a change merely because it was generated would be another mistake.
Before you check
Underline every factual claim in a response. Beside it, write its supporting source or “unverified.” Keep style preferences separate from factual corrections.
Practice and fresh transfer
The packets below are fictional and contain the facts needed for these cases. External references are optional background. Judge each response independently: it may be supported, contradicted, or unresolved. Select the passages needed to justify your judgment and write why the distinction matters before revealing feedback.
Assess the response
Use Accept when all material claims are supported. Use Revise when a supplied fact or requirement is contradicted. Use Evidence is insufficient when a key fact cannot be established either way. If a response contains both an unknown and a direct contradiction, choose Revise and explain both problems. Conflicting claims with no established authority remain insufficient; a claim does not become a governing fact merely because a source asserts it.
Some responses are fully supported. Others need correction or more evidence. Judge each on its sources; do not edit a correct answer just to change it.
Your written notes stay in this page and disappear when you leave. Only a self-reviewed completion can be saved to your learning account. These practice checks do not establish independent proficiency.
Practice
Case 1
Compare the meaning of three fictional generated summaries of the same desk-hours notice.
Source packet
- Source 1
- Current notice: The desk is open Monday–Friday, 9–5, and closed Saturday and Sunday.
- Source 2
- Draft A: Open weekdays 9–5; closed weekends. Draft B: Monday through Friday, 9 to 5; no weekend opening. Draft C: Open every day, 9–5.
Response to assess
A and B preserve the same hours despite different wording. C changes the weekend rule.
Practice
Case 2
Consider a fictional claim after three generated replies agreed.
Source packet
- Source 1
- Source notice: The members’ room requires a reservation.
- Source 2
- Run record: Three generated replies each said no reservation is needed.
Response to assess
No reservation is needed; three agreeing replies prove it.
Fresh transfer
Case 3
Consider two fictional generated interpretations of a small address-formatting evaluation.
Source packet
- Source 1
- Test record: The system matched the rubric on 18 of 20 address-formatting examples. No other tasks were evaluated.
- Source 2
- Report A: It matched 90% of these address-formatting examples. Report B: It is 90% correct on every kind of task.
Response to assess
A preserves the measured result and scope; B expands the claim to tasks this test did not evaluate.
Fresh transfer
Case 4
Consider whether a fictional confident answer can be forwarded as verified.
Source packet
- Source 1
- Question: How many visitors arrived on Tuesday?
- Source 2
- Available record: The packet contains Wednesday’s count of 46; no Tuesday count is supplied.
Response to assess
Exactly 46 visitors arrived Tuesday. I am completely certain.
Save your self-review
Completion records that you reviewed the cases. Your explanation and transfer performance need a facilitator to establish independent learning.
Sign in with your learning-center account to save completion.
Review the explanation for every case before saving.
Summary and next step
Apply the checklist to a new task. Preserve supported content, explain any change with evidence, and name what remains unresolved. Saving records self-review, not independently demonstrated proficiency. A facilitator must assess the explanation and fresh transfer for human learning evidence.