Verification Before—and After—Generation: Why AI Answers Still Need to Be Checked
By Cheval John | Vallano Media
This article was created with the assistance of artificial intelligence.
The final version was reviewed, edited and fact-checked by the author
Graphics created with AI-assisted design tools.
Visual concepts, editorial direction, composition, and final creative decisions were developed by Vallano Media.
Once the research foundation was in place, artificial intelligence became much more useful.
It could compare sources, organize timelines, identify patterns across documents and suggest explanations for why different accounts appeared to conflict.
But that created a different problem: What happens when the AI gives an answer that sounds right?
An obviously poor answer is relatively easy to catch.
A polished answer is harder.
If an AI system produces a clear explanation, uses professional language and connects events in a logical sequence, there is a natural temptation to accept the result and move forward.
But coherence is not confirmation.
For Vallano Media, that led to another rule: Generation did not end the research process.
It triggered another verification process.
A Confident Answer Is Still Just an Answer
Generative AI can produce language that sounds remarkably certain.
That can be useful.
It can also create risk.
A response may contain accurate information, incomplete information, outdated information, unsupported connections or several of those at once.
Those categories are not always obvious from the writing itself.
There is no automatic warning attached to the sentence that happens to be wrong.
That changes how AI-generated material has to be read.
The question cannot simply be: Does this explanation make sense?
The better question is: What evidence would allow us to publish this explanation?
During the FIFA reporting process, useful AI responses were treated as working material rather than finished conclusions.
An answer could suggest a direction.
It could produce a hypothesis.
It could identify something worth investigating.
But before an AI-generated explanation influenced published copy, its important factual claims still had to be traced back to evidence.
Verification Happens at the Claim Level
An AI-generated paragraph can contain several different claims at once.
Imagine a response saying that one organization introduced a proposal, another group opposed it, that opposition caused the proposal to stall, and the controversy demonstrated a larger governance problem.
That may read like one coherent explanation.
It is actually several claims.
Did the organization introduce the proposal?
Did the other group oppose it?
Did that opposition actually cause the proposal to stall?
And is the governance conclusion an established fact or an interpretation?
Each deserves a level of scrutiny appropriate to the claim.
Routine background information may be relatively straightforward to confirm.
Consequential, disputed, causal or unusual claims generally require greater attention.
The first two claims in the example may be factual questions.
The causal claim is more demanding.
The final conclusion may be legitimate analysis, but it should still be presented as analysis rather than established fact.
This is why verification increasingly had to happen at the claim level.
A paragraph can be mostly accurate and still contain one sentence that changes the meaning of the entire story.
That sentence matters.
Sequence Does Not Automatically Mean Cause
Chronology can create another trap.
Event A happens.
Then Event B.
Then Event C.
It is easy to turn that sequence into a story in which A caused B and B caused C.
Sometimes that interpretation is supported.
Sometimes the events simply occurred in that order.
Journalism has to distinguish chronology from causation.
That means asking: Is there evidence that one development actually caused the next?
Did a participant say that it did?
Did independent reporting establish the relationship?
Or are we connecting the events because the sequence looks persuasive?
AI can be useful for turning complicated timelines into understandable narratives.
But that usefulness requires caution.
A compelling narrative becomes a problem when it adds certainty the evidence does not support.
Words such as because, caused, forced, prompted, resulted in and led to therefore deserve extra scrutiny.
A single verb can turn analysis into an unsupported factual claim.
Asking AI to Challenge Its Own Answer
One of the more useful changes in the workflow came when AI stopped being used only to produce answers.
It also began being prompted to challenge them.
After an explanation was generated, the next questions could include: What is weak about this conclusion?
Which claims are not fully supported?
What assumptions are being made?
What alternative explanation fits the same evidence?
Where might the analysis be confusing sequence with causation?
What would a skeptical editor challenge?
What evidence would weaken or disprove this interpretation?
This changed the role of AI.
Instead of functioning only as a generator, it could become part of an adversarial review process.
That did not turn AI into an independent fact-checker.
An AI critique still requires verification and should not be treated as independent evidence.
Its value was in surfacing questions, weaknesses and assumptions that could then be investigated against the evidence.
That introduced useful friction before publication.
And in journalism, friction can be valuable.
The fastest route from prompt to publication is not necessarily the safest one.
The First Answer Was Never Sacred
Another useful habit was refusing to treat the first AI response as authoritative.
The first answer could still be useful.
Sometimes its greatest value was showing how the system was framing the story.
That framing could then be examined.
Was it relying too heavily on one source?
Was it smoothing over uncertainty?
Was it turning a possible explanation into a definite one?
Was it giving too much weight to the most dramatic interpretation?
Was it leaving out an important stakeholder?
Was it treating secondary reporting as though it were primary evidence?
Those questions could produce a better second answer.
Then that answer could be challenged again.
This is not the same as repeatedly prompting AI until it gives the journalist the answer they already wanted.
That would simply automate confirmation bias.
The objective was the opposite: Make the conclusion harder to defend unless the evidence actually supports it.
Verification Sometimes Sends the Reporting Backward
Verification does not always move a story closer to publication.
Sometimes it sends the reporting backward.
An AI-generated claim might expose the need for another source.
A timeline comparison might reveal a date discrepancy.
A causal explanation might fall apart once the underlying reporting is examined.
A quote might need to be traced back to its original context.
In those situations, the next step is not better wording.
It is more reporting.
The process can therefore look less like: Research → AI → Draft → Publish
and more like: Research → AI analysis → verification → additional research → revised analysis → draft → verification again.
That loop may repeat several times.
From a pure speed perspective, that can look inefficient.
From a journalism perspective, that is the safeguard.
AI is useful not only when it helps finish the work faster, but also when it exposes where the work is not finished.
Verification Also Means Looking for What Is Missing
Errors are not limited to false statements.
Omissions can distort a story too.
An AI summary might accurately describe five important facts while leaving out a sixth fact that changes how the first five should be understood.
That means verification has to ask two questions: Is what I am seeing correct?
and: What am I not seeing?
This becomes particularly important in stories involving multiple organizations, competing interests or conflicting accounts.
A response may be technically accurate but still incomplete.
Which perspectives are represented?
Which are absent?
Does one side receive more explanation than another?
Did the summary emphasize conflict while overlooking agreement?
Did it omit a development that complicates the central narrative?
Those are editorial questions.
AI can help surface them.
It cannot resolve them automatically.
Video Courtesy of CrashCourse YouTube Channel
Red-Team Review Became Part of the Workflow
Over time, this adversarial questioning became more systematic.
Vallano Media began using what could be described as an editorial red-team review: deliberately challenging an argument, analysis or draft in an attempt to expose weaknesses before publication.
This is different from formal AI-security or cybersecurity red teaming.
The purpose here was editorial.
The review could look for unsupported factual claims, causal overreach, missing context, weak sourcing, contradictions, wording stronger than the evidence justified and assumptions disguised as conclusions.
The important shift was simple: Do not ask only how to strengthen an argument.
Ask how the argument could fail.
Generative AI naturally encourages forward momentum: Generate → refine → polish → publish.
Red-teaming interrupts that momentum.
It creates a deliberate stage where the objective changes from building the article to trying to break it.
If the argument survives that challenge and the underlying evidence continues to support it, confidence increases.
If it does not, the weakness has been discovered before publication rather than afterward.
Asking AI to Verify Itself Is Not Enough
There is an important distinction between verification and simply asking an AI system: Are you sure?
The system may reconsider its answer.
It may also confidently repeat the same mistake.
Real verification requires returning to evidence outside the generated response.
That may mean opening the original document.
Reading the full article rather than relying on a summary.
Checking the publication date.
Comparing independent reports.
Locating the original quote.
Reviewing the official statement.
Or acknowledging that confirmation is unavailable.
AI can help guide those steps.
It cannot substitute for them.
So the important question is not: Did AI verify the answer?
It is: What evidence ultimately verified the claim?
The Journalist Has to Be Willing to Reject the Output
Perhaps the most important verification skill is also the simplest.
The journalist has to be willing to say: No.
No, that conclusion is too strong.
No, the evidence does not establish causation.
No, that source is not sufficient.
No, those events should not be combined.
No, the answer is plausible but unverified.
No, that does not belong in the article.
This is where subject knowledge, skepticism and editorial judgment become essential.
AI can make suggestions quickly.
The professional decides which suggestions survive.
The goal is not to become better at accepting AI output.
It is to become better at determining which output deserves to survive the editorial process.
Verify Before and After Generation
The title of this chapter contains an important tension.
Verification should happen before generation because AI analysis is only as strong as the evidence being supplied.
But verification also has to happen after generation, because the system can introduce new assumptions, errors or interpretations while processing that evidence.
The workflow therefore becomes: Verify the inputs → generate the analysis → verify the output.
Then repeat whenever necessary.
That cycle became one of the clearest lessons from Vallano Media’s FIFA reporting process.
AI did not reduce the importance of verification.
It created another place where verification had to occur.
What Comes Next
Once the research foundation had been established and AI-generated conclusions had been tested, the next question became much more practical: What was AI actually useful for?
Not theoretically.
Not as a list of possible applications.
But inside the reporting project itself.
Where did it save time?
Where did it help organize information?
Where did it expose weaknesses?
And where did human judgment still have to take over?
Next week, chapter 4 will examine how AI actually assisted the reporting process — and where its role stopped.
Graphics created with AI-assisted design tools.
Visual concepts, editorial direction, composition, and final creative decisions were developed by Vallano Media.
Building the Research Foundation: Why Source Quality Comes Before AI Analysis
By Cheval John | Vallano Media
This article was created with the assistance of artificial intelligence.
The final version was reviewed, edited and fact-checked by the author
Graphics created with AI-assisted design tools.
Visual concepts, editorial direction, composition, and final creative decisions were developed by Vallano Media.
Before artificial intelligence could help analyze the FIFA story, there was a more basic problem to solve.
What information could actually be trusted?
That question matters in any reporting project, but it becomes especially important when AI enters the workflow.
Artificial intelligence can organize information quickly.
It can compare documents, summarize reporting, identify differences between accounts and generate additional questions.
But if the information going into that process is weak, incomplete or incorrectly understood, faster analysis does not solve the problem.
It can simply produce a more polished version of the wrong answer.
For Vallano Media’s reporting on the FIFA Forward Enterprise proposal, the research therefore had to begin somewhere AI could not replace: the sources themselves.
Primary Sources and Secondary Reporting Serve Different Purposes
One of the first steps was separating primary sources from secondary reporting.
Primary sources included material originating from organizations or people directly involved in the story, including official statements and other first-hand material relevant to the proposal.
Those sources were important because they could establish what an organization had actually said, proposed or published.
But being a primary source does not automatically make a source neutral or complete.
An organization speaks from its own perspective.
An official statement may explain a proposal, defend a decision or respond to criticism without answering every question surrounding the issue.
That is where independent reporting becomes important.
Secondary reporting can provide additional context, outside sourcing and information that may not appear in an organization’s own communications.
The goal was not to declare one category universally superior to the other.
It was to understand what each source could legitimately establish.
An official statement might establish what FIFA said.
Independent reporting might establish what journalists learned through additional sourcing.
Analysis might help explain why those developments mattered.
Those are different functions.
Treating them as interchangeable creates problems.
Video Courtesy of CrashCourse YouTube Channel
AI Was Not the Source
That distinction helped establish one of the most important rules of the reporting process: An AI response was not treated as evidence.
Generative AI can make that boundary surprisingly easy to blur.
Ask a system a complicated question and it may produce a detailed answer containing names, dates, chronology and explanations within seconds.
It can look like research.
But the response itself does not establish that those claims are true.
AI could help locate material that deserved examination, organize information that had already been gathered, compare accounts and generate additional questions.
But when a factual claim mattered to the story, the important question remained: Where did this information actually come from?
The underlying source still had to be identified and evaluated.
AI could participate in source discovery.
It could not substitute for the source.
Source Quality Matters More Than Source Quantity
Having more links does not necessarily mean having better reporting.
A dozen websites can repeat the same original claim.
Twenty articles can ultimately trace back to one source.
Search results can therefore create the appearance of widespread confirmation when several publications are simply repeating information from one another.
That makes source quality more important than source volume.
The important questions were familiar journalistic ones.
Who originally reported the information?
Was there a primary source?
Was another credible organization independently confirming it?
Was an article contributing new reporting or repeating somebody else’s work?
Did the source clearly distinguish verified information from allegation or interpretation?
How close was the source to the event being described?
None of those questions were invented by artificial intelligence.
The value of the workflow was not that it created a new form of journalism.
It applied familiar reporting standards to a new tool.
Building the Timeline
Chronology became another important part of the research foundation.
Complex stories rarely arrive in perfect order.
One report may describe something that happened several days earlier.
An official response may appear after independent reporting.
A later development may change how an earlier event should be understood.
If those events are combined incorrectly, the resulting narrative can imply relationships the evidence does not support.
For the FIFA project, establishing the timeline helped separate: what happened, when it happened and what was known at each stage.
That distinction matters because hindsight can distort reporting.
Information that appears obvious after several developments may not have been available when an earlier decision was made.
Likewise, two events that occurred close together are not automatically connected.
AI could help organize dates and compare accounts, but chronology still had to be checked against the underlying reporting.
AI-assisted summaries can collapse events from different moments into a single narrative if that chronology is not checked carefully.
A reliable timeline became one safeguard against that problem.
Facts, Allegations, Analysis and Speculation Are Not the Same Thing
Another step was determining what level of certainty the evidence actually supported.
Not every sentence in a complicated story carries the same weight.
Some information can be stated as established fact.
Some must be attributed because it represents what an organization or person claims.
Some material comes from reported allegations.
Some conclusions are analysis.
And some possibilities remain speculation.
Those categories cannot be casually blended together.
There is a significant difference between saying: An organization did something.
and saying: A news organization reported that sources said the organization was considering doing something.
Removing that distinction might make a sentence shorter.
It can also make the sentence inaccurate.
AI systems can be useful for condensing information, but condensation can erase important qualifications.
Words such as according to, reported, alleged, proposed, and could may appear minor.
They are not.
Sometimes they are the difference between accurately representing the evidence and overstating it.
Contradictions Are Signals, Not Answers
When two credible sources appeared to conflict, the goal was not to immediately decide that one of them was wrong.
The contradiction itself became a reporting question.
Were the sources describing different time periods?
Were they answering different questions?
Did one report contain newer information?
Was one describing an official position while another relied on independent sourcing?
Could both accounts be accurate within different contexts?
This is an area where AI can be particularly useful.
It can place statements next to one another.
It can surface apparent differences.
It can help formulate questions about those differences.
But an apparent contradiction identified by AI is not automatically a contradiction in reality.
It is a signal to investigate further.
AI can surface the problem.
Reporting still has to resolve it — or acknowledge that the available evidence does not yet allow it to be resolved.
Sometimes the Gap Is the Finding
One of the easiest mistakes in research is feeling compelled to fill every blank.
Journalists naturally want answers.
AI systems are also designed to produce answers.
That combination requires caution.
Sometimes the evidence simply does not answer a question.
Responsible reporting does not require constructing the most plausible explanation.
Sometimes the responsible conclusion is simply: The evidence is incomplete.
During the FIFA research process, gaps in available information were treated as gaps rather than invitations for AI to construct connections between them.
This matters because generative AI can produce plausible explanations.
But plausibility is not evidence.
If two verified facts sit on opposite sides of an information gap, AI may be able to suggest several possible explanations connecting them.
Those possibilities can become questions for further research.
They cannot automatically become facts in the story.
Research Became an Iterative Process
The research did not move neatly from beginning to end.
One source raised another question.
A secondary report led back to primary material.
A timeline exposed something that needed additional research.
An AI comparison surfaced a potential inconsistency.
Verification produced new questions.
The process became cyclical: Find information → evaluate the source → verify the claim → compare the evidence → identify questions → research again.
AI could accelerate parts of that process.
It could not eliminate it.
In fact, some of the most useful AI interactions produced more reporting rather than less.
That may sound counterintuitive.
Much of the promise surrounding AI focuses on efficiency: fewer steps, less time and faster output.
But in journalism, a tool can be valuable precisely because it reveals something that deserves another round of research.
Efficiency does not always mean finishing sooner.
Sometimes it means discovering sooner that the work is not finished.
The Foundation Comes Before the Analysis
By the time AI became useful for deeper analysis of the FIFA story, the basic research structure was already taking shape.
Sources had been separated according to their role.
Important claims could be traced back to evidence.
Chronology had been established.
Potential contradictions had been identified for further examination.
Facts had been distinguished from allegations, reporting and interpretation.
And gaps were allowed to remain gaps when the evidence did not justify filling them.
Only then could AI provide meaningful assistance.
That sequence matters.
If AI analysis comes first, there is a risk that the journalist begins building the research around the system’s initial explanation.
If the evidence comes first, AI can instead be asked to work against a foundation the journalist has already examined.
The difference is subtle but important.
One approach asks AI: What happened?
The other begins with evidence and asks: What should we examine next?
That is a much safer place to begin.
What Comes Next
Building a strong research foundation did not eliminate the risks associated with artificial intelligence.
It made those risks easier to recognize and manage.
The next challenge was deciding how much confidence to place in what AI produced once it began analyzing that material.
That required another principle: Verification does not end when AI enters the workflow.
If anything, it becomes more important.
Next Tuesday, chapter 3 will examine verification before generation — including how Vallano Media challenged AI-assisted conclusions, checked confident answers and prevented analysis from outrunning the evidence.
Graphics created with AI-assisted design tools.
Visual concepts, editorial direction, composition, and final creative decisions were developed by Vallano Media.