Verification Before—and After—Generation: Why AI Answers Still Need to Be Checked
By Cheval John | Vallano Media
This article was created with the assistance of artificial intelligence.
The final version was reviewed, edited and fact-checked by the author
Graphics created with AI-assisted design tools.
Visual concepts, editorial direction, composition, and final creative decisions were developed by Vallano Media.
Once the research foundation was in place, artificial intelligence became much more useful.
It could compare sources, organize timelines, identify patterns across documents and suggest explanations for why different accounts appeared to conflict.
But that created a different problem: What happens when the AI gives an answer that sounds right?
An obviously poor answer is relatively easy to catch.
A polished answer is harder.
If an AI system produces a clear explanation, uses professional language and connects events in a logical sequence, there is a natural temptation to accept the result and move forward.
But coherence is not confirmation.
For Vallano Media, that led to another rule: Generation did not end the research process.
It triggered another verification process.
A Confident Answer Is Still Just an Answer
Generative AI can produce language that sounds remarkably certain.
That can be useful.
It can also create risk.
A response may contain accurate information, incomplete information, outdated information, unsupported connections or several of those at once.
Those categories are not always obvious from the writing itself.
There is no automatic warning attached to the sentence that happens to be wrong.
That changes how AI-generated material has to be read.
The question cannot simply be: Does this explanation make sense?
The better question is: What evidence would allow us to publish this explanation?
During the FIFA reporting process, useful AI responses were treated as working material rather than finished conclusions.
An answer could suggest a direction.
It could produce a hypothesis.
It could identify something worth investigating.
But before an AI-generated explanation influenced published copy, its important factual claims still had to be traced back to evidence.
Verification Happens at the Claim Level
An AI-generated paragraph can contain several different claims at once.
Imagine a response saying that one organization introduced a proposal, another group opposed it, that opposition caused the proposal to stall, and the controversy demonstrated a larger governance problem.
That may read like one coherent explanation.
It is actually several claims.
Did the organization introduce the proposal?
Did the other group oppose it?
Did that opposition actually cause the proposal to stall?
And is the governance conclusion an established fact or an interpretation?
Each deserves a level of scrutiny appropriate to the claim.
Routine background information may be relatively straightforward to confirm.
Consequential, disputed, causal or unusual claims generally require greater attention.
The first two claims in the example may be factual questions.
The causal claim is more demanding.
The final conclusion may be legitimate analysis, but it should still be presented as analysis rather than established fact.
This is why verification increasingly had to happen at the claim level.
A paragraph can be mostly accurate and still contain one sentence that changes the meaning of the entire story.
That sentence matters.
Sequence Does Not Automatically Mean Cause
Chronology can create another trap.
Event A happens.
Then Event B.
Then Event C.
It is easy to turn that sequence into a story in which A caused B and B caused C.
Sometimes that interpretation is supported.
Sometimes the events simply occurred in that order.
Journalism has to distinguish chronology from causation.
That means asking: Is there evidence that one development actually caused the next?
Did a participant say that it did?
Did independent reporting establish the relationship?
Or are we connecting the events because the sequence looks persuasive?
AI can be useful for turning complicated timelines into understandable narratives.
But that usefulness requires caution.
A compelling narrative becomes a problem when it adds certainty the evidence does not support.
Words such as because, caused, forced, prompted, resulted in and led to therefore deserve extra scrutiny.
A single verb can turn analysis into an unsupported factual claim.
Asking AI to Challenge Its Own Answer
One of the more useful changes in the workflow came when AI stopped being used only to produce answers.
It also began being prompted to challenge them.
After an explanation was generated, the next questions could include: What is weak about this conclusion?
Which claims are not fully supported?
What assumptions are being made?
What alternative explanation fits the same evidence?
Where might the analysis be confusing sequence with causation?
What would a skeptical editor challenge?
What evidence would weaken or disprove this interpretation?
This changed the role of AI.
Instead of functioning only as a generator, it could become part of an adversarial review process.
That did not turn AI into an independent fact-checker.
An AI critique still requires verification and should not be treated as independent evidence.
Its value was in surfacing questions, weaknesses and assumptions that could then be investigated against the evidence.
That introduced useful friction before publication.
And in journalism, friction can be valuable.
The fastest route from prompt to publication is not necessarily the safest one.
The First Answer Was Never Sacred
Another useful habit was refusing to treat the first AI response as authoritative.
The first answer could still be useful.
Sometimes its greatest value was showing how the system was framing the story.
That framing could then be examined.
Was it relying too heavily on one source?
Was it smoothing over uncertainty?
Was it turning a possible explanation into a definite one?
Was it giving too much weight to the most dramatic interpretation?
Was it leaving out an important stakeholder?
Was it treating secondary reporting as though it were primary evidence?
Those questions could produce a better second answer.
Then that answer could be challenged again.
This is not the same as repeatedly prompting AI until it gives the journalist the answer they already wanted.
That would simply automate confirmation bias.
The objective was the opposite: Make the conclusion harder to defend unless the evidence actually supports it.
Verification Sometimes Sends the Reporting Backward
Verification does not always move a story closer to publication.
Sometimes it sends the reporting backward.
An AI-generated claim might expose the need for another source.
A timeline comparison might reveal a date discrepancy.
A causal explanation might fall apart once the underlying reporting is examined.
A quote might need to be traced back to its original context.
In those situations, the next step is not better wording.
It is more reporting.
The process can therefore look less like: Research → AI → Draft → Publish
and more like: Research → AI analysis → verification → additional research → revised analysis → draft → verification again.
That loop may repeat several times.
From a pure speed perspective, that can look inefficient.
From a journalism perspective, that is the safeguard.
AI is useful not only when it helps finish the work faster, but also when it exposes where the work is not finished.
Verification Also Means Looking for What Is Missing
Errors are not limited to false statements.
Omissions can distort a story too.
An AI summary might accurately describe five important facts while leaving out a sixth fact that changes how the first five should be understood.
That means verification has to ask two questions: Is what I am seeing correct?
and: What am I not seeing?
This becomes particularly important in stories involving multiple organizations, competing interests or conflicting accounts.
A response may be technically accurate but still incomplete.
Which perspectives are represented?
Which are absent?
Does one side receive more explanation than another?
Did the summary emphasize conflict while overlooking agreement?
Did it omit a development that complicates the central narrative?
Those are editorial questions.
AI can help surface them.
It cannot resolve them automatically.
Video Courtesy of CrashCourse YouTube Channel
Red-Team Review Became Part of the Workflow
Over time, this adversarial questioning became more systematic.
Vallano Media began using what could be described as an editorial red-team review: deliberately challenging an argument, analysis or draft in an attempt to expose weaknesses before publication.
This is different from formal AI-security or cybersecurity red teaming.
The purpose here was editorial.
The review could look for unsupported factual claims, causal overreach, missing context, weak sourcing, contradictions, wording stronger than the evidence justified and assumptions disguised as conclusions.
The important shift was simple: Do not ask only how to strengthen an argument.
Ask how the argument could fail.
Generative AI naturally encourages forward momentum: Generate → refine → polish → publish.
Red-teaming interrupts that momentum.
It creates a deliberate stage where the objective changes from building the article to trying to break it.
If the argument survives that challenge and the underlying evidence continues to support it, confidence increases.
If it does not, the weakness has been discovered before publication rather than afterward.
Asking AI to Verify Itself Is Not Enough
There is an important distinction between verification and simply asking an AI system: Are you sure?
The system may reconsider its answer.
It may also confidently repeat the same mistake.
Real verification requires returning to evidence outside the generated response.
That may mean opening the original document.
Reading the full article rather than relying on a summary.
Checking the publication date.
Comparing independent reports.
Locating the original quote.
Reviewing the official statement.
Or acknowledging that confirmation is unavailable.
AI can help guide those steps.
It cannot substitute for them.
So the important question is not: Did AI verify the answer?
It is: What evidence ultimately verified the claim?
The Journalist Has to Be Willing to Reject the Output
Perhaps the most important verification skill is also the simplest.
The journalist has to be willing to say: No.
No, that conclusion is too strong.
No, the evidence does not establish causation.
No, that source is not sufficient.
No, those events should not be combined.
No, the answer is plausible but unverified.
No, that does not belong in the article.
This is where subject knowledge, skepticism and editorial judgment become essential.
AI can make suggestions quickly.
The professional decides which suggestions survive.
The goal is not to become better at accepting AI output.
It is to become better at determining which output deserves to survive the editorial process.
Verify Before and After Generation
The title of this chapter contains an important tension.
Verification should happen before generation because AI analysis is only as strong as the evidence being supplied.
But verification also has to happen after generation, because the system can introduce new assumptions, errors or interpretations while processing that evidence.
The workflow therefore becomes: Verify the inputs → generate the analysis → verify the output.
Then repeat whenever necessary.
That cycle became one of the clearest lessons from Vallano Media’s FIFA reporting process.
AI did not reduce the importance of verification.
It created another place where verification had to occur.
What Comes Next
Once the research foundation had been established and AI-generated conclusions had been tested, the next question became much more practical: What was AI actually useful for?
Not theoretically.
Not as a list of possible applications.
But inside the reporting project itself.
Where did it save time?
Where did it help organize information?
Where did it expose weaknesses?
And where did human judgment still have to take over?
Next week, chapter 4 will examine how AI actually assisted the reporting process — and where its role stopped.
Graphics created with AI-assisted design tools.
Visual concepts, editorial direction, composition, and final creative decisions were developed by Vallano Media.
How AI Actually Assisted the Reporting Process: Where the Tool Helped—and Where It Stopped
By Cheval John | Vallano Media
This article was created with the assistance of artificial intelligence.
The final version was reviewed, edited and fact-checked by the author
Graphics created with AI-assisted design tools.
Visual concepts, editorial direction, composition, and final creative decisions were developed by Vallano Media.
By the time Vallano Media reached this stage of the FIFA project, two things had already been established.
The research foundation had to come first.
And AI-generated conclusions had to be verified before they could influence published work.
That raised the next question: What was artificial intelligence actually useful for?
Not in theory.
Not as a list of generic AI capabilities.
But inside a real reporting process.
For Vallano Media, the answer was not one task.
AI played several different roles depending on what the reporting needed at that moment.
Sometimes it helped organize information.
Sometimes it helped compare sources.
Sometimes it helped surface questions.
Sometimes it helped structure a draft.
And sometimes its most useful contribution was showing where the reporting was still weak.
The important point was that AI did not replace the process.
It moved through the process.
AI Helped Organize a Large Amount of Information
One of the clearest advantages of AI was organization.
A complicated reporting project can generate a large amount of material quickly:
– official statements
– news reports
– timelines
– quotes
– notes
– questions
– competing claims
– background research
– potential angles
Without a system, that material can become difficult to manage.
AI helped turn that information into a more usable structure.
It could group related material.
It could separate recurring themes.
It could compare what different sources were saying about the same event.
It could help identify which questions had already been answered and which ones remained open.
That did not make the research automatically correct.
But it made the research easier to work with.
There is an important distinction there.
AI was not replacing the act of reporting.
It was reducing some of the friction involved in managing what the reporting had already produced.
AI Helped Compare Sources More Quickly
The FIFA story involved multiple organizations, different perspectives and reporting published at different stages of the story.
That made comparison important.
AI could help place different accounts next to one another and ask: Where do these sources agree?
Where do they differ?
Is one source adding new information?
Are two reports describing the same event differently?
Does an official statement conflict with independent reporting?
This kind of comparison can take time when done manually across multiple documents.
AI could accelerate the initial comparison.
But the output still had to be checked.
If AI said two sources contradicted one another, Vallano Media still had to determine whether the contradiction actually existed.
Sometimes the difference was meaningful.
Sometimes the sources were describing different time periods.
Sometimes they were answering different questions.
AI could surface the comparison.
The journalist still had to interpret it.
AI Helped Surface Questions
One of the most useful roles AI played was not answering questions.
It was generating better ones.
A journalist can become deeply familiar with a story.
That familiarity is useful.
It can also create blind spots.
Once a working theory begins to form, it becomes easier to focus on evidence that supports it and overlook questions that challenge it.
AI could be used to interrupt that pattern.
Instead of asking: What does this mean?
the system could be asked: What am I missing?
What would a skeptical reader ask?
Which part of this argument is weakest?
What questions have not been answered yet?
What evidence would contradict this conclusion?
Those prompts often produced more useful results than asking AI to simply summarize the story.
The value was not that the system always found the correct question.
The value was that it could widen the field of inquiry.
In that role, AI became a tool for generating additional lines of inquiry.
AI Helped Organize Working Timelines
Chronology mattered throughout the FIFA reporting project.
Events occurred at different times.
Statements were issued in response to earlier developments.
Independent reporting added information later.
The timing itself sometimes changed how a development should be interpreted.
AI was useful for organizing that sequence into a working timeline.
It could take dates, statements and reported developments and arrange them in a way that made the chronology easier to examine.
That made it easier to see whether two events were actually connected or merely close together.
It also helped identify gaps.
If a timeline jumped from one important development to another without showing what happened in between, that gap could become a new reporting question.
Again, AI did not determine what the timeline meant.
It helped make the chronology visible.
The interpretation still required evidence and editorial judgment.
AI Helped Test an Emerging Argument
As the reporting developed, certain interpretations naturally became stronger than others.
That is where another AI role became valuable.
The system could be asked to challenge the emerging argument.
For example: What assumptions does this conclusion depend on?
What evidence weakens it?
Is there another explanation that fits the same facts?
Where am I overstating causation?
Which claims would require stronger sourcing before publication?
This became part of Vallano Media’s editorial red-team process.
The goal was not to get AI to approve the argument.
The goal was to make the argument survive criticism.
That distinction matters.
If AI is used only to strengthen the direction a journalist has already chosen, the tool can reinforce confirmation bias.
If it is also used to challenge that direction, it becomes more useful.
The best AI response was not always the one that helped finish the article.
Sometimes it was the one that forced the article back into research.
AI Helped Build the Structure Before the Draft
Once the reporting was sufficiently developed, AI also became useful for organization at the writing stage.
Long-form reporting can contain several valid points but still lack a clear structure.
AI could help group those points into sections.
It could suggest an order.
It could identify repetition.
It could show where one section depended on information that had not yet been introduced.
That was especially useful for the FIFA series because the reporting had accumulated across multiple sources and stages.
A strong outline helped prevent the draft from becoming a collection of facts without a clear narrative.
But the outline was still treated as a proposal.
The journalist could accept it.
Change it.
Reorder it.
Discard parts of it.
Or reject it entirely.
AI helped create options.
Editorial judgment decided which structure worked.
AI Assisted With Drafting
AI also assisted with portions of the drafting process.
That is an area where transparency matters.
Vallano Media disclosed AI assistance because the technology was used as part of the writing workflow.
But AI-assisted drafting did not mean an article was generated once and published.
The drafting stage was iterative.
AI could help turn an outline into prose.
It could suggest transitions.
It could help condense repetitive passages.
It could offer alternative wording.
It could help clarify a complicated explanation.
But every useful draft still had to pass through editorial review.
The important question was never: Did AI write this sentence?
The more important questions were: Is the sentence accurate?
Is it properly supported?
Does it reflect what the evidence actually shows?
Does it sound like Vallano Media?
Would we publish it if AI had not been involved?
If the answer to those questions was no, the sentence did not survive.
AI Helped With Revision
Revision became another useful application.
Once a draft existed, AI could help identify:
– repetition
– unclear transitions
– weak section order
– unsupported claims
– missing context
– overly strong language
– areas where the article became too abstract
– sections that needed more evidence
This did not mean every suggestion was accepted.
Some AI edits made the writing cleaner.
Others removed useful nuance.
Some made sentences shorter but less accurate.
Some improved flow while weakening attribution.
That meant revision required the same skepticism as research.
A cleaner sentence is not automatically a better sentence.
Sometimes the complexity is necessary because the evidence is complex.
Video Courtesy of The Poytner Institute YouTube Channel
AI Helped Identify What Needed Fact-Checking
AI could assist the fact-checking process by identifying claims that deserved closer review.
It could flag dates.
It could surface statements that looked too strong.
It could compare a draft against source material provided to it.
It could ask where support appeared to be missing.
Those functions were useful.
But they did not replace independent verification.
A system cannot become its own final authority simply because it reviewed the text it helped create.
The actual verification still required returning to the underlying evidence.
That meant checking:
– names
– dates
– chronology
– quotations
– attribution
– causal language
– descriptions of what organizations said
– distinctions between fact and analysis
AI could help identify where to look.
The evidence still had to settle the question.
AI Helped With More Than Writing
One of the broader lessons from the project was that AI’s usefulness extended beyond the article itself.
It could assist with:
– headline options
– SEO framing
– metadata
– social-media copy
– visual concepts
– graphic direction
– summaries
– repurposing
– publication checklists
That mattered because publishing is more than writing.
A digital article needs presentation.
It needs distribution.
It needs a headline that accurately reflects the story.
It needs visual material.
It may need social promotion.
It may later become part of a larger project, as the FIFA reporting eventually did with this white paper.
AI could support those downstream tasks too.
But the same principle applied.
The tool could generate options.
Vallano Media still made the decision.
The Role Changed Depending on the Task
One reason it is misleading to describe AI simply as a “writer” is that its role was constantly changing.
At different moments, AI functioned more like:
– a research organizer
– a comparison tool
– a brainstorming partner
– a timeline assistant
– a skeptical reviewer
– an outline assistant
– a drafting tool
– a copy editor
– a publication assistant
No single label captures the whole process.
That matters because discussions about AI and journalism often become too narrow.
If the only question is whether AI can write an article, the conversation misses many of the places where the technology may actually be more useful.
Writing was only one part of the workflow.
In some cases, it was not even the most important one.
Where AI’s Role Stopped
For all of those uses, there were clear limits.
AI did not conduct original reporting or create firsthand evidence.
AI did not decide whether a source was trustworthy enough to publish.
AI did not determine whether an allegation had been sufficiently confirmed.
AI did not decide whether causal language was justified.
AI did not determine whether the story was fair.
AI did not decide whether another round of reporting was necessary.
AI did not make the final publication decision.
Those responsibilities stayed with the journalist and publisher.
That is the line that mattered most.
AI could participate in almost every stage. But participation was not authority.
The Professional Still Had to Know What Good Work Looked Like
The usefulness of AI depended heavily on the ability to evaluate it.
A journalist has to recognize weak sourcing.
A journalist has to notice when a timeline does not make sense.
A journalist has to distinguish fact from allegation.
A journalist has to understand attribution.
A journalist has to recognize when a sentence is stronger than the evidence.
Without those skills, AI can make poor work look polished.
With those skills, AI can help make good work more efficient.
That difference is fundamental.
AI does not remove the need for expertise.
It makes expertise more important because someone still has to judge the output.
The Goal Was Better Work, Not Maximum Automation
The FIFA project also reinforced another lesson.
The goal was never to automate as much journalism as possible.
That would have been the wrong measurement.
The better question was: Where does AI improve the work without weakening the standards?
If it saves time organizing research, use it.
If it helps surface a contradiction, investigate it.
If it improves an outline, consider the structure.
If it exposes a weakness, go back to reporting.
If it produces a cleaner sentence that remains accurate, use the sentence.
If it introduces uncertainty, discard it.
The workflow was not built around maximizing AI use. It was built around maximizing usefulness.
That is a very different objective.
What Comes Next
By this point in the process, AI had assisted with research organization, comparison, questioning, analysis, outlining, drafting and revision.
But none of that answered the most important editorial question: Who makes the final call?
That responsibility never moved.
Next week, chapter 5 will examine why human judgment still determines what ultimately gets published.
Graphics created with AI-assisted design tools.
Visual concepts, editorial direction, composition, and final creative decisions were developed by Vallano Media.
Share this: