Tag Archive | Journalism

Verification Before—and After—Generation: Why AI Answers Still Need to Be Checked

By Cheval John | Vallano Media
This article was created with the assistance of artificial intelligence.
The final version was reviewed, edited and fact-checked by the author


Graphics created with AI-assisted design tools.
Visual concepts, editorial direction, composition, and final creative decisions were developed by Vallano Media.

Once the research foundation was in place, artificial intelligence became much more useful.

It could compare sources, organize timelines, identify patterns across documents and suggest explanations for why different accounts appeared to conflict.

But that created a different problem: What happens when the AI gives an answer that sounds right?

An obviously poor answer is relatively easy to catch.

A polished answer is harder.

If an AI system produces a clear explanation, uses professional language and connects events in a logical sequence, there is a natural temptation to accept the result and move forward.

But coherence is not confirmation.

For Vallano Media, that led to another rule: Generation did not end the research process.

It triggered another verification process.

A Confident Answer Is Still Just an Answer

Generative AI can produce language that sounds remarkably certain.

That can be useful.

It can also create risk.

A response may contain accurate information, incomplete information, outdated information, unsupported connections or several of those at once.

Those categories are not always obvious from the writing itself.

There is no automatic warning attached to the sentence that happens to be wrong.

That changes how AI-generated material has to be read.

The question cannot simply be: Does this explanation make sense?

The better question is: What evidence would allow us to publish this explanation?

During the FIFA reporting process, useful AI responses were treated as working material rather than finished conclusions.

An answer could suggest a direction.

It could produce a hypothesis.

It could identify something worth investigating.

But before an AI-generated explanation influenced published copy, its important factual claims still had to be traced back to evidence.

Verification Happens at the Claim Level

An AI-generated paragraph can contain several different claims at once.

Imagine a response saying that one organization introduced a proposal, another group opposed it, that opposition caused the proposal to stall, and the controversy demonstrated a larger governance problem.

That may read like one coherent explanation.

It is actually several claims.

Did the organization introduce the proposal?

Did the other group oppose it?

Did that opposition actually cause the proposal to stall?

And is the governance conclusion an established fact or an interpretation?

Each deserves a level of scrutiny appropriate to the claim.

Routine background information may be relatively straightforward to confirm.

Consequential, disputed, causal or unusual claims generally require greater attention.

The first two claims in the example may be factual questions.

The causal claim is more demanding.

The final conclusion may be legitimate analysis, but it should still be presented as analysis rather than established fact.

This is why verification increasingly had to happen at the claim level.

A paragraph can be mostly accurate and still contain one sentence that changes the meaning of the entire story.

That sentence matters.

Sequence Does Not Automatically Mean Cause

Chronology can create another trap.

Event A happens.

Then Event B.

Then Event C.

It is easy to turn that sequence into a story in which A caused B and B caused C.

Sometimes that interpretation is supported.

Sometimes the events simply occurred in that order.

Journalism has to distinguish chronology from causation.

That means asking: Is there evidence that one development actually caused the next?

Did a participant say that it did?

Did independent reporting establish the relationship?

Or are we connecting the events because the sequence looks persuasive?

AI can be useful for turning complicated timelines into understandable narratives.

But that usefulness requires caution.

A compelling narrative becomes a problem when it adds certainty the evidence does not support.

Words such as because, caused, forced, prompted, resulted in and led to therefore deserve extra scrutiny.

A single verb can turn analysis into an unsupported factual claim.

Asking AI to Challenge Its Own Answer

One of the more useful changes in the workflow came when AI stopped being used only to produce answers.

It also began being prompted to challenge them.

After an explanation was generated, the next questions could include: What is weak about this conclusion?

Which claims are not fully supported?

What assumptions are being made?

What alternative explanation fits the same evidence?

Where might the analysis be confusing sequence with causation?

What would a skeptical editor challenge?

What evidence would weaken or disprove this interpretation?

This changed the role of AI.

Instead of functioning only as a generator, it could become part of an adversarial review process.

That did not turn AI into an independent fact-checker.

An AI critique still requires verification and should not be treated as independent evidence.

Its value was in surfacing questions, weaknesses and assumptions that could then be investigated against the evidence.

That introduced useful friction before publication.

And in journalism, friction can be valuable.

The fastest route from prompt to publication is not necessarily the safest one.

The First Answer Was Never Sacred

Another useful habit was refusing to treat the first AI response as authoritative.

The first answer could still be useful.

Sometimes its greatest value was showing how the system was framing the story.

That framing could then be examined.

Was it relying too heavily on one source?

Was it smoothing over uncertainty?

Was it turning a possible explanation into a definite one?

Was it giving too much weight to the most dramatic interpretation?

Was it leaving out an important stakeholder?

Was it treating secondary reporting as though it were primary evidence?

Those questions could produce a better second answer.

Then that answer could be challenged again.

This is not the same as repeatedly prompting AI until it gives the journalist the answer they already wanted.

That would simply automate confirmation bias.

The objective was the opposite: Make the conclusion harder to defend unless the evidence actually supports it.

Verification Sometimes Sends the Reporting Backward

Verification does not always move a story closer to publication.

Sometimes it sends the reporting backward.

An AI-generated claim might expose the need for another source.

A timeline comparison might reveal a date discrepancy.

A causal explanation might fall apart once the underlying reporting is examined.

A quote might need to be traced back to its original context.

In those situations, the next step is not better wording.

It is more reporting.

The process can therefore look less like: Research → AI → Draft → Publish

and more like: Research → AI analysis → verification → additional research → revised analysis → draft → verification again.

That loop may repeat several times.

From a pure speed perspective, that can look inefficient.

From a journalism perspective, that is the safeguard.

AI is useful not only when it helps finish the work faster, but also when it exposes where the work is not finished.

Verification Also Means Looking for What Is Missing

Errors are not limited to false statements.

Omissions can distort a story too.

An AI summary might accurately describe five important facts while leaving out a sixth fact that changes how the first five should be understood.

That means verification has to ask two questions: Is what I am seeing correct?

and: What am I not seeing?

This becomes particularly important in stories involving multiple organizations, competing interests or conflicting accounts.

A response may be technically accurate but still incomplete.

Which perspectives are represented?

Which are absent?

Does one side receive more explanation than another?

Did the summary emphasize conflict while overlooking agreement?

Did it omit a development that complicates the central narrative?

Those are editorial questions.

AI can help surface them.

It cannot resolve them automatically.

Video Courtesy of CrashCourse YouTube Channel

Red-Team Review Became Part of the Workflow

Over time, this adversarial questioning became more systematic.

Vallano Media began using what could be described as an editorial red-team review: deliberately challenging an argument, analysis or draft in an attempt to expose weaknesses before publication.

This is different from formal AI-security or cybersecurity red teaming.

The purpose here was editorial.

The review could look for unsupported factual claims, causal overreach, missing context, weak sourcing, contradictions, wording stronger than the evidence justified and assumptions disguised as conclusions.

The important shift was simple: Do not ask only how to strengthen an argument.

Ask how the argument could fail.

Generative AI naturally encourages forward momentum: Generate → refine → polish → publish.

Red-teaming interrupts that momentum.

It creates a deliberate stage where the objective changes from building the article to trying to break it.

If the argument survives that challenge and the underlying evidence continues to support it, confidence increases.

If it does not, the weakness has been discovered before publication rather than afterward.

Asking AI to Verify Itself Is Not Enough

There is an important distinction between verification and simply asking an AI system: Are you sure?

The system may reconsider its answer.

It may also confidently repeat the same mistake.

Real verification requires returning to evidence outside the generated response.

That may mean opening the original document.

Reading the full article rather than relying on a summary.

Checking the publication date.

Comparing independent reports.

Locating the original quote.

Reviewing the official statement.

Or acknowledging that confirmation is unavailable.

AI can help guide those steps.

It cannot substitute for them.

So the important question is not: Did AI verify the answer?

It is: What evidence ultimately verified the claim?

The Journalist Has to Be Willing to Reject the Output

Perhaps the most important verification skill is also the simplest.

The journalist has to be willing to say: No.

No, that conclusion is too strong.

No, the evidence does not establish causation.

No, that source is not sufficient.

No, those events should not be combined.

No, the answer is plausible but unverified.

No, that does not belong in the article.

This is where subject knowledge, skepticism and editorial judgment become essential.

AI can make suggestions quickly.

The professional decides which suggestions survive.

The goal is not to become better at accepting AI output.

It is to become better at determining which output deserves to survive the editorial process.

Verify Before and After Generation

The title of this chapter contains an important tension.

Verification should happen before generation because AI analysis is only as strong as the evidence being supplied.

But verification also has to happen after generation, because the system can introduce new assumptions, errors or interpretations while processing that evidence.

The workflow therefore becomes: Verify the inputs → generate the analysis → verify the output.

Then repeat whenever necessary.

That cycle became one of the clearest lessons from Vallano Media’s FIFA reporting process.

AI did not reduce the importance of verification.

It created another place where verification had to occur.

What Comes Next

Once the research foundation had been established and AI-generated conclusions had been tested, the next question became much more practical: What was AI actually useful for?

Not theoretically.

Not as a list of possible applications.

But inside the reporting project itself.

Where did it save time?

Where did it help organize information?

Where did it expose weaknesses?

And where did human judgment still have to take over?

Next week, chapter 4 will examine how AI actually assisted the reporting process — and where its role stopped.

Building Online Presence One Step At A Time

By Cheval John

The old days of having a career at one company is over.

This holds true for journalists too.

The sequence for every journalist back in the day were:

Spend a couple of years in a smaller media market in order to gain work experience before moving on to a top ten media market like New York City, Houston and Chicago.

Move on from the major media market and work for national media like ESPN, NBC, CBS or ABC.

In this environment where technology is changing the way how we consume content, major media and news stations are cutting back on staff to save money.

Sometimes though, people who work in journalism, decides to leave the profession because the work is demanding.

They might end up leaving for positions in media relations for a large company, a professional sports franchise or even the athletic department at a university.

They believe the transition to a media relations role is easy.

They get a rude awakening of how challenging being in media relations is.

Erica Holloway depicted the transition from journalism to public relations with her article here.

Though journalists stayed at one place their entire careers in the past, some still made a change in careers because they were led to.

Sabrina Cadini spent a couple of years as a television announcer in her native Italy before moving to the United States

She transitioned to event planning and is now the owner of La Dolce Idea Wedding and Soiree, a wedding planning company.

She explained to me how she made the transition to wedding planning

One thing to note.

Sabrina knew how important live streaming is for her business and is using Periscope on a weekly basis to share tips with her audience about social media.

Smart people like Sabrina understand the dynamic of live streaming, podcasting and blogging because it saves them a whole lot of money on their marketing budget.

People who are looking to make transitions in their careers should be building their online presence one step at a time because they never know when their department will shut down.

Are you building your online presence now through social media? Leave your comment below