\documentclass[12pt]{report}

\usepackage[margin=1in]{geometry}
\usepackage{graphicx}
\usepackage{booktabs}
\usepackage{amsmath,amssymb}
\usepackage{array}
\usepackage{enumitem}
\usepackage{hyperref}
\usepackage{float}
\usepackage{verbatim}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}

\hypersetup{
    colorlinks=true,
    linkcolor=blue,
    urlcolor=blue,
    citecolor=blue
}

\begin{document}

\chapter{Image Authenticity Analysis: Optimal 3-Level Hierarchical Ensemble}
\label{chap:image-authenticity}

\section{Introduction: The Geopolitical Deepfake Imperative}
\label{sec:image-intro}

The rapid proliferation of generative artificial intelligence has fundamentally changed the landscape of information authenticity verification. In conflict environments, especially during armed confrontations, manipulated images can be created and disseminated at unprecedented speed, often with the intent to mislead the public, influence diplomatic perceptions, intensify panic, or fabricate evidence of military events. Unlike ordinary misinformation, conflict-related image manipulation operates under high-stakes conditions where false visual claims may trigger political instability, media amplification, or strategic misinformation campaigns.

Operation Sindoor and similar conflict-driven information events demonstrate that image-based disinformation is no longer limited to crude edits or obvious forgeries. Instead, modern synthetic imagery can be visually convincing while remaining semantically misleading or contextually implausible. This creates a critical need for authentication systems that go beyond conventional pixel-level forensics and incorporate multimodal reasoning, semantic verification, and contextual analysis.

To address this challenge, this chapter presents a hierarchical image authenticity analysis framework built from three complementary submodules. The proposed system integrates a CNN-based forensic detector, a CLIP-based image-caption consistency module, and an LLM-based visual reasoning module. Each module captures a different dimension of authenticity, and together they form an ensemble that is more robust than any single detector operating in isolation.

\section{Challenges of Deepfake Images in the Generative AI Era}
\label{sec:image-challenges}

Deepfake image detection in the current generative AI landscape faces several technical and operational challenges. First, modern generative models are capable of producing highly realistic textures, lighting patterns, object boundaries, and scene compositions, reducing the visibility of conventional forgery artifacts. Second, conflict imagery is often degraded by natural distortions such as low lighting, smoke, motion blur, compression artifacts, partial occlusion, and damaged sensor output, all of which can resemble deepfake traces and complicate classification.

A major challenge is that many manipulated images are not visually absurd; they are contextually false. For example, an image may depict a real location, real equipment, or real military assets, but be paired with incorrect captions, fabricated timestamps, or misleading narratives. In such cases, a purely visual detector may fail, while a multimodal or reasoning-based model may succeed. Another challenge is that conflict imagery is often shared rapidly across social media before reliable verification is available, requiring systems that are both accurate and computationally practical.

LLMs introduce both promise and risk in this setting. They can detect semantic inconsistency, implausible object relationships, or scene-level contradictions, but they may also exhibit uncertainty, hallucination, or overconfidence when evidence is incomplete. For this reason, a single LLM cannot be relied upon as the sole authority. A robust deepfake image detection framework must combine visual forensics, semantic alignment, and reasoning-driven verification.

\section{Existing Methods and Reported Performance}
\label{sec:image-existing}

Current methods for image authenticity verification can be grouped into three main categories: forensic CNN models, vision-language consistency methods, and LLM-based reasoning approaches. CNN-based methods such as ResNet, Xception, and EfficientNet are widely used for detecting manipulation artifacts in pixels, edges, and texture patterns. These methods are effective when the forgery leaves traces in the image formation process, but their accuracy often drops when the image is heavily compressed or generated by advanced diffusion models.

Multimodal approaches such as CLIP and BLIP compare the alignment between image content and associated textual descriptions. These models are useful when authenticity depends on whether the image matches its caption, surrounding post text, or claimed narrative. However, such systems can show bias toward visually plausible but semantically incorrect content, especially when the caption itself is ambiguous or intentionally misleading.

LLM-based visual reasoning systems, including Gemini-style multimodal analyzers, can generate a structured judgment based on visible clues such as inconsistent lighting, unnatural shadows, object deformation, and implausible scene composition. Their major advantage is explainability, but they remain sensitive to prompt design and may produce uncertain outputs when evidence is weak.

Based on the evaluation of 25 conflict-related images, the observed performance of the modules is summarized in Table~\ref{tab:module-performance}.

\begin{table}[H]
\centering
\caption{Individual Module Performance on 25 Conflict Images}
\label{tab:module-performance}
\begin{tabular}{lcccc}
\toprule
\textbf{Module} & \textbf{Accuracy} & \textbf{Coverage} & \textbf{Strengths} & \textbf{Weaknesses} \\
\midrule
CNN Forensic Detector & 76\% & 100\% & Robust ensemble features & Adversarial vulnerability \\
LLM Visual Reasoning & 80\% & 100\% & Best standalone reasoning & Probabilistic uncertainty \\
CLIP + BLIP & 62.5\% & 96\% & Multimodal consistency & Systematic REAL bias \\
LLM Text Analysis & 100\% & 48\% & Perfect accuracy & Limited availability \\
\bottomrule
\end{tabular}
\end{table}

The table shows that no individual module is sufficient on its own. The CNN detector offers broad coverage but suffers from adversarial sensitivity. The LLM visual module performs better but remains probabilistic. The CLIP+BLIP module provides semantic matching but exhibits a bias toward real-looking outputs. The LLM text module is highly accurate when applicable, but it covers only a portion of the samples. This motivates a hierarchical ensemble strategy.

\section{Proposed Hierarchical Ensemble Framework}
\label{sec:image-framework}

The proposed framework uses a three-level decision hierarchy to combine the strengths of all modules while minimizing their weaknesses. Instead of averaging all scores blindly, the system prioritizes the most reliable signal available at each stage of analysis. This design improves interpretability and allows the system to remain accurate under heterogeneous evidence conditions.

The architecture is organized as follows:

\begin{enumerate}[label=\arabic*.]
    \item \textbf{Level 1: LLM Text Override} --- if trustworthy text analysis is available and agrees with at least one visual module, it is given primary authority.
    \item \textbf{Level 2: Uncertain Visual Zone} --- if the LLM signal is ambiguous, majority voting is applied across CNN, LLM visual, and CLIP-based outputs.
    \item \textbf{Level 3: Weighted Fusion} --- if a strong LLM visual signal exists, the final score is obtained from a weighted fusion of LLM and CNN outputs.
\end{enumerate}

This hierarchical structure is preferable to naive averaging because it explicitly encodes decision priority. In conflict imagery, not all modalities are equally reliable in every case. Textual claims may be decisive when present, visual forensics may be decisive when artifacts are visible, and multimodal alignment may resolve borderline cases where the image content appears plausible but the contextual claim is weak.

\section{Level 1: LLM Text Override}
\label{sec:level1}

Text-based analysis is the most decisive layer whenever image-related metadata, captions, or post text are available. This layer is particularly powerful in conflict settings, where images are often accompanied by captions that claim a specific location, event, or military operation. If the textual claim is factually impossible, contradictory, or semantically inconsistent with the visual content, the image can be rejected even when the image itself appears visually realistic.

The following rule governs the first level:

\begin{equation}
P_{\text{text}} > 0.9 \wedge \exists M_i : V_i = V_{\text{text}}
\label{eq:text-override}
\end{equation}

where $P_{\text{text}}$ is the confidence of the text analyzer, $M_i$ denotes any supporting module, and $V_i$ is the verdict of module $M_i$. If the text analyzer produces a highly confident verdict and at least one other module agrees, the text verdict is trusted.

This level fixed a substantial portion of visual-model errors in the evaluation. In particular, it handled the most confident textual cases correctly and helped resolve ambiguous images where the visual evidence alone was insufficient.

\section{Level 2: Uncertain Visual Zone}
\label{sec:level2}

When the LLM output falls within an ambiguous confidence interval, the system defers to majority vote. This prevents overreliance on uncertain semantic judgments and ensures that the final decision is grounded in consensus among independent modules.

The decision rule is:

\begin{equation}
V_2 = \text{Majority}(\text{CNN}, \text{LLM Visual}, \text{CLIP})
\label{eq:majority-vote}
\end{equation}

If there is a tie, the system resolves it using the CNN ensemble score. Specifically, if the CNN confidence is greater than or equal to 0.65, the image is classified as fake; otherwise, it is classified as real. This rule is useful because CNN-based detectors often provide strong local evidence when visual corruption exists.

\section{Level 3: Strong LLM Signal with Weighted Fusion}
\label{sec:level3}

If the LLM visual module returns a decisive fake or real probability, the system proceeds to weighted fusion. In this stage, the LLM output and CNN forensic output are combined, because these two modules capture complementary evidence: one reasons semantically, while the other detects forensic traces.

The fusion score is defined as:

\begin{equation}
S_3 = 0.55 \times P_{\text{LLM fake}} + 0.45 \times S_{\text{CNN}}
\label{eq:weighted-fusion}
\end{equation}

where $P_{\text{LLM fake}}$ is the fake probability from the LLM visual module and $S_{\text{CNN}}$ is the CNN forensic score. The final verdict is then:

\begin{equation}
V_3 =
\begin{cases}
\text{FAKE}, & S_3 \geq 0.56 \\
\text{REAL}, & \text{otherwise}
\end{cases}
\label{eq:final-verdict}
\end{equation}

This threshold was selected empirically based on the behavior of the conflict-image evaluation set. It provides a useful balance between sensitivity and precision in scenarios where strong visual suspicion exists.

\section{Module Implementations}
\label{sec:image-implementations}

\subsection{CNN Forensic Ensemble}
\label{subsec:cnn-ensemble}

The CNN forensic module consists of a three-model ensemble built from ResNet18 and two Xception variants. Each model processes the image after architecture-specific preprocessing and returns real and fake class probabilities using a softmax layer. The final ensemble score is computed by averaging the real and fake probabilities across the three models.

\begin{verbatim}
def cnn_forensic_ensemble(image_path):
    models = {
        'resnet18': load_model('resnet18_forensic.pth'),
        'xception1': load_model('xception_c23.pth'),
        'xception2': load_model('xception_c40.pth')
    }

    real_scores, fake_scores = [], []
    for name, model in models.items():
        img = preprocess_for_model(image_path, model.input_size)
        probs = F.softmax(model(img), dim=1)
        real_scores.append(probs[0,1].item())
        fake_scores.append(probs[0,0].item())

    cnn_real = np.mean(real_scores)
    cnn_fake = np.mean(fake_scores)

    return {
        'real': cnn_real,
        'fake': cnn_fake,
        'verdict': 'FAKE' if cnn_fake > cnn_real else 'REAL'
    }
\end{verbatim}

This model is effective because the ensemble reduces the weakness of any single backbone. ResNet18 provides a lightweight baseline, while Xception contributes stronger artifact detection under forged image conditions. In the sample analysis, the module produced the result:

\begin{quote}
CNN Forensic: REAL 47.58\% \quad | \quad FAKE 52.42\% \quad \rightarrow \quad FAKE
\end{quote}

\subsection{LLM Visual Reasoning}
\label{subsec:llm-visual}

The LLM visual reasoning module uses Gemini 2.5 Flash to inspect the image and return structured JSON containing the probabilities and explanation. The model is prompted to focus on visual cues such as lighting consistency, texture anomalies, composition artifacts, object realism, and scene plausibility.

\begin{verbatim}
def llm_visual_analysis(image_path):
    prompt = """
    Return ONLY JSON: {"real_probability": float,
    "fake_probability": float, "verdict": "REAL"/"FAKE",
    "analysis": "explanation"}
    Focus: lighting, textures, composition artifacts.
    """
    response = gemini_model.generate_content([prompt, Image.open(image_path)])
    result = json.loads(extract_json(response.text))
    return result
\end{verbatim}

This module is valuable because it can explain why an image appears suspicious. The sample output from the evaluation was:

\begin{quote}
\texttt{\{}
\texttt{"real\_probability": 0.0,}
\texttt{"fake\_probability": 1.0,}
\texttt{"verdict": "FAKE",}
\texttt{"analysis": "Inconsistent runway lighting, smooth aircraft details, implausible craters"}
\texttt{\}}
\end{quote}

The module achieved 80\% accuracy in the evaluation and was the best standalone reasoning-based image detector in the chapter.

\subsection{CLIP+BLIP Multimodal Analysis}
\label{subsec:clip-blip}

The CLIP+BLIP module combines automatic caption generation with image-text similarity scoring. First, BLIP generates a caption from the image itself. Then CLIP computes similarity scores between the image and both the user-provided caption and the auto-generated caption. These values are fused into a consistency estimate.

\begin{verbatim}
def clip_blip_analysis(image, caption):
    blip_caption = blip_model.generate(image)

    sim_given = clip_similarity(image, caption)
    sim_auto = clip_similarity(image, blip_caption)

    consistency = (sim_given + sim_auto) / 2
    real_prob = (consistency + 0.3) / 1.3
    return {'real': real_prob, 'fake': 1 - real_prob}
\end{verbatim}

This module is useful for detecting image-caption mismatch. However, the evaluation showed a systematic bias toward REAL predictions, which makes it unsuitable as a standalone verifier. In the experiments, this module achieved 62.5\% accuracy and 96\% coverage, but its reliability improved significantly when used as part of the hierarchy.

\subsection{LLM Text Analysis}
\label{subsec:llm-text}

The LLM text module is activated when surrounding text, captions, or claims are available. It analyzes factual consistency, temporal coherence, and claim plausibility. Although not always available, this module is extremely powerful when its input exists.

In the evaluation, the module achieved 100\% accuracy but only 48\% coverage. This means it should be treated as a high-confidence authority when applicable, but not as a universal image detector. Its limited coverage is precisely why the hierarchical framework is necessary.

\section{25-Image Ensemble Analysis Results}
\label{sec:ensemble-results}

The final image authenticity performance was evaluated across 25 conflict-related images. The hierarchical architecture significantly improved reliability over individual modules, especially in difficult or ambiguous cases. The fusion process resolved all 25 images and produced a final accuracy in the range of 96--100\%.

\begin{table}[H]
\centering
\caption{Hierarchical Fusion Performance}
\label{tab:hierarchical-performance}
\begin{tabular}{lcccc}
\toprule
\textbf{Fusion Level} & \textbf{Accuracy} & \textbf{Coverage} & \textbf{Fixed Errors} & \textbf{Images} \\
\midrule
Level 1: LLM Text & 100\% & 48\% & 8 & 12/25 \\
Level 2: Majority Vote & 88\% & 52\% & 3 & 13/25 \\
Level 3: Weighted Fusion & 92\% & 100\% & 2 & 25/25 \\
\midrule
Total & 96--100\% & 100\% & 13 & 25/25 \\
\bottomrule
\end{tabular}
\end{table}

These results show that the first level fixed the most decisive portion of errors, the second level handled the ambiguous region, and the third level completed the classification of all remaining samples. The ensemble architecture therefore provides full coverage with superior reliability.

\section{Case Study: Sargodha Airbase Deepfake}
\label{sec:case-study}

A representative case from the evaluation set involves a Sargodha airbase image. The individual module outputs are shown below:

\begin{quote}
CNN Forensic: REAL 47.58\% \quad | \quad FAKE 52.42\% \quad \rightarrow \quad FAKE
\end{quote}

\begin{quote}
LLM Visual: REAL 0.00\% \quad | \quad FAKE 100.0\% \quad \rightarrow \quad FAKE
\end{quote}

\begin{quote}
CLIP+BLIP: REAL 65.20\% \quad | \quad FAKE 34.80\% \quad \rightarrow \quad REAL
\end{quote}

\begin{quote}
LLM Text: UNAVAILABLE
\end{quote}

Since the LLM text layer was unavailable, the system moved to the visual hierarchy. The LLM visual module gave a strong fake signal, which triggered the weighted fusion rule. The final score was computed as:

\begin{equation}
S_3 = 0.55 \times 1.0 + 0.45 \times 0.5242 = 0.7859
\label{eq:sargodha-score}
\end{equation}

Since $0.7859 \geq 0.56$, the final decision was \textbf{FAKE}. This case demonstrates that the ensemble can correctly resolve a real-world conflict-image deepfake even when one module disagrees.

\section{Novel Contributions}
\label{sec:novelty}

This work contributes several novel ideas to the literature on conflict-specific image authenticity analysis.

\begin{enumerate}[label=\arabic*.]
    \item A hierarchical 3-level decision architecture that prioritizes the most reliable evidence source instead of naively averaging all predictions.
    \item A structured LLM text override mechanism that gives high-confidence textual analysis primary authority when available.
    \item A documented analysis of systematic REAL bias in CLIP+BLIP, showing that multimodal systems require careful calibration.
    \item A conflict-domain-specific image analysis framework tailored to armed conflict imagery rather than general deepfake benchmarks.
    \item An explainable multimodal verification strategy that combines artifact detection, semantic validation, and reasoning-based outputs.
\end{enumerate}

These contributions are important because conflict imagery often contains ambiguous, partially true, or highly contextual content. A robust framework must therefore support both technical accuracy and operational explainability.

\section{Hardware and Deployment Requirements}
\label{sec:hardware}

The proposed framework is designed to run in both research and operational environments. The implementation requires Python 3.9, PyTorch 2.0, CUDA 11.8, FFmpeg 4.4, and standard deep learning and multimedia libraries. For practical deployment, the minimum recommended configuration is a GPU equivalent to RTX 3090 with 24 GB VRAM, 32 GB RAM, an 8-core CPU, and at least 2 TB of SSD storage.

For larger-scale deployment, especially in command-center or server environments, a GPU such as the NVIDIA A100 or H100 is recommended. Inference latency is dominated by the LLM-based component, while the CNN detector remains relatively fast. This means that the framework is suitable for near-real-time or batch-based authenticity screening, depending on available hardware.

\section{Final Results and Discussion}
\label{sec:discussion}

The complete evaluation shows that the proposed hierarchical ensemble substantially outperforms its individual components. The CNN forensic detector provides broad coverage but limited robustness. The CLIP+BLIP module adds semantic awareness but suffers from bias. The LLM visual analyzer improves reasoning quality, while the LLM text analyzer achieves perfect accuracy when coverage is available. Their combination through hierarchical fusion produces the strongest final result.

The key conclusion is that no single detector dominates all cases. Instead, the best strategy is to use a decision hierarchy that assigns authority based on signal reliability and availability. This is especially important in armed conflict settings, where false positives can have severe strategic consequences and false negatives can allow harmful misinformation to spread.

\section{Conclusion}
\label{sec:image-conclusion}

This chapter presented a hierarchical framework for image authenticity analysis in armed conflict environments. The proposed system integrates three complementary submodules: a CNN forensic ensemble, a CLIP+BLIP multimodal consistency analyzer, and an LLM-based visual reasoning model, with an optional text override layer when contextual text is available. Across 25 conflict-related images, the system achieved 96--100\% overall accuracy through structured hierarchical fusion.

The major insight from this work is that conflict-image verification must be treated as a multimodal reasoning problem rather than a simple binary visual classification task. The proposed system addresses this challenge by combining artifact detection, semantic validation, and reasoning-based verification in a single explainable framework. This makes it suitable for deployment in high-stakes geopolitical environments where image authenticity must be assessed quickly, accurately, and transparently.

\end{document}