LLM Defense
Evaluation Framework

Executive Summary

The rapid proliferation of open-source large language models — Llama 3, Mistral, Falcon, Phi, and their continuously evolving derivatives — has created an unprecedented opportunity and a complex challenge for defense organizations. These models offer the ability to deploy sovereign AI capabilities within classified enclaves, on air-gapped networks, and in operational environments where commercial API-based LLMs are not viable. They also introduce evaluation obligations that defense organizations are not yet equipped to meet: the commercial model evaluation literature does not translate directly to environments where output failures can affect intelligence assessments, mission planning, force protection, or acquisition decisions.

This white paper, authored by Kurt A. Richardson, PhD, presents the LLM Defense Evaluation Framework (LDEF) — a structured, five-dimension methodology for evaluating open-source LLMs for defense deployment. LDEF addresses the unique evaluation requirements of defense contexts: quality assurance that reflects operational accuracy requirements, not just benchmark leaderboard rankings; fairness analysis that identifies demographic and entity-level biases with operational consequence; security evaluation that goes beyond prompt injection testing to assess model backdoor risks, supply chain integrity, and fine-tuning poisoning vulnerabilities; operational performance characterization under the resource constraints of classified environments; and governance assessment for RMF/ATO compliance.

Introduction: The Open-Source LLM Opportunity

Commercial LLM services — GPT-4, Claude, Gemini — have demonstrated transformative potential for productivity, analysis, and decision support across virtually every domain of human activity. For defense organizations, their use is constrained by a fundamental architecture problem: commercial LLMs are cloud-hosted services that require data to leave the organization's control, traverse public networks, and be processed on infrastructure the organization does not own. For classified programs, sensitive intelligence analysis, and operational planning, this architecture is categorically unacceptable.

Open-source LLMs — models whose weights are publicly available for download, deployment, and modification — solve this architecture problem. A Llama 3.1-70B model can be deployed on government-owned hardware, inside a classified enclave, on an air-gapped network, with full control over data handling, logging, and access. No data leaves the boundary. No external API call is made. The model is the government's to configure, fine-tune, and operate as a sovereign AI capability. This is a genuinely transformative opportunity for defense AI adoption.

The Evaluation Gap

The evaluation challenge is as significant as the opportunity. Open-source LLMs arrive without the testing infrastructure, safety tuning validation, or operational performance documentation that would accompany a commercial enterprise software procurement. The model's license may be permissive; its evaluation documentation is typically absent. What evaluation exists — benchmark scores on academic datasets — was designed to compare models in commercial development contexts, not to assess whether a model is safe to deploy for defense decision support.

NIST AI RMF requires that AI systems deployed in government contexts be evaluated against quantified performance, fairness, and reliability criteria before deployment, and monitored continuously during operation. CDAO's Responsible AI guidance requires documented evaluation of AI systems for bias, reliability, and security. DoD Directive 3000.09 (Autonomous Weapons) creates evaluation obligations for AI systems involved in targeting-adjacent decisions. Meeting these obligations requires a defense-specific evaluation framework — which is what LDEF provides.

Relationship to the Continuum Research Series

LDEF is the evaluation counterpart to the adversarial security framework in WP-CR-2025-04 (Prompt Injection & Adversarial Attacks). Where WP-04 focuses on protecting deployed LLMs against adversarial manipulation, LDEF focuses on evaluating candidate LLMs before deployment to determine whether they meet the performance, safety, and security thresholds required for their intended use case. Together, they form the pre-deployment evaluation + operational security architecture for defense LLM programs.

Why Open-Source LLMs for Defense

The case for open-source LLMs in defense deployments rests on five operational advantages that commercial API-based LLMs cannot match. Each advantage has direct implications for the evaluation framework required.

The Defense Open-Source LLM Landscape

The Five Evaluation Dimensions

LDEF organizes LLM evaluation for defense deployment across five dimensions that together constitute a comprehensive pre-deployment assessment. Each dimension addresses a different failure mode that could affect operational outcomes or create legal, ethical, or national security risks.

How LDEF Differs from Commercial Evaluation

Dimension 1: Quality Assurance

Quality assurance in LDEF measures whether a model produces outputs that are sufficiently accurate, well-reasoned, and calibrated for the specific defense tasks it will be asked to perform. The key insight is that task-specific quality measurement must replace — or heavily supplement — general benchmark scores. A model's MMLU score predicts its performance on academic multiple-choice questions; it does not predict its performance on intelligence analysis summarization, DFARS compliance reasoning, or military doctrine question answering.

Factual Accuracy and Hallucination Rate

Hallucination — the model generating confident, fluent, but factually incorrect outputs — is the most consequential quality failure mode for defense applications. An LLM that hallucinates a citation, an operational procedure, a weapon system capability, or a regulatory requirement does so with the same fluency and confidence as a correct output. There is no syntactic signal that distinguishes a hallucinated fact from a recalled one. Defense evaluation must quantify the hallucination rate for the specific domains relevant to the deployment use case.

LDEF measures hallucination using three complementary approaches:

• Closed-book factual probing: A curated set of domain-specific factual questions with known, verified answers drawn from authoritative defense sources (DoD Instructions, field manuals, acquisition regulations). The model is queried without context documents; responses are scored for factual accuracy against the gold standard.
• RAG faithfulness testing: The model is provided a retrieved document set and asked to answer questions. Responses are scored for faithfulness to the provided documents versus generation of content not in the context — a specific hallucination failure mode in RAG-based deployments relevant to the Secure RAG architecture in CR-04.
• Calibration measurement: The model is asked to express confidence in its answers, and that expressed confidence is compared to its actual accuracy across a large sample. A well-calibrated model that expresses 80% confidence is right approximately 80% of the time. An overconfident model that expresses 95% confidence but is only right 70% of the time is a decision-support liability.

Domain-Specific Task Performance

Quality evaluation must include task-specific performance benchmarks constructed for the program's specific use cases. LDEF defines four primary defense task categories, each requiring a purpose-built evaluation set:

Robustness to Distribution Shift

Defense operational environments change rapidly. A model evaluated in peacetime conditions may encounter significantly different input distributions in crisis conditions — novel terminology, degraded communications, fragmentary reporting, unfamiliar geographic contexts. LDEF tests robustness by evaluating models on out-of-distribution variants of the task evaluation sets: paraphrased questions, incomplete inputs, noisy text (OCR artifacts from scanned documents), and domain-adjacent questions that test whether the model generalizes appropriately or fails in unexpected ways.

Dimension 2: Fairness & Bias

Fairness evaluation for defense LLM deployment extends well beyond the demographic bias analysis that characterizes commercial AI fairness assessments. In defense contexts, the most consequential bias categories are geopolitical, entity-level, and operational — biases that could cause the model to systematically misrepresent adversary capabilities, undervalue allied contributions, stereotype foreign military forces, or apply different analytical standards to different nations or organizations.

Defense-Specific Bias Categories

Bias Measurement Methodology

LDEF measures geopolitical and military bias through counterfactual substitution: identical analytical queries are posed with different nation, force, or entity substitutions, and response quality, completeness, and characterization are compared systematically. For example, a query asking for an analysis of a specific military capability is posed with multiple national subjects (U.S., allied nations, adversary nations), and the responses are evaluated for consistency in analytical depth, factual accuracy, and characterization quality. Systematic differences — consistently shallower analysis for one national category, consistently more negative characterization for another — indicate a bias requiring mitigation before operational deployment.

Demographic Fairness in Defense Contexts

Beyond geopolitical bias, LDEF also addresses traditional demographic fairness — whether the model produces different quality outputs for queries involving different demographic groups of service members, veterans, or civilians. Biased outputs in personnel assessment, medical triage support, administrative processing, or casualty analysis have direct human consequences and legal obligations under DoD equal opportunity and non-discrimination policy.

Dimension 3: Security Evaluation

Security evaluation of open-source LLMs for defense deployment must address threat categories that do not apply to commercial API-based models. Because the model weights are public, any adversary can study them. Because fine-tuning modifies the weights, a supply chain attack on the fine-tuning process can embed persistent behaviors that survive all other safety controls. Because the model will be deployed on classified networks, the consequences of a security failure extend beyond information exposure to potential intelligence compromise.

The Open-Source Security Threat Landscape

• Pre-trained weight integrity: The weights of widely used open-source models are hosted on repositories (HuggingFace, model-specific releases) that could be compromised by an adversary replacing authentic weights with modified versions containing backdoors or degraded capabilities. Supply chain verification — cryptographic hash verification of downloaded weights against authenticated checksums published by the model's authors — is a required pre-deployment step for all LDEF evaluations.
• Backdoor vulnerabilities: Research has demonstrated that LLMs can be fine-tuned to exhibit specific behaviors (producing harmful outputs, bypassing safety controls, returning attacker-specified content) when triggered by specific input patterns, while appearing normal on all other inputs. For a defense deployment using a community fine-tuned model, the risk that the fine-tuning introduced a backdoor requires explicit backdoor detection evaluation.
• Training data memorization: LLMs memorize portions of their training data and can be induced to reproduce them. If the training data included sensitive information — personally identifiable information, proprietary organizational data, operationally sensitive content from open-source repositories — the model may reproduce it under targeted extraction prompts. Memorization testing is a required LDEF evaluation step for models that will be fine-tuned on sensitive data.
• Adversarial input resilience: As detailed in WP-CR-2025-04, open-weight models are susceptible to adversarial suffix attacks generated using white-box gradient access. For operational deployments, the resistance of the deployed model to known adversarial attack techniques — including program-office-specific red-team generated attacks — must be characterized before deployment.

Security Evaluation Protocol

Dimension 4: Operational Performance

Operational performance evaluation assesses whether the candidate model can deliver acceptable quality of service within the hardware and infrastructure constraints of the intended deployment environment. Defense environments impose constraints that commercial LLM deployments do not face: government-approved computing hardware with specific approved products lists, classified network bandwidth limitations, air-gapped deployment without cloud scaling, forward-deployed disconnected operations, and continuous operation requirements without commercial maintenance windows.

Hardware Constraint Evaluation

The primary operational challenge for large open-source models is that the highest-performing variants (70B+ parameters) require specialized high-memory GPU hardware that is not universally available on DoD networks. LDEF evaluates models across a hardware tiering that reflects the actual deployment environments:

Quantization Quality Degradation Assessment

Most tactical and edge deployments require model quantization — reducing model weights from 16-bit floats to 8-bit or 4-bit integers to fit within available VRAM. Quantization reduces model size and inference cost but may degrade quality on specific tasks. LDEF requires quantization quality degradation testing: the full-precision model is evaluated on the defense task suite, then quantized versions are evaluated on the same suite, and the quality delta is measured. If quality degradation exceeds the program's acceptable performance threshold at a given quantization level, that quantization level is rejected for that use case, and a smaller full-precision model may be preferred.

Latency and Throughput Requirements

Operational performance requirements vary significantly by use case. A document summarization tool for analysts can tolerate 30–60 second latency for a complex summary; a real-time conversational assistant for an operational planning session requires sub-5-second first-token latency. LDEF specifies that latency and throughput requirements must be defined as part of the use case specification before model evaluation begins, and candidate models are evaluated against these requirements on the target hardware tier — not on benchmark cloud hardware.

Dimension 5: Governance & Compliance

Governance evaluation ensures that the candidate LLM and its deployment architecture are consistent with the legal, regulatory, and policy frameworks that govern defense AI deployment. This dimension does not evaluate the model itself — it evaluates the program's documentation, process controls, and monitoring posture that together constitute the governance architecture around the model.

NIST AI RMF Alignment

NIST AI RMF 1.0 defines four core functions — GOVERN, MAP, MEASURE, MANAGE — that together constitute a risk management approach for AI systems. LDEF maps the evaluation activities across all five dimensions to the NIST AI RMF MEASURE function requirements, providing programs with documentation that directly supports RMF compliance assertions:

• GOVERN: LLM evaluation policy documented; roles and responsibilities for AI evaluation team defined; escalation procedures for evaluation findings established; executive accountability for deployment decisions documented.
• MAP: Use case context documented; affected stakeholders identified; risk categories and tolerance levels established; deployment context constraints documented.
• MEASURE: All five LDEF dimensions evaluated with quantified metrics; results compared against defined pass/fail thresholds; evaluation methodology documented for reproducibility.
• MANAGE: Deployment decision documented with specific risk acceptance rationale; continuous monitoring plan established; incident response procedures documented; model update and revaluation triggers defined.

License Compliance for Defense Deployment

Open-source model licenses impose use restrictions that must be reviewed by program legal counsel before deployment. The Llama Community License restricts use by organizations with over 700 million monthly active users (not relevant to DoD programs) but has specific terms about disclosure of derivative model development. Apache 2.0 (Mistral, Phi-4) provides broad commercial and government use rights. Custom licenses (Falcon) require specific legal review. Programs must document license compliance as part of the governance assessment, particularly when fine-tuning creates derivative models that may have additional disclosure obligations.

CDAO Responsible AI Principles

The Chief Digital and Artificial Intelligence Office's Responsible AI (RAI) framework requires defense AI systems to be evaluated against five principles: responsible, equitable, traceable, reliable, and governable. LDEF maps to these principles as follows: fairness evaluation → equitable; quality assurance + security evaluation → reliable; hallucination measurement + calibration → traceable; governance evaluation → governable; complete LDEF evaluation documentation → responsible deployment.

Defense Benchmark Suite

The LDEF Defense Benchmark Suite (LDBS) is a purpose-built collection of evaluation datasets and tasks designed specifically to measure LLM performance on defense-relevant activities. Unlike commercial benchmarks that aggregate performance across broad academic domains, LDBS tasks are directly tied to operational use cases and are evaluated by domain experts who can assess the real-world quality of model outputs.

LDBS Task Taxonomy

Constructing Program-Specific Evaluation Sets

LDBS provides the structural template and baseline evaluation sets. Every program should supplement LDBS with program-specific evaluation sets constructed from:

• Representative samples of the actual documents the model will process in the deployed system
• Historical expert-produced outputs for the tasks the model will assist with (analyst reports, requirements documents, acquisition packages), used as gold standards
• Domain expert-constructed adversarial test cases that probe the specific failure modes most consequential for the program's mission
• Historical incidents or near-misses from analogous programs that represent the failure modes the evaluation is designed to prevent

Interactive LLM Comparison Matrix

The following matrix provides a comparative view of leading open-source LLMs across LDEF dimensions, scored against defense-relevant criteria. Filter by evaluation dimension to focus the comparison. Scores represent Continuum's assessment based on available public evaluation data, published research, and operational experience — they are not official government ratings. Program-specific evaluation may yield different results.

Red-Team Evaluation Methodology

Red-team evaluation of LLMs for defense deployment is a structured adversarial exercise in which a team of experts attempts to cause the candidate model to produce outputs that would be unacceptable in the operational context. The methodology draws on the WP-CR-2025-04 adversarial attack taxonomy, adapted for the specific use case and deployment context of the program being evaluated. Red-team evaluation is required for all Tier 1 and Tier 2 defense LLM deployments and recommended for any deployment with access to sensitive operational data.

Defense Red-Team Scope

• Operational context attacks: Red teamers simulate users with legitimate but misused access — an analyst attempting to extract information outside their access authorization, a contractor attempting to use the system for competitive intelligence gathering, or an insider threat attempting to elicit classified information from an AI system trained on classified data.
• Adversarial content injection: Red teamers simulate adversary-controlled content entering the model's context — through documents, email, retrieved web content, or user-submitted materials that contain embedded instructions. This directly applies the indirect injection taxonomy from WP-04 to the specific document types and ingestion pathways of the program's RAG architecture.
• Misinformation and disinformation probing: Red teamers attempt to cause the model to generate content that, if acted upon, would produce incorrect operational outcomes — false analytical conclusions, erroneous regulatory interpretations, incorrect technical specifications. This is distinct from jailbreaking: the goal is not prohibited content but subtly incorrect content that passes superficial quality review.
• Guardrail boundary testing: Red teamers systematically probe the model's content safety boundaries with variants of queries near the refusal threshold, using paraphrasing, role-play framing, and hypothetical framing to identify inconsistencies in the model's safety behavior.

Red-Team Composition for Defense Programs

A defense LLM red-team requires three distinct expertise categories that are rarely co-located in existing program offices:

• AI security specialists: Researchers with specific expertise in adversarial ML, LLM attack techniques, and the tooling required for white-box and black-box adversarial testing. Responsible for the technical security evaluation components.
• Domain experts: Subject matter experts in the mission area the LLM will support — intelligence analysts for intelligence support tools, acquisition professionals for acquisition support tools, military doctors for medical support tools. Responsible for identifying the failure modes that would have operational consequence and that a non-domain expert would not recognize as failures.
• Operational security specialists: Personnel with specific expertise in the threats faced by the program's deployment environment — counterintelligence, information operations, insider threat — who can design red-team scenarios that reflect realistic adversary capabilities and intent.

Red-Team Documentation Requirements

LDEF requires that all red-team findings be documented in a standardized format that includes: the attack technique used, the specific input that caused the failure, the model's output, the expert panel's assessment of operational consequence, the LDEF dimension and sub-dimension the finding falls under, and a remediation recommendation. Red-team findings that rise to the level of Critical or High severity are tracked through a formal remediation process with defined SLAs before deployment is authorized.

ATO & Accreditation Pathway

Authority to Operate for AI systems — and specifically for LLM components within larger systems — is an evolving area where the traditional RMF process was not designed with the unique characteristics of machine learning systems in mind. LDEF provides a structured approach for integrating LLM evaluation documentation into the RMF/ATO package, addressing the specific questions that authorizing officials are likely to ask about AI system risk.

LDEF Documentation Package for ATO

Continuous Monitoring for LLM Systems

ATO for LLM systems requires a continuous monitoring approach that is qualitatively different from traditional continuous monitoring: in addition to infrastructure monitoring (vulnerability scanning, configuration management), programs must monitor model performance — whether the model's output quality is drifting from the evaluation baseline. Model drift can occur without any infrastructure change: as the operational input distribution shifts (new document types, new query patterns, seasonal variation in topics), model performance may degrade in ways that are invisible to infrastructure monitoring but detectable through ongoing sampling and evaluation.

LDEF recommends a minimum continuous monitoring cadence of: quarterly automated benchmark re-evaluation against LDBS core tasks; monthly sampling and expert review of a random 1% of operational outputs; real-time anomaly detection on output quality indicators (response length distribution, refusal rate, sentiment); and immediate re-evaluation triggered by any significant change to the model, its fine-tuning, its retrieval corpus, or its deployment configuration.

Interactive Evaluation Checklist

The following checklist operationalizes all five LDEF dimensions into a structured evaluation workflow. Click items to mark complete, right-click to mark as failed (requiring remediation), and double-click to mark N/A. The progress bar tracks completion across all dimensions. This checklist is designed as a program office tracking tool — not a substitute for the full LDEF evaluation methodology.

Defense Use-Case Analysis

Different defense use cases impose different evaluation priorities. A model adequate for logistics planning support may be inadequate for intelligence analysis support; a model that passes evaluation for unclassified document summarization may fail evaluation for a deployment with access to sensitive acquisition data. This section provides use-case-specific evaluation guidance for the four primary defense LLM deployment contexts.

Use Case 1: Intelligence Analysis Support

Intelligence analysis support is the highest-consequence defense LLM use case — outputs that inform decision-makers about adversary capabilities, intentions, and activities. The LDEF priorities for this use case are: hallucination rate (must be the lowest of any use case — the minimum acceptable is substantially below 5%); calibration (the model must accurately represent its uncertainty — overconfident wrong answers in an intelligence context can directly influence operational decisions); geopolitical bias (systematic bias in adversary characterization is an intelligence analysis failure); and source attribution (the model must correctly trace analytical claims to source documents, not confabulate citations).

Recommended model profile for intelligence analysis: a larger model (70B+) fine-tuned on declassified intelligence analysis tradecraft documentation, with RAG grounding to limit hallucination, deployed in a Tier 1 or Tier 2 environment, with mandatory human review of all analytical outputs before dissemination. Open-source candidates: fine-tuned Llama 3.1-70B or Mixtral 8×22B with domain adaptation.

Use Case 2: Acquisition and Contract Support

Acquisition and contract support involves the model assisting with FAR/DFARS compliance, contract clause interpretation, source selection analysis, and acquisition package development. The highest-consequence failure modes are regulatory misinterpretation (a model confidently providing incorrect regulatory guidance that is acted upon) and bias in source selection analysis (systematic bias in evaluation of contractor proposals that could affect contract outcomes).

LDEF priorities for this use case: accuracy on LDBS-QA regulatory tasks (hard pass/fail threshold for any legal error); entity bias in contractor evaluation (counterfactual testing with different company names, sizes, and ownership categories); and calibration on regulatory questions (the model should express uncertainty on edge cases, not produce confident wrong interpretations). Continuum's operational experience in acquisition support programs directly informs this use case evaluation.

Use Case 3: Technical Requirements and Systems Engineering

Technical requirements generation, review, and traceability support — directly tied to the Embedding-Driven Requirements Management framework in WP-CR-2025-07. For this use case, LDEF priorities are: MBSE terminology accuracy; INCOSE requirements quality criteria compliance; ability to identify ambiguous or inconsistent requirements; and performance on LDBS-RE tasks. The primary failure mode is requirements that appear well-formed but are subtly ambiguous, which a domain expert would catch but a general LLM might miss.

Use Case 4: Disconnected / Tactical Edge Deployment

Tactical edge deployment — an LLM running on a ruggedized laptop or tactical server in a forward operating environment without network connectivity — imposes the most severe operational performance constraints. LDEF priorities: performance on Tier 4/5 hardware (quantization quality under severe quantization); ability to operate from a local context window without RAG (the retrieval corpus cannot be updated in disconnected operations); robustness to degraded input quality (partial, fragmentary, or OCR-impaired documents); and very small model footprint without catastrophic quality degradation.

Recommended candidates for tactical edge: Phi-3-mini (3.8B) or Phi-4-mini at int4 quantization for CPU deployment; Llama 3.2-3B for GPU-assisted tactical systems. These models are substantially less capable than 70B variants but may be sufficient for specific tactical use cases when evaluated against task-specific criteria rather than general benchmarks.

Implementation Roadmap

Building an LDEF evaluation capability within a program office is a phased investment. Programs typically begin with an urgent need to evaluate a specific model for a specific deployment and discover that they lack the infrastructure, methodology, and evaluation datasets to do so rigorously. This roadmap builds the evaluation capability in parallel with the first program-specific evaluation.

The Continuum Approach

Continuum Resources developed the LDEF methodology from the intersection of three capabilities: the AI technical research program directed by Kurt A. Richardson, PhD; the operational AI deployment experience across Continuum's defense program engagements where LLM evaluation questions arise in practical context; and the adversarial security research documented in WP-CR-2025-04. LDEF is not an academic framework constructed for publication — it is the evaluation methodology Continuum applies to LLM selection decisions on active defense programs.

Engagement Models

Conclusion

Open-source large language models have created a genuine strategic opportunity for defense AI: sovereign, air-gappable, fine-tunable AI capability that can be deployed at any classification level without dependence on external commercial providers. Realizing this opportunity responsibly requires evaluation infrastructure and methodology that the defense community is still in the early stages of building. The commercial model evaluation literature is a starting point, not an answer — it was designed to rank models on academic benchmarks, not to assess whether a model is safe and reliable for intelligence analysis, acquisition support, or mission planning.

The LDEF provides a defense-specific answer to the evaluation question: five dimensions that together constitute a comprehensive pre-deployment assessment — quality assurance that measures operational task performance, not just leaderboard rank; fairness analysis that addresses geopolitical and military entity bias, not just demographic parity; security evaluation that includes backdoor detection and supply chain integrity, not just jailbreak resistance; operational performance characterization against government hardware tiers, not commercial cloud benchmarks; and governance documentation that supports RMF/ATO, not just terms-of-service compliance.

References

• [NIST-AI-RMF] National Institute of Standards and Technology — "AI Risk Management Framework (AI RMF 1.0)" — NIST AI 100-1, January 2023. The governing framework for LDEF Dimension 5 governance evaluation.
• [NIST-AI-100-5] National Institute of Standards and Technology — "Adversarial Machine Learning: A Taxonomy and Terminology" — NIST AI 100-4, 2024. Taxonomy basis for LDEF Dimension 3 security evaluation.
• [CDAO-RAI] Chief Digital and Artificial Intelligence Office — "Responsible AI (RAI) Strategy and Implementation Pathway" — CDAO, 2022. CDAO RAI principles mapped to LDEF dimensions throughout.
• [DOD-AI-ETHICS] Department of Defense — "DoD AI Ethics Principles" — CDAO, February 2020. Five AI ethics principles providing normative basis for LDEF fairness and governance evaluation.
• [DODD-3000-09] Department of Defense Directive 3000.09 — "Autonomous Weapons Systems" — DoD, January 2023. Evaluation obligations for AI systems involved in targeting-relevant decisions.
• [LIANG-2022] Liang, P. et al. — "Holistic Evaluation of Language Models (HELM)" — Stanford CRFM, NeurIPS 2022. The most comprehensive academic LLM evaluation framework; LDEF extends HELM methodology to defense-specific dimensions.
• [SRIVASTAVA-2023] Srivastava, A. et al. — "Beyond the Imitation Game: Quantifying and Extrapolating the Capabilities of Language Models (BIG-Bench)" — TMLR, 2023. Multi-task benchmark framework informing LDBS task design methodology.
• [LIN-2022] Lin, S. et al. — "TruthfulQA: Measuring How Models Mimic Human Falsehoods" — ACL 2022. Hallucination measurement methodology adapted for defense domain in LDBS-IS.
• [BOMMASANI-2023] Bommasani, R. et al. — "On the Opportunities and Risks of Foundation Models" — Stanford CRFM, 2023. Foundation model risk taxonomy providing part of the basis for LDEF Dimension 3.
• [WEN-2023] Wen, Y. et al. — "Backdoor Attacks on Language Models" — IEEE S&P, 2023. Backdoor detection methodology applied in LDEF LDBS-SEC security probing.
• [CR-04] Richardson, K.A. — "WP-CR-2025-04: Prompt Injection & Adversarial Attacks on LLM Systems" — Continuum Resources, 2025. Adversarial attack taxonomy that provides the basis for LDEF red-team scope and LDBS-SEC test design.
• [CR-03] Richardson, K.A. — "WP-CR-2025-03: AI Governance for Federal Contractors" — Continuum Resources, 2025. Federal AI governance framework that LDEF Dimension 5 operationalizes for LLM-specific deployment evaluation.
• [OUYANG-2022] Ouyang, L. et al. — "Training Language Models to Follow Instructions with Human Feedback (InstructGPT)" — NeurIPS 2022. RLHF methodology context for understanding alignment tuning quality in evaluated models.