A Supplemental Chapter to Principles and Practices of Model Risk Management

PART II — GenAI Model Development and Independent Model Validation

Revised Version 6 — second-round editorial revision; author comments incorporated

Part II Overview

Part II has two purposes. First, it explains how a GenAI model/system is developed. Second, it explains how an independent validator should validate the completed system. The two processes are related, but they should not be mixed. Development is the responsibility of the first line of defense, model developer and business owner. Validation is the responsibility of the second line of defense, an independent challenge function of the development process, the model/system, its performance, its implementation, and its controls.

The intended reader has basic training in statistics and quantitative modeling but does not need formal training in artificial intelligence. The discussion therefore uses the traditional model-development sequence—data, model structure, model selection, fitting, testing, tuning, limitations, controls, and monitoring—and then shows what changes when the model is a GenAI system.

It is important to emphasize it takes village to develop and validate a Gen AI as it takes more than a standard alone model

4. GenAI Model Development

A GenAI development process is more complex than the development of most traditional statistical models, but the basic discipline is familiar. A developer still needs to define the business purpose, prepare data, select a model structure, estimate or configure the model, test performance, identify limitations, establish controls, define ongoing performance monitoring, and maintain the model after deployment.

A practical development process is shown below.

4.1 Business purpose, intended use, and development scope

Development should begin with business need, not with the technology. The developer should state what the GenAI system is expected to do, who will use it, what decisions it may influence, what data it may access, and what it is not permitted to do. A summarization assistant, a policy Q&A system, a coding assistant, and an agent that can execute transactions are different development problems even if they use the same foundation model.

This first step also determines the appropriate system boundary. For GenAI, the model weights alone rarely describe the complete product. The developer should identify the components that materially affect behavior and risk.

4.2 Data preparation and data design

Data are part of model development, not an activity that occurs only before development. Part I discusses GenAI data risk in detail. Here the focus is narrower: what data decisions must be made in order to develop the system.

The developer should identify and test, as applicable:

• pre-training data and known limitations of the foundation model provider;

• fine-tuning or instruction-tuning data;

• preference/alignment data;

• development, test, and evaluation samples;

• RAG documents, metadata, embeddings, and retrieval indexes;

• runtime prompts, conversation history, and tool/API data; and

• production feedback that may later be used for tuning or evaluation.

For each material dataset, the developer should consider provenance, representativeness, quality, legal right to use, privacy, contamination, sampling design, transformations, and the separation of development from testing. This connects directly to Part I: data risk is not separate from model risk because the data determine what the model learns, what it retrieves, and what information it sees at inference time.

4.3 Model/system structure and components

Before model selection, the developer should describe the system to be built. A useful representation is:

\(S = (W,D,P,R,T,G,U,\varepsilon)\) (4.1)

where W is the foundation model and model version; D is development and runtime data; P is the prompt and configuration structure; R is retrieval/RAG; T is tools and external actions; G is guardrails (regulations, laws) and permissions; U is the use environment, including users and workflow; and ε represents stochastic generation effects.

Equation (4.1) is important because it shows that GenAI development is partly model development and partly system engineering. The same foundation model can produce materially different outcomes when P, R, T, G, or U changes.

4.4 Technical foundation: from text to tokens

Before tokenization and Transformer mechanics, it is useful to distinguish LLM fitting from traditional model estimation. In a regression or other parametric model, the analyst often estimates a relatively small parameter vector with an explicit statistical interpretation. An LLM instead contains a very large neural-network parameter vector—often billions of learned weights—distributed across embeddings, attention projections, feed-forward layers, normalization parameters, and related components. These parameters are fitted jointly from very large token datasets by minimizing a training loss with gradient-based optimization. The training loss is a numerical objective measuring the discrepancy between the model's predicted probability distribution and the observed training tokens; for autoregressive pre-training it is commonly token-level cross-entropy, equivalently negative log-likelihood. Individual parameters generally do not have standalone business interpretations; validation therefore focuses more on architecture, objective, data, empirical behavior, controls, and use-case performance than on coefficient-by-coefficient interpretation.

A useful abstraction is to collect the trainable parameters in a high-dimensional parameter set:

\[\theta = \left\{ W_{E},W_{Q},W_{K},W_{V},W_{O},W_{1},W_{2},\ldots \right\}\]

In this notation, W_E denotes the token-embedding parameters; W_Q, W_K, and W_V are the learned query, key, and value projection matrices used in self-attention; W_O is the learned attention output-projection matrix; and W_1, W_2, … denote learned weight matrices in feed-forward and other neural-network layers.

Given a training corpus containing N token sequences, training seeks parameters that minimize empirical loss over the corpus:

\[\widehat{\theta} = \underset{\theta}{arg\, min}\mspace{6mu}\frac{1}{N}\sum_{i = 1}^{N}\mathcal{L}\left( x^{(i)};\theta \right)\]

For next-token pre-training, the per-sequence loss is commonly cross-entropy/negative log-likelihood:

\[\mathcal{L}(x;\theta) = - \sum_{t = 1}^{n}\log P_{\theta}\left( x_{t} \mid x_{< t} \right)\]

Gradient-based optimization then updates the parameters iteratively. In simplified gradient-descent form:

\[\theta_{k + 1} = \theta_{k} - \eta_{k}\nabla_{\theta}\mathcal{L}\left( \theta_{k} \right)\]

Here η_k is the learning rate at iteration k. Practical LLM optimizers use adaptive learning rates, momentum-like mechanisms, regularization, mini-batching, distributed computation, and mixed-precision training. The important MRM distinction is that this is still parameter fitting, but at a scale and level of nonlinearity where traditional coefficient diagnostics are no longer sufficient.

Tokenization and the Transformer are technical components inside the development process. They are included here because a developer—and later a validator—needs enough understanding of the foundation model to assess conceptual soundness.

An LLM does not directly process a sentence as a human reads it. A tokenizer first converts text into discrete tokens. A token may be a word, part of a word, punctuation mark, number, or other symbol. Each token identifier is then mapped to a numerical vector called an embedding:

\(x_{i} \rightarrow e_{i} \in \mathbb{R}^{d}\) (4.2)

Here xᵢ is token i, eᵢ is its embedding vector, ℝᵈ is a d-dimensional real-valued vector space, and d is the embedding or hidden dimension. In modern LLMs, d is commonly in the thousands and varies by architecture. The exact size is less important for MRM than the implication: each token is represented by many numerical features rather than by one human-interpretable variable.

4.5 Transformer and self-attention

The Transformer architecture was introduced by Vaswani et al. (2017). Its central mechanism is attention. At a simplified level, the current token representations X are projected into Query, Key, and Value matrices:

\(Q = XW_{Q},\quad\quad K = XW_{K},\quad\quad V = XW_{V}\) (4.3)

A Query represents what information a token is looking for; a Key represents what information another token offers for matching; and a Value contains the information passed forward when that token receives attention. The projection matrices W_Q, W_K, and W_V are not calculated anew at inference time. They are learned model parameters. During pre-training, they are initialized and then updated by backpropagation and gradient-based optimization to reduce the training loss. At inference, the learned matrices are fixed for a given model version and are multiplied by the current hidden-state matrix X to produce Q, K, and V.

\(Attention(Q,K,V) = softmax\left( \frac{QK^{T}}{\sqrt{d_{k}}} \right)V\) (4.4)

QKᵀ is the matrix of query-key compatibility scores. The superscript T means matrix transpose. dₖ is the dimension of the key vectors. Dividing by √dₖ keeps the dot products from becoming excessively large as dimension increases.

\(softmax\left( z_{i} \right) = \frac{e^{z_{i}}}{\sum_{j}^{}e^{z_{j}}}\) (4.5)

( j<> i, in 4.5?) Softmax converts a row of raw scores into positive weights that sum to one. For example, raw scores [1, 2, 3] become approximately [0.09, 0.24, 0.67]. Attention therefore works like a context-dependent weighted average: the model gives more weight to tokens judged more relevant to the current token.

Multi-head attention applies this calculation several times with different learned projections:

\({head}_{h} = Attention\left( XW_{Q}^{(h)},XW_{K}^{(h)},XW_{V}^{(h)} \right)\) (4.6)

\(MHA(X) = Concat\left( {head}_{1},\ldots,{head}_{H} \right)W_{O}\) (4.7)

Concat means that the output vectors from the H attention heads are placed side by side to form one larger representation. W₀ (more commonly written W_O) is a learned output-projection matrix that maps the concatenated result back to the model's working dimension. Multiple heads allow the model to represent different relationships at the same time. Individual heads, however, should not automatically be assigned simple human meanings.

Purpose of the Transformer and self-attention. The Transformer provides a scalable way to represent context and dependencies among tokens without relying on strictly sequential recurrence. Self-attention allows each token representation to condition on other relevant tokens in the permitted context. This is essential for language because the meaning of a word, number, pronoun, or instruction often depends on context elsewhere in the prompt. In an autoregressive LLM, causal masking prevents the representation at position i from attending to future positions j > i. During training, this prevents leakage of the token the model is supposed to predict; during generation, it preserves the left-to-right next-token structure. The practical MRM implication is that output behavior is context-dependent: changing prompt wording, ordering, retrieved context, or system instructions can change attention patterns and therefore model output even when model weights are unchanged.

4.6 Feed-forward network, residual computation, and stacked blocks

After attention mixes information across tokens, a feed-forward neural network—often called a multilayer perceptron (MLP)—transforms each token representation.

\(MLP(x) = W_{2}\,\phi\left( W_{1}x + b_{1} \right) + b_{2}\) (4.8)

In Equation (4.8), x is the token representation entering the MLP; W₁ and W₂ are learned weight matrices; b₁ and b₂ are bias vectors; and φ is a nonlinear activation function such as Gaussian Error Linear Unit (GELU). The first layer commonly expands the representation into a larger hidden space; the nonlinear function transforms it; and the second layer projects it back. Attention can be viewed as information mixing across tokens, while the MLP performs nonlinear transformation within each token position.

Residual connections preserve the previous representation while adding the new transformation. Normalization helps stabilize the computation. Many Transformer blocks are stacked. The final behavior therefore emerges from repeated high-dimensional nonlinear transformations. Unlike a regression coefficient, an individual neural-network parameter normally has no standalone business interpretation.

4.7 Pre-training objective and model fitting

For an autoregressive LLM, model fitting during pre-training is commonly based on next-token prediction. Given tokens x₁,…,xₙ, a simplified negative log-likelihood loss is:

\(\mathcal{L}(\theta) = - \sum_{t = 1}^{n - 1}\log P_{\theta}\left( x_{t + 1} \mid x_{1},\ldots,x_{t} \right)\) (4.9)

θ denotes the model parameters. For an autoregressive model, the training loss is typically the average negative log-likelihood (equivalently, token-level cross-entropy) of the observed next tokens under the model. If the model assigns probability pθ(x_t | x_<t) to the observed token x_t, its contribution to loss is −log pθ(x_t | x_<t). A high probability assigned to the correct next token produces a small loss; a low probability produces a large loss. Training uses backpropagation to compute gradients of this loss with respect to θ and an optimizer such as Adam/AdamW to update the parameters. The objective is statistical prediction of tokens—not direct optimization of factuality, compliance, fairness, or business utility.

4.8 Model and component selection / conceptual soundness

Model selection in GenAI is not simply a choice among several foundation models. It can involve the complete system in Equation (4.1). The developer may need to select the foundation model and version, vendor update/version policy, hosting method, context-window size, embedding model, retrieval method, reranker, prompt structure, tool architecture, guardrails, and degree of human oversight. Update frequency is therefore a design and governance choice: frequent vendor/model updates may improve model capability but increase change risk, while pinned versions improve reproducibility but may delay security or capability improvements.

Conceptual soundness asks whether these choices are appropriate for the intended use. A more capable model is not automatically the better model. The selected system should provide enough capability while remaining controllable, testable, secure, and economically reasonable.

4.9 RAG development

Retrieval-Augmented Generation (RAG) combines an LLM with external information retrieved at inference time. RAG should not be treated as an unlimited or undefined information channel. For controlled financial-institution use, the approved retrieval scope should be defined by source classes, ownership, jurisdiction, access entitlement, freshness requirements, and permitted external resources. Open-web search or user-supplied content may be allowed for some use cases, but only as explicitly governed source classes with additional provenance, poisoning, confidentiality, misinformation, and prompt-injection controls. A reproducible RAG design therefore requires more detail than saying that documents are 'added to the model.' A simplified development process is:

For governance purposes, the pipeline can be organized into four stages: (1) knowledge-source governance—authorize sources, parse and clean content, chunk text, and attach metadata; (2) indexing—embed chunks and store/version the index; (3) retrieval—embed the query, retrieve top-k candidates, apply entitlement filters, and rerank; and (4) generation and verification—construct the prompt, generate the answer, cite sources, and verify or abstain where evidence is inadequate.( I would not use the word corpus, as it is more like an AI work instead of a commonly used word)

The twelve RAG steps are grouped below into four control stages to improve readability: knowledge-source governance; indexing; retrieval; and generation/verification.

The approved knowledge base D is first controlled for ownership, effective date, confidentiality, and access rights. Documents and other authorized sources are parsed and divided into chunks. Each chunk receives metadata such as source, date, document type, jurisdiction, and access entitlement. An embedding model converts chunks into vectors that are stored in a searchable index. At runtime, the user's query q is also embedded; the retriever identifies candidate chunks; filters enforce permissions; a reranker may improve ordering; and selected context C_q is inserted into the prompt.

\(q \rightarrow E(q) \rightarrow Retrieve(D,k) \rightarrow C_{q} \rightarrow LLM\left( q,C_{q} \right) \rightarrow y\) (4.10)

Here E(q) is the query embedding, k is the number of retrieved candidates, Cq is the context passed to the LLM, and y is the generated response. A third party should be able to reproduce the design if the developer documents the corpus/version, parser, chunking rules, metadata, embedding model/version, similarity measure, retrieval parameters, filters, reranker, top-k setting, prompt template, generator version, and citation logic.

RAG can improve freshness and grounding, but it redistributes model risk. A failure can come from the corpus, parsing, chunking, metadata, embedding, retrieval, reranking, prompt construction, generation, or citation. For this reason RAG should be developed and tested as a pipeline, not treated as a simple add-on.

What qualifies as RAG documentation? RAG is not limited to formal policy manuals. A controlled retrieval knowledge base may include policies and procedures, regulatory text, product manuals, contracts, research, knowledge-base articles, approved FAQs, database records, structured tables, code repositories, or other authoritative enterprise content. The key MRM question is not the file format but whether the source is appropriate, authorized, current, traceable, access-controlled, and suitable for the use case. Uncontrolled web search or user-uploaded material can also provide retrieval context, but it carries materially different provenance, poisoning, confidentiality, and reliability risks and should be governed accordingly.

Table 4-A summarizes the principal RAG functions, control issues, and their interconnection with the primary model.

4.10 Fine-tuning, instruction tuning, alignment, and outcome analysis

A pretrained model may be adapted through supervised fine-tuning, preference-based methods, system prompts, or other alignment techniques. The development record should state why adaptation is needed, what data are used, what behavior is expected to improve, and what behavior may deteriorate.

Fine-tuning should be followed by outcome analysis. The developer should compare the adapted system with the pre-adaptation baseline, examine performance by important use-case segments, and test for unintended changes in factuality, safety, refusal behavior, privacy, and robustness.

Fine-tuning is model-parameter adaptation: at least some model weights are updated using additional training data. It differs from prompt engineering or RAG, which can change system behavior without changing foundation-model weights. A disciplined fine-tuning loop is: define target behavior and baseline; curate and split adaptation/evaluation data; select the tuning method and trainable parameters; train while monitoring loss and validation metrics; evaluate against held-out and challenge sets; compare against the baseline; analyze regressions and safety effects; revise data/hyperparameters if necessary; freeze/version the accepted checkpoint; and document residual limitations. The stage should output a reproducible model checkpoint or adapter, training-data lineage, hyperparameters, learning curves, comparative evaluation results, regression findings, and an explicit go/no-go decision for the next development stage.

Figure 4-A summarizes the fine-tuning, instruction-tuning, alignment, and outcome-analysis loop.

4.11 Overfitting, memorization, and generalization

This supplement contains a separate technical section on LLM overfitting, memorization, and generalization. The development process should refer to that section rather than repeat the full discussion here.

For development purposes, strong performance on development or familiar benchmarks should not be assumed to represent generalization. Developers should use held-out and newly created tests, temporal holdouts where appropriate, contamination checks, paraphrased/counterfactual cases, and tests for exact or near-exact reproduction of sensitive material. A separate technical subsection on LLM overfitting, memorization, and generalization is appropriate because these topics require more depth than can be provided in the development workflow. The companion technical resource “LLM Overfitting, Generalization and MRM” should be cross-referenced from the supplemental-chapter resources page.

4.12 Model performance, quality, and robustness testing

Before a GenAI system is submitted for independent validation, the development team should perform its own testing. The exact measures depend on the use case, and a single accuracy number is rarely sufficient. Development testing should cover task performance, factuality and grounding, retrieval quality, robustness and prompt sensitivity, compliance, bias/fairness, privacy, cybersecurity, operational resilience, and human-outcome risk. It should also include sensitivity analysis across relevant configurations and environments and stress tests that deliberately push context length, retrieval volume, tool use, concurrency, ambiguity, and adversarial conditions beyond ordinary operation.

Development testing should define measurable acceptance criteria before independent validation. Illustrative measures include: task accuracy or F1 for structured tasks; claim-level factual error rate; grounded-claim rate and citation precision; retrieval recall@k, precision@k and mean reciprocal rank for RAG; abstention precision/recall when evidence is missing; robustness failure rate under meaning-preserving perturbations; jailbreak or prompt-injection attack success rate; unauthorized-data/tool-action rate; latency and failure-rate percentiles; cost per successful task; and human override or material-omission rates. Metrics should be stratified by severity and use-case segment rather than reported only as portfolio averages.

For stochastic outputs, the developer should repeat representative prompts and report distributions rather than a single run. A simple failure-rate estimate is p̂ = k/n, where k is the number of material failures in n independent or approximately independent trials. Confidence intervals or conservative upper bounds are useful when observed failures are rare, especially for high-consequence controls.

Development testing should also include explicit compliance, bias/fairness, sensitivity, and stress testing. Compliance tests should challenge applicable legal, regulatory, policy, disclosure, recordkeeping, suitability, approval, and permission requirements. Sensitivity tests should vary prompts, decoding parameters, model versions, retrieval settings, context construction, and relevant operating environments. Stress tests should examine unusually long or conflicting context, large retrieval sets, component outages, malformed tool responses, high transaction volume, and adversarial inputs.

4.13 Self-identified model risk issues and mitigation

A sound developer should not present the model as if no weaknesses exist. The development package should identify known limitations, explain their business importance, and state the controls used to reduce the risk.

Legal/compliance and intellectual-property/copyright risks should be identified explicitly. Controls may include approved-use policies, legal review, source licensing and provenance checks, copyright-sensitive output testing, citation requirements, disclosure and recordkeeping rules, and escalation where ownership or permitted use is uncertain.

Definitions. Hallucination is unsupported or fabricated content presented as if reliable. Stale knowledge is reliance on information that is no longer current. Prompt sensitivity is material output instability under meaning-preserving prompt changes. Confidential-data risk is unauthorized exposure, retention, or reproduction of protected information. Prompt injection is manipulation of model instructions through malicious direct or retrieved content. Numerical weakness is unreliable arithmetic or quantitative reasoning when a deterministic calculation is required. Excessive agency is the ability of an AI system to take actions or invoke tools beyond the level of autonomy justified by the use case and controls.

4.14 Development documentation, maintenance, and monitoring

Development is not complete when the model first goes into production. Before deployment, the model/system should be documented, independently validated, approved under the applicable governance process, and equipped with defined monitoring and change controls. After deployment, the developer and model owner should maintain performance monitoring, vendor-release review, RAG knowledge-base maintenance, incident handling, and triggers for retesting or revalidation.

Case Study 4-A — Google Gemma 3: Model Development and MRM Assessment for reading materials

Google's Gemma 3 provides a useful public case for illustrating the GenAI development process. The purpose of this case is not to conclude that Gemma is compliant with SR 26-2. Generative AI is outside the formal scope of SR 26-2. Rather, the case asks which publicly documented Gemma development practices are analogous to sound MRM principles and what a financial institution would still need to do for its own use.

Gemma 3 is a family of open-weight multimodal models. Google's model card documents the model family, inputs and outputs, training-data categories and preprocessing, implementation information, benchmark evaluations, safety evaluations, limitations, and release-review practices. For example, the larger Gemma 3 variants use context windows up to 128K tokens; the 27B model was trained on 14 trillion tokens; and the training-data description includes web documents, code, mathematics, and images. Google also describes filtering of sensitive and harmful material.

The assurance-evaluation process is particularly relevant to MRM. Google states that these evaluations are conducted separately from the development team for governance and release decisions; high-level findings may be fed back, while the prompt sets remain held out to reduce overfitting. This resembles two familiar validation principles: independence from development and protection of independent test data.

Source: Google DeepMind, Gemma 3 Model Card; Google Responsible Generative AI Toolkit, 'Evaluate model and system for safety.'

5. GenAI Model Validation

Core concept Gen AI is not a standalone model, it is a system, and the validation is similar to what is called Large Model Framework, or aggregated model risk, the Gen AI risk is not the same as the combination of each single component of a Gen AI system, instead, validator has to validate Gen AI as a system, considering interdependence of each components

Independent validation is the centerpiece of MRM. The objective is not to prove that an LLM is universally correct. The objective is to determine whether the complete GenAI system is conceptually sound for its intended purpose, performs within defined legal and risk-tolerance boundaries, is robust under foreseeable variation and attack, is correctly implemented, and has controls commensurate with its risk.legal boundries and

5.1 Relationship to traditional MRM

SR 26-2 revised interagency MRM guidance superseded SR 11-7 for models within its scope. Generative and agentic AI are outside that scope because they are novel and rapidly evolving, while organizations are expected to determine appropriate governance and controls for tools and systems outside the guidance. The traditional validation principles—conceptual soundness, implementation verification, outcomes analysis, effective challenge, governance, and ongoing monitoring—remain critical foundations, but they require extension for GenAI.

5.2 Validation object and validation boundary

\(S = (W,D,P,R,T,G,U)\) (5.1)

S denotes the GenAI model/system; W represents the foundation model and model version; D the relevant data; P prompts and configuration; R retrieval; T tools; G guardrails and permissions; and U approved users, workflow, and use. Validation should assess each material component and then assess the components in combination. This is analogous to aggregate model risk / Large Model Framework practice: dependencies and interactions can create risks that are not visible in stand-alone component reviews.

Validation establishes an approved operating envelope:

\(\mathcal{V = \{}S:\text{tested configuration, use, and risk remain within approved limits}\}\) (5.2)

A practical component-by-component validation map is:

5.3 Validation procedure

A validator can follow the following sequence:

• Define the validation object, intended use, users, materiality, prohibited uses, and system boundary.

• Review development documentation and identify important assumptions, limitations, vendor information gaps, and developer-identified issues.

• Perform an inherent-risk assessment to determine validation depth and required specialist participation.

• Challenge model and component selection, architecture, data proposition, RAG/tool design, controls, and use-case suitability.

• Design an independent test suite that is separate from development evidence and representative of production risk.

• Test each material component in Equation (5.1) with component-specific evidence. For W, test capability, limitations, stability, and version behavior; for D, test provenance, representativeness, contamination, privacy, and data transformations; for P, test instruction hierarchy, decoding settings, prompt sensitivity, and prohibited-instruction handling; for R, measure retrieval recall/precision, reranking, freshness, entitlement filters, and grounding; for T, test tool selection, parameter construction, permissions, failure handling, and rollback; for G, challenge filters, access controls, human gates, and policy rules; and for U, test actual users, workflows, downstream decisions, and human reliance.

• Test the integrated system with production-like end-to-end scenarios. Include normal cases; difficult and ambiguous cases; boundary cases near approved-use limits; missing or conflicting evidence; long-context and distractor cases; misuse and prohibited-use attempts; direct and indirect prompt injection; unauthorized-data and tool-access attempts; high-consequence actions; component outages; malformed tool outputs; and model/RAG/version changes. Where generation is stochastic, repeat cases across seeds or runs and estimate both average performance and tail failure rates. Record severity, reproducibility, root cause, control effectiveness, and whether the failure breaches a hard limit.

• Verify that the production implementation matches the validated configuration.

• Assess limitations, compensating controls, unresolved issues, and residual risk.

• Issue the validation conclusion and specify monitoring thresholds, conditions, and revalidation triggers.

• Adequately document validation results, including validators’ challenges, communication with model developer and model owner, test design and purpose, validation findings, model deficiency classification and follow up and remediation plan

5.4 Risk-based validation planning

Risk-based validation does not mean simply assigning a numerical score. It means that the scope, depth, sample size, independence, expertise, frequency, and approval requirements should increase with the potential consequence and uncertainty of the system.

5.5 Model and system selection / conceptual soundness

For GenAI, conceptual soundness is a system-selection question. The validator should challenge not only the choice of foundation model but also the choices of data, prompts, RAG, tools, guardrails, deployment method, and human controls.

The validator should ask, among other questions:

• Is GenAI appropriate for the business purpose, or would a deterministic workflow, search engine, statistical model, or narrower ML model be safer and sufficient?

• Why was this foundation model selected over credible alternatives? What trade-offs were made among capability, privacy, cost, latency, transparency, and stability?

• Is RAG needed? If so, why is the chosen knowledge base, embedding model, chunking, retrieval, and reranking design appropriate?

• Are tools necessary? Are their permissions no broader than required?

• Are the prompt and guardrail architecture appropriate for the user population and consequence of error?

• Does the system depend on a context window, vendor feature, or model behavior that is unstable or insufficiently understood?

Does the design identify applicable legal, regulatory, policy, disclosure, recordkeeping, privacy, suitability, and approval requirements, and are guardrails and permissions demonstrably effective rather than merely documented?

• Compliance, guardrails, permissions etc.

• Are human controls realistic, or are they likely to become rubber-stamp approvals?

Vendor foundation models and third-party GenAI services. Most financial institutions will use at least one third-party foundation model, hosted API, embedding model, vector database, safety service, or other external component. Vendor use does not transfer model-risk accountability. Validation should assess vendor technical documentation, model cards, training-data disclosures and limitations, evaluation evidence, versioning and change-notification practices, security and privacy commitments, data-retention and training-use terms, geographic/data-residency constraints, incident history, service dependencies, concentration and substitution risk, and contractual audit/access rights. Where proprietary restrictions prevent full transparency, the institution should document the information gap and compensate with stronger independent empirical testing, tighter operating limits, monitoring, fallback arrangements, and change controls.

5.6 Independent test design: how to build a usable test suite

The independent test set should not be a loose collection of prompts. It should be designed from the approved use cases and failure modes. A practical construction process is:

First, divide production use into meaningful strata? layers? —for example policy questions, summarization, extraction, calculation, or tool actions. Second, identify normal cases, difficult cases, boundary cases, misuse, and adversarial cases within each stratum. Third, define an expected answer, authoritative source, or scoring rubric before seeing model output where possible. Fourth, create independent cases not used in development. Fifth, repeat stochastic tests when variability matters. Finally, classify errors by materiality rather than counting all wording differences equally.

Technical construction of the independent test suite. Define a sampling frame from approved use cases and risk scenarios; stratify by task, user type, data sensitivity, consequence, language, and difficulty; reserve a held-out set that developers have not seen; pre-specify expected answers or scoring rubrics; randomize case order; run repeated trials when temperature or sampling creates stochasticity; use blinded human adjudication for subjective outputs; calculate confidence intervals for key rates; and separately report severe failures rather than allowing them to disappear in averages. For paired robustness tests, compare the baseline prompt with controlled perturbations and measure the change in business-relevant output. For RAG, label relevant source documents so retrieval recall and precision can be measured independently of generation quality.

Example paired robustness statistic:

\[R_{fail} = \frac{1}{n}\sum_{i = 1}^{n}\mathbf{1}\left\{ d\left( y_{i},y_{i}^{'} \right) > \tau_{i} \right\}\]

Here y_i is the baseline output, y_i′ is the output after a controlled meaning-preserving perturbation, d(·) is a business-relevant difference measure, and τ_i is the tolerance. The validator should define d and τ before observing results wherever feasible.

For RAG retrieval, if Rel(q) is the labeled set of relevant chunks and Ret_k(q) is the top-k retrieved set, then recall@k and precision@k can be written as:

\[Recall@k = \frac{\left| Rel(q) \cap {Ret}_{k}(q) \right|}{\left| Rel(q) \right|},\quad\quad Precision@k = \frac{\left| Rel(q) \cap {Ret}_{k}(q) \right|}{k}\]

Example. Suppose four chunks {A, B, C, D} are labeled relevant for a query and the system retrieves five chunks {A, B, E, F, G}. Two of four relevant chunks were retrieved, so Recall@5 = 2/4 = 0.50. Two of five retrieved chunks are relevant, so Precision@5 = 2/5 = 0.40. Recall measures coverage of relevant evidence; precision measures the relevance of retrieved evidence. Both should be interpreted together.

Generation should then be tested separately for claim support, citation correctness, material omissions, and appropriate abstention. This decomposition helps identify whether a bad answer originated in retrieval, generation, or the interaction between them.

Common independent test families include:

Adaptive/recursive-change testing. Where a system can autonomously revise code, prompts, tools, policies, memory, or model components, validation should test authority boundaries, approval gates, rollback, monitoring, and whether iterative changes can compound errors or bypass controls. Ordinary vendor model updates are change-management/version risk rather than recursive self-improvement unless the deployed system itself can initiate or implement material self-modification.

5.7 The GenAI Validation Vector

\(V_{GAI} = (A,F,G,R,S,B,P,C,U)\) (5.3)

A = task accuracy, F = factuality, G = groundedness, R = robustness, S = safety, B = bias/fairness, P = privacy, C = cybersecurity, K = compliance/control adherence, and U = use-case suitability. Compliance is included explicitly because a system can be accurate and factually grounded yet still violate a law, policy, disclosure rule, suitability requirement, permission boundary, or required human-approval step. The vector is not intended to imply that every dimension is measured with the same statistic. Its purpose is to prevent a single performance number from hiding important risk dimensions.

\(Score = \sum_{j = 1}^{m}w_{j}V_{j},\quad\quad\sum_{j}^{}w_{j} = 1\) (5.4)

A weighted score may be useful for summary reporting, but severe failures in non-compensable dimensions—such as confidential-data leakage or unauthorized transaction execution—should be subject to hard limits and should not be offset by strong scores elsewhere.

The purpose of independent validation is not to manufacture a scorecard. Scores can summarize evidence, but validation should identify model/system deficiencies, boundaries, implementation constraints, misuse pathways, and residual risk, and determine whether controls adequately mitigate those risks. Deficiencies should be addressed individually and classified by severity. An issue is a material deficiency that may constrain or prevent approval and requires formal remediation and closure. An observation is a lower-severity weakness or emerging concern that may be tolerable within the approved operating envelope but requires action, monitoring, or follow-up. Severe non-compensable failures should not be averaged away by strong performance elsewhere.

5.8 Task accuracy and outcome analysis

Metrics should follow the task. Classification may use accuracy, precision, recall, F1, ROC-AUC, or calibration. Extraction may use exact match or field-level error. Summarization may require expert scoring for completeness and material omission. The validator should define a material business error, not merely a linguistic difference.

5.9 Factuality and hallucination testing

\(Material\ Claim\ Error\ Rate = \frac{\#\ material\ unsupported/incorrect\ claims}{\#\ material\ claims\ evaluated}\) (5.5)

Testing should separate harmless wording differences from material factual errors and should stratify results by severity and subject matter. Human adjudication may be required. Where reviewers disagree, inter-rater agreement should be measured and disagreement resolved through a documented process.

5.10 RAG validation: retrieval, grounding, and citation

RAG validation should separate retrieval quality from generation quality.

\(Recall@k = \frac{\#\ relevant\ items\ retrieved\ in\ top\ k}{\#\ relevant\ items}\) (5.6)

\(Precision@k = \frac{\#\ relevant\ items\ retrieved\ in\ top\ k}{k}\) (5.7)

The validator should test the corpus and effective dates; parsing and chunking; metadata; embedding model; retrieval and reranking; access filters; conflicting sources; context construction; groundedness; citation correctness; and behavior when no adequate source exists. A system that answers confidently when retrieval fails may be less acceptable than one that abstains.

5.11 Robustness and prompt sensitivity

\(x\mathcal{' \in N}(x)\quad\text{with approximately preserved meaning}\) (5.8)

\(d\left( f(x),f(x') \right) \leq \tau\) (5.9)

x is the baseline prompt, x′ is a meaning-preserving perturbation, d measures the business-relevant difference between outputs, and τ is the acceptable tolerance. Tests should include paraphrases, formatting changes, typos, long context, distractors, multilingual prompts where relevant, and conflicting instructions.

5.12 Bias, fairness, and representativeness

Fairness testing should be tied to the actual use case and applicable requirements. The validator should test whether sensitive attributes, proxies, language varieties, or demographic context produce unjustified differences in quality or treatment. Open-ended output may require expert qualitative review in addition to statistical disparity measures.

5.13 Privacy and memorization testing

Validation should examine whether confidential information can enter prompts or retrieval context, whether the model can reproduce protected information, whether connected tools expose data beyond need-to-know, and whether logs create a second privacy exposure. Controlled secrets/canaries, extraction attempts, and access-control scenarios can be useful.

5.14 Cybersecurity and adversarial testing

The test plan should include direct and indirect prompt injection, jailbreaks, system-prompt extraction, data exfiltration, malicious documents, tool abuse, excessive agency, privilege escalation, unsafe output handling, and poisoning scenarios. For agentic systems, the validator must test both whether the model can be manipulated and what it is technically permitted to do after manipulation.

5.15 Agentic AI and tool-use validation

Agentic validation extends output validation to action validation. Tests should verify tool selection, parameter construction, permissions, transaction limits, confirmation requirements, error recovery, loop termination, and refusal of unauthorized goals.

\(Expected\ Loss \approx P(error) \times Impact(action)\) (5.10)

Even a low error probability can be unacceptable when the action is large, irreversible, financial, legal, or externally consequential. Human approval and deterministic permission controls should therefore be considered according to risk.

5.16 Implementation verification

A conceptually sound design can fail through implementation. The validator should independently verify model and endpoint versions, prompts, retrieval indexes, access permissions, preprocessing, output parsing, fallback logic, caching, temperature/decoding settings, tool schemas, logging, monitoring, and consistency between development, validation, and production.

5.17 Effective challenge

Effective challenge remains central. Independence alone is not enough. The validation function needs sufficient technical competence, business understanding, and organizational authority to challenge model/component selection, data, vendor claims, evaluation design, security controls, and use. Validators should communicate challenges directly to model owners and developers, and material challenges should be resolved or formally dispositioned before validation is complete. Validation findings may be classified as issues or observations according to severity. Where remediation requires additional time, the validation report should specify interim controls, accountable owners, target dates, retesting requirements, and escalation criteria. Unresolved high-severity deficiencies should constrain approval or use rather than be offset by strengths elsewhere.

5.18 Validation limitations and residual risk

No finite test suite can enumerate the full input space of a general-purpose language model. Validation therefore provides evidence about performance within an approved operating envelope rather than proof of universal safety. The report should state untested areas, unavailable vendor information, known weaknesses, uncertainty in human evaluation, benchmark limitations, and dependence on downstream controls.

5.19 Validation conclusion and approval taxonomy

5.20 Independent validation work program

Table 5-B summarizes the expected challenge, remediation, and closure record for validation findings.

5.21 Performance-monitoring handoff

Validation should end by specifying what must be monitored after approval. At minimum, the validator should identify critical metrics, thresholds, sampling frequency, incident triggers, drift indicators, security events, vendor/model-version changes, RAG knowledge-base changes, new tools, and conditions requiring revalidation. Part III develops this ongoing monitoring and dynamic-change framework.

Case Study 5-A — Anthropic Claude: Evaluation, External Testing, and MRM Assessment

Anthropic's Claude system cards provide a useful case for Section 5 because they document model evaluation, safety/capability testing, external testing, deployment decisions, and an explicit risk-governance framework. Again, the case should not be described as SR 26-2 compliance. It is better treated as evidence that can be challenged against MRM principles.

Anthropic publishes system cards for successive Claude models. These cards describe model characteristics, evaluation methods, capability and safety testing, and deployment decisions. Recent system cards also distinguish internal evaluations from evaluations performed by external testers. Anthropic's Responsible Scaling Policy adds capability thresholds, safeguard assessments, documentation, governance, and periodic reassessment.

One especially relevant point is Anthropic's distinction between internal evaluation and external testing. A recent Claude system card states that evaluations are generally run in-house except where external testers are identified, and that external groups have evaluated selected risk areas. Anthropic has also argued that a robust third-party evaluation ecosystem is important because developer self-testing alone has limitations.

This is close to the logic of effective challenge, but the analogy should not be overstated. An external safety evaluator may be independent of the model developer while still not performing the same role as a financial institution's MRM validator. The MRM validator must understand the bank's use case, materiality, data, controls, downstream decisions, and risk appetite.

This principle is also consistent with the vendor-model logic in SR 26-2 for models within its formal scope: proprietary limitations do not remove the banking organization's responsibility to understand conceptual soundness, design, development data and performance, and sound practice includes ongoing monitoring and outcome analysis. For GenAI, which SR 26-2 explicitly excludes from formal scope, this vendor-model principle is best used as an MRM analogy rather than a claim of regulatory applicability.

Source: Anthropic, Model System Cards; Anthropic, Responsible Scaling Policy and related evaluation materials; Anthropic, 'A New Initiative for Developing Third-Party Model Evaluations.'

Part II Summary

Part II separates two activities that should remain distinct. Model development explains how the GenAI system is designed, fitted/configured, tested, controlled, and maintained. Independent validation challenges that work and produces its own evidence.

• GenAI development should follow a disciplined sequence from purpose and data through system structure, component selection, training/configuration, performance testing, tuning, controls, deployment, and maintenance.

• The technical foundation—tokenization, embeddings, attention, MLPs, and next-token training—matters because it explains both the capability and the limitations of LLMs.

• RAG, prompts, tools, guardrails, vendor services, and use are model-relevant system components because they can materially change behavior without changing foundation-model weights.

• Independent validation should test each material component and then test the complete system, with particular attention to interactions.

• A risk-based plan determines the depth and type of testing; it does not replace independent testing with a simple risk score.

• The final validation conclusion should state the approved operating envelope, limitations, conditions, residual risk, monitoring requirements, and revalidation triggers.

Google DeepMind (2025). Gemma 3 Model Card. Google AI for Developers.

Google AI for Developers. Responsible Generative AI Toolkit: Evaluate model and system for safety.

Anthropic. Model System Cards (Claude series), including model capability, safety, external-testing, and deployment assessments.

Anthropic (2024). A New Initiative for Developing Third-Party Model Evaluations.

Anthropic. Responsible Scaling Policy and related implementation/reflection materials.

Part II References and Further Reading

1. Board of Governors of the Federal Reserve System, OCC, and FDIC (2026). Revised Guidance on Model Risk Management, SR 26-2, April 17, 2026.

2. Autio, C., Schwartz, R., Dunietz, J., Jain, S., Stanley, M., Tabassi, E., Hall, P., and Roberts, K. (2024). Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1.

3. NIST (2026). TEVV-Athlon Framework for Evaluating AI Systems, initial public draft materials for NIST AI 200-2.

4. Vaswani, A. et al. (2017). Attention Is All You Need. Advances in Neural Information Processing Systems.

5. Lewis, P. et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Advances in Neural Information Processing Systems.

6. OWASP GenAI Security Project (2025/2026). Guidance on prompt injection, sensitive information disclosure, data/model poisoning, excessive agency, and related GenAI application risks.

7. Board of Governors of the Federal Reserve System and Office of the Comptroller of the Currency (2011). Supervisory Guidance on Model Risk Management, SR 11-7 / OCC 2011-12. The guidance directs banks to consider model risk from individual models and in the aggregate, including interactions and dependencies among models and common assumptions, data, or methodologies.
8. Federal Reserve Bank of St. Louis, Ask the Fed. Model Risk Management supervisory presentation. Notes that many firms were in early stages of assessing aggregate model risk.
9. Barclays (public model-risk role descriptions, 2025–2026). Large Model Framework responsibilities include evaluating the coherence of model interactions and the quality of aggregate results used for regulatory submissions or management decision-making, and assessing framework-level uncertainty.