Machine-Learning Model as a Subject of Examination: Model Version, Parameters, Data Basis and Runtime Environment

AI & Machine Learning

Calling something “the AI model” or handing over a file such as model_final.pkl does not yet define a technically unambiguous subject of examination. Depending on the framework and persistence format, model structure, learned parameters, preprocessing, metadata and runtime dependencies can be stored in different places.

1. Model artifact and technical identification

A machine-learning model must be technically identified

For a technical expert examination, a file or system label is not sufficient on its own. The concrete object may be a serialized model artifact, a PyTorch checkpoint, a TensorFlow SavedModel, an ONNX file, a scikit-learn estimator or pipeline, an inference service deployed in a container, or a historical state stored in a model registry.

The model artifact and the complete AI system are not the same

A stored model can represent only one part of the system that actually produced an output. Data cleaning, feature computation, tokenization, scaling, normalization, input-field selection, thresholds, post-processing and business rules can live outside the artifact. The examination question must therefore state whether it concerns the mathematical model alone or the full processing chain.

Persistence formats also differ in what they contain. The scikit-learn documentation, for example, distinguishes ONNX, skops.io, pickle, joblib and cloudpickle, each with different runtime, portability and security implications. The concrete persistence or export format is therefore part of the technical description.

Names such as model_final.pt, classifier_v3.pkl or production.onnx are organisational labels. They do not by themselves prove which byte sequence, training run, commit or deployed version is present.

Cryptographic hashes can identify a concrete model artifact

A cryptographic hash can technically identify the exact byte state of a model artifact and make later comparisons possible. The hash does not prove which training run created the artifact, whether the intended data were used or whether the resulting model is technically suitable for a given purpose.

2. Model components, interfaces and processing

Model structure and learned parameters must be distinguished

This distinction is particularly visible in PyTorch. A state_dict maps parameter and registered state names to values, but restoring that state requires the matching model structure into which the stored values can be loaded. An examination should therefore determine whether an artifact contains only parameter state, a full model, a checkpoint or an exported program.

Learned parameters are produced by training, such as weights and bias terms. Hyperparameters are chosen or configured for training or model construction, such as learning rate, tree depth, regularisation, batch size or architecture parameters. Both can matter for reconstructing a training process, but they are not the same category of information.

Preprocessing may be embedded in the model, contained in a pipeline, implemented in application code or delegated to another service. Reproducible inference therefore requires the actual transformation sequence applied before the model received its input.

The input schema belongs to the examination basis

A reproducible test needs a sufficiently precise description of expected inputs. Depending on the system, that may include feature count and order, data types, tensor dimensions, categories and encodings, permitted value ranges, tokenization rules and handling of missing values.

The output schema must also be determined. A model can return a class, score, probability, vector, regression value or another technical representation. Only when the output is documented can later application logic be interpreted correctly.

Thresholds and post-processing can live outside the artifact. A classifier may first return a score or probability, while a separately configured threshold determines the final class. Calibration, ranking rules and other post-processing require the same separation.

3. Training data, training recipe and provenance

Training data are part of model provenance

A model state does not arise independently from its data basis. A technical reconstruction can therefore require the concrete training dataset, data preparation and feature generation used to create the model. Data provenance and transformation history are separate technical evidence from the final model file.

Training, validation and test data have different functions. A model can be fitted on training data, selected or tuned using validation data and evaluated on a separate test set. For every reported performance figure, the examination should determine on which dataset state it was actually calculated.

Mitchell et al.'s Model Cards proposal is one possible documentation mechanism and recommends reporting aspects such as model version, intended use, evaluation data, metrics and limitations. A model card remains a documentation source; its statements must still be checked against the available artifacts in a concrete examination.

A persisted model does not necessarily include the code or information required to create the same state again. Relevant evidence can include training scripts or notebooks, source-code version, configuration files, hyperparameters, data splits, random states and library versions.

Training and data splitting can contain stochastic steps. Seeds and other random-state parameters can therefore be relevant, but a seed alone does not guarantee complete reproducibility across every software and hardware configuration.

4. Runtime, inference, deployment and security

The runtime environment can be essential for loading the model

Persistence mechanisms can depend on specific library versions or runtime behavior. scikit-learn explicitly notes that persisted models may require the same package and dependency environment. A historical model artifact that can no longer be loaded under a current environment may therefore require reconstruction of the original software state.

The runtime can include Python version, framework version, additional libraries, operating system, CPU/GPU environment, model-serving software and container or deployment configuration. These elements should be documented when they can affect loading or inference.

PyTorch also distinguishes training and evaluation behavior. Layers such as dropout or batch normalization can behave differently depending on mode, making the actual inference state part of the technical basis.

TensorFlow SavedModel stores a broader executable representation than a simple parameter file. ONNX, in turn, defines an intermediate representation with model, graph, operator-set and version metadata. These differences are relevant when deciding what the artifact proves and what remains external.

Security also matters. Formats based on Python pickling can execute code during loading. A technical examination should not treat unknown serialized artifacts as harmless data; provenance and a controlled analysis environment can be necessary.

5. Artifact formats, versions and reproducibility

Version information should be interpreted at the correct level. ONNX, for example, distinguishes IR version, operator-set imports and model metadata. A single “model version” string is not automatically sufficient to reconstruct the technical state.

An artifact establishes neither model quality nor suitability by itself. Statements about accuracy, robustness, fairness, generalization or other performance properties require a defined examination criterion, a suitable data basis and an appropriate metric.

Reproducibility of inference and reproducibility of training are different questions. Reproducing inference may require the model artifact, input data, preprocessing and runtime. Reproducing training normally requires additional data, training code, hyperparameters, splits and random states.

6. Missing artifacts, documentation and limitations

Missing artifacts limit the technical conclusion

If the model structure is unavailable, an isolated parameter state may not be fully interpretable. If the historical runtime environment is missing, it may be impossible to determine whether an old artifact behaved identically under the original conditions. If training data or code are unavailable, the existing model can still be examined, but its complete creation process does not become reproducible.

A technical report can establish which artifact was available, which parameters or versions are documented, whether a result can be reproduced and which components are missing. These technical findings remain separate from legal assessment.

Practical documentation structure for a model state

• Model name and concrete artifact file or directory.
• Cryptographic hash and persistence or export format.
• Model structure and learned parameter state.
• Relevant hyperparameters.
• Preprocessing, input schema and output schema.
• Thresholds, calibration and post-processing.
• Training, validation and test dataset states.
• Training code or source-code version.
• Framework and library versions.
• Inference mode, runtime and relevant hardware environment.
• Known uncertainties, missing artifacts and limitations.

Further reading

For performance evaluation of a clearly identified model state, see Validating a Machine-Learning Model.

For the quality and representativeness of the underlying data, the English data-cluster articles will be added in a later EN-2 phase. Until then, the reviewed German source articles remain available in the German knowledge base.

For the general documentation standard, see Sources and methodology (German source).

Data Science & AI Assessments

Technical examination of data quality, model validity, reproducibility, bias, robustness and documented AI-system behaviour.

Explore this assessment service