External Models and DPIA – Is It Possible to Assess Risk Without Full Knowledge of the AI?

ANSWER

Knowledge of the model

Most models are already supplied with documentation (known as a model card), although this typically does not provide a complete picture.

In part, the AI Act will provide tools for obtaining the most important information about a model (architecture, training methodology, personal data and copyright considerations).

For open-source models with open weights, a great deal of information about the model can be obtained independently. However, many open-source models do not disclose information such as the source of their training data.

Examples of open-source models compliant with the Open Source Initiative definition include: Pythia (Eleuther AI), OLMo (AI2), Amber and CrystalCoder (LLM360), and T5 (Google). These are models with open weights and an open training dataset.

DPIA

It is not necessary to know every parameter in order to assess risk; however, if adequate due diligence on a model cannot be conducted, that model should not be used.

In other cases, a rigorous DPIA is achievable. Nevertheless, it requires multidisciplinary collaboration within the organisation implementing or building the model.