What is Artificial Intelligence
Artificial intelligence technologies are developing at a rapid pace. Their impact on various sectors of the economy is continually increasing. We are witnessing a revolution in the field of data collection, processing, and utilization. These dynamic changes bring new challenges regarding the respect for confidentiality, which is associated with the requirement to operate in a transparent and responsible manner.
The AI Act introduced a new term: “artificial intelligence system.” According to the definition, it is a system that analyzes the data input into it to generate other data, such as forecasts, new content, recommendations, or even decisions. Such a system can operate at various levels of independence from humans, depending on the configured settings.
It can therefore be said that an artificial intelligence system is a tool that enables the analysis of large amounts of data to gain insights, and then identify patterns and trends that will be used to generate recommendations or decisions.
AI is not only large language models, such as Gemini or ChatGPT (provided as part of the Microsoft Copilot service). Artificial intelligence is used, for example, in streaming services to recommend movies to users based on what they have previously watched or what users with similar movie preferences have watched.
AI can also save lives, as they are used in the medical field to analyze X-ray images and magnetic resonance imaging scans. Thanks to the knowledge gained from large datasets of medical images, it is possible to identify existing patterns and potential anomalies.
The capabilities of AI systems are immense; however, it is essential to ensure that their operation complies with the law.
Principles Arising from GDPR and AI Act
Lawfulness
Article 5(1)(a) of the GDPR requires that data processing be lawful. This means that data may only be processed when there is an appropriate legal basis, such as consent, a contract, a legal obligation, or legitimate interest (Articles 6, 9, and 10 of the GDPR). This requirement must also be met by artificial intelligence systems that process personal data.
The AI Act expands upon this principle by specifying certain types of AI systems whose use is prohibited (Article 5(1) of the AI Act). For example, the use of a system that conducts so-called social scoring of individuals is prohibited if such scoring leads to harmful or disadvantageous treatment of certain natural persons or groups of persons. Another example may be the prohibition of using AI systems that create databases through untargeted scraping of facial images from the internet or recordings from closed-circuit television.
Fairness
Article 5(1)(a) of the GDPR also states that data processing must be fair. This broad concept encompasses the need to consider the principles of social coexistence, including the interests and reasonable expectations of the individuals whose data is being processed.
The AI Act does not define the concept of fairness; however, it reinforces the need for fair practices by introducing a prohibition on the use of AI systems that employ subliminal, manipulative, or misleading techniques (Article 5(1)(a) of the AI Act) or exploit the vulnerabilities of individuals or groups caused by their age, disability, or particular social or economic situation (Article 5(1)(b) of the AI Act).
Transparency
Another principle arising from Article 5(1)(a) of the GDPR is transparency. It requires that data processing be transparent to the individuals whose data is being processed. The AI Act also mandates a basic level of transparency for all artificial intelligence systems that interact directly with humans. Therefore, the user must be informed that they are interacting with AI. In this regard, it is sufficient to provide a message indicating that, for example, the chat is not conducted with a human. Such a message will be unnecessary if the circumstances and context of the situation indicate that conversing with AI is obvious to a suitably informed, attentive, and cautious person (Article 50(1) of the AI Act).
If AI systems generate content in the form of sounds, images, videos, or text, the effects of their operation should be detectable as artificially generated or manipulated (Article 50(2) and (4) of the AI Act). The use of emotion recognition systems or biometric categorization also requires informing the individuals concerned (Article 50(3) of the AI Act).
A higher level of transparency is required for high-risk AI systems. These are AI systems that may adversely affect the safety and fundamental rights of citizens. The provider of a high-risk AI system must provide the entity using such a system with an "operating manual" that informs concisely, completely, accurately, and clearly how the AI system operates (Article 13 of the AI Act). The information provided should facilitate understanding of the factors influencing the choices made by artificial intelligence and the methods used to mitigate potential biases.
Limitation of purpose of processing and data minimization
The GDPR introduced the obligation to limit the purpose of processing (Article 5(1)(b) GDPR) and data minimization (Article 5(1)(c) GDPR). The first principle means that personal data should be collected for specific, explicit, and legitimate purposes, while the second means that such data must be adequate, relevant, and limited to what is necessary for achieving those purposes.
The AI Act clarified the above obligations concerning high-risk AI systems. Such systems should have a suitably defined purpose of operation that reflects the selection of training data forming the knowledge base of the AI system (Article 10 of the AI Act). The purpose must be appropriately documented (Articles 11 and 17 of the AI Act).
Accuracy
According to Article 5(1)(d) GDPR, the processed data should be accurate and, where necessary, kept up to date. The data controller is obliged to take all reasonable steps to ensure that personal data which are inaccurate in relation to the purposes of their processing are erased or rectified without delay.
The above principle also applies to AI systems and is further elaborated in the AI Act. Training datasets should be subject to appropriate operations such as commenting, labeling, cleaning, updating, enriching, and aggregating (Article 10(2)(c) of the AI Act). The EU legislator placed particular emphasis on the obligation to investigate any potential bias in the AI system that could lead to the generation of discriminatory content and outcomes (Article 10(2)(f) of the AI Act).
Storage limitation
As a general rule, the GDPR prohibits the storage of personal data for longer than is necessary to achieve the purposes for which the data was processed (Article 5(1)(e) GDPR).
In this regard, the AI Act does not introduce additional, specific rules regarding data retention. The necessary duration of data storage should be determined on an individual basis, taking into account the general principle arising from the GDPR and the principle of data minimization concerning training data (Article 10 AI Act).
Rules Regarding Automated Decision-Making
Article 22 of the GDPR grants the data subject the right to refuse to be subject to decisions based solely on automated processing. This right applies when the automated decision produces legal effects concerning the data subject or similarly significantly affects them. This means that a person who does not agree to have important matters decided by a machine may request human involvement in the decision-making process. The data controller should inform this person of their rights (Article 13(2)(f) GDPR and Article 14(2)(g) GDPR).
The EU legislator has determined that the above mechanism is insufficient in relation to cases involving AI. The development of artificial intelligence has created countless scenarios in which machines analyze vast amounts of data much faster than humans and make decisions in fractions of a second that can impact human lives. While this allows for the optimization of decision-making processes in the economy, it raises concerns about whether AI decisions will have a "human" and ethical character.
As a result, the AI Act places greater emphasis on human involvement. In the case of high-risk AI systems, appropriate tools must be created at the design stage to enable proactive human oversight (Article 14 of the AI Act). The purpose of this oversight is to eliminate or minimize risks to health, safety, or fundamental rights. Both risks that may arise during the use of the AI system according to its intended purpose and risks that may materialize under reasonably foreseeable conditions of misuse must be taken into account (Article 9(2)(b) of the AI Act). Oversight measures should be proportionate to the identified risk, level of autonomy, and context of the AI system's use. Users should be informed about the capabilities and limitations of the system. They should be warned against excessive reliance on the results generated by AI. Users must have the freedom to decide to refrain from using AI in a specific situation, as well as the ability to directly intervene in the AI system, for example, by halting its operation in a safe state.
In conclusion, while the GDPR required human intervention "upon request," the new regulation on artificial intelligence mandates the implementation of continuous human oversight over high-risk AI systems at all stages of their development, deployment, and use.
How to Ensure Security
In terms of security, the GDPR is technologically neutral due to its general nature. This means that the GDPR does not specify particular safeguards that must be implemented. The data controller should implement such technical and organizational measures that are appropriate in light of the state of technical knowledge, the costs of implementation, and the nature, scope, context, and purposes of processing, taking into account the risk of infringement of the rights or freedoms of natural persons (Article 32 of the GDPR). Consequently, under the GDPR, the data controller should conduct a risk analysis to identify potential threats specific to their activities and then select appropriate security measures.
The AI Act is aimed at addressing more specific threats, such as system bias, which can negatively impact individuals. To mitigate the risk of bias, training data used to build the AI system must be examined for this purpose (Article 10(2)(f) of the AI Act). If the implemented system continues to learn, it must be safeguarded against future influence from potentially biased outcomes of its operations (Article 15(4) of the AI Act). If necessary to eliminate system bias, the processing of special category personal data is permitted based on a specific authorization arising from Article 10(5) of the AI Act.
Preliminary identification of threats, similar to that in the GDPR, should begin at the planning stage (Article 9 of the AI Act). This process is akin to risk analysis under the GDPR but is continuous and repeatable throughout the lifecycle of the AI system.
Article 72 of the AI Act states that after the implementation of a high-risk AI system, it should be monitored by the provider for compliance with the requirements established in Articles 8–15 of the AI Act. The entity using such a system should monitor its operation according to the user manual and, where appropriate, inform the provider of any irregularities in its functioning (Article 26(5) of the AI Act).
An additional safeguard is the aforementioned human oversight of high-risk AI systems. The entity utilizing such a system should assign oversight to a person who possesses the necessary competencies, training, and qualifications, as well as support (Article 26(2) of the AI Act). The provider should ensure that such a person has the appropriate tools to enable effective oversight (Article 14 of the AI Act).
GDPR.
Support is beneficial
Rights of Data Subjects
The GDPR provides individuals whose data is processed with a wide range of rights:
- access to data (Article 15 of the GDPR),
- rectification of data (Article 16 of the GDPR),
- the right to erasure (Article 17 GDPR),
- the right to restriction of processing (Article 18 GDPR),
- the right to data portability (Article 20 GDPR),
- the right to object (Article 21 GDPR),
- the right not to be subject to decisions based solely on automated processing (Article 22 GDPR).
Furthermore, the data controller must provide these individuals with the information referred to in Articles 13 and 14 GDPR.
The AI Act does not limit the aforementioned rights in the context of using AI systems. As mentioned, the scope of informational obligations has been expanded, which must be fulfilled to operate transparently.
Liability for Data Protection Violations
Data protection violations under the GDPR are subject to an administrative fine of up to €10–20 million or 2–4% of the total annual worldwide turnover from the previous financial year (whichever amount is higher).
The administrative financial penalties provided for in the AI Act are equally high. They are:
- €35 million or 7% of the total annual worldwide turnover from the previous financial year (whichever amount is higher) – for violations of prohibited practices or non-compliance with data requirements,
- up to €15 million or 3% of the total annual worldwide turnover from the previous financial year – for non-compliance with any other requirements or obligations arising from the AI Act, including violations of regulations concerning general-purpose AI models,
- up to €7.5 million or 1% of the total annual worldwide turnover from the previous financial year – for providing notified bodies and competent national authorities with incorrect, incomplete, or misleading information in response to a request.
For each category of violation, two maximum amounts of financial penalties are provided. The lower amount is for small and medium-sized enterprises (SMEs), while the higher amount is for other entrepreneurs.
Summary
AI technology is developing very rapidly. Media reports continue to emerge about revolutionary advancements and new applications of AI that fundamentally change the way business can be conducted. The potential of AI is enormous – it can significantly accelerate work and optimize costs. Entrepreneurs who are the first to recognize and leverage new opportunities will leave their competitors far behind in the competitive market. The race is already on, and those who remain indecisive, still waiting for the right moment to join, may wake up too late.
An entrepreneur conducting business in Europe must remember that they operate in a market that is largely regulated by the Union. The Union ensures respect for the existing legal order and the fundamental rights of citizens. This means that the use of modern technologies must be legal and compliant with applicable regulations. Anyone who decides on the directions of the company's development should be prepared for both the necessity of rapid growth and the provision of legal security, which ensures that the technological race does not end in disaster. The combination of the vision of a bold manager and the expertise of data protection specialists allows for the achievement of both stated goals.
