Data security in AI systems is one of those questions that comes too late for most development teams. The system works, the functionality is impressive, the first users are already using it—and only then does someone ask: but how is this data actually being handled?
This is not negligence. It is the natural order of many projects that were built quickly, using tools that democratized access to artificial intelligence and reduced development time from months to weeks. The problem is that development speed and security maturity rarely progress at the same pace.
What Changes When AI Becomes Part of the System
Applications built with artificial intelligence components have several characteristics that distinguish them from more traditional systems from a data protection perspective.
The first is their dependence on external APIs. Many AI systems rely on third-party services, whether language models, computer vision APIs, or processing platforms. Every request sent to these services may include user data, depending on how the system was designed. The question that needs to be answered is: what, exactly, is leaving your application with every request?
The second is the volume of processed data. AI systems tend to process more information per interaction than conventional systems, often storing conversation histories, inferred preferences, or behavioral patterns that qualify as personal data under the LGPD, even if they were not originally collected with that intention.
The third is model opacity. When a system relies on an externally trained model, it is not always clear what happens to the data processed by that model, where it is temporarily retained, or how it is ultimately discarded. This creates a blurred area of responsibility that the law does not recognize as a valid excuse for a lack of control.
Does the LGPD Apply to AI Systems? Yes, Without Exception
The Brazilian General Data Protection Law (LGPD) makes no distinction based on the technology being used. What determines the legal obligation is the processing of personal data belonging to individuals in Brazil. Any system that collects, processes, stores, or transmits this type of information is subject to the same rules, regardless of whether it uses AI, automation, or any other technological resource.
In practice, this means that the developer or company behind the system is the data controller for the information that flows through it. Responsibility for how that data is collected, the legal basis for processing, the purpose of the processing, and how long the information is retained rests with whoever decided to build and deploy the system, as explained in the article on LGPD compliance and the role of the data controller.
Using a third-party API for natural language processing, for example, does not transfer responsibility for user data. The developer remains the data controller, while the API provider becomes a data processor, and the relationship between the two must be formally documented through contractual security and confidentiality obligations.
What to Consider Before Putting the System into Production
Several aspects are often overlooked during the development of AI systems, despite having a direct impact on the protection of user data.
What data is sent to the model. It is common for prompts submitted to language models to include user information that does not actually need to be there. Reviewing the content of every request, eliminating unnecessary data, and using pseudonymization whenever possible are practices that reduce the attack surface.
Where the data is stored and for how long. Conversation histories, usage logs, and inferred preferences accumulate quickly. Without a defined data retention policy, this information remains stored indefinitely, increasing the company’s responsibility and the potential impact of any security incident.
How the system behaves during extraction attempts. AI applications are vulnerable to prompt injection attacks, where users manipulate system inputs to obtain information that should not be accessible. Testing the system against this type of behavior before launch should be part of the security assessment, just like any other vulnerability assessment the technical team would perform on a conventional application.
Whether there is a legal basis for every type of data processing. Consent, legitimate interest, contract performance—every piece of information collected by the system must have a documented legal justification. Systems that collect user data to “improve the model” without explicit user consent operate within a regulatory risk area.
The Risk That No One Maps Before an Incident
A recurring pattern appears in systems that undergo a security review only after a problem occurs: the vulnerabilities that are eventually discovered had been there from the very beginning—they were simply never looked for. No security assessment was performed, no penetration testing was conducted, and no data access policies were formalized. The system worked, and functionality was mistaken for security.
With AI applications, this problem becomes even more significant because the environment is highly dynamic. Models are updated by providers, integrations change versions, and behaviors that did not exist in production just weeks earlier suddenly become part of the system without anyone reviewing their security implications. What was compliant one month may no longer be compliant the next, and only periodic infrastructure reviews can keep pace with these changes.
The good news is that most of the issues described above already have well-established technical solutions. What is missing in most cases is the deliberate decision to pause development, map the actual flow of data, and honestly answer the questions about what is really happening to users’ information.
That moment needs to happen before the system goes live. After deployment, the cost of making corrections becomes significantly higher, and the risk that an incident will expose the problem before the technical team discovers it becomes very real.
STWBrasil supports development teams and business leaders in evaluating the security of AI systems by mapping data flows, identifying vulnerabilities, and verifying LGPD compliance before an incident does that work for you.
