Like many businesses, we at Essent IT faced the challenge of dealing with GDPR compliance in a simple, repeatable way.

To overcome this challenge, we used a technique called pseudonymization. It simply means replacing personal identifiable data with artificial artifacts in a way that still allows analyzing and processing of the data. It complies with EU regulations, and it is possible to get back to the original data when using additional information.

To achieve a scalable solution, we used the standard Kafka connect architecture with a sink as the base. Records that are received by this sink are pseudonymized and in the meantime, a second record is created. This second record contains the additional information that is required to get back to the original data.

The resulting records are placed on two separate output topics and further processing is performed just as we would normally do. For security reasons (and compliance), the topic with the additional information is only available to those that are allowed to see and/or process that information.

This approach allows us to process GDRP-related information in a standardized and simplified manner. By creating a configuration file that specifies fields and the type of processing, we are compliant with the rules. A second benefit is that we can scale easily by creating a new rule set and, if needed, spin up an additional worker.

Have you been struggling to find a scalable approach to comply with the GDPR rules? By pressing the play button in the image above, you can watch the 30 minute long video of the talk Pieter van der Meer delivered at the Kafka Summit London 2023 to better understand the challenge and how we at Essent IT addressed it.

If you have any lingering questions, do not hesitate to drop a comment below!