Data can be classified as structured, unstructured, governed, ungoverned. They are collectively referred to as a data swamp. The 3D’s of data, as I would like to call it, are: Distributed, Dynamic and Diverse. There is yet another D for Data that has gained a lot of attention over the last decade – data is Difficult to manage. This is true because of the sheer volume and the security or privacy problems data poses.
Market Drivers for Data Privacy and Security:
The need for Data Privacy and Security is at an all-time high. The quantity of data being produced every second are astronomical. Several key factors are driving the Data Privacy and Security market:
- Regulation – Given how valuable data is, over 80 countries (encompassing two-thirds of the world’s population) will be covered by Data Protection Regulation by the end of 2023. The challenge is that not all Regulation is the same. Some countries take a “GDPR-like” approach around consent, rights of a Data Subject etc. Other countries have a “Schrems II-like” approach where the data of an EU citizen cannot be sent to the USA. Still others have incorporated data localisation laws to ensure that data is only processed in a certain country.
- Digital Transformation – During Covid, companies were forced to adapt with the working from home mandate. It has been estimated that Digital Transformation accelerated by seven years of growth over a two year period. A great deal of this transformation was fuelled by APIs, so much that Gartner has recently indicated that APIs are the largest IT threat vector, with 91% of organisations having experienced a security incident related to APIs.
- Reputation Risk – As mentioned above, people are now aware of the value of their personal data. Given its value, they expect organisations to protect it. Research shows that 87% of people would not do business with a company that did not adequately protect their data. Another aspect of risk is that following a data breach, the average drop in the share price for publicly traded companies is 7%. It is for this reason that 94% of organisations are now reporting Privacy metrics to their Board.
The Challenges of Data Privacy and Security (and Why it is Hard):
Data is Everywhere – Historically, data in an organisation was stored in a database and it was fed to an application and the exchange of data from database to application was simple. It made logical sense to protect the data in the database as it sat there (“Data at Rest”).
As modern IT architecture has evolved, data is now stored in hundreds if not thousands of databases and it is feeding hundreds if not thousands of applications. These databases may be on-premise or on a Cloud providers database. Or it may be on multiple Cloud providers databases.
Couple this with the digital transformation journeys that many companies have undertaken since Covid, and you now have a sharp increase in Application Programming Interfaces, or (“APIs”), as well as the adoption of Event-Driven Architectures (EDA) and real-time streaming/processing capabilities (such as Apache Kafka). This has led to a complete new set of issues when it comes to data protection and access control.
Now that data is flowing via multiple mechanisms, the traditional paradigm of protecting Data at Rest alone no longer makes sense. The way digital transformation has evolved and how data is now consumed means the backends know less about the consumer as they become abstracted away.
Policies, Policies, Everywhere….
Data Governance – Given that data is everywhere, and we are losing sight as to where it is held or who is consuming it, firms are attempting to mitigate this risk by putting in new controls and policies. Compliance will have a policy, Legal will have a policy, the CISO will have a policy, the Chief Data Officer will have a policy, and the list goes on. These policies are most typically posted to an internal intranet site, and staff are expected to 1) find them and 2) interpret them on their own. Couple that with the external policies and Regulations that a firm must comply with and the challenge of protecting data that is everywhere becomes a monumental task that just cannot be done manually.
Automate the policies I hear you say…
At eXate we had an epiphany moment, we observed how the API gateways were being leveraged to allow simple flows to allow different services to be proxied and intercepted. The gateways already understand about the consumer, are the ones surfacing the data. The bigger aha moment came when we started to see that the gateways were expanding their scope to all data in motion and by leveraging web sockets to allow much larger flows of the data. Here is an opportunity to change how we use our gateways and potentially including internal APIs to use the same pattern. The opportunity is to leverage the gateways (especially the likes of Gravitee and Kong that you can install anywhere) to intercept data flows to ensure we are applying principles of least privilege, ensuring data isn’t falling foul of data regulation and we are only using data for permitted uses. Apigator™ by eXate is solving these challenges today by adding data protection and security at Gateways to centralise and govern data. For more information on this, please contact us at firstname.lastname@example.org