For instance, a user with “Persona A” who belongs to “Department 1” should NOT have access to any financial data but gets full access to claims data.
Departments represent the different line of business in a company.Īn example of a business rule as depicted in the table signifies the complexity of access management.AWS-managed policies can also be managed through pre-defined service roles such as for Amazon EMR and Amazon SageMaker. Services define the permissions as templates, which can be converted into an inline policy or customer-managed policy.Persona defines the profile of the end users who may require a specific list of AWS services.Subject area is a classification of the data within the data lake.
The table below explains various dimensions of such complexity: Subject area/persona/department Some of the rules are also in place due to business needs. Some of the rules applied are created by the security team to ensure each role has exactly the level of access it requires to accomplish the task at hand. Most enterprises either tag data with their subject areas or create independent S3 buckets. Security teams are always looking to govern the process with stringent enforcement. Complexity is increased by introducing various user personas, data subject areas (data classification), type of employees (full time versus contractors), and more.Granting complex roles and permissions to business users on diverse datasets.Demand to log approval processes and track every change for audit purposes.Compliance requirements and sensitivity of data.In creating these rules, they often face challenges: Lake House Access Management ComplexitiesĪs a Lake House comprises a central repository for data-which is a collection from various sources-organizations need a complex set of rules to manage their Lake House access. Increased demand to extract information and knowledge from the data lake has made services like Amazon SageMaker, Amazon Redshift, AWS Glue, and Amazon Athena an integral part of the Lake House ecosystem as well. These stakeholders form the top layer of the Lake House and create value by the extraction of meaningful data and inferences, while Amazon S3 provides a central repository. In a Lake House approach, various stakeholders access the data through Access Management Layer for usage ranging from developing insights, ensuring security, creating and training ML models, and more. This makes Amazon S3 a great choice for data lake, which is a core component for AWS Lake House implementations.įigure 1 – Enterprise Lake House architecture. Most of the organizations today looking to build Lake House-based solutions choose Amazon Web Services (AWS) due to the availability of custom solutions and depth and breadth of AWS offerings.įor instance, Amazon Simple Storage Service (Amazon S3) offers durability, availability, performance, security, and virtually unlimited scalability at low cost. We’ll also discuss the TCS EZLA solution overview, architecture, and functions, and review the benefits of the solution as a case study from a large life science enterprise. In this post, we’ll describe the Lake House ecosystem, complexities, and common challenges. This provides increased efficiencies and easy adoption of the Data Lake House. The EZ Lake Access (EZLA) solution developed by Tata Consultancy Services (TCS), an AWS Premier Consulting Partner, centralizes and simplifies access management of the Data Lake House by codifying most of the enterprise access controls in the form of a rule engine.
They use this information to customize their products and improve customer experience using data lake solutions.Ī Lake House architecture is defined by a central repository (data lake) which allows ingestion of unstructured, structured, and real-time data that’s consumed by various processes like analytics engine, data warehouses, machine learning (ML) models, and visualization tools. Many organizations leverage unstructured data collected from social media feeds, stock streaming, and data clickstream to gain insights about the needs of their customers. By Jitesh Bhattacharjee, Delivery Partner – TCSīy Nicolas Weydert, Chief Architect – TCSīy Sanjay Gupta, Sr.