A data engineer must build an extract, transform, and load (ETL) pipeline to process and load data from 10 source systems into 10 tables that are in an Amazon Redshift database. All the source systems generate .csv, JSON, or Apache Parquet files every 15 minutes. The source systems all deliver files into one Amazon S3 bucket. The file sizes range from 10 MB to 20 GB. The ETL pipeline must function correctly despite changes to the data schema. Which data pipeline solutions will meet these requirements? (Choose two.)
Answer(s): B,D
The correct answers are B and D . Here's a detailed justification:Option B is correct because:EventBridge Scheduling: EventBridge rules provide a reliable and scalable way to trigger ETL pipelines on a schedule (every 15 minutes, as required). AWS Glue Workflow: Glue workflows orchestrate complex ETL processes. They allow chaining multiple jobs and crawlers. On-demand Glue Crawler: The crawler automatically detects schema changes in the source data, addressing the requirement that the pipeline must function despite schema evolution. Crawlers infer the schema and register the tables in the AWS Glue Data Catalog. Glue Job for Transformation and Loading: After the crawler updates the schema, the Glue job transforms the data according to the new schema and loads it into Amazon Redshift. Handles Variety of File Types: Glue can handle .csv, JSON, and Apache Parquet files.Option D is correct because:Lambda Triggering: A Lambda function triggered by S3 events provides a mechanism to initiate the workflow whenever a new file is loaded into the bucket. AWS Glue Workflow with Crawler: This component is the same as in option B, and it handles schema changes. Glue Job for Transformation and Loading: Similar to option B, the Glue job executes the transformation and loading logic into Amazon Redshift, after the schema is updated by the Glue Crawler. Event-Driven Architecture: D uses an event-driven setup using an AWS Lambda function that triggers a Glue Workflow, which enables quick processing of files as they arrive in the S3 bucket.Why other options are incorrect:Option A: A direct Glue job without a crawler is less robust to schema changes. The job could fail if the schema changes. Option C: This option creates a complex interaction between two Lambda functions with an event-driven architecture that is not needed, making it less optimal than the others. Option E: While Lambda can trigger Glue jobs, using Kinesis Data Firehose for loading into Redshift is less efficient and appropriate for large file sizes. Firehose is better suited for streaming data. Glue can natively load data into Redshift with higher throughput. This architecture would be harder to manage and configure.Authoritative Links:Amazon EventBridge: https://aws.amazon.com/eventbridge/ AWS Glue: https://aws.amazon.com/glue/ AWS Glue Workflows: https://docs.aws.amazon.com/glue/latest/dg/workflows-using.html AWS Glue Crawlers: https://docs.aws.amazon.com/glue/latest/dg/add-crawler.html AWS Lambda: https://aws.amazon.com/lambda/ Amazon Redshift: https://aws.amazon.com/redshift/
A financial company wants to use Amazon Athena to run on-demand SQL queries on a petabyte-scale dataset to support a business intelligence (BI) application. An AWS Glue job that runs during non-business hours updates the dataset once every day. The BI application has a standard data refresh frequency of 1 hour to comply with company policies. A data engineer wants to cost optimize the company's use of Amazon Athena without adding any additional infrastructure costs. Which solution will meet these requirements with the LEAST operational overhead?
Answer(s): B
The most cost-effective and least operationally burdensome solution is B: Use the query result reuse feature of Amazon Athena for the SQL queries.Here's a detailed justification:Athena's query result reuse is a built-in feature that automatically caches and reuses query results. If a query is run again within a specified timeframe (configurable), and the underlying data hasn't changed, Athena retrieves the results from the cache instead of re-scanning the data in S3. Since the BI application refreshes every hour and the dataset is updated only daily, the majority of queries during that hourly window will be identical and thus be served from the cache. This significantly reduces data scanned by Athena, translating directly to cost savings because Athena's pricing is primarily based on data scanned per query. This method avoids adding any infrastructure overhead, which is vital considering the constraint.Option A (S3 Lifecycle to Glacier Deep Archive) would dramatically increase query latency and thus negatively impact the BI application's responsiveness. S3 Glacier Deep Archive is designed for long-term archival, not frequent retrieval.Option C (ElastiCache) would introduce infrastructure management overhead. ElastiCache requires provisioning, configuration, scaling, and monitoring, adding unnecessary complexity and potential costs, which directly contradicts the requirement for minimal operational overhead and cost optimization. While it could theoretically cache query results, Athena's built-in feature already fulfills this purpose.Option D (Parquet format) is beneficial for cost optimization in general by reducing data size and improving query performance. However, given the immediate requirement for cost optimization without adding infrastructure costs and with minimal operational overhead, and the already existing dataset and Glue job, changing the data format would introduce substantial initial effort (data conversion) and potentially disrupt existing ETL processes. Also, it may not provide immediate and noticeable impact within the defined timeframe as query result reuse. The query result reuse feature offers immediate cost reduction with zero setup.Therefore, Athena's query result reuse feature directly addresses the problem by leveraging existing capabilities and achieving cost optimization with minimal effort and no additional infrastructure.Authoritative links:Amazon Athena Pricing: https://aws.amazon.com/athena/pricing/ (Shows cost based on data scanned.) Using Query Result Reuse: https://docs.aws.amazon.com/athena/latest/ug/querying-with-reuse.html (Explains the feature and its configuration.)
A company's data engineer needs to optimize the performance of table SQL queries. The company stores data in an Amazon Redshift cluster. The data engineer cannot increase the size of the cluster because of budget constraints. The company stores the data in multiple tables and loads the data by using the EVEN distribution style. Some tables are hundreds of gigabytes in size. Other tables are less than 10 MB in size. Which solution will meet these requirements?
Answer(s): C
The correct answer is C. Let's break down why this solution works and why the others don't.Why C is correct:ALL Distribution for Small, Rarely Updated Tables: The ALL distribution style replicates the entire table to every node in the Amazon Redshift cluster. For small tables (less than 10 MB), the overhead of replication is minimal. Since the tables are rarely updated, the cost of propagating changes across all nodes is also low. This allows each node to perform joins locally without needing to transfer data across the network, significantly improving query performance, especially when joining these small tables with larger distributed tables. This is a key optimization technique in data warehousing. Primary and Foreign Keys (Informational): While not directly related to data distribution, specifying primary and foreign keys provides valuable metadata to the Redshift query optimizer. It can assist in generating more efficient query plans, though the actual impact is less pronounced than the distribution strategy. The query optimizer knows about the data relationships allowing it to make better choices.Why the other options are incorrect:A: Keep using the EVEN distribution style for all tables: The EVEN distribution style distributes rows evenly across all nodes in the cluster. While simple, it doesn't consider data relationships, leading to significant data redistribution during joins. This data movement across the network becomes a bottleneck, especially for large tables. This distribution style does not optimize the location of the data for querying. B: Use the ALL distribution style for large tables: Replicating large tables (hundreds of gigabytes) to every node is highly inefficient. It consumes excessive storage on each node, increases the time for data loading and updates dramatically, and may even lead to performance degradation due to resource contention (memory, disk I/O) on each node. D: Specify a combination of distribution, sort, and partition keys for all tables: While crucial in general Redshift optimization, this isn't the most targeted solution for the described scenario. While distribution keys would help, the prompt clearly identified that some tables are small and this is where a different strategy is needed. Distribution, sort, and partition keys is a general recommendation that does not optimize this particular problem.In summary: ALL distribution is a good strategy for small, relatively static tables in Amazon Redshift, enabling faster local joins. Specifying keys helps the query optimizer find efficient ways to perform queries.Authoritative Links:Amazon Redshift Data Distribution Styles: https://docs.aws.amazon.com/redshift/latest/dg/t_Distributing_data.html Amazon Redshift Key Concepts https://docs.aws.amazon.com/redshift/latest/dg/c_best-practices-sort-dist.html
A company receives .csv files that contain physical address data. The data is in columns that have the following names: Door_No, Street_Name, City, and Zip_Code. The company wants to create a single column to store these values in the following format:Which solution will meet this requirement with the LEAST coding effort?
Use AWS Glue DataBrew to read the files. Use the NEST_TO_MAP transformation to create the new column.
A company receives call logs as Amazon S3 objects that contain sensitive customer information. The company must protect the S3 objects by using encryption. The company must also use encryption keys that only specific employees can access. Which solution will meet these requirements with the LEAST effort?
The correct answer is C: "Use server-side encryption with AWS KMS keys (SSE-KMS) to encrypt the objects that contain customer information. Configure an IAM policy that restricts access to the KMS keys that encrypt the objects."Here's why: The question emphasizes ease of implementation and controlled access to encryption keys. SSE-KMS is the most straightforward approach to achieving both. SSE-KMS integrates seamlessly with S3 and provides fine-grained control over who can use the encryption keys through IAM policies. This means you can easily grant access to the keys only to specific employees as required. When data is uploaded, S3 uses the KMS key to encrypt the object server-side. No code changes are required in the application writing to S3.Option A, using CloudHSM, involves significantly more complexity. CloudHSM requires managing a dedicated hardware security module cluster. This includes tasks such as provisioning, patching, and key management within the HSM. The application writing to S3 needs to be modified to directly interact with CloudHSM for encryption, increasing development and maintenance overhead.Option B, SSE-C, places the responsibility of key management entirely on the user. While it allows restricting access to keys, it adds the burden of securely storing, rotating, and providing these keys with every request. S3 doesn't manage the keys at all.Option D, SSE-S3, offers the least control. Amazon S3 manages the encryption keys, and while IAM can restrict access to the S3 bucket itself, you cannot restrict access to the encryption keys within the service in the same granular way you can with KMS. You cannot specify that only specific employees have access to the encryption keys.In summary, SSE-KMS offers the best balance of security, control, and ease of implementation for encrypting S3 objects with restricted key access. It allows you to easily control which employees can use the keys without needing to manage the key infrastructure, making it the least effort solution.Relevant Links:AWS KMS Encryption: Protecting Data Using Server-Side Encryption: SSE-KMS: IAM Policies:
A company stores petabytes of data in thousands of Amazon S3 buckets in the S3 Standard storage class. The data supports analytics workloads that have unpredictable and variable data access patterns. The company does not access some data for months. However, the company must be able to retrieve all data within milliseconds. The company needs to optimize S3 storage costs. Which solution will meet these requirements with the LEAST operational overhead?
Answer(s): D
The correct answer is D: Use S3 Intelligent-Tiering. Use the default access tier. Here's why:Requirement Match: The problem states the company needs millisecond retrieval times and has variable, unpredictable access patterns with some data being infrequently accessed. S3 Intelligent-Tiering is designed precisely for this scenario. It automatically moves data between frequent, infrequent, and archive access tiers based on access patterns, without any operational overhead.Cost Optimization: S3 Intelligent-Tiering optimizes storage costs by automatically moving infrequently accessed data to lower-cost tiers like the Infrequent Access tier and the Archive Access tier. This helps reduce overall storage expenses compared to keeping all data in S3 Standard.Low Operational Overhead: S3 Intelligent-Tiering requires minimal configuration. Once enabled on a bucket or object, it manages tiering automatically. This eliminates the need for manually creating and refining S3 Lifecycle policies, as suggested in option A.Millisecond Retrieval: Regardless of the tier an object is in within S3 Intelligent-Tiering, the retrieval time remains the same (milliseconds). This satisfies the stringent retrieval time requirement.Why other options are incorrect:Option A: While using S3 Storage Lens and S3 Lifecycle policies can optimize costs, it introduces significant operational overhead. Continuously monitoring metrics and refining policies is time-consuming and error-prone. It doesn't directly leverage the automated intelligence of S3 Intelligent-Tiering. Also, using S3 Glacier, even through lifecycle policies, would violate the millisecond retrieval requirement. Option B: Similar to option A, this involves manual configuration of S3 Lifecycle rules and does not leverage the automated advantages of Intelligent-Tiering. Also, using S3 Glacier violates the millisecond retrieval time requirement. Option C: Activating the Deep Archive Access tier would introduce longer retrieval times (hours), violating the millisecond retrieval requirement.Therefore, S3 Intelligent-Tiering with the default access tier offers the best balance of cost optimization, millisecond retrieval times, and minimal operational overhead for the given scenario.Authoritative Links:S3 Intelligent-Tiering: https://aws.amazon.com/s3/storage-classes/intelligent-tiering/ S3 Storage Classes: https://aws.amazon.com/s3/storage-classes/
During a security review, a company identified a vulnerability in an AWS Glue job. The company discovered that credentials to access an Amazon Redshift cluster were hard coded in the job script. A data engineer must remediate the security vulnerability in the AWS Glue job. The solution must securely store the credentials. Which combination of steps should the data engineer take to meet these requirements? (Choose two.)
Answer(s): D,E
The vulnerability lies in storing database credentials directly within the AWS Glue job script. This is a major security risk as it exposes sensitive information. Options A, B and C are not secure practices. Option A stores credentials in Glue Job parameters, which are often accessible and not designed for secure storage. Option B and C store credentials in an S3 bucket, which, while potentially more obfuscated than hardcoding, are still vulnerable if the bucket permissions are misconfigured or if the file is inadvertently exposed.Option D, storing the credentials in AWS Secrets Manager, is a best practice for securely managing secrets. Secrets Manager is a dedicated service designed to store, rotate, and manage sensitive information like database credentials, API keys, and passwords.Option E, granting the AWS Glue job IAM role access to the stored credentials in Secrets Manager, is crucial for enabling the Glue job to retrieve the credentials securely. The IAM role provides the Glue job with the necessary permissions to call the Secrets Manager API and access the specific secret containing the credentials. Without this IAM permission, the Glue job would be unable to retrieve the credentials, even if they are stored securely in Secrets Manager.Therefore, storing the credentials in AWS Secrets Manager (Option D) and granting the AWS Glue job IAM role access to those credentials (Option E) provides a secure and auditable way to manage and access sensitive information, mitigating the original vulnerability of hardcoding credentials. This approach aligns with the principle of least privilege and reduces the risk of credential compromise.Relevant Links:AWS Secrets Manager: https://aws.amazon.com/secrets-manager/ IAM Roles: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html AWS Glue Security: https://docs.aws.amazon.com/glue/latest/dg/security-iam.html
A data engineer uses Amazon Redshift to run resource-intensive analytics processes once every month. Every month, the data engineer creates a new Redshift provisioned cluster. The data engineer deletes the Redshift provisioned cluster after the analytics processes are complete every month. Before the data engineer deletes the cluster each month, the data engineer unloads backup data from the cluster to an Amazon S3 bucket. The data engineer needs a solution to run the monthly analytics processes that does not require the data engineer to manage the infrastructure manually. Which solution will meet these requirements with the LEAST operational overhead?
The correct answer is B: Use Amazon Redshift Serverless to automatically process the analytics workload.Here's why:Redshift Serverless Benefits: Redshift Serverless eliminates the need for manual infrastructure management. It automatically provisions and scales compute and data warehouse capacity to deliver fast performance for demanding analytics workloads. This aligns perfectly with the requirement of minimizing operational overhead. https://aws.amazon.com/redshift/serverless/Eliminates Cluster Management: Because Redshift Serverless is, well, serverless, the data engineer doesn't have to worry about creating, deleting, pausing, or resuming clusters. This simplifies the monthly process significantly, reducing manual intervention.Cost Optimization: Redshift Serverless automatically scales resources based on demand, so you only pay for what you use. The described workload only requires monthly processing, so Redshift Serverless would be highly efficient and cost-effective, as it would only consume resources during that period.Why other options are less suitable:A (Step Functions with Provisioned Redshift): While Step Functions can automate the process of pausing and resuming a Redshift cluster, it still requires the data engineer to provision and manage the cluster initially, which increases operational overhead compared to Redshift Serverless.C (AWS CLI): The AWS CLI is a powerful tool for managing AWS resources, but it does not, by itself, provide a serverless solution for running analytics. It would require scripting and orchestration to create, manage, and shut down a provisioned Redshift cluster.D (CloudFormation): CloudFormation templates can automate infrastructure provisioning, but they still require the data engineer to define and manage the resources. It does not inherently eliminate infrastructure management in the same way as a serverless service.In summary, Redshift Serverless provides the simplest and most automated solution for running monthly analytics workloads without requiring manual infrastructure management, thus meeting the requirement of least operational overhead.
Share your comments for Amazon Amazon-DEA-C01 exam with other users:
Question 1:
Question 5:
Why this is correct
Question 7:
Question 104:
clustering keys
Q23: Fabric Admin is correct. Because Domain admin cannot create domains. Only Fabric Admin can among the given options. Q51: Wrapping @pipeline.parameter.param1 inside {} will return a string. But question requires the expression to return Int, so correct answer should be @pipeline.parameter.param1 (no {})
Question 62:
ZDX
Analyze Score
Y Engine
Question 32:
Question 3:
date = sys.argv[1]
sys.argv[1]
date = spark.conf.get("date")
input()
date = dbutils.notebooks.getParam("date")
dbutils.notebook.run
Question 528:
Question 23:The correct answer is Domain admin (option B), not Fabric admin.
Question 2:For question 2, the key concept is the Longest Prefix Match. Routers pick the route whose subnet mask is the most specific (largest prefix length) that still matches the destination IP. From the options:
Question 129:Correct answer: CNAME
compute.osAdminLogin
enable-oslogin
Question 2:
Recommend using AI for Solutions rather the Answer(s) submitted here
This is very interesting
Are these the same questions you have to pay for in ExamTopics?
For Question 7 - while the answer description indicates the correct answer, the option no. mentioned is incorrect. Nice and Comprehensive. Thankyou
This is very good and accurate. Explanation is very helpful even thou some are not 100% right but good enough to pass.
The DP-900 exam can be tricky if you aren't familiar with Microsoft’s specific cloud terminology. I used the practice questions from free-braindumps.com and found them incredibly helpful. The site breaks down core data concepts and Azure services in a way that actually mirrors the real test. As a resutl I passed my exam.
interesting
Passed this exam 2 days ago. These questions are in the exam. You are safe to use them.
Helpful to test your preparedness before giving exam
Really helped
Good explanation
very helpful
Question 1, Ans is - Developer,Standard,Professional Direct and Premier
Passed this exam in first appointment. Great resource and valid exam dump.
Today I wrote this exam and passed, i totally relay on this practice exam. The questions were very tough, these questions are valid and I encounter the same.
Anyone used this dump recently?
173 question is A not D
Keeping this site free takes real effort. We constantly battle automated scraping and unauthorized content copying. A quick account helps us protect the community and keep the site free.
To continue studying for your Amazon-DEA-C01, please sign in or create a free account.