Tanya Joshi

Pro

@tanyajoshi

Pro member

I break prod so you do not have to. Spark by day, real-time analytics on weekends.

Ahmedabad, Gujarat2–5 years1 questions7 answers

SparkKafkaSnowflakeGCPAzure

Streak: 14d
XP: 4816
Lessons done: 36
Studio passes: 30

Fortnight Streak25 Lessons25 Tickets2,500 XP

Share on X Share on LinkedIn

Questions

votes

answers

views

Glue job cost doubled after switching to DynamicFrame, why?

Migrated a PySpark script from the DataFrame API to Glue DynamicFrame for catalog integration. Same logic, but DPU hours went from around 12 to 28 per run. Using bookmark and push_down_predicate on the partition column. Anything obvious that makes DynamicFrame more expensive?

AWSawsglue