Post by Rakesh D L

Citi Interview Experience 𝗣𝗼𝘀𝗶𝘁𝗶𝗼𝗻: Data Engineer 𝗘𝘅𝗽𝗲𝗿𝗶𝗲𝗻𝗰𝗲: 3+ Years 𝗣𝗮𝗰𝗸𝗮𝗴𝗲: 15+ LPA 𝗣𝘆𝗦𝗽𝗮𝗿𝗸 & 𝗗𝗶𝘀𝘁𝗿𝗶𝗯𝘂𝘁𝗲𝗱 𝗖𝗼𝗺𝗽𝘂𝘁𝗶𝗻𝗴: 1️⃣ Explain the Catalyst Optimiser workflow. How does it handle rule-based vs. cost-based optimization? 2️⃣ How do you troubleshoot executor memory issues? Detail how to calculate the correct memory overhead. 3️⃣ Difference between Checkpointing and Persistence. When is metadata checkpointing required in streaming? 4️⃣ How do you implement Custom Transformations in PySpark to keep code DRY? 5️⃣ Detail the process of checking skewness specifically via the Spark UI. 𝗔𝘇𝘂𝗿𝗲 & 𝗟𝗮𝗸𝗲𝗵𝗼𝘂𝘀𝗲 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲: 6️⃣ How do you manage ARM Templates using CI/CD pipelines for ADF deployments? 7️⃣ Explain Azure Data Flow partitioning schemes. When is Round Robin better than Hash? 8️⃣ Describe Dynamic Content in ADF. How do you pass runtime parameters to datasets? 9️⃣ How is Data Masking implemented in Azure to protect PII data during the ETL process? 🔟 Detail the two levels of security in ADLS Gen2: RBAC vs. ACLs. 𝗔𝗱𝘃𝗮𝗻𝗰𝗲𝗱 𝗦𝗤𝗟 & 𝗦𝘆𝘀𝘁𝗲𝗺 𝗗𝗲𝘀𝗶𝗴𝗻: 1️⃣1️⃣ Write a SQL query to find the rolling average of sales over a 7-day period. 1️⃣2️⃣ How do you detect and handle gaps in a sequence using Window Functions? 1️⃣3️⃣ Difference between a Correlated Subquery and a nested subquery. Which one impacts performance more at scale? 1️⃣4️⃣ Design a system to detect duplicate events in a real-time stream without storing the entire history. 1️⃣5️⃣ How do you handle scheme drift in a pipeline that consumes data from 50+ different sources? 𝗜 𝗵𝗮𝘃𝗲 𝗽𝗿𝗲𝗽𝗮𝗿𝗲𝗱 𝗖𝗼𝗺𝗽𝗹𝗲𝘁𝗲 𝗜𝗻𝘁𝗲𝗿𝘃𝗶𝗲𝘄 𝗣𝗿𝗲𝗽𝗮𝗿𝗮𝘁𝗶𝗼𝗻 𝗚𝘂𝗶𝗱𝗲 𝗳𝗼𝗿 𝗗𝗮𝘁𝗮 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝘀.. 𝗚𝗲𝘁 𝘁𝗵𝗲 𝗚𝘂𝗶𝗱𝗲 𝗵𝗲𝗿𝗲- https://lnkd.in/gFrNbK87 If you found this useful, a 𝗟𝗶𝗸𝗲 or 𝗥𝗲𝗽𝗼𝘀𝘁 would really help more Data Engineers prepare smarter.