Post by Prateek Jalgaonkar

😴Spark Lazy Evaluation (aka “I’ll do it later” mode) Spark be like: 🙂‍↕️ “You want to filter?” “Cool.” 🙂‍↕️“Add a join?” “Noted.” 🙂‍↕️“Another transformation?” “Relax, I’m just planning.” Nothing actually runs. Spark is busy 😏: Writing things down 📝 Drawing a DAG Optimizing the plan Only when you say .show() / .count() / .write() 👇 -> Spark goes: 🚀 “Alright, NOW we run.” That’s Lazy Evaluation. Plan first. Execute later. Complain never 😄 #ApacheSpark #DataEngineering #BigData #LazyEvaluation #LearningInPublic