I know that job titles varies from company to company, so the title of this post could be interpreted differently.

For my current job, I am a Engineer. I work on batch ETL systems in Python and Airflow. And these systems write into a data warehouse that is built in AWS Redshift. So, I feel like a DW Engineer and an ETL dev. But honestly, I want to get more into concurrent systems, data pipelines, distributed computing, and big data. I feel like the big data engineering world is still using Java and Scala to do most of this work, but I have no real experience on big data systems in production. I would like to get some advice.

I love learning new programming languages, and I really want to learn Scala. Especially because I love the elegant programming principles that come with functional programming.

tl;dr Should I learn Scala for Big Data Engineering? Or since I’m most interested in the functional part of Scala, should I just learn how to write Python in a more functional way?

Source link

No tags for this post.


Please enter your comment!
Please enter your name here