Why the 'Science' in DS?
What is the actual difference between ML and data science?
What’s the difference between data science and “data analytics”?
Currently in Data Science. Should I make the move?
I have to say, I think Data engineering is a better field in many ways than data science. I say this AS a data scientist. Demand FAR outstrips supply, and its not as easy to teach people to be a great data engineer vs Masters students printing out data scientists nowadays.
It's a very broad set of skills, but as long as you have the core competencies, people will overlook a large number of gaps tbh.
In my view of the world, data engineers get paid more too (consulting) frequently than data scientists.
From a technical perspective, a unicorn (and I really stress that) data engineer:
-
DevOps skills (e.g. IaC, CI/CD, Jenkins, unit testing, etc)
-
Cloud-native architecture (specifically around the data space)
-
Docker, Kube
-
Spark
-
Kafka
-
Hadoop knowledge (not extensively, I think its a dying pattern, rightly)
-
data modelling
-
aligned to company tech stack across scala/python/Java
-
Event driven architecture
-
Airflow
-
SQL! Both using it as a user, and designing efficient queries and data models
I'm sure more tools hace cropped up in the meantime too, but its a good start
I'm separating out MLOps/ML Engineering as a separate resource, fwiw
More on reddit.com