Creating data pipelines, ensuring data quality, and implementing ETL processes to migrate and deploy data across systems.
To build and maintain scalable pipelines on Azure Databricks, with strong SQL and relational database expertise and solid DevOps/SDLC practices from design through deployment.
Develop data connectors and pipelines to onboard agency datasets efficiently
Requirement :
Degree in Computer Science with 3 to 5 years hands on experience in Databricks
Hands-on experience building data pipelines and notebooks in Azure Databricks