全职 Data Engineer 工作, 薪水, Neurones IT Asia 公司招聘中 - Ricebowl

We are looking for an experienced Databricks & Structured Streaming Developer with strong expertise in Python fundamentals, data structures, Databricks, and Structured Streaming. The ideal candidate should have hands-on experience building and supporting scalable data processing and streaming solutions.

Key Responsibilities:

• Develop and maintain data processing solutions using Databricks and Python.
• Design and implement real-time data processing pipelines using Apache Spark Structured Streaming.
• Develop efficient and scalable data processing logic using strong Python programming fundamentals and data structures.
• Troubleshoot and optimize streaming jobs and data processing workflows.
• Work with large volumes of data and ensure reliable, scalable, and efficient processing.
• Monitor and resolve issues related to streaming pipelines and Databricks workloads.
• Collaborate with data engineers, technical teams, and stakeholders to understand requirements and deliver effective solutions.
• Follow best practices for code quality, performance, maintainability, and operational support.
• Participate in troubleshooting, debugging, and performance optimization of data pipelines.

Required Skills & Experience:

• 5–10 years of overall experience in data engineering / data development.
• Strong hands-on experience with Databricks.
• Strong experience with Apache Spark Structured Streaming.
• Strong Python programming skills with a good understanding of:
• Experience developing and troubleshooting batch and/or streaming data pipelines.
• Good understanding of data processing concepts and distributed computing.
• Strong problem-solving and analytical skills.
• Ability to work independently as well as collaboratively with technical teams.