全职 Senior Data Engineer - Databricks 工作, 薪水, RiDiK (a Subsidiary Of CLPS. Nasdaq: CLPS) 公司招聘中 - Ricebowl

Essential Technical Skills

Data Engineering: Strong foundation in data engineering principles, ETL/ELT processes, and data pipeline design patterns

PySpark: Proven hands-on experience developing data pipelines using PySpark, including DataFrames API, Spark SQL, and performance optimization

Databricks Platform: Practical experience with Databricks workspace, cluster management, notebooks, and job orchestration

Workspace AI Agent: Knowledge of Databricks Workspace AI Agent capabilities and integration

Data Modelling: Experience implementing data models including dimensional modeling, data vault, or lakehouse architectures

Delta Lake: Understanding of Delta Lake features including ACID transactions, schema evolution, and optimization techniques

Python: Strong Python programming skills for data processing and automation

Additional Technical Skills

SQL proficiency for data querying and transformation

Experience with cloud platforms (Azure, AWS, or GCP)

Understanding of data governance and security best practices

Knowledge of streaming data processing (Structured Streaming)

Familiarity with DevOps practices and CI/CD pipelines

Experience with version control systems (Git)

Understanding of data quality frameworks and testing methodologies

Professional Experience

Minimum 8 years in data engineering or related roles

At least 2-3 years of hands-on experience with Databricks platform

Proven track record of refactoring legacy code to modern frameworks

Experience building and maintaining production data pipelines at scale

Background working across multiple data sources and formats

Experience in agile development environments