We are seeking a Lead GCP Data Engineer to design, build, and optimize scalable data platforms and pipelines on Google Cloud Platform (GCP). Youll lead a team of data engineers, drive data architecture decisions, and ensure the reliability and performance of our data infrastructure supporting analytics, AI/ML, and business intelligence initiatives.
Key Responsibilities
- Lead the design and implementation of end-to-end data pipelines using GCP services (BigQuery, Dataflow, Dataproc, Pub/Sub, Cloud Composer, etc.).
- Architect and optimize data lake and data warehouse solutions for scalability and performance.
- Define and implement data ingestion, transformation, and integration strategies using batch and streaming methods.
- Collaborate with data scientists, analysts, and business stakeholders to define data requirements and deliver insights.
- Enforce data governance, security, and quality best practices.
- Mentor and guide a team of data engineers; review code and ensure adherence to engineering standards.
- Automate and monitor data workflows using CI/CD pipelines and infrastructure-as-code (IaC) (Terraform / Deployment Manager).
- Evaluate new GCP tools and technologies to improve efficiency and reliability.
Required Skills & Qualifications
- Bachelor's or Master's degree in Computer Science, Information Systems, or a related field.
- 7–10 years of experience in data engineering, with at least 3 years in a lead or senior role.
- Strong hands-on expertise with GCP data services (BigQuery, Dataflow, Dataproc, Pub/Sub, Cloud Storage, Composer, etc.).
- Proficiency in Python and SQL for data processing and automation.
- Experience with ETL/ELT design, data modeling, and data warehousing principles.
- Working knowledge of Apache Beam, Spark, or other distributed data frameworks.
- Experience implementing CI/CD pipelines and IaC using Terraform, Cloud Build, or similar tools.
- Familiarity with Airflow, DBT, or other orchestration/transformation frameworks.
- Excellent problem-solving, leadership, and communication skills.