全职 Senior Data Engineer 工作, 薪水, Avenga Federal Territory 公司招聘中 - Ricebowl

Job responsibilities:

• Apply data engineering practices and standards to develop robust and maintainable data pipelines
• Design, develop, and maintain scalable data pipelines to handle large data volumes.
• Integrate data from various sources, ensuring quality, consistency, and reliability.
• Contribute to data architecture design to support analytics and machine learning.
• Collaborate with data scientists, analysts, and stakeholders to meet data needs.
• Monitor, optimize, and document data pipelines and storage solutions.
• Ensure compliance with data protection and privacy regulations.
• Making the data available in the data lake through AWS Glue, Appflow and Lakeformation
• Responsible in writing unit/data tests and monitoring the quality of the overall on-boarding process.
• Analyze and organize raw data ingestion pipelines
• Evaluate business needs and objectives
• Support senior business stakeholders in defining new data product use cases and their value
• Take ownership of data product pipelines and their maintenance
• Explore ways to enhance data quality and reliability, be the "Quality Gatekeeper" for developed Data Products
• Adapt and apply best practices from the Data One community
• Be constantly on the lookout for ways to improve best practices and efficiencies and make concrete proposals.
• Take leadership and collaborate with other teams proactively to keep things moving
• Be flexible and take on other responsibilities within the scope of the Agile Team

Job requirements:

• Bachelor's degree in computer science, systems analysis, or a related study, or equivalent experience
• 7-8 years of relevant experience in Data Engineering.
• 2-3 years of DB/ETL development experience - DBT preferred
• Proficiency in programming languages such as Python, Java, SQL
• Experience with AWS cloud platforms
• Solid knowledge of AWS infrastructure is a must in order to build different connectors such as FTP, API or JDBC integrations
• Familiarity with data warehousing solutions (Redshift, BigQuery, Snowflake) and technical lab informatics platforms (ELNs, Benchling, Dotmatics).
• Experience with containerization (such as Docker, Kubernetes) and orchestration tools.
• Git & CI/CD knowledge is a must
• Knowledge of machine learning, large language models, and data science concepts.
• Familiarity with visualization and data tools (Tableau, Spotfire, Dataiku, Thoughtspot, Plotly).
• AWS Certified Solutions Architect – Associate will be advantageous.