jobs in Net2Source Inc.

Net2Source Inc. Hiring! Full Time Data Architect in Federal Territory - Ricebowl

Data Architect

Net2Source Inc.

Undisclosed

KL City, Federal Territory

Share
Save

Working Location

  • Kuala Lumpur Federal Territory Malaysia

Job Description

Responsibilities

Experience - 12+ Years

Location - Malaysia

Preferred - Local Candidate


  • 12+ years of experience in data engineering using Databricks or Apache Spark-based platforms.
  • Proven track record of building and optimizing ETL/ELT pipelines for batch and streaming data ingestion.
  • Hands-on experience with Azure services such as Azure Data Factory, Azure Data Lake Storage, Azure Synapse Analytics, or Azure SQL Data Warehouse.
  • Proficiency in programming languages such as Python, Scala, or SQL for data processing and transformation.
  • Expertise in Spark (PySpark, Spark SQL, or Scala) and Databricks notebooks for large-scale data processing.
  • Familiarity with Delta Lake, Delta Live Tables, and medallion architecture for data lakehouse implementations.
  • Build and query deltalake storage solutions
  • Process streaming data with Azure Databricks structured streaming
  • Design Azure Databricks security and data protection solutions
  • Flatten nested structures and explode arrays with spark
  • Transfer data outside using sparkpools using pyspark connector
  • Optimizing spark jobs
  • Implementing best practices in spark/databricks
  • Experience with orchestration tools like Azure Data Factory or Databricks Jobs for scheduling and automation.
  • Knowledge of Git for source control and CI/CD integration for Databricks workflows, cost optimization, performance tuning.
  • Familiarity with Unity Catalog, RBAC, or enterprise-level Databricks setups.
  • Ability to create reusable components, templates, and documentation to standardize data engineering workflows.
  • Solutioning and presales - Architecting frameworks, defining roadmaps, and engaging with stakeholders.
  • Experience in defining data strategy, evaluating new tools/technologies, and driving adoption across the organization.
  • Experience with Snowflake and/or Microsoft Fabric is an added advantage.
  • Must have experience of working with streaming data sources and Kafka (preferred).




Eligibility Criteria:

  • Bachelor’s degree in computer science, Information Technology, or related field
  • Proven experience as a Databricks Architect or similar role
  • Complete knowledge in Databricks platform architecture
  • Databricks certification (e.g., Certified Data Engineer)
  • Expertise in Python/Scala/ SQL/R
  • Knowledge of MLOps, data observability, DevOps/CI-CD, and BI tools.
  • Experience with DataOPS & AIOPS
  • Experience with cloud platforms like AWS, Azure, or GCP or Alibaba
  • Strong understanding of data modeling and cloud integration
  • Experience with cluster sizing and security implementation
  • Excellent problem-solving and communication skills

Important Information

Never provide your bank or credit card details when applying for jobs. Do not transfer any money or complete unrelated online surveys. If you see something suspicious, Report this Job ad.

Learn More