Senior Data Engineer / AI Data Platform Engineer (Spark / ETL / Cloud)

About the role

We are seeking a Senior Data Engineer / AI Data Platform Engineer to design and build high‑performance, scalable data pipelines that power AI/ML systems and data‑driven applications. The role focuses on distributed computing, large‑scale data processing, and cloud‑native architectures using Apache Spark, Python, and modern data platforms.

Key responsibilities

• Design and implement scalable data pipelines for AI/ML workloads, supporting both batch and streaming use cases.
• Develop distributed processing systems with Apache Spark, optimizing performance and resource utilization.
• Build and maintain ETL/ELT pipelines that ingest structured and unstructured data from logs, events, and APIs.
• Architect and operate data platforms on AWS, Azure, or Google Cloud, applying best practices for security, observability, and scalability.
• Collaborate with ML Engineers and Software Engineers to provide reliable data for model training and inference.
• Implement parallel processing, partitioning strategies, and performance tuning for large datasets (10 M–1 B+ records).

Required profile

• Proven experience building large‑scale ETL/ELT pipelines and data platforms.
• Strong background in distributed systems and high‑volume data processing.
• Hands‑on expertise with cloud environments (AWS, Azure, or GCP).
• Experience supporting AI/ML pipelines, feature engineering, or data preparation is a plus.

Required skills

• Apache Spark (core, performance optimization, Spark Streaming)
• Python (data‑focused programming)
• Advanced SQL (analytical and optimization)
• Cloud platforms: AWS, Azure, Google Cloud
• ETL/ELT pipeline development
• Kafka or similar real‑time messaging systems
• Databricks, Snowflake, BigQuery
• Docker, Kubernetes
• Knowledge of data lakehouse architectures and MLOps workflows

What we offer

• Competitive USD‑based salary aligned with AI/Engineering market.
• Hybrid remote work model with flexibility.
• Opportunity to work on AI‑driven systems and high‑impact engineering projects.