Job description
The Data Engineer is responsible for designing, building, and maintaining scalable, reliable data pipelines and platforms that support enterprise analytics, business intelligence, and data‑driven decision‑making.
This role works closely with data scientists, analysts, product teams, and business stakeholders to ensure high‑quality data availability across the organization, while adhering to enterprise standards for performance, security, and governance.
Key Responsibilities
This role works closely with data scientists, analysts, product teams, and business stakeholders to ensure high‑quality data availability across the organization, while adhering to enterprise standards for performance, security, and governance.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines and ETL processes for structured and unstructured insurance data.
- Databricks certified or equivalent work experience, you will have experience in various data management architectures such as Data Warehouse, Data Lake, Data Fabrics and Data Hub.
- Demonstrated experience in handling unstructured data and structured data with large, complex, and multiple data sets from various sources.
- Build and maintain production‑ready ML pipelines, including feature engineering, model training, deployment, and monitoring.
- Collaborate with actuaries, underwriters, and business analysts to translate insurance operations requirements into technical solutions.
- Optimize data warehouse and lakehouse architectures to support real-time analytics and machine learning models.
- Partner with cross-functional teams on digital transformation projects, including customer insights, claims automation, and risk modeling.
- Architect and maintain scalable ETL/ELT pipelines with scheduling, caching, partitioning, modelling, schema evolution, and lineage tracking to support both batch and real-time streaming.
- Partner with analytics and product teams to operationalize AI-driven data solutions across the insurance business.
- Bachelor’s or Master’s degree in Computer Science, Computer Engineering, or a related discipline.
- 3–7+ years of experience as a Data Engineer or in a similar role (enterprise environment preferred).
- Strong proficiency in SQL and at least one programming language (Python, Java, or Scala).
- Hands‑on experience with data warehousing and data pipeline development.
- Familiarity with cloud platforms (AWS or Azure) and modern data warehousing concepts.
- Hands-on with CI/CD (GitLab/Jenkins), Terraform, and automation scripting.
- Proven experience in data pipeline development and maintenance.
- Strong expertise in distributed data processing and streaming architectures.
- Proficiency in Python programming for automation and data engineering tasks.
- Solid knowledge of SQL, data modeling, and ETL/ELT processes.
