Data Engineer

About the role

Bentego, a community of engineers helping corporations deliver digital experiences through big data systems and customer journey design on platforms such as Cloudera, Pega and Evam, is looking for a Data Engineer. The role works in a cross-functional squad in a customer environment, building and maintaining batch data pipelines and Spark-based data processing jobs.

Key responsibilities

• Design, develop and maintain batch data pipelines
• Build and optimize Spark-based data processing jobs
• Process and integrate data from multiple sources using Spark-based architectures
• Work with business stakeholders to turn data requirements into data solutions
• Ensure data quality, performance and reliability
• Monitor and troubleshoot data pipeline issues
• Collaborate with engineers and analysts within the customer-side squad
• Schedule and manage data workflows using Apache Airflow
• Contribute to continuous improvement of data engineering standards and best practices

Required profile

• Bachelor's degree in Computer Engineering or a related field
• At least 3 years of experience in data engineering or similar data-focused roles
• Experience working in Linux/Unix environments and knowledge of Git-based workflows
• Ability to work in a cross-functional squad in a customer environment
• B2+ level English proficiency

Required skills

• Python, SQL and Apache Spark
• Kubernetes, including running Spark workloads on it
• Big Data technologies such as Kafka, Hive or Impala, or equivalent open-source tools
• Workflow orchestration with Apache Airflow
• Performance tuning and optimization of Spark applications
• Familiarity with data science / ML concepts and data lakehouse-style architectures

What we offer

• "Great Place to Work" certified culture and continuous learning opportunities
• Net paid salary, cash meal payment and support for remote work expenses
• Company-supported online English platform membership and unlimited private health insurance
• Birthday off and remote or hybrid working