Senior Data Engineer - Weather Forecasting

About the role

Buluttan builds hyper-local weather forecasting solutions using Numerical Weather Prediction and AI models, supporting renewable energy and mobility. It is looking for a Senior Data Engineer to work closely with the Data Science and Atmospheric Sciences team, building and maintaining data pipelines, ensuring data quality and supporting analytics and machine learning. The Data Engineering team is responsible for pipelines that feed the final prediction products, so the role also contributes to product design and engineering.

Key responsibilities

• Design, develop and maintain scalable data pipelines and ELT processes for large volumes of weather data
• Build data ingestion pipelines in Python to extract data from third-party sources
• Implement data validation and monitoring to ensure data quality and integrity
• Optimize and manage the cloud-based data lake and warehouse architecture
• Manage cloud service permissions and security, and build automation tools for them
• Develop and maintain data models in Python and SQL pipelines that feed end products, services and ML and atmospheric models
• Contribute to product design and development with subject-matter expertise on data pipelines, models and architecture
• Help create data visualizations and dashboards for internal and external stakeholders
• Mentor junior engineers and scientists

Required profile

• Proven experience as a Data Engineer, preferably building pipelines that integrate with operational products
• Strong Python and SQL skills, with relational databases and data warehouses (BigQuery, Redshift, MySQL, PostgreSQL)
• Cloud experience (Google Cloud Platform, AWS, Azure) including data warehouses, data lakes, security and permissions
• Understanding of data lakehouse, change data capture, data marts, star/snowflake schemas, data mesh/fabric and platform engineering
• Linux/Unix command line, shell scripting and basic system administration
• Object-oriented programming knowledge and collaboration in GitHub repositories

Required skills

• Data transformation (dbt, DataBricks) and orchestration (Airflow, AWS Glue)
• Monitoring and logging (DataDog, Logstash)
• Nice to have: Grafana, Tableau, Looker; APIs; Zarr and GeoJSON; image and radar file processing; passion for meteorology