Senior Data Engineer

Beschreibung

Job Zusammenfassung In dieser Position entwickelst du Pipelines für die Datenaufnahme aus verschiedenen ERP-Systemen sowie Transformationen zur Erstellung von Finanzberichten mithilfe von Microsoft Fabric und SQL. Job Zusammenfassung In dieser Position entwickelst du Pipelines für die Datenaufnahme aus verschiedenen ERP-Systemen sowie Transformationen zur Erstellung von Finanzberichten mithilfe von Microsoft Fabric und SQL. Deine Rolle im Team Blue Orange Digital is seeking a Senior Data Engineer for a full-time, 22-week engagement (with options to extend or convert) on a large-scale enterprise data warehouse implementation in the global commodities trading and supply chain industry. You will design and build ingestion pipelines, medallion-layer transformations, and Gold-layer data products on Microsoft Fabric-covering Trader P&L, Global Position, Risk Metrics (VaR), and Financial Consolidation-as part of a project-based delivery team. Working closely with the Solutions Architect/Tech Lead and a co-invested client engineering team, you will contribute across all four engagement phases from Day 1 through stabilization and handover. Design and implement Bronze-layer ingestion pipelines from regional ERP systems, CTRM instances (including Irely), and SAP S/4HANA source feeds into OneLake using Microsoft Fabric Data Factory. Build Silver-layer transformation logic in Fabric notebooks using PySpark and SQL against Delta Lake tables, covering data cleansing, standardization, deduplication, and business rule application. Develop Gold-layer data product tables serving Trader P&L, Global Position, VaR risk metrics, and Financial Consolidation reporting domains. Implement and maintain the OneLake medallion architecture following BOD design standards and engagement coding conventions. Perform schema design, data modeling, and query performance optimization across Fabric Lakehouse and Warehouse layers. Collaborate with the Analytics/Power BI Specialist to publish clean, analytics-ready Gold-layer outputs aligned to semantic model requirements. Participate in code reviews, enforce development standards, and contribute to CI/CD pipeline configuration for the Fabric workspace. Work with client-side Fabric/Azure data engineers as a technical peer-aligning on ingestion patterns, shared schemas, and pipeline governance. Unsere Erwartungen an dich Ausbildung Bachelor's degree in Computer Science, Engineering, or a related technical field (or equivalent experience). Qualifikationen Hands-on proficiency with Microsoft Fabric (OneLake, Lakehouse, Notebooks, Data Factory, Warehouse)-the full core stack for this engagement. Familiarity with ERP or CTRM source system integration (SAP S/4HANA or similar strongly preferred). Exposure to commodity trading, financial risk, or P&L reporting data environments. Microsoft Fabric or Azure Data Engineer certification (DP-700 or DP-203). Understanding of data governance, lineage tracking, and schema evolution best practices. Erfahrung 7+ years of data engineering experience with a strong portfolio of end-to-end pipeline implementations. Strong SQL and PySpark skills with production-level experience building and maintaining Delta Lake pipelines. Experience building multi-hop pipelines across Bronze/Silver/Gold medallion architectures. Experience setting up CI/CD for data pipelines using Azure DevOps or GitHub Actions. Unser Angebot Salary: $ 6200 - 8500 [ USD ] monthly. Background checks may be required for certain positions/projects. Themen mit denen du dich im Job beschäftigst AI Data Economy Metadaten Level: Senior Job Feld: Data Anstellung: Vollzeit Vertragsart: Befristetes Dienstverhältnis Arbeitsmodell: Full Remote Unternehmenstyp: Digitale Agentur Branche: Beratung Ort: Berlin | Deutschland

Tech Stack