Principal Data Lead (Data Engineering with Full-Stack Dev) - Investment Systems

POSITION SUMMARY
We are seeking a Principal Data Lead to set the technical direction of our Reference Data Platform — the system of record that masters, governs, and distributes the firm’s core reference data (parties, instruments, portfolios, and accounts. This is a hands-on technical leadership role: you will own the end-to-end architecture of a layered, event-driven data platform — spanning sourcing, ingestion, entity mastering, a bitemporal golden copy, and an API-first distribution layer — and drive it from vision to production. You will be the senior-most engineering voice on reference data, shaping platform strategy, establishing engineering standards across teams, and partnering with product, data, and infrastructure leaders to deliver a scalable, vendor-agnostic master data capability. The ideal candidate is an expert Python engineer and data architect with deep experience in master data management (MDM) and reference data within the financial industry — including market data and vendor feeds — combined with hands-on experience applying Generative AI to data platforms, distributed data processing, and building reusable, independently scalable platform components.

ROLES & RESPONSIBILITIES
•    Own the architecture of a layered reference-data platform, defining the contracts between sourcing, ingestion, mastering, golden-copy, and distribution layers so each is independently buildable, deployable, and scalable.
•    Set technical direction for entity mastering — standardization, blocking, matching, survivorship, and crosswalk/lineage — designing rules as configuration (data, not code) that scale from exact-match fast paths to full-population fuzzy resolution.
•    Architect the durable golden copy: a bitemporal, point-in-time source of truth with match evidence, per-attribute provenance, and a schema-last serving model that adapts to new sources and domains without migration.
•    Define the event-driven backbone and cadence model (start-of-day, intraday, end-of-day, and on-demand) enabling batch today and real-time streaming (CDC / Kafka) without re-architecture.
•    Design the API-first, self-documenting distribution surface (REST, curated data products, subscriptions) governed by data contracts and consumable by applications, analytics, and AI/agents.
•    Drive the platform’s storage and compute strategy over an open lakehouse (Apache Iceberg, Snowflake) and elastic compute (Spark), ensuring an open, vendor-agnostic, no-lock-in foundation.
•    Establish engineering standards, reference patterns, and a reusable component library (a layered monorepo) that make capabilities build-once, reuse-everywhere across domains.
•    Lead cross-team technical design reviews, resolve the hardest architectural trade-offs, and align engineering execution with the firm’s data strategy.
•    Champion data quality, governance, security, entitlements, audit, and observability as first-class, cross-cutting foundations of the platform.
•    Mentor senior and staff engineers,