Data Scientist – AI Extraction Pipelines

About the role

We are looking for a Data Scientist to design, build and maintain multi‑stage document‑extraction pipelines that blend deterministic rule‑based logic with large language model (LLM) powered extraction. The role sits at the intersection of AI, data engineering and enterprise integration, ensuring high‑quality, traceable data flows for downstream business processes.

Key responsibilities

• Design and orchestrate durable, resumable extraction workflows that combine rule‑based logic with LLM prompts.
• Integrate Azure Document Intelligence, OpenAI and other cloud AI services for classification and extraction.
• Own prompt engineering and structured‑output design using Pydantic, guaranteeing strongly typed and validated results.
• Normalize and map extracted information to downstream schemas, handling document variations while preserving provenance.
• Develop automated validation and business‑rule checks to detect non‑compliant claims.
• Integrate results with enterprise source‑of‑truth systems such as SAP (OData), Snowflake and Cosmos DB, maintaining data lineage and auditability.

Required profile

• Strong production‑grade Python development experience with typing and testing.
• Hands‑on expertise in LLM application engineering, prompt design and structured output evaluation.
• Experience measuring model performance (precision/recall, confidence calibration, drift detection).
• Familiarity with cloud AI services, preferably Azure.
• Proficiency in data‑wrangling (pandas) and SQL.
• Experience with Git‑based collaboration, code review and CI/CD pipelines (GitLab a plus).
• Awareness of secure handling of sensitive enterprise data.

Required skills

• Python
• Azure (Document Intelligence, OpenAI, Functions)
• Pydantic
• MLflow
• Snowflake
• Cosmos DB
• Streamlit
• Power BI
• OpenTelemetry
• Grafana
• GitLab CI/CD
• pandas
• SQL
• Git
• Agentic workflow development
• Recommender‑system algorithms
• Retrieval‑augmented generation (RAG)