Data Ingestion Engineer
ÁÒÂ Ìåäèà Ãðóï ÅÀÄ Top employer
more than 300 employees
Data Ingestion Engineer
Sofia
full-time

Data Ingestion Engineer

Sofia full-time

About us

The bTV Media Group is a leading source of information on the Bulgarian market with a high level of trust among viewers. As part of the Central European Media Enterprises (CME) family, owned by the international company PPF Group, we reach millions through television, radio, and digital channels. We stand for innovation, creativity, and excellence while also committing to meaningful social causes and civic initiatives.

We believe in digital transformation as the key to our future success. Embracing new technologies allows us to anticipate user needs and deliver content in innovative ways.

At bTV, you’re part of what matters.

The Role

We are looking for a Data Ingestion Engineer to build and operate the ingestion layer of a modern data platform. This role focuses on integrating source systems, delivering reliable raw-layer data, and running the streaming and batch pipelines that move data from source to platform. You will work across source onboarding, Kafka-based event ingestion, file and database ingestion patterns, infrastructure-level data contracts, and ingestion quality controls to ensure data arrives accurately, consistently, and on time.

Responsibilities

  • Design, build, and maintain source-system integrations for structured, semi-structured, and event-based data.
  • Develop and operate Kafka-based streaming pipelines for near-real-time data ingestion.
  • Build and maintain batch ingestion workflows using tools and frameworks such as Logstash, object storage, Parquet, Python, and orchestration tools.
  • Implement and support MySQL replica synchronization and other source-aligned extraction patterns where appropriate.
  • Deliver reliable source-to-raw data flows and ensure ingestion pipeline execution is stable, observable, and scalable
  • Define and maintain source-to-raw and infrastructure-level data contracts, including schema expectations, freshness requirements, and service levels.
  • Implement ingestion-layer data quality validation such as schema checks, completeness checks, freshness controls, and anomaly detection.
  • Monitor ingestion SLAs, throughput, latency, failure patterns, and pipeline health using dashboards and alerts
  • Support cloud-based ingestion tooling and storage patterns, including object storage and scheduler-orchestrated ingestion jobs.
  • Document source integrations, ingestion logic, runbooks, and incident handling procedures for operational reliability.
  • Collaborate with platform, infrastructure, analytics, and downstream engineering teams to ensure reliable handoff of raw data to later transformation layers.

Requirements and necessary skills

  • Hands-on experience building and operating data ingestion pipelines in both streaming and batch environments.
  • Strong knowledge of Kafka, including producer and consumer patterns, partitioning, offsets, and operational monitoring.
  • Experience with Logstash, object storage ingestion patterns, and file formats such as Parquet, JSON, and CSV.
  • Strong Python and SQL skills for data extraction, pipeline logic, automation, and troubleshooting.
  • Experience with orchestration tools such as Airflow and with scheduling, retries, dependencies, and backfill processes.
  • Practical understanding of relational data sources such as MySQL and analytical or real-time stores such as ClickHouse.
  • Experience working with cloud-based ingestion tooling, storage services, and data contract-oriented delivery patterns.
  • Solid understanding of ingestion design patterns such as CDC, incremental loading, schema evolution, idempotent processing, and raw-layer architecture.
  • Experience with monitoring, observability, alerting, SLA tracking, and operational support for production data pipelines.
  • Familiarity with Git-based development workflows and CI/CD practices.
  • Experience with GCP-based data environments and services supporting ingestion, storage, and orchestration
  • Exposure to Cloud Composer, cloud storage, and modern ingestion frameworks in managed cloud environments.

We offer

  • Discounts and special deals in various stores
  • Professional and cooperative team
  • Trainings
  • Great location (and a metro station near the office)
  • Food vouchers
  • Medical insurance
  • Additional retirement benefits