data-engineering

3 articles

• Dagster vs Airflow for a New Data Platform: Assets, Sensors, and Day-Two Operations

  Dagster and Airflow compared on assets, sensors, partitions, backfills, retries, and observability, so you can choose the right core for a new platform.

  • dagster
  • airflow
  • data-orchestration
  • data-engineering
  Comparison

  Data
  

• Data Quality Gates with Great Expectations: Expectations, Checkpoints, and CI

  Build a working data quality gate with Great Expectations: define an Expectation Suite, run it as a Checkpoint, store the results, and fail CI on bad data.

  • data-quality
  • great-expectations
  • data-engineering
  • ci
  • python
  Tutorial

  Data
  

• Schema Evolution Without Breaking Consumers: Avro, Protobuf, and Compatibility Modes

  A practical guide to evolving Avro and Protobuf schemas safely with compatibility modes, registries, and expand-and-contract deployments.

  • avro
  • protobuf
  • schema-registry
  • data-engineering
  • kafka
  Tutorial

  Data