Data Engineering

The latest data engineering stories, curated continuously from hundreds of sources.

Orchestration and Execution: How JONI Approaches the Agent Layer

Data EngineeringKDnuggetsKDnuggets8h

How energy teams turn theft detection into governed action with Genie and AI business processes

Data EngineeringDatabricks Blog8h

Third-Party Risk Assessments | How Confluent Helps You Move Faster with Confidence

Data EngineeringConfluent BlogBethany Carter8h

Dimitri Fontaine: Plan Advice in PostgreSQL 19

Data EngineeringPostgreSQL Planet8h

Account Usage Views - Snowflake CoCo

Data EngineeringReddit r/snowflake/u/My-Name-Goes-Here9h

Christophe Pettus: 19th Nervous Breakdown

Data EngineeringPostgreSQL Planet9h

Should an Inventory Analyst have SQL/Data Warehouse access?

Data EngineeringReddit r/PowerBI/u/Technical-Bus-142310h

Jimmy Angelakos: Send us a contribution straight from the site

Data EngineeringPostgreSQL Planet10h

Introducing ClickHouse's new TimeSeries Engine: Your drop-In Prometheus replacement

Data EngineeringClickHouse Blog11h

Replica-aware routing public beta

Data EngineeringClickHouse Blog12h

Agility Robotics says its new Digit 5 robot can work next to people without safety fences

Data EngineeringThe DecoderMaximilian Schreiner12h

Jimmy Angelakos: PostgreSQL Commitfest Metrics: PGConf.dev 2026 Part One

Data EngineeringPostgreSQL Planet12h

What is Query Pushdown?

Data EngineeringStarburst Blog17h

Aiven, Confluent, Redpanda, StreamNative and Ververica Form Streamhouse Working Group

Data EngineeringConfluent BlogStreamhouse Working Group18h

Moving Knowledge not just data

Data EngineeringReddit r/snowflake/u/Fearless_Way_183020h

Christophe Pettus: All Your GUCs in a Row: max_index_keys

Data EngineeringPostgreSQL Planet1d

How Databricks’ marketers use data 3x more with Genie, an AI analytics assistant

Data EngineeringDatabricks Blog1d

Should I install the Amplitude SDK?

Data EngineeringAmplitude BlogSean KilcullenSep 15

Why Streamhouse: mission-critical data and AI need infrastructure built for live

Data EngineeringRedpanda BlogSep 15

Managed Postgres: What Lakebase Actually Takes Off Your Plate

Data EngineeringDatabricks Blog1d

Support Help

Data EngineeringReddit r/snowflake/u/Aggressive_Age_10191d

GitHub - dataIcebergChallenge: Who can write the most performant data pipelines? Just a non-serious, fun exercise.

Data EngineeringReddit r/dataengineering/u/averageflatlanders1d

Announcing On-Demand State Repartitioning for Apache Spark™ Structured Streaming on Databricks

Data EngineeringDatabricks Blog1d

how do you handle joining a source that shares no key with anything you already have?

Data EngineeringReddit r/bigquery/u/donewitheverything261d

Need help figuring out a cv project

Data EngineeringReddit r/computervision/u/Ok-Golf44811d

Snowflake Gen AI Specialty cerification

Data EngineeringReddit r/snowflake/u/optimistic_Gunner1d

Diagnose Fabric Data Warehouse workloads with the SQL DW operations skill (Generally Available)

Data EngineeringMicrosoft Fabric BlogMariyaali1d

help in building AWS pipeline (100k records to be processed by an api)

Data EngineeringReddit r/ETL/u/Ok-Seaworthiness18051d

Cornelia Biacsics: Contributions for week 36

Data EngineeringPostgreSQL Planet1d

What is SQL on Anything?

Data EngineeringStarburst Blog1d

KCP: How to Migrate to Confluent Cloud in Days, Not Weeks

Data EngineeringConfluent BlogAhmed Saef Zamzam1d

A letter from Samar: a year of achievements, and what’s next for us

Data EngineeringTemporal BlogSamar AbbasSep 14

Temporal raises $550M at a $12.55B valuation as demand grows for reliable AI infrastructure

Data EngineeringTemporal BlogAllanah HughesSep 14

I got tired of opening htop every time something felt slow so I made this linux debug overlay

Data EngineeringReddit r/dataengineering/u/RK9_20062d

Do you have any success stories on optimizing Snowflake cost. Any hidden tricks or anything worth highlighting?

Data EngineeringReddit r/snowflake/u/thequerylab2d

Christophe Pettus: All Your GUCs in a Row: max_function_args

Data EngineeringPostgreSQL Planet3d

FPGA vs GPU acceleration for distributed shuffle operations: memory bandwidth and pipeline trade-offs

Data EngineeringReddit r/bigdata/u/Ablerol_Magician86263d

Advice on ETL pipeline for large and varied files

Data EngineeringReddit r/ETL/u/boddhya3d

Writing nice unit tests is impossible

Data EngineeringReddit r/apachespark/u/Similar-Bug-3503d

Sales presentation

Data EngineeringReddit r/snowflake/u/Wise-Brief-2083d