DEV Community

# dataengineering

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
How to treat secure data on lakehouse

How to treat secure data on lakehouse

1
Comments
3 min read
Building an Automated Weather Data Pipeline with Apache Kafka and Cassandra

Building an Automated Weather Data Pipeline with Apache Kafka and Cassandra

1
Comments
10 min read
Understanding Data Pipelines: The Backbone of Modern Data Systems

Understanding Data Pipelines: The Backbone of Modern Data Systems

1
Comments
3 min read
Time of YAML/JSON for data engineer

Time of YAML/JSON for data engineer

1
Comments
2 min read
Distributed Model Serving Patterns

Distributed Model Serving Patterns

Comments
4 min read
Building Automated Data Reports from Supabase with GitHub Actions and R Markdown

Building Automated Data Reports from Supabase with GitHub Actions and R Markdown

Comments
12 min read
Matplotlib For Data Visualization

Matplotlib For Data Visualization

Comments
1 min read
Stock Data Extraction Using Apache Kafka

Stock Data Extraction Using Apache Kafka

5
Comments
4 min read
Data Engineering: The Hero Behind Smart Data Decisions

Data Engineering: The Hero Behind Smart Data Decisions

Comments
1 min read
The Ethics of Data Science: Balancing Innovation and Privacy

The Ethics of Data Science: Balancing Innovation and Privacy

1
Comments
4 min read
Building a Self-Optimizing Data Pipeline

Building a Self-Optimizing Data Pipeline

Comments
3 min read
Automating Cryptocurrency Data with Python, Apache Airflow and PostgreSQL

Automating Cryptocurrency Data with Python, Apache Airflow and PostgreSQL

Comments
3 min read
Architecting High-Performance Data Pipelines with Modern ETL | Spiral Mantra

Architecting High-Performance Data Pipelines with Modern ETL | Spiral Mantra

Comments
1 min read
Creating a new Airbyte connector from scratch

Creating a new Airbyte connector from scratch

Comments
4 min read
How to Optimize SQL Queries for Speed and Efficiency

How to Optimize SQL Queries for Speed and Efficiency

2
Comments
5 min read
Software and Data Engineering: How They Power AI and Big Data

Software and Data Engineering: How They Power AI and Big Data

Comments
4 min read
🚀 Building an ETL Pipeline with Python to Scrape Internship Jobs and Load into Excel

🚀 Building an ETL Pipeline with Python to Scrape Internship Jobs and Load into Excel

1
Comments 2
4 min read
Personal Picks: Data Product News (March 19, 2025)

Personal Picks: Data Product News (March 19, 2025)

Comments
5 min read
Unlocking the Power of Delta Lake UNIFORM in Databricks

Unlocking the Power of Delta Lake UNIFORM in Databricks

1
Comments
5 min read
Databricks Optimistic Concurrency Control: Enabling Efficient Collaboration in Big Data Environments

Databricks Optimistic Concurrency Control: Enabling Efficient Collaboration in Big Data Environments

1
Comments
5 min read
Why Pi-Shaped Teams Matter in This AI Era

Why Pi-Shaped Teams Matter in This AI Era

Comments
5 min read
How to Build a Career in Data Science: A Step-by-Step Guide

How to Build a Career in Data Science: A Step-by-Step Guide

Comments
3 min read
Data architecture stack

Data architecture stack

Comments
1 min read
[Snowflake's New Feature] Alert on New Data, and try with Monitoring Tasks and Dynamic Tables

[Snowflake's New Feature] Alert on New Data, and try with Monitoring Tasks and Dynamic Tables

Comments
7 min read
Study Notes 6.15: Kafka & Flink Streaming

Study Notes 6.15: Kafka & Flink Streaming

Comments
5 min read
loading...