Skip to content
Navigation menu
Search
Powered by
Search
Algolia
Log in
Create account
DEV Community
Close
#
dataengineering
Follow
Hide
Posts
Left menu
đź‘‹
Sign in
for the ability to sort posts by
relevant
,
latest
, or
top
.
Right menu
How to treat secure data on lakehouse
kination
kination
kination
Follow
Apr 8
How to treat secure data on lakehouse
#
dataengineering
#
python
#
spark
1
 reaction
Comments
Add Comment
3 min read
Building an Automated Weather Data Pipeline with Apache Kafka and Cassandra
Eric Katumo
Eric Katumo
Eric Katumo
Follow
for
LuxDevHQ
Apr 7
Building an Automated Weather Data Pipeline with Apache Kafka and Cassandra
#
dataengineering
#
data
#
apachekafka
1
 reaction
Comments
Add Comment
10 min read
Understanding Data Pipelines: The Backbone of Modern Data Systems
Rithesh Raj
Rithesh Raj
Rithesh Raj
Follow
Apr 6
Understanding Data Pipelines: The Backbone of Modern Data Systems
#
dataengineering
#
etl
#
python
#
gcp
1
 reaction
Comments
Add Comment
3 min read
Time of YAML/JSON for data engineer
kination
kination
kination
Follow
Apr 6
Time of YAML/JSON for data engineer
#
dataengineering
#
programming
#
config
1
 reaction
Comments
Add Comment
2 min read
Distributed Model Serving Patterns
Emin Mammadov
Emin Mammadov
Emin Mammadov
Follow
Apr 5
Distributed Model Serving Patterns
#
distributedsystems
#
mlops
#
machinelearning
#
dataengineering
Comments
Add Comment
4 min read
Building Automated Data Reports from Supabase with GitHub Actions and R Markdown
Akan
Akan
Akan
Follow
Apr 5
Building Automated Data Reports from Supabase with GitHub Actions and R Markdown
#
automation
#
analytics
#
dataengineering
#
tutorial
Comments
Add Comment
12 min read
Matplotlib For Data Visualization
Bytes Institute
Bytes Institute
Bytes Institute
Follow
Apr 5
Matplotlib For Data Visualization
#
machine
#
datascience
#
dataengineering
#
ai
Comments
Add Comment
1 min read
Stock Data Extraction Using Apache Kafka
Milcah03
Milcah03
Milcah03
Follow
Apr 6
Stock Data Extraction Using Apache Kafka
#
apachekafka
#
python
#
cassandra
#
dataengineering
5
 reactions
Comments
Add Comment
4 min read
Data Engineering: The Hero Behind Smart Data Decisions
Tart Labs
Tart Labs
Tart Labs
Follow
Apr 3
Data Engineering: The Hero Behind Smart Data Decisions
#
dataengineering
#
database
#
webdev
#
datascience
Comments
Add Comment
1 min read
The Ethics of Data Science: Balancing Innovation and Privacy
Aditya Tripathi
Aditya Tripathi
Aditya Tripathi
Follow
Apr 3
The Ethics of Data Science: Balancing Innovation and Privacy
#
datascience
#
dataprivacy
#
dataengineering
#
ai
1
 reaction
Comments
Add Comment
4 min read
Building a Self-Optimizing Data Pipeline
Shreyash Singh
Shreyash Singh
Shreyash Singh
Follow
Apr 1
Building a Self-Optimizing Data Pipeline
#
dataengineering
#
datapipeline
#
etl
#
selfoptimizing
Comments
Add Comment
3 min read
Automating Cryptocurrency Data with Python, Apache Airflow and PostgreSQL
Milcah03
Milcah03
Milcah03
Follow
Apr 1
Automating Cryptocurrency Data with Python, Apache Airflow and PostgreSQL
#
apacheairflow
#
python
#
dataengineering
#
datascience
Comments
Add Comment
3 min read
Architecting High-Performance Data Pipelines with Modern ETL | Spiral Mantra
Karan Singh
Karan Singh
Karan Singh
Follow
Apr 1
Architecting High-Performance Data Pipelines with Modern ETL | Spiral Mantra
#
dataengineering
#
cloud
#
performance
#
bigdata
Comments
Add Comment
1 min read
Creating a new Airbyte connector from scratch
Luiz Gustavo Erthal
Luiz Gustavo Erthal
Luiz Gustavo Erthal
Follow
Mar 31
Creating a new Airbyte connector from scratch
#
airbyte
#
data
#
dataengineering
#
python
Comments
Add Comment
4 min read
How to Optimize SQL Queries for Speed and Efficiency
BiKodes
BiKodes
BiKodes
Follow
Mar 31
How to Optimize SQL Queries for Speed and Efficiency
#
sql
#
sqlserver
#
database
#
dataengineering
2
 reactions
Comments
Add Comment
5 min read
Software and Data Engineering: How They Power AI and Big Data
Vaishali
Vaishali
Vaishali
Follow
Mar 26
Software and Data Engineering: How They Power AI and Big Data
#
aidrivendataengineering
#
dataengineering
#
softwareengineering
#
dataengineeringsolutions
Comments
Add Comment
4 min read
🚀 Building an ETL Pipeline with Python to Scrape Internship Jobs and Load into Excel
Denzel Kanyeki
Denzel Kanyeki
Denzel Kanyeki
Follow
Apr 6
🚀 Building an ETL Pipeline with Python to Scrape Internship Jobs and Load into Excel
#
dataengineering
#
python
#
programming
1
 reaction
Comments
2
 comments
4 min read
Personal Picks: Data Product News (March 19, 2025)
Sagara
Sagara
Sagara
Follow
Mar 23
Personal Picks: Data Product News (March 19, 2025)
#
dataengineering
Comments
Add Comment
5 min read
Unlocking the Power of Delta Lake UNIFORM in Databricks
Anil Kumar Moka
Anil Kumar Moka
Anil Kumar Moka
Follow
Mar 23
Unlocking the Power of Delta Lake UNIFORM in Databricks
#
databricks
#
datamananagement
#
datasecurity
#
dataengineering
1
 reaction
Comments
Add Comment
5 min read
Databricks Optimistic Concurrency Control: Enabling Efficient Collaboration in Big Data Environments
Anil Kumar Moka
Anil Kumar Moka
Anil Kumar Moka
Follow
Mar 20
Databricks Optimistic Concurrency Control: Enabling Efficient Collaboration in Big Data Environments
#
databricks
#
lakehouse
#
dataengineering
#
datascience
1
 reaction
Comments
Add Comment
5 min read
Why Pi-Shaped Teams Matter in This AI Era
Kannan Kalidasan
Kannan Kalidasan
Kannan Kalidasan
Follow
Mar 19
Why Pi-Shaped Teams Matter in This AI Era
#
ai
#
softwareengineering
#
leadership
#
dataengineering
Comments
Add Comment
5 min read
How to Build a Career in Data Science: A Step-by-Step Guide
Smith
Smith
Smith
Follow
Mar 19
How to Build a Career in Data Science: A Step-by-Step Guide
#
science
#
datascience
#
career
#
dataengineering
Comments
Add Comment
3 min read
Data architecture stack
Max Verstappen
Max Verstappen
Max Verstappen
Follow
Mar 19
Data architecture stack
#
dataengineering
#
programming
#
softwareengineering
#
database
Comments
Add Comment
1 min read
[Snowflake's New Feature] Alert on New Data, and try with Monitoring Tasks and Dynamic Tables
Sagara
Sagara
Sagara
Follow
Mar 23
[Snowflake's New Feature] Alert on New Data, and try with Monitoring Tasks and Dynamic Tables
#
snowflake
#
dataengineering
Comments
Add Comment
7 min read
Study Notes 6.15: Kafka & Flink Streaming
Pizofreude
Pizofreude
Pizofreude
Follow
Mar 18
Study Notes 6.15: Kafka & Flink Streaming
#
dataengineering
#
dezoomcamp
#
kafka
#
pyflink
Comments
Add Comment
5 min read
loading...
We're a place where coders share, stay up-to-date and grow their careers.
Log in
Create account