Gloria S.

AI & ML is where all the fuzz is if you work in a data team. But most data teams have only just scratched the surface. dbt surveyed thousands of data professionals in their '2024 State of Analytics Engineering Report'. They found that only a third of respondents currently manage data for AI model training today. But 55% expect to soon. In this post, I've looked into the AI & ML landscape if you work in a data team, including the importance of building on top of solid foundations → State of AI in data teams → AI and ML use cases  → Data quality in AI and ML systems → The five steps to reliable data https://lnkd.in/e-6JTHQm

21

3 σχόλια

Because Amazon #RDS I/O does not directly translate to #Aurora I/O, understanding your db performance requirements and predicting your Aurora spend after migrating from RDS can be initially difficult. Aurora’s “I/O-optimized” configuration might be a good idea in these situations, as it allows you to pay a slightly higher fixed price for your instance and storage but to use I/O freely without extra charges. While I/O-optimized instances are more expensive than with the Standard configuration, they help you lock in a price for your database storage. By starting with Aurora I/O-optimized instances post-migration, you’ll ensure your new Aurora database has the necessary performance and scalability while you evaluate and forecast your actual performance needs. Then once you have a better understanding of your workload requirements, you can consider switching to the appropriate instance type and storage configuration that meets your needs. Sam Clark and Kate Gawron discuss this further in the clip below, which comes from our Cloud Masters podcast deep-diving into Amazon Aurora. Find the full episode linked in the comments below 👇 #AWS #databases

27

1 σχόλιο

Seeking to start orchestrating Ray jobs with Apache Airflow? This integration tutorial demonstrates using the Ray provider package to orchestrate a simple Ray job with Airflow in an existing Ray cluster. If you're looking for more complex implementation examples of the Ray provider package, Astronomer Docs is the place to find additional Ray on Anyscale reference architectures. https://bit.ly/4dtCxzV

20

Very interesting move by Databricks! Would be interesting to see how it impacts the future of Delta Lake and Apache Iceberg!

8

📬 Deploying Spark Streaming with Delta on Kubernetes using Terraform Bringing you a how-to article this time covering the modern technologies widely used across the industry. Terraform template included. This is a guest post at Data Engineering Central, thanks to Daniel Beach! Read Full Article:👉 https://lnkd.in/e2EbcaH5

20