digna’s Post

Share your comments below my short article about unveiling the data revolution on medium 👇🏻 Image generated by Flibbo #data_analysis #data_science #data_revolution

From Raw to Refined: A Journey Through Data Preprocessing — Part 6: Imbalanced Datasets https://bit.ly/3O3geas

I just published an article on Medium where I detail the differences between data analytics and data science. If you have ever found yourself confused about these two fields, this article is tailored just for you. What to Expect: 1. clear definitions of data analytics and data science 2. a relatable story to illustrate the differences 3. key takeaways to help you understand what each role entails. Read the full article here: https://lnkd.in/d3qRdCZf Don't forget to follow my Medium account for more insights on data analytics, Machine Learning, and AI. In addition, your thoughts and feedback would be greatly appreciated. #DataAnalytics

Use RAG with LLMs to democratize data analytics by via WhatIs: RSS Feed URL: https://ift.tt/zMPRITg

Clearing up prejudices in Data Science! 🔬 Have a look a the snippet from last years applydata summit, where Alexander Acker from logsight.ai is lifting the lid on the story behind the amount data scientists need to clean up data. Watch the full talk here: https://lnkd.in/eRW4ePee #applydata #datascience #datacleaning #research

the insight that a data scientist will spend 80% of their time cleaning data is not wrong, still hilarious how that meme happened (see repost) :D I want to elaborate specifically on my experience what I see in the expression "Data Science is 80% Data Cleaning.", I think it is true, but that it is not a bad thing: 1. The raw data for any kind of analytics will NEVER come at you in a way that could just be copy-pasted into any spreadsheet software. For example, most recently I've spent 6 weeks scraping comments and reviews for a market research. So even though the data is there, the plain task of organising it into a useful dataset is where a data scientist will most likely spend their time. 2. The scrape is done, or if you're already in a business setting that has some data literacy, you'll probably have some kind of sql database. Microsoft SQL Server, PostgreSQL, MariaDB, Clickhouse,.. doesn't matter. The data will have to be pre-aggregated into some uniform form. Say, you want to predict if a person will purchase, then you have to set up your customers in a homogeneous dataset, and try to predict "purchase" from there. In a general SQL database a general customer without a purchase and a customer that has purchased could be in totally disjoint parts of the software system. "Legacy software" anyone? :D Not hating on legacy btw, it's just a fact of IT business life. 3. Our intrepid scientist is in the stage the data can be fed into machine learning, e.g. Statistics / General Math like Linear Regression, Gradient Boosted Trees, Topological Data Analysis, Time Series Analysis, or more specifically machine-focused concepts like having an event-driven microservice-monstrosity of a Spark Cluster collect augmenting data to the user data as fast as possible. Hopefully, the scientist has had time for a eureka-moment. Then the anxiety of a data-task finally fades, you know it can be done! Doesn't mean it is done usually .. 4. [ usually I stumble in this, blame exhaustion from 3. :D ] You have an analysis, a recommended course of action, now you need to sell that to people. Dashboards, a chatgpt-supported chatbot, a new piece of analytical software your colleagues will never want to miss, .. i.e.: once a data task is done, it spawns a lot of followup. These feel like actual new tasks, because they are. That's the point of data science, finding out things about business you couldn't have known a week ago. It's extraordinarily insightful fun!! :D Finally: Back to the 80:20-split of the video, if one ignores step 4, which data people, myself included, tend to do too often, then steps 1+2 are what a data scientist will probably agree costs 80% of the time, and step 3 is 20% of the time. Not least it feels like it because step 3 is the rewarding part, numbers saying things. Speaker slanders the source, rightfully so, doesn't mean the 80% are wrong :) Data friends, we're underappreciated how much we clean data, actually 99.95% of the time, right!? :D

"Rather than focusing on data science metrics, predictive analytics efforts must be tied to business outcomes." Joe McKendrick spreads the word about the fundamental problem with today's ML metrics:

Unlocking Data Science: How Gemini Pro and Llama Index Will Transform Your Workflow via #TowardsAI → https://bit.ly/3TBIweY

Data Lakehouse | Big Data Society Short Knowledge EP.1 📊 Blendata presents Big Data Society Short Knowledge, which brings you into the world of Big Data and AI, by sharing concise, insightful knowledge about trending terms, brought to you by experts from Blendata. In this first episode, we introduce the term "Data Lakehouse." 💡 The origin of the Data Lakehouse? 🔍 The concept of the Data Lakehouse? Let's find out together now! https://lnkd.in/g4Bz2n9J #Bigdata #Blendata #DataLakehouse

Barr Moses : From #LLMs transforming the #ModernDataStack to #dataObservability for #vectorDatabases, here are my predictions for the top data engineering trends in 2024 https://lnkd.in/ebJVs9zg