Behdad Karimi

Scary moments for data people 🎃 A social network changes their API results without telling anyone so that "1M" means 1 million in every country except Spain where it means 1 thousand... You run a big script to update & backfill prod data, but you forgot to run screen and the script gets disconnected halfway and the changes aren't idempotent... You mess up data that went to the board... You have a model that seems to give good results, but then you notice it predicted that 80% of Americans had a Fresca within the past 6 weeks... The data person you hired to advise your business leaders doesn't know much about your business... (send them to Top Data People ASAP before your L&D budget resets) Your COO solves a data bug in 15 minutes after you've been working on it for 2 days with no progress... Your payment calculations were off and your company owes partners a couple hundred thousand dollars after years of mistakes... You ask execs "What big things have we accomplished that wouldn't have been possible without the data team?" And nobody has an answer... Didn't even take me 2 minutes to think of these real world stories and more ☠ Who knew SQL and spreadsheets could be so terrifying 🤣

28

8 comentarios

Stop hiring data scientists: Why your company's next big hire should be a DJ. Sounds crazy, right? Let me break it down for you. When I was younger, I spent countless nights DJing in clubs and running my radio shows. It wasn’t just about the music. It was about reading the crowd. Understanding the vibe. And making datadriven decisions on the fly. Much like a DJ, a great data strategist needs to: → Read the room: Understand the company's culture, goals, and needs. → Mix tracks: Combine different data sources and technologies. → Keep the crowd engaged: Ensure stakeholders remain invested and informed. → Adapt to the beat: Pivot strategies when new data or technologies emerge. Here’s how hiring someone with a DJlike mindset can revolutionize your data strategy: Creative Problem Solving: DJs think outside the box. ↳ They experiment with different beats and rhythms. ↳ This creativity can translate to innovative data solutions. Adaptability: DJs adapt to the crowd’s energy. ↳ They pivot quickly when something isn’t working. ↳ Similarly, data strategies need to be dynamic and adaptable. DataDriven Decisions: DJs analyze the crowd’s reactions in realtime. ↳ They make decisions based on data. ↳ Your data team should do the same. Holistic View: DJs understand how different songs fit together. ↳ A data strategist needs a holistic view of the entire data architecture. So next time you’re looking to hire, consider someone with a background in DJing. They might just bring the rhythm your data strategy needs. What unique backgrounds have you seen excel in data roles? Share your thoughts!

5

4 comentarios

Some thoughts after one month at my new workplace as a Data Engineer. I would say that, so far, my experience has been really good. I felt welcome, heard and also understood. I like the way I am working, and learning. There are some things to highlight: - There is no stupid question, or yes, but it is okay. That basically means that there was no possible way that I knew what I was asking about, because there's no place where I could find the information. I am encouraged to ask, fail and proceed. I find this extremely healthy and humbling. Share your knowledge, and you will learn. -There is a true collaboration. In my previous experience everyone would have their own scripts, probably doing the exact same calculations and figures, but only few (probably young) people would share them. Here time is more important than who does it. We work together and push towards an objective. That does not mean we do not put time and effort on something, is just that we optimize it so we do not do the same thing 3 times in 3 slightly different ways. - Having regular meetings with other people within the group. Also, a stand up meeting with the whole division once a month. For me this is really cool as I am not aware of what others divisions do. -How easy it was to start working. I did not have to follow a person to give me the laptop, then spend 4 days installing programs and being ignored by the person in charge. After 2 days I was all set and ready to work. Other places wanted to charge me to use one of their laptops to work. It was a bit overwhelming at the beginning, and still is sometimes. But so far, I am learning a lot, I start to contribute and be a useful member of the group and I want to keep on that track. The learning curve is steep, but nice. 😎 #data #dataengineer #sql #dbt #python

17

The Pitfalls of Data Science Models: A Closer Look Through Football and Food! According to WhoScored ratings, the best player of FC Barcelona in this season is João Cancelo with a score of 7.18 out ot 10. But his performance is perceived quite differntly by the club supporters. He was a key factor of FC Barcelona's elimination in UEFA Champions League, and his two mistakes led to his side's loss against arch-rivals Real Madrid C.F. last night. In fact, his performance in all season has not been as positive as being the best player. The issue at hand is how data science models can sometimes provide misleading outcomes. While I'm not familiar with the specifics of WhoScored's algorithm, I can draw from my own experiences with model-building in different contexts. Suppose you train a model that gets foods ingredients and gives "deliciousness" scores. Such models might learn that sugar significantly enhances taste. However, this relationship is not linear. If you have a recipe of a plate of food with 1 kg of sugar, the model will give a high score to it, but its taste will be awful. So, the correlation stops at some point and then turns negative. Similarly, we have to consider the combination of features. If salt, sugar, and meat have separately a positive correlation with deliciousness, combining them in excessive amounts doesn't necessarily result in a tasty dish. Applying this insight to football, many rating apps calculate a player's performance based on quantifiable actions, such as shots or dribbles. for instance, a player who frequently attempts unnecessary long-range shots might inflate their rating without genuinely contributing positively to the team. The same is also true for pointless dribbles and other factors contributing to higher scores. A deeper understanding of the variables and their interactions is crucial. Simply put, higher isn't always better—sometimes, it's just more.

25

I want to discuss an important question — how to find the balance between doing things quickly and doing them with high quality. When it’s time to act, there are often two extreme approaches: 1️⃣ The first approach — we do everything "in a rush", using temporary solutions, data exports, and quick fixes to show results faster. 2️⃣ The second approach — we say: "No, everything is bad, leave us alone for the next six months while we build a complete system." This involves developing a DWH, website tagging, quality data exports, but visible results will not come for several months. The first approach is typical for businesses. Business wants results "here and now" to make decisions as quickly as possible. The second approach is usually supported by developers and analysts. They want to create a system that works stably and doesn’t require constant "patches." But where is the compromise? Can we always rely on quick fixes? Of course not. Technical debt will only accumulate, consuming the team’s resources and hindering the project’s growth. But focusing entirely on creating the perfect system without interim results isn’t a solution either. Business exists to make money, not just to solve analysts’ interesting challenges. I had an example in one of the in-house analytics teams I led. When we started building the system from scratch, it turned out that everything was so "patched together" that it was impossible to maintain. We had limited resources, but we devised a plan: how to build something new while simultaneously supporting the existing setup. At one of the meetings, I told the top management: "We have a priority task for the next three months. We’ll dedicate 70% of our efforts to it, and the remaining 30% will go towards support and solving urgent tasks." I understood that without this balance, the analytics system would collapse within six months. At first, I faced heavy criticism, especially from the commercial director. For the first two months, I was regularly blamed on calls for not doing things "right." Since I shielded the team from this, they were able to work peacefully, maintaining the 30% to 70% balance. After three months, we delivered the first reworked report. It ran much faster, was more visual, and easier to use. Most importantly, it required significantly less effort to maintain. Along with this, we also presented a roadmap with new reports that the business had requested, and with very appealing deadlines. A couple of months later, the commercial director (who had been the most against the idea and insisted on solving tasks "immediately") approached me and said: "You were right. Thank you for not listening to me. Now I understand why it was necessary." When implementing new systems and maintaining old ones, it’s important not only to find a compromise but also to stand your ground with the business. It will be tough, especially at first, but the result is worth it. If you succeed, the outcome will be impressive

4

This week is another challenge as far as data Analyst training goes: Data exploration with pandas, missingno and python 🐍 together with DBT data warehousing to perform some ETL and then we will play with Power BI for visualising data. 😉

2

https://lnkd.in/e9XbNEXz Interesting article! I also believe that the following actions can help engage users: -Personalized Recommendations. Data products are more valuable when they provide personalized content or reports that help users with specific needs. -Integration with Daily Workflows. When the product integrates seamlessly with daily tasks, it becomes part of the work routine, so users are less likely to see it as an extra step. -Automation and Smart Notifications. Set up the platform to assist users automatically by sending notifications, reports, or actionable insights. -Gamification. Adding gamification elements can increase interest and make using the data product more engaging.

6

1 comentario

We rewatched Game of Thrones (best 70 hours ever spent!), and I became fully immersed in its universe. I enjoy combining my hobbies with data visualization projects, so this time, I created a map to track the movements of Tyrion Lannister across Westeros and Essos. You can see the final animation below, and the code is available here: https://lnkd.in/dSeZjma7 #got #datavisualization #rstats #animation #storytellingwithdata

43

All models are wrong, but some are useful. I recently stumbled upon a discussion about the well-known log-log models, which are often hailed as the industry standard for estimating elasticities—especially in retail, where products are homogenous. But in my humble (and statistically significant 😊) opinion, the best way to estimate elasticities is through causal inference followed by experiments. See: https://lnkd.in/e8eFHqWc I'd love to hear from fellow practitioners. What methods do you use —particularly for heterogeneous products like fashion items (where one size never fits all) or, in my current adventure, short-term vacation rentals?

8

🎓 💙 What do iris flowers, handwritten digits, boston housing prices and the titanic have in common? Only data scieentists know... 🤔 😉 For everyone who does not remember or know instantly: I'm talking about basically THE toy datasets that most of us have started their data science journey with. And I like that quite a lot. For example when I do interviews, I can use one of these and chances are high that my applicant has at least heard of them before. So today I'm celebrating these problems that have helped data scientists get into the field, try out new algorithms and methods, experiment with feature engineering techniques or use them as examples in talks, papers, etc.! What other datasets are your go-to examples? There for sure are more of them...alone with taking iconic kaggle datasets into account :D

14

Want the secrets to data science leadership? Here's a tip: don't be a complete moron. 🤷‍♂️ Start by setting some ground rules for your team, like not being evil with data and algorithms. 🧐 Teach them that bias is bad, unless it's against people you don't like. 😴 Have some awkward conversations about ethics and try not to fall asleep. 📊 Make sure everyone knows that integrity is important, but not as important as hitting your quarterly targets. Or ignore this advice and actually be a good leader. Whatever works for you. 😈 Had any leadership blunders? Share your stories below! ⬇️ #DataScience #Leadership #Ethics

2

How to piss off a Data Scientist in just 7 seconds: - Peek at an AB experiment early, and insist that we can ship the feature now. - Discard their analyses because it doesn't agree with your gut feeling. - Ask for data to support a conclusion that you've already made. - Request an Al solution because "leadership wants one". - Argue that Data Science isn't the sexiest career. - Insist that they're not real scientists. <insert more here> What else did I miss? #datascientist #dataanalysis #dataanalyst

2

When clients ask about starting a data science team, their focus often zooms in on machine learning and deployment, eager to replicate the success of others. 🤩 However, the foundational questions—what data you have, how you process it, and what you aim to achieve—are critical. As the Gen AI wave grows, I'm hopeful we'll see a corresponding increase in data engineering roles. These professionals are essential for building the strong foundations that true innovation requires 🎓 #DataEngineering #MachineLearning #AI #TechTrends

8

1 comentario

If there is one certainty about working with real-world data: it's that you will always have unexpected edge cases to consider. I'm not just talking about bad data quality (we'll save that one for a different post). I'm talking about - crazy workarounds users come up with - unexpected ways that people use products - unexpected ways that processes get broken Your expectations of the data != the actual data. So: before you hard-code that next CASE WHEN statement, take a little time to brainstorm possible edge cases and look for evidence in your data. #data #analytics #sql #dataquality

69

19 comentarios

I've got a friend who is the CFO of a small company and is looking to make her first data hire / build out a data function. She was ready to write a medium-sized check for a traditional data warehouse and modern data stack. However: 🐣 Her data is TINY 📚 The largest table is ~500k rows 💻 She's got access to a replica database to run queries on 🧦 She doesn't have many data sources that need to be combined 📊 She is looking for basic dashboards, nothing fancy My recommendation was that she didn't need the data stack yet. Instead of building a sophisticated architecture and then needing to maintain it, start by building those basic dashboards with the infrastructure you have. Connect a BI tool to the replica, export your one other data source to CSV or google sheets, and then start actually using it. Hire someone that can help you solve business problems vs. someone focused on the architecture. If you need a more sophisticated architecture, you can always migrate later. But better to get value first and then scale it.

282

69 comentarios