Open Systems Technologies’ Post

Growing as an engineer: Augmenting ids In many systems, an object is given a unique id. Uniqueness in the given table is often the only property these ids have when they could (and probably should) have more properties. You can add additional metadata in these ids, e.g. their type, their geo location, some timestamp etc. Just reserve the first few bits for this. How much metadata you want to add depends on your requirements and your system. - Adding the type in your ids means that you can know what a chunk of data is as long as there’s an id inside – you just need to decode the type in the metadata. This is especially useful in case you have to deal with a data leak – you can know at a glance what table was leaked. It also allows you to verify that some ids don’t end up in regions they shouldn’t – e.g. if you don’t want to allow user ids to flow to a central region for logs, you can easily run a query for this. - Adding the geo location where the data should reside allows you to monitor data residency requirement easily without having to do additional joins in your enforcement offline system - Etc. What metadata you should add is a balance between your product/privacy needs and the cost of storing these extra few bytes. Note that an interesting property of encoding the type in the metadata is that you can store all your entities into a single namespace – this means every entity can be linked to another one the same way regardless of what these objects are. I believe this is what Facebook does and is one of the fundamental pieces that allows them to implement deletion at scale – they know how every object is tied to another and how deletion should propagate, from there they “just” cascade the deletion. #softwareengineering #id #uniqueness #metadata Illustration's prompt: cartoon of a software engineer holding a laptop under their arm and adding "metadata" into a magic cauldron labeled "id generator"

✨Do you know that Google caps search queries at 32 𝘸𝘰𝘳𝘥𝘴 or 256 𝘤𝘩𝘢𝘳𝘢𝘤𝘵𝘦𝘳𝘴? 💡It does it so that it can serve user requests in a timely and effective manner.(𝘛𝘩𝘰𝘶𝘨𝘩 𝘪𝘵 𝘸𝘪𝘭𝘭 𝘯𝘰𝘵 𝘴𝘵𝘰𝘱 𝘺𝘰𝘶 𝘧𝘳𝘰𝘮 𝘪𝘯𝘱𝘶𝘵𝘵𝘪𝘯𝘨 𝘵𝘦𝘹𝘵). 💡Whereas LinkedIn allows posts for up to 3,000 𝘤𝘩𝘢𝘳𝘢𝘤𝘵𝘦𝘳𝘴. We may occasionally overlook the significance of text limitations. Adding text limitation gives below results 1.👥𝗘𝗻𝗵𝗮𝗻𝗰𝗲𝗱 𝗨𝘀𝗲𝗿 𝗘𝘅𝗽𝗲𝗿𝗶𝗲𝗻𝗰𝗲: Clear expectations on input length prevent user frustration. 2. 💾𝗗𝗮𝘁𝗮 𝗜𝗻𝘁𝗲𝗴𝗿𝗶𝘁𝘆: Maintain order and structure by curbing overly lengthy entries (e.g., document, database, or application.) 3. 💽𝗗𝗮𝘁𝗮𝗯𝗮𝘀𝗲 𝗘𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝗰𝘆: Trim down storage needs, ramp up query speed 4. 📑𝗙𝗼𝗿𝗺 𝗩𝗮𝗹𝗶𝗱𝗮𝘁𝗶𝗼𝗻: Fewer errors, accurate data. It evaluates client input to reduce errors and improve the accuracy of the data. 5. 🛡️𝗦𝗲𝗰𝘂𝗿𝗶𝘁𝘆: Guard against potential attacks with character constraints, (e.g., SQL injection or buffer overflow) 6. ⚔️𝗣𝗿𝗲𝘃𝗲𝗻𝘁𝗶𝗻𝗴 𝗔𝗯𝘂𝘀𝗲: Combat spam and misuse by setting thoughtful limits 7. 📳𝗠𝗼𝗯𝗶𝗹𝗲 𝗥𝗲𝘀𝗽𝗼𝗻𝘀𝗶𝘃𝗲𝗻𝗲𝘀𝘀:Responsive design on smaller screens - because every character counts. 8. 👌𝗥𝗲𝘀𝗼𝘂𝗿𝗰𝗲 𝗢𝗽𝘁𝗶𝗺𝗶𝘇𝗮𝘁𝗶𝗼𝗻: Efficient allocation where every bit of data impacts performance. Feel free to share your thoughts in below comment... ref: https://lnkd.in/gAUhEwe2 #userexperience #testing #sqa #efficiencymatters #datasecurity

They're After Your Analytics Data 👀💻 As web developers, we live and breathe data – especially the precious insights from Google Analytics / Tag. But what if I told you that your meticulously tracked data is under constant threat from an unlikely source? 💣 Competitors and malicious actors are always on the prowl, seeking ways to pollute and manipulate your analytics to gain an unfair advantage. And one of their primary targets is your exposed Google Analytics tracking code. With just a few lines of code, they can inject fake data into your reports, skewing your metrics and leading you to make poor decisions based on unreliable insights. Want to see a sudden surge in traffic from Timbuktu? They've got you covered. A massive spike in bounce rates? Coming right up! 📈📉 But fear not, there are ways to fortify your defenses and protect your precious data: 1. Server-side tracking 🔒: Render your tracking code server-side to keep it hidden from prying eyes. 2. IP anonymization and filtering 🔍: Obfuscate visitor IPs and filter out suspicious traffic sources. 3. Content Security Policy (CSP) 🚫: Implement a strict CSP to block unauthorized scripts. 4. Regular audits 👀: Routinely review your setup and watch for any anomalies or spam attempts. The war for clean data is never-ending, and competitors are constantly seeking new tactics to undermine your efforts. 🎯 Don't let them gain the upper hand by leaving your analytics exposed. Take action now to secure your tracking code and safeguard your data's integrity. Because in this ruthless digital battleground, knowledge is power – and they want to strip you of that power by any means necessary. 💪 Stay vigilant, stay secure, and protect your data at all costs. The future of your business may depend on it. ⚔️

Understanding the business context or the problem at hand is a crucial step before you start writing an SQL query. Sometimes, the problem might seem complex initially, but once you have a clear understanding of it, writing the SQL query becomes a straightforward task. This is one of those cases. I've successfully solved it. Why not give it a try? Write a query to update the payment status of Facebook advertisers based on the information in the daily_pay table. The output should include the user ID and their current payment status, sorted by the user id. The payment status of advertisers can be classified into the following categories: New: Advertisers who are newly registered and have made their first payment. Existing: Advertisers who have made payments in the past and have recently made a current payment. Churn: Advertisers who have made payments in the past but have not made any recent payment. Resurrect: Advertisers who have not made a recent payment but may have made a previous payment and have made a payment again recently.

📊 Elevate Your Data Talent Hiring with Our Online Global Salary Guide 🌏 🔍 Are you navigating the complex salary landscape in the data world? Whether in Europe or the USA, our comprehensive Salary Benchmark Tool: https://bit.ly/49dfRCs is your gateway to informed hiring in Data Science, Data Engineering, Big Data, and Computer Vision. 💡 Empower your hiring decisions with insights on: 🤖 Data Science & Engineering: Understand the value of these pivotal roles in today's data-driven landscape. 📈 Big Data Expertise: Get up-to-date salary benchmarks for professionals managing vast datasets. 👁️ Computer Vision Innovators: Explore compensation trends for those at the forefront of visual processing technologies. 🌍 Highlights of our online tool: 📋 Tailored Data for Two Continents: Focused insights for both the European and the American markets. 📚 Diverse Roles & Levels: From junior positions to senior experts, gain a comprehensive view. 🔎 Stay Competitive: Leverage our guide to attract and retain top-tier talent in a rapidly evolving field. 🔗 Explore Salary Insights Here: https://bit.ly/49dfRCs 💬 We'd Love to Hear from You! What challenges do you face in recruiting for data-centric roles? Share your thoughts below. #DataEngineering #ComputerVision #SalaryGuide

Working with web data/requests can be a pain, not as straight forward as rectangular data (i.e. tables). When I started working with requests (REST or XML) I came across curl A LOT. This was something new for me and something difficult to get my head around. One of the best resources I came across was the online curl converter. This tool converts curl requests into a number of different languages. As a R users this was, and still is, so useful! #data #api #restful #xml #rstats https://lnkd.in/eJEea5KR

I love Solomons post about data never being perfect. I can imagine many people who come into working with data may be thinking from the point of view of datasets: discrete things that ARE knowable and fixable, and it is the responsible thing to do. Something I realize: once I’m no longer responsible for generating the data, I cannot and really should not own all (or in cases any) of the cleanup. I’ve had a great time working with a colleague to understand and plan reshaping data that _is_ incorrectly designed. Note I don’t talk about the data being “right;” we know they’re wrong: missing values, incomprehensible combinations, blah, blah. What’s cool: we built a dashboard and worked gently with the owners to gently ask questions I understand their process and converted them from saying, “the data are wrong,” to “Oh! We saw these flagged values and have updated them already, thank you; can we request more flags or qcs as we think of them?” Of course! Data is in support of the business, and business workflows often a sort some type of (anomaly. Instead of fixing everything, I aim to fix enough to make sense and understand. Then I can present the anomalies for the owners to react, and we grow from there.

Henk van Ess reports an interesting development. Google Search is testing customizable search filters. So far it's only available to a select few, but if the project is finalized, I'm inclined to agree with Henk and say that it will be a game changer in open data investigations and fact-checking.  You might snicker, we all use search operators, and there's nothing to do in the profession without them, but filters can take search to another level. Existing information suggests that custom filters, if properly configured, will reduce the time and effort we spend searching by an order of magnitude.  The bottom line is that building your own dorks chains and using them in searches will be possible. Priceless for monitoring.  I'm in awe and eager for the opportunity to feel these filters myself https://lnkd.in/eGngQQ4f

You might be perfect, but your data, unfortunately, will never be perfect. You can spend an UNLIMITED amount of money and time investing in data quality. And still, you'll have problems. You'll think you know how many people visited your website, but oh wait, lots of people use ad blockers and were invisible to your tracking! Or your tracking library had a race condition in javascript so some unknown percentage of people never got tracked and your data will be incomplete forever. How do you navigate as a data person? Get comfortable making decisions off incomplete information. Too many people have an all or nothing view around data quality. But the real world doesn't work like that. Even imperfect data can be extremely helpful! If I take off my glasses, everything looks blurry... But I can still see clearly enough to walk around and know if I'm about to walk into the street. Get comfortable helping business folks make good decisions incorporating your understanding of how reliable and complete the information is. It's messier than everyone would like, but real life is messy! The only people who never need to worry about messy data and incomplete information don't actually make important decisions.

Compare your #data against two #values by using #conditionalformatting 📢💙 Follow us for more tips, tricks, hacks, and functions (more than 3 videos daily) 😉 💚 #support us please by #repost so many people can benefit 💚 🌹 All what you need about #data filteration (11 videos) 💥 👉 https://lnkd.in/dDZK_yN3 💥 How to calculate values based on specific criteria 🌹 👉 https://lnkd.in/d-ebe92c 🌹 How to consolidate your data from multiple sheets in one 💥 👉 https://lnkd.in/dEWs5V_K 💥 How to get #information you want from a table (useful for #hr) 🌹 👉 https://lnkd.in/dCZjYTKP 🌹 How to get Max, Min, or Count values based on your criteria 💥 👉 https://lnkd.in/dxH6hkeF 💥 Goal Seek #feature 🌹 👉 https://lnkd.in/duS_qSEP 💣 How to present the highest #value only in a dynamic way 🌹 👉 https://lnkd.in/dPSD-V64 🌹 How to control your #inventory items 💥 👉 https://lnkd.in/dVuhnVUe 💥 Hide your #formula 🌹 👉 https://lnkd.in/dFnqhzWC 🌹 Show your #formula 💥 👉 https://lnkd.in/dnmZfFQR 💥 Insert a chart in 10 Sec 🌹 👉 https://lnkd.in/duaws6ce 🌹 All usage of #ifsum function (10 videos) 💥 👉 https://lnkd.in/dRVZC-ve 💣 #hack the password for a protected Excel sheet 🤟 👉 https://lnkd.in/d3-KDZBi 💥 Make a dynamic drop list based on other 🌹 👉 https://lnkd.in/d-GqdqAX 🌹 Key limitation for the #vlookup function (SOLVED) 💥 👉 https://lnkd.in/dEw_GAd3 💥 How to make a calendar 🌹 👉 https://lnkd.in/dBHS_SEw #excel #exceltips #exceltricks #exceltipsandtricks #msexcel #msexceltraining #sumif #function #accounting #finance #budgeting #forecasting