Barry Spencer

Ethernet always wins. It has already adopted most of the features of Infiniband. Now, Meta reports Ethernet is more reliable.

8

So many projects we build these days rely on API connections to third parties. But all too often, a bad API can take down your entire project. Read more about how to avoid the weak link in this week's blog: https://lnkd.in/esr5-Bif

6

Anthropic has launched Claude 3.5 Sonnet, the inaugural release in their upcoming Claude 3.5 model family. This new AI model sets unprecedented industry benchmarks for intelligence, outperforming competitor models and even Anthropic's own Claude 3 Opus across a wide range of evaluations, including graduate-level reasoning, undergraduate-level knowledge, and coding proficiency. Claude 3.5 Sonnet demonstrates significant improvements in understanding nuance, humor, and complex instructions, while excelling at producing high-quality content with a natural, relatable tone. ➜ Check the news at: https://lnkd.in/dbKMKPm7 ➜ Vendorvise's official Instagram: https://lnkd.in/ebK2KQ2C ➜ Vendorvise's Linktree: https://lnkd.in/dQdbMdth 🤖 If you enjoy reading my posts about AI and tech news, hit that like button and share with your friends to support my content! #ai #artificialintelligence #ainews #aipowered #news #blog #claude3 #sonnet #anthropic #generativeai #llms #tokens #aitools #aitech #tech #technology #hardware #content #socialmedia #future #new #launch #business #techtrends #innovation #success #markets #people #industry

2

These are the boring, yet impactful aspects of API operations I am here for--low-tech, economy impacting, real-world stuff like the API deprecation notice from the Department of Labor.

20

I love looking back to old projects I've done! So nostalgic... One of them is my Kinect floor mapping algorithm from 2017 (I was about 13 back then). I did it in C++ with no external dependencies (apart from the Kinect SDK), and turned the raw depth values returned from the SDK into this beautiful map you can see below! Camera position is the red square. It automatically calibrates to the floor height and then creates a birds-eye-view map like you can see below. Github repository is here: https://lnkd.in/gSiQwcAG (I actually didn't use Git yet & added the repo in 2022 to share with others. This was WAY back in the old days :), before I learned Git)

3

Microservices get a bad rap due to complexity and high TCO. A lot of that is because a microservice-based application works the same way as a bunch of dominoes falling. If one misses, or falls out of order, the whole workflow stops. And you have no control over the process once your microservices start ping-ponging events and RPC's back and forth. That's why #microservices are so expensive to manage. Great blog post below by Colt McNealy about how LittleHorse can solve those problems. Give it a read. https://lnkd.in/gJECBwRu

58

At ScreensAI, we're focused on driving as much accuracy as we can with LLMs. But for contract review and playbook execution, it's often the case that vague instructions and subjectivity are as much to blame for undesirable results as the LLM's reasoning ability. Say you want to use an AI system to help determine if the following standard is met across thousands of contracts in your repository: 📜 The contract should limit the vendor's liability to 12 months’ fees. From our recently published Screens Accuracy Evaluation Report: "While any given individual contract reviewer may have a reasonable understanding of what they’d expect to pass or fail this standard, we see many edge cases in practice. If preferences for those edge cases are not clearly articulated, the LLM is left to assume and make a judgment call. Consider these scenarios: - What if there are major exceptions to this liability cap that weaken it so substantially that it might as well not exist? - What if this liability cap doesn’t apply to beta features? Or some specific set of features? What if the service is free? - What if the liability is capped at either 12 months’ fees or some fixed amount, whichever is greater? Or whichever is lesser? What if the fixed amount is very large? Or trivially small? - What if there is a secondary cap or a backup cap that applies under certain conditions at a much different amount than 12 months’ fees? - What if the cap is only for fees paid for the services that gave rise to the liability rather than fees paid in general under the agreement? In any of these cases, the LLM is left to assume. The leading LLMs are capable of reasoning through this type of nuance, but if they are told they must make a decision as to whether or not the standard passes, they might not make the same decision that you would if you don’t expose your preferences." 📊 If you want quantitative aggregate results to assess the risk of a portfolio of contracts, thousands of verbose "it depends" responses won't cut it. Contract professionals have strong opinions, institutional knowledge, and well-defined playbooks. ScreensAI helps them articulate this to the platform up front, systematically validating the types of LLM-powered results they'll get for the given set of expressions of their preferences. Then they can deploy this expression of preferences at scale in perpetuity. 🧙‍♂️ We think this is where a lot of the magic happens in LLM-powered contract review.

21

5 Comments

I frequently emphasize the need to study Domain-Driven Design deeply. If you have a business challenge, it's essential to invest in obtaining a suitable and customized solution. The domain is the heart of every business and often brings out the details of each context. We always use generic themes in presentations for the same reason. A good source for reading are the books by Vaughn Vernon and Eric Evans. 🏃🏆🏆🏆🏆 #software #ddd #domain #domaindrivendesign

2

1 Comment

“In the same way that “good” software testing examines a system as a whole—from individual components to complex interactions across modules—effective LLM evaluations should help practitioners understand not just the quality of outputs in isolation, but also how different input patterns affect those outputs and how the model’s decision-making mechanisms operate within the context of the use case.” — Shayan Mohanty, John Singleton and Parag Mahajani. In the past few months I've been talking about the problem of trust when using GenAI, and about the open research problem that is LLM 'evals' — evaluating the output of an LLM as good, bad, trustworthy or untrustworthy. This first article offers a mental model for thinking about LLM benchmarks, evals and tests, all of which are separate concepts, and comes courtesy of the Thoughtworks AI Research Lab. https://lnkd.in/e72MDywH

66

I do a lot of AI automation work, much of it involving batch processing and reprocessing of vast amounts of text. The low cost and high quality of Claude 3 Haiku (25 cents per million tokens in and $1.25 per million tokens out) has been fantastic. But today, OpenAI released GPT-4o-mini for use in API calls, driving the cost of AI automations even lower: 15 cents per million tokens in and 60 cents per million tokens out! To give you a sense of the savings: I once accidentally ran my batch processor using the high-end GPT-4o model. With 12 million tokens submitted and 2.5 million tokens returned, it cost me $97.50. With GPT-4o-mini, today that cost is only $3.30. That’s over a 96% savings! 😲 I have reviewed the Llama 3 8b + 70b Meta open-source models for these automations. The cost is lower still (the model is free, but hardware time is not). However, in my tests, the quality just isn't the same. Plus, considering the additional effort to host an API to access the Llama 3 models, my engineering time and money have so far been better spent using the low-cost models from OpenAI and Anthropic. Anyone else finding similar results in savings + quality?

34

1 Comment

This will unlock a lot of possibilities, and comes with some really nice advantages like: - More efficient chatbots - Easier builiding proces - Better customer experience - Cheaper prices for chatbots Now's the best time to start integrating customer support chatbots in your business to deal with customer while you focus on selling 💯

2

Two weeks ago I posted a question "AI vs API vs UI?": https://lnkd.in/e3Z2cXd8 It asked whether the value of UIs will diminish in a world of AI Agents, a counter to Zdenek's question of whether we should still be building APIs at all? No doubt that there are a lot more UIs out there in the world than public APIs today. This reminds me of the discussion in the world of self-driving cars: many folks thought that we'd only achieve self-driving if we updated all the road infrastructure to be "machine-optimized". Many also think that we can only have self-driving when ALL cars are self-driving. Both seem ridiculous to me; the world is messy and the only scalable system is one that can deal with that. Now people are starting to see that we are on the path to self-driving which is mostly vision-based - like its human counterparts - and drive (more) safely alongside human drivers on existing infrastructure. The recent Claude announcement of AI computer use is a great demonstration of the capability of modern AI to deal with that messiness: https://lnkd.in/ePhEbz7w Arguably this strengthens the case for UI > API in the future. I think this might be true for legacy interfaces - why go to the effort of creating a nice API if you've already got a UI that AI can deal with? However, what about net new development? I still think the cost and difficulty of building decent UI vs good API means that focus will shift to API >> UI for agents. We're talking long-term here though, maybe 10 years. One interesting thing though, is how will businesses handle a shift to agents interacting on behalf of users? Such a strong disintermediation is going to create an even stronger deployment of 'aggregation theory' (https://lnkd.in/epMKwjdM) with winning agents having enormous power, more even than Google has today. What do you think?

4

In case anyone is interested in how I created an end-to-end fully automated podcast show -- I've documented the full technology stack and provided a logic flow diagram of the automation process here.

14

Shouldn't tracking inputs/outputs of LLM chains and workflows be simple? Mirascope's computed fields not only make this easy but also ensure you retain full control and access to all variables across all steps. No more of that "RunnablePassthrough" garbage

7

A meta note about this talk: I am, as a lot of folks out there know, not exactly bullish on LLMs in general. I think most of the hype is nonsense—even while I think some of their capabilities are really astonishing!—but I do think they’re here to stay. So: First of all, if we’re going to be working in a world where people are authoring code with LLMs (Copilot etc.) and using them to “do things” as agent-based systems, the world of inputs to the models matters, as do the environments in which they run. They matter a *lot*! Second, I think a lot of engineering organizations pretty consistently underinvest in foundations, because of the simple realities that many kinds of foundations projects take longer and have a less direct/measurable connection to profitability. They just do, no way around it. Third, then, what are the ways that engineering leaders—including managers, but especially Staff+ engineers—can mitigate the risks of LLMs and improve the chances that they are net neutral or even positive, rather than negative, for software quality, UX, and business outcomes? Secretly (and yes, I am saying it out loud now!) it is predicated on my belief that THESE ARE THE THINGS WE SHOULD BE DOING ANYWAY… but this is a moment when we have a really obvious reason how and why these things matter, with which we can make that case to decision-makers. And because these are the things we should be doing anyway, an organization which uses this LLM-hype-moment to improve the foundations on which we run LLM-related systems… will be in a better spot two years from now *even if the LLM ecosystem craters*. Seems like a win.

10

Austin, Head of Technology at Creative NewsHave you ever encountered the challenge of managing empty lists within large data sets and wondered if compiler optimizations could lend a hand? I recently embarked on this exact journey, determined to squeeze every ounce of efficiency out of my code. Initially, I was skeptical about the impact compiler optimizations could have on something as seemingly trivial as handling empty lists. However, I decided to dive into the realm of compiler flags and see what I could discover. By enabling optimizations such as `-O2` or `-O3` in GCC, I found that the compiler was able to streamline my code, eliminating unnecessary checks and enhancing overall performance when dealing with empty lists. One particularly fascinating technique I learned was loop unrolling. This optimization allows the compiler to reduce the overhead of loop control structures, which, even when dealing with empty lists, can significantly cut down processing time in large data sets. If you’re working with large data sets, I encourage you to explore your compiler's optimization settings. Have you considered what hidden efficiencies might be lurking in your code? Tweaking these settings could be the key to unlocking a more performant application. Give it a try and let me know what results you achieve!

1

Thinking of how to turn your LLM idea into an actual product? You might want to read this post I wrote 😊 https://lnkd.in/gXUFcg-g

26

3 Comments

For all those engineers out there building AI agents. You need to check out this #github repo. This is a great team tackling an important set of problems that engineers face, and this solution is significantly better than Langchain's approach. #ai #agents #ML

9

1 Comment