Jordan Slott

It seems that with C/C++ -> Rust, specifically, you could do this by comparing the output compiled code as one of your verifications of correctness for ML learning This is something that e.g. copilot doesn't do - but it'd be straightforward to do so when doing training The expectation would be that the compiled rust code should be equivalent to the compiled C code, i.e. the assembly should be semantically no different As the ML learns, it may end up doing some funky rust code simply to get that to match - but that's probably a good thing, as this is the part that would require some manual intervention & you'd at least be guaranteed equivalent behavior

1

Elon Musk’s xAI and the Surprise Launch of Grok-2 Elon Musk's company xAI has launched Grok-2 and Grok-2 mini in beta, positioning xAI as a formidable competitor against AI giants like OpenAI and Google. Previously seen as a minor player, Grok-2 now reportedly surpasses GPT-4 and Claude 3.5 Opus in key benchmarks. Grok-2 boasts significant improvements in reasoning, reading, math, and science, particularly excelling in visual tasks. A strategic partnership with Black Forest Labs, known for their cutting-edge image generation technology, further enhances Grok-2’s capabilities. Independent validation is needed, but this move heightens the pressure on OpenAI and Google, making the industry eager to see their next steps. Link: https://x.ai/

5

1 Comment

Today, I found a fascinating way to think about “Tokens” in LLMs. It came from a conversation between Jensen Huang and Patrick Collison at a recent Stripe event. When asked about the future of compute capacity in five years, Jensen smartly dodged that question a bit, but then gave an excellent explanation and analogy about why tokens are the new forces that will power the next decades of humanity. He explained that we're now producing something unprecedented (and at scale): floating point numbers that possess “value”, which we now call tokens. These tokens are valuable because they encapsulate “intelligence”. People are now taking these tokens and transforming them into English, French, images, videos, chemicals, proteins, robotic movements, etc. And many are working hard to expand the range of concepts and ideas we can create with these tokens. He then goes on to made a compelling comparison between tokens and electricity. In the past industrial revolution, we successfully found a way to convert “atoms” into “electrons” (by boiling water to power electricity turbines). And now, we've discovered a way to convert "electrons" into "tokens" (by using energy to power data centers that train and run LLMs). When electricity was first introduced, few people understood its value. Today, paying for kilowatts is routine. The same will happen with tokens. Right now, only early adopters and builders are paying tokens. Soon, everyone will be paying for tokens on a daily basis to supercharge productivity and power new products and services. Many new industries will be born from and built on top of tokens. When I first heard about this way of thinking, I got goosebumps. Perhaps it’s because Jensen has a good storytelling skill that helps him sell what he builds, but I have to admit this way of thinking gave me a brief surge of pride. I'm proud of humanity's collective effort and creativity, which have taken us from living under rocks to building machines that can "think." That's nothing short but a miracle.

18

5 Comments

Cut Down Your Platform Engineering Org By 50%. Your Business Will Be OK! If you are a large enough tech company, you have a bloated platform engineering team! A couple of decades ago, Joel Spolsky coined the term 'Architectural Astronauts'—engineers who create grandiose, abstract designs that often fail to address real-world problems. Unfortunately, this issue has only worsened, especially in platform engineering. One of my biggest career regrets? Not fighting hard enough against these platform architects. Forced top-down architectures could have been replaced by simple open-source frameworks. I remember it taking over a year to build a self-service API because of a top-down, inversion-of-control framework forced upon all engineering orgs. Read about what killed popular products like Digg, Orkut, NetScape etc.  You can trace it back to bad engineering thought leadership. The amount of platform engineering waste is staggering. Most large tech companies are a graveyard of abandoned platforms and frameworks. Most in-production frameworks have one tenant and are on the deprecation path. The engineering 'thought leaders' who build these systems got promoted and will still rationalize their decisions by passionately talking about edge cases that will never happen. Almost every tech stack begins with off-the-shelf standard frameworks and platforms, mostly open source. But no two businesses are the same. And companies often start building their own platform pieces. As these platform teams grow, they become out of touch with the actual needs of the business. Led by powerful, disconnected engineering leaders, these teams wield more power than they should. Their solution to an inadequate platform? Build a new one, rather than improving the existing one. Unfortunately most of your IP & patents are from these platform teams. Most of your top engineering brain power is working on these platform teams. You will be warned how you will lose the top talent when you question these investments. If you have a 100+ people platform engineering org, cut it down by 50%. Your business will be OK.

30

10 Comments

As I continue to experiment, I realized chunking is an unsung hero in RAG architecture. Dynamic chunking can be used to elevate the performance of your RAG architecture considerably!

8

I remember as a kid in junior high, Michael Coffey and I did a science project on "Could Humans Live on Mars?". At that time, the Ozone layer and CFCs were a big topic and that was part of our plan to warm and terraform Mars. Today I saw a story that researchers have come up with an idea that could use resources already on Mars. There are some science projects that have stayed with me over time. What are some science projects that you remember? https://lnkd.in/ezCnnvbp

7

Appreciate seeing that App Store review time has roughly halved since the summer (down to about 4hrs compared to 9hrs). I can't help but wonder why starting a review for Testflight external beta is still at the 8hr mark (though I certainly understand that those should be lower priority compared to production builds). I +1'd the Google Play IN_REVIEW status issue and recommend others do the same! On the green goblin [Android] side of things, I've personally experienced a lot of variance lately, resulting in some unexpectedly [days long] reviews. I also see that Google recommends waiting for production reviews while alpha reviews are in-progress, citing longer review times. My experience has been the opposite, where the synchronous reviews results in a MUCH longer review cycle. The numbers point to improvement, but there's still further progress to be made, both for timing and consistency of policy enforcement.

9

1 Comment

Sharing my thoughts on How Waymo (formerly known as the Google Self-Driving Car Project) is Pioneering Autonomous Driving with AI. Waymo has become a trailblazer in self-driving technology, and at the core of it all is AI. Leveraging advanced machine learning and deep neural networks, Waymo’s AI-powered system continuously improves through real-world data, making autonomous vehicles smarter, safer, and more reliable. Perception: AI allows Waymo’s cars to "see" and understand their surroundings by processing data from sensors like cameras, LIDAR, and radar. This helps detect pedestrians, other vehicles, and objects in real time. Prediction: The AI system predicts the behavior of everyone around the vehicle—whether it’s cars changing lanes or pedestrians crossing the street—ensuring smooth and safe driving decisions. Planning & Control: AI makes split-second decisions about accelerating, braking, and steering based on the analysis of complex traffic patterns, enabling Waymo vehicles to safely navigate roads. With millions of autonomous miles driven, Waymo’s AI is constantly learning and improving, helping create the future of safer and smarter transportation. The road ahead is AI-driven! www.rxsmeta.com #AI #AutonomousVehicles #Waymo #SelfDrivingCars #MachineLearning #SmartMobility #TransportationTech #FutureOfTransport #AIinAction #Innovation

2

1 Comment

I just realized: we should start saying "build memoization" rather than "build caching." We aren't really talking about least-recently-used or time-to-live algorithms, we're talking about "don't bother recompiling this piece if none of its inputs changed." (It has been puzzling me that some people are so deeply fearful of "build cache" as a concept, and honestly I think it's mostly because most build systems have non-hermetic state scattered all over the place. Maybe a slightly more precise term for the type of caching we're concerned with would clarify the semantics in question.)

83

9 Comments

Confession: I am lazy. A while ago I showed a Prescient Designer/Node-RED subflow with multiple "virtual" inputs. Not too hard, but not simple enough for me. So I made a couple of nodes to make this super fast and dead simple. Watch how in my latest video! https://lnkd.in/e7WxhxpH #designer #nodered #lazy

11

2 Comments

At ScreensAI, we're focused on driving as much accuracy as we can with LLMs. But for contract review and playbook execution, it's often the case that vague instructions and subjectivity are as much to blame for undesirable results as the LLM's reasoning ability. Say you want to use an AI system to help determine if the following standard is met across thousands of contracts in your repository: 📜 The contract should limit the vendor's liability to 12 months’ fees. From our recently published Screens Accuracy Evaluation Report: "While any given individual contract reviewer may have a reasonable understanding of what they’d expect to pass or fail this standard, we see many edge cases in practice. If preferences for those edge cases are not clearly articulated, the LLM is left to assume and make a judgment call. Consider these scenarios: - What if there are major exceptions to this liability cap that weaken it so substantially that it might as well not exist? - What if this liability cap doesn’t apply to beta features? Or some specific set of features? What if the service is free? - What if the liability is capped at either 12 months’ fees or some fixed amount, whichever is greater? Or whichever is lesser? What if the fixed amount is very large? Or trivially small? - What if there is a secondary cap or a backup cap that applies under certain conditions at a much different amount than 12 months’ fees? - What if the cap is only for fees paid for the services that gave rise to the liability rather than fees paid in general under the agreement? In any of these cases, the LLM is left to assume. The leading LLMs are capable of reasoning through this type of nuance, but if they are told they must make a decision as to whether or not the standard passes, they might not make the same decision that you would if you don’t expose your preferences." 📊 If you want quantitative aggregate results to assess the risk of a portfolio of contracts, thousands of verbose "it depends" responses won't cut it. Contract professionals have strong opinions, institutional knowledge, and well-defined playbooks. ScreensAI helps them articulate this to the platform up front, systematically validating the types of LLM-powered results they'll get for the given set of expressions of their preferences. Then they can deploy this expression of preferences at scale in perpetuity. 🧙‍♂️ We think this is where a lot of the magic happens in LLM-powered contract review.

21

5 Comments

Web3 is ready to touch down in the aviation industry. A recent study by Aventus and Airport Perishables Handling (APH) at Heathrow Airport found that integrating Web3 technologies into cargo handling processes could save airlines up to 7% in costs. On the surface, this may seem like just another efficiency improvement, but in reality, it represents a significant shift in how the aviation industry manages its supply chain. Here’s why: Traditional cargo handling is burdened by manual processes, fragmented systems, and a lack of real-time visibility. These inefficiencies lead to delayed shipments, miscommunication, and increased operational costs– to as much as $1.6 billion each year. Web3 addresses these pain points by introducing smart contracts and decentralized ledgers that enable seamless, automated workflows. Imagine every cargo shipment being tracked on a decentralized platform, where all parties—airlines, freight forwarders, customs, and even customers—have real-time access to a single, immutable record enabling real-time visibility into ULD location, custodianship, and condition. No more scrambling for lost paperwork or dealing with delayed payments. Smart contracts ensure that payments and updates are triggered automatically when certain conditions are met, drastically reducing errors and delays. At Heathrow Airport, Aventus' blockchain-based system, operating on Polkadot, cut manual paperwork time by 83%, reduced errors by 90%, and sped up ULD (Unit Load Device) stock updates from 3-4 hours to 30 minutes. But the potential impact goes beyond cost savings. The transparency and security provided by Web3 can rebuild trust across the supply chain, strengthening relationships between airlines and their logistics partners. This could lead to more reliable and faster shipments, turning logistical efficiency into a competitive edge. In short, Web3 isn’t just shaving off 7% in costs—it’s laying the groundwork for a more agile, trustworthy, and efficient air cargo industry. #web3 #blockchaininaviation #supplychaininnovation #logisticsrevolution #cargoefficiency

136

Alex Komoroske's writing is so fun to read and so easy to draw lines into our reality: "Sarumans can’t see the value of systems; it will seem too fuzzy and abstract; [...] For similar reasons, a Saruman is deeply skeptical of other people’s perspectives on what’s a constraint, because the only constraints that are real are physical constraints, and everything else is just a fiction in people’s minds that can be overcome with boldness. Sarumans are the kinds of people who build impressive, powerful cathedrals. When you’re building a cathedral, you can’t have naysayers pointing out imagined constraints. You need decisiveness. As they get more powerful, accumulating followers, capital, and respect, it becomes easier and easier to cast the naysayers away. [...] Radagasts intuitively understand the power of the swarm: the system, in its buzzing, blooming, emergent glory. Radagasts know how to let go and dance madly with the system. They don’t try to control the system and its components. They love the system and its components: a love borne out of respect." https://lnkd.in/e9TNgNXb

2

sharpening writing skill is one of killer skill!

2

Interesting paper showing that LoRA doesn't learn your data that well which raises questions on how effective it is at fine tuning your Gen AI model. LoRA Learns Less and Forgets Less - https://lnkd.in/eG4tEuSe

19

A very interesting article on DSPy and CoT. As complex queries becomes more popular the need to build pipelines is becoming more of a necessity. "When we are building a pipeline, it is not just me trying to convince an LLM to give an output in a certain way, but more of the output should be restricted in such a way that it can work as input for other modules in the bigger pipeline." The shift from prompting to programming to LLM as a module adapting its behavior automatically is going to be very interesting.

9

It was hard, but we now have a primitive at Gadget that lets us do absolutely 0 downtime atomic changes to an application's structure or data under the hood, with 0 requests dropped. I call it "request catching" but it was inspired by this Braintree eng post I read long ago For stateless workloads like serverless functions, 0-downtime deploys are pretty easy. You stand up the new one next to the old one, use a load balancer to start sending requests to the new one, and slowly stop sending requests to the old one. But for stateful workloads, it's much harder. You can't just start a new Postgres and send some queries to the new one and some to the old one without lying to somebody about state. If you want to be consistent, there has to be one source of truth. Fancy DBs have quorums for this, but PG doesn't. If you want to be consistent, you're effectively forced to have some tiny-but-real window of downtime in between draining the old one and starting the new one. That said, there's nothing forcing you to return errors to clients while that is happening. If your infrastructure supports it, you can pause requests while this changeover is happening under the hood, and then resume them once it is over. Clients will notice a longer request that normal, but it will be served like normal and they'll be none the wiser! What's required to do this though is a deep deep integration between the load balancing infrastructure and the thing doing the database deploy under the hood. We've built this up using a distributed locking tool, where all requests start by taking out a shared lock. When we're ready to do the maintenance under the hood, the maintenance workflow takes out an exclusive lock, blocking all requests from proceeding. The workflow does the atomic change it needs to make knowing it has exclusive access which means nothing will change under its feet, and then releases the exclusive lock as fast as it can so those requests can proceed. As an optimization, we only check the lock on the request path if an app is marked as near a maintenance window. This is only possible because of Gadget's architecture where we bundle together the load balancer, the request processing layer, and the database access tier such that we can reliably intercept all requests and all database transactions and do this lock checking. Its these nasty, in-the-weeds details that make me so proud of Gadget -- we can do world-class engineering stuff under the hood on behalf of all our apps without them ever noticing. If you're interested in doing this on your stack some OSS tools have the primitives you need! We use nginx for our load balancer, Temporal for the reliable workflow execution and we're currently using postgres advisory locks for hte locking. I think the Braintree post has been lost to time after they got acquired, but @simonw has a great post on the subject here: https://lnkd.in/eZmiktGT

51

Anthropic's latest Claude AI model is taking human-AI interaction to a whole new level! 🖥️🤖 The new Claude 3.5 Sonnet can now use computers just like a human would - by looking at the screen, moving the cursor, clicking buttons, and typing text. This is a major leap forward in AI capabilities. Key features: Operates based on screenshots, emulating human computer use Can perform complex multi-step tasks across different applications Self-corrects and retries when encountering obstacles2 Accessible via API for developers to build with Why it's game-changing: This opens up incredible possibilities for AI assistance in data entry, form filling, software testing, and automating repetitive computer tasks13. Imagine having an AI assistant that can seamlessly work alongside you, handling tedious computer work while you focus on higher-level tasks. While still in beta, this technology demonstrates how AI is rapidly advancing toward more human-like interaction with digital interfaces. As it develops further, it could revolutionize how we work with computers and boost productivity across industries. The future of human-AI collaboration just got a lot more exciting! What do you think about this new capability? #AIInnovation #FutureOfWork Here is a video about it: https://lnkd.in/gh8aNYEu

5