Yutong P.

Facebook just released a very different kind of LLMs. These are focused on code optimization and compiler optimization. This is super important because it can fundamentally change the way code is executed on any machine. Here's how it works right now: Programs written in many languages like C/C++ have intermediate steps before they are run on the computer. For example they are converted into Byte Code and then converted into machine code via LLVM (Low level virtual machine). But Facebook's new model can take in the byte code as input and convert it to into a more efficient representation so that when this new code is converted into machine code, it will be faster. "In theory", it could mean that all existing code might be made faster without any change by developers. Our games can have higher frame rates on the same GPUs. Photoshop might run faster on the same laptop. Our software might complete tasks faster without any changes at all. But that will probably take quite a long time. To be honest I don't expect these models to actually be used in production since unless they were 100% as good as compilers they will probably break the code. And are they 100% good? No, not even close. But the speed at which improvements are happening, they will probably reach that level eventually. And that's when we will see models like these everywhere. 

11

Unit Tests Part 2: Cartesian Parameterization! TIL about multiplying parameterizations in pytest. Now, we all know and love pytests parameterization, where you can pass in parameters to a test and thereby re-use the same test code for multiple situations. For example you can pass in these parameters: @pytest.mark.parametrize("input, output", [('hello', 'world'), ('goodbye', 'mom')]) Then in your pytest you will get these two parameter scenarios to use as variables and execute your tests against. - input: hello, output: world - input: goodbye, output: mom It's so sweet I can taste it just writing it! Now - I had no idea but I found out (thank you ChatGPT) today that you can multiple those parameters to get a cartesian product of the sets which will give you every possible permutations of your test parameters. So if you do this: @pytest.mark.parametrize("fruit", ['apple', 'orange']) @pytest.mark.parametrize("ripeness", ['underripe', 'ripe']) You will get these test cases: - fruit: apple, ripeness: underripe - fruit: apple, ripeness: ripe - fruit: orange, ripeness: underripe - fruit: orange, ripeness: ripe This might seem trivial at first - but this is so powerful for test code re-use and flexibility. Below you can see how in the actual test code I am using for one of my projects: At first we had one file to render with 4 different parameter sets. Easy! Then we added another file but we want to test the same 4 parameters against each file rendering. Super easy with pytest, check it out and I hope you try it yourself. That's all for now, go forth and code with delight!

1

A recent breakthrough titled "Matrix Multiplication-Free LLMs" demonstrates a huge advancement in the area of Large Language Models (LLMs) by reducing computational costs. The authors have eliminated MatMul operations from LLMs, claiming to 10 times reduction in memory usage and a 25.6% increase in training speed, all while maintaining strong performance at billion-parameter scales. Paper link: https://lnkd.in/ggph8qXc #AI #machinelearning #deeplearning #LLMs

16

From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future Haolin Jin, Linghan Huang, Haipeng Cai, Jun Yan, Bo Li, Huaming Chen University of Sydney 2024 https://lnkd.in/gVZ3r8G7 "As LLMs have demonstrated significant capabilities in tasks such as code generation and vulnerability detection, researchers are increasingly interested in their potential applications. However, the study highlights the limitations of LLMs, including their lack of autonomy and self-improvement, which LLM-based agents aim to address. These agents represent a promising advancement towards achieving Artificial General Intelligence (AGI) by integrating LLMs for decision-making and action-taking. Despite the growing body of research on LLMs in software engineering, there remains a lack of clarity distinguishing between LLMs and LLM-based agents, as well as a need for standardized benchmarks to evaluate these technologies. The survey systematically investigates current practices and solutions related to both LLMs and LLM-based agents, focusing on six key areas: requirement engineering, code generation, autonomous decision-making, software design, test generation, and software maintenance. By reviewing and contrasting the contributions of LLMs and LLM-based agents across these topics, the survey provides insights into their respective tasks, benchmarks, and evaluation metrics. It also discusses the models and benchmarks utilized in this domain, offering a comprehensive analysis of their effectiveness and applications in software engineering. Ultimately, the authors aim to illuminate the path forward for LLM-based agents in software engineering, paving the way for future research and development in this promising area."

2

Prof. Rao's post is a good starting point for LLM practitioners to become familiar with classic (i.e., predating LLMs) AI concepts and theories, for two reasons: 📌 To understand that some things are theoretically beyond the reach of LLMs and not waste resources chasing them. 📌 To hedge against the possibility that LLMs may be replaced by other technologies and/or undergo fundamental technological revamps, which would render all existing investments in LLMs obsolete. #artificialintelligence #machinelearning #deeplearning

41

3 Comments