Alex Zisis

If you're in a career where you are concerned that your job will be subsumed by AI, then it's time to learn AI so you can outperform both your peers and AI alone. Learn prompt engineering as it relates to your interests and professions. Learn RLHF to increase your value at your workplace before they even start to think about AI. The basics of both concepts can be understood in an afternoon, and then learning more about them is as easy as chatting with any of the freely available Large Language Models (Meta.AI, Claude.AI, OpenAI's GPT3.5, etc.) You can even do what I did a year ago and set up an AI conversation that is specifically about training you on these things, and it can result in responses like this to help you along the way (Courtesy of Meta's Meta.AI): Understanding RLHF (Reinforced Learning through Human Feedback) RLHF is a powerful technique for fine-tuning large language models like myself. It involves training the model to generate responses that are not only coherent and informative but also aligned with human preferences. Key Challenges Defining the reward function (how you tell the AI it did a good job): Crafting reward functions that accurately capture human preferences is crucial. You'll need to balance multiple objectives, such as fluency, accuracy, and engagement. Handling feedback variability: Human feedback can be noisy and inconsistent. You'll need to develop strategies to handle this variability and ensure the model generalizes well across different feedback styles. Avoiding over-optimization: RLHF can lead to over-optimization, where the model becomes too specialized in generating responses that maximize the reward function (the AI wants the pat on the head more than being right). Be mindful of this and incorporate techniques to promote diversity and exploration. Best Practices Use high-quality training data: Ensure your training data is diverse, relevant, and representative of the tasks and domains you want the model to excel in. Employ techniques for exploration: Regularly introduce new prompts, topics, or feedback styles to encourage the model to explore different response types and avoid over-optimization. Monitor and adjust the reward function: Continuously evaluate the reward function's effectiveness and make adjustments as needed to ensure the model is aligned with human preferences. Leverage human evaluation: Regularly conduct human evaluations to assess the model's performance and provide feedback to the model. Stay up-to-date with advancements: RLHF is an active area of research, so stay informed about new techniques, architectures, and best practices to incorporate into your work. Additional Tips Be mindful of potential biases in the training data and feedback, and take steps to mitigate them. Consider using techniques like inverse reinforcement learning to learn from human behavior and preferences. Don't hesitate to experiment and try new approaches – RLHF is an iterative process!

1

1 Comment

Any dev will be able to tell you the importance of iteration speed when building software. One pattern that I'm seeing folks building with Superblocks is increasing ramping up that iteration speed to a whole new level. The killer combination: An engineer + someone from the design or product team + an end user of the internal application. Having everyone being able to collaborate takes a linear process and basically squashes it down to a single meeting. For example: 1. After a 60-90 minute call, it's common to leave these meetings with at least the scaffolding of a usable end application as it can be built in real time with real data. 2. A lot of the time they'll walk out of the meeting with a v2 or v3 of the project just because they were able to fit so many iterations of feedback from experimenting with different ideas. 3.. The end-user feels invested in the tool. This is especially important for internal software where getting adoption to use the tool is frequently a challenge.

24

And here we go... Langroid crosses 2,000 stars on GitHub! 🎉⭐️✨ Thank you to the dev community for your support, and a special shout-out to our awesome core contributors Nils Palumbo and Mohannad Alhanahnah, as well as our early adopters and contributors. Langroid is an open-source Python framework that simplifies building LLM applications, using a multi-agent paradigm inspired by the Actor framework. It is a good time to reflect on some key principles Somesh Jha and I had in mind when we co-founded Langroid: 🤖 Why multi-agents? There are many reasons, but I'll highlight a couple: ⦿ Cognitive load: using a single agent for a complex task leads to unreliable results. Instead, break it down into smaller tasks, each handled by a specialized agent, and have them iteratively collaborate on the task. ⦿ Feedback: "Critic" agents can provide feedback to "actor" agents, leading to better performance. ⦿ Cost: Specialized agents can use weaker LLMs for narrow tasks ➡️ faster, cheaper. But having multiple agents "collaborate" is not trivial -- Langroid is founded on a clean, principled agent communication architecture that seamlessly allows user-interaction, tool-handling and task delegation. 📦 Why use a framework? In theory, you don't need a framework. But you can be far more productive with a good framework -- one that gives the right primitives and abstractions, but is also transparent and extensible. Langroid strives to do just that, and we are happy to see companies adopting Langroid in production, after evaluating other frameworks. Here's what they love: 🤖 Clean, extensible codebase with high test coverage. 💬 Specialized agents to "chat" with documents, csv tables, SQL databases. 📄 Performant, extensible state-of-the art RAG implementation. 🧠 Works with practically any LLM served via an OpenAI-Compatible API, via groq, ollama, oobabooga, litellm. 🛠️ Tools/function-calling/structured-output for any LLM using an elegant Pydantic-based specification. Check out Langroid here: https://lnkd.in/dvDgGiXY Get started with this quickstart Colab notebook: https://lnkd.in/gnWm58he Or follow this quick-start tutorial: https://lnkd.in/gftdtcAE

70

6 Comments

It took 4 or 5 years longer than I originally predicted, but no-code's promise to non-technical founders is here. It looks different than I thought... instead of founders building products themselves for free, they're using agencies and freelancers and just paying less. I'm getting more and more leads from founders who originally got expensive quotes from dev shops. We end up doing the job for ~1/5 cost and sometimes 1/10 time. I launched a web app for a client last month. We built it in two weeks, without code. It already has over 100 paying customers. Real, profitable businesses are being built by leveraging experienced no-code builders! 5 years ago I went all in and converted my solo consultancy to a full no-code shop. It's finally paying off 😂. If you'd like to build something and hiring devs has been prohibitive, consider the no-code route. I'm happy to talk anytime!

12

Software Engineering and OSS: Recently, I’ve been sharing CUDA examples on here, for which I also have a GitHub repository. Last night, one of you opened a PR to improve one of the examples - and I merged it today. I really appreciate that. This is why open source is fun, and this is why I like sharing on here. So often I get to learn from you all. If you’d like to contribute to my CUDA examples repository, I’d appreciate that. Some guidelines (I don’t have a contributors file on there yet): 1. Keep examples simple 2. Either explain the details of the example in the PR or include a readme. I really appreciate contributions because it takes me time to add new stuff. This would be a great way to get some experience while also contributing to help people learn. Thanks in advance, and I look forward to collaborating! Repository in the comments. If you like my content, feel free to follow or connect! #softwareengineering #cuda

39

1 Comment