Lansing Lee

Last week Meta made huge announcements in AI with the release of Llama 3, the latest version of its open-source LLM, and that their Meta AI would be integrated across Meta properties. To discuss the launch of Llama 3, the speed of AI releases, and its impact across big tech, we brought Alex Kantrowitz, CEO of Big Technology onto the Taking Inventory podcast. With a top 10 podcast, 1M+ annual downloads, 150K+ newsletter subscribers and interviews with the biggest names in AI and tech like Ahmad Al-Dahle, Demis Hassabis, Jack Clark, Bryan Catanzaro, Marissa Mayer, there's no one better than Alex to break it all down. Don't miss our conversation as we go deep on: 1. Meta's latest AI announcements  2. The battle between "startups" (OpenAI, Anthropic) and big tech (Facebook, Google) 3. The changing culture in tech from summer camp to workplace 4. Self-driving as a joint effort between AI + humans Listen to our episode with Alex and subscribe here to never miss an episode: Spotify: https://lnkd.in/gBaThmXq Apple: https://lnkd.in/gJPPrfm5 Everywhere: https://lnkd.in/gRSt7XJv James Borow Greg Lieber Sean Friedland Shachar Scott Kent Moy Emad Hasan Cory Pitt Michael Carney Ali Manning

5

2 Comments

Been catching up on LLM papers and this new one on Dual Chunk Attention (DCA) jumped up my list after the Qwen 2 release. It looks like a game-changer. 🤩 They've figured out how to extend the context window of LLMs without any additional training, which is substantial for Llama devs. Compared to RoPE this does offer an edge. ➡️ Paper: https://lnkd.in/g465s8Ka ➡️ Code: https://lnkd.in/gZ7HP6Kk DCA cleverly sidesteps the usual limitations of positional embeddings by breaking down the attention mechanism into smaller, more manageable chunks. It uses three types of attention – intra-chunk, inter-chunk, and successive-chunk – to maintain those crucial relationships between words, even when they're miles apart in the input. This means we can now feed our models WAY more text and still get amazing results on tasks like summarizing lengthy documents or having those deep, meaningful conversations. The context length scaling does not drop off performance as dramatically as it did with other older scaling techniques, which is a major contribution (see the attached screenshot of Together/Llama2 versions compared to their "Chunk" versions (which applies DCA to these models to scale context length to over 190k). DCA is already powering the Qwen 2 release! 🙌 ➡️ Check it out: https://lnkd.in/gdg8X9aD Credit to Min Wei, PhD for sharing the Qwen 2 release news, which made me dig deeper into the scaling methods used by them. #LLM #DCA #NLP #AI #Llama #Innovation #MachineLearning #MachineLearning #ContextLengthScaling #DCA #ICML2024

10

Want a front-row seat to the frontier of tech? We’ve got the role (or two) for you! We’re expanding our frontier-tech team at Blackbird by hiring a Frontier Tech Investments Associate and Foundry Fellows! Got questions? We've got answers... 1️⃣ What are the roles? 🧪 Associate = a full-time VC investment gig in our Blackbird Investments team, working directly with me. We’re looking for someone with a science and/or engineering background and more curiosity than they can handle. You’ll grow your own investment brand and practice, support our portfolio founders, and will help build Foundry, our early-stage frontier-tech accelerator. ✨ Foundry Fellow = a casual/contract gig in our Blackbird Investments team, ~15h per week for 3 months. The Fellowship is ideal for PhD students and ECRs who want to learn more about startups and VC. You’ll go deep on emerging areas relevant to your expertise (or curiosity!), get a front-row seat to groundbreaking companies in those areas, build out your non-academic network, and develop a solid writing practice. 2️⃣ Why are you hiring? We love frontier tech, and we’re ready to grow our team. 3️⃣ Wow, it’s so great that you’re starting to look at deep tech! Look, we get it: we don’t make a lot of noise about our frontier tech investing. Buuuut we’ve been deep-tech investors since we backed Tim Kentley-Klay to found Zoox back in 2014—we’ve been on incredible journeys with PsiQuantum (building the world's first utility-scale quantum computer right here in Australia!), Inventia Life Science (transforming drug discovery with high-fidelity cell models), Remedy Robotics (surgical robots for remote endovascular procedures), Opto Biosystems (minimally-invasive neural implants to treat cancer), and more. We believe that frontier technologies, and great frontier-tech investing, will be part of the solutions to the greatest problems humanity faces today. 4️⃣ When do applications close? May 31st at 11:59pm AEST. 5️⃣ I have more questions! I’m sure you do! Clare Birch and I are hosting an AMA to answer any and all questions about these roles. Want to know what a week in the life of our team looks like? What’s keeping us up at night? What our ideal candidate looks like? Come along and find out - registration link in the comments 👇 Apply for these roles: Associate - https://lnkd.in/gCfj4EUJ Foundry Fellowships - https://lnkd.in/gj6ATZVZ If you know anyone that we should meet, send me their details! Cameron Elise Ben Andrew Robin Joseph Adelaide James Olivia Lucinda Raghav Jesse Christie Mohamed Tom Amee Pablo Haya Loong Hon Joshua Benjamin Megan Harry Denzil Matthew Diana Daniel Tom Deanna Justin Amar Lilly Stone Thomas

114

4 Comments

Harvey going from strength to strength… USD $1.5B is a big number 📈, but maybe this sentiment is more important: “The feedback from our clients, the validation from our investors, and the support of our team give us immense confidence in what Harvey can be: a #pioneer of the greatest #technology of our generation and a catalyst to more effective decision making in professional services. Continuing to push the #frontier of the world’s progress remains, as always, our ultimate goal.” 🌐⚖ #PwC #AI #Ḁї4law | #HarveyAI

3

I love YC Demo Day season. It’s an insane concentration of driven founders building something they deeply believe in at the frontier of tech. I know it's more trendy for VCs to complain about YC. But I believe Garry Tan when he says 5-10% of companies become worth a billion dollars. Maven’s hit rate from YC is even higher: 16% have had $B+ exits, and we’re trending toward a 40% graduation rate from Seed to Unicorn with investments we’ve made alongside YC. Since Maven’s founding in 2013, we’ve funded 12 YC startups out of the 66 companies we’ve invested in. Here are a few: Cruise: acquired by GM for $1B+ Embark: $4B SPAC, acquired by Applied Intuition Chariot: acquired by Ford before Series A May Mobility: raised $300M+ in follow on funding, including from Toyota Carrot Fertility: raised $100M+ in follow on funding, including from CRV and USVP Daybreak Health: Series A from Lightspeed, Series B from USV Moment Robotics: acquired by Cruise founder Kyle Vogt’s new company, The Bot Co (Maven is an investor) Our strategy differs from other YC investors. Many funds build a YC “index fund”, investing in many startups per batch. It’s not a bad move. But we focus on concentration, meeting all the companies in our thesis area and investing in one or two. Gratefully, I think every company from the list above can point to Maven as one of their most trusted partners that made a difference: intros to customers, closing follow-on funding from top VCs, connecting key hires, and offering genuine care and advice in both the best and toughest of moments. What are we looking for in a YC investment? 1) Team - a technical product team with startup DNA and hustle that can achieve a bold vision. Brilliant, determined founders no one would bet against who are ambitious enough to build an iconic company. 2) Vision worth fighting for - what are you bringing into the world that is worth all the time, effort and money required for success? Why aren’t existing solutions good enough? Are we proud to help bring this idea to the world? 3) Massive market - because of the valuation premium YC demands, even a $B outcome may not return today’s average Seed fund. Honestly, that math sucks. In order to pay the premium, we have to believe that if we’re right, we’ll be wildly successful. 4) Consumer trend - Maven invests in tech companies addressing emerging consumer trends. Even if some investments are B2B or B2B2C, they are always grounded in a consumer insight: autonomous vehicles in 2014, fertility care in 2017, mental health in 2020. Today, we’re looking at consumer applications of AI and frontier consumer health. Many successful companies we fund are outside of YC as well – like Zoom, Hello Heart, Epic!, Class, Wildtype. So we match YC’s Unicorn % across the rest of our portfolio, too 💪 YC founders in consumer tech and digital health: if you want to build an iconic company that delivers a positive impact for millions of consumers - I can’t wait to talk to you!

80

6 Comments

In the first innings of GenAI, much of the attention and $s have flowed into foundation models and GPUs. But that could change significantly. The foundation model layer is moving so fast that it may soon commoditize itself and move the action to higher layers of the tech stack. New, increasingly capable foundation models are coming out every day. Open source models are catching up rapidly with premier closed models. AI apps are being designed to select models for each request based on cost and task complexity. Model pricing is coming down rapidly even as quality goes up. For most pure-play, horizontal foundation models, marginal pricing may approach marginal cost, even as usage rises exponentially. In previous platform shifts such as PC, Internet and Mobile, a large portion of value eventually accrued to applications (e.g. Google, Amazon, Facebook), software platforms (e.g. Windows, Cloud platforms, Salesforce), and full solutions (e.g. Apple, Tesla). Some leading pure-play innovators in the hardware and enabling layers did create huge successes (e.g. Intel, Cisco, TSMC), but wider value accretion over time happened in the layers above. I did a rough, directional exercise looking through the top 50 public tech companies today by market cap, and classified them by the primary category they operate in (highly subjective, as many large companies eventually straddle multiple categories). Over half of the ~$20T combined market value of these top 50 tech companies comes from those I classified as Applications and Platforms. This included Microsoft, Alphabet, Amazon, Oracle & many others. A quarter came from those classified as Full Solutions, including Apple & Tesla. And only about a fifth is from those classified as Hardware or Enabling Tech, including Nvidia, TSMC, Intel et al. This is in spite of the rapid recent ascent in market caps in this category from the promise of AI. In contrast, the list of top 10 tech companies by market cap in the early Internet era frequently included pioneers in Hardware and Networking (key Enabling Tech for the Internet), including Cisco, Intel, Sun, Dell, Compaq, Lucent, 3Com, AT&T and Silicon Graphics. A large % of the premier tech talent in those days wanted to work in Hardware and Networking. Investors were disproportionately focused on those areas too. This led to those layers becoming so good that much of the new value accretion moved to the layers above. Similarly, apps and platforms for the enterprise AI era are just getting started, and that’s going to be a multi-decade, multi-trillion journey. Some leading foundation model players will expand to offer wider platforms or full solutions they can monetize better. Others leading models may come from those with a highly profitable, data-generating business at the layers above. But there will be many, many opportunities for pure-play app and platform companies that smartly build upon the underlying layers that seem to be getting ready for prime time.

58

4 Comments

YC is running its first fall batch. Comment below if you're thinking of applying, and we can try and connect you with a YC alum to review your application.

28

17 Comments