Jonathan Schoenberg

 A pre-print of my summer intern's recent paper, entitled “Domain decomposition-based coupling of operator inference reduced order models via the Schwarz alternating method”, is available on ArXiv here: https://lnkd.in/g8qZ94CA. The paper presents and evaluates an alternating Schwarz-based approach for coupling together subdomain-local reduced order models constructed via non-intrusive Operator Inference (OpInf) with each other and with subdomain-local full order models (FOMs).

53

Yet another innovative paper on State Space Models https://lnkd.in/eevz_GiE achieves arbitrarily low memory decay (meaning arbitrarily long memory) by filtering the input sequence with a precomputed (not learned) spectral filter. Recurrent models, such as SSMs, have a general form of x_t = A x_t-1 = A^2 x_t-2 = … = A^t x_0. Because of the power term A^t, they model nonlinearity well. However, this comes with a cost, ||A|| has to be < 1. Otherwise, the model is “explosive” and will grow out of control rather quickly. For any A satisfying ||A|| < 1, A^t goes to 0 very quickly, meaning the effect of x_0 on x_t diminishes rapidly, which is known as “memory decay”. This limitation is why SSMs may not outperform transformers in the needle-in-the-haystack test. That is until this paper. And the trick? View the signal in its waveform. If something occurs repeatedly without decay, it must be a wave. By using a spectral filter that transforms signals from the time domain to the frequency domain, the paper proves that Spectral SSMs can have a memory longer than any given L. And the best part? The spectral filter is precomputed, not trained. Like any other SSMs, a hybrid architecture leads to better performance. Intuitively, this is because some signals can be efficiently expressed in the frequency domain, while some other signals can be efficiently expressed in the time domain, and having both helps. #artificialintelligence #machinelearning #deeplearning

74

3 Comments

New preprint: Evaluating AI-generated code for C++, Fortran, Go, Java, Julia, Matlab, Python, R, and Rust This study evaluates the capabilities of ChatGPT versions 3.5 and 4 in generating code across a diverse range of programming languages. Our objective is to assess the effectiveness of these AI models for generating scientific programs. To this end, we asked ChatGPT to generate three distinct codes: a simple numerical integration, a conjugate gradient solver, and a parallel 1D stencil-based heat equation solver. The focus of our analysis was on the compilation, runtime performance, and accuracy of the codes. While both versions of ChatGPT successfully created codes that compiled and ran (with some help), some languages were easier for the AI to use than others (possibly because of the size of the training sets used). Parallel codes -- even the simple example we chose to study here -- also difficult for the AI to generate correctly. https://lnkd.in/g7byA9ZH

18

3 Comments

🚀 Advanced Information Retrieval with Qdrant & ColBERT in Action! 🚀 In my latest article, I dive into how cutting-edge information retrieval techniques can revolutionize search accuracy and efficiency. 🧠 🌟 Key highlights: 1. The MaxSim algorithm driving the power of ColBERT and Late Interaction 2. Using Qdrant for efficient vector search on large-scale datasets 3. How dense embeddings and term-wise token interaction boost precision Curious to see how modern search techniques are pushing the boundaries of relevance and speed? Check it out and learn how these technologies are shaping the future of information retrieval! 🔍⚡ #GenAI, #LLM

16

Id say the hardest thing about integrating LLMs(Ai) into current workflows is it requires the need to eliminate the countless layers of single case calluses brought on by human error in the current systems. In many cases you can almost eliminate complete workflows, Ui's, and touch points. *The issue is you need to think in terms of subtraction not addition or you get stuck in the middle and the results are horrible. (maybe this is what is slowing linear adoption)

5

2 Comments

https://lnkd.in/gXVrPZE One of my favorite papers. It got basically no recognition, but is really useful for discrete time quantum walks on graphs.

3

Amazon's annual shareholder letters are always a great read. This time on building Primitives :) "However, when you think in primitives, like our Robotics team does, you prioritize the building blocks, picking important initiatives that can benefit from each of these primitives, but which build the tool chest to compose more freely (and quickly) for future and complex needs. Our Robotics team has built primitives in each of the above domains that will be lynchpins in our next set of automation, which includes multi-floor storage, trailer loading and unloading, large pallet mobility, and more flexible sortation across our outbound processes (including in vehicles). The team is also building a set of foundation AI models to better identify products in complex environments, optimize the movement of our growing robotic fleet, and better manage the bottlenecks in our facilities."

5

🚀 Why Qdrant? Vector search made simple and powerful! 🚀 Whether you're building the next-gen search engine or enhancing your ML pipeline, Qdrant delivers speed, accuracy, and ease of use. #GenAI, #LLM, #VectorSearch

38

2 Comments

I’m thrilled to share that our paper titled "MACM: Utilizing a Multi-Agent System for Condition Mining in Solving Complex Mathematical Problems" has been accepted at NeurIPS 2024! 🎉 This work is a collaboration with Bin Lei, Yi Zhang, Shan Zuo, and Caiwen Ding, and I couldn't be more grateful for their insights and hard work. Our algorithm, MACM, is now the top performer on the MATH dataset, one of the toughest benchmarks for mathematical reasoning in LLMs (https://lnkd.in/g4g6c9fU) In this research, we tackled one of the biggest challenges facing large language models like GPT-4: solving advanced mathematical problems that require multi-step logical reasoning. MACM, significantly boosts the problem-solving accuracy of LLMs on the most challenging level-five mathematical problems in the MATH dataset, increasing it from 54.68% to an impressive 76.73%! 🚀 If you're interested in diving deeper into our work, you can find the full paper here: https://lnkd.in/gvdyxm8i. I'd love to hear your thoughts and feedback! #NeurIPS2024 #AIResearch #MathematicalReasoning #LLM #CiscoResearch #CiscoOutshift #Cisco

83

6 Comments