Showing 1–26 of 26 results for author: Buehler, M J

AtomAgents: Alloy design and discovery through physics-aware multi-modal multi-agent artificial intelligence

Authors: Alireza Ghafarollahi, Markus J. Buehler

Abstract: The design of alloys is a multi-scale problem that requires a holistic approach that involves retrieving relevant knowledge, applying advanced computational methods, conducting experimental validations, and analyzing the results, a process that is typically reserved for human experts. Machine learning (ML) can help accelerate this process, for instance, through the use of deep surrogate models tha… ▽ More The design of alloys is a multi-scale problem that requires a holistic approach that involves retrieving relevant knowledge, applying advanced computational methods, conducting experimental validations, and analyzing the results, a process that is typically reserved for human experts. Machine learning (ML) can help accelerate this process, for instance, through the use of deep surrogate models that connect structural features to material properties, or vice versa. However, existing data-driven models often target specific material objectives, offering limited flexibility to integrate out-of-domain knowledge and cannot adapt to new, unforeseen challenges. Here, we overcome these limitations by leveraging the distinct capabilities of multiple AI agents that collaborate autonomously within a dynamic environment to solve complex materials design tasks. The proposed physics-aware generative AI platform, AtomAgents, synergizes the intelligence of large language models (LLM) the dynamic collaboration among AI agents with expertise in various domains, including knowledge retrieval, multi-modal data integration, physics-based simulations, and comprehensive results analysis across modalities that includes numerical data and images of physical simulation results. The concerted effort of the multi-agent system allows for addressing complex materials design problems, as demonstrated by examples that include autonomously designing metallic alloys with enhanced properties compared to their pure counterparts. Our results enable accurate prediction of key characteristics across alloys and highlight the crucial role of solid solution alloying to steer the development of advanced metallic alloys. Our framework enhances the efficiency of complex multi-objective design tasks and opens new avenues in fields such as biomedical materials engineering, renewable energy, and environmental sustainability. △ Less

Submitted 13 July, 2024; originally announced July 2024.

Multicell-Fold: geometric learning in folding multicellular life

Authors: Haiqian Yang, Anh Q. Nguyen, Dapeng Bi, Markus J. Buehler, Ming Guo

Abstract: During developmental processes such as embryogenesis, how a group of cells fold into specific structures, is a central question in biology that defines how living organisms form. Establishing tissue-level morphology critically relies on how every single cell decides to position itself relative to its neighboring cells. Despite its importance, it remains a major challenge to understand and predict… ▽ More During developmental processes such as embryogenesis, how a group of cells fold into specific structures, is a central question in biology that defines how living organisms form. Establishing tissue-level morphology critically relies on how every single cell decides to position itself relative to its neighboring cells. Despite its importance, it remains a major challenge to understand and predict the behavior of every cell within the living tissue over time during such intricate processes. To tackle this question, we propose a geometric deep learning model that can predict multicellular folding and embryogenesis, accurately capturing the highly convoluted spatial interactions among cells. We demonstrate that multicellular data can be represented with both granular and foam-like physical pictures through a unified graph data structure, considering both cellular interactions and cell junction networks. We successfully use our model to achieve two important tasks, interpretable 4-D morphological sequence alignment, and predicting local cell rearrangements before they occur at single-cell resolution. Furthermore, using an activation map and ablation studies, we demonstrate that cell geometries and cell junction networks together regulate local cell rearrangement which is critical for embryo morphogenesis. This approach provides a novel paradigm to study morphogenesis, highlighting a unified data structure and harnessing the power of geometric deep learning to accurately model the mechanisms and behaviors of cells during development. It offers a pathway toward creating a unified dynamic morphological atlas for a variety of developmental processes such as embryogenesis. △ Less

Submitted 22 July, 2024; v1 submitted 9 July, 2024; originally announced July 2024.

Cephalo: Multi-Modal Vision-Language Models for Bio-Inspired Materials Analysis and Design

Authors: Markus J. Buehler

Abstract: We present Cephalo, a series of multimodal vision large language models (V-LLMs) designed for materials science applications, integrating visual and linguistic data for enhanced understanding. A key innovation of Cephalo is its advanced dataset generation method. Cephalo is trained on integrated image and text data from thousands of scientific papers and science-focused Wikipedia data demonstrates… ▽ More We present Cephalo, a series of multimodal vision large language models (V-LLMs) designed for materials science applications, integrating visual and linguistic data for enhanced understanding. A key innovation of Cephalo is its advanced dataset generation method. Cephalo is trained on integrated image and text data from thousands of scientific papers and science-focused Wikipedia data demonstrates can interpret complex visual scenes, generate precise language descriptions, and answer queries about images effectively. The combination of a vision encoder with an autoregressive transformer supports multimodal natural language understanding, which can be coupled with other generative methods to create an image-to-text-to-3D pipeline. To develop more capable models from smaller ones, we report both mixture-of-expert methods and model merging. We examine the models in diverse use cases that incorporate biological materials, fracture and engineering analysis, protein biophysics, and bio-inspired design based on insect behavior. Generative applications include bio-inspired designs, including pollen-inspired architected materials, as well as the synthesis of bio-inspired material microstructures from a photograph of a solar eclipse. Additional model fine-tuning with a series of molecular dynamics results demonstrate Cephalo's enhanced capabilities to accurately predict statistical features of stress and atomic energy distributions, as well as crack dynamics and damage in materials. △ Less

Submitted 15 July, 2024; v1 submitted 29 May, 2024; originally announced May 2024.

Accelerating Scientific Discovery with Generative Knowledge Extraction, Graph-Based Representation, and Multimodal Intelligent Graph Reasoning

Authors: Markus J. Buehler

Abstract: Leveraging generative Artificial Intelligence (AI), we have transformed a dataset comprising 1,000 scientific papers into an ontological knowledge graph. Through an in-depth structural analysis, we have calculated node degrees, identified communities and connectivities, and evaluated clustering coefficients and betweenness centrality of pivotal nodes, uncovering fascinating knowledge architectures… ▽ More Leveraging generative Artificial Intelligence (AI), we have transformed a dataset comprising 1,000 scientific papers into an ontological knowledge graph. Through an in-depth structural analysis, we have calculated node degrees, identified communities and connectivities, and evaluated clustering coefficients and betweenness centrality of pivotal nodes, uncovering fascinating knowledge architectures. The graph has an inherently scale-free nature, is highly connected, and can be used for graph reasoning by taking advantage of transitive and isomorphic properties that reveal unprecedented interdisciplinary relationships that can be used to answer queries, identify gaps in knowledge, propose never-before-seen material designs, and predict material behaviors. We compute deep node embeddings for combinatorial node similarity ranking for use in a path sampling strategy links dissimilar concepts that have previously not been related. One comparison revealed structural parallels between biological materials and Beethoven's 9th Symphony, highlighting shared patterns of complexity through isomorphic mapping. In another example, the algorithm proposed a hierarchical mycelium-based composite based on integrating path sampling with principles extracted from Kandinsky's 'Composition VII' painting. The resulting material integrates an innovative set of concepts that include a balance of chaos/order, adjustable porosity, mechanical strength, and complex patterned chemical functionalization. We uncover other isomorphisms across science, technology and art, revealing a nuanced ontology of immanence that reveal a context-dependent heterarchical interplay of constituents. Graph-based generative AI achieves a far higher degree of novelty, explorative capacity, and technical detail, than conventional approaches and establishes a widely useful framework for innovation by revealing hidden connections. △ Less

Submitted 10 June, 2024; v1 submitted 18 March, 2024; originally announced March 2024.

X-LoRA: Mixture of Low-Rank Adapter Experts, a Flexible Framework for Large Language Models with Applications in Protein Mechanics and Molecular Design

Authors: Eric L. Buehler, Markus J. Buehler

Abstract: We report a mixture of expert strategy to create fine-tuned large language models using a deep layer-wise token-level approach based on low-rank adaptation (LoRA). Starting with a set of pre-trained LoRA adapters, our gating strategy uses the hidden states to dynamically mix adapted layers, allowing the resulting X-LoRA model to draw upon different capabilities and create never-before-used deep la… ▽ More We report a mixture of expert strategy to create fine-tuned large language models using a deep layer-wise token-level approach based on low-rank adaptation (LoRA). Starting with a set of pre-trained LoRA adapters, our gating strategy uses the hidden states to dynamically mix adapted layers, allowing the resulting X-LoRA model to draw upon different capabilities and create never-before-used deep layer-wise combinations to solve tasks. The design is inspired by the biological principles of universality and diversity, where neural network building blocks are reused in different hierarchical manifestations. Hence, the X-LoRA model can be easily implemented for any existing large language model (LLM) without a need for modifications of the underlying structure. We develop a tailored X-LoRA model that offers scientific capabilities including forward/inverse analysis tasks and enhanced reasoning capability, focused on biomaterial analysis, protein mechanics and design. The impact of this work include access to readily expandable and adaptable models with strong domain knowledge and the capability to integrate across areas of knowledge. Featuring experts in biology, mathematics, reasoning, bio-inspired materials, mechanics and materials, chemistry, protein biophysics, mechanics and quantum-mechanics based molecular properties, we conduct a series of physics-focused case studies. We examine knowledge recall, protein mechanics forward/inverse tasks, protein design, adversarial agentic modeling including ontological knowledge graph construction, as well as molecular design. The model is capable not only of making quantitative predictions of nanomechanical properties of proteins or quantum mechanical molecular properties, but also reasons over the results and correctly predicts likely mechanisms that explain distinct molecular behaviors. △ Less

Submitted 30 March, 2024; v1 submitted 11 February, 2024; originally announced February 2024.

ProtAgents: Protein discovery via large language model multi-agent collaborations combining physics and machine learning

Authors: A. Ghafarollahi, M. J. Buehler

Abstract: Designing de novo proteins beyond those found in nature holds significant promise for advancements in both scientific and engineering applications. Current methodologies for protein design often rely on AI-based models, such as surrogate models that address end-to-end problems by linking protein structure to material properties or vice versa. However, these models frequently focus on specific mate… ▽ More Designing de novo proteins beyond those found in nature holds significant promise for advancements in both scientific and engineering applications. Current methodologies for protein design often rely on AI-based models, such as surrogate models that address end-to-end problems by linking protein structure to material properties or vice versa. However, these models frequently focus on specific material objectives or structural properties, limiting their flexibility when incorporating out-of-domain knowledge into the design process or comprehensive data analysis is required. In this study, we introduce ProtAgents, a platform for de novo protein design based on Large Language Models (LLMs), where multiple AI agents with distinct capabilities collaboratively address complex tasks within a dynamic environment. The versatility in agent development allows for expertise in diverse domains, including knowledge retrieval, protein structure analysis, physics-based simulations, and results analysis. The dynamic collaboration between agents, empowered by LLMs, provides a versatile approach to tackling protein design and analysis problems, as demonstrated through diverse examples in this study. The problems of interest encompass designing new proteins, analyzing protein structures and obtaining new first-principles data -- natural vibrational frequencies -- via physics simulations. The concerted effort of the system allows for powerful automated and synergistic design of de novo proteins with targeted mechanical properties. The flexibility in designing the agents, on one hand, and their capacity in autonomous collaboration through the dynamic LLM-based multi-agent environment on the other hand, unleashes great potentials of LLMs in addressing multi-objective materials problems and opens up new avenues for autonomous materials discovery and design. △ Less

Submitted 27 January, 2024; originally announced February 2024.

Valorizing Sewage Sludge: Using Nature-Inspired Architecture to Overcome Intrinsic Weaknesses of Waste-Based Materials

Authors: Sabrina C. Shen, Branden Spitzer, Damian Stefaniuk, Shengfei Zhou, Admir Masic, Markus J. Buehler

Abstract: Sewage sludge, a biosolid product of wastewater processing, is an often-overlooked source of rich organic waste. Hydrothermal processing (HTP), which uses heat and pressure to convert biomass into various solid, liquid, and gaseous products, has shown promise in converting sewage sludge into new materials with potential application in biofuels, asphalt binders, and bioplastics. In this study we fo… ▽ More Sewage sludge, a biosolid product of wastewater processing, is an often-overlooked source of rich organic waste. Hydrothermal processing (HTP), which uses heat and pressure to convert biomass into various solid, liquid, and gaseous products, has shown promise in converting sewage sludge into new materials with potential application in biofuels, asphalt binders, and bioplastics. In this study we focus on hydrochar, the carbonaceous HTP solid phase, and investigate its use as a bio-based filler in additive manufacturing technologies. We explore the impact of HTP and subsequent thermal activation on chemical and structural properties of sewage sludge and discuss the role of atypical metallic and metalloid dopants in organic material processing. In additive manufacturing composites, although the addition of hydrochar generally decreases mechanical performance, we show that toughness and strain can be recovered with hierarchical microstructures, much like biological materials that achieve outstanding properties by architecting relatively weak building blocks. △ Less

Submitted 23 January, 2024; originally announced January 2024.

Learning Dynamics from Multicellular Graphs with Deep Neural Networks

Authors: Haiqian Yang, Florian Meyer, Shaoxun Huang, Liu Yang, Cristiana Lungu, Monilola A. Olayioye, Markus J. Buehler, Ming Guo

Abstract: Multicellular self-assembly into functional structures is a dynamic process that is critical in the development and diseases, including embryo development, organ formation, tumor invasion, and others. Being able to infer collective cell migratory dynamics from their static configuration is valuable for both understanding and predicting these complex processes. However, the identification of struct… ▽ More Multicellular self-assembly into functional structures is a dynamic process that is critical in the development and diseases, including embryo development, organ formation, tumor invasion, and others. Being able to infer collective cell migratory dynamics from their static configuration is valuable for both understanding and predicting these complex processes. However, the identification of structural features that can indicate multicellular motion has been difficult, and existing metrics largely rely on physical instincts. Here we show that using a graph neural network (GNN), the motion of multicellular collectives can be inferred from a static snapshot of cell positions, in both experimental and synthetic datasets. △ Less

Submitted 8 July, 2024; v1 submitted 22 January, 2024; originally announced January 2024.

MechAgents: Large language model multi-agent collaborations can solve mechanics problems, generate new data, and integrate knowledge

Authors: Bo Ni, Markus J. Buehler

Abstract: Solving mechanics problems using numerical methods requires comprehensive intelligent capability of retrieving relevant knowledge and theory, constructing and executing codes, analyzing the results, a task that has thus far mainly been reserved for humans. While emerging AI methods can provide effective approaches to solve end-to-end problems, for instance via the use of deep surrogate models or v… ▽ More Solving mechanics problems using numerical methods requires comprehensive intelligent capability of retrieving relevant knowledge and theory, constructing and executing codes, analyzing the results, a task that has thus far mainly been reserved for humans. While emerging AI methods can provide effective approaches to solve end-to-end problems, for instance via the use of deep surrogate models or various data analytics strategies, they often lack physical intuition since knowledge is baked into the parametric complement through training, offering less flexibility when it comes to incorporating mathematical or physical insights. By leveraging diverse capabilities of multiple dynamically interacting large language models (LLMs), we can overcome the limitations of conventional approaches and develop a new class of physics-inspired generative machine learning platform, here referred to as MechAgents. A set of AI agents can solve mechanics tasks, here demonstrated for elasticity problems, via autonomous collaborations. A two-agent team can effectively write, execute and self-correct code, in order to apply finite element methods to solve classical elasticity problems in various flavors (different boundary conditions, domain geometries, meshes, small/finite deformation and linear/hyper-elastic constitutive laws, and others). For more complex tasks, we construct a larger group of agents with enhanced division of labor among planning, formulating, coding, executing and criticizing the process and results. The agents mutually correct each other to improve the overall team-work performance in understanding, formulating and validating the solution. Our framework shows the potential of synergizing the intelligence of language models, the reliability of physics-based modeling, and the dynamic collaborations among diverse agents, opening novel avenues for automation of solving engineering problems. △ Less

Submitted 14 November, 2023; originally announced November 2023.

Generative retrieval-augmented ontologic graph and multi-agent strategies for interpretive large language model-based materials design

Authors: Markus J. Buehler

Abstract: Transformer neural networks show promising capabilities, in particular for uses in materials analysis, design and manufacturing, including their capacity to work effectively with both human language, symbols, code, and numerical data. Here we explore the use of large language models (LLMs) as a tool that can support engineering analysis of materials, applied to retrieving key information about sub… ▽ More Transformer neural networks show promising capabilities, in particular for uses in materials analysis, design and manufacturing, including their capacity to work effectively with both human language, symbols, code, and numerical data. Here we explore the use of large language models (LLMs) as a tool that can support engineering analysis of materials, applied to retrieving key information about subject areas, developing research hypotheses, discovery of mechanistic relationships across disparate areas of knowledge, and writing and executing simulation codes for active knowledge generation based on physical ground truths. When used as sets of AI agents with specific features, capabilities, and instructions, LLMs can provide powerful problem solution strategies for applications in analysis and design problems. Our experiments focus on using a fine-tuned model, MechGPT, developed based on training data in the mechanics of materials domain. We first affirm how finetuning endows LLMs with reasonable understanding of domain knowledge. However, when queried outside the context of learned matter, LLMs can have difficulty to recall correct information. We show how this can be addressed using retrieval-augmented Ontological Knowledge Graph strategies that discern how the model understands what concepts are important and how they are related. Illustrated for a use case of relating distinct areas of knowledge - here, music and proteins - such strategies can also provide an interpretable graph structure with rich information at the node, edge and subgraph level. We discuss nonlinear sampling strategies and agent-based modeling applied to complex question answering, code generation and execution in the context of automated force field development from actively learned Density Functional Theory (DFT) modeling, and data analysis. △ Less

Submitted 30 October, 2023; originally announced October 2023.

ForceGen: End-to-end de novo protein generation based on nonlinear mechanical unfolding responses using a protein language diffusion model

Authors: Bo Ni, David L. Kaplan, Markus J. Buehler

Abstract: Through evolution, nature has presented a set of remarkable protein materials, including elastins, silks, keratins and collagens with superior mechanical performances that play crucial roles in mechanobiology. However, going beyond natural designs to discover proteins that meet specified mechanical properties remains challenging. Here we report a generative model that predicts protein designs to m… ▽ More Through evolution, nature has presented a set of remarkable protein materials, including elastins, silks, keratins and collagens with superior mechanical performances that play crucial roles in mechanobiology. However, going beyond natural designs to discover proteins that meet specified mechanical properties remains challenging. Here we report a generative model that predicts protein designs to meet complex nonlinear mechanical property-design objectives. Our model leverages deep knowledge on protein sequences from a pre-trained protein language model and maps mechanical unfolding responses to create novel proteins. Via full-atom molecular simulations for direct validation, we demonstrate that the designed proteins are novel, and fulfill the targeted mechanical properties, including unfolding energy and mechanical strength, as well as the detailed unfolding force-separation curves. Our model offers rapid pathways to explore the enormous mechanobiological protein sequence space unconstrained by biological synthesis, using mechanical features as target to enable the discovery of protein materials with superior mechanical properties. △ Less

Submitted 15 December, 2023; v1 submitted 16 October, 2023; originally announced October 2023.

MechGPT, a language-based strategy for mechanics and materials modeling that connects knowledge across scales, disciplines and modalities

Authors: Markus J. Buehler

Abstract: For centuries, researchers have sought out ways to connect disparate areas of knowledge. While early scholars (Galileo, da Vinci, etc.) were experts across fields, specialization has taken hold later. With the advent of Artificial Intelligence, we can now explore relationships across areas (e.g., mechanics-biology) or disparate domains (e.g., failure mechanics-art). To achieve this, we use a fine-… ▽ More For centuries, researchers have sought out ways to connect disparate areas of knowledge. While early scholars (Galileo, da Vinci, etc.) were experts across fields, specialization has taken hold later. With the advent of Artificial Intelligence, we can now explore relationships across areas (e.g., mechanics-biology) or disparate domains (e.g., failure mechanics-art). To achieve this, we use a fine-tuned Large Language Model (LLM), here for a subset of knowledge in multiscale materials failure. The approach includes the use of a general-purpose LLM to distill question-answer pairs from raw sources followed by LLM fine-tuning. The resulting MechGPT LLM foundation model is used in a series of computational experiments to explore its capacity for knowledge retrieval, various language tasks, hypothesis generation, and connecting knowledge across disparate areas. While the model has some ability to recall knowledge from training, we find that LLMs are particularly useful to extract structural insights through Ontological Knowledge Graphs. These interpretable graph structures provide explanatory insights, frameworks for new research questions, and visual representations of knowledge that also can be used in retrieval-augmented generation. Three versions of MechGPT are discussed, featuring different sizes from 13 billion to 70 billion parameters, and reaching context lengths of more than 10,000 tokens. This provides ample capacity for sophisticated retrieval augmented strategies, as well as agent-based modeling where multiple LLMs interact collaboratively and/or adversarially, the incorporation of new data from the literature or web searches, as well as multimodality. △ Less

Submitted 16 October, 2023; originally announced October 2023.

Crosslinker energy landscape effects on dynamic mechanical properties of ideal polymer hydrogels

Authors: Eesha Khare, Amadeus Alcantara, Nic Lee, Munir S. Skaf, Markus J. Buehler

Abstract: Reversible crosslinkers can enable several desirable mechanical properties, such as improved toughness and self-healing, when incorporated in polymer networks for bioengineering and structural applications. In this work, we performed coarse-grained molecular dynamics to investigate the effect of the energy landscape of reversible crosslinkers on the dynamic mechanical properties of crosslinked pol… ▽ More Reversible crosslinkers can enable several desirable mechanical properties, such as improved toughness and self-healing, when incorporated in polymer networks for bioengineering and structural applications. In this work, we performed coarse-grained molecular dynamics to investigate the effect of the energy landscape of reversible crosslinkers on the dynamic mechanical properties of crosslinked polymer network hydrogels. We report that, for an ideal network, the energy potential of the crosslinker interaction drives the viscosity of the network, where a stronger potential results in a higher viscosity. Additional topographical analyses reveal a mechanistic understanding of the structural rearrangement of the network as it deforms and indicate that as the number of defects increases in the network, the viscosity of the network increases. As an important validation for the relationship between the energy landscape of a crosslinker chemistry and the resulting dynamic mechanical properties of a crosslinked ideal network hydrogel, this work enhances our understanding of deformation mechanisms in polymer networks that cannot easily be revealed by experiment and reveals design ideas that can lead to better performance of the polymer network at the macroscale. △ Less

Submitted 3 October, 2023; originally announced October 2023.

Generative modeling, design and analysis of spider silk protein sequences for enhanced mechanical properties

Authors: Wei Lu, David L. Kaplan, Markus J. Buehler

Abstract: Spider silks are remarkable materials characterized by superb mechanical properties such as strength, extensibility and lightweightedness. Yet, to date, limited models are available to fully explore sequence-property relationships for analysis and design. Here we propose a custom generative large-language model to enable design of novel spider silk protein sequences to meet complex combinations of… ▽ More Spider silks are remarkable materials characterized by superb mechanical properties such as strength, extensibility and lightweightedness. Yet, to date, limited models are available to fully explore sequence-property relationships for analysis and design. Here we propose a custom generative large-language model to enable design of novel spider silk protein sequences to meet complex combinations of target mechanical properties. The model, pretrained on a large set of protein sequences, is fine-tuned on ~1,000 major ampullate spidroin (MaSp) sequences for which associated fiber-level mechanical properties exist, to yield an end-to-end forward and inverse generative strategy. Performance is assessed through: (1), a novelty analysis and protein type classification for generated spidroin sequences through BLAST searches, (2) property evaluation and comparison with similar sequences, (3) comparison of molecular structures, as well as, and (4) a detailed sequence motif analyses. We generate silk sequences with property combinations that do not exist in nature, and develop a deep understanding the mechanistic roles of sequence patterns in achieving overarching key mechanical properties (elastic modulus, strength, toughness, failure strain). The model provides an efficient approach to expand the silkome dataset, facilitating further sequence-structure analyses of silks, and establishes a foundation for synthetic silk design and optimization. △ Less

Submitted 18 September, 2023; originally announced September 2023.

BioinspiredLLM: Conversational Large Language Model for the Mechanics of Biological and Bio-inspired Materials

Authors: Rachel K. Luu, Markus J. Buehler

Abstract: The study of biological materials and bio-inspired materials science is well established; however, surprisingly little knowledge has been systematically translated to engineering solutions. To accelerate discovery and guide insights, an open-source autoregressive transformer large language model (LLM), BioinspiredLLM, is reported. The model was finetuned with a corpus of over a thousand peer-revie… ▽ More The study of biological materials and bio-inspired materials science is well established; however, surprisingly little knowledge has been systematically translated to engineering solutions. To accelerate discovery and guide insights, an open-source autoregressive transformer large language model (LLM), BioinspiredLLM, is reported. The model was finetuned with a corpus of over a thousand peer-reviewed articles in the field of structural biological and bio-inspired materials and can be prompted to recall information, assist with research tasks, and function as an engine for creativity. The model has proven that it is able to accurately recall information about biological materials and is further enhanced with enhanced reasoning ability, as well as with retrieval-augmented generation to incorporate new data during generation that can also help to traceback sources, update the knowledge base, and connect knowledge domains. BioinspiredLLM also has been shown to develop sound hypotheses regarding biological materials design and remarkably so for materials that have never been explicitly studied before. Lastly, the model showed impressive promise in collaborating with other generative artificial intelligence models in a workflow that can reshape the traditional materials design process. This collaborative generative artificial intelligence method can stimulate and enhance bio-inspired materials design workflows. Biological materials are at a critical intersection of multiple scientific fields and models like BioinspiredLLM help to connect knowledge domains. △ Less

Submitted 11 December, 2023; v1 submitted 15 September, 2023; originally announced September 2023.

MeLM, a generative pretrained language modeling framework that solves forward and inverse mechanics problems

Authors: Markus J. Buehler

Abstract: We report a flexible multi-modal mechanics language model, MeLM, applied to solve various nonlinear forward and inverse problems, that can deal with a set of instructions, numbers and microstructure data. The framework is applied to various examples including bio-inspired hierarchical honeycomb design, carbon nanotube mechanics, and protein unfolding. In spite of the flexible nature of the model-w… ▽ More We report a flexible multi-modal mechanics language model, MeLM, applied to solve various nonlinear forward and inverse problems, that can deal with a set of instructions, numbers and microstructure data. The framework is applied to various examples including bio-inspired hierarchical honeycomb design, carbon nanotube mechanics, and protein unfolding. In spite of the flexible nature of the model-which allows us to easily incorporate diverse materials, scales, and mechanical features-it performs well across disparate forward and inverse tasks. Based on an autoregressive attention-model, MeLM effectively represents a large multi-particle system consisting of hundreds of millions of neurons, where the interaction potentials are discovered through graph-forming self-attention mechanisms that are then used to identify relationships from emergent structures, while taking advantage of synergies discovered in the training data. We show that the model can solve complex degenerate mechanics design problems and determine novel material architectures across a range of hierarchical levels, providing an avenue for materials discovery and analysis. Looking beyond the demonstrations reported in this paper, we discuss other opportunities in applied mechanics and general considerations about the use of large language models in modeling, design, and analysis that can span a broad spectrum of material properties from mechanical, thermal, optical, to electronic. △ Less

Submitted 30 June, 2023; originally announced June 2023.

Robust Myco-Composites as a Platform for Versatile Hybrid-Living Structural Materials

Authors: Sabrina C. Shen, Nicolas A. Lee, William J. Lockett, Aliai D. Acuil, Hannah B. Gazdus, Branden N. Spitzer, Markus J. Buehler

Abstract: Fungal mycelium, a living network of filamentous threads, thrives on lignocellulosic waste and exhibits rapid growth, hydrophobicity, and intrinsic regeneration, offering a potential means to create next-generation sustainable and functional composites. However, existing hybrid-living mycelium composites (myco-composites) are tremendously constrained by conventional mold-based manufacturing proces… ▽ More Fungal mycelium, a living network of filamentous threads, thrives on lignocellulosic waste and exhibits rapid growth, hydrophobicity, and intrinsic regeneration, offering a potential means to create next-generation sustainable and functional composites. However, existing hybrid-living mycelium composites (myco-composites) are tremendously constrained by conventional mold-based manufacturing processes, which are only compatible with simple geometries and coarse biomass substrates that enable gas exchange. Here we introduce a class of structural myco-composites manufactured with a novel platform that harnesses high-resolution biocomposite additive manufacturing and robust mycelium colonization with indirect inoculation. We leverage principles of hierarchical composite design and selective nutritional provision to create a robust myco-composite that is scalable, tunable, and compatible with complex geometries. To illustrate the versatility of this platform, we characterize the impact of mycelium colonization on mechanical and surface properties of the composite, finding that it yields the strongest mycelium composite reported to date, and demonstrate fabrication of unique foldable bio-welded containers and flexible mycelium textiles. This study bridges the gap between biocomposite and hybrid-living materials research, opening the door to advanced structural mycelium applications and demonstrating a novel platform for development of diverse hybrid-living materials. △ Less

Submitted 12 August, 2023; v1 submitted 20 May, 2023; originally announced May 2023.

Generative Pretrained Autoregressive Transformer Graph Neural Network applied to the Analysis and Discovery of Novel Proteins

Authors: Markus J. Buehler

Abstract: We report a flexible language-model based deep learning strategy, applied here to solve complex forward and inverse problems in protein modeling, based on an attention neural network that integrates transformer and graph convolutional architectures in a causal multi-headed graph mechanism, to realize a generative pretrained model. The model is applied to predict secondary structure content (per-re… ▽ More We report a flexible language-model based deep learning strategy, applied here to solve complex forward and inverse problems in protein modeling, based on an attention neural network that integrates transformer and graph convolutional architectures in a causal multi-headed graph mechanism, to realize a generative pretrained model. The model is applied to predict secondary structure content (per-residue level and overall content), protein solubility, and sequencing tasks. Further trained on inverse tasks, the model is rendered capable of designing proteins with these properties as target features. The model is formulated as a general framework, completely prompt-based, and can be adapted for a variety of downstream tasks. We find that adding additional tasks yields emergent synergies that the model exploits in improving overall performance, beyond what would be possible by training a model on each dataset alone. Case studies are presented to validate the method, yielding protein designs specifically focused on structural proteins, but also exploring the applicability in the design of soluble, antimicrobial biomaterials. While our model is trained to ultimately perform 8 distinct tasks, with available datasets it can be extended to solve additional problems. In a broader sense, this work illustrates a form of multiscale modeling that relates a set of ultimate building blocks (here, byte-level utf8 characters that define the nature of the physical system at hand) to complex output. This materiomic scheme captures complex emergent relationships between universal building block and resulting properties via a synergizing learning capacity to express a set of potentialities embedded in the knowledge used in training, via the interplay of universality and diversity. △ Less

Submitted 11 July, 2023; v1 submitted 7 May, 2023; originally announced May 2023.

Generative Discovery of Novel Chemical Designs using Diffusion Modeling and Transformer Deep Neural Networks with Application to Deep Eutectic Solvents

Authors: Rachel K. Luu, Marcin Wysokowski, Markus J. Buehler

Abstract: We report a series of deep learning models to solve complex forward and inverse design problems in molecular modeling and design. Using both diffusion models inspired by nonequilibrium thermodynamics and attention-based transformer architectures, we demonstrate a flexible framework to capture complex chemical structures. First trained on the QM9 dataset and a series of quantum mechanical propertie… ▽ More We report a series of deep learning models to solve complex forward and inverse design problems in molecular modeling and design. Using both diffusion models inspired by nonequilibrium thermodynamics and attention-based transformer architectures, we demonstrate a flexible framework to capture complex chemical structures. First trained on the QM9 dataset and a series of quantum mechanical properties (e.g. homo, lumo, free energy, heat capacity, etc.), we then generalize the model to study and design key properties of deep eutectic solvents. In addition to separate forward and inverse models, we also report an integrated fully prompt-based multi-task generative pretrained transformer model that solves multiple forward, inverse design, and prediction tasks, flexibly and within one model. We show that the multi-task generative model has the overall best performance and allows for flexible integration of multiple objectives, within one model, and for distinct chemistries, suggesting that synergies emerge during training of this large language model. Trained jointly in tasks related to the QM9 dataset and deep eutectic solvents (DESs), the model can predict various quantum mechanical properties and critical properties to achieve deep eutectic solvent behavior. Several novel combinations of DESs are proposed based on this framework. △ Less

Submitted 24 April, 2023; originally announced April 2023.

Modeling and design of heterogeneous hierarchical bioinspired spider web structures using generative deep learning and additive manufacturing

Authors: Wei Lu, Nic A. Lee, Markus J. Buehler

Abstract: Spider webs are incredible biological structures, comprising thin but strong silk filament and arranged into complex hierarchical architectures with striking mechanical properties (e.g., lightweight but high strength, achieving diverse mechanical responses). While simple 2D orb webs can easily be mimicked, the modeling and synthesis of 3D-based web structures remain challenging, partly due to the… ▽ More Spider webs are incredible biological structures, comprising thin but strong silk filament and arranged into complex hierarchical architectures with striking mechanical properties (e.g., lightweight but high strength, achieving diverse mechanical responses). While simple 2D orb webs can easily be mimicked, the modeling and synthesis of 3D-based web structures remain challenging, partly due to the rich set of design features. Here we provide a detailed analysis of the heterogenous graph structures of spider webs, and use deep learning as a way to model and then synthesize artificial, bio-inspired 3D web structures. The generative AI models are conditioned based on key geometric parameters (including average edge length, number of nodes, average node degree, and others). To identify graph construction principles, we use inductive representation sampling of large experimentally determined spider web graphs, to yield a dataset that is used to train three conditional generative models: 1) An analog diffusion model inspired by nonequilibrium thermodynamics, with sparse neighbor representation, 2) a discrete diffusion model with full neighbor representation, and 3) an autoregressive transformer architecture with full neighbor representation. All three models are scalable, produce complex, de novo bio-inspired spider web mimics, and successfully construct graphs that meet the design objectives. We further propose algorithm that assembles web samples produced by the generative models into larger-scale structures based on a series of geometric design targets, including helical and parametric shapes, mimicking, and extending natural design principles towards integration with diverging engineering objectives. Several webs are manufactured using 3D printing and tested to assess mechanical properties. △ Less

Submitted 11 April, 2023; originally announced April 2023.

Diatom-inspired architected materials using language-based deep learning: Perception, transformation and manufacturing

Authors: Markus J. Buehler

Abstract: Learning from nature has been a quest of humanity for millennia. While this has taken the form of humans assessing natural designs such as bones, butterfly wings, or spider webs, we can now achieve generating designs using advanced computational algorithms. In this paper we report novel biologically inspired designs of diatom structures, enabled using transformer neural networks, using natural lan… ▽ More Learning from nature has been a quest of humanity for millennia. While this has taken the form of humans assessing natural designs such as bones, butterfly wings, or spider webs, we can now achieve generating designs using advanced computational algorithms. In this paper we report novel biologically inspired designs of diatom structures, enabled using transformer neural networks, using natural language models to learn, process and transfer insights across manifestations. We illustrate a series of novel diatom-based designs and also report a manufactured specimen, created using additive manufacturing. The method applied here could be expanded to focus on other biological design cues, implement a systematic optimization to meet certain design targets, and include a hybrid set of material design sets. △ Less

Submitted 14 January, 2023; originally announced January 2023.

Journal ref: In: Perspectives on the Mechanics of Fracture & Biological Materials, ISBN 978-1-4716-1942-7, 2022

Architected Materials for Mechanical Compression: Design via Simulation, Deep Learning, and Experimentation

Authors: Andrew J. Lew, Kai Jin, Markus J. Buehler

Abstract: Architected materials can achieve enhanced properties compared to their plain counterparts. Specific architecting serves as a powerful design lever to achieve targeted behavior without changing the base material. Thus, the connection between architected structure and resultant properties remains an open field of great interest to many fields, from aerospace to civil to automotive applications. Her… ▽ More Architected materials can achieve enhanced properties compared to their plain counterparts. Specific architecting serves as a powerful design lever to achieve targeted behavior without changing the base material. Thus, the connection between architected structure and resultant properties remains an open field of great interest to many fields, from aerospace to civil to automotive applications. Here, we focus on properties related to mechanical compression, and design hierarchical honeycomb structures to meet specific values of stiffness and compressive stress. To do so, we employ a combination of techniques in a singular workflow, starting with molecular dynamics simulation of the forward design problem, augmenting with data-driven artificial intelligence models to address the inverse design problem, and verifying the behavior of de novo structures with experimentation of additively manufactured samples. We thereby demonstrate an approach for architected design that is generalizable to multiple material properties and agnostic to the identity of the base material. △ Less

Submitted 13 February, 2023; v1 submitted 5 December, 2022; originally announced December 2022.

DyFraNet: Forecasting and Backcasting Dynamic Fracture Mechanics in Space and Time Using a 2D-to-3D Deep Neural Network

Authors: Yu-Chuan Hsu, Markus J. Buehler

Abstract: The dynamics of materials failure is one of the most critical phenomena in a range of scientific and engineering fields, from healthcare to structural materials to transportation. In this paper we propose a specially designed deep neural network, DyFraNet, which can predict dynamic fracture behaviors by identifying a complete history of fracture propagation - from cracking onset, as a crack grows… ▽ More The dynamics of materials failure is one of the most critical phenomena in a range of scientific and engineering fields, from healthcare to structural materials to transportation. In this paper we propose a specially designed deep neural network, DyFraNet, which can predict dynamic fracture behaviors by identifying a complete history of fracture propagation - from cracking onset, as a crack grows through the material, modeled as a series of frames evolving over time and dependent on each other. Furthermore, this model can not only forecast future fracture processes but also backcast to elucidate the past fracture history. In this scenario, once provided with the outcome of a fracture event, the model will elucidate past events that led to this state and will predict the future evolution of the failure process. By comparing the predicted results with atomistic-level simulations and theory, we show that DyFraNet can capture dynamic fracture mechanics by accurately predicting how cracks develop over time, including measures such as the crack speed, as well as when cracks become unstable. We use GradCAM to interpret how DyFraNet perceives the relationship between geometric conditions and fracture dynamics and we find DyFraNet pays special attention to the areas around crack tips, which have a critical influence in the early stage of fracture propagation. In later stages, the model pays increased attention to the existing or newly formed damage distribution in the material. The proposed approach offers significant potential to accelerate the exploration of the dynamics in material design against fracture failures and can be beneficially adapted for all kinds of dynamical engineering problems. △ Less

Submitted 15 November, 2022; originally announced November 2022.

Comments: Deep learning, dynamic fracture mechanics, crack speed, molecular dynamics, crystalline solids, next-frame prediction, forecasting, backcasting

Sub-Nanometer Channels Embedded in Two-Dimensional Materials

Authors: Yimo Han, Ming-Yang Li, Gang-Seob Jung, Mark A. Marsalis, Zhao Qin, Markus J. Buehler, Lain-Jong Li, David A. Muller

Abstract: Two-dimensional (2D) materials are among the most promising candidates for next-generation electronics due to their atomic thinness, allowing for flexible transparent electronics and ultimate length scaling. Thus far, atomically-thin p-n junctions, metal-semiconductor contacts, and metal-insulator barriers have been demonstrated. While 2D materials achieve the thinnest possible devices, precise na… ▽ More Two-dimensional (2D) materials are among the most promising candidates for next-generation electronics due to their atomic thinness, allowing for flexible transparent electronics and ultimate length scaling. Thus far, atomically-thin p-n junctions, metal-semiconductor contacts, and metal-insulator barriers have been demonstrated. While 2D materials achieve the thinnest possible devices, precise nanoscale control over the lateral dimensions is also necessary. Here, we report the direct synthesis of sub-nanometer-wide 1D MoS2 channels embedded within WSe2 monolayers, using a dislocation-catalyzed approach. The 1D channels have edges free of misfit dislocations and dangling bonds, forming a coherent interface with the embedding 2D matrix. Periodic dislocation arrays produce 2D superlattices of coherent MoS2 1D channels in WSe2. Using molecular dynamics simulations, we have identified other combinations of 2D materials where 1D channels can also be formed. The electronic band structure of these 1D channels offer the promise of carrier confinement in a direct-gap material and charge separation needed to access the ultimate length scales necessary for future electronic applications. △ Less

Submitted 23 January, 2018; v1 submitted 31 July, 2017; originally announced July 2017.

Comments: 22 pages main manuscript and methods, 4 main figures, 30 pages supplementary materials, 16 extended figures

Journal ref: Nature Materials 2017

Category theoretic analysis of hierarchical protein materials and social networks

Authors: David I. Spivak, Tristan Giesa, Elizabeth Wood, Markus J. Buehler

Abstract: Materials in biology span all the scales from Angstroms to meters and typically consist of complex hierarchical assemblies of simple building blocks. Here we describe an application of category theory to describe structural and resulting functional properties of biological protein materials by developing so-called ologs. An olog is like a "concept web" or "semantic network" except that it follows… ▽ More Materials in biology span all the scales from Angstroms to meters and typically consist of complex hierarchical assemblies of simple building blocks. Here we describe an application of category theory to describe structural and resulting functional properties of biological protein materials by developing so-called ologs. An olog is like a "concept web" or "semantic network" except that it follows a rigorous mathematical formulation based on category theory. This key difference ensures that an olog is unambiguous, highly adaptable to evolution and change, and suitable for sharing concepts with other olog. We consider simple cases of alpha-helical and amyloid-like protein filaments subjected to axial extension and develop an olog representation of their structural and resulting mechanical properties. We also construct a representation of a social network in which people send text-messages to their nearest neighbors and act as a team to perform a task. We show that the olog for the protein and the olog for the social network feature identical category-theoretic representations, and we proceed to precisely explicate the analogy or isomorphism between them. The examples presented here demonstrate that the intrinsic nature of a complex system, which in particular includes a precise relationship between structure and function at different hierarchical levels, can be effectively represented by an olog. This, in turn, allows for comparative studies between disparate materials or fields of application, and results in novel approaches to derive functionality in the design of de novo hierarchical systems. We discuss opportunities and challenges associated with the description of complex biological materials by using ologs as a powerful tool for analysis and design in the context of materiomics, and we present the potential impact of this approach for engineering, life sciences, and medicine. △ Less

Submitted 10 July, 2011; v1 submitted 11 March, 2011; originally announced March 2011.

Comments: 6 Figures, 3 Tables

MSC Class: 00A69; 18A99

Tearing Graphene Sheets From Adhesive Substrates Produces Tapered Nanoribbons

Authors: Dipanjan Sen, Kostya S. Novoselov, Pedro M. Reis, Markus J. Buehler

Abstract: Graphene is a truly two-dimensional atomic crystal with exceptional electronic and mechanical properties. Whereas conventional bulk and thin-film materials have been studied extensively, the key mechanical properties of graphene, such as tearing and cracking, remain unknown, partly due to its two-dimensional nature and ultimate single-atom-layer thickness, which result in the breakdown of conventi… ▽ More Graphene is a truly two-dimensional atomic crystal with exceptional electronic and mechanical properties. Whereas conventional bulk and thin-film materials have been studied extensively, the key mechanical properties of graphene, such as tearing and cracking, remain unknown, partly due to its two-dimensional nature and ultimate single-atom-layer thickness, which result in the breakdown of conventional material models. By combining first-principles ReaxFF molecular dynamics and experimental studies, a bottom-up investigation of the tearing of graphene sheets from adhesive substrates is reported, including the observation of the formation of tapered graphene nanoribbons. Through a careful analysis of the underlying molecular rupture mechanisms, it is shown that the resulting nanoribbon geometry is controlled by both the graphene-substrate adhesion energy and by the number of torn graphene layers. By considering graphene as a model material for a broader class of two-dimensional atomic crystals, these results provide fundamental insights into the tearing and cracking mechanisms of highly confined nanomaterials. △ Less

Submitted 24 May, 2010; originally announced May 2010.

Journal ref: Small 6(10), 1108-1116 (2010)