Communication

Challenges and Limitations of ChatGPT and Artificial Intelligence

for Scientific Research: A Perspective from Organic Materials
Hao-Wen Cheng

Department of Chemical Engineering, Stanford University, Stanford, CA 94305, USA; [email protected]

Abstract: Artificial Intelligence (AI) has emerged as a transformative technology in the scientific
community with the potential to accelerate and enhance research in various fields. ChatGPT, a
popular language model, is one such AI-based system that is increasingly being discussed and being
adapted in scientific research. However, as with any technology, there are challenges and limitations
that need to be addressed. This paper focuses on the challenges and limitations that ChatGPT faces
in the domain of organic materials research. This paper will take organic materials as examples in the
use of ChatGPT. Overall, this paper aims to provide insights into the challenges and limitations of
researchers working in the field of organic materials.

Keywords: artificial intelligence; ChatGPT; organic materials

1. Introduction
In late 2022, OpenAI, an artificial intelligence (AI) company, introduced an interactive
chatbot named ChatGPT which attracts significant attention from both economic and
academic fields. ChatGPT quickly reached 1 million users within just five days [1], setting
a new benchmark for speedy user acquisition. To provide perspective, ChatGPT’s growth
rate was 15 times faster than TikTok, the fastest-growing social media platform. Moreover,
ChatGPT continued to set an unparalleled record for growth, reaching 100 million users
within two months of its launch [2]. The concept of whether machines can think was
Citation: Cheng, H.-W. Challenges
and Limitations of ChatGPT and
proposed by A.M. Turing, in 1950, demonstrated a famous test to determine whether
Artificial Intelligence for Scientific
humans can distinguish between conversations with humans and machines [3]. ChatGPT
Research: A Perspective from easily passes the Turing test, indicating that the era of AI has indeed arrived.
Organic Materials. AI 2023, 4, OpenAI’s first large language model (LLM), called GPT-1, was launched in 2017
401–405. https://doi.org/10.3390/ followed by subsequent versions of GPT-2, GPT-3, and the widely-discussed ChatGPT. In
ai4020021 scientific society, ChatGPT is recently acclaimed for its ability to assist in summarizing
research papers, provide general experimental procedures, and compare experiment results,
Academic Editor: José Machado
thus providing a more efficient approach than internet surfing [4–7]. It is widely believed
Received: 13 March 2023 that LLMs, such as ChatGPT, will make disruptive changes, becoming indispensable
Revised: 13 April 2023 assistants for professional use.
Accepted: 28 April 2023 The future impacts of ChatGPT on scientific writing and publishing have been quickly
Published: 4 May 2023 realized and discussed by many researchers [4,6,7]. However, how this open-accessible
AI tool can be utilized to facilitate scientific research remains unexplored. This article will
utilize ChatGPT as an LLM model to demonstrate how this technique can assist in scientific
research, using organic materials as an example due to their recent attention in the field
Copyright: © 2023 by the author.
of flexible and wearable electronics [8–12]. Additionally, I will examine the limitations of
Licensee MDPI, Basel, Switzerland.
ChatGPT for research purposes and these challenges will be discussed into three categories.
This article is an open access article
distributed under the terms and
2. Results
conditions of the Creative Commons
Attribution (CC BY) license (https://
Starting a new research project can be a challenging task, especially when delving into
creativecommons.org/licenses/by/
unfamiliar areas for beginners. It often requires a substantial amount of academic literature
4.0/). to come up with ideas which can be a time-consuming process. However, ChatGPT offers

AI 2023, 4, 401–405. https://doi.org/10.3390/ai4020021 https://www.mdpi.com/journal/ai

AI 2023, 4 402

a convenient solution by providing quick background and directions within seconds.

For instance, by asking ChatGPT to “Summarize the recent organic light emitting diode
development in polymers” or “Give me five top cited papers on stretchable polymer
semiconductors for organic transistors”, the researcher can quickly obtain knowledge
through the summaries from ChatGPT. The other example is asking “List the most popular
research directions for organic solar cells” which yields a range of potential directions,
including materials development, device optimization, tandem and hybrid devices, large
area and flexible devices, and stability and durability (see Supporting Information S1).
One of the significant advantages of ChatGPT is that it generates information based on the
user’s input, making it more relevant to their needs.
To delve deeper into the materials aspect of organic solar cells, I requested more
specific details from ChatGPT. Here, the issue comes up. While the inorganic material
“perovskite” was suggested, it is essential to note that this material is not included in organic
solar cells. This indicates ChatGPT lacks the capability to give us useful information. For a
more comprehensive response, please refer to Supporting Information S2.
In my pursuit to leverage the capabilities of the AI-powered chatbot to augment our
research potential, I have identified three potential limitations in its responses. These in-
clude (i) computational calculation constraints, (ii) the potential for inaccurate or false facts,
and (iii) inadequacies in inferential capability. It is crucial to acknowledge these limitations
to ensure that the chatbot’s responses are interpreted and applied with appropriate caution
and context.
The first issue is the computational capability of ChatGPT. For example, I asked “The
material provided an NIR absorption spectrum with an onset of 945 nm, what is the optical
bandgap?”. Here is what ChatGPT replied:
“To calculate the optical bandgap, we can use the following formula:

Eg = hc/λ,

where Eg is the bandgap energy, h is Planck’s constant, c is the speed of light, and λ is the
wavelength corresponding to the onset of the absorption spectrum.
In this case, the onset wavelength is 945 nm. Plugging this value into the formula,
we get:
Eg = (6.626 × 10−34 J s) × (2.998 × 108 m/s)/(945 × 10−9 m),

Eg = 2.08 eV.
Therefore, the optical bandgap of the material is approximately 2.08 eV.”
It has become evident that ChatGPT may occasionally provide incorrect responses,
such as when it miscalculates units, leading to an inaccurate answer. For example, when
asked “What is the value of 99,999,999 + 0.9”, the chatbot’s response was “The result of
99,999,999 + 0.9 is 100,000,000.9” instead of “99,999,999.9”. This issue has been previously
investigated by a group at Oxford [13]. To address this concern, it is recommended to
double check and clarify queries to ensure accurate responses. A possible approach is to
request a recalculation and provide additional details to guide ChatGPT’s response. For
instance, by typing “I think there might be an error. Can you please recalculate the bandgap
with these parameters?”.
“I apologize for the mistake in my previous response. Based on the information given,
if the onset of the near-infrared (NIR) absorption spectrum is near 945 nm, then the optical
bandgap can be calculated as follows:

1 eV = 1240/λ (in nm).

Therefore, the energy corresponding to an onset of 945 nm is

E = 1240/945 = 1.31 eV.

AI 2023, 4 403

So, the optical bandgap is 1.31 eV.”

What amazed me is that ChatGPT admitted the fault and successfully replied with the
correct answer.
Second, it is noteworthy that ChatGPT failed to response to false facts. For instance,
I asked the question “Why polymers mobility is higher than small molecules mobility?”.
In common cases, the mobility of small molecule single crystals is generally higher than
polymers. However, ChatGPT did not correct the question but generated responses based
on false facts and combined the facts into paragraphs. See full response in Supporting
Information S3. Thus, when asking these kinds of questions, users should be careful.
Furthermore, it is essential to note that ChatGPT may lack inferential capabilities in
certain scenarios. As evident from the example provided where the scenario that is “Now
you have three donor materials for organic solar cells to match with the same acceptor.
The highest occupied molecular orbital (HOMO) energy levels of A, B, and C are −5 eV,
−5.1 eV, and −5.2 eV. The lowest unoccupied molecular orbital (LUMO) energy level of the
acceptor is −4 eV. Which combination of materials has the highest and lowest open-circuit
voltage (V oc )?” Here is the response.
“Assuming that the V oc is determined only by the energy levels of the donor and
acceptor materials, the V oc of a solar cell can be calculated by the difference between the
HOMO of the donor and the LUMO of the acceptor.
For the given materials, the V oc can be calculated as follows:
A: V oc = (−5 eV) − (−4 eV) = −1 eV
B: V oc = (−5.1 eV) − (−4 eV) = −1.1 eV
C: V oc = (−5.2 eV) − (−4 eV) = −1.2 eV
Therefore, the combination of materials with the highest V oc is Material A, and the
combination with the lowest V oc is Material C”.
Apparently, the V oc will not have negative values and the answers were wrong.
Therefore, it is advisable to refrain from asking ChatGPT inferential questions as it may
lead to incorrect answers being generated.
Lastly, the use of ChatGPT in organic materials research raises various ethical consid-
erations that need to be addressed. One potential issue is the risk of biases in generated
content due to the data biases that users provide. The use of AI tools, such as ChatGPT,
could also have implications for the scientific community, potentially shifting the research
focus away from important areas. Privacy and security concerns arise when processing
sensitive or unpublished data, and researchers must ensure that appropriate measures are
in place to prevent unauthorized access or misuse of data. The generated content may be
subject to intellectual property laws, creating legal challenges and the risk of copyright
infringement. Accountability is another important factor that needs to be considered in
the use of AI tools, such as ChatGPT, in scientific research, and researchers must be trans-
parent about their use and accountable for the research outcomes, including any potential
inaccuracies or biases in the generated content.

3. Conclusions
To conclude, ChatGPT is able to provide assistance in many new research areas, and
we are still at the very early stage of exploring its application scope. I have no doubt that
AI tools will become a game-changing player in all fields, including our scientific society.
However, there are many challenges in terms of research purposes. So far ChatGPT is
limited by computational constraints, the ability to point out inaccurate information or
false faces, and inferential capability for scientific research. All those limitations may lead
to misunderstandings or misinterpretation. To address these challenges, researchers are
immersing themselves to improve the accuracy and reliability of the model.
Emerging technologies and advancements in AI have had a significant impact on the
development and performance of language models, such as ChatGPT. For example, new
AI techniques, such as deep learning and neural networks, have enabled the creation of
AI 2023, 4 404

more sophisticated and accurate language models that can process vast amounts of data
and learn from it.
In addition, the rise of big data and cloud computing has made it possible to train
and run language models at scale, opening up new possibilities for applications, such as
chatbots, virtual assistants, and language translation. Moreover, the development of pre-
trained language models, such as GPT-3, has reduced the amount of data and computing
resources required to train new models, making it easier for researchers and developers to
create their own language models.
For example, collaboration with linguists can help to refine language models by incor-
porating a better understanding of the nuances of human language and communication.
Collaboration with computer scientists can lead to the development of more efficient and
effective algorithms for training and running language models. Collaborations with psy-
chologists and neuroscientists can help to refine language models by providing insights
into how the human brain processes language.
Overall, the impact of emerging technologies and advancements in AI on the de-
velopment and performance of language models, such as ChatGPT is significant, and
the potential for interdisciplinary collaboration is vast. By leveraging these advances
and collaborating across disciplines, we can continue to drive innovation in the field of
language modeling and unlock new possibilities for applications in a wide range of in-
dustries and contexts. I strongly believe that good use of the fast-growing LLM models,
such as ChatGPT, can benefit scientific research as a revolutionary tool and can boost
technology development.

Supplementary Materials: The following supporting information can be downloaded at: https://
www.mdpi.com/article/10.3390/ai4020021/s1, The supplementary covers the ChatGPT records
being discussed in the article, including “List the most popular research directions for organic solar
cells”, “Give me more details on the materials aspect”, and “Why polymers mobility is higher than
small molecules mobility”.
Funding: This research received no external funding.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: All responses were collected from chat.openai.com.
Acknowledgments: I would like to express my appreciation to Xiaozhou Ji for engaging in valuable
discussions with me.
Conflicts of Interest: The author declares no conflict of interest.

References
1. ChatGPT Sprints to One Million Users. Available online: https://www.statista.com/chart/29174/time-to-one-million-users/
(accessed on 10 March 2023).
2. ChatGPT Reaches 100 Million Users Two Months after Launch. Available online: https://www.theguardian.com/technology/20
23/feb/02/chatgpt-100-million-users-open-ai-fastest-growing-app (accessed on 10 March 2023).
3. Turing, A.M. Computing Machinery and Intelligence. Mind 1950, 236, 433–460. [CrossRef]
4. Thorp, H.H. ChatGPT Is Fun, but Not an Author. Science 2023, 379, 313. [CrossRef] [PubMed]
5. Kirmani, A.R. Artificial Intelligence-Enabled Science Poetry. ACS Energy Lett. 2022, 8, 574–576. [CrossRef]
6. Grimaldi, G.; Ehrler, B. AI et al.: Machines Are About to Change Scientific Publishing Forever. ACS Energy Lett. 2023, 8, 878–880.
[CrossRef]
7. Buriak, J.M.; Akinwande, D.; Artzi, N.; Jeffrey Brinker, C.; Burrows, C.; Chan, W.C.W.; Chen, C.; Chen, X.; Chhowalla, M.; Chi, L.; et al.
Best Practices for Using AI When Writing Scientific Manuscripts. ACS Nano 2023, 17, 4091–4093. [CrossRef]
8. Cheng, H.-W.; Zhao, Y.; Yang, Y. Toward High-Performance Semitransparent Organic Photovoltaics with Narrow-Bandgap
Donors and Non-Fullerene Acceptors. Adv. Energy Mater. 2021, 12, 2102908. [CrossRef]
9. Cheng, H.-W.; Zhang, S.; Michalek, L.; Ji, X.; Luo, S.; Cooper, C.B.; Gong, H.; Nikzad, S.; Chiong, J.A.; Wu, Y.; et al. Realizing
Intrinsically Stretchable Semiconducting Polymer Films by Nontoxic Additives. ACS Mater. Lett. 2022, 4, 2328–2336. [CrossRef]
10. Huang, H.-C.; Lin, Y.-C.; Chen, C.-H.; Wei, K.-H.; Su, Y.-W.; Chen, P.-T. Density Functional Theory Study of Donor–Acceptor
Conjugated Polymers with Substituent Effect. J. Polym. Res. 2021, 28, 427. [CrossRef]
AI 2023, 4 405

11. Su, Y.-W.; Huang, Y.-S.; Huang, H.-C.; Chen, P.-T. Optoelectronic Properties of a Benzodithiophene-Based Organic Photovoltaic.
ECS J. Solid State Sci. Technol. 2021, 10, 075003. [CrossRef]
12. Cheng, H.-W.; Juan, C.-Y.; Mohapatra, A.; Chen, C.-H.; Lin, Y.-C.; Chang, B.; Cheng, P.; Wang, H.-C.; Chu, C.W.; Yang, Y.; et al.
High-Performance Organic Photovoltaics Incorporating an Active Layer with a Few Nanometer-Thick Third-Component Layer
on a Binary Blend Layer. Nano Lett. 2021, 21, 2207–2215. [CrossRef] [PubMed]
13. Frieder, S.; Pinchetti, L.; Griffiths, R.; Salvatori, T.; Lukasiewicz, T.; Petersen, P.C.; Chevalier, A.; Berner, J. Mathematical
Capabilities of ChatGPT. arXiv 2023, arXiv:2301.13867. [CrossRef]

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.