Volume 9, Issue 9, September– 2024 International Journal of Innovative Science and Research Technology

ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/IJISRT24SEP1335

Implementation of Real-Time Audio-to-Text

Conversion and Processing for Seamless
Transformation in Classroom Environment
Likith P1 Sahana N D2
AI&ML AI&ML
DSCE Bangalore, India DSCE Bangalore, India

Puneeth P3 Niranthar M4
AI&ML AI&ML
DSCE Bangalore, India DSCE Bangalore, India

Aruna M G5
AI & ML Faculty
Associate Proffesor
DSCE Bangalore, India

Abstract:- In modern educational settings, ensuring that I. INTRODUCTION

all students effectively comprehend lecture content is a
significant challenge, particularly when language barriers In contemporary educational environments, one of the
and varying levels of cognitive processing ability come critical challenges is ensuring that all students can effectively
into play. Traditional classroom instruction often fails to understand and engage with lecture content. This issue is
accommodate the diverse needs of all learners, leading to particularly pronounced in classrooms with diverse linguistic
gaps in understanding and academic performance. This backgrounds, varying levels of cognitive processing abilities,
research explores the implementation of a real-time audio and different learning needs. Traditional instructional
transcription system as a solution to enhance classroom methods often fall short in addressing these disparities,
inclusivity and comprehension. By leveraging mobile resulting in comprehension gaps and hindered academic
technology, students can access live speech-to-text performance for many students. To tackle this issue, the
transcriptions of lectures directly on their devices. This integration of real-time audio transcription technology in
system is designed to assist students who struggle with classrooms presents a promising solution.
listening due to language differences, hearing
impairments, or other cognitive challenges. Speech-to-text (STT) models have made significant
strides in recent years, powered by advancements in machine
The study delves into the technological framework of learning and natural language processing (NLP). These
the transcription service, evaluating its accuracy, latency, models convert spoken language into written text in real-
and usability. It also considers the practical implications time, offering a valuable tool for enhancing classroom
of integrating such technology into various classroom inclusivity. Major providers of STT services, such as Google
settings, from elementary schools to higher education Cloud Speech-to-Text, Microsoft Azure Speech, and IBM
institutions. Through surveys and interviews with Watson, utilize sophisticated algorithms to ensure high
students and teachers, the research assesses the impact of accuracy and speed. Features such as language detection,
live transcription on student engagement, participation, speaker diarization, and noise cancellation have further
and academic performance. Preliminary findings suggest improved the reliability and usability of these systems.
that live audio transcription can significantly bridge the
comprehension gap, offering a practical tool to foster a Incorporating multilingual capabilities in STT models is
more inclusive and effective learning environment. crucial for classrooms with diverse student populations.
Additionally, the study explores potential challenges and These models can recognize and transcribe multiple
solutions for widespread implementation, aiming to languages simultaneously, allowing students who are non-
provide a comprehensive analysis of the benefits and native speakers of the instruction language to follow along
limitations of this innovative educational tool. more easily. By providing real-time transcription on mobile
devices, students can access written versions of lectures,
facilitating better understanding and retention of the material.

IJISRT24SEP1335 www.ijisrt.com 2328

Volume 9, Issue 9, September– 2024 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/IJISRT24SEP1335

Despite these technological advancements, students still the availability of live transcripts enhances understanding and
face several challenges that hinder their learning experience. retention of lecture content. Additionally, it explores learners'
Language barriers, hearing impairments, and cognitive attitudes towards this tool, assessing its perceived usefulness
processing differences can significantly impact a student's and any potential drawbacks, such as over-reliance on
ability to keep up with verbal instruction. Live transcription transcripts that might impede the development of listening
can mitigate these issues by offering a visual aid that skills. Overall, the study seeks to provide insights into
complements auditory learning. Furthermore, it can support optimizing educational strategies for diverse L2 learners.
students with learning disabilities, such as dyslexia, by
providing a written reference that can be reviewed at their  Research Gaps
own pace. Despite the promise shown by integrating real-time
audio transcription technology in classrooms, several
The introduction of STT technology in educational research gaps and areas for improvement remain. One
settings involves several considerations, including the significant gap is the need for comprehensive transcription
accuracy of transcriptions, the latency of the service, and the that includes all verbal activities within the classroom, not
practicality of device integration. Additionally, the just the lectures. This would provide students with a more
effectiveness of these solutions must be evaluated through complete and valuable learning resource. Additionally,
direct feedback from students and educators. By examining merely providing transcription is insufficient; students would
the benefits and potential obstacles of implementing live benefit greatly from summaries that offer an overview of
audio transcription in classrooms, this research aims to class content. Implementing natural language processing
provide a comprehensive overview of how such technologies (NLP) techniques to generate these summaries and highlight
can foster a more inclusive and effective learning key terms could significantly aid students in reviewing and
environment. reinforcing their learning. Furthermore, integrating these
summaries and key term lists into daily homework
II. RELATED WORK assignments could improve student performance and
engagement. Leveraging large language models (LLMs) to
"Using Speech Recognition for Real-Time Captioning provide detailed explanations and conclusive details on
and Lecture Transcription in the Classroom"[1] extracted key terms offers an opportunity to enhance
personalized learning, helping to bridge comprehension gaps.
The paper examines the application of speech Research should also focus on ensuring the accessibility and
recognition technology to deliver real-time captioning and usability of STT technology for all students, including those
transcription in educational environments, aiming to improve with disabilities, by designing user-friendly interfaces.
accessibility for students, especially those who are deaf or Effective implementation requires that teachers are
hard of hearing. By leveraging advanced speech recognition adequately trained to use these tools, so investigating best
systems, the research seeks to create an inclusive learning practices for teacher training is essential. Moreover,
atmosphere where spoken content is immediately transcribed addressing data privacy and security concerns through robust
and displayed as text. This facilitates better comprehension protocols and guidelines is crucial, as is exploring students'
and participation for students with hearing impairments, and parents' perceptions of these issues. By addressing these
ensuring they receive the same information as their peers. research gaps, we can develop a more comprehensive and
The study evaluates the technology's effectiveness, accuracy, effective approach to using real-time transcription technology
and practicality in classrooms, addressing challenges such as in education, ensuring that all students, regardless of their
background noise and technical jargon to refine the system linguistic or cognitive challenges, benefit from a more
for optimal educational use. inclusive and supportive learning environment.

The role of live transcripts in synchronous online L2 III. DESIGN

classrooms: Learning outcomes and learner perceptions[2]
The proposed real-time audio transcription system
The study investigates the effects of live transcripts on leverages a combination of hardware and software
learning outcomes and student perceptions in synchronous components to deliver an effective and inclusive learning
online second language (L2) classrooms. It focuses on how tool. The figure:1 represents the proposed architecture which
automatically generated live transcripts affect students' provides the seamless integration and usability in classroom
comprehension and engagement, especially among different environments, ensuring that students can access
proficiency levels. By examining both high and low transcriptions and summaries of classroom discussions in
proficiency learners, the research aims to determine whether real-time.

IJISRT24SEP1335 www.ijisrt.com 2329

Volume 9, Issue 9, September– 2024 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/IJISRT24SEP1335

Fig 1 System Architecture

 System Workflow: the network. This streaming is facilitated by configuring

the Raspberry Pi to host the web application via the
 Audio Capture: Audio is captured in real-time using the router’s IP address. This setup ensures that any device
Raspberry Pi’s microphone and streamed to the backend connected to the same router can access the audio stream.
using PyAudio.
 Audio Processing and Transcription The streamed audio  Processing and Transcription
is sent to the Deepgram API via the Flask backend. The
Deepgram API processes the audio and returns the  Flask Server: The backend server is implemented using
transcriptions in real-time. Python Flask, which handles the routing and processing
 Display and Interaction: The transcriptions are sent from of audio data. Flask receives the audio stream from
the Flask server to the front-end interface, where they are PyAudio and forwards it to the Deepgram API for
displayed to students. The front-end interface also allows transcription.
students to access summarized content and highlighted  Deepgram API Integration: The Deepgram API processes
key terms generated from the transcriptions. the audio stream in real-time, converting the spoken
words into text. The API leverages advanced speech
This architecture ensures that the system is both recognition models to ensure high accuracy and fast
scalable and flexible, capable of adapting to different transcription.
classroom environments and technological setups. The use of
Raspberry Pi for both hardware and processing, combined  Real-Time Display and user Access
with modern web technologies and powerful transcription
APIs, provides a robust and reliable solution for enhancing  Transcription Display: The transcriptions generated by
classroom inclusivity and learning effectiveness. the Deepgram API are sent back to the Flask server,
which then forwards the text data to the front-end
IV. METHODOLOGY interface. Students connected to the router can access the
web app via the provided URL and view the live
The real-time audio transcription system is designed to transcriptions on their devices.
facilitate seamless access to classroom transcriptions via a
web application, ensuring that students can follow along  User Interaction: The front-end interface allows students
regardless of linguistic or auditory challenges. The to follow along with the live transcriptions as the teacher
methodology integrates various hardware and software speaks. This setup ensures that students who face
components to focus more on the content delivery in the difficulties in understanding spoken language due to
classroom rather than spending time dictating down the linguistic barriers, hearing impairments, or other
notes. challenges can access and comprehend the lecture content
in real-time.
 Audio Capture and Streaming
By utilizing the Raspberry Pi for both audio capture and
 Audio Capture: The system begins with the Raspberry Pi processing, and leveraging modern web technologies and
capturing the audio from the classroom using a robust transcription APIs, the system provides a
microphone. PyAudio is utilized for this purpose, comprehensive solution for enhancing classroom inclusivity.
providing an interface to the audio input and streaming The use of a local network setup ensures that all students in
the captured audio data. the classroom can easily access the web application, making
 Streaming Setup: The audio data captured by the this methodology both practical and effective for real-time
Raspberry Pi is converted into packets and streamed over educational support.

IJISRT24SEP1335 www.ijisrt.com 2330

Volume 9, Issue 9, September– 2024 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/IJISRT24SEP1335

V. RESULTS AND DISCUSSIONS  Discussions

The real-world application of the real-time audio
 Results transcription system involved a classroom scenario with ten
During the implementation and testing phase of the real- participants, including students and a teacher, focusing on a
time audio transcription system, it was observed that there science topic. The session lasted for ten minutes, during
was typically a three-second lag between the teacher's speech which the system's performance and user experience were
and the corresponding transcription displayed on the student's evaluated. Following the session, participants were queried
device. This delay, while noticeable, did not significantly about their experiences, providing valuable insights into the
impede students' ability to follow along with the lecture system's effectiveness and usability.
content. However, it was noted that under conditions of poor
network connectivity, the lag could increase, leading to a Of the ten participants, two students encountered
slightly delayed transcription response. network issues during the session. These issues resulted in
sporadic disruptions to the transcription service, leading to
Despite the presence of occasional lag, the system delays in receiving real-time transcriptions. While these
successfully transcribed the teacher's speech in real-time, interruptions were disruptive, they underscored the
allowing students to keep pace with the lecture. Feedback importance of robust network infrastructure in supporting the
from users indicated that the transcriptions provided by the seamless operation of real-time transcription applications.
app were accurate and reliable, enabling students to Additionally, they highlighted the need for contingency
effectively comprehend the spoken content. This was measures to address network instability and ensure
particularly beneficial for students who faced challenges in uninterrupted access to educational content.
understanding spoken language due to linguistic barriers or
hearing impairments. Despite these challenges, the majority of participants,
comprising eight students and the teacher, reported a positive
Additionally, the system's ability to generate real-time experience with the real-time transcription system. They
transcriptions facilitated active engagement and participation found the transcriptions to be accurate and reliable,
among students. By providing a visual representation of the facilitating their understanding and engagement with the
spoken content, the app encouraged students to follow along classroom lecture. However, participants noted a slight lag of
attentively and actively contribute to classroom discussions. approximately three seconds between the teacher's speech
This enhanced level of engagement was reflected in and the corresponding transcription display. This lag, while
increased interaction between students and teachers, fostering noticeable, was deemed acceptable given the continuous
a more dynamic and collaborative learning environment. nature of the transcription service and did not significantly
hinder comprehension or learning.
Moreover, the system's performance under varying
network conditions highlighted the importance of robust The feedback from participants underscores the
network infrastructure in supporting real-time transcription potential of real-time audio transcription technology to
applications. While the three-second lag was manageable in enhance the learning experience in educational settings. By
most cases, instances of network instability resulted in providing students with accessible and inclusive access to
increased transcription delay, impacting the user experience. lecture content, regardless of linguistic or auditory
Future iterations of the system will focus on optimizing challenges, the system promotes active engagement and
network connectivity and implementing buffering participation in classroom discussions. Moreover, the
mechanisms to mitigate the effects of network fluctuations. system's ability to adapt to varying network conditions
demonstrates its resilience and versatility in real-world
Furthermore, user feedback and analytics data provided environments.
valuable insights into the usability and effectiveness of the
system. Students reported a high degree of satisfaction with Moving forward, efforts to address network stability
the real-time transcription feature, noting its role in issues and optimize system performance will be paramount to
enhancing their understanding and retention of classroom maximizing the benefits of real-time transcription technology
content. Teachers also expressed enthusiasm for the system, in education. This includes implementing measures to
highlighting its potential to accommodate diverse learning mitigate network disruptions, improving buffering
needs and foster inclusive classroom environments. mechanisms to minimize transcription delays, and enhancing
user interfaces for seamless navigation and interaction.
Overall, the results demonstrated the effectiveness of Additionally, ongoing user feedback and iterative testing will
the real-time audio transcription system in supporting student be essential to refining the system and ensuring its continued
learning. Despite minor delays in transcription response time, effectiveness and relevance in supporting student learning
the system successfully met the needs of students by and academic success.
providing accurate and timely transcriptions of classroom
lectures. Moving forward, efforts to optimize network
connectivity, enhance user experience, and gather additional
feedback will further refine the system and ensure its
continued success in educational settings.

IJISRT24SEP1335 www.ijisrt.com 2331

Volume 9, Issue 9, September– 2024 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/IJISRT24SEP1335

VI. CONCLUSION [3]. IoT Application Development Based on Java and

Raspberry Pi by Zheng Lu and Xing Liu, 2021 IEEE
In this comprehensive exploration of real-time audio 12th Annual Information Technology, Electronics and
transcription technology in educational settings, we have Mobile Communication Conference (IEMCON).
delved into various aspects of its implementation, [4]. Vietnamese Voice2Text: A Web Application for
functionality, and impact. Beginning with the introduction, Whisper Implementation in Vietnamese Automatic
we identified the pressing need to address comprehension Speech Recognition Tasks” by Quangphuoc Nguyen,
gaps among students, particularly those facing linguistic Ngocminh Nguyen, Thanhluan Dang, and Vanha
barriers and cognitive challenges. The proposed solution Tran.
involved integrating a real-time audio transcription system, [5]. The implementation of Speech to Text Conversion
leveraging Raspberry Pi, Python Flask, PyAudio, and the Using Hidden Markov Model by A. Elakkiya, K. Jaya
Deepgram API. Surya, Konduru Venkatesh, and S. Aakash.
[6]. The Developing of the System for Automatic Audio to
Through meticulous methodology, we outlined the Text Conversion Oleh Basystiuk, Natalya
systematic approach to implementing and testing the system, Shakhovska, Violetta Bilynska, Oleksij Syvokon,
emphasizing its usability, effectiveness, and adaptability in Oleksii Shamuratov, Volodymyr Kuchkovskiy.
classroom environments. The front-end interface, powered by [7]. Self-Supervised Audio-and-Text Pre-training with
HTML, CSS, and JavaScript, provided students with intuitive Extremely Low-Resource Parallel Data Authors Yu
access to live transcriptions and additional learning Kang TAL Education Group, Tianqiao Liu TAL
resources. Meanwhile, the back-end infrastructure, facilitated Education Group, Hang Li TAL Education Group,
by Python Flask, PyAudio, and the Deepgram API, enabled Yang Hao TAL Education Group, Wenbiao Ding
seamless audio capture, streaming, transcription, and display. TAL Education Group, Tencent.
[8]. Different Methods Review for Speech to Text and
In the discussion, we examined the practical application Text to Speech Conversion by Deep Kothadiya Post
of the system in a classroom scenario, where participants Graduate Student MIT, Nitin Pise, PhD Professor
engaged in a ten-minute session focused on a science topic. MIT, Mangesh Bedekar Professor MIT, International
Feedback from participants highlighted both the successes Journal of Computer Applications September 2020.
and challenges of the system. While network issues affected [9]. Speech to Text Translation Enabling Multilingualism
some users, most found the transcriptions accurate and by Shahana Bano, Pavuluri Jithendra, Gorsa Lakshmi
beneficial, despite minor delays. This underscores the Niharika and Yalavarthi Sikhi - Department Of CSE,
potential of real-time transcription technology to enhance Koneru Lakshmaiah Education Foundation,
classroom inclusivity and student engagement. Vaddeswaram, India - 2020 IEEE International
Conference for Innovation in Technology (INOCON).
In conclusion, our exploration of real-time audio
transcription technology has revealed its significant potential
to revolutionize education by providing accessible and
inclusive access to lecture content. While challenges such as
network stability and transcription delays exist, ongoing
efforts to optimize system performance and gather user
feedback will be crucial in realizing the full benefits of this
innovative educational tool. By leveraging modern
technology and pedagogical insights, we can create more
inclusive and effective learning environments, ensuring that
all students have the opportunity to thrive and succeed
academically.

REFERENCES

[1]. Text to Speech Conversion using Raspberry - PI

Vinaya Phutak, Richa Kamble, Sharmila Gore, Minal
Alave, R.R.Kulkarni Department of Electronics and
Telecom- munication Engineering
[2]. “Internet of things applications using Raspberry-Pi:
asurveyKhalid M. Hosny1, Amal Magdi1, Ahmad
Salah2, Osama El-Komy1,Nabil A.
Lashin11Department of Infor- mation Technology,
Zagazig University, Zagazig, Egypt 2Department of
Computer Science, Zagazig University, Zagazig,
Egypt.

IJISRT24SEP1335 www.ijisrt.com 2332