Newres 5
Newres 5
Nsie Erimola María Reina Agripina*, Hua Shen, Blessed Shinga Mafukidze
Subject Areas
Information Management, Machine Learning
Keywords
Federated Learning, Decentralized Technology, Machine Learning, Data
1. Introduction
Federated learning has emerged as a transformative paradigm in machine learn-
ing, offering a decentralized approach to collaboratively train models across dis-
tributed data sources while preserving data privacy and security. This innovative
as well as network failures and adversarial attacks that could compromise model
privacy and integrity. To address these challenges, researchers are exploring novel
solutions, such as decentralized learning scheduling algorithms, adaptive feder-
ated optimizers, and secure federated learning frameworks with differential pri-
vacy guarantees He (2020) [2]. Recent developments in FL research have focused
on expanding the applicability of federated learning to new domains, such as
healthcare, financial services, and IoT networks, where data privacy and regula-
tory compliance are paramount. Collaborative efforts between academia, indus-
try, and policymakers are driving the development of standards and best practices
for federated learning deployment in real-world settings. Additionally, the inte-
gration of edge computing capabilities and federated learning technologies is pav-
ing the way for decentralized AI applications that deliver personalized services
and insights at the network edge Yang (2019) [3].
Federated learning continues to evolve as a transformative paradigm for collab-
orative and privacy-preserving machine learning. Advances in algorithms, chal-
lenges in data heterogeneity and security, and recent developments in new appli-
cation domains are shaping the landscape of FL research and innovation, laying
the foundation for the next generation of decentralized AI systems.
various training data with the same properties, is used in the initial FL Google
keyboard use case. This is illustrated in Figure 2.
Figure 2. Shaheen et al. (2024) [6], Application of Federated learning: Taxonomy, Challenges, Research
Trends.
is trained locally on each device using its respective data, and model updates are
then aggregated and shared among devices. This allows for personalized model
training while benefiting from the diversity of data from various devices. By shar-
ing only model updates and not raw data, privacy and security are maintained.
For example, consider a user who uses a smartphone, tablet, and laptop for various
tasks such as web browsing, email, and social media. These devices can collaborate
through Cross-Device Federated Learning to train a personalized, predictive text
model that adapts to the user’s typing patterns and language preferences across all
devices.
Figure 3. Farooq et al. (2023) [7], an example application of federated learning for the task of next-word prediction
on mobile phones.
be, this approach frequently has high communication costs and a slow conver-
gence rate. Sketched algorithms are also naturally suited for FL because original
data must be traced back through additional techniques and data IDs are not
maintained in order to strengthen confidentially, Smith and Sekar (2019) [12]
built a link between FL and the sketching algorithm.
friendly and can grow easily. Now, it can be used in many different situations
where bandwidth & computing strength are low. This includes things like smart
gadgets, mobile apps, or industrial systems that are spread out.
In healthcare, federated learning (FL) holds great promise. It encourages team-
work among different institutions without putting patient privacy at risk. For in-
stance, think about several hospitals or medical centers teaming up to create
predictive models for diagnosing diseases or suggesting treatments. They can
do this without swapping any patient records. Keeping the data within their
own walls allows healthcare providers to tap into bigger datasets while making
sure sensitive medical info stays protected. This approach is super handy in
medical imaging. With FL, diagnostic models can be trained using data col-
lected from various healthcare spots. This boosts accuracy and outcomes while
safeguarding privacy.
In finance, FL is a really helpful tool for managing risk & catching fraud. Banks,
credit institutions, and financial service providers can work together to train ma-
chine learning models for things like credit scoring or figuring out loan risks. They
do this without needing to share any customer info. This way, they can combine
their knowledge & make their models better while still following strict financial
rules. By allowing secure teamwork, FL makes decision-making better in fi-
nance and helps drive innovation without putting individual client privacy at
risk.
The IoT sector is another place were FL shines brightly. IoT gadgets like sen-
sors, wearables, and smart home devices create tons of data. A lot of it is to spread
out or sensitive to just put in one place. FL lets these devices train models locally
and only share updates about those models. This means it’s perfect for making
devices work better while keeping data private. For instance, in smart homes, FL
can help save energy or enhance personalized services without sending user data
to a central hub. In industrial IoT, FL can be useful for predictive maintenance.
Here, sensors on machines team up to forecast equipment failures, boosting effi-
ciency & cutting down downtime. Federated Learning (FL) mixed with edge com-
puting makes it easier to work in situations where quick decisions are needed. In
edge computing, devices can do their work right there, without always needing to
connect to a main server. This is super important for things like self-driving cars,
which need to analyze data in real-time.
With FL, these cars can get better at making choices. They can train their mod-
els using local data like sensor info or traffic updates without putting sensitive data
at risk. Keeping that info safe is a big deal. FL is a practical answer for many fields
like healthcare, finance, the Internet of Things (IoT), and self-driving cars. Its way
of handling data helps keep it secure while still allowing for teamwork in develop-
ing models. Plus, it tackles big issues like privacy, how well systems talk to each
other, and the heavy lifting of computations. As research moves forward, we’ll
likely see FL grow in different areas. It’s becoming a must-have tech for businesses
that care about privacy and want to use their data smartly.
back to a main server. This speedy integration boosts training times and cuts down
on delays—a huge bonus for real-time applications, like self-driving cars or ana-
lyzing data instantly in IoT systems.
enabling organizations to train models across different areas without violating le-
gal obligations. This is particularly advantageous for sectors like healthcare and
finance, where data privacy is a regulatory imperative.
FL also reduces the risk linked to single points of failure, which represent a sig-
nificant vulnerability in centralized ML systems. In conventional setups, a breach
of the central server can expose all the stored data. In FL, data is distributed
over multiple nodes, so even if one participant is compromised, the breach is
confined and limited. This greatly lowers the overall risk of a large-scale data
breach.
models and send updates to a central server for aggregation. This decentralized
framework makes it possible for malicious entities to disrupt the model training
process by submitting erroneous, poisoned, or manipulated updates.
Malicious nodes can carry out various types of attacks, including:
Data Poisoning: A malicious node can insert biased or false data into the train-
ing pipeline, altering the local model updates sent to the server. This could lead
to the global model converging inaccurately, resulting in flawed or prejudiced
predictions. For example, in a financial modeling context, a malicious partici-
pant might introduce fraudulent data that skews the model towards favorable
predictions for certain behaviors.
Model Poisoning: Rather than tampering with the data itself, a harmful node
can modify the local model parameters before submitting them to the server,
thereby injecting detrimental updates into the global model. This action can
compromise the integrity of the overall model and potentially lead to wide-
spread system failures.
Backdoor Attacks: Malicious participants can also embed backdoors within
the global model, which may remain inactive until a specific trigger occurs.
This poses significant security risks since attackers can activate these back-
doors after the model is deployed, causing the model to operate unpredictably
or harmfully.
While secure aggregation techniques and differential privacy are commonly
employed to alleviate some of these risks, the challenges remain considerable.
Cryptographic strategies like secure multi-party computation can help ensure that
model updates are kept private and unaltered, but they can be resource-intensive
and may introduce latency. Furthermore, identifying and eliminating malicious
updates in a decentralized environment—without direct access to original data—
continues to be a challenge, as current methods might inadvertently dismiss legit-
imate updates or fail to detect sophisticated attacks.
(non-IID) nature of data across multiple clients. Different devices typically gather
data that varies in distribution, which can lead to biased models if not appropri-
ately managed. To address this, researchers are investigating personalized feder-
ated learning, which involves developing customized models tailored to specific
groups of clients with similar data distributions or behaviors, rather than creating
a single global model. This method enables FL to provide more relevant models
to individual clients while still leveraging shared learning across the network. An-
other proposed solution is federated meta-learning, or “learning to learn,” de-
signed to enhance the global model’s adaptability to diverse data distributions.
This could help the global model quickly adjust to new trends and variations in
client data. Additionally, the creation of dynamic aggregation strategies can im-
prove FL’s efficiency by weighting updates according to data quality or represent-
ativeness instead of treating all updates uniformly as in current methods like Fed-
erated Averaging.
10. Conclusions
Federated learning is a novel approach in machine learning that allows decentral-
ized model training while preserving privacy and security of users’ data. By over-
coming inherent drawbacks of centralized learning, such as data aggregation, FL
provides a groundbreaking solution to privacy, regulatory compliance, and secu-
rity problems. Its benefit is the capacity to train models across devices or servers
without moving sensitive data, which appeals to areas with high privacy require-
ments, such as healthcare, finance, or the Internet of Things.
While federated learning has great potential, there are several challenges that
need to be overcome to unlock it fully. Firstly, one of the more critical challenges
is heterogeneous data, indicated by the varying differences in data distribution
across devices. This leads to problems with the generalization of global models
and most often results in the local model being biased. Second, there are signifi-
cant communication/computation costs that are also critical in the context of sys-
tems that depend on a high number of resource-constrained devices. Their limited
resources often create problems both in terms of communication and data pro-
cessing requirements. The necessity to frequently send updates to the central
server may lead to overextending the devices’ processing capabilities, while the
deficiencies in computational power may lead to certain limitations on the com-
plexity of the models that can be developed and deployed.
Besides, security concerns are of particular importance in FL. Since the FL set-
ting is decentralized, the system is subject to adversarial attacks, such as data or
model poisoning, whereby malicious participants corrupt the aggregation process
to mislead the central server about the true global model. The resilience of FL
systems against such threats and application of secure aggregation techniques,
which guarantee that security and privacy of the training data are maintained, are
much required for the safe and reliable usage of the approach.
As a preliminary, it must be mentioned that to overcome the outlined chal-
lenges, further research should be concerted around several core aspects. In par-
ticular, the considerable extension of the existing federated optimization tech-
niques and the dynamic aggregation strategies would help to solve the problems
of data heterogeneity and enhance the model’s performance. Moreover, more ef-
fective privacy-preserving mechanisms, such as homomorphic encryption and
differential privacy, should be developed to ensure the optimal level of privacy and
high model accuracy. Lastly, more efficient communication protocols, such as
model compression, and more effective approaches to asynchronous updates,
could make the FL more scalable and less resource-consuming for larger deploy-
ments.
To conclude, federated learning presents a viable solution to collaborative ma-
chine learning in privacy-preserving and distributed settings. As the field ad-
vances, overcoming some of its existing constraints will allow it to become a sig-
nificant part of future AI-driven applications, enabling industries to harness their
data’s potential across the board without compromising security or privacy. Fur-
ther progression in the area will be allowed by academia, industry, and policymak-
ers alike, promoting the synthesis of decentralized AI systems in a variety of fields.
Conflicts of Interest
The author declares no conflicts of interest.
References
[1] McMahan, H.B. and Moore, E. (2016) Communication-Efficient Learning of Deep
Networks from Decentralized Data. Proceedings of the 20th International Confer-
ence on Artificial Intelligence and Statistics (AISTATS), Fort Lauderdale, 1273-1282.
[2] He, K., Zhang, Y., Ren, S. and Sun, J. (2020) Securing Federated Learning against
Malicious Clients. Proceedings of the IEEE Symposium on Security and Privacy
(S&P), 1-16.
[3] Yang, Q., Liu, Y., Tian, Z., Yu, S. and Chen, K. (2020) Federated Learning: Challenges,
Methods, and Future Directions. IEEE Transactions on Parallel and Distributed Sys-
tems, 30, 1799-1819.
[4] Ho, Q.B., Cichocki, A. and Hong, T.P. (2013) Learning Processes in Decentralized
Collaborative Working Environments. International Conference on Industrial Tech-
nology, 793-798.
[5] Beltrán, E.T.M., Pérez, M.Q., Sánchez, P.M.S., Bernal, S.L., Bovet, G., Pérez, M.G.,
Pérez, G.M. and Celdrán, A.H. (2023) Decentralized Federated Learning: Fundamen-
tals, State of the Art, Frameworks, Trends, and Challenges. IEEE Communications
Surveys & Tutorials, 25, 2983-3013. https://doi.org/10.1109/COMST.2023.3315746
[6] Shaheen, M., Farooq, M.S. and Umer, T. (2024) AI-Empowered Mobile Edge Compu-
ting: Inducing Balanced Federated Learning Strategy over Edge for Balanced Data
and Optimized Computation Cost. Journal of Cloud Computing, 13, Article No. 52.
https://doi.org/10.1186/s13677-024-00614-y
[7] Farooq, U., Naseem, S., Li, J., Mahmood, T., Rehman, A., Saba, T. and Mustafa, L.
(2023) Analysis of the Factors Influencing the Predictive Learning Performance Us-
ing Federated Learning. Preprint. https://doi.org/10.21203/rs.3.rs-3243194/v1
[8] Mothukuri, V., Parizi, R.M., Pouriyeh, S., Huang, Y., Dehghantanha, A. and Sri-
vastava, G. (2021) A Survey on Security and Privacy of Federated Learning. Future
Generation Computer Systems, 115, 619-640.
https://doi.org/10.1016/j.future.2020.10.007
[9] Pasquini, C. and Böhme, R. (2020) Trembling Triggers: Exploring the Sensitivity of
Backdoors in DNN-Based Face Recognition. EURASIP Journal on Information Se-
curity, 2020, Article No. 12. https://doi.org/10.1186/s13635-020-00104-z
[10] Truex, S., Elsabrouty, M., Mhamdi, L., Felber, P. and Raynal, M. (2019) Hybrid-One:
Enhancing Privacy and Utility in Federated Learning with Hybrid-Privacy Strategies.
IEEE Transactions on Parallel and Distributed Systems, 31, 1736-1749.
[11] Smith, J. and Sekar, R. (2019) An Analysis of Fuzzy Logic Applications in Industrial
Engineering: Current Use and Future Prospects. Journal of Industrial Engineering
Research, 15, 45-58.
[12] Smith, G. and Sekar, V. (2019) Federated Learning: A Privacy-Preserving Collabora-
tive Machine Learning Framework. Proceedings of the ACM International Confer-
ence on Knowledge Discovery and Data Mining (KDD), 78-87.
[13] Nasr, M., Shokri, R. and Houmansadr, A. (2019) Comprehensive Privacy Analysis of
Deep Learning: Passive and Active White-Box Inference Attacks against Centralized
and Federated Learning. 2019 IEEE Symposium on Security and Privacy (SP), San
Francisco, 19-23 May 2019, 739-753. https://doi.org/10.1109/sp.2019.00065
[14] Cai, J. and Venkatasubramanian, K. (2018) Anomaly Detection for Wearable Health
Monitoring Systems: A K-Nearest Neighbor Model with Kernel Density Estimation.
Proceedings of the 2018 IEEE International Conference on Big Data (Big Data), 4488-
4493.