Redundancy Avoidance for Big Data in Data Centers: A Conventional Neural Network

Approach

Abstract:

As the innovative data collection technologies are applying to every aspect of our society, the data
volume is skyrocketing. Such phenomenon poses tremendous challenges to data centers with respect to
enabling storage. In this paper, a hybrid-stream big data analytics model is proposed to perform
multimedia big data analysis. This model contains four procedures, i.e., data pre-processing, data
classification, data recognition and data load reduction. Specifically, an innovative multi-dimensional
Convolution Neural Network (CNN) is proposed to assess the importance of each video frame. Thus,
those unimportant frames can be dropped by a reliable decision-making algorithm. In order to ensure
video quality, minimal correlation and minimal redundancy (MCMR) are combined to optimize the
decision-making algorithm. Simulation results show that the amount of processed video is significantly
reduced, and the quality of video is preserved due to the addition of MCMR. The simulation also proves
that the proposed model performs steadily and is robust enough to scale up to accommodate the big data
crush in data centers.

Existing System:

A strategy that could reduce the transmission load among data centers and leverage the storage system is
required to address the above problems. Some studies on reducing the transmission load have focused on
optimizing route selection as well as detecting and dropping anomaly traffic. Vertical handoff (VHO)
decision algorithm performs well in heterogeneous wireless networks [13], but when the time dimension
is considered in multimedia transmission and storage, VHO becomes extremely complex. Convolution
Neural Network, which incorporates pooling, can improve generalization on pattern recognition problems
by sharing weights and biases. Hybrid Convolutional Neural Network (HCNN) combines CNN and
winner-takesall mechanism to further boost the recognition speed [14].

Proposed System:

This paper focuses on finding a proper proportion of reduced redundancy data, which reduces important
data loss. Specifically, four main steps are presented, including Video Pre-processing, Frame
Classification, Frame-Load-Reduction Processing, and Video Decision. Pre-processing provides an
intermediate layer to adapt input video streams to our model. The purpose of Frame Classification is to
evaluate the significance of each frame. Simultaneously, Frame-Load-Reduction Processing and Video
Decision are proposed to perform data redundancy avoidance and ensure vital videos are not dropped.
Generally, our work focuses on enabling real-time video processing by enhancing the storage efficiency
in terms of storing useful data, and by further relieving multimedia data redundancy in data centers.

CONCLUSIONS:

In this paper, a hybrid-stream big data analytics model has been proposed to enhance the classification
precision and relieve the data centers’ network and storage overload. The model can improve the speed to
deal with the videos and recognizing, deciding the important frames and whether to drop the unimportant
ones in every video. Compared to conventional methods like deep learning to address image analysis
problems, this paper has improved the method to deal with video analysis. Besides, this network and
storage overload problem of video is considered as an optimization problem, which can show a practical
algorithm over a largescale of real-time data from numerous nodes. The conducted simulations represent
that our model performs well in most of the data sets. Moreover, the hybrid-stream big data analytics
model and the improved video with recognized algorithm can lead to a fairly good video stream and save
storage space in the Internet of Things. Our algorithm also provides a way to relieve the network and
storage load. The model can reduce network and storage overload, and it will not destroy the truly
important videos as well.

REFERENCES:

[1] J. Wu, I. Bisio, C. Gniady, E. Hossain, M. Valla, and H. Li, “Contextaware networking and
communications: Part 1 [guest editorial],” IEEE Communications Magazine, vol. 52, no. 6, pp. 14–15,
June 2014.

[2] X. He, K. Wang, H. Huang, and B. Liu, “Qoe-driven big data architecture for smart city,” IEEE
Communications Magazine, vol. 56, no. 2, pp. 88–93, Feb 2018.

[3] C. Ge, Z. Sun, N. Wang, K. Xu, and J. Wu, “Energy management in cross-domain content delivery
networks: A theoretical perspective,” IEEE Transactions on Network and Service Management, vol. 11,
no. 3, pp. 264–277, Sept 2014.

[4] K. Wang, Y. Shao, L. Shu, C. Zhu, and Y. Zhang, “Mobile big data fault-tolerant processing for
ehealth networks,” IEEE Network, vol. 30, no. 1, pp. 36–42, January 2016.

[5] H. Jiang, K. Wang, Y. Wang, M. Gao, and Y. Zhang, “Energy big data: A survey,” IEEE Access, vol.
4, pp. 3844–3861, 2016.

[6] L. Gu, D. Zeng, S. Guo, Y. Xiang, and J. Hu, “A general communication cost optimization framework
for big data stream processing in geo-distributed data centers,” IEEE Transactions on Computers, vol. 65,
no. 1, pp. 19–29, Jan 2016.

[7] Z. B. Liu, “The realization of virtual storage networking based on asymmetric architecture,”
Computer Science, vol. 31, no. 6, pp. 52–55, 2004.

[8] C. F. Lai, Y. X. Lai, M. S. Wang, and J. W. Niu, “An adaptive energyefficient stream decoding
system for cloud multimedia network on multicore architectures,” IEEE Systems Journal, vol. 8, no. 1,
pp. 194–201, March 2014.

[9] J. Wu, S. Guo, J. Li, and D. Zeng, “Big data meet green challenges: Big data toward green
applications,” IEEE Systems Journal, vol. 10, no. 3, pp. 888–900, Sept 2016.

[10] K. Wang, Y. Wang, X. Hu, Y. Sun, D. J. Deng, A. Vinel, and Y. Zhang, “Wireless big data
computing in smart grid,” IEEE Wireless Communications, vol. 24, no. 2, pp. 58–64, April 2017.

[11] K. Wang, H. Li, Y. Feng, and G. Tian, “Big data analytics for system stability evaluation strategy in
the energy internet,” IEEE Transactions on Industrial Informatics, vol. 13, no. 4, pp. 1969–1978, Aug
2017.
[12] H. Liu, S. Liu, X. Meng, C. Yang, and Y. Zhang, “Lbvs: A load balancing strategy for virtual
storage,” in 2010 International Conference on Service Sciences, May 2010, pp. 257–262.

[13] S. Lee, K. Sriram, K. Kim, Y. H. Kim, and N. Golmie, “Vertical handoff decision algorithms for
providing optimized performance in heterogeneous wireless networks,” IEEE Transactions on Vehicular
Technology, vol. 58, no. 2, pp. 865–881, Feb 2009.

[14] K. Wang, J. Mi, C. Xu, Q. Zhu, L. Shu, and D.-J. Deng, “Real-time load reduction in multimedia big
data for mobile internet,” ACM Trans. Multimedia Comput. Commun. Appl., vol. 12, no. 5s, pp. 76:1–
76:20, oct 2016. [Online]. Available: http://doi.acm.org/10.1145/2990473

[15] K. Simonyan and A. Zisserman, “Two-stream convolutional networks for action recognition in
videos,” in Proceedings of the 27th International Conference on Neural Information Processing Systems,
ser. NIPS’14. Cambridge, MA, USA: MIT Press, 2014, pp. 568–576. [Online]. Available:
http://dl.acm.org/citation.cfm?id=2968826.2968890

[16] H. Ye, Z. Wu, R.-W. Zhao, X. Wang, Y.-G. Jiang, and X. Xue, “Evaluating two-stream cnn for
video classification,” in Proceedings of the 5th ACM on International Conference on Multimedia
Retrieval, ser. ICMR ’15. New York, NY, USA: ACM, 2015, pp. 435–442. [Online]. Available:
http://doi.acm.org/10.1145/2671188.2749406

[17] B. Prabavathy, K. Priya, and C. Babu, “A load balancing algorithm for private cloud storage,” in
2013 Fourth International Conference on Computing, Communications and Networking Technologies
(ICCCNT), July 2013, pp. 1–6.

[18] L. Zhou, Y. C. Wang, J. L. Zhang, J. Wan, and Y. J. Ren, “Optimize block-level cloud storage
system with load-balance strategy,” in 2012 IEEE 26th International Parallel and Distributed Processing
Symposium Workshops PhD Forum, May 2012, pp. 2162–2167.

[19] M. Baktashmotlagh, M. Harandi, B. C. Lovell, and M. Salzmann, “Discriminative non-linear

[20] X. He, K. Wang, H. Huang, T. Miyazaki, Y. Wang, and S. Guo, “Green resource allocation based on
deep reinforcement learning in content-centric iot,” IEEE Transactions on Emerging Topics in
Computing, pp. 1–1, 2018.