Scribd Logo
Upload

Welcome to Scribd!

  • 90 views

    البيانات الضخمة Big Data

    Uploaded by

    laila
    This document discusses big data and how it is processed using Hadoop Distributed File System (HDFS). It defines big data as data that is too large to process and store on a single machine. It describes the 5 V's of big data: volume, variety, velocity, value, and veracity. It then explains how HDFS works by breaking files into blocks and storing replicas across clusters to allow for reliable access. A few applications of big data are also provided like Netflix recommendations, driverless cars, education, and government/political analysis.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    البيانات الضخمة Big Data

    Uploaded by

    laila
    90 views30 pages
    This document discusses big data and how it is processed using Hadoop Distributed File System (HDFS). It defines big data as data that is too large to process and store on a single machine. It describes the 5 V's of big data: volume, variety, velocity, value, and veracity. It then explains how HDFS works by breaking files into blocks and storing replicas across clusters to allow for reliable access. A few applications of big data are also provided like Netflix recommendations, driverless cars, education, and government/political analysis.

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    This document discusses big data and how it is processed using Hadoop Distributed File System (HDFS). It defines big data as data that is too large to process and store on a single machine. It describes the 5 V's of big data: volume, variety, velocity, value, and veracity. It then explains how HDFS works by breaking files into blocks and storing replicas across clusters to allow for reliable access. A few applications of big data are also provided like Netflix recommendations, driverless cars, education, and government/political analysis.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    90 views30 pages

    البيانات الضخمة Big Data

    Uploaded by

    laila
    This document discusses big data and how it is processed using Hadoop Distributed File System (HDFS). It defines big data as data that is too large to process and store on a single machine. It describes the 5 V's of big data: volume, variety, velocity, value, and veracity. It then explains how HDFS works by breaking files into blocks and storing replicas across clusters to allow for reliable access. A few applications of big data are also provided like Netflix recommendations, driverless cars, education, and government/political analysis.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 30

    BIG DATA ‫البيانات الضخمة‬

    BY ADIY TWEISSI
    EVOLUTION OF TECHNOLOGY
    IOT
    SOCIAL MEDIA
    WHAT IS BIG DATA?
    • When data is ‘Big’ that means we don’t have enough
    processing power + enough space to deal with it.
    THE 5 V’S OF BIG DATA
    VOLUME
    VARIETY
    VELOCITY
    VALUE
    VERACITY
    WE NEED TO PROCESS BIG DATA WITH:
    HDFS

    • Hadoop Distributed File System ( HDFS )


    HDFS

     Write Once, Read Many times (WORM)

     Divide files into big blocks and distribute across the cluster

     Store multiple replicas of each block for reliability

     Programs can ask "where do the pieces of my file live?”


    10110100101 Cluster
    00100111001
    1
    11111001010
    01110100101
    00101100100
    1 3 2
    10101001100
    01010010111
    2
    01011101011 4 1 3
    11011011010
    Blocks 10110100101
    01001010101
    3
    01011100100
    11010111010
    2 4
    0 4 2 3
    1
    4

    Logical File
    WHAT IS HDFS?

    “I have a 200 TB file that


    I need to store.”

    “Wow - that is big data! I will


    need to distribute that across
    a cluster.”

    “Sounds risky! What


    Hadoop Client happens if a drive fails?”
    HDFS

    “No need to worry! I am


    designed for failover.”
    APPLICATIONS
    • Entertainment: Netflix and Amazon use Big Data to make shows and movie recommendations to their users.
    • Insurance: Uses Big data to predict illness, accidents and price their products accordingly.
    • Driver-less Cars: Google’s driver-less cars collect about one gigabyte of data per second. These experiments
    require more and more data for their successful execution.
    • Education: Opting for big data powered technology as a learning tool instead of traditional lecture methods,
    which enhanced the learning of students as well aided the teacher to track their performance better.
    • Automobile: Rolls Royce has embraced Big Data by fitting hundreds of sensors into its engines and propulsion
    systems, which record every tiny detail about their operation. The changes in data in real-time are reported to
    engineers who will decide the best course of action such as scheduling maintenance or dispatching
    engineering teams should the problem require it.
    • Government: A very interesting use of Big Data is in the field of politics to analyse patterns and influence
    election results. Cambridge Analytica Ltd. is one such organisation which completely drives on data to change
    audience behavior and plays a major role in the electoral process.
    AMAZON +
    BOOKING WEBSITE
    THE CASE OF SOCIAL MEDIA
    • We’ve been involved in big data because of social media.
    • Review the social network report.
    PSYCHOGRAPHICS
    VS
    DEMOGRAPHICS
    PRACTICE ON FACEBOOK PAGE
    THE CASE OF
    E-LEARNING DATA

    • Simulations (Size of universe +


    universe game + circuit simulation)
    • Real-time data
    PRACTICE ON DATA SELFIE AND ALEXA
    INFOGRAM FOR VISUALIZATION

    • Examples
    ONS CASE

    • Traditional methodology involves a paper questionnaire and a team of door-


    knockers; in 2011 the Office for National Statistics (ONS) employed 35,000
    temporary field staff. Then they recognized only ONE team is required.
    ONS CASE
    MORE EXAMPLES:

    • Data Visualization In VR
    • Fugro GIS

    You might also like