Scribd Logo
Upload

Welcome to Scribd!

  • 5 views

    Hadoop Is A Framework That Is Widely Used For Storing and Managing Big Data

    Uploaded by

    Seid Hussen

    Copyright:

    © All Rights Reserved
    Download as DOCX, PDF, TXT or read online from Scribd

    Hadoop Is A Framework That Is Widely Used For Storing and Managing Big Data

    Uploaded by

    Seid Hussen
    5 views2 pages

    Original Title

    Hadoop is a Framework That is Widely Used for Storing and Managing Big Data

    Copyright

    © © All Rights Reserved

    Available Formats

    DOCX, PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Copyright:

    © All Rights Reserved
    Download as DOCX, PDF, TXT or read online from Scribd
    Download as docx, pdf, or txt
    5 views2 pages

    Hadoop Is A Framework That Is Widely Used For Storing and Managing Big Data

    Uploaded by

    Seid Hussen

    Copyright:

    © All Rights Reserved
    Download as DOCX, PDF, TXT or read online from Scribd
    Download as docx, pdf, or txt
    You are on page 1of 2

    Hadoop is a framework that is widely used for storing and managing big data.

    It consists of
    several components that work together to provide a comprehensive solution for handling
    large datasets.

    HADOOP, or High Availability Distributed Object-Oriented Platform, is an open source,


    Java-based software platform that manages data processing and storage for big data
    applications (refer Data Bricks Glossary — HADOOP). Instead of using one large computer
    to store and process the data, Hadoop allows clustering multiple computers to analyze
    massive datasets in parallel more quickly (refer AWS — What Is HADOOP).

    Here are the main components of the Hadoop ecosystem:

    Hadoop Distributed File System (HDFS): HDFS is the primary storage component of
    Hadoop. It is designed to store large datasets across multiple nodes in a distributed manner.
    HDFS divides the data into blocks and replicates them across different nodes for fault
    tolerance and high availability.

    MapReduce: MapReduce is a programming model and processing framework in Hadoop. It


    allows for distributed processing of large datasets by dividing the work into map and reduce
    tasks. The map tasks process the data in parallel, and the reduce tasks aggregate the results to
    produce the final output.

    Yet Another Resource Negotiator (YARN): YARN is the resource management component
    of Hadoop. It manages and allocates resources to different applications running on the
    Hadoop cluster. YARN enables efficient utilization of cluster resources and supports various
    types of processing frameworks, including MapReduce, Spark, and others.

    Apache Spark: Spark is a fast and general-purpose data processing engine that is often used
    in conjunction with Hadoop. It provides in-memory processing capabilities, making it
    suitable for real-time and iterative data processing tasks. Spark can be used for various data
    processing tasks, including batch processing, machine learning, and stream processing.

    Apache Hive: Hive is a data warehousing and SQL-like query language for Hadoop. It
    provides a high-level interface for querying and analyzing data stored in Hadoop. Hive
    translates SQL-like queries into MapReduce or Spark jobs, allowing users to leverage their
    SQL skills for big data analysis.
    Apache Pig: Pig is a high-level scripting language for data analysis and processing in
    Hadoop. It provides a platform for expressing data transformations and complex workflows.
    Pig scripts are translated into MapReduce or Tez jobs, enabling users to perform data
    processing tasks without writing low-level code.

    Apache HBase: HBase is a NoSQL database that runs on top of Hadoop. It provides random
    access to large amounts of structured and semi-structured data. HBase is suitable for real-time
    read and write operations and is often used for applications that require low-latency data
    access.

    These are some of the key components of the Hadoop ecosystem. Each component plays a
    specific role in enabling the storage, processing, and analysis of big data in a distributed and
    scalable manner.

    You might also like