Introduction to Data Analytics: Comprehen: 1. Sources and Nature of Data a. Sources of Data 1. Primary Data Suurces: © Data collected firsthand for a specific purpose. This includes: Surveys: Structured questionnaires used to collect opinions, preferences, and other types of informetion, Experiments: Data obtained through controlled experiments, often used in scientific research, Observations: Information collected through direct observation, like recording human behavior or environmental changes. Sensors: Data from devices like loT sensors, GPS, weather sensors, and biometric devices. 2. Secondary Data Sources: © Data that has already been collected and published by others, which can be used for analysis: b. Nature of Data Public Databases: Census data, World Bank data, and other publicly available datasets Research Reports: Published research findings, industry reports, and white papers. Web Data: Data scraped from websites, social media platforms, and online forums. Data Repositori Repositories like Kaggle, UCI Machine Leaming Repository, and Google Dataset Search. 1. Qualitative Data: © Descriptive data that cannot be measured numerically but provides deep insights into trends and patterns Notes Designed By Er. Gaurav Vishwa(CClO, NAVY SSB+) 1© Examples: Interview transcripts, user feedback, open-ended survey responses. © Analysis Methods: Thematic analysis, content analysis, and narrative analysis. 2. Quantitative Data: © Numerical data that can be measured and quantified, © Examples: Sales figures, temperature readings, and test scores. © Analysis Methods: Statistical anclysis, regression analysis, and machine learning models. 2. Classification of Data a. Structured Data: © Data that is organized in a fixed format, usually in rows and columns. © Storage: Relational databases like MySQL, PostgreSQL. © Examples: Financial Transactions: Transaction records stored in banking databases. © Customer Information: Details such as name, address, and purchase history in CRM systems. b. Semi-Structured Data: © Data that does not conform to a strict structure but contains tags or markers to separate elements. © Storage: NoSQL databases like MongoDB, Cassandra. © Cxamples © JSON and XML Files: Often used in web data interchange. © Email Contents: Contains structured data (headers) and unstructured data (body) c. Unstructured Data: © Data that has no predefined format or structure, making it complex to analyze. © Storage: Hadoop Distributed File System (HDFS), data lakes © Examples © Text Documents: Articles, reports, and books. Notes Designed By Er. Gaurav Vishwa(CClO, NAVY SSB+) 2© Multimedia Files: Images, videos, and audio recordings. © Social Media Posts: Tweets, Facedook posts, comments. 3. Characteristics of Data: 1. Volume: The sheer size of data being generated every second. Requires scalable storage and processing solutions, © Examples: Data generated by social media platforms, IoT devices, financial markets, 2. Velocity: ‘The speed at which new data is generated and needs to be processed. © Examples: Real-time analytics for stock prices, fraud detection systems in banking, 3. Variety: The different types and formats of data, including structured, semi-structured, and unstructured. © Examples: Customer reviews (text), video surveillance footage, transactional data. 4. Introduction to Big Data Platform a. Definition and Concept © Big Data refers to datasets that are too large, complex, or fast-moving to be processed using traditional data-processing techniques. It requires advanced tools and methods for storage, processing, and analysis. b. Big Data Characteristics: © 3 Vs Model: Volume: Huge amounts of data generated from various sources. Velocity: The rapid generation and flow of data. Variety: The diverse formats and types of data. Notes Designed By Er. Gaurav Vishwa(CClO, NAVY SSB+) 35. Need for Data Analytics a. Extract Actionable Insights: © Identify trends, patterns, and correlations that can inform business decisions, optimize processes, and improve customer experiences b. Support Decision-Making: © Data-driven decision-making helps businesses reduce uncertainty and make more informed choices. Predictive Capabilities: Predict future trends, customer behavior, and market changes to gain a competitive advantage. d. Opera nal Efficiency: © Improve operations by identifying bottlenecks and optimizing processes. 6. Evolution of Analytic Scalability: #19803: Data Warchousing Introduction of data warchouses, centralizing data for better reporting and analysis ¢1990s; OLAP Emergence of Online Analytical Processing (OLAP) tools for multidimensional analysis, enabling faster queries, ©2000s; Big Data Rise of big data technologies like Hadoop (2006) and Spark (2010), allowing istributed processing across clusters and supporting large datasets. 2010s: Cloud Computing, Machine Learning and Real-Time Analytics. 1©2020s: Automated Analytics, Privacy ievance. Notes Designed By Er. Gaurav Vishwa(CClO, NAVY SSB+) 47. Analytic Process and Tools a. Data Collection: © Gathering raw data from sources like databases, sensors, APIs, and web scraping. b. Data Cleaning: © Removing or correcting inaccuracies, duplicates, and missing values to ensure data quality. c. Data Integration: ‘© Combining data from different sources into a single, coherent dataset for analysis. d. Data Transformation: © Converting data into a format suitable for analysis, such as normalization, aggregation, and encoding categorical variables. e. Data Modeling: © Creating mathematical models to analyze relationships within the data © Techniques: Regression analysis, classification, clustering, and time-series analysis. © Presenting data in a visual format such as charts, graphs, and dashboards. © Tools: Tableau, Power BI, Matplotlib, D3 js. g. Popular Analytics Tools: © Data Processin; © Python (pandas, NumPy): Data manipulation and numerical analysis. © R: Statistical computing and graphics Notes Designed By Er. Gaurav Vishwa(CCIO, NAVY SSB+) 5© SAS: Advanced analytics, business intelligence, and data management. © Data Visualization: © Tableau, Power BI: Interactive dashboards and reports. © Matplotlib, Seaborn: Visualization libraries in Python. 8. Analysis vs Reporting a. Analysis: In-depth exploration of data to identify patterns, trends, and insights. © Focuses on answering 'why' and ‘how’ questions Techniques: Descriptive statistics, predictive modeling, clustering Tools: Python, R, machine learning libraries (Scikit-learn, TensorFlow). b. Reporting: © Presenting data in a summarized form for stakeholders to understand the outcomes. © Focuses on ‘what’ questions. © Techniques: Summary statistics, data visualization. Tools: Excel, Tableau, Power BI. 9. Applications of Data Analytics a. Business Intelligence: ‘© Enhances decision-making by providing comprehensive insights into business performance. © Use Cases: Sales forecasting, customer segmentation, performance analysis. b. Healthcare: ‘© Improves patient care through predictive analytics and personalized treatment plans © Use Cases: Disease prediction, patient monitoring, drug discovery. Notes Designed By Er. Gaurav Vishwa(CClO, NAVY SSB+) 6¢. Financ © Enables better risk management and fraud detection. © Use Cases: Credit scoring, fraud detection, investment strategy optimization d. Marketing: Allows for targeted marketing campaigns and customer engagement. © Use Cases: Customer segmentation, chur prediction, campaign analysis. e. Supply Chain Management: © Optimizes logistics, inventory management, and demand forecasting. © Use Cases: Route optimization, supply chain risk management, demand planning. f. Social Media Analytics: © Analyzes social media data to understand public sentiment and influence. © Use Cases: Sentiment analysis, trend identification, influencer marketing. 10. Key Roles for Successful Analytic Project : Business User : © The business user is the one who understands the main area of the project and is also basically benefited from the results ‘© This user gives advice and consult the team working on the project about the value of the results obtained and how the operations on the outputs are done. Notes Designed By Er. Gaurav Vishwa(CClO, NAVY SSB+) 7Project Sponsor : '* The Project Sponsor is the one who is responsible to initiate the project. Project Sponsor provides the actual requirements for the project and presents the basic business issue. Project Manager : ‘® This person ensures that the key milestone and purpose of the project is met on time and of the expected quality. Business Intelligence Analyst : ‘* Business Intelligence Analyst provides business domain perfection based on a detailed and deep understanding of the data, key performance indicators (KPIs), key ‘matrix, and business intelligence from a reporting point of view. Database Administrator (DBA) : © DBA facilitates and arrange the database environment to support the analytics need of the team working on a project. ‘© His responsibilities may include providing permission to key databases or tables and making sure that the appropriate security stages are in their correct places related to the data repositories or not. Data Engineer : The data engineer works jointly with the data scientist to help build data in correct ways for analysis. Notes Designed By Er. Gaurav Vishwa(CCIO, NAVY SSB+) 811- Various Phases of Data Analytics Lifecycle: Phase 1: Discovery — ‘© The data science team learns and investigates the problem ‘© Develop context and understanding. ‘© Come to know about data sources needed and available for the project. Phase 2: Data Preparation — © Steps to explore, preprocess, and condition data before modeling and analysis, © Data preparation tasks are likely to be performed multiple times and not in predefined order. Phase 3: Model Planning - © The team explores data to learn about relationships between variables and subsequently, selects key variables and the most suitable models. © In this phase, the data science team develops data sets for training, testing, and production purposes. Phase 4: Model Building ‘© Team develops datasets for testing, training, and production purposes. Phase 5: Communication Results — © After executing model team need to compare outcomes of modeling to criteria established for success and failure. Notes Designed By Er. Gaurav Vishwa(CClO, NAVY SSB+) 9© Team considers how best to articulate findings and outcomes to various team ‘members and stakeholders, taking into account warning, assumptions. Phase 6: Operationalize — © The team communicates benefits of project more broadly and sets up pilot project to deploy work in controlled way before broadet 1g the work to full enterprise of users. © This approach enables team to lear about performance and related constraints of the model in production environment on small scale which make adjustments before full deployment. © The team delivers final reports, briefings, codes. Data Analytics Lifecycle ned By Er. Gaurav Vishwa(CClO, NAVY SSB+) 10Exercise Short Answer Questions eRe eas an . What are primary data sources? Provide two examples. . What is the nature of qualitative data? . Define structured data with one example. . What does the 'Variety' characteristic of Big Data refer to? What is the purpose of data cleaning in the analytic process? Differentiate between analysis and reporting. . Name two popular data visualization tools. What role does a project sponsor play in an analytic project? . What is the first phase in the Data Analytics Lifecycle? Short Type Questions 1 SB ws w p Explain the differences between primary and secondary data sources with examples. Describe the characteristics of semi-structured data with examples, . How does real-time data velocity impact data analytics? Provide an example. . What are the 3 Vs of Big Data, and how do they influence data analysis? . What are the main steps involved in data preparation? . Discuss the key differences between structured, semi-structured, and unstructured data. How does data visualization help in data analytics? Mention two visualization techniques. . Describe the role ofa Business Intelligence Analyst in an analytic project. .. Explain the process of model building in the Data Analytics Lifecycle. Notes Designed By Er. Gaurav Vishwa(CClO, NAVY SSB+) "1Long Type Questions 1 Discuss the various sources of data in detail, including primary and secondary sources. How do these sources influence data analytics projects? Explain the classification of data into structured, semi-structured, and unstructured formats. Provide relevant examples and discuss the storage solutions for each type. . What are the main characteristics of Big Data (Volume, Velocity, Variety)? Discuss their significance and challenges in data processing and analysis. Elaborate on the need for data analytics in modem businesses. Discuss how data analytics supports decision-making, predictive capabilities, and operational efficiency. Trace the evolution of analytic scalability from the 1980s to the present, How have advancements in technology, such as Big Data platforms and cloud computing, impacted data analytics? Describe the analytic process and tools used in data analytics, from data collection to data visualization. Highlight the importance of each step in obtaining actionable insights. Compare and contrast analysis and reporting in the context of data analytics. What tools and techniques are used for each, and what are their respective purposes? Discuss the applications of data analytics in various sectors, including business intelligence, healthcare, finance, and marketing. Provide specific use cases for each sector. Describe the key roles required for a successful analytic project. What are the responsibilities of each role, and how do they contribute to the project’s success? Notes Designed By Er. Gaurav Vishwa(CCIO, NAVY SSB+) 1210.Outline the phases of the Data Analytics Lifecycle. Explain the significance of each phase and how they contribute to the overall success of data analytics projects. Notes Designed By Er. Gaurav Vishwa(CCIO, NAVY SSB+) 13