5105-BHARATHIDASAN ENGINEERING COLLEGE NATTRAMPALLI (CS3352-FOUNDATIONS OF DATA SCIENCE UNIT-4 INTRODUCTION «Define Data Science and Big Data.(NoviDec 2022) List an overview of common errors in retrieving data and which cleansing solutions to be employed. (Nov/Dec 2022) . Define Data Science Process. . Write any 4 benefits of data science. . Write any 4 facets of data. . Define research goal. Write the uses of data science. . What is data retrieval? . How will you prepare the data? 10. Mention the 6 phases of Data Science Process. 11.What is Data Cleansing? 12, What is Data Transformation? 13.What is Data Integration? 14. Write example for Joining and Appending the data. 15. Define Data Exploration. 16.Write about building the model 17.What is Data Mining? 18. Define Data Warehousing. 19.What are the statistical methods used? 20.What are the skills required for Data Scientist? 21.Write the role of Data Scientist and define Statistics. 22. What is KDD? 23. Mention the common types of errors occur during data preparation. 24. Mention any 5 applications of Data Mining. 25. Mention any 5 applications of Data Warehouses. 26.What are the goals of Data Warehouse? PART-B 1. What is Data Science? Write the benefits, disadvantages and uses of Data Science. 2. Explain about the facets of data with the challenges in their processing. (Nov/Dec 2022) 3. Explore the various steps associated with data science process and explain any three steps of it with suitable diagrams and example. (Nov/Dec 2022) Write the overview of Data Science Process, with neat diagram. .. Explain in detail about Data Preparation.What is Data Mining? Write in detail about the features of data mining. What is Data Warehousing? Write in detail about the features of data warehousing. . Explain about the basic statistical descriptions of data. 9. Write in detail about the building applications of data science 10. Explain the research goal and data retrieval process. 11. Explain building the model.oe oe) 5105 - BARATHIDASAN ENGINEERING COLLEGE, NATRAMPALLI ees S3352 - FOUNDATIONS OF DATA SCIENCE [ REGULATION-2021] BE. CSE -Ill- SEMESTER UNIT-2-DESCRIBING DATA PAR’ . Define Data. Mention the types of data, . What is Qualitative data? What is Quantitative data? . What is Ranked data? . Indicate whether each of the following terms are qualitative, quantitative or ranked data. a) Gender b) 1Q score ¢) Second place d) Academic major What is Nominal and Ordinal Measurement? . Define Variable and Constant. Define Variable. Mention the types of variables. .. Classify the below list of data into their types: a) ethnic group b) age c) family size d) academic major e) sexual preference f) !Q score g) net worth(dollars) h) third-place finish i) gender j) temperature and write a brief note on them. (Nov/Dec-2022) 10, Differentiate discrete and continuous variables. (Nov/Dec-2022) 11, What is Discrete Variable? Give Example. 12, What is Continuous Variable? Give Example. 13, What is Approximate Numbers? Give Example. 14. Indicate whether the following quantitative observations are discrete or continuous. a) Ia b) Age ©) Litter of dog d) Speed of a jetliner 15.Write about Independent variable and Dependent variable. 16.What is Frequency Distribution? 17.What is Frequency distribution for ungrouped data? 18. What is Frequency distribution for grouped data? 19. Construct the Frequency distribution for the given ungrouped data. 1 2 1 3 620. Mention the essential guidelines for Frequency Distributions 21.What is Outlier? Give examples. 22.What is Relative frequency distribution? 23.How will you construct relative frequency distribution. 24.What is Cumulative frequency distribution? 25.How will you construct cumulative frequency distribution. 26. What is percentile rank of an observation? 27. From the below given table, Find the approximate class of a) 220-229 b) 160-169 (CUMULATIVE FREQUENCY DISTRIBUTION percentile rank of any Weight in the ‘CUMULATIVE weigiT 1 (CUMULATIVE f PERCENT 240-269 1 53 100 B23 0 52 98 m8 3 52 93 210-219 o 9 2 209-203 2 48 92 190-198 4 a 2 120-188 3 3 at 370-178 7 “0 5 160-168 12 3 e 150-159 7 2 40 140-148 1 4 8 1304 2 3 6 Total 33 28.Define Graph. 29.Define Histogram. 30.What is Frequency Polygon? 31.Define Stem and Leaf Displays. 32. What are the typical shapes of smooth frequency polygon? 33, Explain Bimodal Shape. 34. What is Positively Skewed and Negatively Skewed Distributions? 35.Define Mode. 36.Define Bimodal and Multimodal 37. Determine the mode for the following retirement ages: 58,62,65,60,62,60,65,62,55. 38.Define Median. 39. Find the Median for the following retirement ages: 60,61,58,60,62,61,60,62,62 40.Find the Median for the following retirement ages: 20.3, 22.2, 20.2, 23.5, 20.1, 225 41. How will you calculate the Mean? 42. What is Population mean and Sample Mean? 43. White the formula for population mean and sample mean. 44,Find the mean for the following retirement ages: 60, 63, 65, 50, 60, 63, 70, 58.626! 45, Indicate whether the following terms or symbols are associated with the population mean, the sample mean, or both means. (a) N (b) varies (c) 5 (c) n (d) constant (e)subset 46.Define range and variance. 47.Wite the Definition formula and Computational formula of Sum of squares fo" Population. p 42, the Definition formula and Computational formula of Sum of squates * 5 49, How will you calculate standard deviation for population and sample.50.What is degrees of freedom? 51.Determine the values of the range and the IQR for the following set of data. Retirement ages: 60, 63, 45, 63, 65, 70, 55, 63, 60, 65, 63. 52. Define Z-Score and write its formula. 53.What is Standard score and Transformed standard score? PART-B 1. The IQ scores for a group of 35 high school dropouts are as follows: (a) Construct a frequency distribution for grouped data. (©) Specify the real limits for the lowest class interval in this frequency distribution. 2. Demonstrate the different types of variables used in data analysis with an example for each. (Nov/Dec-2022) 3. GRE scores for a group of graduate school applicants are distributed as follows- GRE 725-749 700-724 675-699 650-674 625-649 600-624 575-599 550-574 525-549 500-524 475-499 2 Total 200 a) Convert to a relative frequency distribution, b) Convert to a cumulative frequency distribution. ©) Convert to a cumulative percent frequency distribution. 4. Explain the types of graphs for quantitative data.“5. Write briefly the typical shapes of frequency distribution with neat diagram and” examples. 6. Describe the graphs for Qualitative data and misleading graphs 7. The number of friends reported by Facebook users is summarized in the following frequency distribution (Nov/Dec-2022) FRIENDS 400 - above 350 - 399 300 - 349 250 — 299 200 - 249 150 - 199 100-149 50-99 0-49 Total 200 (a) What is the shape of this distribution? (b) Find the relative frequencies. (c) Find the approximate percentile rank of the interval 300-349. (d) Convert to a histogram. (e) Why would it not be possible to convert to a stem and leaf display? 8. Explain in detail about describing data with averages. 9. a) Explain about Positively skewed averages and negatively skewed averages with neat diagram. b) Write briefly of averages for Qualitative and Ranked data. 10. (i) Indicate whether each of the following distributions is positively or negatively skew The distribution of (a) incomes of taxpayers has a mean of $48,000 and a median of $43,000. (b) GPAs for all students at some college has a mean of 3.01 and a median of 3.20. (0) number of ‘romantic affairs” reported anonymously by young adults has a mean 1? affairs and a median of 1.9 affairs. : Fann ja (@) daly TV viewing times for preschool children has a mean of 58 minutes and @™° of 73 minutes. Dus “9 (i) During their first swim through a water maze, 15 laboratory rats made the flO", number of errors (blind alleyway entrances): 2, 17, 5, 3, 28, 7, 5, 8, 5; 6: 2: 10,% Find the mode, median, and mean forthese data.“44. With neat diagrams explain Intuitive approach of describing variability. 12 i) Using the definition formula for the sum of squares, calculate the sample standard deviation for the following four scores: 1, 3, 4, 4. ii) Using the computation formula for the sum of squares, calculate the population standard deviation for the scores in (a) and the sample standard deviation for the scores in (b). (a) 1, 3,7, 2,0, 4, 7,3 (b) 10, 8, 5, 0, 1, 1,7, 9,2 13. Explain in detail about degrees of freedom and Interquartile range. 14. Write in brief of finding scores. 15. Explain in detail about Normal curve and Z score.DESCRIBING RELATIONSHIPS PART-A 1. What is correlation? 2. What is a percentile rank? Give an example. (Nov/Dec 2022) 3. Consider Helen sent 10 greeting cards to her friends and she received back 8 cards, what is the kind of relationship it is? Brief on it. (Nov/Dec 2022) 4, Define positive and negative correlation. 5. What is cause and effect relationship? 6. Explain advantages of Scatter diagram. 7. What is regression? 8. What are assumptions of regression? 9. What is regression analysis used for? 10, What are the types of regressions? 11, What do you mean by least square method? 12, What is correlation analysis? 13. State the multiple regression equation? 14, What is a scatterplot? 15. What is the need for correlation? 16. What is a Linear Relationship? 17. What is Nonlinear Relationship? 18. List the types of nonlinear relationship. 19. What is the regression line? 20. What is the Interpretation of °? 21. What is Standard Error of Estimate? 22, What is a correlation matrix? 23. What is a curvilinear relationship? 24, What is an outlier? 25, What are the key properties of correlation coefficient r? 26. What are the types of regression models? 27. Compare Correlation and Regression. 28. What is Regression towards the mean? 29. When does Regression Fallacy occur? 30. Indicate whether the following statements suggest a positive or negative relationship: a) More densely populated areas have higher crime rates. b) School children who often watch TV perform more poorly on academic achievement tests. c) Heavier automobiles yield poorer gas mileage. 4) Better-educated people have higher incomes. ¢) More anxious people voluntarily spend more time performing a simple repetitive task. PART-B 1. Elaborate in detail the significance of correlation and the various types of correlation. 2. What are Scatterplots? Elaborate on the various types with suitable examples, (Nov/Dec 2022) 3. Highlight the significance of the correlation coefficient r. Compare the various correlation coefficients. 4, Calculate and analyze the correlation coefficient between the number of study hours and the number of sleeping hours of different students.), Find the standard error of the mean weight of high school football players using the data | No. of study hours 74a] eao ea) cet). | No. of sleeping hours What is the significance of ati : A) Each of the following pairs represents the number of licensed drivers(X) and the number op [es 7 |. = 0p el [ 25 Give a detailed interpretation ofr? (Nov/Dec 2022) cars(Y) for seven houses in my neighborhood: WovDee my) Drivers (X) | Cars (¥) 5 4 5 3 2 2 2 2 3 2 I i 2 2 i) Construct a scatterplot to verily a lack of pronounced curvilinearity. ii) Determine the least squares equation for these data (Remember, you will first have to calculate r, SSyand SS;) iii) Determine the standard error of estimate, Sys, given that n=7. B) In studies dating back over 100 years, it’s well established that regression toward the mean oce between the heights of fathers and the heights of their adult Sons. Indicate whether the following statements are true or false. i) Sons of tall fathers will tend to be shorter than their fathers. ii) Sons of short fathers will tend to be taller than the mean for all sons. iii) Every son of a tall father will be shorter than his father. iv) Taken asa group, adult sons are shorter than their fathers. v) Fathers of tall sons will tend to be taller than their sons. vi) Fathers of short sons will tend to be taller than their sons but shorter than the mean forall fathers. Discuss the importance of regression. Elaborate on types of Regression. Assume that an r of -.80 describes the strong negative relationship between years of heavy smokit (X) and life expectancy (Y). (Nov/Dec 2022 ‘Assume, furthermore, that the distributions of heavy smoking and life expectancy each having t# following means and sum of squares : 5 60 35 70 x y X Y S$ SS i) Determine the least squares regression equation for predicting life expectancy from ye#* heavy smoking. ii) Determine the standard error of estimate, Sy, assuming that the correlation of 8095" ‘on n=50 pairs of observations. iii) Supply a rough interpretation of Sy. iv) __ Predict the life expectancy for John, who was smoked heavily for 8 years. vy) Predict the life expectancy for Katie, who has never smoked heavily. Explain the significance of regression line and least squares regression equation. gl weights of the players. Player Number _| Weight in Pounds | 1 150 |late regression towards the mean, Explain regression fallacy and state how it can be avoided. . Find the value of the correlation coefficient from the following table: Subject Agex 1 43 2 21 3 5 4 am 3 37 7 6 9 a1 reading and math scores on the SAT test for students A,B,C,D,E,F,G,H and Tare shown in the following scatterplot: a) Which student(s) scored about the same on both tests? b) Which student(s) scored higher on the critical reading test than on the math test? ©) Which student(s) will be eligible for an honors program that requires minimum scores of 700 in critical reading and 500 in math? 4) Is there a negative relationship between the critical reading and math scores? 15. A sample of 12 fathers and their sons gave the following data about their heights in inches, Calculate the coefficient of rank correlation, Fathers Height | 65 | 63 | 67 | 64 | 68 | 62] 70 66 | 68 | 67 | 69 [ 71 Sons Height 68 | 66 | 68 | 65 | 69 | 66 | 68 | 65 | 71 67 | 68 | 70 16. Find Karl Pearson's correlation coefficient for the following paired data, [ Wages. 100 | 101 | 102 [ 102 | 100 | 99 97 98 96. 95. [Costof living | 98 [99 [99 [97 | 95 | 92 | 95 | 94 | 90 [91 17, A random sample of 5 college students is selected and their grades in operaiing system and software engineering are found to be? Subject i z Bene 3 Operating system [85 | 60 | 73 | 40 | 90Calculate Pearson's rank correlation coefficient? UNIT-4- FA‘ PYTHON LIBRARIES FOR DATA WRANGLING PART-A er AawEyyo 10. . Where is Numpy used? . Identified the details maintained by python to store an integer. . Write Python code to create 1D,2D and 3D Numpy arrays. 14, 15. 12. 1B 16, 20. 21 22. 23, 30. 31. 32. . How to get the items of series A not present in series B? 33. Define Data Wrangling? What is Python? What is Numpy? ‘What is an aggregation function? What is Structured Arrays? Describe Pandas. How to manipulate and create categorical variables? Explain hierarchical indexing, What is Pivot table? List the uses of Numpy. How can you verify the shape of 1D,2D and 3D array respectively? ‘Compare Python list with array. . How can you identify the datatype of a given Numpy array? 17. 18. 19. Write short note on python array object. How to perform slicing to access the elements of a numpy array? ‘What is the procedure to count the number of times a given value appears in a numpy array of integers? List the attributes of a numpy array. Give an example. (Nov/Dec 2022) Create a data frame with key and data pairs as Key-Data pair as A-10, B-20, A-40, C-5, B-10, Cl Find the sum of each key and display the result as each key group. (Nov/Dec 2022) “List is mutable.”-Justify with suitable example. . Explain how to create a dictionary in python? 24, 25. 26. 21. 28, 29. What is indexing and negative indexing in Tuple? Write the list of aggregate functions of numpy. What are Universal Functions (Ufunc)? What is array broadcasting in Numpy? Identify the limitations of broadcasting. Develop a python program to list all the values from the array which are less than the mean of entire array. Explain the application of Boolean arrays as masks in numpy. What is Fancy indexing? Explain Reindexing in pandas? PART. Extract from the array np.array([3,4,6,10,24,89,45,43,46,99,100]) with Bootean masking al number: © Which are not divisible by 3 ® Which are divisible by 5* Which are divisible by 3 and 5 * Which are divisible by 3 and set them to 42 List the prime numbers between 0 and 100 by u 2. ng a Boolean array, 3. Elaborate on indexing and slicing operations of numpy arrays, 4. Demonstrate on how vertical and horizontal spitting are done in ndarrays. 5. Discuss the array aggregation operations of numpy arrays with example, 6. What are ufunes in Python?Explain with examples. 7. Explain comparison and masking operations, 8. Assess the benefits of fancy indexing. 9. Explain about the Pandas objects. 10. Develop python programs to create pandas series through different methods. 11. Demonstrate different ways of creating pandas dataframe. 12, How indexing is done in pandas? Explain. 13. Describe various methods of handling the missing data in pandas, 14. Exhibit the benefits of multiple indexing. 15, Discuss the approaches to combine datasets and identify the challenges. 16. Explain Hierarchical indexing, 17. Explain about operating on data. 18. Explain data selection in data frame and data selection in series. 19. What is data wrangling? Explain iterative steps of data wrangling. 20. Imagine you have a series of data that represents the amount of precipitation each day for a year ina given city. Load the daily rainfall statistics for the city of Chennai in 2021 which is given in esv file Chennairainfall2021.esv using Pandas generate a histogram for rainy days, and find out the days that have high rainfall. (Nov/Dec 2022) 21. Consider that, an E-Commerce organization like Amazon, have different regions sales as NorthSales, SouthSales, WestSales, EastSales.csv files. They want to combine North and West region sales and South and East sales to find the aggregate sales of these collaborating regions Help them to do so using Python code. (Nov/Dec 2022) UNIT-5 DATA VISUALIZATION PART-A 1. What is data visualization? 2. Write the concepts used in data visualization. 3. List the benefits of data visualization, 4. Why big data visualization is important? 5. Explain Matplotlib. 6. What is the purpose of error bar function in Matplotlib? Give an example. (Nov/Dec 2022) 7. Showcase 3-dimensional drawing in Matplotlib with corresponding Python Code. (Nov/Dec 2022) 8. What is contour plot? 9. Explain legends. 10. What is subplots? 11, What is the use of tick? 12, Describe in short Basemap. 13. What is Seaborn? 14. List the line styles supported by pyplot. 15. List the applications of Line plot. 16. What is scatterplot?17. Write the syntax of 18, Enumerate the classes of color 19. What is the significance of 20. What is a histogram? 21. Define Kemel Density Estimation, 22. Write about text transforms 23. List the ways to customize Matplotlib, 24. Brief on Basemap tool kit. 25. Write the features of Seaborn module. 26. What is density plot? 27. What are Pair plots? er() method. in scatterplot. Error bat PART-B 1. Write Python program to plot line chart by assuming your own data and explain the various attributes of line chart, 2. Write python program to visualize the dataset using scatterplot and explain its parameters. 3. Elaborate the error visualization methods in pyplot. 4. Explain different methods of showing three-dimensional surface on a two-dimensional plane with example. 5. Demonstrate the usage of histograms for data exploration and explain its attributes. 6. Elaborate the concept of subplots and its applications. 7. Discuss in detail about the three-dimensional plotting functions of matplotlib module. 8. Explain in detail about the functions of mpl_tool kit for Geographic data visualization. 9. What is scatter plots? How to create scatter plot by using plt.scatter() and plt.plot() method? Explain with example. 10. What is Legend? How legend helps for data visualization? Explain with various example. 11. How text and image annotations are done using Python? Give an example of your own with appropriate Python code. (Nov/Dec 2022) 12. Appraise the following (i) Python code. 13. Elaborate visualization with seaborn. istograms (ii) Binnings (iii) Density with appropriate (Nov/Dec 2022) 14. Perform an exploratory data analysis for the following data with different types of plots: The dataset contains cases from a study that was conducted between 1958 and 1970 at the University of Chicago's Billings Hospital on the survival of patients who had undergone surgery for breast cancer. ‘Mee 2022) Data attributes: - eee eat Age of patient at the time of operation (numerical) Patient's year of operation (year-1900, numerical) Number of positive axillary nodes detected (numerical) Survival status (class attribute) 1 = the pati ive ithit eae (class attribute) 1 = the patient survived 5 Years or longer, 2 = the patient died within