il. Explain about the data analysis sequenc: Answer: The data analysis sequence is a process of getting the data, cleaning the data, organizing the data and sizing, the data. Such analysis helps in decision making. 5. There are different types of analysis, that describe the data. Descriptive Data Analysis Its the most easiest type of analysis, where the data is collected and arranged to make the data set more manageable. It is usually displayed in image format. Exploratory Data Analysis Itis an important process of data analysis, where the structure and content of the dataset can be discovered. It is commonly called as EDA. Inferential Statistical Analysis It is a process of making the conclusions and predictions on vast data by assuming the tiny amount of sample data from the original data. Causal Data Analysis Itis a process that helps to point out or track the cause of problem. Mechanistic Data Analysis Itis a process of estimating how one variable influences other. Q12. Write about data acquisition pipeline. Answer : (Model Peper, G1ata ata acquisition means a group of various devices that are combined in an orderly manner, consist of raw data from a multiple sources, allowed to move and transform the data to reach a target source as represented in the below figure. cE Source Format Representation ProcessingDATA ENGINEERING WITH ‘There are three major data sources, 2 Introduction 1. Internet Inroduetion presents a preview ofthe 2. Databases problem to the reader before studying the ‘art ofthe report. It should be easy, simple and ao the reader's attention to study the furher pars, fy ‘The large-scale of data formats may exist inthe | contents included in introduction are, amis. The following are the mostcommenly k90¥® |) pevous stuies/esachon he sme op 1. Unstructured Data (©) Objectives Scope ofthe study Ie means plaintext, ike English or Chinese | (©) Name and dts ofthe writer ae (© Major theories, principles used inthe study ™ a (©) Sources of collecting data Hconsist of tabular data and tagged data, | (7) — Methodology of the study (Tabular data in Comma Separated Values | (4) Cepanization of the repo (esv ties) (©) Organization ofthe report ca i 3. Method . Itis also known as procedure, this part contin if) Tagged data in HTML or XML “ the details of the problem i. location, amount and iv) Tagged data in 1SON. collection of data FFor the further processing of extracted dats, | Example ‘python data structures specialized in lst and dictionaries ‘oradvanced data structures specialized in numpy arrays If itisa science lab report, the following questions ‘and pandas data frames are also used. ‘must be answered, A filly automated data pipeline includes | What apparatus was used? © How the experiment was carried out? What precautions are taken to increase accuraey? % How many times did you repeat the procedure? 4. Discussion 3. Data integration from other sources before 7 ‘it reaches to destination source. Lae ie modularity, reusability and Discussion forms the major part ofthe repo big pipeline i divided into number of | 1! includes results part and analysis of facts and opie MONON ened rut ee morc | interpretation ofthe das snd ony tne eal ithe pickle or ISON files. draw a perfect solution to the given problem. I ale BO Titeiestonofpipetineresutsinreprodecibie | Pvides benefits and limitations of the sohdi Seidel meses sot of python scripts that can be | 1 includes ilustrations, tables, figures Where oy anyone to transform the raw data into inal | MESES. Discussion forms strong. base 10 dA effective conclusions 5. Conclusion ‘Conclusions are the important points drawn ‘analysing and interpreting the data in the disc Conclusion may be represented either in the ‘narration or in points. «a‘There are two types of files in python, 1. Text ile 2. Binary file, 1, Textile A text file contains only textual information like alphabets, digits and special symbols, The ASCII code ofthese characters are stored in text files. ‘Some of the features of text files areas follows, (@ intext file, data is stored in the human readable form. (i) Anew line character (\n) is used to terminate ‘every line of data in the text file. (Git) A special character marker called “EOF” (End. OfFile) is used to indicate the end of a file. ‘The following are some ofthe common extensions text file formats, (© WebStandars: hm xm, sv, jon, 8 * () Source Code: c,h s,s, epPs java, py tb, ph. (© Documents: x, markdown, af, ex, ps @ Configuration: ef, ini re, (©) Tabular Data oxy, 2 binary Files A binary file is a collection of bytes. This ‘lection isa compiled version of any program (Ee ‘Program!.exe), or music data stored ina wave file oF 8 Picture stored in a graphic file. Some of the features of binary files areas follows, [No new line characteris used to indicate the end ofa ine. ‘Images: png, jpg, aif, bmp, pss Audio: mp3, way fla, aac, mka, ma. Videos: mp4, mov, vob, avi, mk, MPR (@ Database: mdb, fm, acede, sali. (©) Documents: doc, pdf, ppt, docx, xls. (Archive: zi, iso ar, tar (g) Executable: exe, dll, 50, class. QI5. Explain the fle operations along with examples. oR Write about the list of methods that are associated with fle object. Answer : Model rape, a13(0) ‘The python provides builtin functions for the following file operations. 1. Opening a file 2. Reading from a file 3. Writing to a file 4. Closing a file 1. Opening a File A file is opened by the open¢ ) function. This function returns a file object which is also known as handle, Therefore the file can ether be read or modified appropriately by making use ofthe file object. This file jobject consist of two parameter which are as follows, (@) Filename o syntax Filemode, file object = openfile_name,file_ mode) © The “file_name” parameter specifies the file ‘name with an extension such as “Data.txt” of “Programi.c” and s0 on. ‘The “file_mode” parameter specifies the mode in which file can be opened. The various modes ‘of fil ae, ‘read mode’ (r), ‘write mode’ (w), “append mode’ (a) ‘r +’ mode for reading iting ec. However, default mode is read ‘as by‘This mode opens the already ex- ising file for both reading and tering purpose in binary format only. This mode opens the existing file (or ereates a new file for writing purpose only: This mode opens the existing ile (or) creates new file for both writing and reading purpose. This mode opens the existing (or) new file forthe purpose of werting in binary format only This mode opens the file for the purpose of both reading and swingin binary format only. This made pens the ile for the Purpose of appending the text This mode opens the fe forthe purpose of both reading snd ap Pending ext This mode opens the file for the purpose of append ext in binary format only This mode opens the fe forthe purpose of bth reading and ap Pending the tex in binary form aly = opent "file tat, ‘open "file we) pent", wy open "fileant, open("filetnt open "flea opent "flex penileThi ines from the fle. iitilty “reads and retums the fist line ofthe fle. Then, ‘it returns the next line when it is called next Each time whenever itis called it etums a new lin. I retums an empty value, i the end of file isreached. syntax freadine) fxomple f= opent “lel ext print realine )) print realint )) readlines( ) Method This method reads alist of lines in the fie. syntax freadlines) Iretums an empty valve if end of files reached. Hisalso used to read a specific line from the file. >>> f= open(“filetxt”/r') >>> 3. Whiting to 0 File BAe in Python can be writen by fist opening ‘tin write mode ‘Syntax {= open( filename." “There ae various methods fr writing file They ae, write) Method This method writes data into a file It doesnot {84a new line character ("sn") at he end ofthe ‘rng, Tis method canbe called many times and the method a new line is writen. = freadlines() ) easy language”) his F opent “file txt"w') {writelineslist-of strings) 4. Closing a File Afilchas to be closed as soon as the user finishes performing operations on it like reading, writimay appending ete. This will release the resources associated withthe file. A file can be closed using the closet ) method. Syntax feloset) Where, fis te file object Example = openifilename.tt"'") feloset) Ths method does not eturn any value. It flashes unvriten data and closes the il object. File operations cannot be performed on close file. Am attempt to do this will rase a Value Error exception In Python, file is closed by default when the reference object is re-assigned to another fil, However, the close( ) method is not considered to be safe. The program code may exit if an exception occurs while performing any operation on the file. This leaves the file linclosed. Hence, it ia good practice and a safe method tousea ty. finally block to ensure thatthe file is closed ‘even afer an exception is Example‘the above main operations, there are few more operations which of them are as follows, _ tell() Method : This method takes no parameter and returns an integer. It is used to get the current position file object. ‘Syntax fitell() Where, fis the file handler. 2. seek() Method This method takes two parameters, offset and ‘from_what’. It is used to change the position of to a specific given position. Syntax f.seek(offset, from_what) ‘Where, % —_fisa file handler or file object offset is a number of positions to move forward ) % —from_whatis a point of reference accepts three values, 0, | and 2. ‘0° sets the reference pointat the Of the file, ‘1’ sets the reference point at the position of file and ‘2’ sets the reference point at the file. By default, itis set to ‘0°UO ce GAA. Wille about the following, (a) Use of with statement (b) File object attributes. Answer (a) Use of with Statement In python, with statem pesources such as file streams. It hely closed automatically after execut syntax with open (filenai Statement-1 Program def rea king with unmanaged readable. A file can be(b) File Object Attributes File objects have file attributes other than file methods. They contain auxiliary data of file objects such as filename, file mode and a flag that indicates whether an additional space is required to be displayed before any successive data items. Various attributes of file objects are as follows, file.encoding It is a type of encoding to convert the unicode strings into byte] | strings | file.closed It returns true if file is closed, | ee otherwise false is returned