i 12 Python Pandas - | Outline ‘1.1 Introduction 1.2. Using Pandas 13. Pandas Data Stuctues Introduction Pandas or Python Pandas is Python's library for data 1.4 Series Data Structure analysis. Pandas has derived its name from “panel data 1.5 Accessing a Series Object system”, which is an ecometrics term for multi- ‘and fs Elements dimensional, structured data sets. Today, Pandas has 16°. Operations on Series Object LT. Series Objects vs. 1D Data Structures and 1D Numpy Array 1.8 DataFrame Data Structure 1.9 Creating and Displaying a DF 1.10 DataFrame Attributes 111 Dataframe vs. Series and become a popular choice for data analysis. As you must be aware of that data analysis refers to process of evaluating big data sets using analytical and statistical tools so as to discover useful information and conclusions to support business decision-making. Pandas makes available various tools for data analysis 2D Numpy Array and makes it a simple and easy process as compared to «1.12 Selecting or Accessing Data other available tools. The main author of Pandas is Wes 1.13 _Adding/Moditying Rows'/Columns" McKinney. \Values in DataFrames 4.14 DeletingRenaming ColumnsiRows 1.15. More on DataFrame Indexing ~ Boolean Indexing This chapter will introduce Python's Pandas library, its data structures series and dataframes and some useful functions with these data structures. “TMPORTANT Please note tha although Pandas i separate brary yet it uses Num as it support library and hence many datatypes, constants and functions of NumPy ae frequent! used with Pandas, Since NumPy was removed from lat year's class 11th syllabus because of COVID situation, students could not study NumPy. Thus, we are giving full NumPy chapter © of class XL as a SUPPORT MATERIAL in SIPO App. To get this chapter, open SIPO App ‘and then Open Support tab to find it. Using Pandas Pandas is an open source, BSD library built for Python programming language Pandas offers high-performance, easy-to-use data structures and data analysis tools. In order to work with pandas in Python, you need to import pandas library in your python environment. You can do this in either on the shell prompt or in your script file (py) by writing: import pandas as pd 1 wy2 . Pee a aime, Ma Ms ea ce” fr mre ee : i me | tan aassngens HO YAIN Ny Pig oan ee ‘Why Pondos ® sar brary in the cet Pylon ecsystet for doing data analysis Pandas isthe most popular any tasks incuding Pome boenea ferent dat formats (nfeger, flat, doublet). «9 Ttean read or wate in many ata is organized ie, across rows and down columns. ie toy old ining dt a oe eter = pandas.series( ) series() Saupe ‘The above statement will create an empty Series type object with no value having default datatype which is float64. Consider following statement S25 ead» oe Series) pre Series), tye: Floste) 1. Creating non-empty Series objects ‘To create non-empty Series objects, you need to specify arguments for data and indexes as per following syntax ‘Series object = pd.Series(data, index-idx) ‘where idx isa valid Numpy datatype and data is the data part ofthe Series ok oni ae Part ofthe Series object, it can be one © A Python sequence © An ndarray ‘© A Python dictionary © Ascalar value CChopter 1: PYTHON PANDAS — Folk ° owing Slsetions talk about the ways to creat Series objets as pera () Specify dota os Python Sequence mores Simplest way to create Series i sin ‘ype objet isto givea sequence of values as attribute to Series), ‘Series Object> = Series () It will return an object of Series ries type. For instance, consider following example ing some Python sequences Seampiesatenens at create two Series type objects ust : Series cjectobjt cE >> print(ndat) (3 osu.) rndod = np.arange(3, 13, 2.5) >>>-serl = pd.Series(ndat) print(ndat) : ser = pd. Sertes(ndat) eee print(sera) lo 30 hos You can ako combine te code ines of yg ae above code. Carefully read the following : arte Pd-serses(DB-3range(3, 13, 3.5)) aoe se een inv cale 7 tc naray deel hoo ‘ou can create a Series object from any ndarray created from any fun i Zeepn cae Sere op ty ndarray fom any function, Consider following ‘Chapter I: PYTHON PANDAS: 7 EXAMPLE EI Wate progam eaea Sow object using an nara ‘hat has 5 elements inthe range 2 to 64 * Output SOLUTION eens smote ap ae S6=pd.Series (np. Linspace(24, 64, 5)) 2 print(s6) pace type: Flares EXAMPLE T Wites pono ten S lit gm ny lS oe HST soic). SOLUTION ae {import pandas as pa ar 57 = paSerses (op t10(3,5},2)) 3 print(s7) an type: int32 (ii) Specify data as a Python Dictionary ‘The sequence that you provide with Series() can be any sequence, including dictionaries. Let us see how you can create a Series abject by specifying indexes and values through a dictionary. {ee below) pa-Series( (an voy Feb? 38) ars YD) ee, ef otra oak the Keys of parsed as indexes and values form te data values (Aso, notice nat indexes ae nol inthe same mer 38 given inthe dictonary above ) Here, one thing is noteworthy that if you are creating a Series object from a dictionary object then keys ofthe dictionary become index of the Series and the values of dictionary become the data of Series object. Also, the indexes, which ar created from keys may not bein the same order as you have typed them. EXAMPLE Bo) Write program to create a Series bject using a dictionary tha stores te number of students in cach section of class 12 in your school. Output SOLUTION ire import pandas as pd aaaal stu {°A':39, 'B:41, "C142, con “4 = pd.Series(stu! ° See type: int6e print (s8)INFORMATICS PRACTICES ~ Xit 8 (@) Specify data os © Scolar Volue tase veo 2 N The daca ine mt nder gent ganas a ee value, BUT if data is sala provided. The scalar va on une be prod to Sexiest) function must Pe Pan the length of index iven as data) will be repé enor agents 1B 2 labels of any type) ¢. consider this jes(10, index = range(®, 2)) re Cintra “ile sequence of rumbers oF is ergamens wil specify he inde for Sones ce lem ‘nedalskon = pd. Ser’ rnedals2 « pd.Seres(15, index =.ra Series("Vet to start’, index = serz=| ee below the Series objects created as per above statements Yet to start Yet to start! Sar ae TOD ear SEMPLE Epp re St so ab fej art ofte war, QO and QP ouput coon gtrl 50000 import panda 3 4 eaten seecrrentestsne, ince [tet "Qe2", Rr", “@ee'D) gerd 50000 aera 50000 prwees) dtype: int64 Spann NI ar ct on 20 er Univers goes deer allrat yer tod bree Sei jt hat sores thse meals forges fb hd the decade 2020-2028, SOLUTION Ostpnt 2020 200 nport pandas as pd pata 510 « pd.Series(209, index = range(2020,2028,2)) 2028 200 print(ste) cee ee 2028 200 dtype: ints 1.4.2 Creating Series Objects ~ Additional Functionality Now that you have a fair idea about how to create Series type objects, let us talk about ‘ditional functionality of Series) that you can use to create Pandas Series objec. ) Specitying/Adding NoN values in o Series Object Sometimes you need to create a series obj vlna a ee cea Of rinsizebut you do nothave complete dla "can fill missing data with a NaN (Not a Number) value (i) Specity index(es) as well as data with Series( ‘Chapter I= PYTHON PANDAS - | 9 'Py module and hence you can use np.NaN to specify a wing code Legal empty value NaN is defined in Num missing value, or use None eg, see follo >>> ong 23 >>> obfa Gas 0 hays cope : Soran 2 ctype: Use apna or None to add missing data, While creating Series type object is that along with values, you also provide indexes. Both. values and indexes are sequences, ‘The syntax for this is as follows : Tees ou sip tee parte aer © [31, 28, 31, 30] Laon © [ant “Fi, Mh’, “toe) Sere ajo Sead wi bob vs wa EE ‘oven as soquenoas >>> obj3 « pd.Series(date = arr, index = won) poe obi feo 28 Sores elevated wih above sora] ivr 32 aor 3 {etype: intse |Sonobia = pdsertet date © (92, 34, 25], index © 0s Temes prereset ‘You could skip keyword data also, ic, following statement will also do the same as above ‘obj3 = pd.Series(arr, index = non) ‘You may use loop for defining index sequence also, si =pd,Series(range(1, 15, 3), index = [x for xn ‘abede'])10 “The above code will create a series objet as shown belt LSS aa pd. Series(range(ts 15+ 3+ indexz[x for * 31 Caution! / indexes equal to the number of speci indees expt sigan index sevens YOu mi prov inso a values in data anay; providing to0 few or too many indices will Error TCA, B,C, Def clas 12 in your schol. sa aon es : feel we charity fund endorsed by the school. Write “Another list contri stores the contribution made by these students toa itactee Sree tha stares the contrition anount ste vals he ection names asthe eves SOLUTION Output Snport pandas a pd oreo sections ('A', ‘81, °C, 0°) 25600 contri = (6700, 5600, 5860, 5200) a s1t «pd. Series(data = contri, index= section) > $200 print(sit) type: snes (i) Specify Dato Type along with dato and index ‘You can also specify datatype along with data and index with Series()as per following syntax ) To understand its functioning, consider following example statements NamPy arava created as ‘supports vectrzed operation >>> print(a) it 9 4811 12) 59> obj? = pd.Series(index <4, date = a * 2) ita seccene dren a8 37) 1s, oer vaua of a doubled >>> objB = pd.Series(index = , date = a ** 2) >>> aba ow Ho 389 ini in la ‘type: int? The vectorized operations on a NumPy array (¢g.,* 2 or a "2 ) will be applied on every clement of NumPy array and stored as data part of Series object. Now consider another code: SSSust = (9, 20, 12, 12] Notice thstime the eqresion or ata aay mole tad se ow thas impacted he irl data aay “The above code tries to create data array for Series object obj8 by giving expression 2 *Lst where Lst is a Python list and as you know a number multiplied with alist replicates the list those many times and hence the data array has the same list values replicated twice (= 2* Lst)INFORMATICS PRACTICES ~ XII 12 previous example erent fom the slowing example. Iti sit lar yet differen Carefully read the fol Ta. Fe oc dre math ll be doubled. AMPLE BEM Sequences section ne the rept 670, S600, S00, 5200, i Contin as made by ac set ie the dain YS iit cde to crete a Series objec that stores the conriution indees with datatype a float32. SOLUTION ers eon aso i import pandas as pd Te a econ ae reg {nport nunpy 35°F ne oll cho conten section =['A', '8", 'C, ‘D's E'T $200, mt) contri = ap.array( {678,560,520 seo Serenata contri 2, inde ected PFISTZ2) print(s12) nis wit double ack va of the comer array. This is Fe ile the sols count 0 char at ———— aed NOTE A 1400.0 3200.0 nen joustore a NaN ave na sees © 30000.0 bjs Pandas rege the datatype > 1000.0 conn he us pfs Sts cist sing poet ye. Even You re Spiraea specly an integer type, Pande wil ype: Floneaz «et om dtl at promate to» Roane pnt ype {automataly) because NAN is not supported by integer types IMPORTANT ‘While creating a Series object, when you give index array as a sequence then there is no compulsion for the uniqueness of indexes. That i, you can have duplicate entries in the index array and Python won't raise any error, eg, see figure below : shar Tienes 6, 37) — fsoowa [scat ets rray([ 2.75, 12.5 , 22.25, 32. 41.75]) >opeb3 = pd.Series (val, index = ['2", pias te) soooba 27s 12150 225 32.09 fe as None {type floats “aomenenanaennnearenanarnnnnnnnnnnnnnnnnanay (tt need not be unique in Pandas ‘Seties Object. This will only cause an ‘eror if/when you perform an operation that requires unique indice. Chopter 1 - PYTHON PANDAS - 13 ‘You have read till now that Series can store he >) sms es ee set pe ‘explore this statement litle more. Have a look atthe following examples, a 1. Creating © Series object that stores three integer values de> BLS pa.sezten( (11, 22, 3317 1 2 ‘Se, hs Sores oe trig ol neg BB mero tat ef Seyper ince Iatger dtanpes Since all the values being stored are integers, an integer datarype Is chesen by Pandas (int here), so that all values can be efficiently stored and processed. All values are of int64 types —hence homogeneous dataype 2. Creating @ Series object thot stores thee floating type values S93 a2 = palserieet (01-1, 12.2, 13.31) 33 a2 oo na ‘Sc, hs Serie bet i strng al ‘oaig pales nd ts. ts type: Eloates << stanpe ison of the float datas Since al the values being stored are floating point numbers, a floating-point dataype is chosen by Pandas (float her), so that all values can be efficiently stored and processed. Al values are of loat6# types — hence homogeneous datatype. 3. Creating a Series object that stores 0 mix of inieger and floating type values, 553 a3 = paSeriee( (i, 22.2, 23-39) 333 83 Achy Se, ths Sees be string all nr — ono npr an agp values ad s,s dba ton ofthe Rot datatypes o 2 deype: Hoates sscommadate all pes of amber Since all the values being stored are numbers - some integers and some floating point numbers, a floating- point datatype is chosen by Pandas (float here), so tha all values can be efficiently stored and processed, All values ae of float64 types — hence homogeneous datatype, 4, Creating o Series object thot stores o mix of different type values op a = pdsdeeben! (14, 17-3) 38.8, TF ° Sx, th Series bjt is rae roles whase apes are diferes. Tas, Pandas wil chose Atay abet) whch con aorta the Since all the values being stored have diferent datatypes (integers, floating point numbers, string etc), Pandas will select a datatype, which is capable of holding all these values, Hence, Pandas will chose is datatype as abject, which is capable of holding any typeof value. All values ar of object types ~ hence homogeneous datatype.INFORMATICS PRACTICES = yg), 4 1.43 Series Object Atributes ‘When you create Series type object through atibutes. You can use these a set information about the Series obec: ‘Series object> sue The me common at 1 Series object ae listed inthe table bel a oe fr th rageot tee Me flow the Table 12 atlinformation related toitis available ttributes in the following format to Tobie 1.2 Common attributes of Series objects Attbute Description “Series cbject>.index | The index axis abel) of the Series “Series ebject>-index.nane | Name ofthe index; can be use to asign new name to index ‘Series object».values | Rum Series as ndamay or ndareay-ke depending on the diype Series object».dtype | retum the dtype object of the undecying data Series object».shape | retum.a tuple of the shape of the underiying data Series objecto.nbytes | retum the rumber of bytes inthe underlying data Series cbjecto.ndin | etum the numberof dimensions of the underlying data Series object>.size | retum the umber of elements in the underlying data Series abject>.itensize | retum the Sze ofthe dlype ofthe te ofthe underlying data ‘Series object>-hasnans | retum True if there are any NaN values; otherwise return False Series object>.empty | retum True if the Series object i empty fab otherwise Series object>.nane | retu orasign name to Series objet Following examples show how to view various attributes of Series objects. () Retrieving Index Array (index atribute) & Data Ary (values atribute) of a Series Object You can acces the index array and data values’ array of an existing Series object objS as shown ‘See, .name attribute can be used to get or set the name of a Series object, eg, >>> serd aa b 2 63 type: inte >>> sert.name >>> sent aa b2 3 Nane: Mys, dtype: ints >>> serd.nane a {¢) Retrieving Data Type (dtype) and Size of Type (itemsize) ‘To retrieve the data type of individual elements of a series object, use type. ‘To know about the type of Series object itself, you can use typel) of Python. You can use itemsize attribute to know the number of bytes allocated to each data item eg, ‘atime ype a ata vais stored in eres cect {Sosebia.dtiee {ype("floatse’) | sosobj2.itensis {>>> type(obs2) | pandas. core. series. Seriesee 16 ————— 1 tae svn seia i (se arte tl ust NOTE ‘The shape of # Ser including missing oT elements it comtins s theshape of series object els how bigit is sf may rere isl 08 2888S yee conan ‘ince yas (NAN Cape itis shown 38: Chements in he objet soo prine(ab. shape» OSBIMPE) oyna t= G0 Sra aon ) where is the mumber of Tt or empty values Nah). bute), rieving Dimension {number ‘of axis : ndim attr 8 = ear Number of Bytes (nbytes attribute) ber of axis), use .ndim. ‘To know about the dimension (num cee ‘Tokaow about the number of elements inthe Series object, use bytes (nbytes is qual tothe size *itemsize) >> obf2andin 3 >o> print (obf2 sie, ob3.size) a 92 print (obj2.nbytes, obj3.nbytes) obj2 ha 4 eemess, hance ee ee i ion ce 8 (g} Checking Emptiness (empty ottribute) ‘and Presence of NoNs {hasnans attribute) See we have created an empty Series object namely obj1 also and then checked emptiness of ‘obj along with emptiness of objects obj2 and obj3 of Reference 1.2. (given on previous page). Scie be dimension obec 82 has flan tg has 3 eons 52 band oj oer [S5S0uga empty The cbject oti onety but ob/2 and ob) = | False (ot 102) 3 ot ompty ae, asthe contin some data {>> >0bj2.enpty ‘alse © Similarly, to check if Series object contains some NOTE [NaN value or not, you can use hasnans attribute a shown below (using objects of reference 2.2) ‘© You can use fen) o get total number of elements and .count) method with Series object to {get the count of non-NaN values in a series objec, £8 S>3ebi2hesnans Fratee IM you use len) on a Series object, then it Fetus total elements in including NaN but eres count) returns ony the count of por-Na values ina Seres objec. {>>> ten(obia) Tuan abe ale Ihe presence of NaN vues and cunt tur zur oon_NaN vais f>>>0852.count() 4 S>n0bf3.hasnans Trae |>>20b3-count() a CChopter} : PYTHON PANDAS — 1 7 EXAMPLE NB Goer eo do apn ed ca TTR Prt te arnt at eae vee oes tn Attribute name object si2._—object s12 Data type Shape No. oF bytes No, oF dinensions Item size Has ais? Enpty > SOLUTION {import pandas as pd { statenents here to create objects S11 and s12 from previous exanples print ("Attribute nane \t\t Object si \t object s12") print(" \t\e ) print ("Data type( type) \", sLL.dtype, "\t\t", si12.dtype) print("Shape (shape) |\t", sUL.shape, "\t\e", s12.shape) print("No. of bytes (.nbytes) :\t", std.nbytes, "\t\t", s12.nbytes) Print("No. of dimensions(.ndin) :\t", s12.ndin, "\t\t", s12.ndim) print("Ttem size (.itensize) :\t", sid.itensize, "\t\t', s12.itensize) print("has NaNs? (.hasnans) :\t", si-hasnans, "\t\t", s12-hasnans) print("*enpty? (empty) \t", siL-enpty, "\t\t", s12.enpty) Output actribute nane object sil object siz Data type. dtype) ines Floats? Shape (.shape) » a G6. No. of bytes (.nbytes) 3 20 no. of dimensions(.ndim) : 1 a ten size (itemize): 8 4 as naNs? (hasnans) ralse True empty? Cenpty) False False18 ts 15 Accessing a Series Object ands Flere sd Series type objet sit in many ways. You cam access its ect itis o created Series Jo eee paividal elements and slices you have e Shap a os 8 07 28 0 E19) roving Series objet ()5, OPIS: a reese ow. Coie oo ny fa jo 81 jie 1088 ta aa jo ia Udevees ieta2 Figure 1.3 Some sample pandas Series Objects ror al the flowing operations we shall be using te sample objets shown in Fig. 13, 1.5.1. Accessing Individucl Elements from 0 Series Object “To access individual elements ofa Series object, you can give it with its name, ie, as ts index in square brackets along “Series Object nave>[«>ob350 Feb] 28 “Seo a theve object legal ndore are ured to access individual lent of those bjt. ‘As you seein above figure, we have use only valid or legal indexes (ie, which exist in series object) to access an element. Ifthe Series object has duplicate indexes, then giving an index with the Series object will return all the entries with that index, eg, see below Sosa fSssouat'e'T i dpa deve i 2 Seis object, all je a.78 fo soba") ni with he same e258 bas je il ees dtype: floats Rots Fae ag OTT fetype: Hoste Tegtnderes ar 90 12-refer stove hace heer. BUT if you try to give an index which is not a legal index for a Series object, it will give you an error, See the adjacent figure. RR ile “cipython snes 075057438560", Lbne 1, fn asa] (Chapter 1 TIONS PARAS — | 9 1.5.2. Extracting Slices from Series Object Like other sequences, you can extract slice too from a Series object to retrieve subsets, Let us see how you can extract slices from Series objects. Here, you need to understand an important thing about slicing, which is that Slicing takes place postion wise and noc the index wise ina series object. ‘Tounderstand this, let us consider the same Series objects as given in Fig. 13. Internally there isa pposition associated with element ~ first element gets the position as 0, second element gets the pposition as 1 and so on. Irrespective of their indexes, positions always start with O and go on like 1, 2, and so on. (ee Fig. 1.4) objs obis 7 Position index Dota Position Index _Doto Position Index Dota ¢ [Feb] 28 eo Ley a efL>| # 2 ana Hi a a ea 2 Darr : 2 7 2[ nba H ae 3 [ape 4 2 “india loots Rave poston mbes arg rm 0 cnwards a, or freeman, for slomenl and so on Figue 1.4 Postion number associated with each element of Series object When you have toextract slices, then you need to specify slices as start: end step ] like you do for other sequences, but the start and stop signify the positions of elements not the indexes. Consider following examples fpceersircy io Latype: ints? {étype: int32 Al other rules of slices apply, ie, NOTE you can specify steps, you can Teverse the elements, the range of A slie object is created from Series object using a syntax of Mice can be oubide fhe range of AIS acs mt mcs Tube inal Saks poste ae ‘object is also a panda Series type object. Even houghobIr has pesebiats 1) indexes 10,11, 12 but iSSsapiiesy ino Series((], stype: int32) ino * iio 20 is a8 not indox wise ‘gvoes int32INFORMATICS PRACTICES ~ a 20 Fans ch ti of sas hat tres the mur i = A 8 ae Se y- per ticket as utput seat cpensontegcr 30-78 Op tet oe an 1.6 Operations on Series Object 1 offers flexibility with storage as well as operations _ASeiess a one-dimensional trcture which offers fle nella pe aetna about how you can perform various types of operations on Pandas Series abject 1.6.1 Modifying Elements of Series Object “The data values ofa Series cbect can be easily modified through item assignment, ie, “Sersesobjects [inde ] = ‘Above assignment will change the data value ofthe given index in the Series object. «sertesobject3(start : stop] Above assignment will replace all the values falling in given slice. Consider following screenshots ew data value> ‘floatee ‘Chopter 1 : PYTHON PANDAS — 1 21 1.6.2 Renaming Indexes ‘You can even change or rename indexes of a Series obj it lexes of a Series object by assigning new index aray to its index attribute, i.e., :