0% found this document useful (0 votes)
41 views12 pages

Age 40

The document describes functions to analyze and summarize data from a CSV file on individuals under age 40 in the state of Johor, Malaysia. The functions read in the CSV file, clean the data by removing unnecessary columns and formatting date fields, calculate age, filter to individuals under 40, and output a new CSV file. For Johor data, it returns the first 3 rows of the cleaned dataframe, showing 453,743 rows met the age criteria. The same process is then shown for Kedah state data, returning 326,639 rows.
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
41 views12 pages

Age 40

The document describes functions to analyze and summarize data from a CSV file on individuals under age 40 in the state of Johor, Malaysia. The functions read in the CSV file, clean the data by removing unnecessary columns and formatting date fields, calculate age, filter to individuals under 40, and output a new CSV file. For Johor data, it returns the first 3 rows of the cleaned dataframe, showing 453,743 rows met the age criteria. The same process is then shown for Kedah state data, returning 326,639 rows.
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 12

7/12/23, 4:20 PM age40

STR Analytics 2023


In [1]: import time
start = time.time()
print(start)

1689148354.0411217

In [2]: import pandas as pd


import numpy as np
import os
import plotly.express as px
import datetime
pd.set_option('display.max_columns', None)
pd.set_option('display.max_rows', None)

Function
In [3]: def str1():
global m1
global m2
global m3
global m4
global m5
global df

m1 = 'Output/df_'
m3 = '.csv'
m4 = 'Age40'
m5 = '_'
m = m1+m2+m3
df = m4+m5+m2
df = pd.read_csv(m,
sep='|',
skipinitialspace=True,
dtype = 'str',
engine='python',
quoting=3)
df = df.drop(['Indeks', 'PekerjaanPemohon',
'JantinaPemohon', 'StatusKahwin',
'Pendapatan', 'Alamat1',
'Alamat2', 'Alamat3',
'Poskod','Negeri',
'KOD_NG','Daerah','KOD_DP'], axis=1)

def str2():
global df
global length

df['TahunLahir'] = df['NoKPPemohon'].str[:2]
df['TahunLahir'] = '19'+df['TahunLahir']
df.loc[df['TahunLahir'] == '1900', 'TahunLahir'] = '2000'
df.loc[df['TahunLahir'] == '1901', 'TahunLahir'] = '2001'
df.loc[df['TahunLahir'] == '1902', 'TahunLahir'] = '2002'
df.loc[df['TahunLahir'] == '1903', 'TahunLahir'] = '2003'
df.loc[df['TahunLahir'] == '1904', 'TahunLahir'] = '2004'
df.loc[df['TahunLahir'] == '1905', 'TahunLahir'] = '2005'
df.loc[df['TahunLahir'] == '1906', 'TahunLahir'] = '2006'
localhost:8888/nbconvert/html/OneDrive/MOF/FED/SEKSYEN EK/MODEL/STR ANALYTICS/age40.ipynb?download=false 1/12
7/12/23, 4:20 PM age40

df.loc[df['TahunLahir'] == '1907', 'TahunLahir'] = '2007'


df.loc[df['TahunLahir'] == '1908', 'TahunLahir'] = '2008'
df.loc[df['TahunLahir'] == '1909', 'TahunLahir'] = '2009'
df.loc[df['TahunLahir'] == '1910', 'TahunLahir'] = '2010'

today = datetime.date.today()
year = today.year
df['TahunSemasa'] = year

df = df.astype({'TahunLahir':'int'})
df['Umur'] = df['TahunSemasa']-df['TahunLahir']
df = df.loc[df['Umur'] < 40]
length = len(df)

df = df.drop(['TahunLahir', 'TahunSemasa'], axis=1)


df['Negeri'] = m2

def str3():
global path
global df

path = m4 + '/' + m2 + m3
df.NoKPPemohon = df.NoKPPemohon.astype('str')
df.to_csv(path, sep = ',',
index=False)

Johor Darul Takzim


In [4]: m2 = 'Johor'
str1()
str2()
print('Number of selected rows are',length)
str3()
df.head(3)

Number of selected rows are 453743


Out[4]: NoKPPemohon NamaPemohon NoTelPemohon Emel Alamat Umur

NO. 146,
JALAN
NUR HANISYA
SONGKET
0 000101010364 SYAFIRA BINTI 01128811236 / [email protected] 23
2, TAMAN
MUSTAFA
ANJARIA,
81000...

NO2JALAN
PERMATA
TAMAN
HWI SHENG
1 000101010743 0185724630 / [email protected] NORA 23
LONG
81800,
ULU
TIRAM, ...

NO 362
JALAN
PUTERI BAYAN,
2 000101011156 SYAFINAZ BINTI 0132240103 / [email protected] FELDA 23
JAMIUN MEDOI,
SEGAMAT,
8500...

localhost:8888/nbconvert/html/OneDrive/MOF/FED/SEKSYEN EK/MODEL/STR ANALYTICS/age40.ipynb?download=false 2/12


7/12/23, 4:20 PM age40

Kedah Darul Aman


In [5]: m2 = 'Kedah'
str1()
str2()
print('Number of selected rows are',length)
str3()
df.head(3)

Number of selected rows are 320639


Out[5]: NoKPPemohon NamaPemohon NoTelPemohon Emel Alamat Umu

NO 70
JALAN
NURUL
KEMUNTING
0 000101020650 SYAFINAZ BINTI 0184674326 / [email protected] 2
1/6, TAMAN
FAUZLIM
KEMUNTING
, K...

LOT 112-A ,
LORONG
MUHAMMAD
TENANG,
1 000101020837 AFIQ IKMAL BIN 0174935741 / [email protected] 2
KAMPUNG
ZAINI
AIR PUTIH,
...

JALAN
KAMPUNG
ABDUL HALIM YOOI,
0184068500 /
2 000101020845 BIN ABDUL [email protected] MUKIM 2
0184068500
RAHMAN BOHOR,
LANGKAWI,
070...

Kelantan Darul Naim


In [6]: m2 = 'Kelantan'
str1()
str2()
print('Number of selected rows are',length)
str3()
df.head(3)

Number of selected rows are 297439

localhost:8888/nbconvert/html/OneDrive/MOF/FED/SEKSYEN EK/MODEL/STR ANALYTICS/age40.ipynb?download=false 3/12


7/12/23, 4:20 PM age40

Out[6]: NoKPPemohon NamaPemohon NoTelPemohon Emel Alamat Umur

KG HAJI
WAN
OMAR , , ,
FAEZNOR BINTI
0 000101030082 01161039915 / [email protected] 16700 23
AWANG
CHERANG
RUKU
KELA...

KAMPUNG
BUKIT
AHMAD
JERING, , ,
FATHUL HAKIMI
1 000101030189 0139512671 / [email protected] 17610 23
BIN
KUALA
MUHAMMAD
BALAH
KE...

KAMPUNG
PAUH
NUR AFIFAH BUTUT,
2 000101030314 BINTI 0142065248 / [email protected] PERINGAT, 23
ZAINUDIN KOTA
BHARU,
1640...

Melaka Bandaraya Bersejarah


In [7]: m2 = 'Melaka'
str1()
str2()
print('Number of selected rows are',length)
str3()
df.head(3)

Number of selected rows are 121021


Out[7]: NoKPPemohon NamaPemohon NoTelPemohon Emel Alamat

BATU 23
PASIR
AHMAD FITHRI
GEMBUR,
0 000101040299 BIN 01131783985 / [email protected]
KAMPUNG
KHAIRUDDIN
TANJUNG
BIDARA, ...

LOT 1558
JALAN
MUHAMMAD HAJI
1 000103040161 REZAL BIN 0192493272 / [email protected] OTHMAN,
ROSLAN BALIK
BUKIT,
BUKIT...

NO 8
JALAN
SAUJANA
ALEX A/L
2 000103040321 0146530487 / [email protected] 2/10,
THOMAS
TAMAN
SAUJANA
SEK 2,,...

localhost:8888/nbconvert/html/OneDrive/MOF/FED/SEKSYEN EK/MODEL/STR ANALYTICS/age40.ipynb?download=false 4/12


7/12/23, 4:20 PM age40

Negeri Sembilan Darul Khusus


In [8]: m2 = 'N9'
str1()
str2()
print('Number of selected rows are',length)
str3()
df.head(3)

Number of selected rows are 150143


Out[8]: NoKPPemohon NamaPemohon NoTelPemohon Emel Alamat Umur Ne

NO 6
JALAN
MUHAMMAD DAHLIA
0137870165 /
0 000101050091 AIRIL BIN [email protected] KAMPUNG 23
01160579756
MAZLAN ANAK AIR
, DURIAN
DA...

167,
JALAN
MANTAU
NURUL FATIN
INDAH
1 000101050470 FATIHAH BINTI 0142353951 / [email protected] 23
3/2,
SHAMSUDIN
TAMAN
MANTAU
INDA...

NO35
LORONG
MOHAMAD SRI
FAIZ IMAN BIN ANGGERIK
2 000101060513 0182588540 / [email protected] 23
MOHAMAD 2/5 ,
NOOR TAMAN
SRI
ANGGE...

Pahang Darul Aman


In [9]: m2 = 'Pahang'
str1()
str2()
print('Number of selected rows are',length)
str3()
df.head(3)

Number of selected rows are 222646

localhost:8888/nbconvert/html/OneDrive/MOF/FED/SEKSYEN EK/MODEL/STR ANALYTICS/age40.ipynb?download=false 5/12


7/12/23, 4:20 PM age40

Out[9]: NoKPPemohon NamaPemohon NoTelPemohon Emel Alamat Umur

NO 11,
LORONG
MUHAMMAD
KAMPUNG
AMERUL IQMAL
0 000101020503 01131784531 / [email protected] PADANG 23
BIN MOHD
JAYA 28,
ROSLAN
TAMAN
KA...

NO 60,
LORONG
ENA MASNIE 17 TAMAN
1 000101060222 01111989223 / [email protected] 23
BINTI YUSRI SATELITE
BESERAH, ,
, 2...

RUMAH
PPRT
KAMPUNG
NUR IMAN BIN
2 000101060599 01151218680 / [email protected] BUKIT 23
SABLIL
PAMPONG,
, , 27400
DO...

Pulau Pinang Pulau Mutiara


In [10]: m2 = 'Penang'
str1()
str2()
print('Number of selected rows are',length)
str3()
df.head(3)

Number of selected rows are 177184


Out[10]: NoKPPemohon NamaPemohon NoTelPemohon Emel Alamat Um

NO 96
MUHAMMAD PERMATANG
0 000101020271 HAFIZ IZUAN 0175051474 / [email protected] TOK LABU,
BIN ZAMRI 13100 PENAGA,
PULAU ...

NO 46 TAMAN
NORSYAZATUL
CEBDERAWASIH
1 000101070076 NAJWA BINTI 0146040188 / [email protected]
16, , , 14300
ABDUL HALID
NIBONG ...

4-02 RUMAH
NURUL ASYIKIN PANGSA DESA
2 000101071052 BINTI 0175664319 / [email protected] PROMINENCE,
ABDULLAH BANDAR
PERD...

Perak Darul Ridzuan


In [11]: m2 = 'Perak'
str1()
localhost:8888/nbconvert/html/OneDrive/MOF/FED/SEKSYEN EK/MODEL/STR ANALYTICS/age40.ipynb?download=false 6/12
7/12/23, 4:20 PM age40

str2()
print('Number of selected rows are',length)
str3()
df.head(3)

Number of selected rows are 336739


Out[11]: NoKPPemohon NamaPemohon NoTelPemohon Emel Alamat Umur N

NO 238
JALAN
NURULSHAZLIN 19A ,
0 000101080020 BINTI SHAH 0175828385 / [email protected] TAMAN 23
ARIFIN KUNING
SARI,
JALAN AI...

4,BAGAN
PANCHOR,
ANG MAY , , 34900
1 000101080098 0183661020 / [email protected] 23
LIANG PANTAI
REMIS
PERAK

LOT3394,
KAMPUNG
MUHAMMAD
BUKIT
2 000101080549 ARIF BIN MOHD 0134361395 / [email protected] 23
BERTAM,
ZAILANI
TAIPING,
34500 ...

Perlis Indera Kayangan


In [12]: m2 = 'Perlis'
str1()
str2()
print('Number of selected rows are',length)
str3()
df.head(3)

Number of selected rows are 39411

localhost:8888/nbconvert/html/OneDrive/MOF/FED/SEKSYEN EK/MODEL/STR ANALYTICS/age40.ipynb?download=false 7/12


7/12/23, 4:20 PM age40

Out[12]: NoKPPemohon NamaPemohon NoTelPemohon Emel Alamat Umur Neger

KAMPUNG
TIT TINGGI
SUZANA BINTI JALAN
0 000102020310 0175523250 / [email protected] 23 Perl
AYUB SERI
SEPAKAT ,
, , 02...

NO 11
LORONG
4, TAMAN
SITI ZAHARAH 0132554460 /
1 000104090048 [email protected] EMAS, 23 Perl
BINTI ALWI 049384206
BESERI,
02450
KANG...

NO 17,
JALAN
MUHAMMAD
BAKAU 6,
AIDIL IDHAM
2 000104090101 0169423167 / NaN RPA 23 Perl
BIN MOHD
KUALA
FHAUZI
PERLIS, ,
0200...

Selangor Darul Ehsan


In [13]: m2 = 'Selangor'
str1()
str2()
print('Number of selected rows are',length)
str3()
df.head(3)

Number of selected rows are 614362


Out[13]: NoKPPemohon NamaPemohon NoTelPemohon Emel Alamat Umur

LOT 2125,
SITI NOOR FATIN JALAN HAJI
0 000101031384 NABILA BINTI 0172451488 / [email protected] IBRAHIM,KG 23
MOHD IZWANI JENJAROM,
, , ...

SG 10/10
TAMAN
ROOBENDHIRAN 0167892780 / PINGGIRAN,
1 000101070703 [email protected] 23
A/L VEERAPPA 0167892783 BATU
CAVES
68100, SE...

TB-3
BLOK31,
NURUL AINA BANDAR
2 000101081760 BINTI AHMAD 0174404808 / [email protected] BARU 23
SHAH SULTAN
SULAIMAN,
4200...

Terengganu Darul Iman


localhost:8888/nbconvert/html/OneDrive/MOF/FED/SEKSYEN EK/MODEL/STR ANALYTICS/age40.ipynb?download=false 8/12
7/12/23, 4:20 PM age40

In [14]: m2 = 'Terengganu'
str1()
str2()
print('Number of selected rows are',length)
str3()
df.head(3)

Number of selected rows are 193385


Out[14]: NoKPPemohon NamaPemohon NoTelPemohon Emel Alamat Umur

NO 54 PPRT
MUHAMMAD JALAN BARAT,
0 000101101645 ALIF BIN MOHD 0179334083 / NaN , , 21450 23 T
AL ANSARI CHALOK
TEREN...

C 220
MUHAMMAD KAMPUNG
IMAN ARSYAD KIJING, , ,
1 000101110445 0139831405 / [email protected] 23 T
BIN MOHD 21600
KHAIRI MARANG
TERENGGANU

LOT 4233
KAMPUNG
NUR AFIQAH
2 000102110180 0133334532 / [email protected] PECAH 23 T
BINTI NORMI
ROTAN, BATU
RAKIT, KUAL...

Sabah Negeri di Bawah Bayu


In [15]: m2 = 'Sabah'
str1()
str2()
print('Number of selected rows are',length)
str3()
df.head(3)

Number of selected rows are 398470

localhost:8888/nbconvert/html/OneDrive/MOF/FED/SEKSYEN EK/MODEL/STR ANALYTICS/age40.ipynb?download=false 9/12


7/12/23, 4:20 PM age40

Out[15]: NoKPPemohon NamaPemohon NoTelPemohon Emel Alamat Umur

WARUNG
PADANG
NUR NAJWA JAWA NO 6
0 000101090042 0137496330 / [email protected] 23
BINTI NORIZAN BLOCK
SEGRD ,,
FLR LOR...

KG TENGAH
MOHD. KINARUT, , ,
1 000101120061 HIDAYATULLAH 01126366401 / [email protected] 89600 23
BIN ISMAIL PAPAR
SABAH

KG
SONSOGON
MASTINA BINTI PALIU, , ,
2 000101120272 01171158946 / [email protected] 23
ALOG 89100 KOTA
MARUDU
SABAH

Sarawak Bumi Kenyalang


In [16]: m2 = 'Sarawak'
str1()
str2()
print('Number of selected rows are',length)
str3()
df.head(3)

Number of selected rows are 397584


Out[16]: NoKPPemohon NamaPemohon NoTelPemohon Emel Alamat Umur

6E-JALAN
TIONG
SHANIA TAN HUA, , ,
0 000101130112 01156931690 / [email protected] 23 S
CHII YI 96000
SIBU
SARAWAK

NO 63,
LOT 690, ,
JALAN
1 000101130294 LEE XIN 01155036146 / [email protected] 23 S
CALYCINA,
TAMAN
PIN FO...

134
KAMPUNG
SEJINGKAT,
NUR MANISYA
2 000101130382 0135208304 / NaN JALAN 23 S
BINTI JALANI
BAKO, ,
93050
KUC...

Kuala Lumpur Ibu Negara


localhost:8888/nbconvert/html/OneDrive/MOF/FED/SEKSYEN EK/MODEL/STR ANALYTICS/age40.ipynb?download=false 10/12
7/12/23, 4:20 PM age40

In [17]: m2 = 'KL'
str1()
str2()
print('Number of selected rows are',length)
str3()
df.head(3)

Number of selected rows are 188963


Out[17]: NoKPPemohon NamaPemohon NoTelPemohon Emel Alamat

P-35 JALAN
MERANT
THARSHINI A/P TAMAN
0 000101080370 0122431200 / [email protected]
RAJENDRAN SETAPAK
KUALA
LUMPUR..

NO 40-4-1
NURUL
KELUMPUK
FATIHAH AYUNI
1 000101101012 0176155478 / [email protected] PINANG
BINTI MOHD
SETIAWANGSA
HOD
54200..

DLUQMAN J -11-14 PPR


ADAM 0149247613 / PINGGIRAN
2 000101101565 [email protected]
FADLULLAH BIN 0143145865 BUKIT JALIL
SANSUDIN JALAN, BUKI..

Labuan
In [18]: m2 = 'Labuan'
str1()
str2()
print('Number of selected rows are',length)
str3()
df.head(3)

Number of selected rows are 8596


Out[18]: NoKPPemohon NamaPemohon NoTelPemohon Emel Alamat Um

KAMPUNG
GANGGARAK,
HASRUL BIN
0 000107120345 0195982824 / [email protected] , , 87000
MISE
LABUAN W.P.
LABUAN

NO A8-32
TINGKAT 7,
SITI NOR
RUMAH
1 000109121712 FAZIRAH BINTI 01131689821 / [email protected]
MESRA
SAHRI
RAKYAT , KG
SG...

KG BELUKUT,
AFFENDY BIN , , 87000
2 000128150025 0196325537 / [email protected]
BAHARI LABUAN W.P.
LABUAN

localhost:8888/nbconvert/html/OneDrive/MOF/FED/SEKSYEN EK/MODEL/STR ANALYTICS/age40.ipynb?download=false 11/12


7/12/23, 4:20 PM age40

Putrajaya Bandar Pentadbiran


In [19]: m2 = 'Putrajaya'
str1()
str2()
print('Number of selected rows are',length)
str3()
df.head(3)

Number of selected rows are 6781


Out[19]: NoKPPemohon NamaPemohon NoTelPemohon Emel Alamat Umu

APARTMENT
NUR FATIN BLOK A1
0197719816 /
0 000117140374 SYAZWANI [email protected] TO3-UO5 2
0388889816
BINTI SIDEK PRESINT 16,
PUTRAJAY...

C-T08-U20 ,
AMAR BLOCK C
1 000202050613 ZAQWAN BIN 01126738757 / [email protected] PANGSAPURI 2
MOHD NADZRI , PUTRA
DAMAI, ...

A-08-11
PPAM
MUHAMMAD
SERULING,
2 000207010653 DARWISY BIN 0169559432 / [email protected] 2
NO 2 JALAN
NOR AZMAN
SERULING
P5A...

Tahniah Mr Harun!
In [20]: end = time.time()
time = end - start
minute = time/60

import numpy as np
time = np.round(time,2)
minute = np.round(minute,2)
print('Total time elapsed is',time,'s and equivalent to',minute,'minutes')

Total time elapsed is 267.89 s and equivalent to 4.46 minutes

localhost:8888/nbconvert/html/OneDrive/MOF/FED/SEKSYEN EK/MODEL/STR ANALYTICS/age40.ipynb?download=false 12/12

You might also like