DataWareHousing - ETL Project Life Cycle ( Simple to understand )
-> Datawarehousing projects are categorized into 4 types. 1) Development Projects. 2) nhancement Projects !) "igration Projects 4) Production support Projects. -> #he $ollowing are the di$$erent phases involved in 1) &usiness 'e(uirement )ollection * &'D ) 2) +ystem 'e(uirement )ollection * +'D ) !) Design Phase a) ,igh %evel Design Document * ,'D ) -) %ow level Design Document * %%D ) c) "apping Design 4) )ode 'eview .) Peer 'eview /) #esting a) 0nit #esting -) +ystem 1ntegration #esting. c) 0+er 2cceptance #esting * 02# ) 3) Pre - Production 4) Production * 5o-%ive ) #% project development li$e cycle.
&usiness 'e(uirement )ollection 6---------------------------------------------> #he -usiness re(uirement gathering start -y -usiness 2nalyst7 onsite technical lead and client -usiness users. -> 1n this phase7a &usiness 2nalyst prepares &usiness 'e(uirement Document * &'D ) *or) &usiness 'e(uirement +peci$ications * &'+ ) -> &' collection ta8es place at client location. -> #he o9p $rom &' 2nalysis are -> &'+ 6- &usiness 2nalyst will gather the &usiness 'e(uirement and document in &'+ -> +'+ 6- +enior technical people *or) #% architect will prepare the +'+ which contains s9w and h9w re(uirements. #he +'+ will includes a) :9+ to -e used * windows or uni; ) -) 'D&"+ re(uired to -uild data-ase * oracle7 #eradata etc ) c) #% tools re(uired * 1n$ormatica7Datastage ) d) :%2P tools re(uired * )ognos 7&: ) #he +'+ is also called as #echnical 'e(uirement +peci$ications * #'+ ) Designing and Planning the solutions 6------------------------------------------------> #he o9p $rom design and planning phase is a) ,%D * ,igh %evel Design ) Document -)%%D * %ow %evel Design ) Document ,%D * ,igh %evel Design ) Document 6 2n #% 2rchitect and D<, 2rchitect participate in designing a solution to -uild a D<,. 2n ,%D document is prepared -ased on &usiness 'e(uirement. %%D * %ow %evel Design ) Document 6 &ased on ,%D7a senior #% developer prepare %ow %evel Design Document #he %%D contains more technical details o$ an #% +ystem. 2n %%D contains data $low diagram * D=D )7 details o$ source and targets o$ each mapping. 2n %%D also contains in$ormation a-out $ull and incremental load. 2$ter %%D then Development Phase will start Development Phase * )oding ) 6--------------------------------------------------> &ased an %%D7 the #% team will create mapping * #% )ode ) -> 2$ter designing the mappings7 the code * "appings ) will -e reviewed -y developers. )ode 'eview 6-> )ode 'eview will -e done -y developer. -> 1n code review7the developer will review the code and the logic -ut not the data. -> #he $ollowing activities ta8es place in code review -> >ou have to chec8 the naming standards o$ trans$ormation7mappings o$ data etc.
-> +ource and target mapping * Placed the correct logic or not in mapping ) Peer 'eview 6-> #he code will reviewed -y your team mem-er * third party developer ) #esting6-------------------------------#he $ollowing various types testing carried out in testing environment. 1) 0nit #esting 2) Development 1ntegration #esting !) +ystem 1ntegration #esting 4) 0ser 2cceptance #esting 0nit #esting 6-> 2 unit test $or the D<, is a white &o; testing71t should chec8 the #% procedure and "appings. -> #he $ollowing are the test cases can -e e;ecuted -y an #% developer. 1) ?eri$y data loss 2) @o.o$ records in the source and target !) Dataload91nsert 4) Dataload90pdate .) 1ncremental load /) Data accuracy 3) veri$y @aming standards. 4) ?eri$y column "apping -> #he 0nit #est will -e carried -y #% developer in development phase. -> #% developer has to do the data validations also in this phase. Development 1ntegration #esting -> 'un all the mappings in the se(uence order. -> =irst 'un the source to stage mappings. -> #hen run the mappings related to dimensions and $acts. +ystem 1ntegration #esting 6-> 2$ter development phase7we have to move our code to A2 environment. -> 1n this environment7we are giving read-only permission to testing people. -> #hey will test all the wor8$lows. -> 2nd they will test our code according to their standards. 0ser 2cceptance #esting * 02# ) 6-> #his test is carried out in the presence o$ client side technical users to veri$y the data migration $rom source to destination. Production nvironment 6---------------------------------> "igrate the code into the 5o-%ive environment $rom test environment * A2 nvironment ).
