Abinitio Scenarios Question
Abinitio Scenarios Question
How to find out malformed data?.... For example we have 1000000 records in that I need only
malformed data..
Like
Share
Hazra Ayon I would use is_valid metadata function in next transform component and put threshold value as
never reject.
Gagandeep Singh i think we can use m_dump with -show-partial option.
January 31 at 2:43am · Like
Nikhil Kumar Abinitio itself shows that record number in its error message
April 7 at 9:59pm · Like
can anyone explain about mvs file , intermediate file ,db_update in abinitio
thanx in advance
Hi, what is the max limit of temporary files created using sort?
i/p*3/maxcore
Hi All,i have in0 todays file & in1 yesterdays file,i have some new records in the todays file and
updated records also in the todays file,if i use join here new records will go unused0 port ,non
matching records from in1 go to unused1 port,here how can we write a condition for update records
going to o/p port?how can we write a flag condtions here?
Like
Share
Ramesh Reddy
December 1, 2014 · Edited
Share
Sudheer Kumar Baratam I guess ur actually trying to refer to partition by expression? Is dat r8
Hi All,I have a i/p file with 20 records ,i want to send all these records to rejects port using filter by
expression?
Like
Share
Sarvesh Mishra if (1==1) force_error("reject")
Ramesh Reddy
November 27, 2014
how to print a 5th record from a file & also having dml from backend?
i mean like m_dump pathofdml pathofdatafile -start 5 -end is it the right
syntax?help me
Like
Share
2 people like this.
Madhusudhan Reddy in view data u can select 5th recrd.
m_dump dml data -record 5.
Ramesh Reddy thank u madhu
Srikanth Arasavalli M_dump dml file - record start5 end 6
Share
Arjunreddy Syamala
November 24, 2014
Share
Shadab Mohd Replicate the original flow and then in one flow Use dedup with keep first record and then in
reformat give seq no and join with another flow ....
Aman Madan When are you going to use help and your brains? Kindly tell me what options have thou tried
yourself first. You really won't learn much if you don't practise and fail
Nikhil Kulkarni read data by character
record
char(1) a;
char(1) b;
char("\n") c;
end;
Then u can declare a variable in reformate and save value of field "c"
Also u can reverse the order of fields c then b then a
let me know if this helps
Hi All,i have a duplicate records in a file using rollup we can eliminate duplicate records using built in
functions like first & last in rollupp but i want only uniq records from a file without using
dedupsort.how can we achieve?help
Like
Share
Arjunreddy Syamala
November 22, 2014
.i have 4 way multifile I want 3 rd record from 1st partion and from 4th partition 5th record in the
output how can we do this?
Like
Share
Irfan Patel Please try this i am not sure though...m_dump <DML> <MFS_FILE> -select "this_partition()==1" -record 3
Arjunreddy Syamala
November 22, 2014
1. Hi,What is diff b/w fast load and multiload in terrada,explain?which one is better?
Channi Tata 1.Best practice is to use plan. 2. Use any other scheduler available like ctrlM, autosys, eventengine and
use dependencies.
3.The next best thing is to use a shell script and do following...(chk syntax, error handling)...See More
Shadab Mohd If u want to run second graph irrespective of first graph's success or failure then write air sandbox
run command sequntially without checking command's output...
Mahesh Kumar How To Write Ascript Please explain..
Arjunreddy Syamala
November 19, 2014
Hi All, I have a graph ,i want to see all objects like(xfr's,dmls etc) belongs to the particular graph?
how can we find it?pls help me.
Like
Share
Hazra Ayon Right click on the object name in ur sandbox pane..... u will get a option like Required files..... go into
it....u will get all the files required bythat graph to run.
Arjunreddy Syamala thank u Hazra Ayon
Hazra Ayon Check it b4 saying thanks...i may b wrong..😎
November 19, 2014 at 8:39pm · Like
Madhusudhan Reddy check this air category list-member /abc/mp
Madhusudhan Reddy air sandbox get required files cmnd also .
Channi Tata required files thingy on graph works only if the graph is non-generic. i.e. XFRs and DMLs are not
parameterized.
If parameterized/generic graph then do air command for required list on pset where all parameters are defined.
make sure proper config file is included if necessary.
Arjunreddy Syamala
October 31, 2014
Share
Madhusudhan Reddy multuple serial files with same dml..then we can use adhoc multifiles.
Arjunreddy Syamala thank u madhu
Share
Madhusudhan Reddy string_concat(tmp.combine,",",in.name);
this will wrk.
Share
Channi Tata Read input as... string and array
Use normalizer with array position value as incremented value. Check ab help file for example. It is explained quite
clearly there.
Hi ,What is .cfg file ?where we will use this & where we find this file?
Like
Share
Arjunreddy Syamala
October 30, 2014
i have dml like i want to insert the data for this dml in the file
record
decimal(",")custid;
decimal(",")ntrans;
record
date("YYYY.MM.DD")transdate;
decimal(",")amount;
end.transaction(ntrans);
end;
Like
Share
Shashank Vaze Looks like a nested dml....you may check Normalize component for this...so that it can vector the
data into the sub dml....when you view the data on this file, you should be able to see a + symbol for nested DML
data...
Arjunreddy Syamala yes it is nested dml ,i want to prepare test data for this dml,how to insert data like
1,02,1990.08.23, give me an example
Shashank Vaze Btw ... this dml is copied from Ab Initio help document....so you can better use help....
Nikhil Kumar use create data/generate records component then u can normalize the data to make it multiple
records.
In utility mode at the time of loading data into a table constaints & indexes also will disable & will it
recreate i mean after loading data and recreate indexes at the time of constructing a table is it
correct?
Like
Share
Arjunreddy Syamala
October 29, 2014
Hi,In Utility mode constraints will disable at the time of loading into the database.there is a chance to
insert the duplicate data in the table ,should we handle duplicate data before loading into the table ?
which interface is best?pls help
Like
Share
Shashank Vaze For lower loads of data, api is best, but if the data volume is huge, you can use utility. It would be
advisable to handle duplicates, data cleansing, data filtration and data transformation before loading the data...
possibly in that order...
Shashank Vaze Yes
Arjunreddy Syamala
October 28, 2014
Share
Nikhil Saha Use normalize with length as number of subjects...key-student no
Arjunreddy Syamala can we do using rollup ?
Nikhil Saha No..you can use meta pivot as well...which will transpose your rows to columns
Arjunreddy Syamala i used metapivot and after that i used reformat to drop the fieldname column,for ex- record
format like this fieldname,fieldvalue,studentno got the result. without using reformat after the metapivot can we
do this?
Arjunreddy Syamala
October 27, 2014 · Edited
I want to run a graph 10 times how can we write a shell script ?how can we loop it?
Like
Share
Ankur Jain Use looping plans
Anvesh Vuppu Why 10 times immediately? If it needs to be run in some intervals, You can also control this from
scheduler.
Arjunreddy Syamala
October 25, 2014 · Edited
Hi All,i try to create an empty multifile it is failing what is the reasons behis this?
Like
Share
Ankur Jain How did you try creating it ? And whats the error message ?
Try m_touch
Arjunreddy Syamala
October 27, 2014
Share
Arjunreddy Syamala
October 21, 2014
Channi Tata Do cat on mfs file. You'll get list of files with absolute path. Now do cat on the partition you want.
Channi Tata I guess u can also use m_dump. Somthing like
Nikhil Saha M_cat <filename> -partition <number>
Share
Ankur Jain Y dont you use plans to implement this ?
Channi Tata It will work...but as ankur said, itz better to use plans unless u have a specific requirement to use a
script
Arjunreddy Syamala we can use the plans to run sequentially graphs,but above one will or not?
October 21, 2014 at 2:43pm · Edited · Like
Channi Tata pl re-read my comment again.
Ankur Jain Arjunreddy Syamala - No offence, but your 'will or will not' questions will be best answered if you try
implementing the same. Moreover, that will give you a confidence on the features available !
Channi Tata Ankur Jain like emoticon
Arjunreddy Syamala...if it is a yes/no case run it once in dev environment using some sample graph. that's the
way to learn than getting some instant answers.
Arjunreddy Syamala
October 21, 2014
Hi ALL,we can run the .ksh files through wrapper script.can we also run the .mp file through wrapper
script?
Like
Share
Channi Tata if it is air command that you are using then both .mp or .ksh will work.for more information check abinitio
help
Arjunreddy Syamala
October 20, 2014
i created a save file, after by using air object load command loading save file into the dev eme only?
after that what i have to do?i want to move to production env how can we do,please give the steps
after creating save file?
Like
Share
Arjunreddy Syamala
October 19, 2014
Share
Ismail Mohammed m_cat has an option .. (something like m_cat -patition n) to select data from the parition 'n'
Sam Jacob Try m_dump... It has several options making it preferable over other options....
Arjunreddy Syamala
October 18, 2014 · Edited
I have a serial file ename salary with 100 records I want only 50 to 60 record in o/p ?can we do this
using single filterbyexpression or single reformat?
Like
Share
Nikhil Saha Yes..use next_in_sequence>40 in either filter by expression or select parameter of reformat...depends
which 50/60 records you want.If if first 50 then change your expression accordingly
i have lookup multifile and i have a source multifile. source 1st data partition matching records in
lookup 2nd data partition how can i retrive them?
Like
Share
Ismail Mohammed Both of them should b partitioned on the same key for accurate retrievals
Anvesh Vuppu if the data is less, use lookup instead of lookup_local
Srikanth Arasavalli Use lookup .... Lookup local work with its corresponding partitions ...
Arjunreddy Syamala
October 17, 2014
Share
Channi Tata ...every time you connect to abinitio EME, i.e. whenever you use "air" commands or db related
commands.
Share
Ismail Mohammed whats ur database ? For Teradata, u can use BTEQ
Nikhil Saha For db2 u can use...db2 connect...make sure your db2 path is set in .profile
Arjunreddy Syamala Oracle database
Chakri Majety
October 11, 2014 · Edited
HI,
Does anyone have Abinitio Help file ? Can someone please send it to me?
Thanks,
Chakri Majety
Like
Share
Gopi Nath me tooo plz send @ [email protected] plzz
Share
Ismail Mohammed Please watch all the 12 very short videos in this playlist. This should answer most of
your datawarehouse fundamental questions. Worth the time.
http://www.youtube.com/watch?v=cwpL-3rkRYQ...
Arjunreddy Syamala
October 16, 2014
I have a primarykey in the table,why we are creating a surrogate key pls help me
Like
Share
Arjunreddy Syamala
October 11, 2014
how to write a force error function to fail if there are duplicates in input file.
Like
Share
Ismail Mohammed I think you are looking for force_abort and not force_error
how can u measure(report) the data skew of a particular graph? can we see the data skew report after running a
graph?
Like
Share
Sivakumar Ponnusamy you can set AB_REPORT with appropriate value to get skew
Arjunreddy Syamala
October 10, 2014 · Edited
man Madan You will have to define that as env parameter. Then graph will pick from environment when u call the
graph thru wrapper
Arjunreddy Syamala without wrapeer i want to run from backend through mp file
Aman Madan Mp never runs. You can run ksh.
Arjunreddy Syamala air sandbox run graphname parametername value
Aman Madan In this case define it as keyword parameter. And u can run like this
Arjunreddy Syamala thank u madan
Aman Madan Wc
Arjunreddy Syamala
October 10, 2014
Share
Arjunreddy Syamala
October 8, 2014
Share
Arjunreddy Syamala
October 8, 2014
what is candidate key ?is it a primary key or what ?
Like
Share
Arjunreddy Syamala
October 8, 2014
Share
Arjunreddy Syamala
September 26, 2014
Share
Arjunreddy Syamala it is the only one purpose is there any other use?some body asked one more advantage?
Ankur Jain 1. To do some pre-processing ( like checking any pre-requisites; setting run specific parameters
etcc..) outside the GDE environment before you run a plan/graph
2. To design/control the flow of graphs( you can do that using plans as well though !)
3. To do any post processing once the execution is compeleted...See More
Arjunreddy Syamala
September 29, 2014
Share
Arjunreddy Syamala
September 26, 2014
how to convert a string to dateformat in abinitio i have i/p file dml like this
decimal(",")id;
string(",") name;
i want to conver this name to date formalt like this date("yyyy/mm/dd")?
Like
Share
Channi Tata As long as data is right(months b/w 1-12, dates b/w 1-31), u can cast it directly using following
out.date_col::(date(yyyy/mm/dd))(in.string_col)...See More
Nikhil Saha out.name :: (date("yyyy/mm/dd"))(date("yyyymmdd"))in.name;
Arjunreddy Syamala
September 26, 2014
Arjunreddy Syamala it is asked by some one in interview
A vauge way would be to concatenate the data of each partition manually into a serial file....See More
Arjunreddy Syamala Ankur i asked my fnd he has given the answer is m_cat multifilename >> serailfilename
Nikhil Saha m_cat mfs_filename > serial_dir/serial_filename
Arjunreddy Syamala
September 22, 2014
what is the diff b/w input parameters and pld what i understood is pdl must be enclosed in [ ] this
bracket please explain?
Like
Share
Arjunreddy Syamala
September 17, 2014 · Edited
i loaded 200 record at the of the day i want to load from 201 from next day?
tell me how to do?
rjunreddy Syamala manually do i need to code in fbe
Arjunreddy Syamala madhu can we do using o/p table component in insert statement creating any sequence
no?
Shadab Mohd Use sort,scan then fbe ...nd put records after count>200 in a file having same name and location
as ip file it will over write the file ....thus your job will run daily nd load only 200 records in a day ....
Shadab Mohd You can achieve this without scan also ...
Puneet Sharma Create a graph where you Unload the table again, join with new file... load unused records in
append mode to the table.
Arjunreddy Syamala
September 18, 2014 · Edited
i have a file loading into database its failed after loading 5 records
i rectified error and again i run the graph i want to load from 6th record how can we do that?
Like
Share
Jai Chidambaram try to use a commit table
Arjunreddy Syamala
September 18, 2014 · Edited
Share
Nikhil Saha One deletes and loads..other appends to existing load
Arjunreddy Syamala
September 17, 2014 · Edited
Share
Arjunreddy Syamala
September 15, 2014
what is key_change in rollup when we will use this give an example i read previous i/p record and
current input record the are same group?
Like
Share
Arjunreddy Syamala
September 15, 2014
Hi All,I have a graph with 200 records i run the graph the graph failed after loading 100 reords i
rectified the error so again i run the graph but now i want to load from 101th record what should i do
in the output table or update table component?
Like
Share
Shadab Mohd Use utilitiy mode ..see help for more info
September 15, 2014 at 7:29pm · Like
Arjunreddy Syamala Shadab in the access tab if i use load in append mode it will load from the 101 th record
yes?
Shadab Mohd
September 15, 2014
Hi ,
How to create dataset for "call stored procedure" component by mapping data tab to eme location?
Like
Share
Arjunreddy Syamala
September 15, 2014
Hi All I have a i/p file it is a 3 way multifile and i have a lookup file it is a 3 way multifie in i/p file 1st
partion matching records in the second partion of lookup file like etc how can i retrive the matching
records please write a code with an example?
Like
Share
Ankur Jain Try using lookup_local() for faster processing; though lookup() function should also work .
Arjunreddy Syamala if i use lookup_local i will fetch the records if the matching records are in the first partition
of lookup file but here matching records are in the second partition
Channi Tata prerequisite for lookup with MFS files is "lookup file and input file should be partitioned and sorted
on same key". if that is not the case you either do pbks on both files(same) again or atleast make them serial.
Abinitio.. i have data column like this id 1,2,3,.......10 iam using output_indexes likes this. count
parameter is 4
out::output_indexes(in)=
begin
out:1:if(in.id==3)[1,3];
out:2:if(in.id==5)[vector 2,4];
end;
if i use above function the remaining records will go to output or not?
Like
Share
Arjunreddy Syamala thank u
Aman Madan No
Arjunreddy Syamala
September 12, 2014
what is the diff b/w max-memory and max-core ?max-memory is available input must be sorted it is
fine max-core is available input need not be sore it is fine? after exceeding max-core intermediate
result write to the disk if exceeds max core the performace will decrease?
Like
Share
Arjunreddy Syamala
September 12, 2014
input parameters and graph parameters are same is it different?if it is not same what is the
difference?
Like
Share
Shadab Mohd Graph has input as well as non input param ... If a param is an ip param then it must hav a value
otherwise graph will fail at first state ...where as if a param is not an ip param then it may hav a value or not ...it will
not fail in first step ...
Aman Madan Different. Chk help
Shariff Mohammed One more difference is..Input parameters are the ones mostly derived from shell .. graph
parameters are mostly set within graph runtime, localized through private,pub,common projects..
Hi...Does anyone have the Version Difference script code base? Scpt_utils.func and the
script...Please pass it on to me..
Like
Share
Vaibhav Upadhyay it is with welssfargo team at infy pune ph. 1
Vaibhav Upadhyay *wells fargo
Ismail Mohammed Varun Nischal - Dude ?
Anup Singh Lehal Kya sir kabhi toh khud se kuch kiya karo smile emoticon
Shashank Vaze Re-usability is sophisticated name for Jugaad!!!!
can any one plz explain me how to create .ksh (script) from .mp(graph) and how its work in unix
environment
Like Comment Share
Gowrisan San R u asking abt deployed script? As far as I know once u deploy or save graph, deployed script
gets generated in run directory and u can run normally in unix environment as u run unix scripts...
January 14 at 7:42am · Like · 1
Gopal Patil You will always need co-op to run the deployed script.
January 14 at 10:44am · Like · 3
Urs Jagadish IS it possible in GDE with Sandbox ?
April 9 at 12:21am · Like
Togati Vijay On gde u can run .mp .pset .plan objects ...if u want run scrits use run_command component then
place your script with path .....orelse go to f-8 and then execute script
July 8 at 11:12pm · Like
Write a comment...
Arjunreddy Syamala
November 19, 2014
Hi All, I have a graph ,i want to see all objects like(xfr's,dmls etc) belongs to the particular graph?
how can we find it?pls help me.
Like Comment Share
Yogesh Shetty Right click on the graph name, click on 'Required Files'!
Niyas Ahamed u can use air command also..
Pankaj Gupta air object uses <graphname along with eme path >
Gopal Patil air project get-required-files
Roshan Dupkhaiya Perfect Gopal Patil.... smile emoticon
Urs Jagadish If i don't have EME or sandbox... then ?
April 9 at 12:32am · Like
Togati Vijay Air object get-required-files eme pathof the project space mp/graph name .....so it wil give u the list
of dml"s xfr's etc...
July 8 at 11:09pm · Like
Write a comment...
Gowrisan San
May 6
Just want to share what i learned today... To identify the difference between different versions of a
same object we can use below
air object changed <object detail> -version1 <verno> -version2 <verno> -diff
Like Comment Share
7 people like this.
Togati Vijay But the output is understabke 100% if you use web eme link its giving good understanding
July 8 at 11:04pm · Like
Write a comment...
Debabrata Roy
November 14, 2014
Ramanjaneyulu Kancharla It having all servers connections information
Vikas Rajodia Server connection info...
Phani Muthuluri it consists system configuration information your host want to access another server
Phani Muthuluri it consists system configuration information your host want to access another server
Togati Vijay We can able to connect to all the servers mentioned in roc file
June 28 at 12:02pm · Like
Togati Vijay Fir example we configure that file in dev env with qa,prod details etc..then we can connect to that
severs
June 28 at 12:04pm · Like
Togati Vijay After configuring abinitiorc file go to one server type ssh othersrver name if u give correct passwd
then it wi connect succefully
July 8 at 10:58pm · Like
Write a comment...
Spandana Reddy
November 13, 2014
what is the critical situation you faced in your project means what should i have to tell? can anyone
pls tell me its very urgent
Like Comment Share
Togati Vijay Any complex logic u have applied ..or u have given a generic graph or pset fir any problem