Hashed File Distribution
Day 5 – Session 3
Hashed File Distribution
Dates
25/02/2011 Tech Support Program
Venue
Microcred, Senagal
Objective
Objective:
At the end of this session, participants will:
• Know the need and features of File Distribution
• Distribution algorithm
• Part file creation
• Attaching part files
• Testing data integrity after distribution
3 Thesys Training Centre
Introduction to File Distribution
File Distribution
In jBase, the size of J4 and J3 file size should not exceed 2GB
File will get corrupted if the size exceeds 2GB
We need to perform regular archival activity in order to
maintain the file size
Some T24 files like STMT.ENTRY , CATEG.ENTRY and
RE.CONSOL.SPEC.ENTRY grows very fast because all the
financial transactions hit these files
To solve this problem, one file can be divided into many part
files
5 Thesys Training Centre
File Distribution
It is possible to split one file into “n” number of part files
For example, If FBNK.STMT.ENTRY file size is 1.9GB, then
this can be divided into 13 Part files
Dividing algorithm is required for this activity
The file saving logic would be, one part file will be created
for each month. Each month’s data will be saved in the
corresponding part file
Remaining data or data that does not fall in any part file will
be stored in the last part file (13th Part File)
6 Thesys Training Centre
File Distribution
Logic can be designed as, all Jan month’s STMT.ENTRY
records will be stored in PART FILE 1
All Feb month’s STMT.ENTRY records will be stored in
PART FILE 2
Error files or remaining files will be stored in the last part file
7 Thesys Training Centre
File Distribution
File Distribution – STEP I
Fix the number of part files and arrive at the Algorithm
• Let us take the example of FBNK.STMT.ENTRY file
• Letus say we have decided to distribute this file based on the
day of the month
• To achieve this we require 31 STMT.ENTRY files one pertaining
to each day
• Additionally, we also should have a 32nd file to act as a catch all,
where the records that would defy the algorithm could be stored
9 Thesys Training Centre
File Distribution Algorithm
This algorithm should be based on the ID of the file that is
being distributed
In our example of FBNK.STMT.ENTRY the record ID would
be of the format
119040005344258.000001
However, you might also find ID’s like
F20001213.6265153350 - Forward Entries
DUMMY.ID19 - Temporary Entries
10 Thesys Training Centre
File Distribution Algorithm
For this file we can use an algorithm based on the first five
digits of ID that represent a date in machine format
Write a program for this logic
Compile and Catalog the program
11 Thesys Training Centre
File Distribution Algorithm
SUBROUTINE STMT.PART.ALGO(RESERV,REC.ID,YKEY)
DATENO = REC.ID[1,5]
FIRSTCHAR = REC.ID[1,1]
IF ALPHA(FIRSTCHAR) THEN
ENTRYDT = "32“
END ELSE
ENTRYDATE = OCONV(DATENO,"D")
ENTRYDT = ENTRYDATE[1,2]
END
YKEY = ENTRYDT
RETURN
END
12 Thesys Training Centre
File Distribution – STEP II
FBNK.STMT.ENTRY file to a backup file and delete
FBNK.STMT.ENTRY file
CT VOC FBNK.STMT.ENTRY
F
../mbdemo.data/ac/FBNK.STMT008
../mbdemo.dict/F.STMT.000]D
13 Thesys Training Centre
File Distribution – STEP II
From Unix level move the Physical file and dict file into some
other name
•$ pwd
• /globus/mbdemo/mbdemo.data/ac
•$ mv FBNK.STMT008 FBNK.STMT008.BEFDISTB
Make a copy of the dictionary file for future use
•$ pwd
• /globus/mbdemo/mbdemo.dict
•$ cp F.STMT.000]D F.STMT.000]D.BEFDISTB
14 Thesys Training Centre
File Distribution – STEP II
Create VOC for the backup file
jsh -->JED VOC FBNK.STMT.ENTRY.BEFDISTB
F
../mbdemo.data/ac/FBNK.STMT008.BEFDISTB
../mbdemo.dict/F.STMT.000]D.BEFDISTB
15 Thesys Training Centre
File Distribution – STEP II
Delete the VOC of the FBNK.STMT.ENTRY as follows
jsh -->JED VOC FBNK.STMT.ENTRY
Confirm deletion of record (Y/N) : Y
Command->FD
F
../mbdemo.data/ac/FBNK.STMT008
../mbdemo.dict/F.STMT.000]D
16 Thesys Training Centre
File Distribution – STEP II
Check the FBNK.STMT.ENTRY file .It should not exist
• SELECT FBNK.STMT.ENTRY
• No file name could be found for your query
17 Thesys Training Centre
File Distribution – STEP III
In our example, we have to create 32 data files to act as
part-files. First decide the modulo for each of these part files.
• CREATE-FILE DATA FBNK.STMT.ENTRY.01 { modulo }
• CREATE-FILE DATA FBNK.STMT.ENTRY.02 { modulo }
……..
……..
• CREATE.FILE DATA FBNK.STMT.ENTRY.31 {modulo}
• CREATE.FILE DATE FBNK.STMT.ENTRY.32 {modulo}
18 Thesys Training Centre
File Distribution – STEP IV
CREATE-FILE DATA FBNK.STMT.ENTRY TYPE=DISTRIB
FBNK.STMT.ENTRY and all the 32 part files should be
available in bnk.data/ac directory
Create / Edit the VOC entries of the distributed file
FBNK.STMT.ENTRY and all the part files to make sure that
they point the correct unix file and correct dictionary file
19 Thesys Training Centre
File Distribution – STEP V
Add the distribution algorithm and part files to the distributed
file created to make it complete
• CREATE-DISTRIB -pUSER, STMT.PART.ALGO FBNK.STMT.ENTRY
• CREATE-DISTRIB –a FBNK.STMT.ENTRY 1 FBNK.STMT.ENTRY.01
• CREATE-DISTRIB –a FBNK.STMT.ENTRY 2 FBNK.STMT.ENTRY.02
………
………
• CREATE-DISTRIB –a FBNK.STMT.ENTRY 31 FBNK.STMT.ENTRY.31
• CREATE-DISTRIB –a FBNK.STMT.ENTRY 32 FBNK.STMT.ENTRY.32
20 Thesys Training Centre
File Distribution – STEP VI
VERIFY-DISTRIB FBNK.STMT.ENTRY
This will check whether distributed file has been created
properly
Copy the records from the backup to distributed file.
• COPY FROM FBNK.STMT.ENTRY.BEFDISTB TO
FBNK.STMT.ENTRY ALL
21 Thesys Training Centre
Data Integrity Check
Testing Integrity
Check whether the numbers of records match before and
after distribution
Use SELECT, COUNT, LIST commands on the file to see
that they return values without error
Log on to GLOBUS / T24. List the file and verify the
correctness
Execute an enquiry that uses this file and verify the
correctness
Execute an EOD and make sure that EOD runs without any
problem
23 Thesys Training Centre
Summary
The participants have learnt,
• The need and features of File Distribution
• Distribution algorithm
• Part file creation
• Attaching part files
• Testing data integrity after distribution
24 Thesys Training Centre
All product names and other company names used herein are for identification purposes only and
may be trademarks or registered trademarks of their respective owners. Errors and omissions excepted,
all specifications are subject to change without notice.
© 2009 Thesys Technologies Incorporated. All rights reserved.
FOR MORE INFORMATION
Visit : www.thesys.co.in
email :
[email protected]
