File Splitter

Version: 1.0

By Sudheer Anantha

Page 1 of 14

Contents
1. Objective: ..............................................................................................................................3 2. High Level View: ....................................................................................................................4 3. Flowchart 5 4. Low Level Details and Logic: ...................................................................................................6 5. Code Components:.................................................................................................................7 6. Test Cases: ........................................................................................................................... 14

Page 2 of 14

Objective:
The Purpose of the document is to explain the working of the File Splitter program. It explains the big picture of the program first and gives explains the low level details in the later part of the document. The Big Picture tells about where the File Splitter is used and how. And gives a basic idea of how it works. And the low level details explain each function in the code component in details. The Program is explained in detail so that any future changes to the code or future enhancements can be easily made after reading the Document.

Page 3 of 14

High Level View:

The High Level View gives the big picture of the working of splitter. The following diagram shows
the basic operation of BizTalk server for a 270 batch file.

NOTE: The purpose of the diagram is to show where the Batch Splitter is placed inside the BizTalk server and where the re-batching is done so dont mind if any other components are misplaced or named differently when compared with other design documents. When a 27x batch file is sent from the Trading partner the receiver pipeline inside the Biztalk server gets it and each component of the pipeline works on the incoming message doing their respective operations. When the message reaches the disassembler part of the Receiver Pipeline the File Splitter program will split the incoming message stream into multiple messages streams. Each of the messages is later sent for processing and the details are archived into the database. Note: A copy is saved before it is split so that any error in the processing or un-process able messages can be processed later.

Page 4 of 14

Flowchart
270 Batch File
ISA*-----------------------------------------GS------------------------------------------ST------------------\nSE-----------------------------------------------------------------ST------------------\nSE------------------GE------------------------------------------IEA-------------------------------------------

ISA or GS Header Line Starts With ?

IEA or GE Trailer

ST-SEs ST or BHT ST Header Line Starts With ? SE STTrailer

HLs

ST Header

DP HashTable

0 Trailer

HL line ends with ?

1 SB Hashtable

Header Single HL Component

HL Components For the Single HL

ST Trailer

Simple 270 File Simple 270 File Process the next ST component Process the next HL ARCHIVE Simple 270 File Simple 270 File

Yes

More Elemensts to process in HashTable?

NO

Yes

More ST Components to be processed?

NO

Page 5 of 14

Low Level Details and Logic: (for developers)

Once the File Splitter gets the input stream the File Splitter saves the stream into a text file and archives the file. And now the input stream is divided into the following components. 1) Header: The First 2 lines of the stream are loaded in here. The lines which start with ISA and GS lines are stored here 2) Trailer: The last 2 lines of the stream are loaded in here. The lines which start with GE and IEA lines are stored here 3) stHeader : It has the first lines of the ST component. The lines which start with ST and BHT lines are stored here 4) stTrailer: It has the first lines of the SE component. The line which start with SE is stored here. 5) HL components: All the HL components are stored in HashMaps. It is stored in 2 different types of HashMaps subTable: It contains all the HL components for which the HL line of the component ends with 1. depTable: It contains all the HL components for which the HL line of the component ends with 0.

All the HL components are divided into 2 parts hlInfo and hlValue. The line that started with HL is stored as hlInfo and the other lines are made as hlValue. After all the following components are made, for each item in the depTable, a file is made which is an independent and a valid 270 File with only 1 component and is ready to be processed and is sent to the later stages of the pipeline. A acknowledge message is sent to the trading partner who sent the file and also acknowledge message is saved for each file that is created and this is done internally by BizTalk and not by the code.

Page 6 of 14

Code Components: (for developers)

The Following elements are created and stored in memory so that creating the split files can be easy as creating each of them will require each of elements be there in memory.
// The Following elements are stored in memory while the program is executed StringBuilder header = new StringBuilder(); //The ISA and GS Lines StringBuilder trailer= new StringBuilder(); //The GE and IEA Lines Hashtable depTable = new Hashtable(); //This table contains all the HL's which end with 0 Hashtable subTable = new Hashtable(); //This table contains all the HL's which end with 1 StringBuilder stHeader = new StringBuilder(); //The ST and BHT line StringBuilder stTrailer = new StringBuilder(); //The Se Line LinkedList<string> filenames = new LinkedList<string>(); //ALl filenames string filename;

This Function will created the header and footer elements and stores St sections in array. // This function loads all the file into respective stirngbuilders private void processBtn_Click(object sender, EventArgs e) { StreamReader sr = new StreamReader(@"" + txtBrowse.Text + ""); StringBuilder s1 = new StringBuilder(); LinkedList<string> stfile = new LinkedList<string>(); string line = null; ; int i = 0; while((line=sr.ReadLine())!=null) { if(line.StartsWith("ISA")) { header.Append(line.ToString() + "\r\n"); } else if (line.StartsWith("GS")) { header.Append(line.ToString()); } else if (line.StartsWith("ST")) { s1.Append(line + "\r\n"); } else if (line.StartsWith("SE")) {

Page 7 of 14

s1.Append(line + "\r\n"); stfile.AddLast(s1.ToString()); s1.Length = 0; ++i; } else if (line.StartsWith("GE") || line.StartsWith("IEA")) { trailer.Append(line.ToString() + "\r\n"); } else { s1.Append(line + "\r\n"); } } ProcessFiles(stfile); } The Following Function will create the stHeader, stTrailer. The hlinfo and hlValue for each of the HL component and calls other functions for furthur processing. // This function deals with the each individual ST section private void ProcessFiles(LinkedList<string> stfile) { int i = 0; foreach(string file in stfile) { String line = null; String hlinfo = null; StringBuilder hlValue = new StringBuilder(); StringReader reader = new StringReader(file.ToString()); while ((line = reader.ReadLine()) != null) { if (line.StartsWith("ST")) { stHeader.Append(line.ToString() + "\r\n"); } else if (line.StartsWith("BHT")) { stHeader.Append(line.ToString()); } else if (line.StartsWith("HL")) { if (hlValue.Length != 0) { addToHash(hlinfo + "\r\n", hlValue); } hlinfo = null; hlValue.Length = 0; hlinfo = line;

Page 8 of 14

} else if (line.StartsWith("SE")) { if (hlValue.Length != 0) { addToHash(hlinfo, hlValue); hlValue.Length = 0; } stTrailer.Append(line.ToString() + "\r\n"); } Else { hlValue.Append("\r\n" + line.ToString()); } } CreateSplits(i); formatFiles(); clearMemory(); i++; } } The function clears the memory after every St section is processed so that the previous elements can be used the next time. //Clears the memory after a ST section is processed private void clearMemory() { depTable.Clear(); subTable.Clear(); stHeader.Length = 0; stTrailer.Length = 0; filenames.Clear(); } The following function gets the level of the HL number form the token number we created for processing. //gets the level of the HL private int getLevel(int x) { int temp = 0; temp = x / 10000; int temp2 = x - (temp * 10000); return temp2; }

Page 9 of 14

The following section is the critical aspect of the code as it deals with the various scenarious of the input Batch File. If the code fails for a particular scenario then the following function should be altered to make the program to work. // This part creates teh split files. private void CreateSplits(int i) { Hashtable list = new Hashtable(); foreach (int num in depTable.Keys) { foreach (int x in subTable.Keys) { if (num > x) { int temp2 = getLevel(x); int dep = getLevel(num); int value = 0; if (list.ContainsKey(temp2) == true) { value = (int)list[temp2]; } if (list.ContainsKey(temp2) == false && temp2 != dep) { list.Add(temp2, x); } else if (value < x && temp2 != dep) { list.Remove(temp2); list.Add(temp2, x); } } } LinkedList<int> key = new LinkedList<int>(); key = sortlist(list); writeToFiles(key, num, i); list.Clear(); } }

Page 10 of 14

The following function write the elements into the files. It uses all the elemts declared at the start of the component. //All the indiviual files are made private void writeToFiles(LinkedList<int> keys,int num, int k) { System.IO.StreamWriter file = new System.IO.StreamWriter(outputFolder.Text + "\\" + filename + "_" + num + k.ToString() + ".txt"); filenames.AddLast(filename + "_" + num + k.ToString() + ".txt"); file.WriteLine(header); file.WriteLine(stHeader); StringBuilder temp = new StringBuilder(); foreach (int item in keys) { temp.Append(subTable[item]); file.WriteLine(temp); temp.Length = 0; } file.WriteLine(depTable[num]); file.WriteLine(stTrailer); file.WriteLine(trailer); file.Close(); } The Following code does 2 things: 1) serializes the HL components i.e., all the HL lines are changed so that all the HL lines would be correctly. 2) Formats the code i.e., removes all theempty lines. //Formats the file to remove blank lines and //All the HL lines are serialized. private void formatFiles() { foreach (string file in filenames) { int count = 1; string[] text = File.ReadAllLines(outputFolder.Text+"\\" file).Where(s => s.Trim() != string.Empty).ToArray(); for (int i = 0; i < text.Length; i++) { if (text[i].ToString().StartsWith("HL")) { StringBuilder sb = new StringBuilder(); string[] words = text[i].ToString().Split('*'); int num = Convert.ToInt32(words[1].ToString()); words[1] = count.ToString();

Page 11 of 14

for (int j = 0; j < words.Length; j++) { sb.Append(words[j] + "*"); } text[i] = sb.ToString().Substring(0, sb.Length - 1); sb.Length = 0; count++; } } File.Delete(file); File.WriteAllLines(file, text); listBox.Items.Add(file); } }

This function is responsible for sorting the hashtable values so that the correct order is decided which is needed before the split files are made. //The given hashtable would be sorted. private LinkedList<int> sortlist(Hashtable list) { int i = 0; int[] keys = new int[list.Count]; LinkedList<int> values = new LinkedList<int>(); foreach (int item in list.Keys) { keys[i] = item; i++; } Array.Sort(keys); for (int j = 0; j < list.Count; j++ ) { int val = (int)list[keys[j]]; if(values.Count ==0 || val>values.Last()) values.AddLast(val); } return values; } The following function is where the Hashtables subTable and depTable are loaded with data. //All the hastables are made using this function private void addToHash(String hlinfo, StringBuilder hlValue) { string[] words = hlinfo.Split('*'); int level = Convert.ToInt32(words[3].ToString());

Page 12 of 14

int num = Convert.ToInt32(words[1].ToString()); int dep = Convert.ToInt32(words[4].ToString().Substring(0,1)); int key = 0; StringBuilder hl = new StringBuilder(); if (dep == 0 ) { key = num * 10000 + level; hl.Append(hlinfo.ToString()); hl.Append(hlValue); depTable.Add(key, hl); } if (dep == 1) { key = num * 10000 + level; hl.Append(hlinfo.ToString()); hl.Append(hlValue); subTable.Add(key, hl); } }

Page 13 of 14

Test Cases:
The Program works for single ST section in a File. The Program works for multiple ST sections in a File. The Program Works for single Dependent. The Program works for multiple Dependencies. The Program works for the different multiple levels of the subscribers and dependencies. The Program works for files which use any character instead of line feed.

Page 14 of 14