Scribd
Upload
70 views5 pages

Large File Handling

This document discusses handling large files in webMethods Integration Server. It describes how to stream flat files and XML files into the server to avoid crashes from files over 100MB. For flat files, it recommends using the FilePolling port or pub.file:getFile service with the "load as stream" option. For XML, it recommends using pub.xml:xmlStringToXMLNode and pub.xml:getXMLNodeIterator services to iterate through nodes one by one instead of loading the entire file at once.

Uploaded by

Guda Vanitha
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
70 views5 pages

Large File Handling

This document discusses handling large files in webMethods Integration Server. It describes how to stream flat files and XML files into the server to avoid crashes from files over 100MB. For flat files, it recommends using the FilePolling port or pub.file:getFile service with the "load as stream" option. For XML, it recommends using pub.xml:xmlStringToXMLNode and pub.xml:getXMLNodeIterator services to iterate through nodes one by one instead of loading the entire file at once.

Uploaded by

Guda Vanitha
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 5

Large File Handling webMethods

In middleware integration we generally face a issue how to handle


large data of flatfiles and xml in the case of large files(suppose
greater than 20MB) , it will slowdown the Integration server if directly
loaded into the server , if the file is of huge size (greater than 100MB)
then in this case it might crash the server . So, in order to save ourself
from this situation we must take a precautive measure , this can be
done by streaming the file into the server . So, in this blog we will see
basically how to do large file handling in Integration server in the case
of both Flat File and xml files.

Large File Handling (Flat Files)


Now we will see how to do large file handling in the case of flat files.
we can use webMethods FilePolling Port , pub.file:getFile , or using
FTP (i.e using this service pub.client:ftp) to get the file depending on
whether the file is on localsystem or remote FTP system w.r.t to
webMethods Integration server(IS).
webMethods FilePolling port by default streams the file into the IS , for
pub.file:getFile select the optionsal input parameter as load as stream
and it will stream the file into the server .

Normally we use pub.flatFile:convertToValues service to convert the


flatfile into IS document , this time also we will use the same service
with slight modification we will set the iterate input parameter as true.

pub.flatFile:convertToValues: This service is used in convert the flat file


data to IS Document. Iterate = true value ensures that the records from
the flat file is read one By one. Output ffIterator is passed at the input
of the next invocation of this service.

PUBLIC
If ffIterator is null, exit from the loop. It means all the available records
in the flat file are processed.

PUBLIC
Large File Handling XML
Generally in real time scenarion for large xml it has been seen that
some nodes are there that are being repeated several tomes making
the xml file large enough . The large file handling for xml in
webMethods targets this point and in built services are present to hanle
these kind of scenarios.

Let's first see the inbuilt services present to do large file handling in
the case of xml.
pub.xml:xmlStringToXMLNode – This service converts the input
from String to XML node , this is a general service used in webMethods
where we want to convert xml string data to xml node . This service is
good in built service having great performance it plays a vital role in
large file handling.

pub.xml:getXMLNodeIterator : This service will get the node one


by one according to the Criteria set in the pipeline. Over here the
criteria is set as EmployeeDetails and OFCDetails. This is the main
service used for large file handling for XML , Just like flat file service ,
this service also iterates over the node and process it one by one .

PUBLIC
pub.xml:getNextXMLNode : This service is used in order to get the
next node .

pub.xml:xmlNodeToDocument : This service converts the XMLNode to


Document

Now lets see the structure of the main flow service where Large file
Handling for xml has been done .

PUBLIC
Large File Handling:

First create flatfile schema and record definition and fields. Record definition name must match
the value of the record identifier in a flat file. Record definition is 0 and fields start from 1.
For mapping purpose, create an IS document type from this flat file schema by keeping ‘Max
Repeat’ 1. Now set ‘Max Repeat’ to Unlimited for parsing.
Create the service keep the service pub.file.getfile file name is path and load as bytes.
Take repeat step make it repeat on success, take convert to values step from pub.flatfile
Bytes to ffdata in pipein, copy ffschema and paste in ffschema which is under pipeline make it
iterate true to read the records one by one. Take the document reference from schema DT and
map it from ffValues for further usage in the pipeine.
Here I am just storing the records in database with an insert adapter service. Map the values.
Take branch step
When the ffiterator variable is null, you should use an EXIT step to exit from the REPEAT step and
discontinuing processing. Exit from loop. Copy ffiterator and paste in switch

PUBLIC

You might also like