Configure SFTP Shell Script File Transfer
Configure SFTP Shell Script File Transfer
Trademarks
RSA, the RSA Logo and EMC are either registered trademarks or trademarks of EMC Corporation in the
United States and/or other countries. All other trademarks used herein are the property of their respective
owners. For a list of EMC trademarks, go to www.emc.com/legal/emc-corporation-trademarks.htm.
License Agreement
This software and the associated documentation are proprietary and confidential to EMC, are furnished
under license, and may be used and copied only in accordance with the terms of such license and with the
inclusion of the copyright notice below. This software and the documentation, and any copies thereof, may
not be provided or otherwise made available to any other person.
No title to or ownership of the software or documentation or any intellectual property rights thereto is
hereby transferred. Any unauthorized use or reproduction of this software and the documentation may be
subject to civil and/or criminal liability. This software is subject to change without notice and should not be
construed as a commitment by EMC.
Third-Party Licenses
This product may include software developed by parties other than RSA.
Distribution
Use, copying, and distribution of any EMC software described in this publication requires an applicable
software license. EMC believes the information in this publication is accurate as of its publication date.
The information is subject to change without notice.
THE INFORMATION IN THIS PUBLICATION IS PROVIDED "AS IS." EMC CORPORATION
MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE
INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Contents
l The script runs on all POSIX compliant Unix/Linux systems and shells.
l RSA recommends that you set up a cron job to run the script at specified time intervals. However, if you
do set up a cron job, make sure to run it as a user that has access to the logs that need to be sent to RSA
NetWitness Suite.
l Instructions for Upgrading the Agent: follow these steps if you are currently running version 2.7 of the
agent
l Instructions for Installing the Agent: follow these steps if you are downloading the agent for the first
time
l Instructions to Configure the RSA NetWitness Log Collector to Receive Log Files
You must perform the following steps to complete the installation and configuration of the agent:
I. Install or Upgrade the agent, depending on whether or not you are running it currently.
II. Configure the the RSA NetWitness Log Collector to Receive Log Files.
Configure SFTP Shell Script
l Encourages the user to keep configuration separate from script source. Warning is logged if the user does
not do so.
l The agent has been made highly configurable so that it can be configured to run from anywhere, and
create a persistent state directory anywhere. For example, non-root users can persist state information
to their home directory, by specifying an alternate persistent state directory in the configuration file.
l Command line options (-C or --config) have been added to point the agent to custom configuration.
l Logs are written to /var/log/rsa/sasftpagent.log. Logging levels have been introduced so that the
logs may be filtered for WARN, ERROR and FATAL entries, to help troubleshoot issues. These log
entries can now be used to perform troubleshooting after the fact.
l If the user forgets to edit the configuration, a FATAL log entry is generated. The entry contains a clear
message that the user needs to edit the configuration before the script can be used.
IV. Run the mvpersinfo.sh script to move persistent information to the location used by version 3.
l The configuration may have been specified separately; wherever the CONFIG_FILE parameter in the
sasftpagent.sh script is set.
For any parameters that you edited within the script or that you specified in the separate configuration file,
you need to move them to a separate file in the following location: /etc/rsa/sasftpagent.conf.
The following parameters are the ones that users often change during configuration:
l SA
l DATA_DIRECTORY
l FILESPEC
l FLAG_REMOVE_AFTER_SEND (only set if you wanted to remove data files automatically after
transferring them to the Log Collector).
To move the persistent information to its new location, perform the following steps:
1. Copy mvpersinfo.sh to the system where you run version 2.7 of the agent.
2. Open mvpersinfo.sh with a text editor, and confirm that OLD_PERSINFO_DIRECTORY is set to the
value set for PERSINFO_DIRECTORY in your 2.7 configuration.
If there are no errors, and the script runs successfully, it does not generate any output.
After you run the script, you can confirm the successful movement by using the following procedure:
1. Run the following command to get a list of the old tracking files:
find /usr/local/sa -name "*-*last.line"
2. Run the following command to get a list of the new tracking files:
find /var/lib/rsa/sasftpagent -name "*-*last.line"
3. Compare the output from the two commands. The output should be similar, with the only difference being
the paths to the files. Here is a sample of the results of running these commands after moving the
persistent files:
$ find /usr/local/sa -name "*-*last.line"
/usr/local/sa/opt/log/bar.log-sa.last.line
/usr/local/sa/opt/log/foo.log-sa.last.line
usr/local/sa/var/log/foo.log-sa.last.line
/usr/local/sa/var/log/fob.log-sa.last.line
/var/lib/rsa/sasftpagent/track/opt/log/bar.test-last.line
/var/lib/rsa/sasftpagent/track/opt/log/foo.test-last.line
/var/lib/rsa/sasftpagent/track/var/log/foo.test-last.line
/var/lib/rsa/sasftpagent/track/var/log/fob.test-last.line
IV. Schedule the Agent to Run Periodically: Configure cron or your OS scheduler to automate running the
script at your desired interval.
2. Click RSA NetWitness Unix SFTP Agent and save the file anywhere on your file system.
3. Set execute permissions on the sasftpagent.sh file. For example, run the following command:
chmod 755 /usr/local/sa/sasftpagent.sh
l This script creates the sftp user for transferring the log files.
2.
Caution: It is important to run this command as the same user who will run it when it is automated.
SA Name or IP address The name or IP address of your RSA NetWitness Log Collector
host.
DATA_ Directory path or paths, The local source for the log data. For example:
DATA_DIRECTORY=/var/log:/var/log/audit
DIRECTORY separated by colons (:).
You can specify one or more folders.
Note: All folders that you specify are searched for the file
names that you specify in the FILESPEC parameter.
FILESPEC File name or names, File mask that matches the log files to be processed by the
script.
separated by colons (:).
Note: The script supports line-by-line text data. Thus, .xml,
.zip, .gz, .exe and other non-text formats are not supported.
SA_ The directory name of The destination folder name. For example:
/upload/apache/muditapache
DIRECTORY your NetWitness Log
Collector host
TRANSFER_ SFTP, or SCP SFTP is the default (and recommended) transfer protocol
METHOD
USEHEAD A non-negative integer, The number of lines in each log file to be considered as a
header that must be transferred to Log Collector in each
representing the number
transfer.
of header lines
You can set this to 0 to indicate that there are no header lines.
DEPTH A positive integer, rep- Governs how many levels deep the script searches to find logs
under the configured DATA_DIRECTORY.
resenting number of
Defaults to DEPTH=1, which causes the script to search for
folder levels
data files directly under the directories configured in DATA_
DIRECTORY, but not in any sub-folders.
USERNAME sftp Default setting for SSH daemon on the RSA NetWitness Suite
platform.
IDENTITY File path Location of the private key used to connect to RSA NetWitness
Suite. For instructions on generating keys, see Install and
Update the SFTP Agent.
The default value is the following:
$HOME/.ssh/id_rsa
This command creates id_rsa in OpenSSH format, which is used by RSA NetWitness Suite. If your Linux
system creates IETF SECSH format by default, run the following command to convert it:
ssh-keygen -f ~/.ssh/id_rsa.pub -i
2. Add the public key into the log collector, as described in Install and Update the SFTP Agent.