IBM Aspera Desktop Client 4.4 For Linux
IBM Aspera Desktop Client 4.4 For Linux
IBM
Contents
ii
High-Speed Transfer Client Admin Guide for Linux
Welcome to the High-Speed Transfer Client documentation, where you can find information about how to
install, maintain, and use the High-Speed Transfer Client.
Introduction
Thanks for choosing Aspera and welcome to the world of unbelievably fast and secure data transfer.
The Basics
Aspera high-speed transfers begin when an Aspera client authenticates to an Aspera server and requests
a transfer. If the client user has authorization, then transfer tools are launched on the client and server
and the transfer proceeds.
For example, an IBM Aspera Desktop Client user connects to an IBM Aspera High-Speed Transfer Server
and initiates a transfer:
Depending on the user's transfer request, files and folders can be transferred to the server from the client
(uploaded) or transferred to the client from the server (downloaded). The source and destination can be
cloud storage, an NFS or CIFS mount, and IBM Spectrum Scale storage, to name a few.
What is the Server?
The Aspera server receives transfer requests from Aspera clients, determines if the user has permission
to access the server and authorization to the target area of the file system (source or destination with read
or write access), and participates in transfers. The server can be:
• an on-premises installation of HSTS, IBM Aspera High-Speed Transfer Endpoint (which permits one
client connection),
• a HSTS installed as part of IBM Aspera Faspex, or
• an HSTS deployed in object storage as an IBM Aspera On Demand instance, an IBM Aspera on Cloud
transfer service node, or an IBM Aspera Transfer Cluster Manager node.
What is the Client?
The Aspera client is the program that requests a transfer with the Aspera server. Aspera applications that
can act as clients include:
• Desktop Client,
• IBM Aspera Drive,
• IBM Aspera Connect,
• IBM Aspera Command-Line Interface,
• HSTS and HSTE
What is FASP?
At the heart of your Aspera ecosystem are the FASP transfer engines Ascp and Ascp 4. Ascp maximizes
data transport over any network and is particularly suited to large files. It is a powerful command-line tool
and also drives transfers started in the GUI.
Requirements
System requirements for Desktop Client.
• Product-specific Aspera license file.
• Ubuntu 14.04 LTS, 16.04 LTS, 17.10; RHEL 6-7; CentOS 6-7; SLES 11-12; Debian 7-9; Fedora 26-27;
Kernel 2.4 or higher and Glibc 2.5+. Linux distributions/kernels released after the product release date
might not be compatible.
• Glibc 2.5 or higher.
• Screen resolution 1024 x 768 or higher for graphical user interface.
6. If you modified the daemon startup scripts for Aspera Central and asperanoded (for example, as part
of an Aspera API integration), back up the modified files. These files are overwritten during an upgrade
and you will need to copy your modifications into the new files after upgrading.
OS Commands
4. Installation troubleshooting.
If the installer freezes during installation, another Aspera product might be running on your
computer. To stop all FASP transfer-related applications and connections, see “Before Upgrading or
Downgrading” on page 4.
The following is basic information for configuring your firewall to allow Aspera file transfers. The outbound
TCP port for SSH may differ depending on your organization's unique network settings. Although TCP/
33001 is the default setting, refer to your IT Department for questions related to which SSH port(s) are
open for file transfer. Consult your operating system's documentation for instructions on configuring your
firewall. If your client host is behind a firewall that does not allow outbound connections, you will need to
allow the following ports:
• Outbound TCP/33001: Allow outbound connections from the Aspera client on the TCP port (TCP/33001
by default, when connecting to a Windows server, or on another non-default port for other server
operating systems).
• Outbound UDP/33001: Allow outbound connections from the Aspera client on the FASP UDP port
(33001, by default).
• Local firewall: If you have a local firewall on the client (such as iptables), verify that it is not blocking
your SSH and FASP transfer ports (such as TCP/UDP 33001).
Testing a Transfer
To make sure the software is working properly, set up a connection with a server and test downloads and
uploads.
Linux users may use the GUI or command line, and instructions for both are shown below.
1. Launch the application.
Run the following command in a terminal window:
$ asperascp
Field Value
Host my_demo.example.com
User my_user_name
Authentication my_password
(Password)
3. Test your connection to the remote server.
Click Test Connection to determine whether you can reach the remote server with the settings you
configured. An alert box opens and reports whether the connection is successful.
4. Connect to the server and download test files.
From the main window, select the server entry and click the Connect button.
On the server file browser (right panel), browse to a folder with a file that you want to use, select the
file, and click to download it to your local machine.
You should see the session appear in the Transfer pane.
5. Upload to the server.
Select the same file (100MB) on the local file browser (left panel), go to the folder /Upload on the
server, and click to upload it.
6. Download test files from the server.
Use the ascp command to download, press y to accept the server's key, and enter your password
when prompted. For example
Item Value
server address my_demo.example.com
Login account xeno
Test file /test-dir-large/100MB
Download location /tmp/
Transfer settings Fair transfer policy, target rate 10M, minimum rate 1M, encryption disabled.
Item Description
100 MB The name of the file that is being transferred.
28% The percentage completed.
28 MB The amount transferred.
Uninstalling
Desktop Client can be uninstalled without removing existing configuration files.
1. If you are uninstalling in order to upgrade your Aspera product, review the upgrade preparation steps
in “Before Upgrading or Downgrading” on page 4.
2. Close or stop the following applications and services:
• FASP transfers
• SSH connections
• user interface
3. Uninstall by running the following command:
Platform Command
Note: This process does not remove Aspera configuration files. If you reinstall an Aspera product, these
configuration files are applied to the new installation.
# asperascp
Item Description
A Click Transfer to enter the transfer viewing mode. This is the default view when you
launch the application, which shows the local and remote file browsers. For more
information, see “Transferring Content” on page 26.
B Select a transfer from the bottom pane and click Details to enter the transfer details
viewing mode. This view shows the details of the selected transfer session, as well as the
transfer control options. For more information, see “Managing Transfers” on page 27.
C Click Connections to open the Connection Manager window in which you can manage
the remote endpoints. For more information, see “Adding and Editing Connections” on
page 15.
D Click Preferences to set the local computer's default transfer settings, such as the FASP
global bandwidth and the number of simultaneous transfers in the queue, and the SMTP
server's information for transfer notifications.
E Browse the local file system to view files to transfer.
F When not connected, a list of the saved connections is displayed. When connected (by
clicking on a Connection Name and clicking Connect), browse the remote file system.
G Display previous, ongoing, and queued transfers. Manage the priority.
1. Launch the application with administrator privileges and click Tools > Global Preferences.
3. To limit total bandwidth usage by FASP uploads and downloads, edit System-Wide Settings.
System-wide settings set the aggregated bandwidth cap for all FASP transfers on this computer.
To override the default bandwidth limits, under System-Wide Settings select the boxes next to Limit
Download Bandwidth and Limit Upload Bandwidth and enter new values in the fields. The global
settings for download and upload bandwidth limits cannot be reset by non-admin users. However,
users can view the global limit from the Preferences > Transfers dialog.
4. To set default target rates for all users, edit Default Target Rate.
Non-admin users can adjust their personal default target rates above or below the global default value.
5. To limit the number of active transfers, edit Maximum Active Transfers.
Non-admin users can adjust their personal maximum active transfers above or below the global
default value.
6. Click OK to activate your changes.
# asperascp
Click to duplicate a selected connection (to copy all information into a new profile) and to delete
a connection profile.
4. Configure the connection name, if desired.
By default, connections are named username@host.
To name or rename a connection, click the connection name and enter the new name in the pop-up.
Click OK to save your changes.
Connection Description
Option
Host The server's address, such as 192.168.1.10 or companyname.com. For Shares,
Node API, or AoCts connections, enter the URL and port for asperanoded, such
as https://ats-aws-us-west-2.aspera.io:443.
User The transfer user's username, the Shares user, Node API credentials, or an
access key ID.
Authentication The authentication method. Select Password to authenticate with the transfer
user's password, the Shares user's password, the Node API user password, or
an access key secret (such as for AoCts or ATC Manager).
Select Public Key to authenticate with the transfer user's public SSH key. For
more information, see “Creating SSH Keys in the GUI” on page 23 and .
Storage Type The default option is local storage. Use this option to connect to:
• on-premises servers
• AoCts
• cloud-based servers when the transfer user has the storage credentials
configured in their docroot on the server
When the server is in the cloud but the storage credentials are not configured
in the transfer user's docroot, use the drop-down menu to select the object
storage type and enter credentials.
Supported object storages include the following:
• Akamai NetStorage
• Amazon S3: Requires your Access Id, Secret Access Key, and bucket
path. The local machine must be reasonably time-synchronized in order to
communicate with the Amazon servers. You can also select the Advanced
button to modify the following settings:
– Host: Amazon S3 hostname (default: s3.amazonaws.com).
– Port: Default is port 443.
– HTTPS connection for file browsing: Enable for secure browsing.
– Server-side file encryption: Enable for AES256 encryption.
– Reduced redundancy storage class: Assign objects to a to the "reduced
redundancy" storage class (durability of 99.99%).
• Google Storage: Requires your Project Number and bucket path.
• Limelight: Requires your Account, Username, and Password.
• Windows Azure: Requires your Storage Account and Access Key.
Azure storage is set to use the Azure block blob REST API by default. To
specify the REST API for page blobs, enter your account credentials then click
Advanced. Select PAGE from the drop-down menu next to Api and click OK.
• Windows Azure SAS: Requires your Shared URL.
• Azure Files: Requires your File service endpoint and password.
Connection Description
Option
Target Directory The default directory when connecting to this computer. When left blank,
(or Bucket Path browsing the remote host brings up either the user's docroot or the last-visited
for most object folder. When a path is set, the connection opens to the exact directory.
storage)
Advanced Settings The TCP port for SSH connections. Default: 33001. If the application cannot
> SSH Port (TCP) connect on 33001, it automatically attempts a connection on port 22. If the
connection on 22 succeeds, the setting is updated to 22.
Speed Select Override default transfer rate settings to specify the transfer rate.
The target rate is constrained by the global bandwidth settings; for more
information, see “Global Bandwidth Settings” on page 10.
Retry Select to automatically retry the transfer after a recoverable failure for a set
amount of time, in seconds, minutes or hours. You may set the initial and
maximum retry intervals by clicking the More Options button.
• Initial interval: The first retry waits for the initial interval. Input in seconds,
minutes or hours.
• Maximum interval: After the initial interval, the next interval doubles until
the maximum interval is met, and then stops retrying after the retry time is
reached. Input in seconds, minutes or hours.
For example, if the retry is set for 180 seconds, the initial interval is 10
seconds, and the maximum interval is 60 seconds, then the transfer will retry
at 10, 30, 70, 130, and 180 seconds after the first try, such that the interval
progression is 10, 20, 40, 60, 60, and 50 seconds. The last interval is not the
maximum because the retry period ends with a final retry.
As another example, if the retry is set for 600 seconds, the initial interval is 30
seconds, and the maximum interval is 120 seconds, then the transfer will retry
at 30, 90, 210, 330, 450, 570, and 600 seconds after the first try, such that
the interval progression is 30, 60, 120, 120, 120, 120, and 30 seconds. Again,
the last interval is not the maximum because the retry period ends with a final
retry.
Show Advanced Click Show Advanced Settings to edit the following options:
Settings
• Specify FASP datagram size (MTU): By default, the detected path MTU is
used. Select to specify a value between 296 and 10000 bytes.
• Disable calculation of source files size before transferring: Select to turn
off job size calculation on the client side, if allowed by the server.
For more information on filter rule syntax, see “Using Filters to Include and Exclude Files” on page
64.
10. Configure security settings, if needed.
On the Security tab, configure non-default transfer settings by editing any of the following settings:
Encryption Select the encryption cipher. Aspera supports three sizes of AES cipher keys
(128, 192, and 256 bits) and supports two encryption modes, cipher feedback
mode (CFB) and Galois/counter mode (GCM). The GCM mode encrypts data
faster and increases transfer speeds compared to the CFB mode, but the server
must support and permit it.
Cipher rules
The encryption cipher that you are allowed to use depends on the server
configuration and the version of the client and server:
• When you request a cipher key that is shorter than the cipher key that is
configured on the server, the transfer is automatically upgraded to the server
configuration. For example, when the server setting is AES-192 and you
request AES-128, the server enforces AES-192.
• When the server requires GCM, you must use GCM (requires version 3.9.0 or
newer) or the transfer fails.
• When you request GCM and the server is older than 3.8.1 or explicitly
requires CFB, the transfer fails.
• When the server setting is "any", you can use any encryption cipher. The only
exception is when the server is 3.8.1 or older and does not support GCM
mode; in this case, you cannot request GCM mode encryption.
• When the server setting is "none", you must use "none". Transfer requests
that specify an encryption cipher are refused by the server.
Cipher Values
AES-128 Use the GCM or CFB All client and server versions.
AES-192 encryption mode, depending
AES-256 on the server configuration
and version (see cipher
negotiation matrix).
NONE Do not encrypt data in transit. All client and server versions.
Aspera strongly recommends
against using this setting.
The following table shows which encryption mode is used depending on the
server and client versions and settings:
Content Select Encrypt uploaded files with a password to encrypt the uploaded files
Protection with the specified password (client-side encryption at rest). The protected
file has the extension .aspera-env appended to the file name. Anyone
downloading the file must have the password to decrypt it.
Select Decrypt password-protected files downloaded to prompt for the
decryption password when downloading encrypted files.
For more information about client-side encryption at rest, see “Client-Side
Encryption-at-Rest (EAR)” on page 75.
File Attributes • Select Preserve Access Time to set the access time of the destination file to
the same value as that of the source file.
• Select Preserve Modification Timeto set the modification time of the
destination file to the same value as that of the source file.
• Select Preserve Source Access Time to keep the access time of the source
file the same as its value before the transfer.
Note: Access, modification, and source access times cannot be preserved for
node and Shares connections that are using cloud storage.
Source Handling Select Automatically delete source files after transfer to delete the files that
transferred successfully from the source.
Select Automatically move uploaded source files to a directory after
transfer and specify the location on the source machine to which they should
be moved. Only a path to an existing location on the client can be specified.
Select Delete empty source subdirectories to remove empty subdirectories
from the source once the files that they contain transfer successfully. This
option is usually used to clean up the Hot Folder when source files are moved
or deleted after transfer.
Import Connections
1. Right-click the remote server panel and select Import.
2. Enter the following information:
• Source file: The file with the exported connections.
• Options: Select the appropriate option, depending on how the connections were exported.
• Password: If you select the Passwords are encrypted option, enter the password that was set when
the connections were exported.
3. Click OK to import the connection information.
4. Before deleting the source file, confirm that the import process was successful by testing your
connections.
# asperascp
The SSH Keys dialog is also available from the Connection tab in the Connection Manager. When you
select Public Key for authentication, the Manage Keys button appears; clicking it opens the SSH Keys
dialog.
4. In the New SSH Key Pair window, enter the requested information.
Field Description
Identity Name your key pair, such as with your user name.
Passphrase (Optional) Set a passphrase on your SSH key, which will be prompted for
whenever it needs to use the key. If you don't want the user to be prompted
for passphrase when logging in, leave this field blank.
Type Select RSA (default) or ECDSA key.
Access When sharing a connection with public key authentication, or a connection that
is has a Hot Folder (on Windows machines), this option must be checked.
Importing keys:
To import keys created outside the GUI, go to Tools > Manage Keys to open the SSH Keys dialog. Click
the button in the upper-left corner of the dialog to open a file browser. You can import the key pair by
selecting either the private key or the public key; this will copy both keys into the user's .ssh directory.
You cannot import a key pair if a key pair with the same identity already exists in the .ssh directory.
Imported key pairs can be shared with other users. In the SSH Keys dialog, select a key and click the
button to open the Edit SSH Key Pair dialog. Select Access to allow shared connections to use this key.
Shared keys are moved to the Aspera etc directory.
Transferring Content
The GUI provides an easy, intuitive way to transfer content between the local computer and a remote
host.
Note: Do not use the following characters in file or folder names:
4. Connect to the remote server by either double-clicking the connection name, or select it and click
Connect.
5. Select the content to transfer (from the local or remote file system) and do any of the following:
• click the upload or download arrow
• drag and drop the files between the windows
• copy and paste the files between the windows
6. Once a transfer is started, you can manage the transfer rate, transfer policy, and priority. For
information, see “Managing Transfers” on page 27.
Managing Transfers
The Desktop Client GUI enables you to start, stop, and reorder transfers, as well as adjust transfer rates
and policies and configure transfer preferences.
D Transfer Monitor The transfer graph. Use the sliders on the vertical axis to adjust the
transfer rate up or down (if allowed).
Item Description
Global Bandwidth Limits The aggregated bandwidth cap for all FASP transfers on this computer.
Default Target Rate The initial download and upload rates for all transfers.
Maximum Active Transfers The maximum number of concurrent upload transfers and download
transfers.
For information about Email settings, see “Configuring Transfer Notifications” on page 31.
Tab Description
Transfer The transfer session-related options, such as the transfer speed and retry rules.
Tracking Options for tracking the transfer session, including the confirmation receipt and the
email notifications.
Filters Create filters to skip or include files that match certain patterns.
Security Enable the transfer encryption and the content protection.
File Handling Set up resume rule, preserve transferred file attributes, and remove or move source
files.
Scheduling Schedule the transfer.
3. Schedule the transfer.
Note: When scheduling transfers, ensure that the Desktop Client GUI stays open and running.
Scheduled transfers do not run when the application is closed.
To enable transfer scheduling, select Schedule this transfer.
The following scheduling options are available in the Transfer repeats drop-down menu:
Does not repeat
Set the time and date for a single transfer.
# asperascp
c) To turn on email notifications for all users, select Enable email notifications.
Enter the email address from which the notifications are sent in the From Address field and enter
the outgoing email server host name in the Host field. The other values are optional.
d) To test your settings, click Send test email, which sends a test message to the From Address.
3. Set your personal mail preferences.
Personal mail preferences override global settings.
a) Click Preferences.
b) Click Mail and edit the inherited global default values.
To restore your settings to global values, click Restore Defaults.
3. For new templates, name the template and select its base template.
Select an existing template from the Based On menu. Click OK.
4. Edit the template text.
The Edit Template window has four fields:
Field Description
Name The template name.
HTML The HTML mail body. Click Insert Image to insert an image into the template.
The image is copied to the template directory. Preview the template by clicking
Preview.
Text The plain text mail body. Preview the template by clicking Preview.
Access Select Share this template with all users on this computer to allow other
system users to access this template.
To generate content only under specific conditions, use a conditional statement. To construct a
conditional statement, use #if, #else, and #end, with the following syntax:
#if
...
#else
...
#end
All conditional statements are categorized in four parts: the conditional (what must occur to trigger the
action), session information (what action is triggered), time, and statistics.
Conditional
Use conditional tests in an if statement. For example:
#if ($event.isFailed())
...
#end
Statement Description
$event.isStarted() If the transfer session is started.
Session Information
Statement Description
$event.getSourceHost() The source host name (or host address if the host
name is not discoverable).
$event.getSourceHostAddress() The source host address.
$event.getSourcePaths() The source file path.
$event.getDestinationHost() The destination host name (or host address if the host
name is not discoverable).
$event.getDestinationHostAddress() The destination host address.
$event.getDestinationPath() The destination file path.
$event.getInitiatingHost() The session-initiating host name (or host address if
the host name is not discoverable).
$event.getInitiatingHostAddress() The session-initiating host address.
$event.getId() The session ID.
$event.getName() The session name.
$event.getType().getDescription() The session state. Three outputs: "STARTED",
"FAILED", and "COMPLETED".
$event.getUser() The transfer login.
$event.getFiles() The files that are being transferred. Use this
statement in a foreach loop: (Any text after ## is
a comment)
Time
Statement Description
$formatter.date(var, Formatting the date and time output. Enter three values in the
"lang", "format") parenthesis:
• var is either $event.getStartTime() or
$event.getEndTime()
• lang is an abbreviated language name; for example, en for English.
• format is the display format. Use these symbols:
– yyyy The year; for example, 2010.
– MM Month of the year; for example, 03.
– dd Day of the month; for example, 26.
– HH Hour of the day; for example, 16.
– mm Minute.
– ss Second.
– z Time zone.
– EEE The abbreviated weekday name; for example, Fri.
For example,
Statistics
Statement Description
$event.getSourceFileCount() The number of source files.
$event.getCompletedFileCount() The number of files that successfully transferred.
$event.getFailedFileCount() The number of files that failed to transfer.
$event.getAverageRatePercentage() The average transfer rate in bps. Enclose this
statement with $formatter.formatRate() to
simplify the output.
$event.getAverageLossPercentage() The average packet loss percentage.
$event.getSourceSizeB() The source file size. Enclose this statement with
$formatter.toBestUnit() to simplify the output.
$event.getTransferredB() The transferred file size. Enclose this statement with
$formatter.toBestUnit() to simplify the output.
$event.getWrittenB() The destination file size. Enclose this statement with
$formatter.toBestUnit() to simplify the output.
5. Click OK to save your changes.
Item Description
When Check the events for which to send notifications.
Subject Customize the subject line, which can use the same template fields as
described in “Configuring Transfer Notifications” on page 31.
To Enter the recipients, comma separated.
Template Select a mail template.
Message Optionally enter a message to include in the notifications.
c) Click OK to save your changes.
Ascp Syntax
ascp options [[username@]src_host:]source1[ source2 ...] [[username@]dest_host:]dest_path
username
The username of the Aspera transfer user can be specified as part of the source or destination,
whichever is the remote server. It can also be specified with the --user option. If you do not specify
a username for the transfer, the local username is authenticated by default.
Note: If you are authenticating on a Windows computer as a domain user, the transfer server
strips the domain from the username. For example, Administrator is authenticated rather than
DOMAIN\Administrator. For this reason, you must specify the domain explicitly.
src_host
The name or IP address of the computer where the files or directories to be transferred reside.
source
The file or directory to be transferred. Separate multiple arguments with spaces.
The growing files feature can be used with the source option to start transferring files to the target
directory while they are still being written to the source directory.
Note: To use the growing files feature, the source file must be on a native file system. Growing files
cannot be larger than 8 exabyte - 1 (9,223,372,036,854,775,807 bytes). However, the maximum file
size of the file system will override a setting that is larger.
Growing files use can also be configured statically with aspera.conf, see aspera.conf - File System
Configuration. See also “Ascp General Examples” on page 53.
To use the growing files feature with ascp, the source parameter can be used with the following
syntax:
A file transfer is deemed to be complete when the time since last update to the source file reaches the
wait_time value (in seconds). However, the time is only sampled when all currently available source
data has been transferred. In other words, if more data arrives after the wait time has elapsed, but the
transfer is still in progress, the additional data will still be transferred.
Any value for mtime that meets this criteria is acceptable to flag this condition except mtime =
0,which is used to flag a file error. You can, for example, use touch 1. If mtime is not adjusted
before the timeout is reached, an error will be generated.
An alternative method for defining the wait_time value is to use modifiers for powers of 1,024.
However, the value must be less than 8 * 2^60. The modifier must consist of an integer, and a unit
specifier. The unit specifiers are:
• k or K for 1 * 1024
• m or M for 1 * 1024 * 1024
• g or G for 1 * 1024 * 1024 * 1024
dest_host
The name or IP address of the computer where the source files or directories are to be transferred.
dest_path
The destination directory where the source files or directories are to be transferred.
• If the source is a single file, the destination can be a filename. However, if there are multiple source
arguments, the destination must be a directory.
• To transfer to the transfer user's docroot, specify "." as the destination.
• If the destination is a symbolic link, then the file or directory is written to the target of the symbolic
link.
Environment Variables
The following environment variables can be used with the ascp command. The total size for environment
variables depends on your operating system and transfer session. Aspera recommends that each
environment variable value should not exceed 4096 characters.
ASPERA_DST_PASS=password
The password to authenticate a URI destination.
ASPERA_LOCAL_TOKEN=token
A token that authenticates the user to the client (in place of SSH authentication).
Note: If the local token is a basic or bearer token, the access key settings for cipher and
preserve_time are not respected and the server settings are used. To set the cipher and timestamp
preservation options as a client, set them in the command line.
Ascp Options
-6
Enable IPv6 address support. When specifying an IPv6 numeric host for src_host or dest_host, write
it in brackets. For example, username@[2001:0:4137:9e50:201b:63d3:ba92:da]:/path or
--host=[fe80::21b:21ff:fe1c:5072%eth1].
-@ range_start:range_end
Transfer only part of a file: range_start is the first byte to send, and range_end is the last. If either
position is unspecified, the file's first and last bytes (respectively) are assumed. This option only works
for downloads of a single file and does not support transfer resume.
-A, --version
Display version and license information.
--apply-local-docroot
Apply the local docroot that is set in aspera.conf for the transfer user. Use to avoid entering object
storage access credentials in the command line. This option is equivalent to setting the environment
variable ASPERA_SCP_DOCROOT.
-C nodeid:nodecount
Enable multi-session transfers (also known as parallel transfers) on a multi-node/multi-core system.
A node ID (nodeid) and count (nodecount) are required for each session. nodeid and nodecount can
be 1-128, but nodeid must be less than or equal to nodecount, such as 1:2, 2:2. Each session must
use a different UDP port specified with the -O option. Large files can be split across sessions, see
--multi-session-threshold. For more information, see the IBM Aspera High-Speed Transfer
Server.
-c cipher
Encrypt in-transit file data using the specified cipher. Aspera supports three sizes of AES cipher
keys (128, 192, and 256 bits) and supports two encryption modes, cipher feedback mode (CFB)
and Galois/counter mode (GCM). The GCM mode encrypts data faster and increases transfer speeds
compared to the CFB mode, but the server must support and permit it.
Cipher rules
The encryption cipher that you are allowed to use depends on the server configuration and the version
of the client and server:
• When you request a cipher key that is shorter than the cipher key that is configured on the server,
the transfer is automatically upgraded to the server configuration. For example, when the server
setting is AES-192 and you request AES-128, the server enforces AES-192.
none Do not encrypt data in transit. Aspera All client and server versions.
strongly recommends against using
this setting.
--check-sshfp=fingerprint
Compare fingerprint to the server SSH host key fingerprint that is set with
<ssh_host_key_fingerprint> in aspera.conf. Aspera fingerprint convention is to use a hex
/tmp/code/compute.php
doc_dir
images/iris.png
images/rose.png
then the destination, in this case the transfer user's docroot, will contain the following:
compute.php
doc_dir (and its contents)
iris.png
rose.png
Restrictions:
• The command line cannot use the user@host:source syntax. Instead, specify this information
with the options --mode, --host, and --user.
• Paths specified in the file list cannot use the user@host:source syntax.
• Because multiple sources are being transferred, the destination must be a directory.
• Only one --file-list or --file-pair-list option is allowed per ascp session. If multiple lists
are specified, only the last one is used.
• Only files and directories specified in the file list are transferred; any sources specified on the
command line are ignored.
• If the source paths are URIs, the size of the file list cannot exceed 24 KB.
To create a file list that also specifies destination paths, use --file-pair-list.
--file-manifest={none|text}
Generate a list of all transferred files when set to text. Requires --file-manifest-path to specify
the location of the list. (Default: none)
--file-manifest-path=directory
Save the file manifest to the specified location when using --file-manifest=text. File manifests
must be stored locally. For cloud or other non-local storage, specify a local manifest path.
--file-manifest-inprogress-suffix=suffix
Apply the specified suffix to the file manifest's temporary file. For use with --file-manifest=text.
(Default suffix: .aspera-inprogress)
--file-pair-list=file
Transfer files and directories listed in file to their corresponding destinations. Each source is specified
on a separate line, with its destination on the line following it.
Specify destinations relative to the transfer user's docroot. Even if a destination is specified as an
absolute path, the path at the destination is still relative to the docroot. Destination paths specified in
the list are created automatically if they do not already exist.
Dir1
Dir2
my_images/iris.png
project_images/iris.png
/tmp/code/compute.php
/tmp/code/compute.php
/tmp/tests/testfile
testfile2
then the destination, in this case the transfer user's docroot, now contains the following:
Restrictions:
• The command line cannot use the user@host:source syntax. Instead, specify this information
with the options --mode, --host, and --user.
• The user@host:source syntax cannot be used with paths specified in the file list.
• Because multiple sources are being transferred, the destination specified on the command line
must be a directory.
• Only one --file-pair-list or --file-list option is allowed per ascp session. If multiple lists
are specified, only the last one is used.
• Only files from the file pair list are transferred; any additional source files specified on the command
line are ignored.
• If the source paths are URIs, the file list cannot exceed 24 KB.
For additional examples, see “Ascp General Examples” on page 53.
-G write_size
If the transfer destination is a server, use the specified write-block size, which is the maximum
number of bytes that the receiver can write to disk at a time. Default: 256 KB, Range: up to 500 MB.
This option accepts suffixes "M" or "m" for mega and "K" or "k" for kilo, such that a write_size of 1M is
one MB.
This is a performance-tuning option that overrides the write_block_size set in the client's
aspera.conf. However, the -G setting is overridden by the write_block_size set in the
server's aspera.conf. The receiving server never uses the write_block_size set in the client's
aspera.conf.
-g read_size
If the transfer source is a server, use the specified read-block size, which is the maximum number of
bytes that the sender reads from the source disk at a time. Default: 256 KB, Range: up to 500 MB. This
option accepts suffixes "M" or "m" for mega and "K" or "k" for kilo, such that a read_size of 1M is one
MB.
This is a performance-tuning option that overrides the read_block_size set in the client's
aspera.conf. However, the -g setting is overridden by the read_block_size set in the server's
aspera.conf. When set to the default value, the read size is the default internal buffer size of the
server, which might vary by operating system. The sending server never uses the read_block_size
set in the client's aspera.conf.
-h, --help
Display the help text.
To support this command, the ascp.conf file would have to include the following configuration:
<default>
<file_system>
<access>
<paths><path><absolute>
file:////tmp?grow=120;wait_start=null_read
</absolute></path></paths>
</access>
</file_system>
</default>
For more information, see the discussion of ascp.conf configuration for growing files in aspera.conf -
File System Configuration.
• Fair-policy transfer
Fair-policy transfer with maximum rate 100 Mbps and minimum at 1 Mbps, without encryption, transfer
all files in \local-dir\files to 10.0.0.2:
• Fixed-policy transfer
Fixed-policy transfer with target rate 100 Mbps, without encryption, transfer all files in \local-
dir\files to 10.0.0.2:
/tmp/folder1
tmp1
/tmp/folder2
tmp2
/tmp1/folder1
/tmp2/folder2
The file is saved on the server as file.aspera-env, with the extension indicating that the file is
encrypted. See the next example for how to download and decrypt an encrypted file from the server.
• Download with content protection and decryption
Download an encrypted file, file.aspera-env, from the server 10.0.0.2 and decrypt while
transferring:
• Transfer a file but do not save results to disk (no destination storage required)
Transfer the file /tmp/sample as user root to 10.0.0.2, but do not save results to disk:
• Transfer random data and do not save result to disk (no source or destination storage required)
Transfer 10 MB of random data from 10.0.0.2 as user root and do not save result to disk:
• Upload only the contents of a directory (not the directory itself) by using the --src-base option:
Upload only the contents of /data/ to the /storage/ directory at the destination. Strip the /src/
data/ portion of the source path and preserve the remainder of the file structure at the destination:
• Upload a directory and its contents to a new directory by using the -d option.
Upload the /data/ directory to the server and if it doesn't already exist, create the new folder /
storage2/ to contain it, resulting in /storage2/data/ at the destination.
• Upload the contents of a directory, but not the directory itself, by using the --src-base option:
Upload all folders and files in the /clips/out/ folder, but not the out/ folder itself, to the /in/ folder
at the destination.
Result: The source folders and their content appear in the in directory at the destination:
Without --src-base, the example command transfers not only the contents of the out/ folder, but the
folder itself.
Note: Sources located outside the source base are not transferred. No errors or warnings are
issued, but the skipped files are logged. For example, if /clips/file4 were included in the above
example sources, it would not be transferred because it is located outside the specified source base, /
clips/out/.
• Upload only the contents of a file and a directory to a new directory by using --src-base
Upload a file, /monday/file1, and a directory, /tuesday/*, to the /storage/ directory on the
server, while stripping the srcbase path and preserving the rest of the file structure. The content is
saved as /storage/monday/file1and /storage/tuesday/* on the server.
• Download only the contents of a file and a directory to a new directory by using --src-base
Download a file, /monday/file1, and a directory, /tuesday/*, from the server, while stripping the
srcbase path and preserving the rest of the file structure. The content is saved as /data/monday/
file1 and /data/tuesday/* on the client.
• Move the source file on the client after it is uploaded to the server by using --move-after-
transfer
Uploadfile0012 to Pat's docroot on the server at 10.0.0.1, and move (not copy) the file from C:/
Users/Pat/srcdir/ to C:/Users/Pat/Archive on the client.
• Move the source file on the server after it is downloaded to the client by using --move-after-
transfer
Download srcdir from the server to C:/Users/Pat on the client, and move (not copy) srcdir to the
archive directory /Archive on the server.
• Move the source file on the client after it is uploaded to the server and preserve the file structure
one level above it by using --src-base and --move-after-transfer
Upload file0012 to Pat's docroot on the server at 10.0.0.1, and save it as /srcdir/
file0012 (stripped of C:/Users/Pat). Also move file0012 from C:/Users/Pat/srcdir/ to
C:/Users/Pat/Archive on the client, where it is saved as C:/Users/Pat/Archive/srcdir/
file0012.
• Delete a local directory once it is uploaded to the remote server by using --remove-after-
transfer and --remove-empty-directories
Upload /content/ to the server, then delete its contents (excluding partial files) and any empty
directories on the client.
• Delete a local directory once its contents have been transferred to the remote server by using
--src-base, --remove-after-transfer, and --remove-empty-directories
Upload /content/ to the server, while stripping the srcbase path and preserving the rest of the
file structure. The content is saved as /storage/* on the server. On the client, the contents of /
content/, including empty directories but excluding partial files, are deleted.
With ASPERA_DEST_PASS and ASPERA_SCP_PASS set, run ascp with the syntax listed in the table for
transfers with no docroot configured, except that you do not need to include the storage password or
access key, and are not prompted for the Aspera password upon running ascp.
Upload example:
Download syntax:
Download example:
Upload example:
Download syntax:
Download example:
# ascp --mode=recv --
user=AS037d8eda429737d6 --host=dev920350144d2.azure.asperaondemand.com
azu://astransfer:[email protected]/abc /downloads
Google Cloud Note: The examples below require that the VMI running the Aspera server is a
Storage Google Compute instance.
Upload example:
Download syntax:
Download example:
HDFS Aspera recommends running ascp transfers with HDFS with a docroot configured.
IBM COS - S3 Upload syntax:
Upload example:
Download syntax:
Download example:
{
"aspera": {
"cloud-metadata": [
{"key1":"value1"},
{"key2":"value2"},
...
] } }
• Download the file remote_file from the server at 10.0.0.2 to stdout on the client. Transfer at 100
Mbps.
• Upload the file local_file to the server at 10.0.0.2 to the named pipe /tmp/outpipe. Transfer at
100 Mbps.
Multi-File Transfers
Ascp can transfer one or more files in an encoded, streamed interface, similar to single file transfers. The
primary difference is that the stream includes headers that demarcate data from individual files.
To upload files that are piped into stdin, set the source as stdio-tar://. The file modification time is
set to the time at which the upload starts.
[0 - n blank lines]
File: /path/to/file_1
Size: file_size
Offset: bytes
file_1 data
[0 - n blank lines]
File: /path/to/file_2
Size: file_size
file 2 data
...
To download one or more files to stdout, set the destination as stdio-tar://. Normal status output to
stdout is suppressed during downloads because the transfer output is streamed to stdout. The data sent
to stdout has the same encoding as described for uploads.
To download to a named pipe, set the destination to stdio-tar:////path, where path is the path of
the named pipe.
When an offset is specified, the bytes that are sent replace the existing bytes in the destination file (if it
exists). The bytes added to the destination file can extend beyond the current file size. If no offset is set,
the bytes overwrite the file if overwrite conditions are met.
Restrictions:
• When downloading to stdio-tar://, the source list must consist of individual files only. Directories
are not allowed.
• Only --overwrite=always or --overwrite=never are supported with stdio-tar://. The
behavior of --overwrite=diff and --overwrite=diff+older is undefined.
• Offsets are only supported if the destination files are located in the native file system. Offsets are not
supported for cloud destinations.
Multi-file Transfer Examples:
• Upload two files, myfile1 (1025 bytes) and myfile2 (20 bytes), to /remote-dir on the server at
10.0.0.2. Transfer at 100 Mbps.
File: myfile1
Size: 1025
• Uploading multiple files from stdin by using a persistent session is the same as a non-persistent
session.
• Update bytes 10-19 in file /remote-dir/myfile1 on the server at 10.0.0.2 at 100 Mbps.
File: myfile1
Size: 10
Offset: 10
• Upload two files, myfile1 and myfile2, to the named pipe /tmp/mypipe (streaming output) on the
server at 10.0.0.2. Transfer at 100 Mbps.
This sends an encoded stream of myfile1 and myfile2 (with the format of sourcefile in the upload
example) to the pipe /tmp/mypipe. If /tmp/mypipe does not exist, it is created.
• Download the files from the previous example from 10.0.0.2 to stdout. Transfer at 100 Mbps.
FASPMGR 2
Type: START
Source: /tmp/myfile1
Destination: mynewfile1
FASPMGR 2
Type: START
Source: /tmp/myfile2
Destination: mynewfile2
FASPMGR 2
Type: DONE
File: mynewfile1
Size: file_size
mynewfile1_data
File: mynewfile2
Size: file_size
mynewfile2_data
• Upload two files, myfile1 and myfile2, to named pipe /tmp/mypipe on the server at 10.0.0.2.
Transfer at 100 Mbps.
File: myfile1
Size: 1025
-N 'rule1' -E 'rule2' Transfer all files and directories with names that match rule1, as well as all
other files and directories except those with names that match rule2.
-E 'rule1' -N 'rule2' Transfer all files and directories except those with names that match rule1. All
files and directories not already excluded by rule1 are included because no
additional exclude rule follows -N 'rule2'.
To transfer only the files and directories with names that do not match rule1
but do match rule2 use:
Rule Patterns
Rule patterns (globs) use standard globbing syntax that includes wildcards and special characters, as well
as several Aspera extensions to the standard.
• Character case: Case always matters, even if the file system does not enforce such a distinction. For
example, on Windows FAT or NTFS file systems and macOS HPFS+, a file system search for "DEBUG"
returns files "Debug" and "debug". In contrast, Ascp filter rules use exact comparison, such that "debug"
does not match "Debug". To match both, use "[Dd]ebug".
• Partial matches: With globs, unlike standard regular expressions, the entire filename or directory name
must match the pattern. For example, the pattern abc*f matches abcdef but not abcdefg.
Standard Globbing: Wildcards and Special Characters
Where the user is "my_user_name", and the target is the Upload directory.
At the prompt, enter my_user_name's password.
3. Create a destination directory on your computer, for example /tmp/dest.
4. Download your files from the demo server to /tmp/dest to test your filtering rules. For example:
5. Compare the destination directory with the source to determine if the filter behaved as expected.
The diff output shows the missing files and directories (those that were not transferred).
AAA/abc/def
AAA/abc/.def
AAA/abc/.wxy/def
AAA/abc/wxy/def
AAA/abc/wxy/.def
AAA/abc/wxy/tuv/def
AAA/abc/xyz/def/wxy
AAA/wxyfile
AAA/wxy/xyx/
AAA/wxy/xyxfile
-N '*' -E 'AAA/**'
Results:
AAA/abc/def
AAA/abc/wxy/def
AAA/abc/wxy/tuv/def
AAA/abc/xyz/def/wxy
AAA/wxyfile
AAA/wxy/xyx/
-E 'wxy*'
Results:
AAA/abc/def
AAA/abc/.def
AAA/abc/.wxy/def
AAA/abc/xyz/def/
< AAA/abc/wxy/def
< AAA/abc/wxy/.def
< AAA/abc/wxy/tuv/def
< AAA/abc/xyz/def/wxy
< AAA/wxyfile
< AAA/wxy/xyx/
< AAA/wxy/xyxfile
3. Include directories and files that start with "wxy" if they fall directly under AAA:
-N 'wxy*' -E 'AAA/**'
Results:
AAA/wxy/
AAA/wxyfile
< AAA/abc/def
< AAA/abc/.def
< AAA/abc/.wxy/def
< AAA/abc/wxy/def
< AAA/abc/wxy/.def
< AAA/abc/wxy/tuv/def
< AAA/abc/xyz/def/wxy
< AAA/wxy/xyx/
< AAA/wxy/xyxfile
4. Include directories and files at any level that start with wxy, but do not include dot-files, dot-
directories, or any files under the wxy directories (unless they start with wxy). However, subdirectories
under wxy will be included:
-N '*/wxy*' -E 'AAA/**'
Results:
AAA/abc/wxy/tuv/
AAA/abc/xyz/def/wxy
AAA/wxyfile
AAA/wxy/xyx/
< AAA/abc/def
< AAA/abc/.def
< AAA/abc/.wxy/def
< AAA/abc/wxy/def *
< AAA/abc/wxy/.def
< AAA/abc/wxy/tuv/def
< AAA/wxy/xyxfile
Results:
AAA/abc/wxy/def
AAA/abc/wxy/.def
AAA/abc/wxy/tuv/def
6. Exclude directories and files starting with wxy, but only those found at a specific location in the tree:
-E '/AAA/abc/wxy*'
Results:
AAA/abc/def
AAA/abc/.def
AAA/abc/.wxy/def
AAA/abc/xyz/def/wxy
AAA/wxyfile
AAA/wxy/xyx/
AAA/wxy/xyxfile
< AAA/abc/wxy/def
< AAA/abc/wxy/.def
< AAA/abc/wxy/tuv/def
7. Include the wxy directory at a specific location, and include all its subdirectories and files, including
those starting with ".":
-N 'AAA/abc/wxy/**' -E 'AAA/**'
Results:
AAA/abc/wxy/def
AAA/abc/wxy/.def
AAA/abc/wxy/tuv/def
< AAA/abc/def
< AAA/abc/.def
< AAA/abc/.wxy/def
< AAA/abc/xyz/def/wxy
< AAA/wxyfile
< AAA/wxy/xyx/
< AAA/wxy/xyxfile
Client setting = Copy and Copy and Copy and Copy and replace Copy and
copy+force replace any replace any replace any any existing files. replace any
existing files. existing existing existing
files. files. files.
Client setting = skip Skip Skip Skip Skip Skip
# asconfigurator -x "set_node_data;symbolic_links,value"
# asconfigurator -x "set_user_data;user_name,username;symbolic_links,value"
$ mkdir /home/username/.ssh
$ cd /home/username/.ssh
3. As the root user, make sure that the SSH key is owned by the transfer user and that proper restrictive
permissions are set. SSH keys must only be readable by the key owner.
Use the following command syntax, where username is the transfer user name and id_rsa is the
key-pair's filename.
In this example, you are connecting to the server (10.0.0.2, directory /space) with the user account
jane and the private key ~/.ssh/id_rsa.
Reporting Checksums
File checksums are useful for trouble-shooting file corruption, allowing you to determine at what point in
the transfer file corruption occurred. Aspera servers can report source file checksums that are calculated
on-the-fly during transfer and then sent from the source to the destination.
To support checksum reporting, the transfer must meet both of the following requirements:
• Both the server and client computers must be running HSTS or HSTE.
• The transfer must be encrypted. Encryption is enabled by default.
The user on the destination can calculate a checksum for the received file and compare it (manually or
programmatically) to the checksum reported by the sender. The checksum reported by the source can be
retrieved in the destination logs, a manifest file, in IBM Aspera Console, or as an environment variable.
Instructions for comparing checksums follow the instructions for enabling checksum reporting.
Checksum reporting is disabled by default. Enable and configure checksum reporting on the server by
using the following methods:
• Edit aspera.conf with asconfigurator.
• Set options in the client GUI.
• Set ascp command-line options (per-transfer configuration).
Command-line options override the settings in aspera.conf and the GUI.
Important: When checksum reporting is enabled, transfers of very large files (>TB) take a long time to
resume because the entire file must be reread.
The location where manifest files are written. The location can be
file_manifest_path
an absolute path or a path relative to the transfer user's home
File Manifest Path
directory. If no path is specified (default), the file is generated
--file_manifest_path=path
under the destination path at the receiver, and under the first
source path at the sender.
Note: File manifests can be stored only locally. Thus, if you are
using S3 or other non-local storage, you must specify a local
manifest path.
# asconfigurator -x "set_node_data;file_checksum,checksum"
To enable and configure the file manifest where checksum report data is stored, run the following
commands:
# asconfigurator -x "set_node_data;file_manifest,text"
# asconfigurator -x "set_node_data;file_manifest_path,filepath"
These commands create lines in aspera.conf as shown in the following example, where checksum type
is md5, file manifest is enabled, and the path is /tmp.
<file_system>
...
<file_checksum>md5</file_checksum>
<file_manifest>text</file_manifest>
<file_manifest_path>/tmp</file_manifest_path>
...
</file_system>
In the examples above, the manifest is generated when files are transferred and saved as a text file called
aspera-transfer-transfer_id-manifest.txt in the directory /tmp.
Comparing Checksums
If you open a file that you downloaded with Aspera and find that it is corrupted, you can determine when
the corruption occurred by comparing the checksum that is reported by Aspera to the checksums of the
files on the destination and on the source.
1. Retrieve the checksum that was calculated by Aspera as the file was transferred.
• If you specified a file manifest and file manifest path as part of an ascp transfer or Lua transfer event
script, the checksums are in that file in the specified location.
• If you specified a file manifest and file manifest path in the GUI or aspera.conf, the checksums are
in a file that is named aspera-transfer-transfer_id-manifest.txt in the specified location.
# md5sum filepath
3. Compare the checksum reported by Aspera with the checksum that you calculated for the corrupted
file.
• If they do not match, then corruption occurred as the file was written to the destination. Download
the file again and confirm that it is not corrupted. If it is corrupted, compare the checksums again.
If they do not match, investigate the write process or attempt another download. If they match,
continue to the next step.
• If they match, then corruption might have occurred as the file was read from the source. Continue to
the next step.
4. Calculate the checksums for the file on the source. These examples use the MD5 checksum method;
replace MD5 with the appropriate checksum method if you use a different one.
Windows:
Mac OS X:
$ md5 filepath
# md5sum filepath
AIX:
Solaris:
5. Compare the checksum of the file on the source with the one reported by Aspera.
• If they do not match, then corruption occurred when the file was read from the source. Download
the file again and confirm that it is not corrupted on the destination. If it is corrupted, continue to the
next step.
• If they match, confirm that the source file is not corrupted. If the source file is corrupted, replace it
with an uncorrupted one, if possible, and then download the file again.
# export ASPERA_SCP_FILEPASS=password
For more command line examples, see “Ascp General Examples” on page 53.
Note: When a transfer to HSTS falls back to HTTP or HTTPS, client-side EAR is no longer supported. If
HTTP fallback occurs while uploading, then the files are NOT encrypted. If HTTP fallback occurs while
downloading, then the files remain encrypted.
• To download client-side-encrypted files without decrypting them immediately, run the transfer without
decryption enabled (clear Decrypt password-protected files downloaded in the GUI or do not specify
--file-crypt=decrypt on the ascp command line).
• To decrypt encrypted files once they are on a computer with no network access, run the following
command:
Ascp Ascp4
-6
-@[range_low:range_high]
-A, --version -A, --version
--apply-local-docroot
-C nodeid:nodecount
-D | -DD | -DDD
-d
--delete-before
--delete-before-transfer** --delete-before-transfer**
--dest64 --dest64
-E pattern -E pattern
-e prepost_filepath
--exclude-newer-than=mtime
--exclude-older-than=mtime
-f config_file -f config_file
--faspmgr-io
--file-checksum=hash
--file-list=filepath** --file-list=filepath**
--file-pair-list=filepath
-G write_size
-g read_size
-h, --help -h, --help
-i private_key_file_path** -i private_key_file_path
-K probe_rate
-k {0|1|2|3} -k {0|1|2|3}
--keepalive --keepalive
-l max_rate -l max_rate
-L local_log_dir[:size] -L local_log_dir[:size]
-m min_rate -m min_rate
--memory=bytes
--meta-threads=num
--mode={send|recv} --mode={send|recv}
--move-after-transfer=archivedir
--multi-session-threshold=threshold
-O fasp_port -O fasp_port
--overwrite=method** --overwrite=method**
-P ssh-port -P ssh-port
-p -p
--partial-file-suffix=suffix --partial-file-suffix=suffix
--policy={fixed|high|fair|low} --policy={fixed|high|fair|low}
--precalculate-job-size
--preserve-access-time
--preserve-acls=mode
--preserve-creation-time
--preserve-file-owner-gid --preserve-file-owner-gid
--preserve-file-owner-uid --preserve-file-owner-uid
--preserve-modification-time
--preserve-source-access-time
--preserve-xattrs=mode
--proxy=proxy_url
-q -q
-R remote_log_dir -R remote_log_dir
--read-threads=num
--remote-memory=bytes
--remote-preserve-acls=mode
--remote-preserve-xattrs=mode
--remove-after-transfer
--remove-empty-source-directory
--resume (similar to -k)
--retry-timeout=secs
-S remote_ascp
--save-before-overwrite
--scan-threads=num
--source-prefix=prefix
--source-prefix64=prefix
--symbolic-links=method** --symbolic-links=method**
-T -T
-u user_string -u user_string
--user=username --user=username
-v
-W token_string | @token_filepath
-w{r|f}
-X rexmsg_size -X rexmsg_size
-Z dgram_size -Z dgram_size
Ascp FAQs
Answers to some common questions about controlling transfer behavior, such as bandwidth usage,
resuming files, and overwriting files.
1. How do I control the transfer speed?
You can specify a transfer policy that determines how a FASP transfer utilizes the network resource,
and you can specify target and minimum transfer rates where applicable. In an ascp command, use
the following flags to specify transfer policies that are fixed, fair, high, or low:
To improve the transfer speed, also consider upgrading the following hardware components:
Component Description
Hard disk The I/O throughput, the disk bus architecture (such as RAID, IDE, SCSI, ATA, and
Fiber Channel).
Network I/O The interface card, the internal bus of the computer.
CPU Overall CPU performance affects the transfer, especially when encryption is
enabled.
3. How do I ensure that if the transfer is interrupted or fails to finish, it will resume without re-
transferring the files?
Use the -k flag to enable resume, and specify a resume rule:
-k 0 – Always re-transfer the entire file.
-k 1 – Compare file attributes and resume if they match, and re-transfer if they do not.
Introduction to Ascp4
Ascp4 is a FASP transfer binary that is optimized for sending extremely large sets of individual files. The
executable, ascp4, is similar to ascp and shares many of the same options and capabilities, in addition to
data streaming capabilities.
Both Ascp4 and Ascp are automatically installed with IBM Aspera High-Speed Transfer Server, IBM
Aspera High-Speed Transfer Endpoint, and IBM Aspera Desktop Client.
ascp4 Syntax
ascp4 options [[user@]srcHost:]source_file1[,source_file2,...] [[user@]destHost:]dest_path
User
The username of the Aspera transfer user can be specified as part of the as part of the source or
destination, whichever is the remote server or with the --user option. If you do not specify a username
for the transfer, the local username is authenticated by default.
Note: If you are authenticating on a Windows machine as a domain user, the transfer server
strips the domain from the username. For example, Administrator is authenticated rather than
DOMAIN\Administrator. Thus, you must specify the domain explicitly.
Source and destination paths
• If there are multiple source arguments, then the target path must be a directory.
• To describe filepaths, use single quotes (' ') and forward slashes (/) on all platforms.
• To transfer to the transfer user's docroot, specify "." as the destination.
• Avoid the following characters in filenames: / \ " : ' ? > < & * |.
• If the destination is a symbolic link, then the file is written to the target of the symbolic link. However,
if the symbolic link path is a relative path to a file (not a directory) and a partial file name suffix is
configured on the receiver, then the destination path is relative to the user's home directory. Files within
directories that are sent to symbolic links that use relative paths are not affected.
URI paths: URI paths are supported, but only with the following restrictions:
• If the source paths are URIs, they must all be in the same cloud storage account. No docroot
(download), local docroot (upload), or source prefix can be specified.
• If a destination path is a URI, no docroot (upload) or local docroot (download) can be specified.
• The special schemes stdio:// and stdio-tar:// are supported only on the client side. They cannot
be used as an upload destination or download source.
• If required, specify the URI passphrase as part of the URI or set it as an environment variable
(ASPERA_SRC_PASS or ASPERA_DST_PASS, depending on the direction of transfer).
UNC paths: If the server is Windows and the path on the server is a UNC path (a path that points to a
shared directory or file on Windows operating systems) then it can be specified in an ascp4 command
using one of the following conventions:
1. UNC path that uses backslashes ( \ )
If the client side is a Windows machine, the UNC path can be used with no alteration. For example,
\\192.168.0.10\temp. If the client is not a Windows machine, every backslash in the UNC path
must be replaced with two backslashes. For example, \\\\192.168.0.10\\temp.
2. UNC path that uses forward slashes ( / )
Replace each backslash in the UNC path with a forward slash. For example, if the UNC path is
\\192.168.0.10\temp, change it to //192.168.0.10/temp. This format can be used with any
client-side operating system.
Environment Variables
If needed, you can set the following environment variables for use with an ascp4 session. The total size
for environment variables depends on your operating system and transfer session. Aspera recommends
that each environment variable value should not exceed 4096 characters.
ASPERA_SCP_PASS=password
The password that is used for SSH authentication of the transfer user.
ASPERA_SCP_TOKEN=token
Set the transfer user authorization token. Ascp4 currently supports transfer tokens, which must be
created by using astokengen with the --full-paths option. For more information, see "Transfer
Token Generation (astokengen)" in the IBM Aspera High-Speed
ASPERA_SCP_COOKIE=cookie
A cookie string that is passed to monitoring services.
ASPERA_SRC_PASS=password
The password that is used to authenticate to a URI source.
ASPERA_DST_PASS=password
Set the password that is used to authenticate to a URI destination.
ASPERA_LOCAL_TOKEN=token
A token that authenticates the user to the client (in place of SSH authentication).
Note: If the local token is a basic or bearer token, the access key settings for cipher and
preserve_time are not respected and the server settings are used. To set the cipher and timestamp
preservation options as a client, set them in the command line.
Ascp4 Options
-A, --version
Display version and license information.
-c {aes128|aes192|aes256|none}
Encrypt in-transit file data using the specified cipher. This option overrides the
<encryption_cipher> setting in aspera.conf.
--check-sshfp=fingerprint
Compare fingerprint to the server SSH host key fingerprint that is set with
<ssh_host_key_fingerprint> in aspera.conf. Aspera fingerprint convention is to use a hex
string without the colons; for example, f74e5de9ed0d62feaf0616ed1e851133c42a0082. For more
information on SSH host key fingerprints, see the Admin Guide: Securing your SSH Server.
--chunk-size=bytes
Perform storage read/write operations with the specified buffer size. Also use the buffer size as an
internal transmission and compression block. Valid range: 4 KB - 128 MB. For transfers with object
src
src2
...
srcN
To read a file list from standard input, use "-" in place of filepath (as ascp4 --file-list=- …).
UTF-8 file format is supported. Use with -d if the destination folder does not exist.
Restrictions:
• Paths in file lists cannot use user@host:filepath syntax. You must use --user with --file-
list.
• Only one --file-list option is allowed per ascp4 session. If multiple file lists are specified, all
but the last are ignored.
• Only files and directories from the file list are transferred, and any additional source files or
directories specified on the command line are ignored.
• If more than one read thread is specified (default is 2) for a transfer that uses --file-list, the
files in the file list must be unique. Duplicates can produce unexpected results on the destination.
• Because multiple sources are being transferred, the destination must be a directory.
• If the source paths are URIs, the size of the file list cannot exceed 24 KB.
For very large file lists (~100 MB+), use with --memory to increase available buffer space.
--file-manifest={none|text}
Generate a list of all transferred files when set to text. Requires --file-manifest-path to specify
the location of the list. (Default: none)
--file-manifest-path=directory
Save the file manifest to the specified location when using --file-manifest=text. File manifests
must be stored locally. For cloud or other non-local storage, specify a local manifest path.
--file-manifest-inprogress-suffix=suffix
Apply the specified suffix to the file manifest's temporary file. For use with --file-manifest=text.
(Default suffix: .aspera-inprogress)
-h, --help
Display the usage summary.
--host=host
Transfer to the specified host name or address. Requires --mode. This option can be used instead of
specifying the host as part of the filename (as hostname:filepath).
-i private_key_file
Authenticate the transfer using public key authentication with the specified SSH private key file
(specified with a full or relative path). The private key file is typically in the directory $HOME/.ssh/. If
multiple -i options are specified, only the last one is used.
-k {0|1|2|3}
Enable the resumption of partially transferred files at the specified resume level. Default: 0. This
option must be specified for your first transfer or it does not work for subsequent transfers. Resume
levels:
• -k 0: Always re-transfer the entire file (same as --overwrite=always).
• -k 1: Compare file modification time and size and resume if they match (same as --
overwrite=diff --compare=size --resume).
• -k 2: Compare sparse checksum and resume if they match (same as --overwrite=diff --
compare=md5-sparse --resume).
-X rexmsg_size
Limit the size of retransmission requests to no larger than the specified size, in bytes. Max: 1440.
-Z dgram_size
Use the specified datagram size (MTU) for FASP transfers. Range: 296-65535 bytes. Default: the
detected path MTU.
As of version 3.3, datagram size can be specified on the server by setting <datagram_size> in
aspera.conf. The server setting overrides the client setting, unless the client is using a version of
ascp that is older than 3.3, in which case the client setting is used. If the pre-3.3 client does not set
-Z, the datagram size is the discovered MTU and the server logs the message "LOG Peer client doesn't
support alternative datagram size".
# asconfigurator -x "set_node_data;transfer_protocol_options_chunk_size,value"
# asconfigurator -x "set_user_data;user_name,
username;transfer_protocol_options_chunk_size,value"
2. Set the chunk size in the client's aspera.conf to the Trapd chunk size.
If Trapd is using the default chunk size, run the following command to set the chunk size to 1 MB:
# asconfigurator -x "set_node_data;transfer_protocol_options_chunk_size,1048576"
Ascp4 Examples
The command options for ascp4 are generally similar to those for ascp. The following examples
demonstrate options that are unique to Ascp4. These options enable reading management commands
and enable read/write concurrency.
For Ascp examples, see “Ascp Command Reference” on page 38. See “Comparison of Ascp and Ascp4
Options” on page 76 for differences in option availability and behavior.
• Read FASP4 management commands
Read management commands V4 from management port 5000 and execute the management
commands. The management commands version 4 are PUT, WRITE and CLOSE.
• Increase concurrency
The following command runs ascp4 with two scan threads and eight read threads on the client, and
eight meta threads and 16 write threads on the server.
Appendix
Restarting Aspera Services
When you change product settings, you might need to restart certain Aspera services in order for the new
values to take effect.
faux:///dirname?file=file&count=count&size=size&inc=increment&seq=sequence&buf_init=buf_option
Where:
• dirname is a name for the directory (required)
• file is the root for file names, default is "file" (optional)
• count is the number of files in the directory (required)
• size is the size of the first file in the directory, default 0 (optional). size can be set with modifiers (k/K,
m/M, g/G, t/T, p/P, or e/E) to a maximum of 7x260 bytes (7 EiB).
• increment is the increment of bytes to use to determine the file size of the next file, default 0 (optional)
• sequence is how to determine the size of the next file: "sequential" or "random". Default is "sequential"
(optional). When set to "sequential", file size is calculated as:
Where rand is a random number between zero and one. If necessary, increment is automatically
adjusted to prevent the file size from being negative.
For both options, increment is adjusted to prevent the file size from from exceeding 7x260 bytes.
• buf_option is how faux source data are initialized: "none", "zero", or "random". Default is "zero". "none" is
not allowed for downloads (Ascp run with --mode=recv).
When the defaults are used, Ascp sends a directory that is named dirname and that contains count
number of zero-byte files that are named file_count.
For example, to transfer a faux directory ("mydir") that contains 1 million files to /tmp on 10.0.0.2, and
the files in mydir are named "testfile" and file size increases sequentially from 0 to 2 MB by an increment
of 2 bytes:
Faux Target
To send data but not save the results to disk at the destination (do not write to the target), specify the
target as faux://.
To leave more network resources for other high-priority traffic, use the Fair policy and adjust the target
rate and minimum rate by sliding the arrows or entering values.
2. Test the maximum bandwidth.
Note: This test will typically occupy a majority of the network's bandwidth. Aspera recommends
performing it on a dedicated file transfer line or during a time of very low network activity.
Use Fixed policy for the maximum transfer speed. Start with a lower transfer rate and increase
gradually toward the network bandwidth.
Component Description
Hard disk The I/O throughput, the disk bus architecture (such as RAID, IDE, SCSI, ATA, and
Fiber Channel).
aclean Reference
The Aspera aclean command-line tool is a fast method of deleting directories and files from local and
object storage. Directories and files can be filtered based on their last modified times. For Windows
operating systems, the created time (CTIME) and modified time (MTIME) are used as the matching criteria.
You can do a dry run of an aclean command to test what content will be deleted. aclean can be run on
any platform on which Ascp4 is supported.
Note: The directory specified in an aclean command is not deleted. Only the content in the directory that
matches the options is deleted.
Syntax
Examples
Delete the contents of the local directory /temp/logs-test/:
$ aclean /temp/logs-test/
Delete files that have a last-modified time older than March 27, 2017 13:34 from Azure object storage:
Log Files
The application log file includes detailed transfer information and can be useful for review and support
requests. You can configure log rotation and redirect Aspera logging so that it is not recorded in the
system log file.
Note: To view logs from the command line in Linux, you must have a functional web-browser or other
default application for opening HTML files.
To set the logging level for transfers, open the My Preferences dialog by clicking Tools > Preferences or
by clicking Preferences in the upper-right corner of the application window.
The five logging levels to select from are: Off, Error, Warn, Info, and Debug. The system default is Info.
*.info;mail.none;authpriv.none;cron.none /var/log/messages
Change it to:
*.info;mail.none;authpriv.none;cron.none;local2.none /var/log/messages
Next, forward local2.info log messages to your new file. For example, to write to /var/log/
aspera.log, add the following line just below the line you modified above:
local2.info -/var/log/aspera.log
The log file name should be separated from the log facility (local2.info) by tab characters, not spaces
and be preceded by a hyphen. The hyphen before the log file name allows for asynchronous logging.
Next, restart the syslog daemon to have it load the new configuration:
filter f_local { facility(local0, local1, local2, local3, local4, local5, local6, local7); };
# rcsyslog restart
2. Edit /etc/logrotate.conf by adding the configuration after the line "# system-specific
logs may also be configured here." The following example compresses and rotates 10 logs
/var/log/aspera.log {
rotate 10
size 100M
create 664 root
postrotate
/usr/bin/killall -HUP syslogd
endscript
compress
}
The following example compresses and rotates 10 logs once daily. Instead of moving the original log file
and creating a new one, the copytruncate option tells logrotate to first copy the original log file, then
truncate it to zero bytes.
/var/log/aspera.log {
daily
rotate 10
copytruncate
compress
}
<ascmd>
<log_cmd>
<as_info>no</as_info>
<as_ls>no</as_ls>
<as_rm>no</as_rm>
<as_du>no</as_du>
<as_df>no</as_df>
<as_mkdir>no</as_mkdir>
<as_cp>no</as_cp>
<as_mv>no</as_mv>
<as_md5sum>no</as_md5sum>
</log_cmd>
</ascmd>
As an example of asconfigurator usage, the following command specifies that any deletions from the
server file system by user xeno are logged:
asconfigurator -x "set_user_data;user_name,xeno;ascmd_log_cmd_as_rm,yes"
<ascmd>
<log_cmd>
<as_rm>yes</as_rm>
</log_cmd>
</ascmd>