Transfer acceleration

The Transfer CFT transfer acceleration option takes advantage of an available combination of latency and high bandwidth, which improves transfer speeds, especially when dealing with large files.

This page begins with basic concepts, and describes:

Concepts

Bandwidth

In the context of network capacity, bandwidth is the bit-rate of available or consumed information capacity expressed typically in metric multiples of bits per second.

Latency

Latency refers to the round-trip delay time (RTD) or round-trip time (RTT), which is the length of time it takes for a signal to be sent plus the length of time it takes for an acknowledgment of that signal to be received.

TCP Windows

When using the TCP protocol, sent data needs to be acknowledged before sending the next bit of data. TCP Windows is the amount of data that can be sent without getting any acknowledgment.

Typically the TCP Windows is 65,535 bytes. On some operating system or network equipment, TCP Windows may be increased from 65,535 bytes to 1 gigabyte.

Bandwidth Delay Product (BDP)

Bandwidth-delay product refers to the result of multiplying the data link's capacity (in bits per second) by its round-trip delay time (in seconds).

Packet loss

Packet loss reflects network congestion between a sender and receiver, which may however be due to factors other than network congestion, such as hardware defaults.

Typical network use cases

The following table displays use cases where it may be helpful to implement transfer acceleration.

Use case pTCP UDT
Low Bandwidth Network No benefit. No benefit.
Low quality Network (lot of disconnections and/or packets corruption or lost) No benefit.

Benefits from acceleration.

UDT should help with lost packets or packets corruption.

High BDP network where the TCP Windows is smaller than the BDP

Benefits from acceleration.

Benefits from acceleration.
No dedicated network and there is a lot of concurrent activities Benefits from acceleration. Benefits from acceleration because UDT is fair.
High BDP network but the TCP Windows is greater than the BDP To benefit you must set the TCP Windows value to less than the BDP. Benefits from acceleration.

When to use pTCP or UDT

Before implementing either protocol:

  1. Evaluate your network bandwidth and RTT, which gives you the BDP (Bandwidth Delay Product).
  2. Evaluate the TCP receive window (RWIN).
  3. Determine if acceleration is possible based on factors such as your Interoperability support and network characteristics.
  4. Estimate if you can benefit from using pTCP or UDT based on the recommendations in the following table. These are typical performance findings, however actual performance is dependent on additional factors, such as the packet loss rate.

To determine the BDP using this table, multiply the latency by the bandwidth. This table is based on a 64 KB RWIN.

  Latency ms Bandwidth
1 Mb/s 10 Mb/s 100 Mb/s 1 Gb/s
0.1 No benefit No benefit No benefit No benefit
1 No benefit No benefit No benefit Possible benefit
10 No benefit No benefit Possible benefit Benefit
60 No benefit Possible benefit Benefit Benefit
100 No benefit Possible benefit Benefit Benefit
200 No benefit Possible benefit Benefit Benefit
400 No benefit Benefit Benefit Benefit
  1. Decide on the protocol, pTCP or UDT, depending on the platform support.
  • If you select pTCP, consider:
    • Number of connections should be greater than the BDP divided by TCP Windows.
    • The packet size must be less than the TCP Windows size.
  • If you select UDT, consider:
    • Platform support.
    • You must configure the firewall for UDT (while the pTCP protocol is firewall friendly).

Interoperability support

Protocol Transfer CFT SecureTransport
pTCP

Supported on Unix/Windows

All ST platforms
UDT Supported on LINUX/Windows only No support

How to implement pTCP or UDT

About pTCP

The pTCP protocol provides increased performance by opening a customizable number of connections to allow sending data faster, instead of sending the data through a single TCP connection.

Pros

Cons

  • Not supported by all applications.
  • Congestion may actually decrease performance.
  • In some cases, increases congestion and packet loss rate by opening a high number of connections.
  • Difficult to tune due to variations in networks and Transfer CFT (for example modifying packet and buffer sizes).
  • May be unfair to other flows.
  • Not beneficial if your usage is typically small-sized files.

How to use pTCP

When using pTCP, remember that both partners must be configured to use pTCP.

  • If you are using Transfer CFT with Central Governance, add pTCP to the application configuration, modify the pTCP settings as needed, and then select pTCP as the protocol when defining the flow.
  • If you are defining pTCP in Transfer CFT standalone, see UCONF: Acceleration.

Fine tuning

When implementing pTCP, you can modify the following using the Central Governance interface:

  • Number of connections (default is 10): The number of connections should be greater than the BDP divided by TCP Windows, and the higher the value the more you increase the TCP Windows. The default value of 10 may be inadequate depending on your configuration and use.
  • Packets size: In the context of Transfer CFT, the packet size is the amount of data sent to one connection before moving on to the next connection. Setting the value too low creates too many connections; too large of a value may be inefficient as time may be lost waiting for a TCP Windows acknowledgment on a connection. We recommend that you use a value less than the TCP Windows size.

 

About UDT

UDT is a connectionless transmission type protocol that relies on UDP, which can transfer data at a much higher speed than TCP. It features a configurable framework that can accommodate various congestion control algorithms. While UDT requires firewall configuration, multiple UDT flows can share a single UDP port meaning that a firewall can open a single UDP port for all UDT connections.

For more information, refer to udt.sourceforge.net.

Pros

  • Fast: UDT is designed for extremely high speed networks.
  • Fair: Concurrent UDT flows can share the available bandwidth leaving enough bandwidth for TCP.

Cons

  • Platform restrictions: When using in Transfer CFT the protocol is limited to LINUX/Windows only.
  •  

How to use UDT

When using UDT, remember that both partners must be configured to use UDT.

  • If you are using Transfer CFT with Central Governance, add UDT to your configuration, modify UDT settings as needed, and select UDT as the protocol when defining the flow.
  • If you are defining UDT in Transfer CFT standalone, see UCONF: Acceleration.

Troubleshooting

This section lists a few error types and corrective actions when using pTCP or UDT.

Partner is not enabled for transfer acceleration

The following types of errors may display when one of the partners is not defined transfer acceleration.

Local definition

In this example the protocol is defined only locally, but not on the remote partner, which creates a local network connection error:

CFTA01I Acceleration peer NET0_PESIT_CtoP, session 4 : connection from component

CFTA03E PTcpResponseRecv(peer:NET0_PESIT_CtoP): error 0 in recv: errno: 0 strerror: Error 0

CFTA03E readTCPtoPTCP(peer:NET0_PESIT_CtoP, session:4, cnx:0): error in SetupOneConnectionSync

CFTA03E readTCPtoPTCP(peer:NET0_PESIT_CtoP, session:4): SOCKS request rejected or failed

CFTA03E readTCPtoPTCP(peer:NET0_PESIT_CtoP, session:4): thread end on error

CFTH10E Network connect reject <PART=SUN38 RECOV=9 L=0 R=9 D=0 NET=TCP>

Remote definition

In this example the protocol is not defined locally (only on remote), which creates an immediate local network connection error:

CFTH11E Error Opening session <PART=RS58 EV=VNRELI ST=CN0022>

CFTT75E connect reject <IDTU=A000000G PART=RS58 IDF=T2 IDT=A1318100 302 R 0 2f2>

Incompatible versions

This section describes the use of a Transfer CFT v3.1.3 SP3 or lower pTCP implementation when communicating with Transfer CFT v3.1.3 SP4 and higher. If you have the following types of messages or issues, check the partner’s Transfer CFT version and verify the pTCP implementation.

Older Transfer CFT version connecting with a more recent pTCP implementation

Requester

The catalog displays a 302 diagnostic code, and the log messages similar to the following are displayed:

CFTH11E Error Opening session <PART=SUN38 EV=VNRELI ST=CN0022>

CFTT75E connect reject <IDTU=A00000GX PART=SUN38 IDF=PTCP IDT=A1418194 302 R 0 2f2>

Server

Nothing displays in the catalog, but a log message is displayed indicating an invalid pTCP packet:

CFTA03E PTcpRequestRecv(peer:NET0_PESIT_PtoC): invalid received PTCP packet

Recent pTCP implementation connecting with an older version of Transfer CFT

Requester

The catalog displays a 302 diagnostic code, and the log messages similar to the following are displayed:

CFTA01I Acceleration peer NET0_PESIT_CtoP, session 0 : connection from component

CFTA03E PTcpResponseRecv(peer:NET0_PESIT_CtoP): error -1 in recv: errno: 131 strerror: Connection reset by peer

CFTA03E readTCPtoPTCP(peer:NET0_PESIT_CtoP, session:0, cnx:0): error in SetupOneConnectionSync

CFTA03E readTCPtoPTCP(peer:NET0_PESIT_CtoP, session:0): SOCKS request rejected or failed

CFTA03E readTCPtoPTCP(peer:NET0_PESIT_CtoP, session:0): thread end on error

CFTH10E Network connect reject <PART=WIN RECOV=9 L=0 R=9 D=0 NET=TCP>

Server

There is no indication in either the log or catalog that the partner has a different Transfer CFT version pTCP implementation.

Related Links