Implement a custom protocol connector

The SecureTransport Pluggable Transfer Site SPI provides the com.axway.st.plugins.site.Connection interface that should be implemented in order to transfer files to and from the partner. The Connection interface version 1.0 declares the following methods:

  • void getFile(DestinationFile file) throws IOException;
  • void putFile(SourceFile file) throws IOException;
  • void finalizeExecution() throws IOException;
  • List<FileItem> list() throws IOException;

The Connection interface is extended by com.axway.st.plugins.site.CustomSite abstract class which is also part of Pluggable Transfer Site SPI. Its main purpose is to hold a UIBean instance containing connection parameters as key-value pairs saved into the database as custom transfer site properties. The parameters are loaded from the database by SecureTransport and populated in the UIBean instance inside the specific CustomSite implementation when starting server-initiated transfers.

The following steps should be followed to create the custom protocol connector:

  1. Define a class that extends the com.axway.st.plugins.site.CustomSite:
  2. import com.axway.st.plugins.site.CustomSite
    public class FTPSite extends CustomSite {
    }
    
  3. Define the UIBean instance that will hold the connector parameters:
  4. /** The UIBean implementation. */
    private FtpBean mFtpBean = new FtpBean();
  5. Define the default constructor for FTPSite class which should set the instantiated FtpBean as the UIBean instance in the CustomSite parent class.
  6. When SecureTransport starts the transfer, it will populate the connection properties into UIBean instance defined in the CustomSite class:
  7. public FTPSite() {
          setUIBean(mFtpBean);
    }

Site SPI versioning

Each Site SPI is versioned. Тhe initial SPI release is version 1.0. Every version extends the previous one and adds additional content. The SPI methods and classes added in a specific version have the @since annotation in their Javadoc documentation.

To declare what SPI version is used in Site implementation, place the Plugin-Info.yaml file inside the Meta-Inf directory. This file contains the plug-in information, similar to the MANIFEST.MF. The MANIFEST.MF file (if any) is with lower priority when there is a Plugin-Info.yaml file.

If the version is not specified in the MANIFEST.MF, the default one (1.0) will be used.

Only the methods and classes available in the specified version can be used.

The current SPI version, introduced along with SecureTransport5.3.6 patch 41, is 1.0.

Connection interface implementation - version 1.0

SecureTransport calls putFile (SourceFile file) method from the Connection interface when executing server-initiated push. The SourceFile instance will contain the InputStream to read the local file content that will be sent, the remote file name to write the content to, and the size of the local file (number of bytes the InputStream could read). Generally, the implementation of putFile() method should connect to the remote partner and write the content from the local file input stream to the remote file. The parameters for the connection can be retrieved from the already defined UIBean instance.

When calling getInputStream(RemotePartner descriptor) from the SourceFile instance, you must pass a com.axway.st.plugins.site.RemotePartner instance as argument. This instance should hold the connection parameters which will be reported by SecureTransport File Tracking. Currently, these parameters are the remote host, the remote folder to which the file will be pushed, the parameter that shows whether the remote connection is secure and the new remote file name which should be reported as Post Transmission Action if such is configured in transfer site.

Let's assume you have an FTPConnector interface and a simple FTP client implementation based on your interface which has the following methods:

  • connect(FtpBean ftpBean)
  • upload(InputStream inputStream, String remoteDir, String remoteFile)

The connect() method retrieves connection parameters from the FtpBean instance and uses them to configure the FTP client and then connects to the remote partner.

The upload() method sends the content from the inputStream to the remote partner and saves it to remoteFile in the remoteDir.

Note The remoteDir is actually the remote upload folder to which the file will be sent. The Send To Partner step in the Advanced Routing cannot overwrite the upload folder of a Custom Transfer Site.

The Send to Partner step has option to override the upload folder configured in the transfer site only if the transfer site explicitly allows upload folder overriding which is configurable in all built-in transfer sites. For all custom transfer sites, the overriding of the upload folder by the Send to Partner step is not allowed and is not configurable. If the Send to Partner step is configured to override the upload folder of custom transfer site, the overriding is ignored.

So, in the putFile method implementation inside the FTPSite class, you should connect to the remote partner and then upload the file.

Note Do not disconnect from the remote partner.

First, you should use the connect() method to connect to the FTP server:

/** The FTP client implementation. Allows connecting/disconnecting/transferring files. */
private AbstractFTPConnector mFtpConnection;

/** Connects to a FTP server.
 *
 * @throws IOException on error
 */
public String connect() throws IOException {
    mFtpConnection = new FTPConnectorBuilder().build(mFtpBean);
    mFtpConnection.setCertificateService(mCertificateService);
    mFtpConnection.setProxyService(mProxyService);
    return mFtpConnection.connect();
}

Then use it in the putFile() method to upload the file:

@Override
public void putFile(SourceFile file) throws IOException {

    String resolvedHost = "127.0.0.1";
    if (mFtpConnection == null) {
        resolvedHost = connect();
    }

    RemotePartner descriptor = new RemotePartner(resolvedHost, 
    mFtpBean.getPartnerUploadFolder(),
            mFtpBean.isFtps(), getSentFileAs());

    mFtpConnection.upload(file.getInputStream(descriptor), 
    descriptor.getRemoteHost(), file.getName());
}
/**
 * Gets the "Send File As" value or
 * return empty string if "Send File As" option is not enabled.
 *
 * @return "Send File As" value or, if no value is found it returns 
   empty string
 */
private String getSentFileAs() {
    String resultFileAs = "";
    if (mFtpBean.isSendFileAsEnabled()) {
        resultFileAs = mFtpBean.getSendFileAs();
        }
    return resultFileAs;
}

List() method

The list() method implementation from the Connection interface should return the list of files from the remote partner that should be downloaded. These files should be described using the com.axway.st.plugins.site.FileItem class. This class holds the remote file name that should be downloaded and the name which will be used to save the downloaded files.

When SecureTransport starts the server-initiated pull transfer, the list() method from the Connection interface is always called before the file pull. The parameters for configuring file pull are remoteDir and remotePattern where the pattern could be a filename or expression. After the list() method returns the list of files that should be downloaded, described using the FileItem class, SecureTransport calls the getFile() method from Connection interface for every item in the list. In the list() method, you can implement logic for listing files in a predefined remote directory only, or recursively listing all files in a specific directory. It is important to remember that the file names (and paths in case of recursive listing) returned by the list() method should be relative to the predefined remote directory.

In the following example, the FtpBean and FtpConnector will be used to implement the list() method:

Let's assume you have an FTPConnector interface and a simple FTP client implementation based on your interface which has the following methods:

  • connect(FtpBean ftpBean)
  • listFiles(String remoteDir, String remotePattern)

The connect() method will retrieve connection parameters from the FtpBean instance and use them to configure the FTP client and then connect to the remote partner.

The listFiles() method returns the list of file names matching the remotePattern inside the remoteDir folder.

In the list() method implementation you must first connect to the remote partner, then execute the listFiles() method to get the list of file names that match the specific pattern and then construct the result list of the FileItem instances.

First, you should use the connect() method to connect to the FTP server:

/** The FTP client implementation. Allows connecting/disconnecting/transferring files. */
private AbstractFTPConnector mFtpConnection;

/** Connects to a FTP server.
 *
 * @throws IOException on error
 */
public String connect() throws IOException {
    mFtpConnection = new FTPConnectorBuilder().build(mFtpBean);
    mFtpConnection.setCertificateService(mCertificateService);
    mFtpConnection.setProxyService(mProxyService);
    return mFtpConnection.connect();
}

Then use it to list the files from the remote partner that match the specified criteria (remote folder and file name pattern) defined in the FtpBean instance.

Note Do not disconnect from the external server.
@Override
public List<FileItem> list() throws IOException {
    if (mFtpConnection == null) {
        connect();
    }

    List<String> names = mFtpConnection.listFiles(mFtpBean.getPartnerDownloadFolder(),
            mFtpBean.getPartnerDownloadPattern());
    List<FileItem> result = new ArrayList<FileItem>();
    if (names != null && names.size() > 0) {
        for (String name : names) {
            result.add(new FileItem(name));
        }
    }
    return result;
}

Note that in the example above, the file will be saved locally with the same name as the name of the remote file when pulled.

If you want to save the file with different name, you can use the other constructor from the FileItem class when creating the FileItem instance the files:

  • new FileItem(String fileName, String receiveFileAs);

getFile(DestinationFile file) method

The next method that should be implement is getFile(DestinationFile file).

SecureTransport calls the getFile(DestinationFile file) method from the Connection interface when executing a server-initiated pull. The DestinationFile instance will contain the OutputStream to write the remote file content to and the remote file name which will be pulled. Generally, the getFile() method should write the content of the remote file to the output stream held by the DestinationFile instance. When calling the getOutputStream(RemotePartner descriptor); from DestinationFile instance, you must pass as argument the com.axway.st.plugins.site.RemotePartner instance. This instance should hold the connection parameters which will be reported by SecureTransport File Tracking. Currently, these parameters are the remote host, remote folder from which the file will be downloaded and the parameter that shows whether the remote connection is secure.

Let's assume you have an FTPConnector interface and a simple FTP client implementation based on your interface which has the following methods:

  • connect(FtpBean ftpBean)
  • download(OutputStream outputStream, String remoteDir, String remoteFile)

The connect() method retrieves the connection parameters from the FtpBean instance and uses them to configure the FTP client and then connects to the remote partner.

The download() method retrieves the content from the remoteFile which is in the remoteDir and writes its content to the outputStream instance.

So, in the getFile method implementation inside the FTPSite class, you should first connect to the remote partner and then download the file.

First, you should use the connect() method to connect to the FTP server:

/** The FTP client implementation. Allows connecting/disconnecting/transferring files. */
private AbstractFTPConnector mFtpConnection;

/** Connects to a FTP server.
 *
 * @throws IOException on error
 */
public String connect() throws IOException {
    mFtpConnection = new FTPConnectorBuilder().build(mFtpBean);
    mFtpConnection.setCertificateService(mCertificateService);
    mFtpConnection.setProxyService(mProxyService);
    return mFtpConnection.connect();
}

Then use it in the getFile() method to download the file.

Note Do not disconnect from the external server.
@Override
public void getFile(DestinationFile file) throws IOException {
    String resolvedHost = "127.0.0.1";
    if (mFtpConnection == null) {
        resolvedHost = connect();
    }
    mFtpConnection
        .download(
            file.getOutputStream(
                    new RemotePartner(resolvedHost, 
           mFtpBean.getPartnerDownloadFolder(), mFtpBean.isFtps())),
            mFtpBean.getPartnerDownloadFolder(), file.getName());
}

public void finalizeExecution() throws IOException; method

The last method from Connection interface which you should implement is:

  • public void finalizeExecution() throws IOException;

This method is called by SecureTransport when finalizing the server-initiated transfer after executing the list method, getFile method, and putFile method regardless if they are completed successfully or not.

In this method you should release any occupied resource, for instance disconnecting the specific client from the remote partner.

Note If the external server requires an explicit logout, it needs to be implemented either here or in the disconnect implementation.
@Override
public void finalizeExecution() throws IOException {
    if (mFtpConnection != null) {
        mFtpConnection.disconnect();
        mFtpConnection = null;
    }
}

Exception handling

SecureTransport will only retry the transfer by default if some of the Connection interface methods throw an IOException instance if there are retry attempts remaining. The SecureTransport Pluggable Transfer Site SPI provides a special exception named TransferFailedException, which is of type java.io.IOException. This exception indicates a transfer failure; if the error is permanent (for example, an authentication failure) and is not expect to be resolved on retry, an isPermanentFailure flag should be raised. Otherwise, it should be left set to false, (for example, on connection timeout or any other transient error), allowing the transfer to succeed on the next retry. Generally, any exception caused by configuration or miss configuration is an exception that should not be retried. If an error occurs while implementing your Custom Connector, a TransferFailedException instance should be thrown with the flag set to indicate if the exception is permanent or temporary.

There are two available constructors for instantiating com.axway.st.plugins.site.TransferFailedException:

  • public TransferFailedException(String message, Throwable cause);
  • public TransferFailedException(String message, Throwable cause, boolean isPermanentFailure);

The first one will set the isPermanetFailure flag to false and will force the SecureTransport to retry the transfer if retry attempts are available. The number of retries and the interval between retries for a transient failure of transfer can be configured by a SecureTransport administrator. For more information about configuring retry parameters for server-initiated transfers, refer to the SecureTransport Administrator's Guide.

Note Only adminstrator configurable retries should be attempted. Internal retries should not be attempted. Internal retries are hidden from SecureTransport and no file tracking information will be generated.

SecureTransport exposed services

The SecureTransport Pluggable Transfer Site SPI also exposes services from SecureTransport Server that can be consumed by the Custom Connector implementation. The implementation can @Inject the service interface in its extension of CustomSite class and then consume the service via provided interface.

To use the certificate service, declare a variable with @Inject annotation (javax.inject.Inject) to the interface of the certificate service provided in the API:

    @Inject
    private CertificateService certificateService;

The certificate service provides capabilities for:

  • KeyPair getKeyPair(String certId) throws SecurityException;
  • SSLContext createSSLContext(String certId, boolean verifyPeer) throws SecurityException;
  • SSLContext createSSLContext(String certId, boolean verifyPeer, String sslProtocol, String keyAlgorithm, String trustAlgorithm) throws SecurityException;
  • SSLSocketFactory configSSLContexFactory(SSLContext sslContext, String[] protocols, String[] cipherSuites) throws SecurityException;
  • SSLSocketFactory configSSLContexFactory(String certId, boolean verifyPeer, String sslProtocol, String keyAlgorithm, String trustAlgorithm, String[] protocols, String[] cipherSuites) throws SecurityException;

To use the proxy service, declare variable with @Inject annotation to the interface of the proxy service provided in the API:

    @Inject
    private ProxyService proxyService;

The proxy service provides capabilities for:

  • ProxyInfo getProxy(ProxyType proxyProtocol, String dmzName) throws NoSuchZoneException;
  • boolean isRemoteDnsResolutionEnabled(String zoneName) throws NoSuchZoneException;
  • String getPublicURLPrefix(String zoneName) throws NoSuchZoneException

The getProxy method of the proxy service will return a "null" object, if there is no proxy with the specified type defined in the selected DMZ zone. The specific custom connector implementation decides whether to use this proxy for the transfer or not.

The default SecureTransport zone is Private.

Related Links