FTP client scripting interface

The FTP client in the trading engineB2Bi includes a scripting interface to accommodate non-standard interaction with FTP servers. The default FTP client implementation uses an XML file, called the command set document. The XML file defines meta-commands comprised of one or more FTP commands to be sent to an FTP server. The default command set document works with most FTP servers without any modifications.

The following topics describe how to change the default FTP client implementation when the client must interact with an FTP server that requires non-standard commands or expects commands in a different order than the default in the command set document. Such changes can range from editing the command set document to creating Java classes that implement non-standard FTP commands.

Any change to the FTP client requires familiarity with the trading engineB2Bi and the FTP protocol as described in RFC959. Simple modifications require editing the XML command set document. Advanced modifications require familiarity with the Java programming language and with writing and compiling Java classes.

Concepts:

Reference:

Levels of scripting

The following describe the three levels of possible scripting changes, ordered from simplest to most complex.

Edit the command set document. At this level, you edit the default command set document to change the order of commands sent to the FTP server or remove commands. For example, if an FTP server immediately prompts for a password rather than first prompting for a username, you can remove the line that sends the User command from the authenticate meta-command.

Change Java classes containing commands. A Java class implements each command (for example, User). To change the behavior of a command or to add a command, you must edit or create the Java class that implements the command, then compile the class and make it available to the FTP client.

Write a custom FTP client implementation. The default client implementation is the framework of the FTP client. It includes classes that manage the connections with the server and that read and execute the commands in the script. You can replace the default client implementation with one that does not use a script. For example, if a user has an FTP client implementation written in Java, it could be modified to work with the trading engineB2Bi by replacing the default client implementation with an interface to the user’s existing implementation.

The following topics describe each level of scripting in greater detail:

Editing the command set document

The script that specifies commands sent to the host is the XML command set document, named ftpcommandset.xml and located in <install directory>/conf. It contains a set of meta-commands, each consisting of one or more FTP commands to be sent to an FTP server.

While interacting with the FTP client, the trading engineB2Bi executes meta-commands defined in the script, such as receive and send. For each meta-command, the script specifies the FTP commands to send to the FTP server and in what order. The trading engineB2Bi is not aware of the specific FTP commands being sent to the FTP server, which means changes to the command set document do not require changes to the trading engineB2Bi.

One example of the changes that can be made to the command set document is reordering the FTP commands comprising a meta-command. As a simple example, the receive meta-command sends the Type and Size commands, in that order. You could reverse the order by transposing those two lines in the script.

Users of the FTP client, such as the ftpTester program and the trading engineB2Bi itself, invoke the connect and authenticate meta-commands before issuing any other commands.

Changes made to the script take effect the next time the trading engineB2Bi needs to access the FTP server; the trading enginethe B2Bi server does not have to be restarted.

To use a command set document other than ftpcommandset.xml, you must add an entry for your document in <install directory>/conf/filereg.xml and restart the trading enginethe B2Bi server. You also must do this if you want to use a different command set document for each FTP delivery exchange.

Changing Java classes

If re-arranging commands in the command set document is insufficient for your needs, you might need to edit a command or create a command and add it to the command set document. Changes of this type require knowledge of the Java programming language and an understanding of the default implementation of the commands. For example, the delete meta-command is implemented by a single FTP command, Dele.

The following is the default implementation of this command:

// Copyright (c) 2006 Cyclone Commerce, Inc. All Rights Reserved.

// Please refer to the file "LICENSE" for further important

// copyright and licensing information. Please also refer to

// the documentation for additional copyright notices.

package com.cyclonecommerce.tradingengine.transport.ftp. commands;

import com.cyclonecommerce.tradingengine.transport. ftp.FtpClientSettings;

import com.cyclonecommerce.tradingengine.transport.ftp. ControlConnection;

import com.cyclonecommerce.tradingengine.transport.ftp. FtpReplyException;

import com.cyclonecommerce.tradingengine.transport.ftp.FtpReply;

import com.cyclonecommerce.util.StringUtil;

 

public class Dele

extends FtpCommand

{

public Dele(FtpClientSettings settings, CommandState commandState, ControlConnection controlConnection)

{

super(settings, commandState, controlConnection);

}

 

/**

Ensure that this object is initialized correctly for execution.

*/

protected void validateParameters()

throws FtpCommandException

{

super.validateParameters();

 

if (StringUtil.isNullEmptyOrBlank(getCommandState().getHostFilePath()))

{

throw new FtpCommandException("Name of file to delete cannot be empty or null");

}

}

 

protected String getCommand()

{

return "DELE " + getCommandState().getHostFilePath();

}

 

protected boolean shouldExecute()

{

return getCommandState().doDelete();

}

 

/**

Override of base checkReply that allows us to ignore errors in the case of a send operation, since we call delete before STOR without first checking to see if the file already exists. (Some servers don't even support STAT, so we really couldn't check first even if we wanted to.)

*/

protected void checkReply()

throws FtpReplyException

{

FtpReply reply = getReply();

 

if (getCommandState().isSend())

{

// note: we ignore all 5xx errors because some servers return

//other than 550 for file not found

if (!reply.isSuccessReply() && !reply.isErrorReply())

{

FtpReplyException fre = new FtpReplyException(buildErrorMessage(

"Expected success (2xx) or 5xx (file not found) reply from FTP server, but received:", reply));

fre.setResponseCode(reply.getReplyCode());

throw fre;

}

}

else

{

if (!reply.isSuccessReply())

{

FtpReplyException fre = new FtpReplyException(buildErrorMessage(

"Expected success (2xx) reply from FTP server, but received:", reply));

fre.setResponseCode(reply.getReplyCode());

throw fre;

}

}

}

}

 

Dele extends FtpCommand, and FtpCommand extends AbstractFtpCommand.

 

Refer to the following class hierarchy:

FTP Class Hierarchy Example

FtpCommand and FtpDataCommand are separate due to the FTP protocol requirement for a separate control connection and data connection.

Commands that can potentially return large volumes of data, such as Stor, Retr and Nlist, extend FtpDataCommand. Other commands extend FtpCommand, and they use only the control connection. Dele falls into this category. RFC959 defines which commands do and do not use the data connection.

 

A review of Dele.java reveals the Dele class implements the getCommand method, which returns the command string to be sent to the server. All command classes must implement getCommand because it is abstract in the base class.

 

In addition, Dele overrides the validateParameters method in AbstractFtpCommand. This allows Dele to ensure its argument is not empty. Many other commands that require an argument also override validateParameters for the same reason.

Writing a custom FTP client

If your requirements cannot be met using the methods already described, you can implement your own FTP client. The client you write must extend the FtpClient base class (which is abstract), allowing the trading engineB2Bi to interact with your client. The methods in your implementation must accept input parameters and return output values in the same form as the default implementation. For conformity, use as a reference the file CustomFtpClient.java, which is located at sdk/java-src/examples/transport.

After creating and compiling an FTP client implementation, instruct the trading engineB2Bi to use it by defining a property called com.cyclonecommerce.ftp.client on the JVM invocation line in the startServer script in the bin directory (GatewayInterchangeService.ini if running as a Windows service). This property must contain the name of the class that implements FtpClient. For example: com.cyclonecommerce.ftp.client=com.me.MyFtpClient.

Make sure the directory containing your class is in the classpath specified in the environment file in the bin directory. For example, you could put the class in <install directory>/classes/com/me/MyFtpClient.class.

All FTP delivery exchanges must use the same FTP client implementation because it is a system property in the JVM.

FTP tester tool

You can use the ftpTester tool to exercise the FTP client outside of the trading engineB2Bi. The script to start ftpTester can be found in <install directory>/tools.

ftpTester is a console-based application that can verify the operation of the FTP client in the trading engineB2Bi and a partner’s FTP server. The trading engineB2Bi server does not have to be running to use this tool. You can use it on UNIX or Windows.

ftpTester duplicates the way the trading engineB2Bi accesses an FTP server. It is a test program to verify interoperability with FTP servers. If you can send, list, receive and delete files on a FTP server using ftpTester, this is a good indication that the trading engineB2Bi can interoperate with the server.

ftpTester prompts for all of the information it needs, as the following example illustrates. Two views are shown, depending on whether you are testing receiving (consuming) or producing (sending).

Consumer options

**** Welcome to the Cyclone FTP test program ****

-> Enter host: localhost

-> Enter user: ftpuser

-> Enter password: cyclone

-> Enter account (leave blank for most servers):

-> Enter pickup directory (blank for "pickup"):

-> Enter c for CONSUMER client (list, receive, delete), p for PRODUCER (send). (Blank assumes c): c

-> Should temp files be used to avoid read/write collisions? (Y/N - default is Y):

-> Should a separate inbox dir be used (instead of putting temp files in the pickup dir)? (Y/N - default is Y):

-> Enter inbox dir where files will initially be uploaded before being moved to pickup dir (blank for "inbox"):

-> Use SSL? (Y/N - default is N):

-> Should restarts be attempted for binary files if supported by the host? (Y/N - default is Y):

-> Enter minimum bytes a file must contain to be eligible for restarts (blank for 100000):

FtpClientSettings - host=localhost pickupDir=pickup ctlPort=21 passive=true type=Binary mode=Stream struct=File user=ftpuser account= transportId=FtpTester receiveTempDir=/Applications/cyclone/b1095/bin/ftpRestart_FtpTester connectTimeoutMs=30000 readTimeoutMs=30000 attemptRestarts=true evaluateRestartable=true restartableMinBytes=100000 commandSet=../conf/ftpcommandset.xml preserveFilename=true overwriteIfDupe=true fixOutputFilenames=false tempFileExt=.tmp useDebounce=true useInbox=true isSSLEnabled=false

Host working directory: "/Users/ftpuser" is the current directory.

Host pickup directory (used by REC, DEL and LIST): pickup

Local working directory (used by REC and LLIST): /Applications/cyclone/b1095/bin (LCD to change)

 

-> Enter CONSUMER command (e.g. ?, LIST, RECeive, DELete, LLIST, ASCII, BIN, LCD, QUIT): ?

CONSUMER metacommands (abbreviations shown in upper case):

? help

LIST list files on host

RECeive filename retrieve file from host

DELete filename delete file from host

ASCII perform ASCII data transfers

BINary perform binary data transfers

LCD change local working directory

LLIST list files in local working directory

PASV perform data transfers in passive mode (the default)

PORT perform data transfers in port mode

QUIT/EXIT/BYE exitNormal FtpTester

Note: Metacommands are scriptable via conf/ftpcommandset.xml

Host working directory: "/Users/ftpuser" is the current directory.

Host pickup directory (used by REC, DEL and LIST): pickup

Local working directory (used by REC and LLIST): /Applications/cyclone/b1095/bin (LCD to change)

 

-> Enter CONSUMER command (e.g. ?, LIST, RECeive, DELete, LLIST, ASCII, BIN, LCD, QUIT):

Producer options

**** Welcome to the Cyclone FTP test program ****

-> Enter host: localhost

-> Enter user: ftpuser

-> Enter password: cyclone

-> Enter account (leave blank for most servers):

-> Enter pickup directory (blank for "pickup"):

-> Enter c for CONSUMER client (list, receive, delete), p for PRODUCER (send). (Blank assumes c): p

-> Should original filenames be preserved when sending? (Y/N - default is Y):

-> Should existing files be overwritten? (Y/N - default is Y):

-> Should temp files be used to avoid read/write collisions? (Y/N - default is Y):

-> Should a separate inbox dir be used (instead of putting temp files in the pickup dir)? (Y/N - default is Y):

-> Enter inbox dir where files will initially be uploaded before being moved to pickup dir (blank for "inbox"):

-> Use SSL? (Y/N - default is N):

-> Should restarts be attempted for binary files if supported by the host? (Y/N - default is Y):

-> Enter minimum bytes a file must contain to be eligible for restarts (blank for 100000):

FtpClientSettings - host=localhost pickupDir=pickup ctlPort=21 passive=true type=Binary mode=Stream struct=File user=ftpuser account= transportId=FtpTester receiveTempDir=/Applications/cyclone/b1095/bin/ftpRestart_FtpTester connectTimeoutMs=30000 readTimeoutMs=30000 attemptRestarts=true evaluateRestartable=true restartableMinBytes=100000 commandSet=../conf/ftpcommandset.xml preserveFilename=true overwriteIfDupe=true fixOutputFilenames=false tempFileExt=.tmp useDebounce=true useInbox=true isSSLEnabled=false

Host working directory: "/Users/ftpuser" is the current directory.

Host inbox directory (used by SEND for uploading, after which files are moved to pickup): inbox

Host pickup directory (used by SEND and LIST): pickup

Local working directory (used by SEND and LLIST): /Applications/cyclone/b1095/bin (LCD to change)

 

-> Enter PRODUCER command (e.g. ?, SEND, LLIST, ASCII, BIN, LCD, QUIT): ?

PRODUCER metacommands (abbreviations shown in upper case):

? help

SEND filename write file to host

ASCII perform ASCII data transfers

BINary perform binary data transfers

LCD change local working directory

LLIST list files in local working directory

PASV perform data transfers in passive mode (the default)

PORT perform data transfers in port mode

QUIT/EXIT/BYE exitNormal FtpTester

Note: Metacommands are scriptable via conf/ftpcommandset.xml

Host working directory: "/Users/ftpuser" is the current directory.

Host inbox directory (used by SEND for uploading, after which files are moved to pickup): inbox

Host pickup directory (used by SEND and LIST): pickup

Local working directory (used by SEND and LLIST): /Applications/cyclone/b1095/bin (LCD to change)

 

-> Enter PRODUCER command (e.g. ?, SEND, LLIST, ASCII, BIN, LCD, QUIT):

After prompting for the initial configuration information such as the host, user and password, FtpTester displays a main prompt that allows you to enter meta-commands to perform simple operations such as list, send and receive. You can enter a question mark (?) at this point to obtain more information.

Related Links