NAME

SYNOPSIS

bbftp -v

bbftp [ Options ] -u RemoteUsername -i ControlFile RemoteHost

bbftp [ Options ] -u RemoteUsername -e ControlCommands RemoteHost

DESCRIPTION

Use the bbftp command to transfer files between the local host and a remote host.

In order to get better performance on a loaded Wide Area Network than transferring with the standard ftp command, use the bbftp command. bbftp has been designed to take advantage of the RFC 1323 and uses multiple streams is order to speed up transfer. It also implements an automatic retry in case of failure of commands contained in the ControlFile or in the ControlCommands.

bbftp allow two methods of connection to the remote host, a direct connection on the control port of the remote bbftpd daemon or the ability to remotely start a bbftpd daemon through an ssh tunnel. For the first method, the bbftpd daemon has to be started (by inetd or as a standalone daemon) on the remote host and all user information ( username, password) are transmitted encrypted to the daemon; for the second the bbftpd binary has to be accessible somewhere on the remote host and all control data are transmitted through the ssh tunnel.

The behaviour of bbftp is controled by the ControlFile which contains the commands to be executed (option -i) or by commands separated by semicolons (option -e). The format of these commands are given in CONTROL COMMANDS.

bbftp may be used in one of the following ways:

bbftp

to print a short help

bbftp -v

to write the version of the software and default values to standard output.

bbftp [ Options ] -i ControlFile -u RemoteUsername RemoteHost

bbftp [ Options ] -e ControlCommands -u RemoteUsername RemoteHost

to request the execution of commands contained in the control file ControlFile or the ControlCommands using RemoteUsername on RemoteHost.

Depending on the Options a password for the RemoteUsername on RemoteHost may be requested interactively (see OPTIONS section, for a full description of the Options ).

OPTIONS

CONNECTION OPTIONS

-s

Use this option to use ssh to remotely start a bbftpd daemon. It usually starts the binary "bbftpd -s", but this can be changed througth the -E option.

-S

Same as -s but ask ssh to run without asking a question (no password, no host key checking). It implies the usage of the identity file (for those familiar to ssh, this in done in setting the ssh options BatchMode to yes and the option StrictHostKeyChecking to no).

-E Server command to run

This option has to be used if the binary to be started on the remote host is not the default one (usually "bbftpd -s"). This option also implies the usage of the ssh mode (no need to add the -s option)

-I SSH identity file

This option has to be used if the ssh identity file is not the default one (usually $HOME/.ssh/identity). This option also implies the usage of the ssh mode (no need to add the -s option)

-L SSH command

If your ssh command is not the default one (usually "ssh -q"), use this option to change it. This option also implies the usage of the ssh mode (no need to add the -s option)

-P Private authentication string

If your client have been compiled with a private authentication schema, this option allow to pass an arbitrary string to the authentication module. You can determine if your client is using a private authentication module with the -v option.

-w Control port number

Use this option to change the control port for connection to on the RemoteHost. This option is meaningless in ssh mode.

BEHAVIOUR OPTIONS

-b

Use this option to run bbftp in the background after all interactive requests.

-c

Use this option to gzip the data during transmission. Compression and uncompression are done "on the fly" and do not require any additionnal disk space. Do not use it if the files to transmit are not compressible, as this will only lead to a waste of CPU and time. This option can be overridden by the control command setoption gzip

-f ErrorFile

Use this option to redirect the data generated on the standard error to ErrorFile

-m

Use to special output on file transfer. The ouput is in the following format:

Direction NumberOfBytes NumberOfSeconds BuffersizePerStream SendWinSize RecvWinSize NumberOfStreams Compression

BuffersizePerStream, SendWinSize, RecvWinSize are expressed in KiloBytes

In exemple :

get 10000 10 256 256 256 2 gzip

means that 10000 Bytes have been transfered in 10 seconds in compressed mode using 2 streams, a buffer size per stream of 256 KBytes, a TCP send window size of 256 KBytes and a TCP receive window size of 256 KBytes.

This option is usefull when trying to choose the best parameters between two sites.

When this option is set -V , -W and -t options are not available.

-o OutputFile

Use this option to redirect the data generated on the standard output to OutputFile

-p NumberOfParallelStreams

Use this option to increase the number of streams to use during the file transfer. Default is 1. This option can be overridden by the control command setnbstream

-r NumberOfTries

Use this option to change the number of tries to use when a transfer fails. The default is usually 5.

-R .bbftprc file

After a successful connection to the daemon the client is going to execute all control commands located in the $HOME/.bbftprc file. The location of this file can be changed with this option. Take care, not all control commands are allowed in the .bbftprc file (See CONTROL COMMANDS to know the authorized one)

-t

Use this option to have a timestamp on all output (overridden by -m option).

-V

Use this option to set the client in verbose mode (overridden by -m option).

-W

Use this option to print warnings to stderr (overridden by -m option).

CONNECTION EXAMPLES

bbftp -i ctrlfile -u jon -p 5 -c cchost.in2p3.fr

means that bbftp is going to connect to remote host cchost.in2p3.fr using username jon. If the connection is successful then the commands in ctrlfile will be executed. All transfer commands will use five streams and gzip compression.

bbftp -i ctrlfile -u phg -s cchost.in2p3.fr

means that bbftp is going to start a remote bbftp via sshd on host cchost.in2p3.fr using username phg. ssh will first try an RSAAuthentication if it is allowed by cchost.in2p3.fr; otherwise ssh will ask for a password for user phg on cchost.in2p3.fr. Then the sshd on cchost.in2p3.fr will log user phg and try to start the command "bbftpd -s"

bbftp -i ctrlfile -u jon -E '/tmp/bbftpd -s' cchost.in2p3.fr

Same behaviour as preceding, except that the remote command will be "/tmp/bbftpd -s"

bbftp -i ctrlfile -u gilles -S cchost.in2p3.fr

means that bbftp is going to start using ssh a remote bbftpd on host cchost.in2p3.fr using username gilles. ssh will try an RSAAuthentication if it is allowed by cchost.in2p3.fr, otherwise the connection will be broken.

bbftp -e 'setrecvwinsize 1024 ; put file1 file2' -u jon cchost.in2p3.fr

means that bbftp is is going to connect to remote host cchost.in2p3.fr using jon username. If the connection is successful then the commands and put file1 file2 will be executed. All tranfer commands will use one stream.

CONTROL COMMANDS

The control commands are either contained by an ASCII file ( -i option) or written on the bbftp line ( -e option). They can be divided into two classes, the "File related commands" and the "Behaviour commands".

All "Behaviour commands" may be put in a .bbftprc file, but all "File related commands" are forbiden in that file.

IMPORTANT NOTE

Under the RFIO mode (see setoption remoterfio and setoption localrfio ) all file related commands have to be given in absolute mode.

FILE RELATED COMMANDS

cd RemoteDirectory

Change the current directory of the daemon on the remote host.

If the RemoteDirectory is given in relative mode (not beginning by a /), it is supposed to be relative to the directory where the daemon is currently running. After the first connection, the current directory is the home directory of the RemoteUsername.

The client keeps in mind the current remote directory so in case of broken connection during a transfer, it can reset the current directory of the daemon to the correct directory.

If the daemon has been set in RFIO mode (see setoption remoterfio ) this option is unavailable.

get RemoteFile LocalFile

Transfer the remote file RemoteFile to the local host with the name LocalFile. If the local file already exists it is overwritten (only in case of successful transfer if the setoption tmpfile has been set). Under some circumstances (No space left on device, Access denied, File is a directory ...), no retry is done and the next command is processed.

If the RemoteFile is given in relative mode (not beginning by a /), it is supposed to be relative to the current directory on the remote host (which is set to the home directory of the RemoteUsername at the beginning).

If the LocalFile is given in relative mode (not beginning by a /) the file is created relative to the directory where the bbftp command is running (which may have been changed with the lcd command).

get RemoteFile

Transfer the remote file RemoteFile to the local host with the name RemoteFile.

If the RemoteFile is given in relative mode (not beginning by a /), it is supposed to be relative to the current directory on the remote host (which is set to the home directory of the RemoteUsername at the beginning) and created on the local host relative to the directory where the bbftp command is running (which may have been changed with the lcd command).

lcd LocalDirectory

Change the current directory on the local host.

If the LocalDirectory is given in relative mode (not beginning by a /), it is supposed to be relative to the directory where the client is currently.

If the client has been set into RFIO mode (see setoption localrfio ) this option is unavailable.

mget RemoteFiles LocalDirectory

Expand the RemoteFiles on the remote machine and do a "get" for each file name thus produced. Files are transferred into the LocalDirectory.

mget RemoteFiles

Expand the RemoteFiles on the remote machine and do a "get" for each file name thus produced. Files are transferred into the local working directory, which can be changed with the lcd command.

mkdir RemoteDirectory

Create the directory RemoteDirectory on the remote host. If the directory already exist no retry is done and the next command of the file is processed. If the RemoteDirectory is given in relative mode (not beginning by a /) the directory is created relative to the current remote directory.

If one directory in the given path does not exist the command will fail if the setoption nocreatedir is set. If the setoption createdir has been set all unexisting directories will be created.

mput LocalFiles RemoteDirectory

Expand wild cards in the list of local files given as arguments and do a "put" for each file in the resulting list. Files are transfered into the RemoteDirectory.

mput LocalFiles

Expand wild cards in the list of local files given as arguments and do a "put" for each file in the resulting list. Files are transfered into the current remote directory which can be changed with the cd command.

put LocalFile RemoteFile

Transfer the local file LocalFile to the remote host with the name RemoteFile. If the remote file already exists it is overwritten. Under some circumstances (No space left on device, Access denied ...), no retry is done and the next command of the file is processed.

If the LocalFile is given in relative mode (not beginning by a /) the file is supposed to be relative to the directory where the bbftp command is running (which may have been changed with the lcd command).

If the RemoteFile is given in relative mode (not beginning by a /), it is created relative to the current directory on the remote host (which is set to the home directory of the RemoteUsername at the beginning).

put LocalFile

Transfer the local file LocalFile to the remote host with the name LocalFile.

If the LocalFile is given in relative mode (not beginning by a /) the file is supposed to be relative to the directory where the bbftp command is running (which may have been changed with the lcd command) and created relative to the current directory on the remote host (which is set to the home directory of the RemoteUsername at the beginning).

BEHAVIOUR COMMANDS

setoption Option

To negate an option just add no before the option (ie setoption nocreatedir). The options are the following :

createdir

All file-related commands will create missing directories if needed (default createdir).

gzip

All file transfers will be compressed using the gzip algorythm (default nogzip).

keepaccess

The access time and modify time will be kept on each file transferred (default keepaccess).

keepmode

The file mode will be kept on each file transferred (default keepmode).

localrfio

All local files will be created with RFIO functions (default nolocalrfio).

remoterfio

All remote files will be created with RFIO functions (default noremoterfio).

tmpfile

All files will be created under a temporary name (FileName.bbftp.tmp) and renamed to the correct file name if transfer is successful (default tmpfile)

setbuffersize Buffersize

Set the size in Kbytes of the buffer used for reading or writing the files. This command set the local and remote buffer size. (Each stream will use the same buffer size)

setlocalcos LocalCos

Set the local COS to the value specified by LocalCos. This COS will be used for further rfio funtions. It is used if the setoption localrfio has been set.

setlocalumask LocalUmask

Set the local umask to the value specified by LocalUmask. This umask will be used for further i/o funtions. The LocalUmask has to be given in OCTAL

setnbstream NumberOfParallelStreams

Set the number of parallel streams to NumberOfParallelStreams. This number will be used for further transfer commands.

setremotecos RemoteCos

Set the remote COS to the value specified by RemoteCos. This COS will be used for further rfio funtions. It is used if the setoption remoterfio has been set.

setremoteumask RemoteUmask

Set the remote umask to the value specified by RemoteUmask. This remote umask will be used for further i/o funtions. The RemoteUmask has to be given in OCTAL

setrecvwinsize WindowSize

Set size in Kbytes of the receive TCP window of each stream of the bbftpd daemon. This also set the send window size of the client to the same value.

setsendwinsize WindowSize

Set size in Kbytes of the send TCP window of each stream of the bbftpd daemon. This also set the receive window size of the client to the same value.

NOTES

If the option tmpfile is used then if the new file ( RemoteFile for a put or LocalFile for a get) did not exist before, bbftp ensures that the file transfer was correct if the file exists.

In case of an already existing file, if the size, the last access and modification time are correct (if option keepaccess has been set) bbftp ensures that the file transfer was correct.

WARNING

Do not believe that the transfer is ended because the new file ( RemoteFile for a put or LocalFile for a get) has the expected size, IT IS CREATED WITH ITS EXPECTED SIZE AT THE BEGINNING OF THE TRANSFER.

EXIT STATUS

0

if all commands were successfuly executed

>0

if one command failed.

It may happend that a non-zero value is returned even if all files were correctly transfered, if during one transfer a retry was needed. This will be corrected in future releases.

MESSAGES AND ERRORS

All informative messages are written to the standard ouput (or to the OutputFile ). All error messages are written to the standard error (or to the ErrorFile ).

WARNING

RESULT FILE

If the -i option was used a result file will be created in the same directory as the ControlFile. Its name is ControlFile with the extension ".res". It contains the same lines as the ControlFile plus the keyword OK, in case of success, or FAILED, in case of failure.

If the -e option was used and the -V option was not used, the software will print the command executed plus the keyword OK, in case of success, or FAILED, in case of failure to standard output.

EXAMPLE

User jon want to transfer files from host localhost to remotehost on the account bbrdist. The bbrdist account has /home/babar/bbrdist as default directory on remotehost but has no subdirectories. We are going to study a control file in order to understand bbftp behaviour (we do not care here about the connection method; see CONNECTION EXAMPLES for that).

User jon on the local host is on the /home/babar/jon directory and has the following control file (all lines have a number which must not exists but which are there just for clarity) :

1

setnbstream 20

2

setremoteumask 022

3

setoption nocreatedir

4

put /home/babar/jon/f1 /home/babar/bbrdist/newfiles/f1

5

setoption createdir

6

put /home/babar/jon/f1 /home/babar/bbrdist/newfiles/f1

7

setnbstream 5

8

setrecvwinsize 1024

9

setoption gzip

10

put /home/babar/jon/f2 /home/babar/bbrdist/newfiles/f2

Command 1 just sets the number of parallel streams to 20 for further get or put commands.

Command 2 sets the remote umask for further put commands.

Command 3 indicates that no directory has to be created on further put or get commands.

Command 4 tries to send the local file /home/babar/jon/f1 to /home/babar/bbrdist/newfiles/f1. This command will fail because bbrdist has no subdirectory and directory creation is inhibed.

Command 5 resets the createdir option.

Command 6 will be successful (if the connection does not break), because the creation of the directory /home/babar/bbrdist/newfiles has been authorized by the createdir option.

Command 7 reduces the number of streams to 5

Command 8 sets the receive TCP window size to 1024 Kbytes on remotehost and the send TCP window size to 1024 Kbytes on localhost.

Command 9 sets the gzip option for further get or put commands.

Command 10 will transfer file /home/babar/jon/f2 to /home/babar/bbrdist/newfiles/f2 with 5 streams in compressed mode.

SEE ALSO

AUTHOR

bbftp was developed by Gilles Farrache (farrache@cc.in2p3.fr) from IN2P3 Computing Center , Villeurbanne (FRANCE). All the ssh-related stuff is based on ideas and software written by Tim Adye (T.J.Adye@RL.AC.UK) from Rutherford Appleton Laboratory , UK.

This software uses the cryptolib 0.9.5 of the OpenSSL project and zlib 1.1.3 written by Jean-Loup Gally and Mark Adler.
 

Index

NAME

SYNOPSIS

DESCRIPTION

OPTIONS

CONNECTION EXAMPLES

CONTROL COMMANDS

EXIT STATUS

MESSAGES AND ERRORS

WARNING

RESULT FILE

EXAMPLE

SEE ALSO

AUTHOR