bbFTP
  
Home
Overview
License
Contributors
 
Releases
Download
ChangeLog
 
Documentation
3.2.1
3.2.0
3.1.0
3.0.2
3.0.1
3.0.0
2.2.2
2.2.1
2.2.0
2.1.0
2.0.2
 
Help
BBFTP-L List
    Register
    Archives
FAQ
Error messages
 
Email Us
bbftp@in2p3.fr
  

On-line documentation for release 3.2.0

 
Main changes
Installing the client
Installing the server
Ports used by BBFTP
Working with Transfer URLs (TURL)
Working with external software
Using bbFTP with a certificate
Building transfer statistics on server side
Implementing a private authentication module
Known bugs
Man bbftp
Man bbftpd
Warning

Main changes

  • New features
    • New client program "bbftpstatus"
    • Add message STATUS to server in order to send to client "bbftpstatus"
    • Port to Darwin (by Andrew Goodney)
    • Port to SX (By Gilles Gallot)
  • Fixes
    • Bug: empty result file (*.res) with -m option
    • Bug: multiple IP addresses now works with SSH (by Gilles Gallot)
    • Configure: add --with-globus-flavor option (gcc32dbg by default)

    Installing the client

    Configuring the client

    In the bbftpc directory, run:
    % ./configure

    The configure script accepts the following standard options:

    --prefix=DIR Defines the directory to install BBFTP client

    The configure script accepts the following additional options:

    --with-gzip Enable compression (automatic search + dynamic link if possible) (DEFAULT)
    --with-gzip=DIR Enable compression where libs are in DIR/lib and includes in DIR/include (static link)
    --without-gzip Disable compression
    --with-ssl Enable encryption (automatic search + dynamic link if possible) (DEFAULT)
    --with-ssl=DIR Enable encryption where libs are in DIR/lib and includes in DIR/include (static link)
    --without-ssl Disable OpenSSL encryption. Cannot be used in private authentication mode
    --with-rfio Enable RFIO interface (automatic search in "/usr/local" + static link) (DEFAULT)
    --with-rfio=DIR Enable RFIO interface where libs are in DIR/lib and includes in DIR/include (static link)
    --without-rfio Disable RFIO interface
    --with-afs Use AFS for standard authentication
    --with-afs=DIR Use AFS for standard authentication where libs are in DIR/lib and includes in DIR/include
    --without-afs Do not use AFS for standard authentication (DEFAULT)
    --enable-authentication=private Use the private authentication module. This mode requires OpenSSL
    --enable-authentication=certificates Use the certificate authentication module

    The following options should be used in certificate mode only:

    --with-gsi=DIR Defines where to find the Globus GSI software (DEFAULT=$GLOBUS_LOCATION or "/opt/globus", static link)

    Building the client

    In the bbftpc directory, run:
    % make

    Installing the client

    In the bbftpc directory, run:
    % make install

    Binary Windows installation

    unzip the file bbftp-VERSION-client-cygwin.zip. The file cygwin1.dll must be installed in the same directory.

    Installing the server

    Configuring the server

    In the bbftpd directory, run:
    % ./configure

    The configure script accepts the following standard options:

    --prefix=DIR Defines the directory to install BBFTP server

    The configure script accepts the following additional options:

    --with-gzip Enable compression (automatic search + dynamic link) (DEFAULT)
    --with-gzip=DIR Enable compression where libs are in DIR/lib and includes in DIR/include (static link)
    --without-gzip Disable compression
    --with-ssl Enable encryption (automatic search + dynamic link) (DEFAULT)
    --with-ssl=DIR Enable encryption where libs are in DIR/lib and includes in DIR/include (static link)
    --with-rfio Enable RFIO interface (automatic search in "/usr/local", static link) (DEFAULT)
    --with-rfio=DIR Enable RFIO interface where libs are in DIR/lib and includes in DIR/include (static link)
    --without-rfio Disable RFIO interface
    --with-afs Use AFS for standard authentication
    --with-afs=DIR Use AFS for standard authentication where libs are in DIR/lib and includes in DIR/include
    --without-afs Do not use AFS for standard authentication (DEFAULT)
    --with-pam Use PAM
    --without-pam Do not use PAM (DEFAULT)
    --with-ephemeral-ports-range=RANGE The server will choose data ports in range RANGE, defined as MINPORT:MAXPORT
    --without-ephemeral-ports-range The server will choose data ports in the system-dependent default range (DEFAULT)
    --enable-authentication=private Use the private authentication module
    --enable-authentication=certificates Use the certificate authentication module

    The following options should be used in certificate mode only:

    --with-gsi=DIR Defines where to find the Globus GSI software (DEFAULT=$GLOBUS_LOCATION or "/opt/globus", static link)
    --with-x509-user-cert=FILE Defines where to find the host certificate (DEFAULT=/etc/grid-security/hostcert.pem)
    --with-x509-user-key=FILE Defines where to find the host private key (DEFAULT=/etc/grid-security/hostkey.pem)
    --with-x509-cert-dir=DIR Defines where to find the CA certificates (DEFAULT=/etc/grid-security/certificates)
    --with-gridmap=FILE Defines where to find the gridmap file (DEFAULT=/etc/grid-security/grid-mapfile)

    Building the server

    In the bbftpd directory, run:
    % make

    Installing the server

    In the bbftpd directory, run:
    % make install

    Configuration

    It is possible to make the daemon start with options defined in a configuration file. This file is /etc/bbftpd.conf. The syntax is the same of the configuration file for client users (.bbftprc). Available options are described in the BBFTPD man page

    Post installation

    If you have chosen to run the server through inetd

    • Add the line
      bbftp    CONTROLPORT/tcp
      in your /etc/services file where CONTROLPORT is the number you have set in the includes/config.h (usually 5021)
    • Add the line
      bbftp stream tcp nowait root ${bindir}/bbftpd bbftpd
      in the /etc/inetd.conf

    If your system authentication is PAM

    • If you have a /etc/pam.conf file
      • Verify the lines begining by other auth and other account allow login. If it is not the case add special lines for bbftp. The lines to be added will look like:
        bbftp auth required /lib/security/pam_pwdb.so shadow nullok
        bbftp account required /lib/security/pam_pwdb.so

    • If you have a /etc/pam.d directory
      • Verify the file other and allow login. If it is not the case create a new file called bbftp which will contain two lines looking like:
        auth required /lib/security/pam_pwdb.so shadow nullok
        account required /lib/security/pam_pwdb.so

    The name of the PAM library may change.

    If you have chosen to run the server as a daemon

    • Modify the script ${prefix}/etc/bbftpd to add the options you need and to change the values of the GSI variables if you did not use the --with-x509 options during the configuration process.
    • To start the daemon run:
      ${prefix}/etc/bbftpd start
    • To stop the daemon, run:
      ${prefix}/etc/bbftpd stop
    • To restart the daemon (stop and start), run:
      ${prefix}/etc/bbftpd restart
    • To see the daemon's status, run:
      ${prefix}/etc/bbftpd status

    Ports used by BBFTP

    Control connection

    In the standard mode, the control connection is initiated by the client. The connection is required on port 5021 on the server side. This port can be changed at compile-time (#define CONTROLPORT in includes/config.h) or/and at run-time (option -w). If the server port is not the default one (5021), a client which want to connect to this server must specify the port (option -w).

    In the SSH mode, the server listens on port std+1 (ie 5022 by default). It cannot be changed at run-time.

    Data connection

    Whereas the FTP protocol implements one data connection only, BBFTP opens multiple data connections depending on the number of streams required (client option -p and server option -m).
    • Passive mode (client 3 against server 3 only)
      The server listens on ephemeral ports which are sent to the client. These ports can be chosen in a range defined at compile-time or at run-time.
    • Non-passive mode (other configurations)
      The server initiates the data connection(s) after the client has sent him its private IP address and the listening port(s). This(these) port(s) is(are) ephemeral ports (>1023). (it is possible to define a range of ports when starting the client using the -D option) The server can then establish the connection(s) to the specified port(s) from its port std-1 (ie 5020 by default).

    Working with Transfer URLs (TURL)

    The TURL notation can be used with commands relatives to files and directory. If you use this notation, you should not specify the server name in command.

    BBFTP support 3 types of TURL:


    Working with external software

    Zlib

    This software allows BBFTP to compress/uncompress data on-the-fly.
    It can be downloaded at
    http://www.gzip.org/zlib. The recommended version is 1.1.4 which fixes a potential security problem.
    This library is optional for both client and server.
    The default behavior of the BBFTP build process is to search Zlib automatically.

    OpenSSL

    This software allows BBFTP to encrypt user connection details (user name + password) when using the standard authentication method.
    It can be downloaded at http://www.openssl.org.
    This library is optional for BBFTP client. It is mandatory for BBFTP server except in certificates mode (The Globus Toolkit includes it).
    The default behavior of the BBFTP build process is to search OpenSSL automatically.

    Globus Toolkit 2

    This software allows BBFTP to use certificates as authentication mode. The recommended version is Globus Toolkit 2.4.3. It is divided into several components. The one needed by BBFTP is GSI (Grid Security Infrastructure) Bundle. It can be downloaded at http://www-fp.globus.org/gt2.4/download.html or directly http://www-unix.globus.org/ftppub/gt2/2.4/2.4-latest/bundles/src/globus-gsi-2.4.3-src_bundle.tar.gz.
    Security updates must be installed as well (download at: http://www-unix.globus.org/toolkit/advisories.html?version=2.4
    Theses libraries are mandatory when using certificates.
    If BBFTP is built in certificate mode, the default behavior of the build process is to search GSI automatically, else GSI is not included in the build process.

    AFS

    BBFTP is interfaced with AFS so that password entered in standard authentication mode is handled by AFS.
    By default, AFS is not included in the build process.

    RFIO

    This software allows BBFTP to access files directly from the hierarchical storage systems HPSS or Castor.
    RFIO64 is also supported.
    The default behavior of the BBFTP build process is to search RFIO automatically.

    Using bbFTP with a certificate

    External software required (on both client and server hosts)

    • GSI (Grid Security Infrastructure) from the Globus 2 Toolkit (2.0 or 2.2)

    Refer to the Working with external software section.

    System requirements

    • The certificate mode has been tested for Linux intel and Solaris only

    Certificates

    2 certificates needed:

    • A host certificate or a user certificate on the BBFTP server host.
    • A user certificate on the BBFTP client host.

    Build process

    For both client and server, use the --enable-authentication=certificates option for the configure script.
    See the installation sections for the client and for the server.

    Running the BBFTPD daemon

    Before starting the daemon, some environment variables can be set to override default values for certificates paths, mapfile location... If you use the --with-x509... options of the configure script, these variables will be automatically set.
    Refer to the Globus documentation for more details.

    Run the daemon "bbftpd -b"

    If you use a host certificate, you must be root to start the daemon. The CN field of the certificate must be the full name of the host (i.e. host.domain). If it is something else (for example a service name), it must be specified by the client with the -g option.
    If you want to start your own daemon, you must use another certificate (a host certificate you can read or your own certificate). In this case, only you can use the bbftp client against this server, and you will have to use the -g option (see BBFTP Client man page).

    Running the client

    Before running the client, you must create a temporary proxy using the grid-proxy-init command.

    Run the client "bbftp -e<command_list>|-i<control_file> <remote_host>"
    Please, note that the -u option is not used in the certificate authentication mode. If you use it, BBFTP will use the standard authentication mode.

    More information

    Additionnal information can be found on the GLOBUS web site

    Building transfer statistics on server side

    Collecting transfer data

    Transfer information are logged in the system log with the facility daemon and level notice if and only if the server is launched with a trace level greater than notice:
    % bbftpd -b -lNOTICE
    The output format is:
    action user filename bytes_transfered seconds kbytes/s Mbits/s
    action is GET or PUT.

    Building statistics

    Nothing is provided with BBFTP to build statistics with the data collected. You should use a third party tool run on a periodical basis.

    Implementing a private authentication module

    Starting with release 2.1.0 it is possible to implement a private authentication mechanism without. The private authentication mode disables all the other authentication modes (standard, ssh and certificates).

    On the client side

    Write your authentication code in the file bbftp/bbftp_private_user.c.

    • extern char *username;

      This variable contains the username given on the command line (-u option).

    • extern char *privatestr;

      This variable contains the string given on the command line with the -P option (or NULL if not used).

    • int bbftp_private_getargs(char *logmessage)

      This routine is called at the begining of the login sequence (just before setting the process in background if needed) in order to allow the programmer to set variables or to request input from the user.

      The return code and the variables are explained in the file.

      After having called this routine the main program will exchange RSA keys with the server in order to crypt all messages sent during the authentication procedure. Then the hand will be given to the next routine :

    • int bbftp_private_auth(char *logmessage)

      This routine will allow the user exchange data between the client and the server. For that it will use two routines bbftp_private_recv and bbftp_private_send whose descriptions are given in the bbftp_private_user.c file.

      When all this data exchange has ended, the routine will return to main code with a return code of 0 in case of success or with a return code of -1 and the string logmessage filled in case of error.

    On the server side

    Write your authentication code in the file bbftpd/bbftpd_private_user.c.

    • extern char currentusername[MAXLEN];

      It has to be filled by the bbftpd_private_auth routine.

    • int bbftpd_private_auth(char *logmessage)

      As on the client side, this routine will exchange data using bbftpd_private_send and bbftpd_private_recv routines (whose decriptions are given in the bbftpd_private_user.c file), do all checks needed and return 0 in case of success or -1 and the string logmessage filled in case of error.

    Building BBFTP with a private authentication module

    On the client side

    In the bbftpc directory, run
    ./configure --enable-authentication=private

    Then run:
    make

    Then run:
    make install

    On the server side

    In the bbftpd directory, run
    ./configure --enable-authentication=private

    Then run:
    make

    Then run:
    make install

    See the installation pages for the client and for the server for more information about the BBFTP build process.

    Known bugs

    AFS/RFIO64 incompatibility on Linux

    On Linux, linking with RFIO64 requires the posix threads library (libpthread). But this library makes the AFS 3.0 authentication routine in the BBFTP server (ka_UserAuthenticate) crash with a segmentation fault. From the client side, you get an error "error reading answer message" because the socket does not exist anymore.

    AFS/Globus incompatibility

    This is another incompatibility problem on the server: when built with AFS and Globus GSI, the AFS 3.0 authentication routine (ka_UserAuthenticate) always returns 0 with an error message "Incorrect password" even if the right password is given. This is due to a conflict between AFS code and GSI code related to cryptography. From the client side, you get an error "Incorrect password".

    OpenSSL bug on Linux 2.4

    On Linux 2.4, the server crashes if it is statically linked with OpenSSL prior to 0.9.6e (routine RAND_seed in bbftpd_daemon.c crashes). Upgrade your version of OpenSSL or link it dynamically.

    Warning

    Since bbftp include hooks to cryptography, the following information from OpenSSL applies to bbftp as well.

    PLEASE REMEMBER THAT EXPORT/IMPORT AND/OR USE OF STRONG CRYPTOGRAPHY SOFTWARE, PROVIDING CRYPTOGRAPHY HOOKS OR EVEN JUST COMMUNICATING TECHNICAL DETAILS ABOUT CRYPTOGRAPHY SOFTWARE IS ILLEGAL IN SOME PARTS OF THE WORLD. SO, WHEN YOU IMPORT THIS PACKAGE TO YOUR COUNTRY, RE-DISTRIBUTE IT FROM THERE OR EVEN JUST EMAIL TECHNICAL SUGGESTIONS OR EVEN SOURCE PATCHES TO THE AUTHOR OR OTHER PEOPLE YOU ARE STRONGLY ADVISED TO PAY CLOSE ATTENTION TO ANY EXPORT/IMPORT AND/OR USE LAWS WHICH APPLY TO YOU. THE AUTHORS ARE NOT LIABLE FOR ANY VIOLATIONS YOU MAKE HERE. SO BE CAREFUL, IT IS YOUR RESPONSIBILITY.

  • Last modified on Thu, 07 Feb 2013 16:27:24 +0000
    This page was generated in Python using ht2html