File systems (shared file systems for clusters)

B2Bi supports active-active clustering. To configure a cluster, you require a shared file system and a shared database between the nodes.

The following shared file systems are supported for B2Bi cluster implementations:

Linux / UNIX

Supported file system

NFS v4 server Linux ext4 (true ext4 not ext3 mounted as ext4)

NFS read lease bug workaround for Linux distributions running kernel version 2.x (2012/11/14):

NFS uses a read lease to guarantee that a client can perform local read opens without informing the server. Axway tests show that under some circumstances this read lease is not updated correctly and causes inconsistency on what the different nodes see on the shared file system. This can cause the nodes in a cluster to stop and restart unexpectedly and repeatedly.

Because this issue is not currently resolved in the 2.x Linux kernel, Axway recommends that you turn off the read lease function on the NFS server. This is done by setting a flag in the /proc/sys file system to tell the kernel to not allow any use of this feature.

The following procedure provides an example of how to set the flag on a RHEL machine acting as NFS server. Similar procedures can be adapted for other distributions.

  1. Important: Stop B2Bi before you perform this procedure.
  2. As root, execute the command:
  3. echo 0 > /proc/sys/fs/leases-enable
  4. Restart the nfs deamon:
  5. /etc/init.d/nfs restart
  6. After you complete the previous steps, unmount and re-mount from the NFS clients.

The change implemented by this procedure disappears when you reboot the server. To make this change persistent over machine restarts, add the following lines to a start-up script that is executed before the NFS daemon is started. A good place for this is in /etc/init.d/nfs in the "start" section, after the check for root UID but before the nfsd is started (insertion in bold below):

# Only root can start the service

[ $uid -ne 0 ] && exit 4

# Hotfix for the read-leases problem

echo 0 > /proc/sys/fs/leases-enable

Disable noac

When using NFS as the shared file system in B2Bi cluster environments, the selection of the "noac" option is no longer a prerequisite. Disabling "noac" enables a significant performance improvement.

NFS version

B2Bi 2.3.1 supports configurations with an NFS v4-mounted shared file system only. Existing customers must update their NFS share to be NFS v4 based (instead of NFS v3).

Linux RHEL 6

Supported file system

SAN: Symantec Storage Foundation Cluster File System (High Availability) 6.2

SAN 6.2 hotfix

In order to run B2Bi on SAN 6.2 you must obtain the VxFS 6.2.0.003 HotFix from Symantec Support and apply the HotFix to each Veritas cluster node.

Windows

Supported file systems

  • SMB2 (Windows 2008 R2)
  • SMB3 (Windows 2012 R1 and 2012 R2)

SMB 2.0 limitation

Running multiple B2Bi nodes using a shared file system on a network environment that uses SMB 2.0 can cause files on the shared file system to be corrupted. For information and hotfix see the Microsoft support link: http://support.microsoft.com/kb/2028965

SMB file locking errors

To avoid file locking errors on SMB2 (Windows 2008) / SMB3 (Windows 2012) platforms, set the following registry values:

  • On the client machine, set the SessTimeout parameter to the recommended value of 240 seconds.
  • Registry path: HKLM\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters
  • On the server machine, set the SessionKeepAlive registry parameter to the recommended value of 30000 milliseconds (30 seconds).
  • Registry path: HKLM\SYSTEM\CurrentControlSet\Services\NetBT\Parameters

SMB Samba limitation

B2Bi has been validated only with the SMB2 / SMB3 shares provided by Windows Server. It has not been tested with the Samba implementation of the SMB protocol.

SMB 2.0 and Windows Server 2008 cache lifetime issue

With the release of SMB 2.0 in Windows Server 2008, three file metadata caches were implemented to speed up the return of the most recently accessed file and directory information. These caches also reduce the number of interactions a client requires with a SMB server for common file-browsing operations. Axway tests show that this behavior causes inconsistency on what the different nodes see on the shared file system, affecting the stability of the system.

When SMB 2.0 is the negotiated protocol, the client SMB network redirector uses three different caches.

Cache name, type, ID Description Registry key that controls the cache Default setting

File Information Cache

[DWORD]

FileInfoCacheLifetime

File attribute information contained in the File_Network_Open_Information structure which is useful in conserving network IO for retrieving common file metadata information. To disable or turn off the caching behavior, the value of this registry key should be set to 0. Turning off the file information cache is not recommended as it could nearly double the number of network transactions required for executing a given scenario. HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Lanmanworkstation\Parameters 10 seconds

File Not Found Cache

[DWORD]

FileNotFoundCacheLifetime

Cache of files that the client failed to open because the file was not present on the server. This prevents the client from repeatedly attempting to open files which are known not to exist on the server. This cache is likely to affect distributed applications running on multiple computers accessing a set of files on a server - where the applications use an out of band mechanism to signal each other about addition/deletion of files on the server. HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Lanmanworkstation\Parameters 5 seconds

Directory Cache

[DWORD]

DirectoryCacheLifetime

Cache of recent directory enumerations performed by the client. Subsequent enumeration requests made by client applications as well as metadata queries for files in the directory can be satisfied from the cache. The client also uses the directory cache to determine the presence or absence of a file in the directory and uses that information to prevent clients from repeatedly attempting to open files which are known not to exist on the server. This cache is likely to affect distributed applications running on multiple computers accessing a set of files on a server - where the applications use an out of band mechanism to signal each other about modification/addition/deletion of files on the server. HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Lanmanworkstation\Parameters 10 seconds

Disable all three caches by adding (or changing) the registry keys:

  • Directory Cache - Set DirectoryCacheLifetime to ZERO.
  • File Not Found Cache - Set FileNotFoundCacheLifetime to ZERO.
  • File Information Cache - Set FileInfoCacheLifetime to ZERO.

For additional information about SMB redirector cache settings, see the website:
http://technet.microsoft.com/library/ff686200(WS.10).aspx

Related Links