Remote log streaming with Filebeat

Description

This configuration is designed to stream log files from multiple sources and gathering them in a single centralized environment, which can be achieved by configuring Filebeat to collect logs from files and forward them to an instance of Decision Insight for processing. Filebeat uses the lumberjack protocol to send messages. You can listen for this messages and process them using the Camel Lumberjack component and process them in Data Integration.

Architecture


System prerequisites

Install Axway Decision Insight (DI) following the prerequisites and steps detailed under Installation.

Filebeat should be installed on a Linux Operating System. Solaris and Windows operating systems were not tested with (DI) and are not covered by this guide.

Filebeat version

This documentation pertains to Filebeat release 1.2.3 and Filebeat release 5.2.0 on Linux.

No other release versions of Filebeat have been validated with Decision Insight.

Configuration prerequisites

Please check the following configuration settings have been properly applied to the environment where Filebeat will be installed:

Accounts

Account
Required abilities
<Filebeat account>
  • Permissions
Directory
Permissions
<install directory> read, write, execute

<registry_file directory>

read, write

<monitored log files directories>

read
 

<install directory> is the directory where filebeat will be installed

<registry_file directory> is the directory where the index registry for parsed logs will be stored (by default, <registry_file directory> is the directory from which filebeat is launched)

<monitored log files directories> are the storage directories for the log files to be monitored.


Related Links