Translate

Search

IBM QRadar 5) Collecting File Logs

We will see how to collect file logs in this section.

Basically, we have to use FTP to collect file logs.

It has interval time, and logs are collected with interval time.

For security reason, many company don't allow FTP but we can use SFTP(Secure File Transfer Protocol).

SFTP is using SSH(Secure Shell) protocl to get file logs.

I am seeing most of servers allow FTP or SFTP, if there are some server that don't allow any connection you have to use tail2syslog that be used for forward file logs as syslog type.

I will show you about tail2syslog later.

FTP and SFTP's configuration are same without protocol selection.

Officially, let me show you how to configure to collect file logs.

1. Run Log Sources Menu


Move 'Admin' tab and click 'Log Sources' icon.

2. Add Log Source


Click 'Add' to add log list

3. Defualt view of Log Sources menu


 4. Select file protocol (FTP or SFTP)


Select file protocol to collect file logs.
  ●  Choose 'Universal DSM' in 'Log Source Type'
  ●  Choose 'Log File' in 'Protocol Configuration'
  ●  Choose 'FTP' or 'SFTP' in 'Service Type'

5. Fill information about detail information of log


In this section, we will type ip address, file location, file name, interval time and othters.

You don't need to fill all blanks that you saw as above picture.

'Log Source Name' and 'Log Source Description' don't give any problem to log. so you can fill this field as you want to do.

And other information like server ip address and port, authentication must be corrent to collect logs.

Just you want simple, you can fill these blank as number 5 picture.

Please see below information if you want to know every options.

 ●  'Log Source Name' : log's name
 ●  'Log Source Description' : log's description
 ●  'Log Source Type' : log's type
 ●  'Protocol Configuration' : form of logs
 ●  'Service Type' : log's protocol
 ●  'Remote IP or Hostname' : log server's ip address or hostname
 ●  'Remote Port' : FTP / SSH port
 ●  'Remote User' : account that have read permission
 ●  'Remote Password' : account's password
 ●  'Confirm Password' : repeat password
 ●  'SSH Key File' : SSH Key File informatin(it is not necessary)
 ●  'Remote Directory' : log file location
 ●  'Recursive' : collecting schedule
 ●  'FTP File Pattern' : log file name(here, regex can be used.)
 ●  'Start Time' : first collecting time
 ●  'Recurrence' : collecting interval time
 ●  'Run on Save' : auto save logs.
 ●  'EPS Throttle' : maximum of EPS
 ●  'Processor' : if log files are compressed, check it
 ●  'Ignore Previously Processed File(s)' : don't collect file that be processed
 ●  'Change Local Directory' : change log location in QRadar
 ●  'Event Generator' : form of logs
 ●  'File Encoding' : setting encoding of logs
 ●  'Folder Separator' : it is used if log directories are diffrent
 ●  'Enabled' : enable collecting log after configure
 ●  'Credibility' : important asset
 ●  'Target Event Collector' : event processor that collect logs
 ●  'Coalescing Payload' : many logs that have same information will be shown one log
 ●  'Store Event Payload' : save original log payload
 ●  'Log Source Language' : set language of log

7 comments:

  1. Hello,

    Thanks for the tutorial :)
    I did what you wrote there, but I have a little error :

    ERROR - Authentication Status: Auth Failed: ssh connection failed to root@/192.168.X.X:22 with exception: com.jcraft.jsch.JSchException: Auth fail
    ERROR - File Transfer Status: Could not transfer file(s)
    ERROR - Event Collection Status: Problem gathering/parsing events

    Do you happen to know what is this from ?

    ReplyDelete
    Replies
    1. Hi, I guess your account information is fail.

      You have to compare your root password for using SCP protocol.

      If it is not solution, maybe your ssh configuration in your server denied accessing your server.

      This configuration is placed on /etc/ssh/sshd_config file.

      Set 'PermitRootLogin' parameter to 'yes'.

      Sorry about too late reply.

      Delete
  2. Hello good job very useful for understanding the Qradar base !

    Are plan to write more blog about Qradar ? like IBM Qradar #6-7-8

    ReplyDelete
  3. For this issue I created private & public key of external server. Then copied the public key to qradar appliance and fill the form (SSH Key file). Then I made a first connection using putty in order to see that I can access external server from Qradar console. Then Qradar could connect without password to the external server and parse the logs

    ReplyDelete
  4. hello the authentication is successful for me and file transfer too but i have this error message :
    INFO - Authentication Status: Successful
    INFO - File Transfer Status: File(s) transferred successfully
    ERROR - Event Collection Status: One or more files could not be processed, some desired events will not be collected .

    can you give me a solution about it .

    ReplyDelete
  5. Getting Error

    Authentication Status: Auth Failed: ssh connection failed to User@/IP:22 with exception: com.jcraft.jsch.JSchException: SSH_MSG_DISCONNECT: 2 Too many authentication failures File Transfer Status: Could not transfer file(s)Event Collection Status: Problem gathering/parsing events

    ReplyDelete