Overview

A log data monitor is a customized monitor that extracts information from log files. Extremely useful when you have to extract specific data from logs like response time, HTTP request, response size etc. The log data monitor fetches data based on field numbers and/or a pattern that is followed in your logs.

Pre Requisites

• To successfully fetch logs from your environment, make sure that the log files that you want to monitor are present.
• he application, whose logs are to be monitored, should be up & running.

Configuration

Provide the details in the following fields for the configuration of the log data monitor:

• • Metric Group Name - The name of the metric group under which all the log data being captured will be shown. The log data will be shown as metrics, belonging to this metric group.
  • Metric Group Description - To provide a suitable metric group description corresponding to the metric group name.
  • File Name Selection(Use File name) - Getting log using File Name path
  • File Name Selection(Use Command to get the file name) - Getting logs through the command.
  • File Name Selection(Use JournalD) - JournalD is the system service for collecting and storing log data introduced with systemd. Getting log using type “JournalD” and based on the following option:
    • All Logs
    • Specified Value
  • Number of header lines - To skip the header lines. By default is 0.
  • Trailing Characters - To exclude trailing characters
  • Log line Pattern - The pattern is to be used to read the logs afterwards.
• Separators
  • Double Quotes, Single Quotes, Tab, Space, Comma, Backslash, Semicolon, Dash, Equal, Others - Select the separator which separates the different log fields in your logs. It is a mandatory field.
• Metrics
  • Field Type - Select the field type from the drop-down. You have the following option:
    • Field Number - Log field number of the log data that you want to capture.
    • Pattern - The pattern will be used to find matching data in the log(s).
    • Field Number and Pattern - Specify the field number and pattern to retrieve the log data.
  • Field Number - Enter the field number of the log data that is to be fetched.
  • Pattern - Specify the pattern which will be used to find the matching values in the logs. The data matching this pattern will be captured and shown under the metric name.
  • Metric Name - Provide a suitable metric name under which we will be able to see the metrics being captured.
  • Data Type - It is a drop-down list containing “Sample”, “Rate”, “Cumulative”, “times”, and “timesStd”. The log data being captured will be shown as this data type in the graphs.
  • Formula Type - Formula to be applied on the data value for the graph
    • Divide By: Divide the data value by Formula Value.
    • Multiply By: Multiply the data value by Formula Value.
    • Bytes to kbps: Converting bytes to kbps
  • Formula Value - It is the value provided to the formula type for calculation.
• Advance Settings
  • Dump Server Logs inside the Test Run - Select this box if you want to dump server logs inside the Test run.
  • Read Multiple Log Files -To read multiple log files, check this box.

Deployments of Log Data Monitors

• When the configurations have been added, you can view the deployed log data monitors in the Deployment section. You can edit the configurations of the configured monitor in case you want to change the File name selection/Instance Name. Advanced settings contains the options to edit the number of instances, option to dump server logs inside the test run and specifying the frequency of the data being captured. To delete the selected configured monitor, click on the delete icon.
• You can deploy the monitors in the deployments screen.

Deploy Monitors

Deploy the configured monitors using our machine agent in the deployments screen. The following fields need to be filled in to deploy:

Tier: Tier is used to select the tier on which the monitor(s) will be deployed. Select the required tier from the drop-down list.

Server: You can select the server(s) of the selected tier on which the configured monitor(s) will get deployed. Based on the requirement, you can select any one of the following:

• All -- Deploy the configured monitors on all the servers of the selected tier.
• Any – any one of the servers where monitors will be deployed.
• Specified Server - Select one or multiple servers of the chosen tier on which the monitor(s) will be deployed.

It means that with monitors whether we require all servers, any server, or a specified server.

Validation

• To validate the successful implementation of custom log data monitor is validated after completing the above fields, it automatically comes on the monitoring homepage UI with the number of monitors are successfully running or failed with details.
• Also, validate the Custom log data metrics under the given instance name in the metrics tree.