My Elastic Search Projects

 

Back to main page

 

Project 1: Elasticsearch TLS Encryption HTTPS Communication

https://isc.sans.edu/forums/diary/Secure+Communication+using+TLS+in+Elasticsearch/26902/

This document is a compilation of the various references listed in this document; it combines all the necessary steps I used to setup TLS encryption. Using Elasticsearch elasticsearch-certutil self-signed certificate authority, it provides secure communication for Linux and Windows between Elasticsearch nodes, Kibana, logstash and the various beats.

The complete installation document TLS_elasticsearch_configuration.pdf is located here.

 

Project 2: Building IDS Sensor with Suricata & Zeek with Logs to Elasticsearch

Follow the steps listed in this document to build the sensor. This Internet Storm Center (SANS ISC) provides a summary to quickly install the sensor.

ISC Diary: https://isc.sans.edu/diary/27296

Installation Document : https://handlers.sans.edu/gbruneau/elk/Building_Custom_IDS_Sensor.pdf

The primary goal of this document is to provide a framework to build your own sensor(s) using CentOS 7 with Suricata and Zeek. It also has information to capture netflow data using softflowd. The two tarball listed below, are used to preconfigure the sensor after CentOS7 has been installed with the two /nsm partitions. It includes configuration scripts, startup scripts and some partly configure .yml files for filebeat, metricbeat and packetbeat.

Note: Modify both these tarball and adapt them for your local network.

References:

[1] https://handlers.sans.edu/gbruneau/scripts/installation.tgz

[2] https://handlers.sans.edu/gbruneau/scripts/sensor.tgz

 

Project 3: Pihole Configuration Files

 
Configure /etc/filebeat/filebeat.yml as follow:
 
filebeat.inputs:
 
- type: log
  enabled: true
  paths:
    - "/var/log/pihole.log"
  fields_under_root: true
  fields:
    region: Ottawa
 
output.logstash:
  hosts: ["127.0.0.1:5044"]
 
Download the following logstash script to send logs to ELK. Adjust your Elastic IP accordingly.
 
The pihole.conf file was updated using the Elastic Common Schema (ECS) Reference and the dashboard was updated accordingly.
 
Logstash Pihole parser (Updated 20 Feb 2020)
Pihole Dashboard (21 Feb 2020)
 
 
 

Project 4: tcp-honeypot Configuration Files

 
Configure /etc/filebeat/filebeat.yml as follow:
 
filebeat.inputs:
 
- type: log
  enabled: true
  paths:
    - "/opt/logs/tcp-honeypot-*.log"
  fields_under_root: true
  fields:
    region: Ottawa
 
output.logstash:
  hosts: ["127.0.0.1:5044"]
 
Download the following logstash.yml script to send logs to ELK. Adjust your Elastic IP accordingly. His honeypot script is located here.
 
Logstash tcp-honeypot (20 Jun 2020)
tcp-honeypot Dashboard (20 Jun 2020)

 

 

Project 5: Windows DHCP Server Logs Configuration Files

 

A screen shot of a computer

Description automatically generated with low confidence

Note: See Post on https://isc.sans.edu/diary/27198 

Configure /etc/filebeat/filebeat.yml as follow:

 

# This filebeat shipper is used with

# for Microsoft DHCP logs

 

# 9 Jan 2021

# Version: 1.0

 

filebeat.inputs:

 

# Filebeat input for Microsoft DHCP logs

 

- type: log

  paths:

    - "C:/Windows/System32/dhcp/DhcpSrvLog-*.log"

  include_lines: ["^[0-9]{2},"]

  fields_under_root: true

 

#==================== Queued Event ====================

#queue.mem:

#  events: 4096

#  flush.min_events: 512

#  flush.timeout: 5s

 

#queue.disk:

#  path: "/op/filebeat/diskqueue"

#  max_size: 10GB

 

#==================== Output Event ====================

output.logstash:

  hosts: ["192.168.2.23:5044"]

Download the following logstash script to send logs to ELK. Adjust your Elastic IP accordingly.
 
The pihole.conf file was updated using the Elastic Common Schema (ECS) Reference and the dashboard was updated accordingly.
 
Logstash Microsoft DHCP Parser (12 March 2021)
Microsoft DHCP Dashboard (12 March 2021)
Windows DHCP Template (12 March 2021)
Windows DHCP ILM Policy (12 March 2021)