Centralized Log Management via the Elastic Stack Deployment on CentOS 7

The implementation of the Elastic Stack, historically and widely recognized as the ELK Stack, represents a critical architectural decision for modern IT environments requiring robust observability and centralized logging. By integrating Elasticsearch, Logstash, and Kibana, organizations can transform raw, unstructured log data from disparate sources into actionable intelligence. On a CentOS 7 platform, this stack provides a stable, enterprise-grade foundation for searching, analyzing, and visualizing logs generated from any source in any format. This practice of centralized logging is indispensable when diagnosing complex application failures or server instabilities, as it eliminates the need for administrators to manually SSH into individual machines to grep through local files. Instead, it enables the correlation of logs across multiple servers within a specific time frame, allowing for the identification of systemic issues that span across an entire distributed architecture.

The Elastic Stack is composed of four primary components that work in a linear pipeline: Beats for data collection, Logstash for processing, Elasticsearch for storage and indexing, and Kibana for visualization. In a standard CentOS 7 deployment, these components must be synchronized in versioning; using the same version across the entire stack is a mandatory requirement to ensure API compatibility and prevent data corruption or communication failures between the nodes. While the stack can be installed manually via package managers, the scale of modern infrastructure often necessitates the use of automation frameworks such as Ansible. This ensures that the environment is reproducible, consistent, and devoid of human error during the configuration of complex components like Kibana enrollment tokens or Elasticsearch cluster settings.

Architectural Components of the Elastic Stack

The Elastic Stack is not a single application but a suite of integrated tools. Each component serves a specific role in the telemetry pipeline, moving data from the edge of the network to the administrator's dashboard.

• Elasticsearch: This is the heart of the stack. It is a distributed, RESTful search and discovery engine based on Apache Lucene. It provides a highly scalable way to store and index large volumes of data in near real-time. In a CentOS 7 environment, it acts as the primary database where all processed logs are indexed for rapid retrieval.
• Logstash: This is the server-side data processing pipeline. Logstash ingests data from multiple sources, transforms it (parsing, filtering, and enriching), and then sends it to a "sink," typically Elasticsearch. It allows for the normalization of data, ensuring that logs from different operating systems or applications follow a consistent schema.
• Kibana: This is the visualization layer. It provides a graphical user interface (GUI) that allows users to query the Elasticsearch indices and create dashboards, histograms, and maps. Because Kibana is typically bound to the localhost by default, production deployments often require a reverse proxy, such as Nginx, to make the interface accessible over a web browser to remote administrators.
• Beats: These are lightweight data shippers. Unlike Logstash, which is resource-intensive, Beats are designed to reside on the edge servers (clients) to forward logs and metrics to Logstash or Elasticsearch. Filebeat is a common example used for forwarding and centralizing logs and files.

Manual Installation Procedures on CentOS 7

For administrators performing a manual setup, the process involves configuring specific repositories and ensuring the Java runtime environment is present, as the Elastic Stack is Java-based.

Elasticsearch Installation and Configuration

The installation begins with the preparation of the system and the import of the official GPG keys to ensure package integrity.

bash yum install java-openjdk -y rpm --import http://packages.elastic.co/GPG-KEY-elasticsearch

The repository must be defined in the yum configuration to point to the correct version of the software. For an older 2.x deployment, the following configuration is used:

bash cat <<EOF > /etc/yum.repos.d/elasticsearch.repo [elasticsearch-2.x] name=Elasticsearch repository for 2.x packages baseurl=http://packages.elastic.co/elasticsearch/2.x/centos gpgcheck=1 gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch enabled=1 EOF

Once the repository is established, the software is installed and the service is enabled to ensure it starts on boot:

bash yum install elasticsearch -y systemctl enable elasticsearch systemctl start elasticsearch

Kibana Installation and Configuration

Kibana follows a similar installation pattern, requiring its own repository definition to ensure the correct version (e.g., 4.4.x) is pulled from the Elastic servers.

bash cat <<EOF > /etc/yum.repos.d/kibana.repo [kibana-4.4] name=Kibana repository for 4.4.x packages baseurl=http://packages.elastic.co/kibana/4.4/centos gpgcheck=1 gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch enabled=1 EOF

Following the repository setup, the installation and service activation are performed:

bash yum install kibana -y systemctl enable kibana systemctl start kibana

Logstash Installation and Configuration

Logstash completes the core triad of the ELK stack. Its repository is distinct and must be configured as follows for version 2.2:

bash cat <<EOF > /etc/yum.repos.d/logstash.repo [logstash-2.2] name=logstash repository for 2.2 packages baseurl=http://packages.elasticsearch.org/logstash/2.2/centos gpgcheck=1 gpgkey=http://packages.elasticsearch.org/GPG-KEY-elasticsearch enabled=1 EOF

The installation and service activation are then executed:

bash yum install logstash -y systemctl enable logstash systemctl start logstash

Automation via Ansible for Enterprise Scaling

In professional environments, manual installation is discarded in favor of Ansible playbooks. This approach allows for the orchestration of multiple nodes, ensuring that the Elasticsearch cluster and Kibana servers are configured identically.

Inventory and Node Management

The first step in an automated deployment is the creation of an Ansible inventory file. This file contains the IP addresses or hostnames of the CentOS 7 servers that will comprise the Elasticsearch cluster and the Kibana nodes. This allows the administrator to target specific groups (e.g., master-elk-server, kibana-server) with tailored tasks.

Kibana Enrollment and Token Management

A critical part of the modern Elastic Stack setup is the enrollment process, which secures the connection between Kibana and Elasticsearch. The automation process involves generating a token on the master server and distributing it to the Kibana nodes.

The token is generated via the following shell command on the master server:

bash /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana

Ansible handles the distribution of this token using a combination of register and copy modules. If the deployment involves multiple Kibana nodes, the token must be shared across servers:

yaml - name: Keep Kibana tokens on different servers. copy: content: "{{ hostvars[groups['kibana-server'][0]]['kibana_token_node']['stdout'] }}" dest: /etc/kibana/kibana_token_node.txt loop: "{{ groups['kibana-server'] }}" when: - inventory_hostname in groups["kibana-server"] - inventory_hostname != item delegate_to: "{{ item }}"

Once the token is present on the node, the enrollment is finalized:

bash /usr/share/kibana/bin/kibana-setup --enrollment-token {{ kibana_token.stdout }}

Configuration and RPM Deployment

Ansible can also manage the installation of specific RPMs to ensure version parity across the environment. This is done by downloading the specific version from the Elastic artifacts repository:

```yaml
- name: Download Kibana RPM
geturl:
url: https://artifacts.elastic.co/downloads/kibana/kibana-{{ elkversion }}-x8664.rpm
dest: /tmp/kibana.rpm
when: inventoryhostname in groups["kibana-server"]

• name: Install kibana RPM
  yum:
  name: /tmp/kibana.rpm
  state: present
  when: inventory_hostname in groups["kibana-server"]
  ```

Furthermore, the kibana.yml configuration file is managed via the blockinfile and lineinfile modules to ensure that the elasticsearch.hosts setting is correctly configured or commented out as needed:

yaml - name: Delete line elasticsearch.hosts lineinfile: path: /etc/kibana/kibana.yml regexp: '^elasticsearch.hosts:' line: '#' when: inventory_hostname in groups["kibana-server"]

Advanced Data Ingestion and Beats Integration

To maximize the utility of the ELK stack, data must be ingested using Beats. This allows the system to push logs from various CentOS 7 client machines to the Logstash service.

Filebeat and Index Templates

Filebeat is used to forward logs, but for the data to be searchable and organized, index templates must be loaded into Elasticsearch. These templates define how the logs are handled and stored.

The process of loading these templates involves using curl to send a PUT request to the Elasticsearch API:

bash curl -XPUT 'http://localhost:9200/_template/filebeat?pretty' -d@/etc/filebeat/filebeat.template.json curl -XPUT 'http://localhost:9200/_template/topbeat?pretty' -d@/etc/topbeat/topbeat.template.json curl -XPUT 'http://localhost:9200/_template/packetbeat?pretty' -d@/etc/packetbeat/packetbeat.template.json

For environments that incorporate Windows machines, the winlogbeat.template.json must be uploaded to the server via SCP and then loaded similarly:

bash curl -XPUT 'http://localhost:9200/_template/winlogbeat?pretty' -d@/tmp/winlogbeat.template.json

Dashboard Deployment and Customization

To provide a visual interface for the incoming data, dashboard zip files must be installed and loaded.

bash yum install unzip -y unzip beats-dashboards-*.zip cd beats-dashboards-* ./load.sh

For organizations that require specific corporate branding, the Kibana interface can be rebranded by replacing the default logo and favicon files:

bash cp /tmp/kibana.svg /opt/kibana/optimize/bundles/src/ui/public/images/kibana.svg cp /tmp/favicon.ico /opt/kibana/optimize/bundles/src/ui/public/images/elk.ico

Technical Specifications and Comparison

The following table outlines the primary components and their roles within the CentOS 7 deployment.

Security and Infrastructure Hardening

A production-ready ELK stack on CentOS 7 must implement security layers to prevent unauthorized access to sensitive log data.

TLS and SSL Implementation

To secure the communication between components, TLS (Transport Layer Security) support must be added. This involves the use of openssl to generate certificates. For the Kibana server, a self-signed certificate can be generated using the internal Elasticsearch utility:

bash /usr/share/elasticsearch/bin/elasticsearch-certutil cert -name kibana-server --self-signed

This ensures that data in transit between the Kibana GUI and the Elasticsearch backend is encrypted, preventing man-in-the-middle attacks.

Resource Management and Performance

The Elastic Stack is resource-intensive, particularly Elasticsearch, which relies heavily on the JVM heap size. Administrators must tune the jvm.options file to allocate sufficient memory based on the physical RAM of the CentOS 7 server. Furthermore, the use of a reverse proxy like Nginx is not only for accessibility but for security, as it allows the administrator to implement SSL termination and basic authentication before requests reach the Kibana service.

Analysis of Deployment Methodologies

The choice between manual installation and Ansible-driven automation depends entirely on the scale of the infrastructure. Manual installation, as detailed in the yum and systemctl commands, is suitable for "proof of concept" scenarios or single-server environments. However, the lack of consistency in manual setups often leads to "configuration drift," where one server's settings deviate from another, causing unpredictable failures during cluster synchronization.

Ansible solves this by treating the infrastructure as code. The use of get_url for RPM downloads and blockinfile for configuration management ensures that every node in the cluster is a perfect mirror of the desired state. The complexity of the Kibana enrollment token process—requiring a token from the master node to be passed to the client nodes—demonstrates why automation is mandatory for the Elastic Stack. Manually copying tokens across dozens of servers is not only inefficient but prone to error, which would result in the Kibana nodes failing to join the cluster.

The integration of Beats further extends the reach of the stack. By deploying Filebeat on all CentOS 7 client machines, the system transitions from a simple log viewer to a comprehensive monitoring hub. The necessity of loading index templates via curl emphasizes that the ELK stack is not "plug-and-play"; it requires a precise sequence of API calls to prepare the database for the specific structure of the incoming logs.

Conclusion

The deployment of the Elastic Stack on CentOS 7 provides a powerful mechanism for centralized logging, transforming fragmented system logs into a unified, searchable data stream. The synergy between Elasticsearch's indexing capabilities, Logstash's processing power, and Kibana's visualization tools allows for an unprecedented level of visibility into infrastructure health. While the manual installation path is straightforward for small-scale tests, the shift toward Ansible automation is essential for maintaining consistency, security, and scalability in enterprise environments. By leveraging Beats for edge collection and implementing TLS for secure transport, administrators can build a resilient observability pipeline that significantly reduces the mean time to resolution (MTTR) for system incidents. The ability to correlate logs across multiple nodes in a single timeframe transforms the troubleshooting process from a manual search into a data-driven analysis.

Sources

1. Cyber Ranges
2. LinkedIn - Sirisoft PCL
3. DigitalOcean
4. Timothy Quinn