The modern infrastructure landscape demands robust observability, where the ability to aggregate, analyze, and visualize log data across distributed systems is critical for operational integrity. The Elastic Stack, historically referred to as the ELK stack (Elasticsearch, Logstash, Kibana), serves as the foundational architecture for centralized logging. This ecosystem allows administrators to search, analyze, and visualize logs from any source in any format, enabling the correlation of issues spanning multiple servers within specific time frames. By centralizing these logs, organizations can identify problems with servers or applications through a single pane of glass, transforming raw telemetry into actionable insights for monitoring, analytics, and troubleshooting.

Core Component Analysis

The Elastic Stack comprises three primary open-source projects, each serving a distinct function within the data pipeline. Understanding the technical role of each component is essential for proper deployment and configuration.

Elasticsearch: The Search and Analytics Engine

Elasticsearch is a distributed, RESTful high-performance search and analytics engine built on Apache Lucene. It is the storage and processing core of the stack. Its architecture is ideal for log analytics due to its support for schema-free JSON documents and compatibility with a myriad of programming languages. Elasticsearch manages the indexing of data, allowing for rapid retrieval and complex queries across massive datasets.

Logstash: The Data Processing Pipeline

Logstash functions as a lightweight, server-side data processing pipeline. Its primary role is to collect data from multiple sources, transform the data structure or content, and ship the processed information to a desired destination, typically Elasticsearch. This component handles the "E" in ETL (Extract, Transform, Load), ensuring that raw logs are normalized and structured before indexing.

Kibana: The Visualization Interface

Kibana serves as the open-source data web UI and data visualization tool. It provides the interface for visualizing logs collected and processed by Logstash and indexed by Elasticsearch. The platform offers powerful, user-friendly features including histograms, line graphs, pie charts, heat maps, and built-in geospatial support. These visualization tools allow engineers to create custom dashboards that provide immediate visibility into system health and application performance.

Prerequisites and System Requirements

Deploying the Elastic Stack requires careful attention to system resources and software dependencies. The following specifications ensure stable operation of the components.

General Prerequisites:
- Ubuntu 20.04 or 22.04 (and 24.04 for newer deployments)
- Total system recommendation: At least 4GB RAM (8GB recommended)
- Root or sudo access
- Java 11 or 17

Java Installation

Since Elasticsearch is developed in Java, installing a compatible Java Runtime Environment (JRE) is a strict prerequisite. The standard approach involves installing OpenJDK 17, the latest stable LTS release.

```bash

Update package lists

sudo apt update

Install OpenJDK 17 JRE (headless version for server environments)

sudo apt install openjdk-17-jre-headless -y

Verify installation

java -version
```

Alternatively, the full JDK can be installed if development tools are required on the same host:
bash sudo apt install openjdk-17-jdk -y

Repository Configuration

The three ELK stack components are not available in Ubuntu’s default apt repositories. Administrators must manually add Elastic’s official package source list to ensure they are using version-aligned binaries. It is critical to use the same version across the entire stack to prevent compatibility issues.

```bash

Import the Elasticsearch GPG key

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg

Add the Elastic repository for version 8.x

echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list

Update the package cache

sudo apt update
```

Elasticsearch Installation and Configuration

After adding the repository, install the Elasticsearch package and configure the core settings.

```bash

Install Elasticsearch

sudo apt install elasticsearch -y

Edit the main configuration file

sudo nano /etc/elasticsearch/elasticsearch.yml
```

Within /etc/elasticsearch/elasticsearch.yml, the following parameters must be defined:

```yaml

Cluster name

cluster.name: elk-cluster

Node name

node.name: node-1

Data and logs paths

path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch

Network binding

network.host: localhost
http.port: 9200
```

For single-node deployments, discovery settings may need adjustment, but the provided configuration restricts network access to localhost for security, requiring a proxy for external access.

Logstash and Kibana Deployment

Following Elasticsearch, install Logstash and Kibana using the apt package manager. Since Kibana is normally only available on the localhost, external access is typically facilitated via a reverse proxy such as Nginx.

```bash

Install Logstash

sudo apt install logstash -y

Install Kibana

sudo apt install kibana -y
```

Filebeat Integration and Dashboard Setup

Filebeat is a lightweight shipper used for forwarding and centralizing logs and metrics. It acts as the initial collector in the pipeline, sending data to Logstash or directly to Elasticsearch.

Loading Index Templates and Pipelines

To ensure Filebeat data is properly structured in Elasticsearch, specific setup commands must be executed. These commands load ingest pipelines, index templates, and pre-built Kibana dashboards.

```bash

Load the ingest pipeline for the system module

sudo filebeat setup --pipelines --modules system

Load the index template into Elasticsearch

sudo filebeat setup --index-management -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["localhost:9200"]'

Load Kibana dashboards and create index patterns

sudo filebeat setup -E output.logstash.enabled=false -E output.elasticsearch.hosts=['localhost:9200'] -E setup.kibana.host=localhost:5601
```

Service Activation

Once configured, Filebeat must be started and enabled to persist across reboots.

bash sudo systemctl start filebeat sudo systemctl enable filebeat sudo systemctl status filebeat

Conclusion

The deployment of the ELK stack on Ubuntu transforms fragmented log data into a coherent, searchable, and visualizable asset. By adhering to strict version alignment across Elasticsearch, Logstash, and Kibana, and by correctly configuring Java dependencies and GPG-signed repositories, administrators establish a robust centralized logging infrastructure. The integration of Filebeat completes the architecture, enabling real-time shipment of logs into the pipeline. This setup not only aids in immediate troubleshooting but also provides long-term analytics capabilities essential for modern DevOps and system administration.

Sources

1. LinuxTechi
2. DigitalOcean
3. OneUptime
4. PortForwarded