The Elastic Stack, historically known as the ELK stack, represents the industry standard for centralized logging, real-time analytics, and infrastructure monitoring. Comprising Elasticsearch, Logstash, Kibana, and Beats (specifically Filebeat), this suite enables organizations to aggregate data from disparate sources, transform it through processing pipelines, and visualize the results in comprehensive dashboards. This technical guide details the precise steps to deploy this stack on Ubuntu Linux systems, covering version compatibility, system requirements, Java prerequisites, repository configuration, and the specific service configurations required for a functional, secure, and scalable monitoring environment.
Architectural Overview and System Requirements
The core value proposition of the Elastic Stack lies in centralized logging. By aggregating logs from multiple servers or applications into a single searchable index, administrators can correlate events across distributed systems, identify latency spikes, and troubleshoot complex failures that span multiple hosts. The standard data flow begins with log sources, which are collected by agents like Filebeat or processed by Logstash before being indexed by Elasticsearch. Kibana then provides the user interface for querying and visualizing this data.
Successful deployment requires strict adherence to version compatibility. All components within the stack—Elasticsearch, Logstash, Kibana, and Filebeat—must utilize the exact same major and minor version to ensure API compatibility and data structure alignment. Incompatibility between versions can result in failed index mappings, broken dashboards, or service startup failures.
Before initiating the installation, the host system must meet specific hardware thresholds to prevent performance degradation or out-of-memory errors during peak indexing loads.
| Component | Minimum RAM | Minimum CPU | Minimum Disk |
|---|---|---|---|
| Elasticsearch | 2GB+ | 2 cores | 50GB+ |
| Logstash | 1GB+ | 1 core | 10GB |
| Kibana | 1GB+ | 1 core | 1GB |
The operating system must be Ubuntu 20.04 or 22.04 (with specific instructions also applicable to 24.04). A minimum of 4GB of total system RAM is required, though 8GB is recommended for production workloads. At least two CPU cores are necessary to handle the concurrent indexing and query processing. Root or sudo access is mandatory for repository addition and service management.
Java Prerequisite Installation
Elasticsearch is built upon Apache Lucene and is developed in Java. Consequently, a compatible Java Runtime Environment (JRE) or Development Kit (JDK) is a strict prerequisite. The Elastic Stack supports Java 11 and Java 17. For Ubuntu-based systems, OpenJDK 17 is the recommended Long Term Support (LTS) release.
Execute the following commands to update the package index and install the Java environment:
bash
sudo apt update
sudo apt install openjdk-17-jre-headless -y
To verify the installation and confirm the correct version is active:
bash
java -version
Configuring the Elastic Repository
The ELK components are not available in the default Ubuntu apt repositories. Installation requires manually adding the official Elastic package source. This involves importing the GPG key to ensure package integrity and adding the specific repository list file.
Import the Elasticsearch GPG key and convert it to the keyring format used by APT:
bash
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
Add the repository source list, ensuring the signed-by parameter points to the newly created keyring:
bash
echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list
Refresh the local package cache to recognize the new source:
bash
sudo apt update
Deploying and Configuring Elasticsearch
Elasticsearch serves as the distributed, RESTful high-performance search and analytics engine. It handles the storage, indexing, and querying of JSON documents.
Install the Elasticsearch package:
bash
sudo apt install elasticsearch -y
Configuration requires editing the main YAML configuration file. For a single-node development or small production setup, the discovery type must be explicitly set to prevent cluster formation errors.
bash
sudo nano /etc/elasticsearch/elasticsearch.yml
The following configuration parameters are critical for a functional node:
yaml
cluster.name: elk-cluster
node.name: node-1
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: localhost
http.port: 9200
discovery.type: single-node
Security is enabled by default in recent versions. The configuration must include SSL settings for both HTTP and transport layers to secure inter-node communication and client access.
yaml
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
xpack.security.http.ssl:
enabled: true
keystore.path: certs/http.p12
xpack.security.transport.ssl:
enabled: true
verification_mode: certificate
keystore.path: certs/transport.p12
truststore.path: certs/transport.p12
The Java Virtual Machine (JVM) heap size must be tuned to prevent garbage collection pauses and out-of-memory crashes. A common best practice is to allocate half of the available system RAM, capped at 31GB. For a system with 4GB RAM, a 2GB heap is appropriate.
Edit the heap configuration file:
bash
sudo nano /etc/elasticsearch/jvm.options.d/heap.options
Insert the following parameters:
-Xms2g
-Xmx2g
Initialize the systemd daemon, enable the service for boot persistence, and start the process:
bash
sudo systemctl daemon-reload
sudo systemctl enable elasticsearch
sudo systemctl start elasticsearch
sudo systemctl status elasticsearch
To secure the default elastic superuser account, reset its password:
bash
sudo /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic
Verify the service is responsive via cURL. If SSL is enabled:
bash
curl -k -u elastic:YOUR_PASSWORD https://localhost:9200
Deploying and Configuring Kibana
Kibana provides the web-based UI for visualizing data. It supports histograms, line graphs, pie charts, and geospatial maps. Since Kibana typically binds to localhost by default, external access requires either direct configuration changes or a reverse proxy (such as Nginx).
Install the Kibana package:
bash
sudo apt install kibana -y
Edit the Kibana configuration to bind to all network interfaces and define the connection to Elasticsearch:
bash
sudo nano /etc/kibana/kibana.yml
Configure the server settings:
yaml
server.port: 5601
server.host: "0.0.0.0"
server.name: "kibana-server"
Integrating Filebeat for Log Collection
Filebeat is a lightweight shipper that collects logs from file sources and forwards them to Elasticsearch or Logstash. It acts as the bridge between raw system logs and the storage engine.
Install the Filebeat package:
bash
sudo apt install filebeat -y
Configure the Apache module to capture access and error logs. Edit the configuration file:
bash
sudo nano /etc/filebeat/filebeat.yml
Enable the Apache module and define the log paths. If left empty, Filebeat defaults to OS-specific paths, but explicit definitions ensure precision.
yaml
var.paths: ["/var/log/apache2/access.log*"]
error:
enabled: true
var.paths: ["/var/log/apache2/error.log*"]
Before starting the service, ingest the necessary index templates and dashboard definitions into the stack. This ensures Kibana recognizes the data structure immediately upon receipt.
Load the index template, pointing to the Elasticsearch host:
bash
sudo filebeat setup --index-management -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["localhost:9200"]'
Load the Kibana dashboards and index patterns:
bash
sudo filebeat setup -E output.logstash.enabled=false -E output.elasticsearch.hosts=['localhost:9200'] -E setup.kibana.host=localhost:5601
For remote servers, replace localhost with the specific IP address of the ELK stack server (e.g., 192.168.x.x).
Start and enable the Filebeat service:
bash
sudo systemctl start filebeat
sudo systemctl enable filebeat
sudo systemctl status filebeat
Conclusion
Deploying the Elastic Stack on Ubuntu establishes a robust foundation for observability. By strictly adhering to version parity, tuning JVM heap sizes, and correctly configuring the YAML parameters for discovery and security, administrators avoid common pitfalls such as split-brain scenarios or unsecured endpoints. The integration of Filebeat completes the pipeline, transforming raw system logs into actionable intelligence within Kibana. This infrastructure enables real-time correlation of events across distributed systems, providing the visibility required for modern DevOps practices and incident response.