The integration of the Elastic Stack (ELK) within a Red Hat OpenShift environment represents a sophisticated approach to cluster observability, transitioning from basic log collection to a comprehensive telemetry ecosystem. OpenShift, as an enterprise-grade Kubernetes distribution, generates a massive volume of ephemeral data across its pods, nodes, and control plane. Managing this data requires a specialized architecture capable of ingestion, indexing, and visualization at scale. The ELK stack—comprising Elasticsearch, Logstash, and Kibana—provides the necessary machinery to transform raw, unstructured logs into actionable operational intelligence. When deployed on OpenShift, this stack must be carefully orchestrated to respect the platform's stringent security constraints, such as Security Context Constraints (SCCs), and its specific resource management policies. Whether utilizing the Elastic Cloud on Kubernetes (ECK) operator for a managed lifecycle or leveraging Helm charts for granular control, the goal is to establish a centralized logging subsystem that remains independent of the monitored objects to ensure data integrity and availability during cluster failures.
The Architectural Components of the ELK Stack
The effectiveness of the ELK stack on OpenShift relies on the symbiotic relationship between its four primary components: Elasticsearch, Logstash, Kibana, and the Beats family of shippers.
Elasticsearch serves as the heart of the stack. It is a distributed, RESTful search and analytics engine capable of storing and indexing vast amounts of data in near real-time. In an OpenShift context, Elasticsearch is critical because it allows sysadmins to aggregate data and discover trends and patterns as the volume of queries and data grows. Its scalability is a primary advantage; as the cluster expands, additional nodes can be deployed to meet the demand. This is particularly vital for analyzing security events or performing high-speed searches across thousands of containers.
Logstash acts as the data processing pipeline. It collects data from multiple sources, transforms it via filters, and then forwards it to Elasticsearch. Logstash is the "transformer" of the stack, taking raw inputs from files or syslogs and converting them into a structured format. This ensures that the data arriving in Elasticsearch is clean, normalized, and ready for analysis.
Kibana provides the visualization layer. It is the window into the data stored within Elasticsearch, allowing users to explore logs through dashboards, graphs, and maps. Beyond visualization, Kibana serves as the administrative console for managing and monitoring the entire ELK stack.
Beats are lightweight shippers designed for efficiency. Because Logstash can be resource-heavy, Beats are installed on target devices to feed information to Logstash or Elasticsearch directly.
- Filebeat: Specifically designed for logs and files.
- Packetbeat: Used for network packet analysis.
- Winlogbeat: Specialized for Windows event logs.
- Metricbeat: Used for collecting system and service statistics.
Deployment Methodologies for OpenShift
There are three primary paths to deploying the ELK stack on Red Hat OpenShift, each offering different levels of abstraction and control.
The ECK (Elastic Cloud on Kubernetes) Operator Path
The Elastic Cloud on Kubernetes (ECK) operator is the official method for simplifying the setup, upgrade, and scaling of the Elastic Stack. By using ECK on OpenShift, administrators can orchestrate a stateful deployment in minutes. The operator embeds built-in best practices that extend beyond the initial installation (Day 1 operations) into ongoing lifecycle management. This method is particularly advantageous for organizations requiring a unified view of OpenShift logs and metrics alongside other infrastructure monitoring data through Elastic Observability.
The Helm Chart Deployment Path
For those requiring more granular control over the configuration, Helm charts provide a flexible mechanism for deployment. This approach involves adding the elastic helm repository and deploying components individually or via a setup script.
To initiate this process, the following sequence of commands is utilized:
First, the elastic helm repository must be added to the local environment:
helm repo add elastic https://helm.elastic.co
Before executing any deployment scripts, a dedicated namespace must be created to isolate the logging components:
oc new-project logging --display-name='Logging'
Once the namespace is established, the stack can be deployed using a comprehensive setup script:
./elk_setup.sh
Alternatively, components can be deployed individually for targeted upgrades or configurations:
For Elasticsearch:
helm upgrade --install elasticsearch elastic/elasticsearch --values elasticsearch/values.yaml
For Logstash:
helm upgrade --install logstash elastic/logstash --values logstash/values.yaml
For Filebeat:
helm upgrade --install filebeat elastic/filebeat --values filebeat/values.yaml
For Kibana:
helm upgrade --install kibana elastic/kibana --values kibana/values.yaml
The Red Hat Logging Subsystem Path
Red Hat provides a native logging subsystem that can be installed via the OpenShift Container Platform web console. This method involves deploying the OpenShift Elasticsearch Operator and the Red Hat OpenShift Logging Operator. The former manages the Elasticsearch cluster specifically for OpenShift logging, while the latter manages the broader logging stack components.
Users who wish to use an external log store can remove the internal Elasticsearch logStore and Kibana visualization components from the ClusterLogging custom resource (CR) to save cluster resources.
Technical Prerequisites and System Configuration
Deploying ELK on OpenShift is not a simple "plug-and-play" operation; it requires specific system-level adjustments to prevent catastrophic failures.
Virtual Memory and Kernel Tuning
Elasticsearch utilizes a mmapfs directory by default to store its indices efficiently. However, the default operating system limits on mmap counts are typically too low for production workloads, which frequently results in out-of-memory (OOM) exceptions. To prevent this, it is strongly recommended to increase the kernel setting vm.max_map_count to 262144. This is a critical step that must be performed on the Kubernetes nodes before the Elasticsearch deployment begins.
Storage Requirements
Elasticsearch is a stateful, memory-intensive application that requires robust persistent storage.
- Each Elasticsearch node must have its own dedicated storage volume.
- Local volumes are supported, but there is a strict prohibition against using raw block volumes.
- In the
LocalVolumeobject, thevolumeMode: blocksetting must not be used, as Elasticsearch cannot operate on raw block volumes.
Administrative Privileges
To perform these installations, the user must possess high-level cluster privileges. Specifically, the user must be a system:admin or have the authority to create:
- Projects
- Custom Resource Definitions (CRDs)
- RBAC (Role-Based Access Control) resources at the cluster level.
Security and Policy Configuration in OpenShift
OpenShift's security model is significantly more restrictive than standard Kubernetes, which necessitates specific policy adjustments for log shippers like Filebeat.
Security Context Constraints (SCC)
Filebeat needs to access host-level logs to ship them to the ELK stack. This requires privileged access to the node's filesystem. By default, OpenShift prevents pods from running as privileged users. Therefore, an administrator must explicitly grant the privileged SCC to the Filebeat service account.
Assuming the project name is logging, the following command is used:
oc adm policy add-scc-to-user privileged system:serviceaccount:logging:filebeat-filebeat
Integration and Agent Security
When using the Kubernetes integration for monitoring, security must be prioritized during the agent enrollment process. When installing agents via the Fleet management system, the manifest provided by the agent policy cannot be applied directly without customization for OpenShift compatibility. A critical security requirement is the creation of a secret that contains the fleet enrollment token to ensure that agents are securely authenticated to the Elastic cluster.
Detailed Monitoring Workflow with Elastic Stack
Monitoring an OpenShift cluster requires a strategy where logs are centralized and independent from the objects being monitored.
Integration Setup
Since OpenShift is based on Kubernetes, the standard Kubernetes integration provided by Elastic is used. The process involves:
1. Navigating to the integration section and selecting the Kubernetes integration.
2. Creating an agent policy that includes the Kubernetes integration and any additional required policies.
3. Navigating to Fleet > Add Agent to retrieve the manifest.
Agent Deployment Process
After obtaining the manifest, the administrator must apply the necessary OpenShift-specific customizations. This ensures that the agent can operate within the OpenShift networking and security constraints while maintaining a secure connection to the fleet via the aforementioned enrollment token secret.
Lifecycle Management and Decommissioning
Maintaining the ELK stack involves not only deployment but also clean removal to prevent resource leakage (such as orphaned PVCs).
To fully clean up the ELK stack, the following steps are required:
- Uninstall the ELK and Filebeat Helm releases.
- Delete the Kibana Route to free up the network endpoint.
- Release the Persistent Volume Claims (PVCs) to reclaim storage on the physical disks.
Technical Specifications Summary
The following table details the critical configuration values and requirements for an enterprise ELK deployment on OpenShift.
| Component/Setting | Required Value/Requirement | Impact of Non-Compliance |
|---|---|---|
vm.max_map_count |
262144 |
Out of Memory (OOM) exceptions |
| Storage Mode | File-system (Not Block) | Elasticsearch failure to start |
| Filebeat Privileges | privileged SCC |
Unable to read host log files |
| User Role | system:admin |
Permission denied for CRD/Project creation |
| Namespace | logging (or custom) |
Deployment failure/Resource collision |
Conclusion
The deployment of the ELK stack on Red Hat OpenShift is a complex but rewarding engineering feat that transforms the way infrastructure is monitored. By moving beyond the built-in monitoring systems of OpenShift and implementing a dedicated Elastic Stack, organizations achieve a level of observability that is both scalable and independent. The technical journey—from tuning the Linux kernel's vm.max_map_count and managing Security Context Constraints for Filebeat, to orchestrating the deployment via ECK or Helm—highlights the necessity of a rigorous approach to stateful application management. The ability to ingest logs via Beats, process them through Logstash, index them in Elasticsearch, and visualize them in Kibana creates a closed-loop system for operational intelligence. This architecture not only supports standard logging but also enables advanced security analysis and performance monitoring, ensuring that as the OpenShift cluster grows in complexity, the visibility into its inner workings remains crystal clear.