In this tutorial, we will show you how to install and configure the ELK Stack on CentOS 7. Elasticsearch, Logstash, and Kibana are the three open-source products which are part of the collection called ELK stack. It is a robust solution for search, analyzes, and visualization of data. As we already know Elasticsearch is distributed RESTful search and analytics NoSQL engine based on Lucene, the light-weight data processing pipeline or Logstash is used for managing events and logs from many different sources and Kibana which is a web application is used for data visualization that works on top of Elsticsearch. This tutorial on installing ELK Stack on CentOS 7, should be good for any other Linux VPS systems, but originally it was intended or in other words written and tested for a CentOS-based virtual private server. The task of installing ELK Stack on CentOS 7 should be easy, you just need to follow the steps which are given below and the job will be done in less than ten minutes. Let’s get started.
Table of Contents
1. Prerequisites
The first thing you need in order to complete this tutorial successfully is root access to the CentOS 7 server.
The volume of logs that you want to gather is important because the amount of CPU, RAM, and storage required by your ELK Server depend on them.
System requirements:
- OS: CentOS 7
- RAM: 4GB
- CPU: 2
2. Install Java
The Elastic stack deployment requires Java. It supports both OpenJDK and Oracle Java. we recommend using the Oracle JDK 1.8. We will install Java 8 from the official Oracle rpm package.
Java 8 JDK can be downloaded with the following wget command.
wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http:%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u77-b02/jdk-8u171-linux-x64.rpm"
The next step is the installation with this rpm command:
rpm -ivh jdk-8u171-linux-x64.rpm
At this point, you should check the Java JDK installation.
java -version
if everything is installed all right, you will see the following output:
java version "1.8.0_172" Java(TM) SE Runtime Environment (build 1.8.0_172-b11) Java HotSpot(TM) 64-Bit Server VM (build 25.172-b11, mixed mode)
3. Install and Configure Elasticsearch on CentOS 7
Now is time for Elasticsearch to be installed and configured. We will use rpm package for this installation which is provided by elastic.co and then we will configure it to run on localhost ( all this is to be sure that we have made the setup secure and to make sure that it can not be reached from the outside).
You should add the elastic.co key to the server prior to the installation of Elasticsearch.
rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
Then you should use wget in order to download Elasticsearch 6.2 and initiate the installation
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.2.4.rpm rpm -ivh elasticsearch-6.2.4.rpm
At this point when Elasticsearch is already installed, you need to go to the configuration directory and edit the elasticsaerch.yml configuration file.
nano /etc/elasticsearch/elasticsearch.yml
Then you should remove a comment on line 40 in order to enable the memory lock for Elasticsearch. This disables memory swapping for Elasticsearch.
bootstrap.memory_lock: true
After that, you need to uncomment the network.host and http.port lines in the “Network” block.
network.host: localhost http.port: 9200
Don`t forget to save the file and then you can exit the editor.
Next, you should edit the elasticsearch.service file for the configuration of the memory lock.
vim /usr/lib/systemd/system/elasticsearch.service
Uncomment LimitMEMLOCK line.
LimitMEMLOCK=infinity
Again, don`t forget to save and then exit.
The next step is to edit the sysconfig configuration file for Elasticsearch.
nano /etc/sysconfig/elasticsearch
Now you need to uncomment line 60 and make sure the value is ‘unlimited’.
MAX_LOCKED_MEMORY=unlimited
Save and exit.
Completing all these steps means that you have finished the Elasticsearch configuration on CentOS 7. The localhost IP address on port 9200 will run the Elasticsearch,
Before you start the service first you should reload systems and enable Elasticsearch to start at boot time.
systemctl daemon-reload systemctl enable elasticsearch systemctl start elasticsearch
After a few seconds the Elasticsearch will start and then you should check the open ports on the server and double check that ‘state’ for port 9200 is ‘LISTEN’.
4.Install and configure Kibana on CentOS 7
When you reach this point, the following thing on the list is the installation and configuration of Kibana with a Nginx web server. Kibana will listen on the localhost IP address and Nginx acts as a reverse proxy for the Kibana application.
You should use wget in order to download Kibana 6.2.4 and then you should install it using the rpm command:
wget https://artifacts.elastic.co/downloads/kibana/kibana-6.2.4-x86_64.rpm rpm -ivh kibana-6.2.4-x86_64.rpm
Next step is editing the Kibana configuration file.
nano /etc/kibana/kibana.yml
Uncomment the configuration lines for server.port, server.host and elasticsearch.url.
server.port: 5601 server.host: "localhost" elasticsearch.url: "http://localhost:9200"
The changes need to be saved and then you can exit.
Start the Kibana service and set it to start automatically on boot:
systemctl enable kibana systemctl start kibana
You have now finished the installation of Kibana.
5. Install and configure Nginx
Next thing is to install Nginx and of course to configure it as a reverse proxy to access Kibana from the public IP address.
Nginx can be found in the Epel repository, install epel-release with yum.
yum install epel-release
Then, you need to install the Nginx and httpd-tools package.
yum install nginx httpd-tools
The tools for the web server are contained in httpd-tools package, you can use htpasswd basic authentication for Kibana.
At this point, it is necessary for us to create a new virtual host configuration file in the conf.d directory.
nano /etc/nginx/conf.d/your_domain.com.conf
so we will be able to add a new virtual host configuration. Paste the configuration below:
server {
listen 80;
server_name your_domain.com;
auth_basic "Restricted Access";
auth_basic_user_file /etc/nginx/htpasswd.kibana;
location / {
proxy_pass http://localhost:5601;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
}
Then a new basic authentication file should be created using the htpasswd command.
sudo htpasswd -c /etc/nginx/htpasswd.kibana admin TYPE YOUR STRONG PASSWORD
Add Nginx to run at the boot time and start Nginx.
systemctl enable nginx systemctl start nginx
6. Install and Configure Logstash
In the following step we would install Logsatash and configure it to centralize server logs from clients with filebeat, then filter and transform the Syslog data and move it into the stash (Elasticsearch).
Use rpm in order to download Logstash and install it.
wget https://artifacts.elastic.co/downloads/logstash/logstash-6.2.4.rpm rpm -ivh logstash-6.2.4.rpm
When the installation of Logstash is completed, start the Logstash service and set it to start automatically on boot:
systemctl restart logstash systemctl enable logstash
The Logstash configuration depends on your personal preferences and the plugins you will use.
That’s it. You have successfully installed the ELK Stack on your CentOS 7 VPS.
Follow this guide on how to Install ELK stack on Ubuntu 20.04
Of course, you don’t have to Install and Configure the ELK Stack on CentOS 7, if you use one of our Managed Elasticsearch VPS Hosting services, in which case you can simply ask our expert Linux admins to install the ELK stack for you. They are available 24×7 and will take care of your request immediately.
PS. If you liked this post please share it with your friends on the social networks using the buttons below or simply leave a comment in the comments section. Thanks.