elasticsearch安装与应用

¶elasticsearch

¶elasticsearch介绍

code

elasticsearch ， 是一个基于[Lucene](https://lucene.apache.org/)的搜索服务器。它提供了一个基于RESTFul web 接口的分布式多用户能力的全文搜索引擎。它处理大数据非常的快，支持非结构化数据的搜索。

¶elasticsearch安装

下载elasticsearch-7.4.3

code

1	wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.4.2-linux-x86_64.tar.gz

安装

code

tar -zxvf elasticsearch-7.4.2-linux-x86_64.tar.gz -C /opt/elk/

# 创建elasticsearch账户
useradd elasticsearch
chown -R elasticsearch:elasticsearch elasticsearch-7.4.0

cd /opt/elk
cd elasticsearch-7.4.2
# es目录结构
ls -l
drwxr-xr-x  2 gnome gnome   4096  9月 27 16:40 bin        # 可执行文件目录
drwxr-xr-x  2 gnome gnome   4096 10月 20 17:11 config     # 配置文件目录
drwxr-xr-x 10 gnome gnome   4096  9月 27 16:40 jdk  		# 自带jdk目录
drwxr-xr-x  3 gnome gnome   4096  9月 27 16:40 lib		# 库目录
-rw-r--r--  1 gnome gnome  13675  9月 27 16:35 LICENSE.txt 
drwxr-xr-x  2 gnome gnome   4096  9月 27 16:40 logs		# 日志目录
drwxr-xr-x 37 gnome gnome   4096  9月 27 16:40 modules	# 模块目录
-rw-r--r--  1 gnome gnome 523209  9月 27 16:40 NOTICE.txt
drwxr-xr-x  2 gnome gnome   4096  9月 27 16:40 plugins	# 插件目录
-rw-r--r--  1 gnome gnome   8500  9月 27 16:35 README.textile

¶elasticsearch配置

¶单机配置

code

cd config

ls
-rw-r----- 1 gnome gnome  2867 10月 20 17:11 elasticsearch.yml # es配置文件
-rw-r----- 1 gnome gnome  3593  9月 27 16:35 jvm.options # jvm虚拟机选项
-rw-r----- 1 gnome gnome 17545  9月 27 16:40 log4j2.properties # 日志文件
-rw-r----- 1 gnome gnome   473  9月 27 16:40 role_mapping.yml  # 角色映射文件
-rw-r----- 1 gnome gnome   197  9月 27 16:40 roles.yml  # 角色文件
-rw-r----- 1 gnome gnome     0  9月 27 16:40 users  # 用户文件
-rw-r----- 1 gnome gnome     0  9月 27 16:40 users_roles # 用户角色

elasticsearch.yml配置文件

bash

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: elasticsearch # 集群名称，一般需要有意义
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node-master # 节点名
node.master: true # 是主节点
# node.data: false 
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /opt/elk/elasticsearch-7.4.0/data # 数据存储路径
#
# Path to log files:
#
path.logs: /opt/elk/elasticsearch-7.4.0/logs	# 日志文件路径
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
# 内存
bootstrap.memory_lock: false
bootstrap.system_call_filter: false

# 允许跨域
http.cors.enabled: true
http.cors.allow-origin: "*"
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 0.0.0.0 # 本节点绑定主机
#
# Set a custom port for HTTP:
#
http.port: 9200 # es开放端口
transport.tcp.port: 9300
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:

cluster.initial_master_nodes: ["127.0.0.1"]
#
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true

jvm.options配置(一般默认)

code

## JVM configuration

################################################################
## IMPORTANT: JVM heap size
################################################################
##
## You should always set the min and max JVM heap
## size to the same value. For example, to set
## the heap to 4 GB, set:
##
## -Xms4g
## -Xmx4g
##
## See https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html
## for more information
##
################################################################

# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space

-Xms1g	# java虚拟机初始堆容量
-Xmx1g	# java虚拟机最大堆容量

################################################################
## Expert settings
################################################################
##
## All settings below this section are considered
## expert settings. Don't tamper with them unless
## you understand what you are doing
##
################################################################

## GC configuration 垃圾回收配置
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly

## G1GC Configuration
# NOTE: G1GC is only supported on JDK version 10 or later.
# To use G1GC uncomment the lines below.
# 10-:-XX:-UseConcMarkSweepGC
# 10-:-XX:-UseCMSInitiatingOccupancyOnly
# 10-:-XX:+UseG1GC
# 10-:-XX:G1ReservePercent=25
# 10-:-XX:InitiatingHeapOccupancyPercent=30

## DNS cache policy
# cache ttl in seconds for positive DNS lookups noting that this overrides the
# JDK security property networkaddress.cache.ttl; set to -1 to cache forever
-Des.networkaddress.cache.ttl=60
# cache ttl in seconds for negative DNS lookups noting that this overrides the
# JDK security property networkaddress.cache.negative ttl; set to -1 to cache
# forever
-Des.networkaddress.cache.negative.ttl=10

## optimizations

# pre-touch memory pages used by the JVM during initialization
-XX:+AlwaysPreTouch

## basic

# explicitly set the stack size
-Xss1m

# set to headless, just in case
-Djava.awt.headless=true

# ensure UTF-8 encoding by default (e.g. filenames)
-Dfile.encoding=UTF-8

# use our provided JNA always versus the system one
-Djna.nosys=true

# turn off a JDK optimization that throws away stack traces for common
# exceptions because stack traces are important for debugging
-XX:-OmitStackTraceInFastThrow

# flags to configure Netty
-Dio.netty.noUnsafe=true
-Dio.netty.noKeySetOptimization=true
-Dio.netty.recycler.maxCapacityPerThread=0
-Dio.netty.allocator.numDirectArenas=0

# log4j 2
-Dlog4j.shutdownHookEnabled=false
-Dlog4j2.disable.jmx=true

-Djava.io.tmpdir=${ES_TMPDIR}

## heap dumps

# generate a heap dump when an allocation from the Java heap fails
# heap dumps are created in the working directory of the JVM
-XX:+HeapDumpOnOutOfMemoryError

# specify an alternative path for heap dumps; ensure the directory exists and
# has sufficient space
-XX:HeapDumpPath=data

# specify an alternative path for JVM fatal error logs
-XX:ErrorFile=logs/hs_err_pid%p.log

## JDK 8 GC logging

8:-XX:+PrintGCDetails
8:-XX:+PrintGCDateStamps
8:-XX:+PrintTenuringDistribution
8:-XX:+PrintGCApplicationStoppedTime
8:-Xloggc:logs/gc.log
8:-XX:+UseGCLogFileRotation
8:-XX:NumberOfGCLogFiles=32
8:-XX:GCLogFileSize=64m

# JDK 9+ GC logging
9-:-Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m
# due to internationalization enhancements in JDK 9 Elasticsearch need to set the provider to COMPAT otherwise
# time/date parsing will break in an incompatible way for some date patterns and locals
9-:-Djava.locale.providers=COMPAT

错误

code

# 虚拟内存问题
[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
# 解决
echo "vm.max_map_count = 262144" >> /etc/sysctl.conf
sudo sysctl -p

# 问题2
[1]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured

# 解决
cluster.initial_master_nodes: ["127.0.0.1"]

配置完成

code

curl http://localhost:9200/
{
  "name" : "node-master",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "_na_",
  "version" : {
    "number" : "7.4.0",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "22e1767283e61a198cb4db791ea66e3f11ab9910",
    "build_date" : "2019-09-27T08:36:48.569419Z",
    "build_snapshot" : false,
    "lucene_version" : "8.2.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

¶elasticsearch

¶elasticsearch介绍

¶elasticsearch安装

¶elasticsearch配置

¶单机配置

¶集群

¶文献引用