文章
181
标签
50
分类
65
主页
时间线
标签
分类
链接
关于
List
目录
  1. 1. elasticsearch
    1. 1.1. elasticsearch介绍
    2. 1.2. elasticsearch安装
    3. 1.3. elasticsearch配置
      1. 1.3.1. 单机配置
      2. 1.3.2. 集群
    4. 1.4. 文献引用
elasticsearch安装与应用
|工具安装与部署
字数总计: 1.6k|阅读时长: 8 分钟|阅读量:

elasticsearch

elasticsearch介绍

1
elasticsearch ， 是一个基于[Lucene](https://lucene.apache.org/)的搜索服务器。它提供了一个基于RESTFul web 接口的分布式多用户能力的全文搜索引擎。它处理大数据非常的快，支持非结构化数据的搜索。

elasticsearch安装

  1. 下载elasticsearch-7.4.3
1
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.4.2-linux-x86_64.tar.gz
  1. 安装
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
tar -zxvf elasticsearch-7.4.2-linux-x86_64.tar.gz -C /opt/elk/

# 创建elasticsearch账户
useradd elasticsearch
chown -R elasticsearch:elasticsearch elasticsearch-7.4.0

cd /opt/elk
cd elasticsearch-7.4.2
# es目录结构
ls -l
drwxr-xr-x  2 gnome gnome   4096  9月 27 16:40 bin        # 可执行文件目录
drwxr-xr-x  2 gnome gnome   4096 10月 20 17:11 config     # 配置文件目录
drwxr-xr-x 10 gnome gnome   4096  9月 27 16:40 jdk  		# 自带jdk目录
drwxr-xr-x  3 gnome gnome   4096  9月 27 16:40 lib		# 库目录
-rw-r--r--  1 gnome gnome  13675  9月 27 16:35 LICENSE.txt 
drwxr-xr-x  2 gnome gnome   4096  9月 27 16:40 logs		# 日志目录
drwxr-xr-x 37 gnome gnome   4096  9月 27 16:40 modules	# 模块目录
-rw-r--r--  1 gnome gnome 523209  9月 27 16:40 NOTICE.txt
drwxr-xr-x  2 gnome gnome   4096  9月 27 16:40 plugins	# 插件目录
-rw-r--r--  1 gnome gnome   8500  9月 27 16:35 README.textile

elasticsearch配置

单机配置

  1. 进入配置目录
1
2
3
4
5
6
7
8
9
10
cd config

ls
-rw-r----- 1 gnome gnome  2867 10月 20 17:11 elasticsearch.yml # es配置文件
-rw-r----- 1 gnome gnome  3593  9月 27 16:35 jvm.options # jvm虚拟机选项
-rw-r----- 1 gnome gnome 17545  9月 27 16:40 log4j2.properties # 日志文件
-rw-r----- 1 gnome gnome   473  9月 27 16:40 role_mapping.yml  # 角色映射文件
-rw-r----- 1 gnome gnome   197  9月 27 16:40 roles.yml  # 角色文件
-rw-r----- 1 gnome gnome     0  9月 27 16:40 users  # 用户文件
-rw-r----- 1 gnome gnome     0  9月 27 16:40 users_roles # 用户角色
  1. elasticsearch.yml配置文件
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: elasticsearch # 集群名称，一般需要有意义
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node-master # 节点名
node.master: true # 是主节点
# node.data: false 
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /opt/elk/elasticsearch-7.4.0/data # 数据存储路径
#
# Path to log files:
#
path.logs: /opt/elk/elasticsearch-7.4.0/logs	# 日志文件路径
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
# 内存
bootstrap.memory_lock: false
bootstrap.system_call_filter: false

# 允许跨域
http.cors.enabled: true
http.cors.allow-origin: "*"
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 0.0.0.0 # 本节点绑定主机
#
# Set a custom port for HTTP:
#
http.port: 9200 # es开放端口
transport.tcp.port: 9300
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:

cluster.initial_master_nodes: ["127.0.0.1"]
#
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
  1. jvm.options配置(一般默认)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
## JVM configuration

################################################################
## IMPORTANT: JVM heap size
################################################################
##
## You should always set the min and max JVM heap
## size to the same value. For example, to set
## the heap to 4 GB, set:
##
## -Xms4g
## -Xmx4g
##
## See https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html
## for more information
##
################################################################

# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space

-Xms1g	# java虚拟机初始堆容量
-Xmx1g	# java虚拟机最大堆容量

################################################################
## Expert settings
################################################################
##
## All settings below this section are considered
## expert settings. Don't tamper with them unless
## you understand what you are doing
##
################################################################

## GC configuration 垃圾回收配置
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly

## G1GC Configuration
# NOTE: G1GC is only supported on JDK version 10 or later.
# To use G1GC uncomment the lines below.
# 10-:-XX:-UseConcMarkSweepGC
# 10-:-XX:-UseCMSInitiatingOccupancyOnly
# 10-:-XX:+UseG1GC
# 10-:-XX:G1ReservePercent=25
# 10-:-XX:InitiatingHeapOccupancyPercent=30

## DNS cache policy
# cache ttl in seconds for positive DNS lookups noting that this overrides the
# JDK security property networkaddress.cache.ttl; set to -1 to cache forever
-Des.networkaddress.cache.ttl=60
# cache ttl in seconds for negative DNS lookups noting that this overrides the
# JDK security property networkaddress.cache.negative ttl; set to -1 to cache
# forever
-Des.networkaddress.cache.negative.ttl=10

## optimizations

# pre-touch memory pages used by the JVM during initialization
-XX:+AlwaysPreTouch

## basic

# explicitly set the stack size
-Xss1m

# set to headless, just in case
-Djava.awt.headless=true

# ensure UTF-8 encoding by default (e.g. filenames)
-Dfile.encoding=UTF-8

# use our provided JNA always versus the system one
-Djna.nosys=true

# turn off a JDK optimization that throws away stack traces for common
# exceptions because stack traces are important for debugging
-XX:-OmitStackTraceInFastThrow

# flags to configure Netty
-Dio.netty.noUnsafe=true
-Dio.netty.noKeySetOptimization=true
-Dio.netty.recycler.maxCapacityPerThread=0
-Dio.netty.allocator.numDirectArenas=0

# log4j 2
-Dlog4j.shutdownHookEnabled=false
-Dlog4j2.disable.jmx=true

-Djava.io.tmpdir=${ES_TMPDIR}

## heap dumps

# generate a heap dump when an allocation from the Java heap fails
# heap dumps are created in the working directory of the JVM
-XX:+HeapDumpOnOutOfMemoryError

# specify an alternative path for heap dumps; ensure the directory exists and
# has sufficient space
-XX:HeapDumpPath=data

# specify an alternative path for JVM fatal error logs
-XX:ErrorFile=logs/hs_err_pid%p.log

## JDK 8 GC logging

8:-XX:+PrintGCDetails
8:-XX:+PrintGCDateStamps
8:-XX:+PrintTenuringDistribution
8:-XX:+PrintGCApplicationStoppedTime
8:-Xloggc:logs/gc.log
8:-XX:+UseGCLogFileRotation
8:-XX:NumberOfGCLogFiles=32
8:-XX:GCLogFileSize=64m

# JDK 9+ GC logging
9-:-Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m
# due to internationalization enhancements in JDK 9 Elasticsearch need to set the provider to COMPAT otherwise
# time/date parsing will break in an incompatible way for some date patterns and locals
9-:-Djava.locale.providers=COMPAT
  1. 错误
1
2
3
4
5
6
7
8
9
10
11
# 虚拟内存问题
[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
# 解决
echo "vm.max_map_count = 262144" >> /etc/sysctl.conf
sudo sysctl -p

# 问题2
[1]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured

# 解决
cluster.initial_master_nodes: ["127.0.0.1"]
  1. 配置完成
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
curl http://localhost:9200/
{
  "name" : "node-master",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "_na_",
  "version" : {
    "number" : "7.4.0",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "22e1767283e61a198cb4db791ea66e3f11ab9910",
    "build_date" : "2019-09-27T08:36:48.569419Z",
    "build_snapshot" : false,
    "lucene_version" : "8.2.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

集群

文献引用

  1. elasticsearch与solr对比
  2. 基于elasticsearch2.x的官方文档
  3. elastic中文社区
  4. elasticsearch安装与配置介绍
  5. Elasticsearch入门（三）：Elasticsearch 7.0.0 集群搭建
  6. Elasticsearch 重要配置详解
  7. 虚拟内存问题解决
文章作者: rack-leen
文章链接: http://yoursite.com/2019/11/26/%E5%B7%A5%E5%85%B7%E5%AE%89%E8%A3%85%E4%B8%8E%E9%83%A8%E7%BD%B2/elasticsearch%E5%AE%89%E8%A3%85%E4%B8%8E%E5%BA%94%E7%94%A8/
版权声明: 本博客所有文章除特别声明外，均采用 CC BY-NC-SA 4.0 许可协议。转载请注明来自 rack-leen's blog！
工具安装与部署
打赏
  • 微信
  • 支付宝
相关推荐
varnish安装与应用
logstash安装与应用
filebeat安装与应用
评论
简