共计 14652 个字符,预计需要花费 37 分钟才能阅读完成。
Elasticsearch 的功能,主要用在搜索领域,这里,我来研究这个,也是项目需要,为公司开发了一款 CMS 系统,网站上的搜索栏功能,我打算采用 Elasticsearch 来实现。Elasticsearch 的高性能,低延时是最大的吸引力。
系统环境:
CentOS6.8,X86_64。
Elasticsearch 的版本:2.3.5,下载地址来自官网,选择的是 RPM 包。
es 安装完毕后,默认安装的路径是 /usr/share/elasticsearch, 配置文件,默认是在 /etc/elasticsearch 目录。这里要重点说说配置文件的信息:
1 [root@CloudGame bin]# cd /etc/elasticsearch/ | |
2 [root@CloudGame elasticsearch]# ll | |
3 total 12 | |
4 -rwxr-x---. 1 root elasticsearch 3189 Jul 27 18:34 elasticsearch.yml | |
5 -rwxr-x---. 1 root elasticsearch 2571 Jul 27 18:34 logging.yml | |
6 drwxr-x---. 2 root elasticsearch 4096 Jul 27 18:44 scripts | |
7 [root@CloudGame elasticsearch]# tree | |
8 . | |
9 ├── elasticsearch.yml #es 的系统配置文件 | |
10 ├── logging.yml #es 的日志配置文件 | |
11 └── scripts | |
12 | |
13 1 directory, 2 files |
单实例的启动很简单,安装完成后,直接 service elasticsearch start 即可启动,日志在 /var/log/elasticsearch 目录,数据在 /var/lib/elasticsearch 目录。
这里重点放在如何配置单机多实例上面。为了简单起见,多实例就选择启动 2 个实例,更多的实例,和 2 个实例是一样的配置,只是简单的相应改改参数即可。
1. 在 /etc 目录下创建 esconf 目录,然后在 esconf 下面创建 esins1 以及 esins2 目录。将原本 /etc/elasticsearch 目录的内容 copy 到 esins1 以及 esins2 目录下。
1 [root@CloudGame esconf]# pwd | |
2 /etc/esconf | |
3 [root@CloudGame esconf]# tree | |
4 . | |
5 ├── esins1 | |
6 │ ├── elasticsearch.yml | |
7 │ ├── logging.yml | |
8 │ └── scripts | |
9 ├── esins2 | |
10 ├── elasticsearch.yml | |
11 ├── logging.yml | |
12 └── scripts | |
13 | |
14 | |
15 4 directories, 6 files |
2. 修改配置文件 elasticsearch.yml, 对 esins1 以及 esins2 下面的这个文件都做修改。主要是修改其节点信息,网络链接信息,以及日志数据路径信息。下面直接上修改好的配置文件内容。
实例 1:esins1 目录下的 elasticsearch.yml 内容
1 [root@CloudGame esins1]# cat elasticsearch.yml | |
2 # ======================== Elasticsearch Configuration ========================= | |
3 # | |
4 # NOTE: Elasticsearch comes with reasonable defaults for most settings. | |
5 # Before you set out to tweak and tune the configuration, make sure you | |
6 # understand what are you trying to accomplish and the consequences. | |
7 # | |
8 # The primary way of configuring a node is via this file. This template lists | |
9 # the most important settings you may want to configure for a production cluster. | |
10 # | |
11 # Please see the documentation for further information on configuration options: | |
12 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/setup-configuration.html> | |
13 # | |
14 # ---------------------------------- Cluster ----------------------------------- | |
15 # | |
16 # Use a descriptive name for your cluster: | |
17 # | |
18 cluster.name: tksearch #es 默认的集群名称是 elasticsearch,注意,集群中是以 name 来区分节点属于那个集群的。19 # | |
20 # ------------------------------------ Node ------------------------------------ | |
21 # | |
22 # Use a descriptive name for the node: | |
23 # | |
24 node.name: node1 # 节点的名称 | |
25 # | |
26 # Add custom attributes to the node: | |
27 # | |
28 # node.rack: r1 | |
29 # | |
30 node.master: true # 是否让这个节点作为默认的 master,若不是,默认会选择集群里面的第一个作为 master,es 有一套选择那个节点作为 master 的机制 | |
31 # ----------------------------------- Paths ------------------------------------ | |
32 # | |
33 # Path to directory where to store the data (separate multiple locations by comma): | |
34 # | |
35 path.data: /home/AppData/es/esins1/data # 配置节点数据存放的目录 | |
36 # | |
37 # Path to log files: | |
38 # | |
39 path.logs: /home/AppData/es/esins1/logs # 配置节点日志存放的目录 | |
40 # | |
41 # ----------------------------------- Memory ----------------------------------- | |
42 # | |
43 # Lock the memory on startup: | |
44 # | |
45 # bootstrap.mlockall: true | |
46 # | |
47 # Make sure that the `ES_HEAP_SIZE` environment variable is set to about half the memory | |
48 # available on the system and that the owner of the process is allowed to use this limit. | |
49 # | |
50 # Elasticsearch performs poorly when the system is swapping the memory. | |
51 # | |
52 # ---------------------------------- Network ----------------------------------- | |
53 # | |
54 # Set the bind address to a specific IP (IPv4 or IPv6): | |
55 # | |
56 network.host: 0.0.0.0 # 配置节点绑定的地址,全 0 表示可以绑定任何地址,当然这里,本机可以是 127.0.0.1 回还地址,也可以是 ifconfig 看到的 eth1 的地址。 | |
57 # | |
58 # Set a custom port for HTTP: | |
59 # | |
60 http.port: 9200 # 配置当前节点对外 http 访问的端口号,默认是 9200,不配的话,es 会从 9200-9299 当中找一个未用过的。 | |
61 # | |
62 # For more information, see the documentation at: | |
63 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html> | |
64 # | |
65 transport.tcp.port: 9300 #es 集群节点之间的通信端口号。默认 9300. | |
66 # --------------------------------- Discovery ---------------------------------- | |
67 # | |
68 # Pass an initial list of hosts to perform discovery when new node is started: | |
69 # The default list of hosts is ["127.0.0.1", "[::1]"] | |
70 # | |
71 discovery.zen.ping.unicast.hosts: ["127.0.0.1:9300"] # 集群多播时发现其他节点的主机列表,真实多机集群环境下,这里会是多个主机的 IP 列表,默认格式“host:port”的数组 | |
72 # | |
73 # Prevent the "split brain" by configuring the majority of nodes (total number of nodes / 2 + 1): | |
74 # | |
75 # discovery.zen.minimum_master_nodes: 3 | |
76 # | |
77 # For more information, see the documentation at: | |
78 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery.html> | |
79 # | |
80 # ---------------------------------- Gateway ----------------------------------- | |
81 # | |
82 # Block initial recovery after a full cluster restart until N nodes are started: | |
83 # | |
84 # gateway.recover_after_nodes: 3 | |
85 # | |
86 # For more information, see the documentation at: | |
87 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-gateway.html> | |
88 # | |
89 # ---------------------------------- Various ----------------------------------- | |
90 # | |
91 # Disable starting multiple nodes on a single system: | |
92 # | |
93 node.max_local_storage_nodes: 2 # 默认情况下,是不建议单机启动多个 node 的,这里这个参数,就是告知 es 单机上启动了几个实例,这里我们配置 2 个,若是要配置 3 个或者更多实例,修改这个数字即可 | |
94 # | |
95 # Require explicit names when deleting indices: | |
96 # | |
97 # action.destructive_requires_name: true |
实例 2:esins2 目录下的 elasticsearch.yml 内容,配置和 esins1 几乎一样。
1 [root@CloudGame esins2]# cat elasticsearch.yml | |
2 # ======================== Elasticsearch Configuration ========================= | |
3 # | |
4 # NOTE: Elasticsearch comes with reasonable defaults for most settings. | |
5 # Before you set out to tweak and tune the configuration, make sure you | |
6 # understand what are you trying to accomplish and the consequences. | |
7 # | |
8 # The primary way of configuring a node is via this file. This template lists | |
9 # the most important settings you may want to configure for a production cluster. | |
10 # | |
11 # Please see the documentation for further information on configuration options: | |
12 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/setup-configuration.html> | |
13 # | |
14 # ---------------------------------- Cluster ----------------------------------- | |
15 # | |
16 # Use a descriptive name for your cluster: | |
17 # | |
18 cluster.name: tksearch | |
19 # | |
20 # ------------------------------------ Node ------------------------------------ | |
21 # | |
22 # Use a descriptive name for the node: | |
23 # | |
24 node.name: node2 | |
25 # | |
26 # Add custom attributes to the node: | |
27 # | |
28 # node.rack: r1 | |
29 # | |
30 node.master: false | |
31 # ----------------------------------- Paths ------------------------------------ | |
32 # | |
33 # Path to directory where to store the data (separate multiple locations by comma): | |
34 # | |
35 path.data: /home/AppData/es/esins2/data | |
36 # | |
37 # Path to log files: | |
38 # | |
39 path.logs: /home/AppData/es/esins2/logs | |
40 # | |
41 # ----------------------------------- Memory ----------------------------------- | |
42 # | |
43 # Lock the memory on startup: | |
44 # | |
45 # bootstrap.mlockall: true | |
46 # | |
47 # Make sure that the `ES_HEAP_SIZE` environment variable is set to about half the memory | |
48 # available on the system and that the owner of the process is allowed to use this limit. | |
49 # | |
50 # Elasticsearch performs poorly when the system is swapping the memory. | |
51 # | |
52 # ---------------------------------- Network ----------------------------------- | |
53 # | |
54 # Set the bind address to a specific IP (IPv4 or IPv6): | |
55 # | |
56 network.host: 0.0.0.0 | |
57 # | |
58 # Set a custom port for HTTP: | |
59 # | |
60 http.port: 9201 | |
61 # | |
62 # For more information, see the documentation at: | |
63 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html> | |
64 # | |
65 transport.tcp.port: 9301 | |
66 # --------------------------------- Discovery ---------------------------------- | |
67 # | |
68 # Pass an initial list of hosts to perform discovery when new node is started: | |
69 # The default list of hosts is ["127.0.0.1", "[::1]"] | |
70 # | |
71 discovery.zen.ping.unicast.hosts: ["127.0.0.1:9300"] | |
72 # | |
73 # Prevent the "split brain" by configuring the majority of nodes (total number of nodes / 2 + 1): | |
74 # | |
75 # discovery.zen.minimum_master_nodes: 3 | |
76 # | |
77 # For more information, see the documentation at: | |
78 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery.html> | |
79 # | |
80 # ---------------------------------- Gateway ----------------------------------- | |
81 # | |
82 # Block initial recovery after a full cluster restart until N nodes are started: | |
83 # | |
84 # gateway.recover_after_nodes: 3 | |
85 # | |
86 # For more information, see the documentation at: | |
87 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-gateway.html> | |
88 # | |
89 # ---------------------------------- Various ----------------------------------- | |
90 # | |
91 # Disable starting multiple nodes on a single system: | |
92 # | |
93 node.max_local_storage_nodes: 2 | |
94 # | |
95 # Require explicit names when deleting indices: | |
96 # | |
97 # action.destructive_requires_name: true |
3. 由于配置中指定了配置文件和数据的路径了,所以,要在相应的路径下创建所需的目录。
1 /home/AppData/es/esins1/data | |
2 /home/AppData/es/esins1/logs | |
3 /home/AppData/es/esins2/data | |
4 /home/AppData/es/esins2/logs |
4. 启动 es,先启动 es 实例 1. 注意,es 启动时指定的配置文件时,要指定配置文件所在的路径,这个路径包含 elasticsearch.yml 以及 logging.yml
1 [root@CloudGame bin]# ./elasticsearch -d -Des.path.conf=/etc/esconf/esins1 -p /etc/esconf/esins1.pid | |
2 [root@CloudGame bin]# Exception in thread "main" Java.lang.RuntimeException: don't run elasticsearch as root. | |
3 at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:93) | |
4 at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:144) | |
5 at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:270) | |
6 at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35) | |
7 Refer to the log for complete error details. |
哟,错了啊, 不能用 root 启动 es,见鬼,再来一次吧。
1 [water@CloudGame bin]$ ./elasticsearch -d -Des.path.conf=/etc/esconf/esins1 -p /etc/esconf/esins1.pid | |
2 [water@CloudGame bin]$ Exception in thread "main" SettingsException[Failed to open stream for url [/etc/esconf/esins1/elasticsearch.yml]]; nested: AccessDeniedException[/etc/esconf/esins1/elasticsearch.yml]; | |
3 Likely root cause: java.nio.file.AccessDeniedException: /etc/esconf/esins1/elasticsearch.yml | |
4 at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) | |
5 at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) | |
6 at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) | |
7 at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) | |
8 at java.nio.file.Files.newByteChannel(Files.java:317) | |
9 at java.nio.file.Files.newByteChannel(Files.java:363) | |
10 at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:380) | |
11 at java.nio.file.Files.newInputStream(Files.java:108) | |
12 at org.elasticsearch.common.settings.Settings$Builder.loadFromPath(Settings.java:1067) | |
13 at org.elasticsearch.node.internal.InternalSettingsPreparer.prepareEnvironment(InternalSettingsPreparer.java:88) | |
14 at org.elasticsearch.bootstrap.Bootstrap.initialSettings(Bootstrap.java:202) | |
15 at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241) | |
16 at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35) | |
17 Refer to the log for complete error details. |
哟嘿,还是错???访问 etc/esconf/esins1/elasticsearch.yml 被阻止了???想起来了,这个文件是在 root 用户下创建配置的。修改一下用户组吧: chown -R water:water /etc/esconf/esins1。再来一次。。。
1 [water@CloudGame bin]$ ./elasticsearch -d -Des.path.conf=/etc/esconf/esins1 -p /etc/esconf/esins1.pid | |
2 [water@CloudGame bin]$ Exception in thread "main" SettingsException[Failed to open stream for url [/etc/esconf/esins1/elasticsearch.yml]]; nested: AccessDeniedException[/etc/esconf/esins1/elasticsearch.yml]; | |
3 Likely root cause: java.nio.file.AccessDeniedException: /etc/esconf/esins1/elasticsearch.yml | |
4 at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) | |
5 at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) | |
6 at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) | |
7 at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) | |
8 at java.nio.file.Files.newByteChannel(Files.java:317) | |
9 at java.nio.file.Files.newByteChannel(Files.java:363) | |
10 at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:380) | |
11 at java.nio.file.Files.newInputStream(Files.java:108) | |
12 at org.elasticsearch.common.settings.Settings$Builder.loadFromPath(Settings.java:1067) | |
13 at org.elasticsearch.node.internal.InternalSettingsPreparer.prepareEnvironment(InternalSettingsPreparer.java:88) | |
14 at org.elasticsearch.bootstrap.Bootstrap.initialSettings(Bootstrap.java:202) | |
15 at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241) | |
16 at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35) | |
17 Refer to the log for complete error details. |
怎么还是一样的错啊,去看看日志吧,靠,/home/AppData/es/esins1/logs 下面什么也没有啊???????倒,这个目录也是在 root 权限下创建的,water 没有写的权限。。。。。。再给这数据和日志目录改属组。。。。。
1 [root@CloudGame es]# ll | |
2 total 8 | |
3 drwxr-xr-x. 4 root root 4096 Oct 18 11:20 esins1 | |
4 drwxr-xr-x. 4 root root 4096 Oct 18 11:21 esins2 | |
5 [root@CloudGame es]# chown -R water:water * | |
6 [root@CloudGame es]# ll | |
7 total 8 | |
8 drwxr-xr-x. 4 water water 4096 Oct 18 11:20 esins1 | |
9 drwxr-xr-x. 4 water water 4096 Oct 18 11:21 esins2 | |
10 [root@CloudGame es]# cd esins1/ | |
11 [root@CloudGame esins1]# ll | |
12 total 8 | |
13 drwxr-xr-x. 2 water water 4096 Oct 18 11:20 data | |
14 drwxr-xr-x. 2 water water 4096 Oct 18 11:49 logs |
再来启动一次,应该可以了吧。。。。
1 [water@CloudGame bin]$ ./elasticsearch -d -Des.path.conf=/etc/esconf/esins1 -p /etc/esconf/esins1.pid | |
2 [water@CloudGame bin]$ Exception in thread "main" java.nio.file.AccessDeniedException: /etc/esconf/esins1.pid | |
3 at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) | |
4 at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) | |
5 at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) | |
6 at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) | |
7 at java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:430) | |
8 at java.nio.file.Files.newOutputStream(Files.java:172) | |
9 at org.elasticsearch.common.PidFile.create(PidFile.java:76) | |
10 at org.elasticsearch.common.PidFile.create(PidFile.java:55) | |
11 at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:247) | |
12 at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35) | |
13 Refer to the log for complete error details. |
去你的,怎么还是权限的问题,来 pid 文件没有办法写入。。。。。再搞一次权限配置。。。。
1 [root@CloudGame etc]# chown -R water:water esconf
再启动,再不好,是不是就不玩了。。。。
1 [water@CloudGame bin]$ ./elasticsearch -d -Des.path.conf=/etc/esconf/esins1 -p /etc/esconf/esins1.pid | |
2 [water@CloudGame bin]$ |
哈哈哈,很好,这次启动不错了,说明八九不离十了。
同样的,参照启动 esins1 的过程,启动 esins2. 一切都 ok。esconf 下面有两个 pid 文件
1 [root@CloudGame esconf]# ll -al | |
2 total 32 | |
3 drwxr-xr-x. 4 water water 4096 Oct 18 12:00 . | |
4 drwxr-xr-x. 126 root root 12288 Oct 18 13:25 .. | |
5 drwxr-xr-x. 3 water water 4096 Oct 18 11:38 esins1 | |
6 -rw-rw-r--. 1 water water 5 Oct 18 11:55 esins1.pid | |
7 drwxr-xr-x. 3 water water 4096 Oct 18 11:39 esins2 | |
8 -rw-rw-r--. 1 water water 5 Oct 18 12:00 esins2.pid |
到此,单机双实例的启动完成。
真的完成了么?要不访问看看,那就验证一下吧,下面来几张截图 demo 一下最后的结果!
esins1 的基本信息:
esins2 的基本信息:
esins1 的健康状态:
esins2 的健康状态:
ElasticSearch 最新版本 2.20 发布下载了 http://www.linuxidc.com/Linux/2016-02/128166.htm
Linux 上安装部署 ElasticSearch 全程记录 http://www.linuxidc.com/Linux/2015-09/123241.htm
Elasticsearch 安装使用教程 http://www.linuxidc.com/Linux/2015-02/113615.htm
ElasticSearch 配置文件译文解析 http://www.linuxidc.com/Linux/2015-02/114244.htm
ElasticSearch 集群搭建实例 http://www.linuxidc.com/Linux/2015-02/114243.htm
分布式搜索 ElasticSearch 单机与服务器环境搭建 http://www.linuxidc.com/Linux/2012-05/60787.htm
ElasticSearch 的工作机制 http://www.linuxidc.com/Linux/2014-11/109922.htm
Elasticsearch 的安装,运行和基本配置 http://www.linuxidc.com/Linux/2016-07/133057.htm
使用 Elasticsearch + Logstash + Kibana 搭建日志集中分析平台实践 http://www.linuxidc.com/Linux/2015-12/126587.htm
Ubuntu 14.04 搭建 ELK 日志分析系统 (Elasticsearch+Logstash+Kibana) http://www.linuxidc.com/Linux/2016-06/132618.htm
Elasticsearch1.7 升级到 2.3 实践总结 http://www.linuxidc.com/Linux/2016-11/137282.htm
ElasticSearch 的详细介绍 :请点这里
ElasticSearch 的下载地址 :请点这里
本文永久更新链接地址 :http://www.linuxidc.com/Linux/2016-11/137359.htm