MongoDB 3.0.2与wiredTiger存储引擎安装测试

193次阅读

共计 16863 个字符，预计需要花费 43 分钟才能阅读完成。

MongoDB 3.0 版本，除了之前默认的 MMAP 存储引擎，还新推出了 wiredTiger 存储引擎。据官网测试报告称，mongodb 3.0 版本使用 wiredTiger 存储引擎后，写性能能够提高 7 -10 倍，数据压缩 30%-80%，运维降低 95%，这三个方面都还是很有吸引力的。

为了进行 mongodb 2.6 和 mongodb 3.0 的性能，先搭建一个 mongodb 3.0 环境，然后用 YCSB 工具进行测试。

mongodb 3.0.2 与 wiredTiger 的安装步骤如下：

1. 下载和安装 mongodb 3.0 软件

从 mongodb 官网下载 mongodb 3.0 的安装介质 mongodb-linux-x86_64-rhel62-3.0.2.tgz，然后登陆的服务器上执行下面的命令安装：

cd /usr/local/

rz mongodb-linux-x86_64-rhel62-3.0.2.tgz

tar -zxvf mongodb-linux-x86_64-rhel62-3.0.2.tgz

ln -s mongodb-linux-x86_64-rhel62-3.0.2 mongodb30

chown -R mongodb:mongodb mongodb-linux-x86_64-rhel62-3.0.2 mongodb30

2. 准备 mongodb 实例目录和配置文件

创建 mongodb 实例所需目录：

rm -fR /data/mongodb30/

mkdir -p /data/mongodb30/{db,logs}

touch /data/mongodb30/logs/mongodb.log

chown -R mongodb:mongodb /data/mongodb30/

创建配置文件，配置启动参数，尤其注意添加 wriedTiger 存储引擎相关参数：

vim /data/mongodb30/mongodb30.conf

storageEngine = wiredTiger
wiredTigerCacheSizeGB = 2
syncdelay = 30
wiredTigerCollectionBlockCompressor = snappy
port=38019
dbpath=/data/mongodb30/db
oplogSize=2048
logpath=/data/mongodb30/logs/mongodb.log
logappend=true
fork=true
rest=true
journal = true

上面的参数中，标红的参数是 wriedTiger 相关参数，storageEngine 是设置存储引擎；wiredTigerCacheSizeGB 是设置 mongodb 存储引擎所用的内容，默认为系统内存的 50%；syncdelay 是设置从内存同步到硬盘的时间间隔，默认为 60 秒，可以设置的少一些，在 mongodb 故障时，丢失的日志会少一些；wiredTigerCollectionBlockCompressor 是设定压缩策略 snappy 是一种压缩速度非常快的压缩策略。

由此可见，在 mongodb 3.0 中想要启用 wriedTiger 存储引擎，只需要增加该引擎的几个参数即可。

注：mongodb 实例的启动，可以用命令行启动，也可以用配置文件启动；为了方便操作，并且可以保存启动的的参数，建议使用配置文件的方式启动 mongodb 实例。

修改实例目录和文件的属主为 mongodb：

chown -R mongodb:mongodb /data/mongodb30/

3. 启动 mongodb 实例，并登录测试

使用 mongod 命令，以配置文件的方式启动 mongodb 实例：

/usr/local/mongodb30/bin/mongod –config /data/mongodb30/mongodb30.conf

确认启动后的进程状态：

ps -ef | grep mongo

使用 mongo 命令登录到 mongodb 实例中：

/usr/local/mongodb30/bin/mongo –port=38019

实际操作过程记录如下：

[root@test07 local]#

[root@test07 local]# rm -fR /data/mongodb30/
[root@test07 local]# mkdir -p /data/mongodb30/{db,logs}
[root@test07 local]# touch /data/mongodb30/logs/mongodb.log
[root@test07 local]# chown -R mongodb:mongodb /data/mongodb30/
[root@test07 local]#
[root@test07 local]# vim /data/mongodb30/mongodb30.conf
[root@test07 local]#
[root@test07 local]#
[root@test07 local]# cat /data/mongodb30/mongodb30.conf
storageEngine = wiredTiger
wiredTigerCacheSizeGB = 2
syncdelay = 30
wiredTigerCollectionBlockCompressor = Snappy
port=38019
dbpath=/data/mongodb30/db
oplogSize=2048
logpath=/data/mongodb30/logs/mongodb.log
logappend=true
fork=true
rest=true
journal = true
[root@test07 local]#
[root@test07 local]# chown -R mongodb:mongodb /data/mongodb30/
[root@test07 local]#
[root@test07 local]# su – mongodb
[mongodb@test07 ~]$
[mongodb@test07 ~]$ /usr/local/mongodb30/bin/mongod –config /data/mongodb30/mongodb30.conf
2015-10-27T11:06:12.889+0800 F CONTROL Failed global initialization: BadValue storage.wiredTiger.collectionConfig.blockCompressor must be a string of the format: (none/snappy/zlib)
[mongodb@test07 ~]$ sed -i ‘s/Snappy/snappy/g’ /data/mongodb30/mongodb30.conf
[mongodb@test07 ~]$
[mongodb@test07 ~]$ /usr/local/mongodb30/bin/mongod –config /data/mongodb30/mongodb30.conf
2015-10-27T11:07:04.640+0800 I CONTROL ** WARNING: –rest is specified without –httpinterface,
2015-10-27T11:07:04.640+0800 I CONTROL ** enabling http interface
about to fork child process, waiting until server is ready for connections.
forked process: 3098
child process started successfully, parent exiting
[mongodb@test07 ~]$
[mongodb@test07 ~]$ ps -ef | grep mysql
mongodb 3344 3063 0 11:42 pts/0 00:00:00 grep mysql
[mongodb@test07 ~]$
[mongodb@test07 ~]$
[mongodb@test07 ~]$ ps -ef|grep mongodb
mongodb 2953 1 0 11:00 ? 00:00:16 /usr/local/mongodb26/bin/mongod –config /data/mongodb26/mongodb26.conf
root 3062 2802 0 11:05 pts/0 00:00:00 su – mongodb
mongodb 3063 3062 0 11:05 pts/0 00:00:00 -bash
mongodb 3098 1 0 11:07 ? 00:00:13 /usr/local/mongodb30/bin/mongod –config /data/mongodb30/mongodb30.conf
mongodb 3346 3063 0 11:42 pts/0 00:00:00 ps -ef
mongodb 3347 3063 0 11:42 pts/0 00:00:00 grep mongodb
[mongodb@test07 ~]$
[mongodb@test07 ~]$
[mongodb@test07 ~]$ /usr/local/mongodb30/bin/mongo –port=38019
MongoDB shell version: 3.0.2
connecting to: 127.0.0.1:38019/test
Server has startup warnings:
2015-10-27T11:07:04.640+0800 I CONTROL ** WARNING: –rest is specified without –httpinterface,
2015-10-27T11:07:04.640+0800 I CONTROL ** enabling http interface
>
> show dbs;
local 0.000GB
> use admin;
switched to db admin
> show collections;
>
>
> db.serverStatus()
{
“host” : “test07:38019”,
“version” : “3.0.2”,
“process” : “mongod”,
“pid” : NumberLong(3098),
“uptime” : 2697,
“uptimeMillis” : NumberLong(2697265),
“uptimeEstimate” : 2457,
“localTime” : ISODate(“2015-10-27T03:52:01.900Z”),
“asserts” : {
“regular” : 0,
“warning” : 0,
“msg” : 0,
“user” : 0,
“rollovers” : 0
},
“connections” : {
“current” : 1,
“available” : 818,
“totalCreated” : NumberLong(2)
},
“cursors” : {
“note” : “deprecated, use server status metrics”,
“clientCursors_size” : 0,
“totalOpen” : 0,
“pinned” : 0,
“totalNoTimeout” : 0,
“timedOut” : 0
},
“extra_info” : {
“note” : “fields vary by platform”,
“heap_usage_bytes” : 31212656,
“page_faults” : 0
},
“globalLock” : {
“totalTime” : NumberLong(“2697266000”),
“currentQueue” : {
“total” : 0,
“readers” : 0,
“writers” : 0
},
“activeClients” : {
“total” : 8,
“readers” : 0,
“writers” : 0
}
},
“locks” : {
“Global” : {
“acquireCount” : {
“r” : NumberLong(724),
“w” : NumberLong(2),
“W” : NumberLong(4)
}
},
“Database” : {
“acquireCount” : {
“r” : NumberLong(723),
“R” : NumberLong(1),
“W” : NumberLong(2)
}
},
“Collection” : {
“acquireCount” : {
“r” : NumberLong(722)
}
}
},
“network” : {
“bytesIn” : 1371,
“bytesOut” : 3116,
“numRequests” : 18
},
“opcounters” : {
“insert” : 0,
“query” : 1,
“update” : 0,
“delete” : 0,
“getmore” : 0,
“command” : 19
},
“opcountersRepl” : {
“insert” : 0,
“query” : 0,
“update” : 0,
“delete” : 0,
“getmore” : 0,
“command” : 0
},
“storageEngine” : {
“name” : “wiredTiger”
},
“wiredTiger” : {
“uri” : “statistics:”,
“LSM” : {
“sleep for LSM checkpoint throttle” : 0,
“sleep for LSM merge throttle” : 0,
“rows merged in an LSM tree” : 0,
“application work units currently queued” : 0,
“merge work units currently queued” : 0,
“tree queue hit maximum” : 0,
“switch work units currently queued” : 0,
“tree maintenance operations scheduled” : 0,
“tree maintenance operations discarded” : 0,
“tree maintenance operations executed” : 0
},
“async” : {
“number of allocation state races” : 0,
“number of operation slots viewed for allocation” : 0,
“current work queue length” : 0,
“number of flush calls” : 0,
“number of times operation allocation failed” : 0,
“maximum work queue length” : 0,
“number of times worker found no work” : 0,
“total allocations” : 0,
“total compact calls” : 0,
“total insert calls” : 0,
“total remove calls” : 0,
“total search calls” : 0,
“total update calls” : 0
},
“block-manager” : {
“mapped bytes read” : 0,
“bytes read” : 12288,
“bytes written” : 126976,
“mapped blocks read” : 0,
“blocks pre-loaded” : 0,
“blocks read” : 3,
“blocks written” : 27
},
“cache” : {
“tracked dirty bytes in the cache” : 0,
“tracked bytes belonging to internal pages in the cache” : 1245,
“bytes currently in the cache” : 14644,
“tracked bytes belonging to leaf pages in the cache” : 2147482403,
“maximum bytes configured” : 2147483648,
“tracked bytes belonging to overflow pages in the cache” : 0,
“bytes read into cache” : 0,
“bytes written from cache” : 20178,
“pages evicted by application threads” : 0,
“checkpoint blocked page eviction” : 0,
“unmodified pages evicted” : 0,
“page split during eviction deepened the tree” : 0,
“modified pages evicted” : 0,
“pages selected for eviction unable to be evicted” : 0,
“pages evicted because they exceeded the in-memory maximum” : 0,
“pages evicted because they had chains of deleted items” : 0,
“failed eviction of pages that exceeded the in-memory maximum” : 0,
“hazard pointer blocked page eviction” : 0,
“internal pages evicted” : 0,
“maximum page size at eviction” : 0,
“eviction server candidate queue empty when topping up” : 0,
“eviction server candidate queue not empty when topping up” : 0,
“eviction server evicting pages” : 0,
“eviction server populating queue, but not evicting pages” : 0,
“eviction server unable to reach eviction goal” : 0,
“pages split during eviction” : 0,
“pages walked for eviction” : 0,
“eviction worker thread evicting pages” : 0,
“in-memory page splits” : 0,
“percentage overhead” : 8,
“tracked dirty pages in the cache” : 0,
“pages currently held in the cache” : 10,
“pages read into cache” : 0,
“pages written from cache” : 16
},
“connection” : {
“pthread mutex condition wait calls” : 59275,
“files currently open” : 7,
“memory allocations” : 21765,
“memory frees” : 19667,
“memory re-allocations” : 64,
“total read I/Os” : 8,
“pthread mutex shared lock read-lock calls” : 1336,
“pthread mutex shared lock write-lock calls” : 2958,
“total write I/Os” : 141
},
“cursor” : {
“cursor create calls” : 568,
“cursor insert calls” : 24,
“cursor next calls” : 12,
“cursor prev calls” : 3,
“cursor remove calls” : 0,
“cursor reset calls” : 119,
“cursor search calls” : 604,
“cursor search near calls” : 3,
“cursor update calls” : 0
},
“data-handle” : {
“connection dhandles swept” : 0,
“connection candidate referenced” : 0,
“connection sweeps” : 269,
“connection time-of-death sets” : 1,
“session dhandles swept” : 0,
“session sweep attempts” : 56
},
“log” : {
“log buffer size increases” : 0,
“total log buffer size” : 1048576,
“log bytes of payload data” : 8051,
“log bytes written” : 18560,
“yields waiting for previous log file close” : 0,
“total size of compressed records” : 5113,
“total in-memory size of compressed records” : 9524,
“log records too small to compress” : 91,
“log records not compressed” : 5,
“log records compressed” : 6,
“maximum log file size” : 104857600,
“pre-allocated log files prepared” : 1,
“number of pre-allocated log files to create” : 1,
“pre-allocated log files used” : 0,
“log read operations” : 0,
“log release advances write LSN” : 102,
“records processed by log scan” : 0,
“log scan records requiring two reads” : 0,
“log scan operations” : 0,
“consolidated slot closures” : 0,
“logging bytes consolidated” : 0,
“consolidated slot joins” : 0,
“consolidated slot join races” : 0,
“slots selected for switching that were unavailable” : 0,
“record size exceeded maximum” : 0,
“failed to find a slot large enough for record” : 0,
“consolidated slot join transitions” : 0,
“log sync operations” : 92,
“log sync_dir operations” : 1,
“log server thread advances write LSN” : 0,
“log write operations” : 102
},
“reconciliation” : {
“page reconciliation calls” : 16,
“page reconciliation calls for eviction” : 0,
“split bytes currently awaiting free” : 0,
“split objects currently awaiting free” : 0
},
“session” : {
“open cursor count” : 54,
“open session count” : 59
},
“thread-yield” : {
“page acquire busy blocked” : 0,
“page acquire eviction blocked” : 0,
“page acquire locked blocked” : 0,
“page acquire read blocked” : 0,
“page acquire time sleeping (usecs)” : 0
},
“transaction” : {
“transaction begins” : 50,
“transaction checkpoints” : 89,
“transaction checkpoint generation” : 89,
“transaction checkpoint currently running” : 0,
“transaction checkpoint max time (msecs)” : 16,
“transaction checkpoint min time (msecs)” : 0,
“transaction checkpoint most recent time (msecs)” : 0,
“transaction checkpoint total time (msecs)” : 52,
“transactions committed” : 3,
“transaction failures due to cache overflow” : 0,
“transaction range of IDs currently pinned by a checkpoint” : 0,
“transaction range of IDs currently pinned” : 1,
“transactions rolled back” : 46
},
“concurrentTransactions” : {
“write” : {
“out” : 0,
“available” : 128,
“totalTickets” : 128
},
“read” : {
“out” : 1,
“available” : 127,
“totalTickets” : 128
}
}
},
“writeBacksQueued” : false,
“mem” : {
“bits” : 64,
“resident” : 37,
“virtual” : 339,
“supported” : true,
“mapped” : 0,
“mappedWithJournal” : 0
},
“metrics” : {
“commands” : {
“getLog” : {
“failed” : NumberLong(0),
“total” : NumberLong(2)
},
“isMaster” : {
“failed” : NumberLong(0),
“total” : NumberLong(10)
},
“listCollections” : {
“failed” : NumberLong(0),
“total” : NumberLong(1)
},
“listDatabases” : {
“failed” : NumberLong(0),
“total” : NumberLong(1)
},
“replSetGetStatus” : {
“failed” : NumberLong(2),
“total” : NumberLong(2)
},
“serverStatus” : {
“failed” : NumberLong(0),
“total” : NumberLong(1)
},
“whatsmyuri” : {
“failed” : NumberLong(0),
“total” : NumberLong(2)
}
},
“cursor” : {
“timedOut” : NumberLong(0),
“open” : {
“noTimeout” : NumberLong(0),
“pinned” : NumberLong(0),
“total” : NumberLong(0)
}
},
“document” : {
“deleted” : NumberLong(0),
“inserted” : NumberLong(0),
“returned” : NumberLong(0),
“updated” : NumberLong(0)
},
“getLastError” : {
“wtime” : {
“num” : 0,
“totalMillis” : 0
},
“wtimeouts” : NumberLong(0)
},
“operation” : {
“fastmod” : NumberLong(0),
“idhack” : NumberLong(0),
“scanAndOrder” : NumberLong(0),
“writeConflicts” : NumberLong(0)
},
“queryExecutor” : {
“scanned” : NumberLong(0),
“scannedObjects” : NumberLong(0)
},
“record” : {
“moves” : NumberLong(0)
},
“repl” : {
“apply” : {
“batches” : {
“num” : 0,
“totalMillis” : 0
},
“ops” : NumberLong(0)
},
“buffer” : {
“count” : NumberLong(0),
“maxSizeBytes” : 268435456,
“sizeBytes” : NumberLong(0)
},
“network” : {
“bytes” : NumberLong(0),
“getmores” : {
“num” : 0,
“totalMillis” : 0
},
“ops” : NumberLong(0),
“readersCreated” : NumberLong(0)
},
“preload” : {
“docs” : {
“num” : 0,
“totalMillis” : 0
},
“indexes” : {
“num” : 0,
“totalMillis” : 0
}
}
},
“storage” : {
“freelist” : {
“search” : {
“bucketExhausted” : NumberLong(0),
“requests” : NumberLong(0),
“scanned” : NumberLong(0)
}
}
},
“ttl” : {
“deletedDocuments” : NumberLong(0),
“passes” : NumberLong(44)
}
},
“ok” : 1
}
>