Openstack对接Ceph时的错误

228次阅读

共计 3984 个字符，预计需要花费 10 分钟才能阅读完成。

本次 Openstack 版本为 N 版 ceph 版本为 J 版在对接 volume 和 glance 时非常轻松不过在对接 nova 的时候坑了一下午。

ERROR nova.compute.manager [instance: b6b53548-2d12-4019-a189-1d006455dfa3] 2017-08-07T14:09:55.209454Z qemu-system-x86_64: -drive file=rbd:vms/b6b53548-2d12-4019-a189-1d006455dfa3_disk:id=cinder:auth_supported=cephx\;none:mon_host=192.168.0.151\:6789,file.password-secret=virtio-disk0-secret0,format=raw,if=none,id=drive-virtio-disk0,cache=none: Unknown protocol ‘rbd’

意思是说不支持 rbd 协议于是乎查看自己的 qemu。
[root@compute01 ~]# qemu-img –help | grep rbd
[root@compute01 ~]#

注此时我已经装了 ceph 相关的包现在已经有了 rbd 的命令内核已经也已经有了 rbd 现在是 qemu 不支持 rbd 协议接下来继续证明这个事情

[root@compute01 ~]# rados lspools
glance
volumes
vms
rbd
images
[root@compute01 ~]# rbd ls volumes
volume-4a898a0e-40db-4846-b944-079891598ae4
[root@compute01 ~]#
[root@compute01 ~]# qemu-img create -f raw rbd:volumes/test 5G
qemu-img: rbd:volumes/test: Unknown protocol ‘rbd’
[root@compute01 ~]#

可以看到直接让 qemu 使用 ceph 都不行报不支持 rbd 协议此时我回想自己编译 qemu 时是默认选项查了好多文档默认是不支持 rbd 的所以要重新编译此次开启 rbd
[root@compute01 qemu-2.9.0]# ./configure –enable-rbd
No C++ compiler available; disabling C++ specific optional code

ERROR: User requested feature rados block device
configure was not able to find it.
Install librbd/ceph devel

报 No C++ 我们来安装包
[root@compute01 qemu-2.9.0]# yum install gcc gcc-c++
Installed:
gcc-c++.x86_64 0:4.8.5-11.el7

Dependency Installed:
libstdc++-devel.x86_64 0:4.8.5-11.el7

Complete!

再次编译
[root@compute01 qemu-2.9.0]# ./configure –enable-rbd

ERROR: User requested feature rados block device
configure was not able to find it.
Install librbd/ceph devel

报找不到 ceph 的 librdb 我们继续装包
[root@compute01 qemu-2.9.0]# yum install ceph-devel
Loaded plugins: fastestmirror, priorities
Loading mirror speeds from cached hostfile
Resolving Dependencies
–> Running transaction check
—> Package ceph-devel-compat.x86_64 1:10.2.9-0.el7 will be installed
Installed:
ceph-devel-compat.x86_64 1:10.2.9-0.el7

Dependency Installed:
libcephfs1-devel.x86_64 1:10.2.9-0.el7 libcephfs_jni1.x86_64 1:10.2.9-0.el7
libcephfs_jni1-devel.x86_64 1:10.2.9-0.el7 librados2-devel.x86_64 1:10.2.9-0.el7
libradosstriper1-devel.x86_64 1:10.2.9-0.el7 librbd1-devel.x86_64 1:10.2.9-0.el7

Complete!

再次编译
[root@compute01 qemu-2.9.0]# ./configure –enable-rbd
Install prefix /usr/local
BIOS directory /usr/local/share/qemu
binary directory /usr/local/bin
library directory /usr/local/lib
module directory /usr/local/lib/qemu
libexec directory /usr/local/libexec
include directory /usr/local/include
config directory /usr/local/etc
local state directory /usr/local/var
Manual directory /usr/local/share/man
ELF interp prefix /usr/gnemul/qemu-%M

rbd support yes ### 重点来了
NUMA host support no
tcmalloc support no
jemalloc support no
avx2 optimization yes
replication support yes

[root@compute01 qemu-2.9.0]# make && make install

大概等了 3 个小时终于编译完成！
install -c -m 0755 qemu-tilegx “/usr/local/bin”
strip “/usr/local/bin/qemu-tilegx”
install -d -m 0755 “/usr/local/bin”
install -c -m 0755 qemu-x86_64 “/usr/local/bin”
strip “/usr/local/bin/qemu-x86_64”
[root@compute01 qemu-2.9.0]#

再次查看是否支持 rbd
[root@compute01 ~]# qemu-img -h|grep rbd
Supported formats: blkdebug blkreplay blkverify bochs cloop dmg file host_cdrom host_device luks nbd null-aio null-co parallels qcow qcow2 qed quorum raw rbd replication sheepdog vdi vhdx vmdk vpc vvfat
[root@compute01 ~]#

qemu 已经支持 rbd 了！此次任务完成！

更多 Ceph 相关教程见以下内容：

在 CentOS 7.1 上安装分布式存储系统 Ceph http://www.linuxidc.com/Linux/2015-08/120990.htm

Ceph 环境配置文档 PDF http://www.linuxidc.com/Linux/2013-05/85212.htm

CentOS7 下部署 Ceph 集群（版本 10.2.2）http://www.linuxidc.com/Linux/2017-02/140728.htm

Ceph 的安装过程 http://www.linuxidc.com/Linux/2013-05/85210.htm

如何升级 Ceph 版本及注意事项 http://www.linuxidc.com/Linux/2017-02/140631.htm

HOWTO Install Ceph On FC12, FC 上安装 Ceph 分布式文件系统 http://www.linuxidc.com/Linux/2013-05/85209.htm

实验环境 Ceph 9.2.1 部署笔记 http://www.linuxidc.com/Linux/2016-11/137094.htm

Ubuntu 16.04 快速安装 Ceph 集群 http://www.linuxidc.com/Linux/2016-09/135261.htm

CentOS 7 下单台物理机搭建 Ceph 集群 http://www.linuxidc.com/Linux/2017-06/144636.htm

Ceph 的详细介绍：请点这里
Ceph 的下载地址：请点这里