storage

NVMe

Install

sudo apt install -y nvme-cli

CLI

# list devices
nvme list
 
nvme smart-log /dev/nvme0n1
 
isdct show -d DeviceStatus,Index,Firmware,FirmwareUpdateAvailable -intelssd

Fix nvme1: ignoring ctrl due to duplicate subnqn
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1803692

# dmesg | grep nvme
[    2.546620] nvme nvme0: pci function 0000:5e:00.0
[    2.552447] nvme nvme1: pci function 0000:5f:00.0
[    2.768347] nvme nvme1: ignoring ctrl due to duplicate subnqn (nqn.2017-12.org.nvmexpress:uuid:11111111-2222-3333-4444-555555555555).
[    2.775422] nvme nvme1: Removing after probe failure status: -22
[    2.779813]  nvme0n1: p1 p2

Fix by upgrade NVMe firmware
http://www.panticz.de/intel/nvme

Delete

nvme format --ses=1 /dev/nvme1

Namespaces
https://www.linuxjournal.com/content/data-flash-part-ii-using-nvme-drives-and-creating-nvme-over-fabrics-network

s3cmd

Install s3cmd (s3 CLI client)

sudo apt install -y s3cmd

Configuration file

s3cmd --configure
${HOME}/.s3cfg

CLI

s3fs

Install

sudo apt install -y s3fs

Create credentials

echo ${BUCKET}:${ACCESS_KEY}:${SECRET_KEY} >> ~/.passwd-s3fs
chmod 600 ~/.passwd-s3fs

Mount S3 bucket

s3fs bucket1 /tmp/mnt -o url=https://s3.example.com

Parameter

-o passwd_file=~/.passwd-s3fs
-o url=https://s3.example.com,allow_other,umask=0000
-o use_cache=/tmp/cache

/etc/fstab

mybucket1.mydomain.org /mnt/mybucket1 fuse.s3fs _netdev,allow_other,passwd_file=/home/ftpuser/.passwd-aws-s3fs,default_acl=public-read,uid=1001,gid=65534 0 0

Links
https://github.com/s3fs-fuse/s3fs-fuse
https://gridscale.io/community/tutorials/s3-fuse-ubuntu/

CEPH

ceph status
ceph health detail
ceph df detail
ceph osd lspools
osd pool ls detail

Monitoring
https://github.com/ceph/ceph-nagios-plugins/releases

Build nagios-plugins-ceph
sudo apt-get install -y devscripts fakeroot build-essential dh-python
git clone https://github.com/ceph/ceph-nagios-plugins.git
cd ceph-nagios-plugins#
make deb
sudo dpkg -i nagios-plugins-ceph_*_all.deb

# create wrapper for kolla-ansible installation
cat < /usr/bin/ceph
#!/bin/bash
docker exec -it ceph_mon ceph $@
EOF
chmod +x /usr/bin/ceph

DRBD 

# cat /etc/drbd.d/global_common.conf 
global {
		usage-count	yes;
}
 
common {
	startup {
		degr-wfc-timeout	0;
	}
 
	net {
		cram-hmac-alg	sha1;
		shared-secret	****************;
	}
 
	disk {
		on-io-error	detach;
	}
}
 
# cat /etc/drbd.d/r0.res 
resource r0 {
	on scld.sedo.de.intern {
		volume 0 {
			device		/dev/drbd0;
			disk		/dev/vg0/lvol0;
			flexible-meta-disk	internal;
		}
		address		192.168.255.1:7788;
	}
	on ubuntu {
		volume 0 {
			device		/dev/drbd0;
			disk		/dev/sda3;
			flexible-meta-disk	internal;
		}
		address		192.168.255.2:7788;
	}
Subscribe to storage