Production ready management for File, Block and Object Storage

For production environments, we suggest to install croit in Docker on a dedicated server or VM. croit is well tested with Debian Linux but works on any other OS with a current version of Docker. This host will be refered to as the management node within this guide.

General support for:

• Monitors
• OSDs
• MDSs
• RGW

More details:

• Authentication (cephx), this can be disabled.
• Supports cluster public and private network.
• Monitors deployment. You can easily start with one monitor and then progressively add new nodes.
  So can deploy one monitor for testing purpose. For production, I recommend to always use an odd
  number of monitors, 3 tends to be the standard.
• Object Storage Daemons. Like the monitors you can start with a certain amount of nodes and then
  grow this number. The playbook either supports a dedicated device for storing the journal or both
  journal and OSD data on the same device (using a tiny partition at the beginning of the device).
• Metadata daemons.
• Collocation. The playbook supports collocating Monitors, OSDs and MDSs on the same machine.
• The playbook was validated on Debian Wheezy, Ubuntu 12.04 LTS and CentOS 6.4.
• Tested on Ceph Dumpling and Emperor.
• A rolling upgrade playbook was written, an upgrade from Dumpling to Emperor was performed and worked.

To give you a little bit of background when the OSD writes into his journal it uses D_SYNC and O_DIRECT. Writing with O_DIRECT bypasses the Kernel page cache, while D_SYNC ensures that the command won’t return until every single write is complete. So yes, basically the OSD forces all the writes to be flushed prior to start the next IO.

A simple exemple to make Replication for RBD.

Based on this post from scuttlemonkey : http://ceph.com/dev-notes/incremental-snapshots-with-rbd/, here is a sample script to synchronize rbd image on a remote cluster (eg for backups). In the example below, the sync is made to an “archive” pool on the same cluster. (For remote host, you need to use ssh key.)

Some notes about Ceph

The aim is showing that it is possible to create a low-cost storage, efficient and scalable, using opensource solutions. In the example below, I am using Ceph for scalability and reliability, combined with EnhanceIO to ensure very good performance.

This week I’m testing a lot with CephFS and one of the things I never tried was re-exporting CephFS using NFS-Ganesha and libcephfs.

NFS-Ganesha is a NFS server which runs in userspace. It has multiple backends (FSALs) it can use and libcephfs is one of them.