Manila

Owned by Chris Green
Last updated: Apr 24, 2024 by david.fairbrother@stfc.ac.ukVersion comment
66 min read

Manila: Shared File Systems as a Service

OpenStack Manila provides services for managing shared file systems and is the third storage OpenStack component alongside Cinder and Swift. Manila provides a way to provision remote, sharable file systems (shares) which can be accessed by multiple instances simultaneously and file systems can have access rules assigned to control read-write and read-only access to the file system. On STFC Cloud, Manila provides a way to create Ceph file sytems (CephFS).

Manila Commands

In OpenStack Train, the Manila CLI is mainly used for handling shares. There are some limited commands in the OpenStack CLI for shares.

 

To use Manila commands in OpenStack Train, the compatible version of the python package python-manilaclient needs to be installed. For OpenStack Train, this is v2.0.0. This can be installed using pip install python-manilaclient=2.0.0.

OpenStack CLI

In the OpenStack CLI the following commands are available:

openstack share --help share create # create a new share share delete # delete a share share list # list all shares share show # view details about a share

Manila CLI

Manila provides a CLI for not only for managing shares, but also for viewing the share types available, managing access rules for a share, and viewing the quota for share in your project. Commands are run using manila <sub-command>. The next section will demonstrate how to do some of the Manila commands.

Usage

View Share Quota

You can view either the overall quota using manila quota-show, or the quota and the total quota used with manila absolute-limits.

$ manila quota-show +-----------------------+----------------------------------+ | Property | Value | +-----------------------+----------------------------------+ | gigabytes | 250 | | id | PROJECT_ID | | share_group_snapshots | 0 | | share_groups | 5 | | share_networks | 0 | | shares | 10 | | snapshot_gigabytes | 0 | | snapshots | 0 | +-----------------------+----------------------------------+
$ manila absolute-limits +----------------------------+-------+ | Name | Value | +----------------------------+-------+ | maxTotalShareGigabytes | 250 | | maxTotalShareNetworks | 0 | | maxTotalShareSnapshots | 0 | | maxTotalShares | 10 | | maxTotalSnapshotGigabytes | 250 | | totalShareGigabytesUsed | 3 | | totalShareNetworksUsed | 0 | | totalShareSnapshotsUsed | 0 | | totalSharesUsed | 3 | | totalSnapshotGigabytesUsed | 0 | +----------------------------+-------+

List Share Types Available

Share types available can be listed using manila share-type list or openstack share list. This will list the share types currently available and supported.

Create a Share

Shares can be created by users using the manila create command. The required arguments are the share protocol and the size of the share in GiB.

Do not set a share to be public. This will make the share visible and accessible to everyone outside of the project.

So if we want to create a new CephFS share of size 1GiB, we can use the following command:

Which outputs:

Update a Share

Once a share has been created in Manila, there are only three properties which can be updated: - Name of the share. - The description for the share. - Change the visibility of the share to public or private.

For example, we can update the name of a share from demo_share to updated_demo_share in the following way:

Then we can see the updated share in the list of shares in the current project:

Extend a Share

The size of a share can be increased using the manila extend command.

For example, if we want to extend a demo share from 1GiBs to 2GiBs, we can do the following:

Viewing the details of the share we can see that the size of the share has been updated.

Shrink a Share

This can only be done through the command line only.

The size of a share can be reduced using the manila shrink command.

Using the example in the previous section, we can reduce the size of a share from 2GiB to 1Gib using:

We can see using manila show updated_demo_share that the size of the share has been updated:

Add Access Rule through CLI

Access rules can be created for a share through the Web UI or on the command line. You can allow either read-write (RW) or read-only (RO) access to the share.

To create a new access rule for a share, e.g. a share named demo_share, we can use the manila access-allow command:

With the arguments required described below:

 

Then we can view the access rules for the share using:

Remove Access Rule through CLI

Access rules can be removed from a share using manila access-deny:

Delete a Share

A share can be deleted by using the manila delete command:

How to access a CephFS Share

There are two methods that can be used to mount a share onto a VM:

 

The following examples assume that the following have been set up:

  • VM (examples use an Ubuntu VM)

  • CephFS Share and have the export path given in the share details copied to use

  • cephx share access rules

Monitor IP addresses

Access a share using the Kernel Client

The kernel client requires the ceph-common package to be installed. To access the filesystem on a share, we can use the mount command of the form:

Note: This is run as a single command, it has been broken down for readability in the documentation

The directory will be owned by root by default. To allow another user to read and write add the appropriate user and group like so:

Access a share using the FUSE Client

In order to use the FUSE client, make sure the version of the ceph-fuse package is at least octopus. The version of the package can be found using:

To use the FUSE client, we need to set up a keyring and a ceph.conf file. If we have named the cephx rule created with the name alice, then the keyring file will need to be named alice.keyring and needs to contain the following:

Then for the ceph.conf file, we need to add the IP addresses for the monitors from the ceph cluster. The IP address for the ceph monitors for the share can be found using manila show <share-id> under the export locations field.

Then the filesystem can be mounted to a test directory using the alice cephx access rule:

Where the mount point is the path to the share on the ceph cluster. The share path can be found using manila show <share-id> under the export locations field, and it looks like /volumes/_nogroup/abc123-efd

Using Shares on Kubernetes Clusters

To use shares on Kubernetes clusters we need the following csi installed and deployed onto the cluster:

The manila-csi is designed to be lightweight and only handles the interaction between Kubernetes and OpenStack to create, delete, and access shares. The ceph-csi handles the CephFS side of the filesystem on the Kubernetes cluster. See https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/manila-csi-plugin/developers-csi-manila.md#notes-on-design for more information on how manila-csi is designed.

Deploying onto the Cluster

Both the manila-csi and ceph-csi can be deployed to a Kubernetes cluster using a Helm chart.

ceph-csi

The ceph-csi can be deployed in the following way:

You should then be able to see the following pods in the ceph-csi-cephfs namespace:

manila-csi

Before deploying the Helm chart for manila-csi, we need to create a mannila-csi-values.yaml file in order to deploy the csi to support CephFS shares:

Then we can deploy the chart:

Then you should see pods similar to the following being created onto the cluster (for this example the chart has been deployed into the default namespace):

Next step is to manually create the secret for the manila-csi to use in order to interact with OpenStack:

  • Create an application credential for the associated account ( )

  • Copy this template:

  • Create the secret: kubectl apply -f manila_csi_secret.yaml

Enabling Dynamic Shares using the storage class

Shares can also be created on demand and attached to specific pods.

First, a storage class needs to be created:

Then the storage class can be created from this template using:

Next, a persistent volume claim needs to be created in order to define the size of the share and the access mode.

To create the pvc:

Then, for example, if we want to have an nginx pod spun up with a new share attached we can use the following template:

Then we can create the pod and inspect it:

In the OpenStack project where the application credential was created for, we should now see a new share has been created with a cephx access rule.

Examples: Provisioning Manila Shares onto the Cluster

Static (Manual) Share Provisioning

You will need the ID for the Share as well as the ID for the access rule the PV should use. See share access list for the getting the ID for the access rule.

  • Create a persistent volume claim (PVC) and persistent volume:

Note

You should be able to create the persistent volume claim created successfully here:

Check if it was successful:

Which should return:

To get the information on the existing persistent volume:

Which should return:

List the persistent volumes:

Which returns:

Get information on the pre-provisioned share:

Which returns:

References: