REX-Ray provides visibility and management of external/underlying storage via guest storage introspection. Available as a Go package, CLI tool, and Linux service, and with built-in third-party support for tools such as Docker, REX-Ray is easily integrated into any workflow. For example, here's how to list storage for a guest hosted on Amazon Web Services (AWS) with REX-Ray:

[0]akutz@pax:~$ export REXRAY_STORAGEDRIVERS=ec2
[0]akutz@pax:~$ export AWS_ACCESS_KEY=access_key
[0]akutz@pax:~$ export AWS_SECRET_KEY=secret_key
[0]akutz@pax:~$ rexray volume get

- providername: ec2
  instanceid: i-695bb6ab
  volumeid: vol-dedbadc3
  devicename: /dev/sda1
  region: us-west-1
  status: attached
- providername: ec2
  instanceid: i-695bb6ab
  volumeid: vol-04c4b219
  devicename: /dev/xvdb
  region: us-west-1
  status: attached

[0]akutz@pax:~$

See the releases area for downloadable binaries.

We have a first release available that support all of the following capabilities!

• Block
  • Cloud infrastructure
    • EC2
    • OpenStack
  • Software-Defined with Kernel Module
    • ScaleIO
  • iSCSI
    • XtremIO (with/without Multipath and Device-Mapper)
• NAS

REX-Ray can be leveraged by Docker (1.7+) as a VolumeDriver. Docker can connect to one or more of these drivers by specifying different --host flags of unix:///run/docker/plugins/name.sock or tcp://127.0.0.1:port when executed. That means getting Docker to leverage REX-Ray is as easy as:

# Start the REX-Ray service. The Docker volume driver is enabled
# by default.
sudo rexray service start

# Create a new container with a volume that leverages the REX-Ray driver
docker run --volume-driver=rexray -v volumename:/pathtomount container

Additionally, the Dogged repo maintains efforts for the EMC {code} team relating to embedding REX-Ray inside of Container Engines such as Docker. Here you will find REX-Ray enabling Docker to manage its own storage via Container Data Volumes.

// TODO

This might currently require upstream additions for the Goamz package to github.com/clintonskitson/goamz at the snapcopy branch.

docker run --rm -it -v $GOPATH:/go -w /go/src/github.com/emccode/rexray/rexray golang:1.5-cross make release

The primary method used to configure the REX-Ray library and influence its behavior is through he use of environment variables.

This section outlines the primary types in the REX-Ray library as well as providing some code examples.

• Volume Driver (Volume Manager Interface Inward)
  • Mount
  • Unmount
  • Path
  • Create
  • Remove
• Storage Driver (External Storage Interface Outward)
  • Get Volume Mappings
  • Get Instance
  • Get Volume
  • Get Volumes Attached
  • Create Snapshot
  • Get Snapshot
  • Remove Snapshot
  • Create Volume
  • Remove Volume
  • Get Next Available Device
  • Attach Volume
  • Detach Volume
  • Copy Snapshot
• OS Driver (Local Management)
  • Get mounts
  • Mounted
  • Mount
  • Unmount
  • Format

These represent the methods that should be available from storage drivers.

type Driver interface {
  // GetVolumeMapping lists the block devices that are attached to the instance.
	GetVolumeMapping() ([]*BlockDevice, error)

	// GetInstance retrieves the local instance.
	GetInstance() (*Instance, error)

	// GetVolume returns all volumes for the instance based on either volumeID or volumeName
	// that are available to the instance.
	GetVolume(volumeID, volumeName string) ([]*Volume, error)

	// GetVolumeAttach returns the attachment details based on volumeID or volumeName
	// where the volume is currently attached.
	GetVolumeAttach(volumeID, instanceID string) ([]*VolumeAttachment, error)

	// CreateSnapshot is a synch/async operation that returns snapshots that have been
	// performed based on supplying a snapshotName, source volumeID, and optional description.
	CreateSnapshot(runAsync bool, snapshotName, volumeID, description string) ([]*Snapshot, error)

	// GetSnapshot returns a list of snapshots for a volume based on volumeID, snapshotID, or snapshotName.
	GetSnapshot(volumeID, snapshotID, snapshotName string) ([]*Snapshot, error)

	// RemoveSnapshot will remove a snapshot based on the snapshotID.
	RemoveSnapshot(snapshotID string) error

	// CreateVolume is sync/async and will create an return a new/existing Volume based on volumeID/snapshotID with
	// a name of volumeName and a size in GB.  Optionally based on the storage driver, a volumeType, IOPS, and availabilityZone
	// could be defined.
	CreateVolume(runAsync bool, volumeName string, volumeID string, snapshotID string, volumeType string, IOPS int64, size int64, availabilityZone string) (*Volume, error)

	// RemoveVolume will remove a volume based on volumeID.
	RemoveVolume(volumeID string) error

	// GetDeviceNextAvailable return a device path that will retrieve the next available disk device that can be used.
	GetDeviceNextAvailable() (string, error)

	// AttachVolume returns a list of VolumeAttachments is sync/async that will attach a volume to an instance based on volumeID and instanceID.
	AttachVolume(runAsync bool, volumeID, instanceID string) ([]*VolumeAttachment, error)

	// DetachVolume is sync/async that will detach the volumeID from the local instance or the instanceID.
	DetachVolume(runAsync bool, volumeID string, instanceID string) error

	// CopySnapshot is a sync/async and returns a snapshot that will copy a snapshot based on volumeID/snapshotID/snapshotName and
	// create a new snapshot of desinationSnapshotName in the destinationRegion location.
	CopySnapshot(runAsync bool, volumeID, snapshotID, snapshotName, destinationSnapshotName, destinationRegion string) (*Snapshot, error)}

These represent the methods that should be available from volume drivers. These drivers allow an abstracted way to access storage drivers.

type Driver interface {
	// Mount will return a mount point path when specifying either a volumeName or volumeID.  If a overwriteFs boolean
	// is specified it will overwrite the FS based on newFsType if it is detected that there is no FS present.
	Mount(volumeName, volumeID string, overwriteFs bool, newFsType string) (string, error)

	// Unmount will unmount the specified volume by volumeName or volumeID.
	Unmount(volumeName, volumeID string) error

	// Path will return the mounted path of the volumeName or volumeID.
	Path(volumeName, volumeID string) (string, error)

	// Create will create a new volume with the volumeName.
	Create(volumeName string) error

	// Remove will remove a volume of volumeName.
	Remove(volumeName string) error

	// Attach will attach a volume based on volumeName to the instance of instanceID.
	Attach(volumeName, instanceID string) (string, error)

	// Detach will detach a volume based on volumeName to the instance of instanceID.
	Detach(volumeName, instanceID string) error

	// NetworkName will return an identifier of a volume that is relevant when corelating a
	// local device to a device that is the volumeName to the local instanceID.
	NetworkName(volumeName, instanceID string) (string, error)
}

The following examples assumes that you have passed proper environment variables based on the guest instance.

instance, err := driver.GetInstance()
if err != nil {
    log.Fatalf("Error: %s", err)
}

allBlockDevices, err := rexray.GetBlockDeviceMapping()
if err != nil {
    log.Fatalf("Error: %s", err)
}

These represent the methods that should be available from OS drivers.

type Driver interface {
    // Shows the existing mount points
    GetMounts(string, string) ([]*mount.MountInfo, error)

    // Check whether path is mounted or not
    Mounted(string) (bool, error)

    // Unmount based on a path
    Unmount(string) error

    // Mount based on a device, target, options, label
    Mount(string, string, string, string) error

    // Format a device with a FS type
    Format(string, string, bool) error
}

REX-Ray can be used independently as a CLI tool that provides guest storage introspection and management. The CLI should be distributed to the system that requires introspection and storage management. It will discover proper drivers to use, and then with proper authorization, will get further details about those devices.

Once the introspection has occurred, REX-Ray can then manage manage storage using initialized drivers in a common manner between storage providers. The providers will attach devices via any method possible to get the device attached as the next available /dev/xvd_ or one that is automatically assigned via the REX-Ray driver.

The REX-Ray CLI has a set of top-level commands that each represent logical groupings of common categorizations. Simply execute them to find out more about them!

[0]akutz@pax:rexray$ rexray
REX-Ray:
  A guest-based storage introspection tool that enables local
  visibility and management from cloud and storage platforms.

Usage: 
  rexray [flags]
  rexray [command]

Available Commands: 
  volume      The volume manager
  snapshot    The snapshot manager
  device      The device manager
  driver      The driver manager
  service     The service controller
  version     Print the version
  help        Help about any command

Flags:
  -c, --config="$HOME/.rexray/config.yaml": The REX-Ray configuration file
  -d, --debug=false: Enables verbose output
  -?, --help=false: Help for rexray
  -h, --host="tcp://127.0.0.1:7979": The REX-Ray service address


Use "rexray [command] --help" for more information about a command.

The follow examples demonstrate how to configure storage platforms and use the REX-Ray CLI to interact with them.

// TODO

export REXRAY_STORAGEDRIVERS=ec2
export AWS_ACCESS_KEY=access_key AWS_SECRET_KEY="secret_key"

./rexray volume get

- providername: ec2
  instanceid: i-695bb6ab
  volumeid: vol-dedbadc3
  devicename: /dev/sda1
  region: us-west-1
  status: attached
- providername: ec2
  instanceid: i-695bb6ab
  volumeid: vol-04c4b219
  devicename: /dev/xvdb
  region: us-west-1
  status: attached

// TODO

// TODO

// TODO

// TODO

// TODO

export REXRAY_STORAGEDRIVERS=rackspace
export OS_AUTH_URL=https://identity.api.rackspacecloud.com/v2.0 OS_USERNAME=username OS_PASSWORD='password'

./rexray volume get

- providername: RackSpace
  instanceid: 5ad7727c-aa5a-43e4-8ab7-a499295032d7
  volumeid: 738ea6b9-8c49-416c-97b7-a5264a799eb6
  devicename: /dev/xvdb
  region: DFW
  status: ""
- providername: RackSpace
  instanceid: 5ad7727c-aa5a-43e4-8ab7-a499295032d7
  volumeid: 43de157d-3dfb-441f-b832-4d2d8cf457cc
  devicename: /dev/xvdd
  region: DFW
  status: ""

export REXRAY_STORAGEDRIVERS=scaleio
export GOSCALEIO_ENDPOINT=https://mdm1.scaleio.local:443/api GOSCALEIO_INSECURE=true GOSCALEIO_USERNAME=admin GOSCALEIO_PASSWORD=Scaleio123 GOSCALEIO_SYSTEMID=1aa75ddc59b6a8f7 GOSCALEIO_PROTECTIONDOMAINID=ea81096700000000 GOSCALEIO_STORAGEPOOLID=1041757800000001

./rexray volume get

export GOXTREMIO_ENDPOINT="https://10.5.132.140/api/json"
export GOXTREMIO_INSECURE="true"
export GOXTREMIO_USERNAME="admin"
export GOXTREMIO_PASSWORD="Xtrem10"
export REXRAY_XTREMIO_MULTIPATH=true

./rexray volume get

// TODO

// TODO

// TODO

REX-Ray can be run as a CLI for interactive usage, but it can also be executed with the --daemon flag to spawn a background process that hosts an HTTP server with a RESTful API.

The REX-ray daemon comes with out-of-the-box support for SysV init scripts and systemd services. For example, in order to configure REX-Ray as a systemd service using the included rexray.service unit file, please follow the commands below:

# copy the service file to the systemd service unit file directory
sudo cp rexray.service /usr/lib/systemd/system/

# notify systemd about the new service & enable it to start on boot
sudo systemctl enable rexray

# start the rexray service
sudo systemctl start rexray

In the case of doing local tests, since it passes HTTP via Unix socket, you can use tools like socat and others like curl-unix-socket to talk with the API.

This can be used for as a simple test of the messages that do not have bodies since it is not HTTP aware. The following will test a basic activation message.

echo -e "GET /Plugin.Activate HTTP/1.1\r\n" | socat unix-connect:/run/docker/plugins/rexray.sock STDIO

This utility is HTTP and Unix socket aware so can do POST messages in a HTTP friendly manner which allows us to specify a body.

/usr/src/go/bin/curl-unix-socket -v -X POST -d '{"Name":"test22"}\r\n' unix:///run/docker/plugins/rexray.sock:/VolumeDriver.Mount

> POST /VolumeDriver.Mount HTTP/1.1
> Socket: /run/docker/plugins/rexray.sock
> Content-Length: 21
>
< HTTP/1.1 200 OK
< Content-Type: appplication/vnd.docker.plugins.v1+json
< Date: Fri, 22 May 2015 15:52:21 GMT
< Content-Length: 49
{"Mountpoint": "/var/lib/docker/volumes/test22"}

We are actively looking for contributors to this project. This can involve any number of area.

• Documentation
• Storage Drivers
• OS Drivers
• A future distributed model

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Please file bugs and issues at the Github issues page. For more general discussions you can contact the EMC Code team at Google Groups or tagged with EMC on Stackoverflow.com. The code and documentation are released with no warranties or SLAs and are intended to be supported through a community driven process.