An API interface to Mantl.

Currently, Mantl API allows you to install and uninstall DCOS packages on Mantl. More capabilities are planned for the future.

Table of Contents

• Mantl API
  • Comparison to Other Software
  • Building
  • Deploying
    • Options
  • Package Repository
    • Synchronizing Repository Sources
  • Usage
    • Installing a Package
    • Uninstalling a Package
  • Endpoints
    • GET /health
    • GET /1/packages
    • GET /1/packages/
    • POST /1/packages
    • DELETE /1/packages/
  • License

Mantl API leverages the Mesosphere DCOS package repository and provides capabilities similar to the package command in the DCOS CLI. The goal is to provide a simple, API-driven way to install and uninstall pre-built packages on Mantl clusters. In the future, mantl-api will contain additional functionality for maintaining and operating Mantl clusters.

As of the 0.5 release, Mantl API is installed by default on Mantl clusters. It is deployed via Marathon and will be running on one of the worker nodes. Mantl automatically discovers the Mantl API instance and puts it behind Nginx on all control nodes at the /api endpoint. By default, it is secured with SSL and basic authentication. As an example, on a default Mantl install, you can retrieve the list of available packages by running a command like the following:

curl -k -u admin:mantlpw https://mantl-control-01/api/1/packages

You just need to update the command with the correct host name and credentials for your cluster. All of the examples in this document will assume you are working with Mantl API on a Mantl cluster and will use urls underneath /api.

docker build -t mantl-api .

You can run Mantl API on your cluster via Marathon. An example mantl-api.json is included below:

{
  "id": "/mantl-api",
  "container": {
    "type": "DOCKER",
    "docker": {
      "image": "ciscocloud/mantl-api:0.1.0",
      "network": "BRIDGE",
      "portMappings": [
        { "containerPort": 4001, "hostPort": 0 }
      ]
    }
  },
  "instances": 1,
  "cpus": 1.0,
  "mem": 512,
  "constraints": [["hostname", "UNIQUE"]],
  "env": {
    "CONSUL_HTTP_SSL_VERIFY": "false",
    "MANTL_API_LOG_LEVEL": "debug",
    "MANTL_API_CONSUL": "https://consul.service.consul:8500",
    "MANTL_API_MESOS_PRINCIPAL": "mantl-api",
    "MANTL_API_MESOS_SECRET": "secret"
  },
  "healthChecks": [
    {
      "protocol": "HTTP",
      "path": "/health",
      "gracePeriodSeconds": 3,
      "intervalSeconds": 10,
      "portIndex": 0,
      "timeoutSeconds": 10,
      "maxConsecutiveFailures": 3
    }
  ]
}

You will need to replace the MANTL_API_MESOS_PRINCIPAL and MANTL_API_MESOS_SECRET variables with valid Mesos credentials. These can be found in the security.yml file that was generated when you ran security-setup for your Mantl cluster.

All Mantl API configuration can be set with environment variables. See below for the additional configuration options that are available.

To install Mantl API, you can submit the above json to Marathon.

curl -k -u admin:mantlpw -X POST -d @mantl-api.json -H "Content-type: application/json" https://mantl-control-01/marathon/v2/apps

You will need to replace admin:mantlpw with valid Marathon credentials (see marathon_http_credentials in your security.yml). You will also replace mantl-control-01 with the host of one your Mantl control nodes.

After a few moments, Mantl API should be running on your cluster.

Every option can be set via environment variables prefixed with MANTL_API. For example, you can use MANTL_API_LOG_LEVEL for log-level, MANTL_API_CONSUL for consul, and so on. You can also specify all configuration from a TOML configuration file using the config-file argument.

mantl-universe contains the list of packages that work out-of-the-box on Mantl today. You can install any of the DCOS packages but you will likely have to customize some of the configuration to work on Mantl. Most of the packages assume that service discovery is provided by Mesos-DNS and need to be converted to work with the Consul DNS interface. Contributions are welcome!

Mantl API is compatible with the package repository structure created for Mesosphere DCOS packages. Each repository contains a directory structure that looks something like this:

repo/packages
├── C
│   ├── cassandra
│   │   ├── 0
│   │   │   ├── config.json
│   │   │   ├── marathon.json
│   │   │   └── package.json
│   │   ├── 1
│   │   │   ├── command.json
│   │   │   ├── config.json
│   │   │   ├── marathon.json
│   │   │   └── package.json
├── H
│   └── hdfs
│       ├── 0
│       │   ├── command.json
│       │   ├── config.json
│       │   ├── marathon.json
│       │   └── package.json
│       ├── 1
│       │   ├── command.json
│       │   ├── config.json
│       │   ├── marathon.json
│       │   └── package.json
|__ ...

For each named package (cassandra and hdfs in the example above), there is a directory of JSON files corresponding to each version.

Mantl API supports the ability to layer repositories. Mantl API is configured to use two repository sources by default:

• Mesosphere Universe
• Mantl Universe

The order of the repositories is important. By default, Mesosphere Universe is configured with index 0 and Mantl Universe is configured with index 1. In the Consul KV store, the repositories are stored as follows:

Repositories with higher indexes are prioritized. When Mantl API receives a request to install a particular package, it will use the package definition found in the repository with the highest index. If there are any required files missing, it will look for the corresponding package version in the repositories with the lower indexes until it is able to construct a valid, installable package. For example, if mantl-install/repository/1/repo/packages/H/hdfs/1 (Mantl Universe) exists but does not have a config.json file, Mantl API will merge in the config.json file from mantl-install/repository/0/repo/packages/H/hdfs/1 (Mesosphere universe). This enables us to use the existing packages in the Mesosphere Universe without having to recreate everything in the Mantl Universe. Packages in the Mantl Universe repository typically just contain only the changes that are needed to run the package on Mantl clusters.

This also enables users to create their own repository to customize packages for their specific needs.

The package repositories are synchronized to the Consul K/V backend. If you want to refresh your repositories, you can run the following command:

mantl-api sync --consul http://consul.service.consul:8500

If you want to use different source repositories, you can specify a configuration file that contains your source definitions using the --config-file argument. Here is an example configuration file that uses 2 sources — one named "mesosphere" from a git repository and the other named "mantl-universe" from the local file system:

[sources]

[sources.mesosphere]
path = "https://github.com/mesosphere/universe.git"
type = "git"
index = 0

[sources.mantl]
path = "/home/ryan/Projects/mantl-universe"
index = 1

The following attributes can be specified for each source:

• path
• type — git or filesystem (default)
• index
• branch — only applicable for the git type

The following example curl commands assume a default Mantl API installation (with a control node called mantl-control-01) on a Mantl cluster with SSL and authentication turned off. You will need to adjust the commands to work with valid URLs and credentials for your cluster.

It is a single API call to install a package. In the example below, we are going to run Cassandra on our Mantl cluster.

curl -X POST -d "{\"name\": \"cassandra\"}" http://mantl-control-01/api/1/packages

You will need to replace mantl-worker-003 and 4001 with the host and port where Mantl API is running. You can use the Marathon API or UI to find this. You'll also need to make sure that the port is accessible to the machine where you are calling the API from. Adjust the security groups or firewall rules for your platform accordingly.

After about 5 minutes, you should have Cassandra up and running on your Mantl cluster.

Uninstalling is just as easy. Run the command below to uninstall Cassandra:

curl -X DELETE http://mantl-control-01/api/1/packages/cassandra

After a moment, Cassandra will have been removed from your cluster. This will also remove the Zookeeper state for the Cassandra framework. In the future, we will add more flexibility in being able to control what is uninstalled.

GET /health: returns OK

curl http://mantl-control-01/api/health

OK

GET /1/packages: returns a JSON representation of packages available to install.

curl http://mantl-control-01/api/1/packages | jq .

[
  {
    "name": "cassandra",
    "description": "Apache Cassandra running on Apache Mesos",
    "framework": true,
    "currentVersion": "0.2.0-1",
    "supported": true,
    "tags": [
      "mesosphere",
      "framework",
      "data",
      "database"
    ],
    "versions": {
      "0.1.0-1": {
        "version": "0.1.0-1",
        "index": "0",
        "supported": true
      },
      "0.2.0-1": {
        "version": "0.2.0-1",
        "index": "1",
        "supported": true
      }
    }
  },
  ...
  {
    "name": "swarm",
    "description": "Swarm is a Docker-native clustering system.",
    "framework": true,
    "currentVersion": "0.4.0",
    "supported": false,
    "tags": [
      "mesosphere",
      "framework",
      "docker"
    ],
    "versions": {
      "0.4.0": {
        "version": "0.4.0",
        "index": "0",
        "supported": false
      }
    }
  }
]

GET /1/packages/<package>: returns a JSON representation of a package.

curl http://mantl-control-01/api/1/packages/cassandra | jq .

{
  "name": "cassandra",
  "description": "Apache Cassandra running on Apache Mesos",
  "framework": true,
  "currentVersion": "0.2.0-1",
  "supported": true,
  "tags": [
    "mesosphere",
    "framework",
    "data",
    "database"
  ],
  "versions": {
    "0.1.0-1": {
      "version": "0.1.0-1",
      "index": "0",
      "supported": true
    },
    "0.2.0-1": {
      "version": "0.2.0-1",
      "index": "1",
      "supported": true
    }
  }
}

POST /1/packages: post a JSON representation of a package to install.

curl -X POST -d "{\"name\": \"cassandra\"}" http://mantl-control-01/api/1/packages | jq .

{
  "id": "/cassandra/dcos-test",
  "cmd": "$(pwd)/jre*/bin/java $JAVA_OPTS -classpath cassandra-mesos-framework.jar io.mesosphere.mesos.frameworks.cassandra.framework.Main",
  "args": null,
  "user": null,
  "env": {
    "CASSANDRA_ZK_TIMEOUT_MS": "10000",
    "JAVA_OPTS": "-Xms256m -Xmx256m",
    "MESOS_AUTHENTICATE": "true",
    "CASSANDRA_RESOURCE_CPU_CORES": "0.1",
    "CASSANDRA_FAILOVER_TIMEOUT_SECONDS": "604800",
    "DEFAULT_SECRET": "secret",
    "CASSANDRA_BOOTSTRAP_GRACE_TIME_SECONDS": "120",
    "CASSANDRA_DEFAULT_DC": "DC1",
    "CASSANDRA_RESOURCE_MEM_MB": "768",
    "MESOS_ZK": "zk://zookeeper.service.consul:2181/mesos",
    "CASSANDRA_DATA_DIRECTORY": ".",
    "DEFAULT_PRINCIPAL": "mantl-install",
    "CASSANDRA_FRAMEWORK_MESOS_ROLE": "*",
    "CASSANDRA_CLUSTER_NAME": "dcos-test",
    "CASSANDRA_RESOURCE_DISK_MB": "16",
    "CASSANDRA_SEED_COUNT": "2",
    "CASSANDRA_ZK": "zk://zookeeper.service.consul:2181/cassandra-mesos/dcos-test",
    "CASSANDRA_NODE_COUNT": "3",
    "CASSANDRA_DEFAULT_RACK": "RAC1",
    "CASSANDRA_HEALTH_CHECK_INTERVAL_SECONDS": "60"
  },
  "instances": 1,
  "cpus": 0.5,
  "mem": 512,
  "disk": 0,
  "executor": "",
  "constraints": [],
  "uris": [
    "https://downloads.mesosphere.io/cassandra-mesos/artifacts/0.2.0-1/cassandra-mesos-0.2.0-1.tar.gz",
    "https://downloads.mesosphere.io/java/jre-7u76-linux-x64.tar.gz"
  ],
  "storeUrls": [],
  "ports": [
    0
  ],
  "requirePorts": false,
  "backoffFactor": 0,
  "container": null,
  "healthChecks": [
    {
      "path": "/health/cluster",
      "protocol": "HTTP",
      "portIndex": 0,
      "command": null,
      "gracePeriodSeconds": 120,
      "intervalSeconds": 15,
      "timeoutSeconds": 5,
      "maxConsecutiveFailures": 0,
      "ignoreHttp1xx": false
    },
    {
      "path": "/health/process",
      "protocol": "HTTP",
      "portIndex": 0,
      "command": null,
      "gracePeriodSeconds": 120,
      "intervalSeconds": 30,
      "timeoutSeconds": 5,
      "maxConsecutiveFailures": 3,
      "ignoreHttp1xx": false
    }
  ],
  "dependencies": [],
  "upgradeStrategy": {
    "minimumHealthCapacity": 0,
    "maximumOverCapacity": 0
  },
  "labels": {
    "MANTL_PACKAGE_NAME": "cassandra",
    "MANTL_PACKAGE_IS_FRAMEWORK": "true",
    "DCOS_PACKAGE_FRAMEWORK_NAME": "cassandra.dcos-test",
    "MANTL_PACKAGE_FRAMEWORK_NAME": "cassandra.dcos-test",
    "MANTL_PACKAGE_INDEX": "1",
    "MANTL_PACKAGE_VERSION": "0.2.0-1"
  },
  "acceptedResourceRoles": null,
  "version": "2015-09-17T22:19:24.576Z",
  "tasks": [],
  "deployments": [
    {
      "id": "dd113528-388c-4438-a524-9eba45aa2908"
    }
  ],
  "tasksStaged": 0,
  "tasksRunning": 0,
  "tasksHealthy": 0,
  "tasksUnhealthy": 0,
  "backoffSeconds": 0,
  "maxLaunchDelaySeconds": 3600
}

DELETE /1/packages/<package>: post a JSON representation of package specific uninstall options.

curl -X DELETE -d "{\"name\": \"cassandra\"}" http://mantl-control-01/api/1/packages/cassandra

mantl-api is released under the Apache 2.0 license (see LICENSE)