Skip to content

Klaudit/lattice

 
 

Repository files navigation

Lattice: Run Containerized Workloads

Website: http://lattice.cf
Mailing List: Subscribe
Archives: [ Nabble | Google Groups ]

Lattice is an open source project for running containerized workloads on a cluster. Lattice bundles up http load-balancing, a cluster scheduler, log aggregation/streaming and health management into an easy-to-deploy and easy-to-use package.

Lattice is based on a number of open source Cloud Foundry components:

  • Diego schedules and monitors containerized workloads
  • Loggregator aggregates and streams application logs
  • Gorouter provides http load-balancing

Demoing Lattice with Redis at Spring One? Please see Troubleshooting.

Get Lattice

Visit Lattice Releases or our Nightly Bundles page to download one of our bundles. These include both the ltc CLI for the appropriate architecture, as well as the Vagrantfile and Terraform examples for a given release or nightly build.

Deploy Lattice

A local deployment of Lattice can be launched with Vagrant.

A scalable cluster deployment of Lattice can be launched with Terraform. We currently support AWS, DigitalOcean, Google Cloud and Openstack.

Use Lattice

The Lattice CLI ltc provides a command line interface for launching docker-based applications.

More complex workloads can be constructed and submitted directly to Lattice's Receptor API which is fully documented here.

Local Deployment

Launching with Vagrant

Make sure you have Vagrant 1.6+ installed, download the Lattice bundle from the latest release or the nightly builds page, and run vagrant up:

unzip lattice-bundle-VERSION-PLATFORM.zip
cd lattice-bundle-VERSION-PLATFORM/vagrant
vagrant up

This spins up a virtual environment that is accessible at 192.168.11.11.

Use the Lattice CLI to target Lattice:

cd lattice-bundle-VERSION-PLATFORM
./ltc target 192.168.11.11.xip.io

NOTE: Ubuntu 14.04 LTS does not install a compatible version of vagrant by default. You can upgrade the version that you get out of the box by downloading the .deb file from Vagrant.

Using Different Providers

You can do this with either VirtualBox or VMware Fusion (version 7 or later):

Virtualbox:

vagrant up --provider virtualbox

VMware Fusion:

vagrant up --provider vmware_fusion

Networking Conflicts

If you are trying to run both the VirtualBox and VMWare providers on the same machine, you'll need to run them on different private networks (subnets) that do not conflict.

Set the System IP to an address that does not conflict with the host networking configuration by passing the LATTICE_SYSTEM_IP environment variable to the vagrant up command:

LATTICE_SYSTEM_IP=192.168.80.100 vagrant up
ltc target 192.168.80.100.xip.io

Miscellaneous

Nightly builds

The latest unsupported nightly build is available for Linux and for OS X.

Building Lattice from source

To build Lattice from source and deploy using Vagrant:

$ git clone git@github.com:cloudfoundry-incubator/lattice.git
$ cd lattice
$ development/setup && development/build && development/run
$ source development/env

More information on developing for Lattice can be found on the development readme.

Updating

Currently, Lattice does not support updating via provision. To update, you have to destroy the box and bring it back up with a new Vagrantfile. If you have copied a new Vagrantfile into an existing directory, make sure to remove any outdated lattice.tgz present in that directory.

Manual install of Lattice

Follow these instructions to install a co-located Lattice cluster to a server that's already deployed. (e.g., vSphere)

Proxy configuration

Install the vagrant-proxyconf plugin as follows:

$ vagrant plugin install vagrant-proxyconf

Setup your environment in the terminal:

$ export http_proxy=http://PROXY_IP:PROXY_PORT
$ export https_proxy=http://PROXY_IP:PROXY_PORT
$ export no_proxy=$(ltc target|head -1|awk '{print $2}')

Then proceed with vagrant up. For ltc create, ltc build-droplet or ltc launch-droplet, you'll need to pass these environment variables into the container. For example:

$ ltc create -e http_proxy -e https_proxy -e no_proxy lattice-docker-app cloudfoundry/lattice-app
$ ltc build-droplet -e http_proxy -e https_proxy -e no_proxy lattice-droplet go
$ ltc launch-droplet -e http_proxy -e https_proxy -e no_proxy lattice-app lattice-droplet

Troubleshooting

Redis Docker hub image incompatibility

As of September 2015, Lattice is incompatible with most tags listed on the redis DockerHub registry. We are actively investigating the issue. In the meantime, please use the 3.0.1 tag:

ltc create redis redis:3.0.1 --run-as-root --memory-mb=256 --tcp-routes=6379:6379 --monitor-command="redis-cli --scan"
redis-cli -h 192.168.11.11

No such host errors

DNS resolution for xip.io addresses can sometimes be flaky, resulting in errors such as the following:

 ltc target 192.168.11.11.xip.io
 Error verifying target: Get http://receptor.192.168.11.11.xip.io/v1/desired_lrps:
 dial tcp: lookup receptor.192.168.11.11.xip.io: no such host

Resolution Steps

  1. Follow these instructions to reset the DNS cache in OS X. There have been several reported issues with DNS resolution on OS X, specifically on Yosemite, insofar as the latest beta build of OS X 10.10.4 has replaced discoveryd with mDNSResponder.

  2. Check your networking DNS settings. Local "forwarding DNS" servers provided by some home routers can have trouble resolving xip.io addresses. Try setting your DNS to point to your real upstream DNS servers, or alternatively try using Google DNS by using 8.8.8.8 and/or 8.8.4.4.

  3. If the above steps don't work (or if you must use a DNS server that doesn't work with xip.io), our recommended alternative is to follow the dnsmasq instructions, pass the LATTICE_SYSTEM_DOMAIN environment variable to the vagrant up command, and target using lattice.dev instead of 192.168.11.11.xip.io to point to the cluster, as follows:

LATTICE_SYSTEM_DOMAIN=lattice.dev vagrant up
ltc target lattice.dev

dnsmasq is currently only supported for vagrant deployments.

Vagrant IP conflict errors

The below errors can come from having multiple vagrant instances using the same IP address (e.g., 192.168.11.11).

$ ltc target 192.168.11.11.xip.io
Error connecting to the receptor. Make sure your lattice target is set, and that lattice is up and running.
  Underlying error: Get http://receptor.192.168.11.11.xip.io/v1/desired_lrps: read tcp 192.168.11.11:80: connection reset by peer

$ ltc target 192.168.11.11.xip.io
Error connecting to the receptor. Make sure your lattice target is set, and that lattice is up and running.
  Underlying error: Get http://receptor.192.168.11.11.xip.io/v1/desired_lrps: use of closed network connection  

$ ltc target 192.168.11.11.xip.io
Error verifying target: Get http://receptor.192.168.11.11.xip.io/v1/desired_lrps: net/http: transport closed before response was received

To check whether multiple VMs might have an IP conflict, run the following:

$ vagrant global-status
id       name    provider   state   directory
----------------------------------------------------------------------------------------------------------------
fb69d90  default virtualbox running /Users/user/workspace/lattice
4debe83  default virtualbox running /Users/user/workspace/lattice-bundle-v0.4.0-osx/vagrant

You can then destroy the appropriate instance with:

$ cd </path/to/vagrant-directory>
$ vagrant destroy

Miscellaneous

If you have trouble running vagrant up --provider virtualbox with the error

default: Warning: Remote connection disconnect. Retrying...
default: Warning: Authentication failure. Retrying...
...

try upgrading to the latest VirtualBox.

Clustered Deployment

This repository contains several Terraform templates to help you deploy on your choice of IaaS. To deploy Lattice in this way you will need:

  • Credentials for your choice of IaaS

  • Terraform

    Lattice Compatible Versions
    v0.3.3+ Terraform 0.6.2+
    v0.3.0 Terraform 0.6.1
    v0.2.7 Terraform 0.6.1

Deploying

Here are some step-by-step instructions for deploying a Lattice cluster via Terraform:

  1. Download a lattice bundle from the latest release or the nightly builds page.
  2. Unzip the bundle, and switch to the directory for your intended provider:
unzip lattice-bundle-VERSION-PLATFORM.zip
cd lattice-bundle-VERSION-PLATFORM/terraform/PROVIDER
  1. Update the lattice.<provider>.tf by filling in the values for the variables. Instructions for each supported platform are here:
  1. Run the following commands in the folder containing the lattice.<platform>.tf file
terraform get -update
terraform apply

This will deploy the cluster.

Upon success, terraform will print the Lattice target:

Outputs:

  lattice_target = x.x.x.x.xip.io
  lattice_username = xxxxxxxx
  lattice_password = xxxxxxxx

which you can use with the Lattice CLI to ltc target x.x.x.x.xip.io.

Terraform will generate a terraform.tfstate file. This file describes the cluster that was built - keep it around in order to modify/tear down the cluster.

Destroying

To destroy the cluster go to the folder containing the terraform.tfstate file and run:

terraform destroy

Updating

To update the cluster, you must destroy your existing cluster and begin the deployment process again using a new lattice bundle.

Contributing

Everyone is encouraged to help improve this project.

Please submit pull requests against the master branch.

Our Concourse CI system is available at ci.lattice.cf.

Here are some ways you can contribute:

  • by using alpha, beta, and prerelease versions
  • by reporting bugs
  • by suggesting new features
  • by writing or editing documentation
  • by writing specifications
  • by writing code (no patch is too small: fix typos, add comments, clean up inconsistent whitespace)
  • by refactoring code
  • by closing issues
  • by reviewing patches

Also see the Development Readme

Submitting an Issue

We use the GitHub issue tracker to track bugs and features. Before submitting a bug report or feature request, check to make sure it hasn't already been submitted. You can indicate support for an existing issue by voting it up. When submitting a bug report, please include a Gist that includes a stack trace and any details that may be necessary to reproduce the bug including the Lattice version.

Submitting a Pull Request

  1. Propose a change by opening an issue.
  2. Fork the project.
  3. Create a topic branch.
  4. Implement your feature or bug fix.
  5. Commit and push your changes.
  6. Submit a pull request.

Choosing Stories From our Tracker

Pivotal Tracker is the way that Cloud Foundry projects organize and prioritize work and releases. With Tracker, work is organized into stories that are actionable and have been prioritized. The team typically works on stories found in the Current and Backlog columns.

Stories not (yet) prioritized are kept in the Icebox. The Icebox is a grab-bag of feature requests, GitHub Issues, or partially-developed ideas. Stories in the Icebox may never be prioritized, and may not always be maintained in the same disciplined manner as the Backlog.

Periodically, the Lattice team goes through through both the Backlog and the Icebox, and tags stories using the 'community' label. These are stories that are particularly well-suited to community contribution, and make excellent candidates for people to work on.

Copyright

See LICENSE for details. Copyright (c) 2015 Pivotal Software, Inc.

Packages

No packages published

Languages

  • Go 91.7%
  • Shell 4.1%
  • HCL 4.0%
  • Other 0.2%