Patterns and some code inspired by Hashicorp vault.
Currently in PoC status - don't even think about using this in a production situation.
This software uses Hashicorp's Vault for storing secret, in this case the private keys for signing tokens. We keep the public keys in there are well, since we don't hand those out to anybody.
The easiest way to get set up with vault in a configuration that requires sealing/unsealing, use of tokens, and so on is to run vault with the file backend.
To do use a configuration that looks like this:
backend "file" {
path = "/cygdrive/c/vault/backend"
}
listener "tcp" {
address = "0.0.0.0:8200"
tls_disable = 1
}
Then run vault pointing to that configuration:
vault server -config vconfig
You will need to initialize the vault (vault init) and unseal the vault (vault unseal) using the key shards
produced by the init process. Note - hang onto the key shards and root token!
To interact with vault from the command line, you will need to set the VAULT_ADDR environment variable to
http://localhost:8200 and set the VAULT_TOKEN environment variable to the root token (or make your owbn token).
Refer to the vault documentation for details.
You will need to set up AWS credentials for a user associated with the following policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1441122614000",
"Effect": "Allow",
"Action": [
"dynamodb:DeleteItem",
"dynamodb:GetItem",
"dynamodb:GetRecords",
"dynamodb:PutItem",
"dynamodb:Query",
"dynamodb:Scan",
"dynamodb:UpdateItem"
],
"Resource": [
"*"
]
}
]
}
The code uses two tables in DynamoDB: Application and Developer. Refer to the go code under the repos/ddl package that can be used to create the tables.
Note that you can use DynamoDB Local as well with this code. To do set, specify local use via an environment variable named LOCAL_DYNAMO_ADDR, and set your local address using this variable, e.g.
export LOCAL_DYNAMO_ADDR=http://localhost:8000
As an alternative to DynamoDB, Roll can be configured to use MariaDB as its database. To set up MariaDB, use the rolldb.sql script in repos/ddl to create the database and user for the application. Minimally, the user should be customized. Once the database and user have been created, the tabledefs.sql script can be run to create the tables and grant the appropriate accesss to the roll db user created in the first step, which means tabledefs.sql will need to be edited to reference the user created in step 1.
Still need to vendor my dependencies, but they are:
- Stretchr Testify
- AWS Golang SDK
- Hashicorp Vault Golang API
- JWT-go
- Go UUID
- Context package from the Gorilla Web Toolkit
- Go MySQL Driver
Use go get github.com/hashicorp/vault/api to install the API portion of Vault
I used the mockery tool to generate the mocks - I don't think there's a runtime dependency but go get it as per its instructions if you see build weirdness.
The following commands downloads all the dependencies needed to build the software:
go get github.com/stretchr/testify/ go get -u github.com/aws/aws-sdk-go/... go get github.com/hashicorp/vault/api go get github.com/dgrijalva/jwt-go go get github.com/nu7hatch/gouuid go get github.com/gorilla/context go get github.com/go-sql-driver/mysql
To run the integration test, install gucumber:
go get github.com/lsegal/gucumber/cmd/gucumber
You will also need the docker client to run the integration tests.
go get github.com/samalba/dockerclient
Running the integration tests requires docker-machine to be installed. You also need AWS credentials and, if behind an http_proxy, set your http_proxy and no_proxy environment variables appropriately.
Below are some older instructions (this documentation is in the process of being rewritten).
In a nutshell, you need to:
- Boot roll in unsecure mode
- In rollsvcs/cmd set the environment using setenv.sh then run
go run rollmain.go -port 3000 -unsecure
- In rollsvcs/cmd set the environment using setenv.sh then run
- Seed a roll application - see seed.js in the Roll Setup repository. Seed.js registers a developer and seeds a portal application associated with that developer. Roll can then being run in secure mode as the registered application created by seed.js
- Reboot roll in secure mode, using the application client_id obtained in the previous step.
- Register a sample application for use with rollsample and rollecho in trying out the grants as outlined below. See register.js in the Roll Setup repository. Note you'll need to grab the client id and secret for the seeded roll app from the application table and set the values of portClientId and clientSecret in register.js
- Start the rollsample and rollecho applications. Roll sample is a sample application that obtains auth tokens via roll, and roll echo is a service that requires an auth token associated with the sample application for admission.
- Do the flows as explained below.
Build the rollsvcs executable (go build in the rollsvcs directory), set the VAULT_ADDR and VAULT_TOKEN environment variables as described above, and start the server. Here we assume the use of port 3000.
Also, if you need a proxy setting to access the internet, set the HTTP_PROXY environment variable.
./rollsvcs -port 3000
Next, build the callback server, which is used for the oauth2 callback for our sample, plus it can mock an XTRAC login if you used xtrac://localhost:2000 as the login provider. We assume the use of port 2000.
./cbserver -port 2000
The following command line examples using curl illustrate how to secure access to roll services to a specific application registered with roll.
It's a two step process. First roll is booted in unsecured mode to allow an initial 'developer' to be created, followed by the creation of the application that will be authorized to call roll. After this has been done, roll is booted in secure mode, afterwhich all access will require an access token obtained using the authorized application.
Note the OAuth endpoints are not restricted, but obtaining access tokens is done in the context of an authorized application, which is the source of the client id, client secret, etc.
#####Unsecured (Bootstrap)
go run rollmain.go -port 3000 -unsecure
curl -v -X PUT -d '
{
"email":"test@dev.com",
"firstName":"Doug",
"lastName":"Dev"
}' -H 'X-Roll-Subject: portal-admin' localhost:3000/v1/developers/test@dev.com
curl -X POST -d '{
"applicationName":"secured dev portal",
"developerEmail":"test@dev.com",
"redirectURI":"http://localhost:2000/oauth2_callback",
"loginProvider":"xtrac://localhost:2000"
}' -H 'X-Roll-Subject: portal-admin' localhost:3000/v1/applications
{"client_id":"14fc366b-9138-46af-4bf7-c47579911028"}
curl -H 'X-Roll-Subject: portal-admin' localhost:3000/v1/applications/14fc366b-9138-46af-4bf7-c47579911028
{"developerEmail":"test@dev.com","DeveloperID":"portal-admin","clientID":"14fc366b-9138-46af-4bf7-c47579911028","applicationName":"secured dev portal","clientSecret":"wW2vFB3aWjOkcj9LoONIpmT+VQ3/KoS8GiRom0yqReo=","redirectURI":"http://localhost:2000/oauth2_callback","loginProvider":"xtrac://localhost:2000","jwtFlowPublicKey":""}
curl --data "client_id=14fc366b-9138-46af-4bf7-c47579911028" --data "grant_type=password" --data-urlencode "client_secret=wW2vFB3aWjOkcj9LoONIpmT+VQ3/KoS8GiRom0yqReo=" --data "username=foo" --data "password=passw0rd" localhost:3000/oauth2/token
{"access_token":"eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJhcHBsaWNhdGlvbiI6InNlY3VyZWQgZGV2IHBvcnRhbCIsImF1ZCI6IjE0ZmMzNjZiLTkxMzgtNDZhZi00YmY3LWM0NzU3OTkxMTAyOCIsImV4cCI6MTQ1MjI4ODc3NCwiaWF0IjoxNDUyMjAyMzc0LCJqdGkiOiJkNWQ1ZWNhYy05ZWU3LTQ0YzctNmZjYS1kMDBhN2RjZTQ3ZmYiLCJzY29wZSI6IiIsInN1YiI6ImZvbyJ9.mgPyalqccJpA_IC2-JB3WcdqpRbxb2ZDkdNAeTqaNnMgxCbw1z6BIjOdpmHiZYc51g5zbTWUJl6XZsyv1Ul3u0U8kpfhjmg12VqN1mxVbsx3Z6wVXKRAaFN85rnoE-h4TaXRJcqq5wXzBXz1QT0qX8462VJN4cYaHo-MmHydTtU","token_type":"Bearer"}
#####Secured
Note - use admins.go in repos/util to seed admin users - required to use a scope of admin.
export ROLL_CLIENTID=1d703e17-fc84-42eb-65b6-9dcb7700b282 go run rollmain.go -port 3000
curl -v -X PUT -d '
{
"email":"new-dev@dev.com",
"firstName":"Doug",
"lastName":"Dev"
}' -H 'X-Roll-Subject: foo' localhost:3000/v1/developers/doug@dev.com
< HTTP/1.1 401 Unauthorized
curl --data "client_id=14fc366b-9138-46af-4bf7-c47579911028" --data "grant_type=password" --data-urlencode "client_secret=wW2vFB3aWjOkcj9LoONIpmT+VQ3/KoS8GiRom0yqReo=" --data "username=newdev" --data "password=passw0rd" localhost:3000/oauth2/token
{"access_token":"eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJhcHBsaWNhdGlvbiI6InNlY3VyZWQgZGV2IHBvcnRhbCIsImF1ZCI6IjY3YmZmYzQwLWVlZDItNDE0ZC01OWUzLWNhZTgyYzg3YzUyMCIsImV4cCI6MTQ1MjI4Njk4NiwiaWF0IjoxNDUyMjAwNTg2LCJqdGkiOiJjYWI2ZjE3OC0xNzA3LTQ2NDMtNGU3Mi03OWUzMDM1NDQwY2QiLCJzY29wZSI6IiIsInN1YiI6Im5ld2RldiJ9.lDWPsru75l-KdchoTZQE63LWaNX6f2dRwXlgWqLXXvbcd76R05zOKutOGUDQadtTTTudoRP4KQ2YHqxttPtVlnk9VSajJEuCysTdfslyegoTsc7qcy3vjxIc17FH6mclP8aqcgjyR20_L33GC7HVKe5CPRjOr615afVZF9GvXNs","token_type":"Bearer"}
curl -v -X PUT -d '
{
"email":"new-dev@dev.com",
"firstName":"Doug",
"lastName":"Dev"
}' -H 'Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJhcHBsaWNhdGlvbiI6InNlY3VyZWQgZGV2IHBvcnRhbCIsImF1ZCI6IjE0ZmMzNjZiLTkxMzgtNDZhZi00YmY3LWM0NzU3OTkxMTAyOCIsImV4cCI6MTQ1MjI4ODkwMSwiaWF0IjoxNDUyMjAyNTAxLCJqdGkiOiJiNjdmYTFiOS1kM2VlLTQyYTMtNDIzNS03Yzg4YmMyNTFmYWYiLCJzY29wZSI6IiIsInN1YiI6Im5ld2RldiJ9.EReJee2p__3KCvzFIj7esj5rGZy0BSWzQ24gwDDA4yFehCcvfPlWMP3M4_31tQeNJjgzo4PxfyfIvHTfKvbZy3h4OJru7Rk9ECkgMHx3yM-mWVKGvGJ3xdHnPcAbT8ArDcQRthS_So5KYJ5I3hq_swc7rOH0wdoF0FDMjD2fuP8' localhost:3000/v1/developers/new-dev@dev.com
curl -H 'Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJhcHBsaWNhdGlvbiI6InNlY3VyZWQgZGV2IHBvcnRhbCIsImF1ZCI6IjE0ZmMzNjZiLTkxMzgtNDZhZi00YmY3LWM0NzU3OTkxMTAyOCIsImV4cCI6MTQ1MjI4ODkwMSwiaWF0IjoxNDUyMjAyNTAxLCJqdGkiOiJiNjdmYTFiOS1kM2VlLTQyYTMtNDIzNS03Yzg4YmMyNTFmYWYiLCJzY29wZSI6IiIsInN1YiI6Im5ld2RldiJ9.EReJee2p__3KCvzFIj7esj5rGZy0BSWzQ24gwDDA4yFehCcvfPlWMP3M4_31tQeNJjgzo4PxfyfIvHTfKvbZy3h4OJru7Rk9ECkgMHx3yM-mWVKGvGJ3xdHnPcAbT8ArDcQRthS_So5KYJ5I3hq_swc7rOH0wdoF0FDMjD2fuP8' localhost:3000/v1/developers/new-dev@dev.com
curl -X POST -d '{
"applicationName":"App No. 5",
"developerEmail":"new-dev@dev.com",
"redirectURI":"http://localhost:2000/oauth2_callback",
"loginProvider":"xtrac://localhost:2000"
}' -H 'Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJhcHBsaWNhdGlvbiI6InNlY3VyZWQgZGV2IHBvcnRhbCIsImF1ZCI6IjE0ZmMzNjZiLTkxMzgtNDZhZi00YmY3LWM0NzU3OTkxMTAyOCIsImV4cCI6MTQ1MjI4ODkwMSwiaWF0IjoxNDUyMjAyNTAxLCJqdGkiOiJiNjdmYTFiOS1kM2VlLTQyYTMtNDIzNS03Yzg4YmMyNTFmYWYiLCJzY29wZSI6IiIsInN1YiI6Im5ld2RldiJ9.EReJee2p__3KCvzFIj7esj5rGZy0BSWzQ24gwDDA4yFehCcvfPlWMP3M4_31tQeNJjgzo4PxfyfIvHTfKvbZy3h4OJru7Rk9ECkgMHx3yM-mWVKGvGJ3xdHnPcAbT8ArDcQRthS_So5KYJ5I3hq_swc7rOH0wdoF0FDMjD2fuP8' localhost:3000/v1/applications
{"client_id":"aaf9404c-ac51-4720-6097-f80c5404773d"}
curl -H 'Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJhcHBsaWNhdGlvbiI6InNlY3VyZWQgZGV2IHBvcnRhbCIsImF1ZCI6IjE0ZmMzNjZiLTkxMzgtNDZhZi00YmY3LWM0NzU3OTkxMTAyOCIsImV4cCI6MTQ1MjI4ODkwMSwiaWF0IjoxNDUyMjAyNTAxLCJqdGkiOiJiNjdmYTFiOS1kM2VlLTQyYTMtNDIzNS03Yzg4YmMyNTFmYWYiLCJzY29wZSI6IiIsInN1YiI6Im5ld2RldiJ9.EReJee2p__3KCvzFIj7esj5rGZy0BSWzQ24gwDDA4yFehCcvfPlWMP3M4_31tQeNJjgzo4PxfyfIvHTfKvbZy3h4OJru7Rk9ECkgMHx3yM-mWVKGvGJ3xdHnPcAbT8ArDcQRthS_So5KYJ5I3hq_swc7rOH0wdoF0FDMjD2fuP8' localhost:3000/v1/applications/aaf9404c-ac51-4720-6097-f80c5404773d
{"developerEmail":"new-dev@dev.com","DeveloperID":"newdev","clientID":"aaf9404c-ac51-4720-6097-f80c5404773d","applicationName":"App No. 5","clientSecret":"JqBUEAYXOxook/VzwqiqyiKPirJ5mBXZkOerrhAfId8=","redirectURI":"http://localhost:2000/oauth2_callback","loginProvider":"xtrac://localhost:2000","jwtFlowPublicKey":""}
curl --data "client_id=aaf9404c-ac51-4720-6097-f80c5404773d" --data scope=admin --data "grant_type=password" --data-urlencode "client_secret=JqBUEAYXOxook/VzwqiqyiKPirJ5mBXZkOerrhAfId8=" --data "username=portal-admin" --data "password=passw0rd" localhost:3000/oauth2/token
{"access_token":"eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJhcHBsaWNhdGlvbiI6IkFwcCBOby4gNSIsImF1ZCI6ImFhZjk0MDRjLWFjNTEtNDcyMC02MDk3LWY4MGM1NDA0NzczZCIsImV4cCI6MTQ1MjI4OTUwOSwiaWF0IjoxNDUyMjAzMTA5LCJqdGkiOiIzODMxMDVhNy1kODMwLTRkZjgtNzQwNy1iMmMyZjZiYzFkMjAiLCJzY29wZSI6ImFkbWluIiwic3ViIjoicG9ydGFsLWFkbWluIn0.k8LgLK1OPDCeN8XzByS80UePhhB16CdnXPBIGdG6lRbJ97qyPMticFNuzvPCQq2ZKJ4Qp1gvIDYEFcq6iawWcMLdl_tuyhGT-wkq3_yeZq2bk2aKmbkws4azKhNOv8Ih-bYg4E7PjVuowplkyUvUqm7_tBqKElFQW-3kYBYkyyk","token_type":"Bearer"}
curl -H 'Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJhcHBsaWNhdGlvbiI6InNlY3VyZWQgZGV2IHBvcnRhbCIsImF1ZCI6IjE0ZmMzNjZiLTkxMzgtNDZhZi00YmY3LWM0NzU3OTkxMTAyOCIsImV4cCI6MTQ1MjI4ODc3NCwiaWF0IjoxNDUyMjAyMzc0LCJqdGkiOiJkNWQ1ZWNhYy05ZWU3LTQ0YzctNmZjYS1kMDBhN2RjZTQ3ZmYiLCJzY29wZSI6IiIsInN1YiI6ImZvbyJ9.mgPyalqccJpA_IC2-JB3WcdqpRbxb2ZDkdNAeTqaNnMgxCbw1z6BIjOdpmHiZYc51g5zbTWUJl6XZsyv1Ul3u0U8kpfhjmg12VqN1mxVbsx3Z6wVXKRAaFN85rnoE-h4TaXRJcqq5wXzBXz1QT0qX8462VJN4cYaHo-MmHydTtU' localhost:3000/v1/applications
Create Developer
curl -v -X PUT -d '
{
"email":"doug@dev.com",
"firstName":"Doug",
"lastName":"Dev"
}' localhost:3000/v1/developers/doug@dev.com
Retrieve a Developer
curl localhost:3000/v1/developers/doug@dev.com
{"firstName":"Doug","lastName":"Dev","email":"doug@dev.com","id":""}
List Developers
curl localhost:3000/v1/developers
[{"firstName":"Doug","lastName":"Dev","email":"doug@dev.com","id":""}]
Register an application
curl -X POST -d '{
"applicationName":"App No. 5",
"developerEmail":"doug@dev.com",
"redirectURI":"http://localhost:2000/oauth2_callback",
"loginProvider":"xtrac://localhost:2000"
}' localhost:3000/v1/applications
{"client_id":"7843541e-d4cb-4903-5b88-ee596c32ecd7"}
Update an application
curl -v -X PUT -d '{
"applicationName":"App No. Four",
"developerEmail":"doug@dev.com",
"redirectURI":"http://localhost:2000/oauth2_callback",
"loginProvider":"xtrac://localhost:2000"
}' localhost:3000/v1/applications/7843541e-d4cb-4903-5b88-ee596c32ecd7
Retrieve an Application
curl localhost:3000/v1/applications/7843541e-d4cb-4903-5b88-ee596c32ecd7
{"developerEmail":"doug@dev.com","clientID":"7843541e-d4cb-4903-5b88-ee596c32ecd7","applicationName":"App No. Four","clientSecret":"bQeH+n/Q9g8gM++Xd9gnqrn6zp92EZpSXrRPofVUbyk=","redirectURI":"http://localhost:2000/oauth2_callback","loginProvider":"xtrac://localhost:2000","jwtFlowPublicKey":""}
Retrieve all applications
curl localhost:3000/v1/applications
[{"developerEmail":"doug@dev.com","clientID":"7843541e-d4cb-4903-5b88-ee596c32ecd7","applicationName":"App No. Four","clientSecret":"bQeH+n/Q9g8gM++Xd9gnqrn6zp92EZpSXrRPofVUbyk=","redirectURI":"http://localhost:2000/oauth2_callback","loginProvider":"xtrac://localhost:2000","jwtFlowPublicKey":""}]
Open the following in your browser. Note I use chrome for this - I assume it will work in other browsers.
http://localhost:3000/oauth2/authorize?client_id=111-222-3333&response_type=token&redirect_uri=http://localhost:2000/oauth2_callback
Note that you need to use the ClientID as the client_id parameter, and the redirect_uri registered for the application.
The url will present you with an authorization page. Fill in some credentials and click approve, or go straight to deny. The browser will be redirected to your registered callback - if you use the supplied callback server it will display your access token or the access denied error. You might also get an error if the client if can't be found, it it's not a valid client id, etc.
Note in ths implementation of the flow, the credentials are sent back to the authorization server, which looks up the authentication (login) endpoint associated with the application, which is embedded in a URL, e.g.
xtrac://localhost:2000
Using the URL scheme, we can accomodate different login providers, currently we know how to authenticate against XTRAC.
This can be be done with the above setup by modifying the above URL to use code
as the grant type:
http://localhost:3000/oauth2/authorize?client_id=111-222-3333&response_type=code&redirect_uri=http://localhost:2000/oauth2_callback
This can be executed directly via curl, e.g.
curl --data "client_id=7843541e-d4cb-4903-5b88-ee596c32ecd7" --data "grant_type=password" --data-urlencode "client_secret=bQeH+n/Q9g8gM++Xd9gnqrn6zp92EZpSXrRPofVUbyk=" --data "username=foo" --data "password=passw0rd" localhost:3000/oauth2/token
The JWT flow allows a security token created in a different fiefdom to be exchanged for an XTRAC token. To enable this flow for an application, a certificate that can be used to validate the foreign JWT is uploaded to the roll server for the application. When the external token is posted to the token endpoint, the application key (client_id) associated with the application is assumed to be carried in the token's iss claim: the public key extracted from the uploaded certificate is used to validate the token signature, and if it checks out a access token is returned.
To try it out, we need to upload a cert, create and sign a JWT with the private key assocaited with the cert, and post the JWT flow payload to the token endpoint -- jwt-sample/jwtflow-sample.go shows how to execute this flow.
Now that an application has been configured and an access token created, we can protect resources via a simple wrapper.
The authzwrapper package contains a simple wrapper that restricts access to requests accompanied by authorization bearer tokens created via the OAuth 2 flows supported by roll.
The echo server provides an example of a protected resource.
To try it out, build the echo server and run it on say port 5000.
If you try it without a token, access will be denied:
curl -X PUT -d 'Hello hello echo echo' localhost:5000/echo Unauthorized
If you use the token obtained through the implicit grant flow, access will be granted.
curl -X PUT -d 'Hello hello echo echo' -H 'Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJhcHBsaWNhdGlvbiI6IkhhbmdyeSBCaXJkeiIsImF1ZCI6IjExMS0yMjItMzMzMyIsImlhdCI6MTQ0MTcyNjA2NywianRpIjoiOGZjM2MzNzYtYjMyMy00Yzg5LTdiZTktOWRkZWE3ZWJhNWM2In0.yOjkodiiJtNnXGSoz2lipBgYNyKmQApjKVHPmkiW-peAVhtyQw-q3nnD-H93-vioiq-qvwKp9R4uj1gkPSXJlPJDDj4A6AtqlbbYElQ3K2q9IPPeYiaOR2fJZtLYsIvoDZimGHq_FjZvxDzYZalFSd7BDFeQ5xmhGWczqs6vNNE' localhost:5000/echo Hello hello echo echo