A process that scales up IronWorkers based on IronMQ queue sizes.
This worker will live for a default of 30 minutes and check queue sizes and worker stats every 10 seconds. Depending on your trigger settings, the process will enqueue 0..N tasks then wait for the next check interval.
If you do not have Go:
❯❯❯ docker run --rm -it -v $PWD:/go/src/a -w /go/src/a -e "CONFIG_FILE=config.json" iron/go:dev sh -c "go get ./... && go run main.go"
If you have Go:
❯❯❯ go get github.com/NickSardo/triggers
❯❯❯ cd $GOPATH/src/github.com/NickSardo/triggers
# Create a scale.json
❯❯❯ vi scale.json
# Run
❯❯❯ go run main.go
You can define the configuration in two places, but configuration set via scale.json
has precedence.
scale.json
file located in the same directory as the triggers executable- Visit hud.iron.io and modify the configuration for the dequeuer worker.
{
"envs": {
"iron_mq":{
"host": "mq-aws-us-east-1-1.iron.io",
"api_version": "3"
},
"projectA":{
"token": "AAA",
"project_id": "BBB",
}
},
"alerts": [
{
"queueName": "sampleQueue",
"queueEnv": "projectA",
"workerName": "dequeuer",
"workerEnv": "projectA",
"triggers":[
{
"type": "ratio",
"value": 10
}
],
"cluster": "ABC",
"interval": 3,
"priority": 1,
"min": 3,
"max": 20
}
],
"cacheEnv":"projectA",
"runtime": 1800
}
envs
: named map of objects describing different Iron.io environments. See dev.iron.io for more information about configuration
alerts
: Array of objects, each object connects a queue to monitor and a worker to start
queueEnv
or workerEnv
: these values are found under defined environments above
cluster
: tasks for this worker are spawned on this cluster
priority
: priority for created tasks. 0 - normal, 1 - medium, 2 - highest (optional, default: 0)
min
: minimum number of tasks queued or running regardless of triggers (optional, default: none)
max
: maximum number of tasks. No more tasks will be created regardless of triggers (optional, default: none)
cacheEnv
: this scaler code caches the last known queue size, provide an environment to Iron Cache
interval
: polling frequency for checking the queue sizes & launching tasks (optional, default: 10 sec)
runtime
: seconds this task will live for (optional, default: 1800 seconds)
Triggers will tell the scaler how many tasks to spawn. Given more than one trigger to an alert object, the max number of tasks generated among each of the triggers will be used.
Have one task queued or running for every {value} messages on the queue.
Example:
Given a ratio value of 10, # of tasks = queue size / 10
If the queue size >= {value}, have at least one task queued or running.
Example:
Given a fixed value of 50, have at least one task queued or running starting at time = 20 min.
If the queue size grows by {value} messages since the last check time, spawn 1 task. Note this is affected by the interval
parameter.
Given a progressive value of 20 and a check interval of 1 minute, spawn two tasks.
# of tasks created = (current - prev) / 20
❯ (105 - 60) / 20
❯ 2 tasks created
Triggers will send a copy of the trigger configuration that caused this task to launch. If the min
setting caused tasks to be created, a trigger is created with type min
.
{
"type": "min",
"value": 1
}
If a ratio
trigger caused N
tasks to be created, each task will have the ratio
setting as defined in the configuration in the payload. See the printer
folder for an example of reading the payload into a struct.
The following process describes one way to deploy your code to IronWorker - bundling the executable (and config file) into a docker image. Another option is to zip your files and run iron worker upload -zip myZip.zip -name triggers iron/base
If you want to skip compiling the code yourself, you can go to step 3 and use nicksardo/triggers:0.1
❯❯❯ docker run --rm -it -v $PWD:/go/src/a -w /go/src/a iron/go:dev sh -c "go get ./... && go build -o triggers"
If you're using scale.json
, modify the enclosed Dockerfile to ADD
the file during image build.
❯❯❯ docker build -t {{youraccount}}/triggers:0.1 .
❯❯❯ docker push {{youraccount}}/triggers:0.1
❯❯❯ vi someConfig.json # create a config file called "someConfig.json" in an empty directory
❯❯❯ docker run --rm -it -e "CONFIG_FILE=someConfig.json" -v $PWD:/app {{youraccount}}/triggers
Specifying the environment variable CONFIG_FILE
isn't necessary if you built the image with scale.json
.
# Upload
❯❯❯ iron register {{youraccount}}/triggers:0.1
If you didn't deploy a scale.json
, visit hud.iron.io and modify the configuration for the {{youraccount}}/triggers
code package.
# Schedule this to run every 30 minutes via CLI or HUD
❯❯❯ iron worker schedule -cluster default -run-every 1800 {{youraccount}}/triggers