Skip to content

smalleni/etcd-load-generator

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

38 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ETCD Load Testing

Note : This test, right now, supports only a single node etcd cluster.

This is a etcd load test module written in go-language. It basically creates artificial load on a running etcd instance. The test runs for a random set of keys and values, which can be specified in the configuration file. You can configure many other parameters in the config file. See the sample config file etcd_load.cfg for more information.

##Features

  • Memory information
  • Gives the memory information about the running etcd instance -- before, after and difference, when requests are made
  • To get this info, use the "-mem" flag while running the module
  • Also, if the instance is running on a remote machine, then you need to use the "-remote" flag as well.
  • The memory information if obtained using the following "pmap" command. Basically the RSS part of the total memory usage, is used.
  • It also supports secure etcd-instances, for example the secure etcd running under openshift. For running the load test on a secure etcd instance, you to specify the following parameters/arguments :
    • The flag "-secure" : It is a must
    • Optional parameters : "-capath", "-ca", "-cert", "-cakey"
    • These are optional because they can be set in the config file as well
      • capath : Path of the folder where all the certificates are located
      • ca : The CA file
      • cert : The client certificate file
      • cakey : the cliend key file
	$ pmap -x $(pidof etcd) | tail -n1 | awk '{print $4}'

Sample Output of : pmap -x $pid_of_etcd

```	  
  3084:   etcd -addr 10.70.1.148:4001
Address           Kbytes     RSS   Dirty Mode  Mapping
0000000000400000    3020    2764       0 r-x-- etcd
00000000006f3000    3316    2996       0 r---- etcd
0000000000a30000     120     116      28 rw--- etcd
0000000000a4e000     116      96      96 rw---   [ anon ]
000000c000000000      36      36      36 rw---   [ anon ]
000000c207de8000   36448   15188   15188 rw---   [ anon ]
00007fcc92ad1000    1728     660     660 rw---   [ anon ]
00007ffc139c0000     132      12      12 rw---   [ stack ]
00007ffc139ed000       8       0       0 r----   [ anon ]
00007ffc139ef000       8       4       0 r-x--   [ anon ]
ffffffffff600000       4       0       0 r-x--   [ anon ]
---------------- ------- ------- ------- 
total kB           44936   21872   16020
```
  • Key value distribution
  • This feature basically allows you to specify the distribution of the key-values, that is how many keys lie in a particular value range, specified by "value-range" parameter under section : "section-args", in etcd_load.cfg
  • See the "pct" parameter under the section :"section-args", in etcd_load.cfg
  • Value range
  • This allows you to specify value ranges, which will be used for the -- Key value distribution feature.
  • Note : length (Value Range) = length (Key Value distribution) + 1
  • There are other configuration options as well, like -- log-file, remote-flag, remote-host-user, etcd. . Some of these can be specified using commandline flags as well, in which case the flags will override the cfg-file values. To know more about them see the default config file -- "etcd_load.cfg" and for help regarding flags, use - go run etcd_load.go -h

##Setup

  • Make sure that the following packages are in your GOPATH. Set your GOPATH if its not already set. In go, to get a "package" you can simply do : go get package. In this use case do it under your GOPATH.
  • Packages :
  • "github.com/coreos/go-etcd/etcd"
  • "code.google.com/p/gcfg"
  • "code.google.com/p/go.crypto/ssh"
  • Example :
  • go get "code.google.com/p/gcfg"
  • Set up a default config file, like the one available in the repo.
  • Below is a sample config file, for more details take a look at etcd_load.cfg

Sample Config File

		[section-args]
		etcdhost="127.0.0.1"
		etcdport="4001"
		operation=create
		keycount="100"
		operation-count="200"
		log-file=log
		threads=5 
		pct="5,74,10,10,1"
		value-range="0,256,512,1024,8192,204800"
		remote-flag=False
		ssh-port=22
		remote-host-user=root
		cpath=/root/username/certificates
		ca="ca.crt"
		client-crt="etcd-client.crt"
		client-key="etcd-client.key"

##Build

  • Now, to run etcd_load test, use the following steps
 $ go build etcd_load.go report.go

##Running the Test

  • ./etcd_load -c config_file [flag]
  • flags : -help , -h , -p , -o , -k , -oc , -log , -mem , -remote
  • For details regarding the flags , use :
    $ go run etcd_load.go -help
    
  • Examples :
[remote etcd instance]
```
$ ./etcd_load -c etcd_load.cfg -mem -remote -o create  
$ ./etcd_load -c etcd_load.cfg -h 10.10.10.1 -p 4001 -o create 
```
[local etcd instance]
```
$ ./etcd_load -c etcd_load.cfg -h 127.0.0.1 -o create 
```
[secure etcd instance]
```
$ ./etcd_load -c etcd_load.cfg -h secure_etcd_host -o create -secure -capath=/path/to/certificates         -ca=ca.crt -cert=client.crt -cakey=client.key
```
[Note]
```
The "-c" flag is compulsory, that is you need to have a default 
config file that must be input using the -c flag
Also if the instance is secure, then it is necessary to specify "-secure"
flag.
To know more about the flags :: do -- go run etcd_load.go -h
```

##Result

  • You can find more runtime details in the log file.
  • The commandline the report looks like :

Summary:
  Total:	2.8454 secs.
  Slowest:	0.1493 secs.
  Fastest:	0.0001 secs.
  Average:	0.0258 secs.
  Requests/sec:	35.1449

Response time histogram:
  0.000 [1]		|
  0.015 [17]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  0.030 [48]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  0.045 [27]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  0.060 [3]		|∎∎
  0.075 [2]		|∎
  0.090 [0]		|
  0.105 [1]		|
  0.119 [0]		|
  0.134 [0]		|
  0.149 [1]		|

Latency distribution:
  10% in 0.0006 secs.
  25% in 0.0226 secs.
  50% in 0.0240 secs.
  75% in 0.0308 secs.
  90% in 0.0332 secs.
  95% in 0.0481 secs.
  99% in 0.1493 secs.

##Credit ::

About

generate load on you etcd instance for testing purpose

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Go 100.0%