elasticsearch - Docker container http requests limit -


i'm new docker so, likely, i'm missing something.

i'm running container elasticsearch, using image.

i'm able setup everyhing correctly. after using script developed collegue in order insert data, querying mysql database , making http requests .

problem is, many of requests stuck until fails. if netstat -tn | grep 9200 get:

tcp6       0      0 ::1:58436               ::1:9200                time_wait   tcp6       0      0 ::1:59274               ::1:9200                time_wait   ... tcp6       0      0 ::1:58436               ::1:9200                time_wait   tcp6       0      0 ::1:59274               ::1:9200                time_wait

with lot of requests. @ point i'm not sure if it's related elastic search or docker. not happen if elasticsearch instaleld on machine.

some info:

$ docker version client version: 1.6.2 client api version: 1.18 go version (client): go1.4.2 git commit (client): 7c8fca2 os/arch (client): linux/amd64 server version: 1.6.2 server api version: 1.18 go version (server): go1.4.2 git commit (server): 7c8fca2 os/arch (server): linux/amd64  $ docker info containers: 6 images: 103 storage driver: devicemapper  pool name: docker-252:1-9188072-pool  pool blocksize: 65.54 kb  backing filesystem: extfs  data file: /dev/loop0  metadata file: /dev/loop1  data space used: 4.255 gb  data space total: 107.4 gb  data space available: 103.1 gb  metadata space used: 6.758 mb  metadata space total: 2.147 gb  metadata space available: 2.141 gb  udev sync supported: false  data loop file: /var/lib/docker/devicemapper/devicemapper/data  metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata  library version: 1.02.82-git (2013-10-04) execution driver: native-0.2 kernel version: 3.14.22-031422-generic operating system: ubuntu 14.04.2 lts cpus: 4 total memory: 15.37 gib  $ docker logs elasticsearch [2015-06-15 09:10:33,761][info ][node                     ] [energizer] version[1.6.0], pid[1], build[cdd3ac4/2015-06-09t13:36:34z] [2015-06-15 09:10:33,762][info ][node                     ] [energizer] initializing ... [2015-06-15 09:10:33,766][info ][plugins                  ] [energizer] loaded [], sites [] [2015-06-15 09:10:33,792][info ][env                      ] [energizer] using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/mapper/ubuntu--vg-root)]], net usable_space [145.3gb], net total_space [204.3gb], types [ext4] [2015-06-15 09:10:35,516][info ][node                     ] [energizer] initialized [2015-06-15 09:10:35,516][info ][node                     ] [energizer] starting ... [2015-06-15 09:10:35,642][info ][transport                ] [energizer] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/172.17.0.5:9300]} [2015-06-15 09:10:35,657][info ][discovery                ] [energizer] elasticsearch/y1zfiri4qo21zrhci-btxa [2015-06-15 09:10:39,426][info ][cluster.service          ] [energizer] new_master [energizer][y1zfiri4qo21zrhci-btxa][76dea3e6d424][inet[/172.17.0.5:9300]], reason: zen-disco-join (elected_as_master) [2015-06-15 09:10:39,446][info ][http                     ] [energizer] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/172.17.0.5:9200]} [2015-06-15 09:10:39,446][info ][node                     ] [energizer] started [2015-06-15 09:10:39,479][info ][gateway                  ] [energizer] recovered [0] indices cluster_state

the important part of script:

for package in c.fetchall():     id_package, tracking_number, order_number, payment_info, shipment_provider_name, package_status_name=package     el['tracking_number'] = tracking_number     el['order_number'] = order_number     el['payment_info'] = payment_info     el['shipment_provider_name'] = shipment_provider_name     el['package_status_name'] = package_status_name      requests.put("http://localhost:9200/packages/package/%s/_create"%(id_package), json=el)

so, wasn't problem either docker or elastic. recap, same script throwning put requests @ elasticsearch setup locally worked, when throwning @ container elasticsearch failed after few thousand documents (20k). note overal number of documents roughtly 800k.

so, happend? when setup somethig running on localhost , make request (in case put request) request goes through loopback interface. in pratice ths means no tcp connection gets created making lot faster.

when docker container setup, ports bound host. although script still makes requests localhost on desired port, tcp connection gets created between host , docker container through docker0 interface. comes @ expense of 2 things:

  • the time setup tcp connection
  • time_wait state

this more realistic scenario. setup elasticsearch on machine , did exact same test , got, expected, same result.

the problem sending requests , each of them creating new connection. due way tcp works, connections cannot closed immediately. meant using available connections until got none use because rate of creation higher actual close rate.

three suggestions fix this:

  1. pause requests every once in while. maybe put sleep @ every x requests making possible time_wait pass , connection closing
  2. send the connection: close header: option sender signal connection closed after completion of response.
  3. reuse connection(s).

i ended going option 3) , rewrote collegue's script , reusing same tcp connection.


Comments

Post a Comment

Popular posts from this blog

javascript - Google App Script ContentService downloadAsFile not working -

i have web app developed using google app script htmlservice , html form, populating excel sheet in google drive using spreadsheetapp . , 1 section calling contentservice download data excel file. function doget(e) { // read excel sheet //getappfile(); // render application html template return htmlservice.createtemplatefromfile('index').evaluate() .settitle('go smart') .setsandboxmode(htmlservice.sandboxmode.iframe); } function downloaddoublequatecsvfile() { var sheetid = propertiesservice.getscriptproperties().getproperty('sheetid'); var ss = spreadsheetapp.openbyid(sheetid).getactivesheet(); //var ss = spreadsheetapp.getactivespreadsheet().getactivesheet(); var maxcolumn = ss.getlastcolumn(); var maxrow = ss.getlastrow(); var data = ss.getrange(1, 1, maxrow, maxcolumn).getvalues(); if (data.length > 1) { var csv = ""; (var row = 0; row < data.length; row++) { (var c...
Read more

javascript - Function overwritting -

is there way overwrite function or event handling in function? using addeventlistener, can have many event handlers on element want var test=document.getelementbyid("div"); test.addeventlistener("click",function(){alert("im first");},false); test.addeventlistener("click",function(){alert("im second");},false); // alerts "im first" , after "im second" but if want overwrite function/event example based on client width example this function one() { alert("first"); } function two() { alert("second"); } window.onresize = function() { var test = document.getelementbyid("div"); if (window.innerwidth > 500) { test.onclick = 1 } else { test.onclick = two; } } is possible in javascript? in case use effective little known approach. addeventlistener can accept object property handleevent event handler. in case it's easy overwr...
Read more

php - Find a regex to take part of Email -

i have following emails need extract part from: bestellnummer xxx: 1 von xxx, 1er pack ------------- anfang der nachricht ------------- foo bar baz foo bar baz // <<<<< need text here ------------- ende der nachricht ------------- ------------- anfang der nachricht ------------- foo bar baz foo bar baz ------------- ende der nachricht ------------- there 0 unlimited occurences of ------------- anfang der nachricht ------------- ------------- ende der nachricht ------------- and i'm able extract first part regex: $re = "/------------- .*? -------------.?(.*?).?------------- .*? -------------/s"; but, i'm quite new on learning regex, i'm pretty sure there must better regex extract part (foo bar baz foo bar baz) of text between ------------- anfang der nachricht ------------- ------------- ende der nachricht ------------- as can in different languages, i'm using .? to match between hyphens. i...
Read more