linux - Bash: Unexpected parallel behavior when reading arguments from file using xargs -


previous

this follow-up this question.

specs

my system dedicated server running ubuntu desktop, release 12.04 (precise) 64-bit, 3.14.32-xxxx-std-ipv6-64. neither release or kernel can upgraded, can install package.

problem

the problem discribed in question above seems solved, doesn't work me. i've installed latest lftp , parallel packages , seem work fine themselves.

  • running lftp works fine.
  • running ./job.sh ftp.microsoft.com works fine, needed chmod -x script
  • running sed 's/|.*$//' end_unique.txt | xargs parallel -j20 ./job.sh ::: not work , produces bash errors in form of /bin/bash: <server>: command not found.

to simplify things, cleaned input file end_unique.txt, has following format each line:

<server>

each line ends in crlf, because imported windows server.

edit 1:

this job.sh script:

#/bin/sh server="$1" lftp -e "find .; exit" "$server" >"$server-files.txt"

edit 2:

i took file , ran against fromdos. should standard unix format, 1 server per line. keep in mind server in file can vary in format:

ftp.server.com www.server.com server.com 123.456.789.190

etc. of servers ftp servers, accessible ftp://<serverfromfile>/.

with :::, parallel expects list of arguments needs complete commands it's going run appear on command line, in

parallel -j20 ./job.sh ::: server1 server2 server3

without ::: reads arguments stdin, serves better in case. can say

parallel -j20 ./job.sh < end_unique.txt

addendum: things can go wrong

make 2 things:

  1. that using gnu parallel , not version (such 1 moreutils), because (as far i'm aware) gnu version supports reading argument list stdin, and

  2. that gnu parallel not configured disable gnu extensions. turned out, after lengthy discussion in comments, disabled default on ubuntu 12.04, not inconceivable sort of thing might found elsewhere (particularly downstream ubuntu). such configuration can hide in

    • the environment variable $parallel,
    • /etc/parallel/config, or
    • ~/.parallel/config

if gnu version of parallel not available you, , if argument list not long shell , none of arguments in contain whitespaces, same thing moreutils parallel is

parallel -j20 job.sh -- $(cat end_unique.txt)

this did not work op because file contained more servers shell willing put command line, might work others similar problems.


Comments

Post a Comment

Popular posts from this blog

c# - Validate object ID from GET to POST -

i have method allows user access comment have made on site. have post method lets them update comment. my normal solution send entire comment model through, let them update it, update on database when post back. involve sending commentid through in hiddenfor , can manipulated. how can verify commentid sent in get method same i'm getting in post , not able alter comment wish? how can verify commentid sent in method same i'm getting in post? basically have validate following things in post - user logged-in user. authenticated users post comments. commentid in post , should valid commentid , should present in database. userid associated logged-in user should same userid associated comment . comment should contain userid column, can check @ time of update. to make sure update happens comment has been sent in get - hold commentid in session , in post action compare commentid value in session .
Read more

node.js - Custom Model Validator SailsJS -

when using custom model validator checks record within model. return within spread doesn't seem end control flow. //post.js model /** * custom validator **/ types: { isuservalid: function(user_id) { var promise = require('bluebird'); promise.all([ user.findone({id: user_id}) ]) .spread(function(user) { console.log(user); if (user === null || user === undefined) { console.log('failed'); return false; }else{ console.log('passed'); return true; } }); } }, my response standard validation failed response. { "error": "e_validation", "status": 400, "summary": "1 attribute invalid", "model": "post", "invalidattributes": { "owner": [ { "rule": "isuservalid", ...
Read more

php - Find a regex to take part of Email -

i have following emails need extract part from: bestellnummer xxx: 1 von xxx, 1er pack ------------- anfang der nachricht ------------- foo bar baz foo bar baz // <<<<< need text here ------------- ende der nachricht ------------- ------------- anfang der nachricht ------------- foo bar baz foo bar baz ------------- ende der nachricht ------------- there 0 unlimited occurences of ------------- anfang der nachricht ------------- ------------- ende der nachricht ------------- and i'm able extract first part regex: $re = "/------------- .*? -------------.?(.*?).?------------- .*? -------------/s"; but, i'm quite new on learning regex, i'm pretty sure there must better regex extract part (foo bar baz foo bar baz) of text between ------------- anfang der nachricht ------------- ------------- ende der nachricht ------------- as can in different languages, i'm using .? to match between hyphens. i...
Read more