html - Python, lxml and xpath: returns "[<Element x at 0x29a9998>] rather than expected value -


i'm trying scrape td asset management pages (example below; can't post more 2 links) in order retrieve "price on" value, i.e. dollar amount in snippet of html:

<div class="td-layout-grid9 td-layout-column td-layout-column-first"> price on: jun 12, 2015 <br> <strong>$14.54  </strong> <strong class="td-copy-red">-0.01 (-0.07%)</strong> </div>

i hoping achieve python, requests, lxml, , xpath, installed follows:

apt-get update apt-get install python python-pip python-dev gcc build-essential libxml2-dev libxslt-dev libffi-dev libssl-dev pip install lxml pip install requests pip install requests[security]

next, retrieve page did this:

python >>> lxml import html >>> import requests >>> page = requests.get('https://www.tdassetmanagement.com/funddetails.form?fundid=6320&lang=en') >>> tree = html.fromstring(page.text)

finally, attempt made retrieve desired dollar value using xpath of relevant element obtained chrome's "inspect element" tool:

>>> price = tree.xpath('//*[@id="fundcardvo"]/div[2]/div[1]/div[1]/div[1]/strong[1]') >>> print price

unfortunately result [<element strong @ 0x29a9998>] rather expected dollar amount $14.54&nbsp;&nbsp;.

to ensure expected data retrieved initial "requests.get", ran this:

>>> print page.content

the result can seen here: http://pastebin.com/f5c4mfqb.

if paste above html tool: http://videlibri.sourceforge.net/cgi-bin/xidelcgi xpath query //*[@id="fundcardvo"]/div[2]/div[1]/div[1]/div[1]/strong[1] returns dollar amount expected.

any hints or tips how might able use python, lxml, , xpath retrieve desired value element appreciated. if there's different way going obtain same result interested in too.

thanks.

after further googling find out elements (they're lists of things attributes tag or text), followed more googling regarding unicodeencodeerror (see unicodeencodeerror: 'ascii' codec can't encode character u'\xa0' in position 20: ordinal not in range(128)) able obtain desired value this:

>>> priceelement = tree.xpath('//*[@id="fundcardvo"]/div[2]/div[1]/div[1]/div[1]/strong[1]') >>> priceascii = priceelement[0].text >>> price = priceascii.encode('utf-8') >>> print price

thanks nudging me in right direction jonrsharpe.

i still not able determine how obtain list of available attributes element though, tag , text available.

i went on number (without dollar symbol , trailing non-breaking spaces) this:

>>> import re >>> p = re.search('[0-9]{1,3}\.[0-9]{2}', price) >>> price = p.group(0) >>> print price

Comments

Post a Comment

Popular posts from this blog

c# - Validate object ID from GET to POST -

i have method allows user access comment have made on site. have post method lets them update comment. my normal solution send entire comment model through, let them update it, update on database when post back. involve sending commentid through in hiddenfor , can manipulated. how can verify commentid sent in get method same i'm getting in post , not able alter comment wish? how can verify commentid sent in method same i'm getting in post? basically have validate following things in post - user logged-in user. authenticated users post comments. commentid in post , should valid commentid , should present in database. userid associated logged-in user should same userid associated comment . comment should contain userid column, can check @ time of update. to make sure update happens comment has been sent in get - hold commentid in session , in post action compare commentid value in session .
Read more

node.js - Custom Model Validator SailsJS -

when using custom model validator checks record within model. return within spread doesn't seem end control flow. //post.js model /** * custom validator **/ types: { isuservalid: function(user_id) { var promise = require('bluebird'); promise.all([ user.findone({id: user_id}) ]) .spread(function(user) { console.log(user); if (user === null || user === undefined) { console.log('failed'); return false; }else{ console.log('passed'); return true; } }); } }, my response standard validation failed response. { "error": "e_validation", "status": 400, "summary": "1 attribute invalid", "model": "post", "invalidattributes": { "owner": [ { "rule": "isuservalid", ...
Read more

php - Find a regex to take part of Email -

i have following emails need extract part from: bestellnummer xxx: 1 von xxx, 1er pack ------------- anfang der nachricht ------------- foo bar baz foo bar baz // <<<<< need text here ------------- ende der nachricht ------------- ------------- anfang der nachricht ------------- foo bar baz foo bar baz ------------- ende der nachricht ------------- there 0 unlimited occurences of ------------- anfang der nachricht ------------- ------------- ende der nachricht ------------- and i'm able extract first part regex: $re = "/------------- .*? -------------.?(.*?).?------------- .*? -------------/s"; but, i'm quite new on learning regex, i'm pretty sure there must better regex extract part (foo bar baz foo bar baz) of text between ------------- anfang der nachricht ------------- ------------- ende der nachricht ------------- as can in different languages, i'm using .? to match between hyphens. i...
Read more