python - Convert url for crawler -


I'm working on crawler Usually, when I type url1 in my browser, then the browser converted it to url2 Does. How can I do this in Python?

url1: www.odevsitesi.com/ara.asp?kelime=doğanın dengesinin bozulması

url2: www.odevsitesi.com/ara.asp? Kelime = do% F0an% FDn% 20dengesinin% 20bozulmas% FD

You can translate the URL properly (ISO-8859-9 in your case), separate it into parts, enter urllib.Query part and keep it together again. I.e .:

  & gt; & Gt; & Gt; Import urlparse & gt; & Gt; & Gt; Import urllib & gt; & Gt; & Gt; X = u'http: //www.odevsitesi.com/ara.asp? Kelime = doğanın dengesinin bozulması '& gt; & Gt; & Gt; Y = x.encode ('iso-8859-9') & gt; & Gt; & Gt; # Just to show how the division of y is visible (we can handle it as a tuple): & gt; & Gt; & Gt; Urlparse.urlsplit (y) SplitResult (schema = 'hp', netlok = 'www.odevsitesi.com', path = '/ era.asp', query = 'kelim = dot \ xf0an \ xfdn dengesinin bozulmas \ xfd', slice = '') & Gt; & Gt; & Gt; Z = urlparse.urlsplit (y) & gt; & Gt; & Gt; Quoted = z [: 3] + (urllib.quote (z.query), z.fragment) & gt; & Gt; & Gt; # Now just to show you how the 'cited' tuple looks like: & gt; & Gt; & Gt; Quoted ('http', 'www.odevsitesi.com', '/ AAP', 'kelim% 3Ddo% F0an% FDn% 20dengesinin% 20bozulmas% FD', '')) & Gt; & Gt; # And finally putting it back together: & gt; & Gt; & Gt; Urlparse.urlunsplit (quoted) 'http://www.odevsitesi.com/ara.asp?kelime%3Ddo%F0an%FDn%20dengesinin%20bozulmas%FD'  

Comments

Popular posts from this blog

windows - Heroku throws SQLITE3 Read only exception -

lex - Building a lexical Analyzer in Java -

python - rename keys in a dictionary -