From the Python 3 documentation:
urllib.parse.quote(string, safe='/', encoding=None, errors=None)
Replace special characters in string using the
%xxescape. Letters, digits, and the characters'_.-~'are never quoted. By default, this function is intended for quoting the path section of a URL. The optional safe parameter specifies additional ASCII characters that should not be quoted — its default value is'/'.
That means passing '' for safe will solve your first issue:
>>> import urllib.parse
>>> urllib.parse.quote('/test')
'/test'
>>> urllib.parse.quote('/test', safe='')
'%2Ftest'
(The function quote was moved from urllib to urllib.parse in Python 3.)
By the way, have a look at urlencode.
About the second issue, there was a bug report about it and it was fixed in Python 3.
For Python 2, you can work around it by encoding as UTF-8 like this:
>>> query = urllib.quote(u"Müller".encode('utf8'))
>>> print urllib.unquote(query).decode('utf8')
Müller
From the Python 3 documentation:
urllib.parse.quote(string, safe='/', encoding=None, errors=None)
Replace special characters in string using the
%xxescape. Letters, digits, and the characters'_.-~'are never quoted. By default, this function is intended for quoting the path section of a URL. The optional safe parameter specifies additional ASCII characters that should not be quoted — its default value is'/'.
That means passing '' for safe will solve your first issue:
>>> import urllib.parse
>>> urllib.parse.quote('/test')
'/test'
>>> urllib.parse.quote('/test', safe='')
'%2Ftest'
(The function quote was moved from urllib to urllib.parse in Python 3.)
By the way, have a look at urlencode.
About the second issue, there was a bug report about it and it was fixed in Python 3.
For Python 2, you can work around it by encoding as UTF-8 like this:
>>> query = urllib.quote(u"Müller".encode('utf8'))
>>> print urllib.unquote(query).decode('utf8')
Müller
In Python 3, urllib.quote has been moved to urllib.parse.quote, and it does handle Unicode by default.
>>> from urllib.parse import quote
>>> quote('/test')
'/test'
>>> quote('/test', safe='')
'%2Ftest'
>>> quote('/El Niño/')
'/El%20Ni%C3%B1o/'
Using urllib package (import urllib) :
Python 2.7
From official documentation :
urllib.unquote(string)Replace
%xxescapes by their single-character equivalent.Example:
unquote('/%7Econnolly/')yields'/~connolly/'.
Python 3
From official documentation :
urllib.parse.unquote(string, encoding='utf-8', errors='replace')[…]
Example:
unquote('/El%20Ni%C3%B1o/')yields'/El Niño/'.
And if you are using Python3 you could use:
import urllib.parse
urllib.parse.unquote(url)
You can escape certain characters by specifying them explicitly as safe argument value
urllib.quote(str, safe='~()*!.\'')
More : https://docs.python.org/3.0/library/urllib.parse.html#urllib.parse.quote
You can do this by adding the query params as a string before hitting the endpoint.
I have used requests for making a request.
For example:
GET Request
import requests
url = "https://www.example.com/?"
query = "A=B,C"
url_final = url + query
url = requests.get(url_final)
print(url.url)
# https://www.example.com/?A=B,C
The comma (along with some other characters) is defined in RFC 3986 as a reserved character. This means the comma has defined meaning at various parts in a URL, and if it is not being used in that context it needs to be percent-encoded.
That said, the query parameter doesn't give the comma any special syntax, so in query parameters, we probably shouldn't be encoding it. That said, it's not entirely Requests' fault: the parameters are encoded using urllib.urlencode(), which is what is percent-encoding the query parameters.
This isn't easy to fix though, because some web services use , and some use %2C, and neither is wrong. You might just have to handle this encoding yourself.