Good instinct. Yes, an attacker being able to supply arbitrary format string is a vulnerability under python.
- The denial of service is probably the most simple to address. In this case, limiting the size of the string or the number of operators within the string will mitigate this issue. There should be a setting where no reasonable user will need to generate a string with more variables than X, and this amount of computation isn't at risk of being exploited in a DoS attack.
- Being able to access attributes within an object could be dangerous.
However, I don't think that the
Objectparent class has any useful information. The object supplied to the format would have to contain something sensitive. In any case, this type of notation can limited with a regular expression. - If the format strings are user supplied then a user might need to know the error message for debugging. However, error mesages can contain senstive information such as local paths or class names. Make sure to limit the information that an attacker can obtain.
Look over the python format string specification and forbid functionality you don't want the user to have with a regex.
Answer from rook on Stack OverflowGood instinct. Yes, an attacker being able to supply arbitrary format string is a vulnerability under python.
- The denial of service is probably the most simple to address. In this case, limiting the size of the string or the number of operators within the string will mitigate this issue. There should be a setting where no reasonable user will need to generate a string with more variables than X, and this amount of computation isn't at risk of being exploited in a DoS attack.
- Being able to access attributes within an object could be dangerous.
However, I don't think that the
Objectparent class has any useful information. The object supplied to the format would have to contain something sensitive. In any case, this type of notation can limited with a regular expression. - If the format strings are user supplied then a user might need to know the error message for debugging. However, error mesages can contain senstive information such as local paths or class names. Make sure to limit the information that an attacker can obtain.
Look over the python format string specification and forbid functionality you don't want the user to have with a regex.
This simple Formatter override blocks users from accessing attributes. It still allows formatting and conversion of types.
from string import Formatter
class SafeFormatter(Formatter):
def get_field(self, field_name, args, kwargs):
if not str.isidentifier(field_name):
raise ValueError(
f"Invalid format string, {field_name!r} is not a valid identifier"
)
return super().get_field(field_name, args, kwargs)
Usage example:
>>> safe_formatter = SafeFormatter()
>>> safe_formatter.format("Number {num} is id={id}", num=1, id="hello")
'Number 1 is hello'
>>> safe_formatter.format("Number {num} is {id.__class__}", num=1, id="hello")
Traceback (most recent call last):
File "<python-input-3>", line 1, in <module>
safe_formatter.format("Number {num} is {id.__class__}", num=1, id="hello")
~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[...omitted...]
File "<python-input-0>", line 6, in get_field
raise ValueError(
f"Invalid format string, {field_name!r} is not a valid identifier"
)
ValueError: Invalid format string, 'id.__class__' is not a valid identifier
>>>
Videos
You could use the partial function from functools which is short, most readable and also describes the coder's intention:
from functools import partial
s = partial("{foo} {bar}".format, foo="FOO")
print s(bar="BAR")
# FOO BAR
If you know in what order you're formatting things:
s = '{foo} {{bar}}'
Use it like this:
ss = s.format(foo='FOO')
print ss
>>> 'FOO {bar}'
print ss.format(bar='BAR')
>>> 'FOO BAR'
You can't specify foo and bar at the same time - you have to do it sequentially.
Python's f-strings are actually safer. Use them!
String formatting may be dangerous when a format string depends on untrusted data. So, when using str.format() or %-formatting, it's important to use static format strings, or to sanitize untrusted parts before applying the formatter function. In contrast, f-strings aren't actually plain strings, but more like syntactic sugar for concatenating strings and expressions. As such, an f-string's format is predetermined and doesn't allow dynamic (potentially untrusted) parts in the first place.
Old-style formatting with str.format()
>>> data_str = 'bob'
>>> format_str = 'hello {name}!'
>>> format_str.format(name=data_str)
'hello bob!'
Here, your Python interpreter doesn't know the difference between a data string and a format string. It just calls a function, str.format(), which runs a replacement algorithm on the format string value at the moment of execution. So, expectedly, the format is just a plain string with curly braces in it:
>>> import dis
>>> dis.dis("'hello {name}!'")
1 0 LOAD_CONST 0 ('hello {name}!')
2 RETURN_VALUE
New-style formatting with f-strings
>>> data_str = 'bob'
>>> f'hello {data_str}!'
'hello bob!'
Here, f'hello {data_str}!' may look like a string constant, but it's not. The interpreter doesn't parse the part between {...} as part of the string that may be expanded later, but as a separate expression:
>>> dis.dis("f'hello {name}!'")
1 0 LOAD_CONST 0 ('hello ')
2 LOAD_NAME 0 (name)
4 FORMAT_VALUE 0
6 LOAD_CONST 1 ('!')
8 BUILD_STRING 3
10 RETURN_VALUE
So, think of "hi {sys.argv[1]}" as (approximately) syntactical sugar for "hi " + sys.argv[1]. At run time, the interpreter doesn't even really know or care that you used an f-string. It just sees instructions to build a string from a the constant "hi " and the formatted value of sys.argv[1].
Vulnerable example
Here is a sample web app which uses str.format() in a vulnerable way:
from http.server import HTTPServer, BaseHTTPRequestHandler
secret = 'abc123'
class Handler(BaseHTTPRequestHandler):
name = 'funtimes'
msg = 'welcome to {site.name}'
def do_GET(self):
res = ('<title>' + self.path + '</title>\n' + self.msg).format(site=self)
self.send_response(200)
self.send_header('content-type', 'text/html')
self.end_headers()
self.wfile.write(res.encode())
HTTPServer(('localhost', 8888), Handler).serve_forever()
$ python3 example.py
$ curl 'http://localhost:8888/test'
<title>/test</title>
welcome to funtimes
Attack
When the res string is built, it usesself.path as part of the format string. Since self.path is user-controlled, we can use it to alter the format string and e.g. exfiltrate the global variable secret:
$ curl -g 'http://localhost:8888/XXX{site.do_GET.__globals__[secret]}'
<title>/XXXabc123</title>
welcome to funtimes
If this basic language feature were this flawed, it probably wouldn't be a feature at all. As long as the contents of the format string are controlled by the programmer at development time, there is nothing that a user can do to abuse them.
The contents of the curly braces are evaluated, but the result of that evaluation is not evaluated again (i.e. sys.argv[1] is evaluated to "1+1", but is not evaluated again, like you have seen).
The problem arises when a user is able to to inject data into a string before it is formatted; see this challenge example. While this is not for an f-string, it is a good demonstration of the attacks that are possible if the user is allowed to control the formatting.