Good instinct. Yes, an attacker being able to supply arbitrary format string is a vulnerability under python.
- The denial of service is probably the most simple to address. In this case, limiting the size of the string or the number of operators within the string will mitigate this issue. There should be a setting where no reasonable user will need to generate a string with more variables than X, and this amount of computation isn't at risk of being exploited in a DoS attack.
- Being able to access attributes within an object could be dangerous.
However, I don't think that the
Objectparent class has any useful information. The object supplied to the format would have to contain something sensitive. In any case, this type of notation can limited with a regular expression. - If the format strings are user supplied then a user might need to know the error message for debugging. However, error mesages can contain senstive information such as local paths or class names. Make sure to limit the information that an attacker can obtain.
Look over the python format string specification and forbid functionality you don't want the user to have with a regex.
Answer from rook on Stack OverflowGood instinct. Yes, an attacker being able to supply arbitrary format string is a vulnerability under python.
- The denial of service is probably the most simple to address. In this case, limiting the size of the string or the number of operators within the string will mitigate this issue. There should be a setting where no reasonable user will need to generate a string with more variables than X, and this amount of computation isn't at risk of being exploited in a DoS attack.
- Being able to access attributes within an object could be dangerous.
However, I don't think that the
Objectparent class has any useful information. The object supplied to the format would have to contain something sensitive. In any case, this type of notation can limited with a regular expression. - If the format strings are user supplied then a user might need to know the error message for debugging. However, error mesages can contain senstive information such as local paths or class names. Make sure to limit the information that an attacker can obtain.
Look over the python format string specification and forbid functionality you don't want the user to have with a regex.
This simple Formatter override blocks users from accessing attributes. It still allows formatting and conversion of types.
from string import Formatter
class SafeFormatter(Formatter):
def get_field(self, field_name, args, kwargs):
if not str.isidentifier(field_name):
raise ValueError(
f"Invalid format string, {field_name!r} is not a valid identifier"
)
return super().get_field(field_name, args, kwargs)
Usage example:
>>> safe_formatter = SafeFormatter()
>>> safe_formatter.format("Number {num} is id={id}", num=1, id="hello")
'Number 1 is hello'
>>> safe_formatter.format("Number {num} is {id.__class__}", num=1, id="hello")
Traceback (most recent call last):
File "<python-input-3>", line 1, in <module>
safe_formatter.format("Number {num} is {id.__class__}", num=1, id="hello")
~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[...omitted...]
File "<python-input-0>", line 6, in get_field
raise ValueError(
f"Invalid format string, {field_name!r} is not a valid identifier"
)
ValueError: Invalid format string, 'id.__class__' is not a valid identifier
>>>
Good instinct. Yes, an attacker being able to supply arbitrary format string is a vulnerability under python.
- The denial of service is probably the most simple to address. In this case, limiting the size of the string or the number of operators within the string will mitigate this issue. There should be a setting where no reasonable user will need to generate a string with more variables than X, and this amount of computation isn't at risk of being exploited in a DoS attack.
- Being able to access attributes within an object could be dangerous.
However, I don't think that the
Objectparent class has any useful information. The object supplied to the format would have to contain something sensitive. In any case, this type of notation can limited with a regular expression. - If the format strings are user supplied then a user might need to know the error message for debugging. However, error mesages can contain senstive information such as local paths or class names. Make sure to limit the information that an attacker can obtain.
Look over the python format string specification and forbid functionality you don't want the user to have with a regex.
Answer from rook on Stack OverflowVideos
Python's f-strings are actually safer. Use them!
String formatting may be dangerous when a format string depends on untrusted data. So, when using str.format() or %-formatting, it's important to use static format strings, or to sanitize untrusted parts before applying the formatter function. In contrast, f-strings aren't actually plain strings, but more like syntactic sugar for concatenating strings and expressions. As such, an f-string's format is predetermined and doesn't allow dynamic (potentially untrusted) parts in the first place.
Old-style formatting with str.format()
>>> data_str = 'bob'
>>> format_str = 'hello {name}!'
>>> format_str.format(name=data_str)
'hello bob!'
Here, your Python interpreter doesn't know the difference between a data string and a format string. It just calls a function, str.format(), which runs a replacement algorithm on the format string value at the moment of execution. So, expectedly, the format is just a plain string with curly braces in it:
>>> import dis
>>> dis.dis("'hello {name}!'")
1 0 LOAD_CONST 0 ('hello {name}!')
2 RETURN_VALUE
New-style formatting with f-strings
>>> data_str = 'bob'
>>> f'hello {data_str}!'
'hello bob!'
Here, f'hello {data_str}!' may look like a string constant, but it's not. The interpreter doesn't parse the part between {...} as part of the string that may be expanded later, but as a separate expression:
>>> dis.dis("f'hello {name}!'")
1 0 LOAD_CONST 0 ('hello ')
2 LOAD_NAME 0 (name)
4 FORMAT_VALUE 0
6 LOAD_CONST 1 ('!')
8 BUILD_STRING 3
10 RETURN_VALUE
So, think of "hi {sys.argv[1]}" as (approximately) syntactical sugar for "hi " + sys.argv[1]. At run time, the interpreter doesn't even really know or care that you used an f-string. It just sees instructions to build a string from a the constant "hi " and the formatted value of sys.argv[1].
Vulnerable example
Here is a sample web app which uses str.format() in a vulnerable way:
from http.server import HTTPServer, BaseHTTPRequestHandler
secret = 'abc123'
class Handler(BaseHTTPRequestHandler):
name = 'funtimes'
msg = 'welcome to {site.name}'
def do_GET(self):
res = ('<title>' + self.path + '</title>\n' + self.msg).format(site=self)
self.send_response(200)
self.send_header('content-type', 'text/html')
self.end_headers()
self.wfile.write(res.encode())
HTTPServer(('localhost', 8888), Handler).serve_forever()
$ python3 example.py
$ curl 'http://localhost:8888/test'
<title>/test</title>
welcome to funtimes
Attack
When the res string is built, it usesself.path as part of the format string. Since self.path is user-controlled, we can use it to alter the format string and e.g. exfiltrate the global variable secret:
$ curl -g 'http://localhost:8888/XXX{site.do_GET.__globals__[secret]}'
<title>/XXXabc123</title>
welcome to funtimes
If this basic language feature were this flawed, it probably wouldn't be a feature at all. As long as the contents of the format string are controlled by the programmer at development time, there is nothing that a user can do to abuse them.
The contents of the curly braces are evaluated, but the result of that evaluation is not evaluated again (i.e. sys.argv[1] is evaluated to "1+1", but is not evaluated again, like you have seen).
The problem arises when a user is able to to inject data into a string before it is formatted; see this challenge example. While this is not for an f-string, it is a good demonstration of the attacks that are possible if the user is allowed to control the formatting.
The Clang & GCC compilers check that format strings and the supplied arguments conform, they cannot do this if the format string is not a literal - hence the error message you see as you are obtaining the format string from the bundle.
To address this issue there is an attribute, format_arg(n) (docs), to mark functions which take a format string; alter it in some way without changing the actual format specifiers, e.g translate it; and then return it. Cocoa provides the convenient macro NS_FORMAT_ARG(n) for this attribute.
To fix your problem you need to do two things:
Wrap up the call to
NSBundlein a function with this attribute specified; andChange your "key" to include the format specifiers.
Second first, your strings file should contain:
"name %@" = "My name is %@"
so the key has the same format specifiers as the result (if you need to reorder the specifiers for a particular language you use positional format specifiers).
Now define a simple function to do the lookup, attributing it as a format translation function. Note we mark it as static inline, using the macro NS_INLINE as a hint to the compiler to both inline it into your macro expansion; the static allows you to include it in multiple files without symbol clashes:
NS_INLINE NSString *localize(NSString *string) NS_FORMAT_ARGUMENT(1);
NSString *localize(NSString *string)
{
return [[NSBundle mainBundle] localizedStringForKey:string value:@"" table:nil];
}
And your macro becomes:
#define GetLocalStr(key, ...) [NSString stringWithFormat:localize(key), ##__VA_ARGS__]
Now when you:
GetLocalStr(@"name %@", @"Foo")
You will get both the localised format string and format checking.
Update
After Greg's comment I went back and checked - I had reproduced your error and so assumed it was down to a missing attribute. However as Greg points out localizedStringForKey:value:table: already has the attribute, so why the error? What I had absentmindedly done in reproducing your error was:
NSLog( GetLocalStr( @"name %@", @"Foo" ) );
and the compiler pointed at the macro definition and not that line - I should have spotted the compiler was misleading me.
So where does that leave you? Maybe you've done something similar? The key is that a format string must either be a literal or the result of a function/method attributed as a format translating function. And don't forget, you must also had the format specifier to your key as above.
Update 2
After your additional comments what you need to use is function, rather than a macro, along with the format attribute, for which Cocoa provides the convenient NS_FORMAT_FUNCTION(f,a) macro. This attribute informs the compiler that the function is a formatting one, the value of f is the number of the format string and a is the number of the first argument to the format. This gives the function declaration:
NSString *GetLocalStr(NSString *key, ...) NS_FORMAT_FUNCTION(1,2);
and the definition (assuming ARC):
NSString *GetLocalStr(NSString *key, ...)
{
va_list args;
va_start(args, key);
NSString *format = [[NSBundle mainBundle] localizedStringForKey:key value:@"" table:nil];
NSString *result = [[NSString alloc] initWithFormat:format arguments:args];
va_end (args);
return result;
}
(which is essentially the same as @A-Live's).
Uses of this will be checked appropriately, for example:
int x;
...
NSString *s1 = GetLocalStr(@"name = %d", x); // OK
NSString *s2 = GetLocalStr(@"name = %d"); // compile warning - More '%" conversions than data arguments
NSString *s3 = GetLocalStr(@"name", x); // compile warning - Data argument not used by format string
NSString *s4 = GetLocalStr(@"name"); // OK
This variant produces no warnings (as there's always a va_list):
NSString* GetLocalStr1(NSString *formatKey, ...) {
va_list ap;
va_start(ap, formatKey);
NSString * format = [[NSBundle mainBundle] localizedStringForKey:formatKey value:@"" table:nil];
NSString *result = [[NSString alloc] initWithFormat:format arguments:ap];
va_end (ap);
return [result autorelease];
}
...
__unused NSString * str = GetLocalStr1( @"name", @"Foo" );
__unused NSString * str1 = GetLocalStr1( @"TestNoArgs" );
__unused NSString * str2 = GetLocalStr1( @"TestArgs", @"Foo" );
NSLog(@"%@", str);
NSLog(@"%@", str1);
NSLog(@"%@", str2);
Result:
my name is Foo
TestNoArgs
Hello world Foo
It doesn't answer the question exactly but might help you to avoid warnings until the solution is found.
The warning is because Rprintf(msg) isn't being called with a string literal. In your case, you could change this to Rprintf("%s", msg); to explicitly show that msg is a string.
The format string refers to the first argument of Rprintf, and that should be a string literal:
Rprintf("ANN WARNING:");
Rprintf("%s", msg);
Or:
Rprintf("ANN WARNING:\n%s", msg);
See also the documentation of the -Wformat-security compiler flag in the GCC manual. (Yes, your warning was issued by Clang, but the Clang manual is much less helpful here.)