You are correct.
You only need to use the secure_filename function if you are using the value of request.files['file'].filename to build a filepath destined for your filesystem - for example as an argument to os.path.join.
As you're using a UUID for the filename, the user input value is disregarded anyway.
Even without S3, it would also be safe NOT to use secure_filename if you used a UUID as the filename segment of the filepath on your local filesystem. For example:
uploaded_file = request.files['file']
if uploaded_file:
file_uuid = uuid.uuid4()
file.save(os.path.join(app.config['UPLOAD_FOLDER'], file_uuid))
# Rest of code
In either scenario you'd then store the UUID somewhere in the database. Whether you store the originally provided request.files['file'].filename value alongside that is your choice.
This might make sense if you want the user to see the original name of the file when they uploaded it. In that case it's definitey wise to run the value through secure_filename anyway, so there's never a situation where the frontend displays a listing to a user which includes a file called ../../../../ohdear.txt
the secure_filename docstring also points out some other functionality:
Pass it a filename and it will return a secure version of it. This filename can then safely be stored on a regular file system and passed to :func:
os.path.join. The filename returned is an ASCII only string for maximum portability. On windows systems the function also makes sure that the file is not named after one of the special device files.
>>> secure_filename("My cool movie.mov")
'My_cool_movie.mov'
>>> secure_filename("../../../etc/passwd")
'etc_passwd'
>>> secure_filename(u'i contain cool \xfcml\xe4uts.txt')
'i_contain_cool_umlauts.txt'
python - Secure user-provided filename - Stack Overflow
Import Error when importing secure_filename from Werkzeug.utils : Forums : PythonAnywhere
'Secure' Filenames
python - flask_uploads: ImportError: cannot import name 'secure_filename' - Stack Overflow
If you're running in a UNIX variant, you might want a chroot jail to prevent access to the system outside your application.
This approach would avoid you having to write your own code to deal with the problem and let you handle it with infrastructure setup. It might not be appropriate if you need to restrict access to some area within the application as it changes what the process thinks is the system root.
This seems like it would work, provided that open uses the same mechanism to resolve paths as os.path.abspath. Are there any flaws to this approach?
import os
def is_safe(filename):
here = os.path.abspath(".")
there = os.path.abspath(filename)
return there.startswith(here)
>>> is_safe("foo.txt")
True
>>> is_safe("foo/bar/baz")
True
>>> is_safe("../../goodies")
False
>>> is_safe("/hax")
False
Flask's WSGI library werkzeug has a utility function called secure_filename - you're intended to pass the filename of a user uploaded file to it but I'm having trouble understanding what exactly the concern is, and what securing a filename could possibly entail.
The Pooco site notes
what does that secure_filename() function actually do? Now the problem is that there is that principle called “never trust user input”. This is also true for the filename of an uploaded file. All submitted form data can be forged, and filenames can be dangerous. For the moment just remember: always use that function to secure a filename before storing it directly on the filesystem.
What's returned is the exact same filename except I guess sanitized in some way. What my concern is is that I don't really understand what this function accomplishes. I plan on changing the filename of any user submitted file anyways (random uuid string with a suffix of '-small', '-large', etc.). Does it do me any good to first run the filename through secure_filename() and then change the actual filename? Can I just initially change the filename?
» pip install filenames-secure
In flask_uploads.py
Change
from werkzeug import secure_filename,FileStorage
to
from werkzeug.utils import secure_filename
from werkzeug.datastructures import FileStorage
According to this issue, it is a bug related to the current version 1.0.0 of workzeug. It's merged but not yet published in pypi.
The workaround know until now is to downgrade from werkzeug=1.0.0 to werkzeug==0.16.0
So for do that you just need run the command:
pip install -U Werkzeug==0.16.0
Looking in the release notes from werkzeug there is a version 0.16.1, but in bug report there is no evidence that using that version could be of any help.
open() expects a pathname to the file. Since the file hasn't been saved to disk, no such path exists. :)
What you actually want to do is call f.read() directly. Reading incoming files is covered here.
Also, definitely use secure_filename() if you work with anything on disk. Don't want to open yourself to any directory traversal attacks down the line.
The objects in request.files are FileStorage objects and they have the same methods as normal file objects in python. So to get the contents of the file as binary, try doing this:
data = request.files['myfile'].read()
Python:
"".join(c for c in filename if c.isalpha() or c.isdigit() or c==' ').rstrip()
this accepts Unicode characters but removes line breaks, etc.
example:
filename = u"ad\nbla'{-+\)(ç?"
gives: adblaç
edit str.isalnum() does alphanumeric on one step. – comment from queueoverflow below. danodonovan hinted on keeping a dot included.
keepcharacters = (' ','.','_')
"".join(c for c in filename if c.isalnum() or c in keepcharacters).rstrip()
I don't recommend using any of the other answers. They're bloated, use bad techniques, and replace tons of legal characters (some even removed all Unicode characters, which is nuts since they're legal in filenames). A few of them even import huge libraries just for this tiny, easy job... that's crazy.
Here's a regex one-liner which efficiently replaces every illegal filesystem character and nothing else. No libraries, no bloat, just a perfectly legal filename in one simple command.
Reference: https://en.wikipedia.org/wiki/Filename#Reserved_characters_and_words
Regex:
clean = re.sub(r"[/\\?%*:|\"<>\x7F\x00-\x1F]", "-", dirty)
Usage:
import re
# Here's a dirty, illegal filename full of control-characters and illegal chars.
dirty = "".join(["\\[/\\?%*:|\"<>0x7F0x00-0x1F]", chr(0x1F) * 15])
# Clean it in one fell swoop.
clean = re.sub(r"[/\\?%*:|\"<>\x7F\x00-\x1F]", "-", dirty)
# Result: "-[----------0x7F0x00-0x1F]---------------"
print(clean)
This was an extreme example where almost every character is illegal, because we constructed the dirty string with the same list of characters that the regex removes, and we even padded with a bunch of "0x1F (ascii 31)" at the end just to show that it also removes illegal control-characters.
This is it. This regex is the only answer you need. It handles every illegal character on modern filesystems (Mac, Windows and Linux). Removing anything more beyond this would fall under the category of "beautifying" and has nothing to do with making legal disk filenames.
More work for Windows users:
After you've run this command, you could optionally also check the result against the list of "special device names" on Windows (a case-insensitive list of words such as "CON", "AUX", "COM0", etc).
The illegal words can be found at https://en.wikipedia.org/wiki/Filename#Comparison_of_filename_limitations in the "Reserved words" and "Comments" columns for the NTFS and FAT filesystems.
Filtering reserved words is only necessary if you plan to store the file on a NTFS or FAT-style disk. Because Windows reserves certain "magic filenames" for internal usage. It reserves them case-insensitively and without caring about the extension, meaning that for example aux.c is an illegal filename on Windows (very silly).
All Mac/Linux filesystems don't have silly limitations like that, so you don't have to do anything else if you're on a good filesystem. Heck, in fact, most of the "illegal characters" we filtered out in the regex are Windows-specific limitations. Mac/Linux filesystems can store most of them. But we filter them anyway since it makes the filenames portable to Windows machines.