You are probably looking for BytesIO or StringIO classes from Python io package, both available in python 2 and python 3. They provide a file-like interface you can use in your code the exact same way you interact with a real file.
StringIO is used to store textual data:
import io
f = io.StringIO("some initial text data")
BytesIO must be used for binary data:
import io
f = io.BytesIO(b"\x00\x00\x00\x00\x00\x00\x00\x00\x01\x01\x01\x01\x01\x01")
To store MP3 file data, you will probably need the BytesIO class. To initialize it from a GET request to a server, proceed like this:
import requests
from pygame import mixer
import io
r = requests.get("http://example.com/somesmallmp3file.mp3")
inmemoryfile = io.BytesIO(r.content)
mixer.music.init()
mixer.music.load(inmemoryfile)
mixer.music.play()
# This will free the memmory from any data
inmemoryfile.close()
Additional note: as both classes inherit from IOBase, they can be used as context manager with the with statement, so you don't need to manually call the close() method anymore:
import requests
from pygame import mixer
import io
r = requests.get("http://example.com/somesmallmp3file.mp3")
with io.BytesIO(r.content) as inmemoryfile:
mixer.music.init()
mixer.music.load(inmemoryfile)
mixer.music.play()
Python
docs.python.org › 3 › library › io.html
io — Core tools for working with streams
The file name. This is the file descriptor of the file when no name is given in the constructor. Buffered I/O streams provide a higher-level interface to an I/O device than raw I/O does. ... A binary stream using an in-memory bytes buffer. It inherits from BufferedIOBase.
(C# || Python/Windows) Is there a way I can create an in-memory file that has a path other processes can access?
in windows you can use CreateFileMapping and MapViewOfFile to create a shared memory that is accessible to multiple programs. https://learn.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-createfilemappinga https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-mapviewoffile the other side must use OpenFileMapping. and you will need to cleanup with UnmapViewOfFile and CloseHandle. this if you have access to the source of the other program, if you don't you might still be able to do it by injecting a dll in the target that hook the file opening api and repleace with the memory opening solution. or better: probably that other third party compiled app that you use to decoding have also a third party compiled dll that you can use from within your project to do stuff instead of using the compiled app. so you just use that dll. EDIT: yet another option that only partially fix your problem is to set a file to be deleted on next reboot (i'm not 100% sure about this, i never used it): MoveFileEx(filePath, NULL, MOVEFILE_DELAY_UNTIL_REBOOT); More on reddit.com
9
3
November 25, 2024Include in-memory fs into python path
I would like to write a module, that I decrypt during program execution, into an in-memory filesystem, so that he decrypted file is not on disk. Then I would like to import that module. How can I add the path of the file in the in-memory fs of pyfilesystem2 to the pythonpath? More on github.com
19
April 23, 2020memfile: Python library to store files in RAM
The official Python community for Reddit! Stay up to date with the latest news, packages, and meta information relating to the Python programming language. More on reddit.com
48
19
February 14, 2025Writing File to Memory
I believe what you need is the open() keyword. This loads the file into memory so you can perform executions on it. As for creating files and then executing them as valid python is a terrible idea. Unless you're creating an IDE you're program should not be able to execute code it hasn't implemented itself. This leaves you open to potential attacks. More on reddit.com
13
10
May 15, 202103:39
Python: Memory-Mapped Files - YouTube
08:08
Understanding Memory and File I/O (Video) – Real Python
python file in memory
01:41
python create in memory file - YouTube
23:02
File I/O With Memory Mapping Using Python mmap - YouTube
04:43
8.9 - Learn Python: Using an In-Memory Stream - YouTube
Medium
medium.com › @abhishekshaw020 › understanding-bytesio-handling-in-memory-files-like-a-pro-e1b767339468
Understanding BytesIO: Handling In-Memory Files Like a Pro | by Abhishek Shaw | Medium
March 31, 2025 - Think of BytesIO as a virtual file that lives in your computer’s memory (RAM) instead of your hard drive. It lets you read and write data just like a normal file, but everything stays in memory—fast, efficient, and no cleanup required!
DEV Community
dev.to › bluepaperbirds › file-like-objects-in-python-1njf
File-Like Objects in Python - DEV Community
February 20, 2020 - >>> myFile.seek(0) 0 >>> myFile.read() 'Data into the file' >>> ... >>> import io >>> myFile = io.StringIO("Feeling good") >>> data = myFile.read() >>> print(data) Feeling good >>> You can write data into the memory file too, by using the .write() method.
Reddit
reddit.com › r/learnprogramming › (c# || python/windows) is there a way i can create an in-memory file that has a path other processes can access?
r/learnprogramming on Reddit: (C# || Python/Windows) Is there a way I can create an in-memory file that has a path other processes can access?
November 25, 2024 -
I feel like this might be a situation where OS features like pipes could come into play but I'm not knowledgable enough about that stuff, and would appreciate any advice or clarification.
I'm writing a program that deals with image data, reading files into arrays of pixels in memory. The library I'm using knows how to decode JPEGs and PNGs, but not some more niche cutting-edge formats like JPEG XL. So currently when I need to deal with those I create a subprocess, call ImageMagick or djxl (JPEG XL -> JPEG decoder program) to create a temporary .JPEG/.PNG file on the disk, read that, then delete it.
Which works fine, but this isn't optimal: the program could leave temp files on the disk if it's killed before cleaning up (e.g. power failure), it could write millions of temp files for large runs creating unnecessary wear and tear on an SSD, it could spin up a sleeping HDD on a laptop causing battery drain, etc. The temp files are never over 3 MB so it'd be nice to create them in memory instead.
But I'm not sure how I'd go about doing that short of creating a ramdisk. The programs I call in the subprocess want file paths to read from and write to.
So is this a situation where you can create a pipe and use the pipe like a file path, and have the C#/Python program read from it? What's the best approach here?
Top answer 1 of 7
4
in windows you can use CreateFileMapping and MapViewOfFile to create a shared memory that is accessible to multiple programs. https://learn.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-createfilemappinga https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-mapviewoffile the other side must use OpenFileMapping. and you will need to cleanup with UnmapViewOfFile and CloseHandle. this if you have access to the source of the other program, if you don't you might still be able to do it by injecting a dll in the target that hook the file opening api and repleace with the memory opening solution. or better: probably that other third party compiled app that you use to decoding have also a third party compiled dll that you can use from within your project to do stuff instead of using the compiled app. so you just use that dll. EDIT: yet another option that only partially fix your problem is to set a file to be deleted on next reboot (i'm not 100% sure about this, i never used it): MoveFileEx(filePath, NULL, MOVEFILE_DELAY_UNTIL_REBOOT);
2 of 7
3
There is probably some technical solution like you're considering that would work really great. But, if you're not dealing with too many files at a time, maybe just add a cleanup step at the beginning of your process? That is, when you go to create a new file, first check for existing files and clean them up. Quick, simple, takes five minutes to code. Upgrade to something more efficient if and when you actually need it.
Read the Docs
rasterio.readthedocs.io › en › latest › topics › memory-files.html
In-Memory Files — rasterio 1.5.1.dev0 documentation
There are different options for Python programs that have streams of bytes, e.g., from a network socket, as their input or output instead of filenames. One is the use of a temporary file on disk. import tempfile with tempfile.NamedTemporaryFile() as tmpfile: tmpfile.write(data) with rasterio.open(tmpfile.name) as dataset: data_array = dataset.read() Another is Rasterio’s MemoryFile, an abstraction for objects in GDAL’s in-memory filesystem.
Simon Willison
til.simonwillison.net › sqlite › python-sqlite-memory-to-file
Saving an in-memory SQLite database to a file in Python | Simon Willison’s TILs
April 8, 2023 - The VACUUM INTO command can be used to save a copy of the database to a new file. Here's how to use it: import sqlite3 db = sqlite3.connect(":memory:") db.execute("create table foo (bar text)") db.execute("vacuum main into '/tmp/saved.db'") # Or with sqlite-utils import sqlite_utils db = sqlite_utils.Database(memory=True) db["foo"].insert({"bar": "Example record"}) db.execute("vacuum main into '/tmp/saved2.db'") spatialite Creating a minimal SpatiaLite database with Python - 2021-12-17
Python
python-list.python.narkive.com › QnxDgTiX › mimicking-a-file-in-memory
mimicking a file in memory
In order to do this, you give mutagen a filename, which it converts into a file object using the python built-in "file" function. Unfortunately, my mp3 files don't live locally. They are on a number of remote servers which I access using urllib2. I don't want to copy the files into a local directory for mutagen's sake, only to have to remove them afterward. Instead, I'd like to load the files into memory and still be able to hand the built-in "file" function a filename to access the file in memory.
GitHub
github.com › PyFilesystem › pyfilesystem2 › issues › 384
Include in-memory fs into python path · Issue #384 · PyFilesystem/pyfilesystem2
April 23, 2020 - I would like to write a module, that I decrypt during program execution, into an in-memory filesystem, so that he decrypted file is not on disk. Then I would like to import that module. How can I add the path of the file in the in-memory fs of pyfilesystem2 to the pythonpath?
Author PyFilesystem
Medium
medium.com › @sarthakshah1920 › harnessing-the-power-of-in-memory-buffers-with-bytesio-0ac6d5493178
Harnessing the Power of In-Memory Buffers with BytesIO | by Sarthak Shah | Medium
December 24, 2023 - Whether dealing with images or files, the traditional approach of saving data to disk can introduce various challenges such as slower I/O operations, security concerns, and the need for manual file cleanup. This article explores a more efficient alternative using in-memory buffers, exemplified by the Python BytesIO module.
LinkedIn
linkedin.com › pulse › handling-large-files-optimizing-file-operations-python-nuno-bispo-ots2e
Handling Large Files and Optimizing File Operations in ...
We cannot provide a description for this page right now
Textual
textual.textualize.io › blog › 2024 › 02 › 11 › file-magic-with-the-python-standard-library
Textual - File magic with the Python standard library
February 11, 2024 - In Python you get an object which behaves like a bytearray, but loads data from disk when it is accessed. The beauty of this module is that you can work with files in much the same way as if you had read the entire file in to memory, while leaving the actual reading of the file to the OS.
PyPI
pypi.org › project › memory-profiler
memory-profiler · PyPI
Execute the code passing the option -m memory_profiler to the python interpreter to load the memory_profiler module and print to stdout the line-by-line analysis. If the file name was example.py, this would result in:
» pip install memory-profiler
w3resource
w3resource.com › python-exercises › extended-data-types › python-extended-data-types-index-memory-views-exercise-5.php
Modifying binary files using Python memory views
Finally, the modified data is written to an output file. ... Write a Python program to open a binary file, create a memory view of its content, modify a specific byte, and then write the modified bytes back to a new file.
Quora
quora.com › When-we-open-a-file-using-Python-will-the-whole-file-be-moved-to-the-RAM-or-will-only-the-file-descriptor-with-the-portion-of-the-file-be-moved-to-the-RAM
When we open a file using Python, will the whole file be moved to the RAM or will only the file descriptor with the portion of the file be moved to the RAM? - Quora
Answer (1 of 4): Opening a file does NOT implicitly read nor load its contents. Even when you do so using Python's context management protocol (the with keyword). For example if you do something like: > [code]with open('./some_big_file.txt', 'r') as f: for each_line in f: do_something(...
Reddit
reddit.com › r › Python › comments › 1ipei7z › memfile_python_library_to_store_files_in_ram
memfile: Python library to store files in RAM
February 14, 2025 - The official Python community for Reddit! Stay up to date with the latest news, packages, and meta information relating to the Python programming language.
GitHub
github.com › pytest-dev › pyfakefs
GitHub - pytest-dev/pyfakefs: Provides a fake file system that mocks the Python file system modules. · GitHub
pyfakefs implements a fake file system that mocks the Python file system modules. Using pyfakefs, your tests operate on a fake file system in memory without touching the real disk.
Starred by 743 users
Forked by 98 users
Languages Python 99.9% | Dockerfile 0.1%
DataCamp
campus.datacamp.com › courses › python-toolbox › using-iterators-in-pythonland
Using iterators to load large files into memory | Python
One solution is to load the data in chunks, perform the desired operation or operations on each chuck, store the result, discard the chunk and then load the next chunk; this sounds like a place where an iterator could be useful!To surmount this challenge, we are going to use the pandas function ...
GitHub
github.com › bloomberg › memray
GitHub - bloomberg/memray: Memray is a memory profiler for Python · GitHub
Example: $ python3 -m memray run ... of the memory usage in the terminal optional arguments: -h, --help Show this help message and exit -v, --verbose Increase verbosity. Option is additive and can be specified up to 3 times -V, --version Displays the current version of Memray Please submit feedback, ideas, and bug reports by filing a new issue at https://github.com/bloomberg/memray/issues · To use Memray over a script or a single python file you can ...
Starred by 15.2K users
Forked by 458 users
Languages Python 70.3% | C++ 19.4% | Cython 5.3% | JavaScript 2.5% | HTML 1.0% | CMake 0.5%
LabEx
labex.io › tutorials › python-how-to-read-large-files-efficiently-434795
Python - How to read large files efficiently
In the world of Python programming, efficiently reading large files is a critical skill for developers working with big data, log analysis, and complex data processing tasks. This tutorial explores advanced techniques to read massive files while minimizing memory consumption and maximizing ...