# Create an example
from io import BytesIO
bytesio_object = BytesIO(b"Hello World!")
# Write the stuff
with open("output.txt", "wb") as f:
f.write(bytesio_object.getbuffer())
Answer from Martin Thoma on Stack Overflow# Create an example
from io import BytesIO
bytesio_object = BytesIO(b"Hello World!")
# Write the stuff
with open("output.txt", "wb") as f:
f.write(bytesio_object.getbuffer())
It would be helpful if you supplied the library you were using to work on excel files, but here's a buckshot of solutions, based on some assumptions I'm making:
- Based on the first paragraph in the io module's documentation, it sounds like all the concrete classes- including BytesIO- are file-like objects. Without knowing what code you've tried so far, I don't know if you have tried passing the BytesIO to the module you're using.
- On the off chance that doesn't work, you can simply convert BytesIO to a another io Writer/Reader/Wrapper by passing it to the constructor. Example:
.
import io
b = io.BytesIO(b"Hello World") ## Some random BytesIO Object
print(type(b)) ## For sanity's sake
with open("test.xlsx") as f: ## Excel File
print(type(f)) ## Open file is TextIOWrapper
bw=io.TextIOWrapper(b) ## Conversion to TextIOWrapper
print(type(bw)) ## Just to confirm
- You may need to check which kind of Reader/Writer/Wrapper is expected by the module you're using to convert the BytesIO to the correct one
- I believe I have heard that (for memory reasons, due to extremely large excel files) excel modules do not load the entire file. If this ends up meaning that what you need is a physical file on the disk, then you can easily write the Excel file temporarily and just delete it when you're done. Example:
.
import io
import os
with open("test.xlsx",'rb') as f:
g=io.BytesIO(f.read()) ## Getting an Excel File represented as a BytesIO Object
temporarylocation="testout.xlsx"
with open(temporarylocation,'wb') as out: ## Open temporary file as bytes
out.write(g.read()) ## Read bytes into file
## Do stuff with module/file
os.remove(temporarylocation) ## Delete file when done
I'll hope that one of these points will solve your problem.
You need to seek back to the beginning of the file after writing the initial in memory file...
myio.seek(0)
How about we write and read gzip content in the same context like this?
#!/usr/bin/env python
from io import BytesIO
import gzip
content = b"does it work"
# write bytes to zip file in memory
gzipped_content = None
with BytesIO() as myio:
with gzip.GzipFile(fileobj=myio, mode='wb') as g:
g.write(content)
gzipped_content = myio.getvalue()
print(gzipped_content)
print(content == gzip.decompress(gzipped_content))
shutil has a utility that will write the file efficiently. It copies in chunks, defaulting to 16K. Any multiple of 4K chunks should be a good cross platform number. I chose 131072 rather arbitrarily because really the file is written to the OS cache in RAM before going to disk and the chunk size isn't that big of a deal.
import shutil
myBytesIOObj.seek(0)
with open('myfile.ext', 'wb') as f:
shutil.copyfileobj(myBytesIOObj, f, length=131072)
BTW, there was no need to close the file object at the end. with defines a scope, and the file object is defined inside that scope. The file handle is therefore closed automatically on exit from the with block.
Since Python 3.2 it's possible to use the BytesIO.getbuffer() method as follows:
from io import BytesIO
buf = BytesIO(b'test')
with open('path/to/file', 'wb') as f:
f.write(buf.getbuffer())
This way it doesn't copy the buffer's content, streaming it straight to the open file.
Note: The StringIO buffer doesn't support the getbuffer() protocol (as of Python 3.9).
Before streaming the BytesIO buffer to file, you might want to set its position to the beginning:
buf.seek(0)
The issue is that you are positioned at the end of the stream. Think of the position like a cursor. Once you have written b' world', your cursor is at the end of the stream. When you try to .read(), you are reading everything after the position of the cursor - which is nothing, so you get the empty bytestring.
To navigate around the stream you can use the .seek method:
>>> import io
>>> in_memory = io.BytesIO(b'hello', )
>>> in_memory.write(b' world')
>>> in_memory.seek(0) # go to the start of the stream
>>> print(in_memory.read())
b' world'
Note that, just like a filestream in write ('w') mode, the initial bytes b'hello' have been overwritten by your writing of b' world'.
.getvalue() just returns the entire contents of the stream regardless of current position.
this is a memory stream but still a stream. The position is stored, so like any other stream if you try to read after having written, you have to re-position:
import io
in_memory = io.BytesIO(b'hello')
in_memory.seek(0,2) # seek to end, else we overwrite
in_memory.write(b' world')
in_memory.seek(0) # seek to start
print( in_memory.read() )
prints:
b'hello world'
while in_memory.getvalue() doesn't need the final seek(0) as it returns the contents of the stream from position 0.