The Pythonic way is to write a generator function. Since this generator handles a file resource which needs to be closed afterwards, we should wrap it in a contextmanager, and consume the chunks within a with block.
from contextlib import contextmanager
@contextmanager
def file_chunks(filename, chunk_size):
f = open(filename, 'rb')
try:
def gen():
b = f.read(chunk_size)
while b:
yield b
b = f.read(chunk_size)
yield gen()
finally:
f.close()
Usage:
with file_chunks('test.raw', 160) as chunks:
for chunk in chunks:
# process the chunk
print(chunk)
Answer from kaya3 on Stack OverflowYou can read your data as a text file instead of binary file.
Here's how:
file = open("/dev/rtlightsensor0", "r")
line = file.readline()
data = [int(each) for each in line.split())
Original answer:
You didn't describe your question clearly enough, but I'm guessing that you have four 32-bit (4 bytes) int in your file, which you want to read into your program.
os.read is a binary file read function, so the variable data you got is bytes instead of a list of numbers.
Python can't infer the length your number as it can be four 32-bit numbers, one 128-bit number, or something else. You need to define it explicitly.
There is also the issue of whether your data are big endian or little endian.
If your data doesn't come from a network communication, it's likely to be little endian. If you find the result from the following code is wrong (for example, unbelievably large), you can try to convert 'big' into 'little'.
You can finish your code like this:
dev = os.open("/dev/rtlightsensor0", os.0_RDWR)
raw_data = os.read(dev,16)
data = []
data.append(int.from_bytes(raw_data[0:4], 'little'))
data.append(int.from_bytes(raw_data[4:8], 'little'))
data.append(int.from_bytes(raw_data[8:12], 'little'))
data.append(int.from_bytes(raw_data[12:16], 'little'))
print(data)
However, DO NOT do it like that because it's really silly. If you can understand the code above, then you can write it better like this:
dev = os.open("/dev/rtlightsensor0", os.0_RDWR)
raw_data = os.read(dev, 16)
data = [int.from_bytes(raw_data[i * 4: (i + 1) * 4], 'little') for i in range(4)]
print(data)
By the way, it's not wise to read binary data like this. Generally speaking, the package numpy is recommended.
It is a byte array. You can convert the byte array to string using decode function
>>> test = b'1 53 -5 1\n'
>>> type(test)
<class 'bytes'>
>>> test[0]
49
>>> test_1 = test.decode("utf-8")
>>> test_1[0]
'1'
You can use buffered input from io with bytearray:
import io
buf = bytearray(1024)
with io.open(filename, 'rb') as fp:
size = fp.readinto(buf)
if not size:
break
# do things with buf considering the size
This is one of the situations that python 3.8's new walrus operator - which both assigns a value to a variable, and returns the value that it just assigned - is really good for. You can use file.read(size) to read in 1024-byte chunks, and simply stop when there's no more file left to read:
buffer_size = 1024
with open('file.txt', 'rb') as f:
while (chunk := f.read(buffer_size)) != b'':
# do things with the variable `chunk`, which should have len() == 1024
Note that the != b'' part of the condition can be safely removed, as the empty string will evaluate to False when used as a boolean expression.