Read the binary file content like this:

with open(fileName, mode='rb') as file: # b is important -> binary
    fileContent = file.read()

then "unpack" binary data using struct.unpack:

The start bytes: struct.unpack("iiiii", fileContent[:20])

The body: ignore the heading bytes and the trailing byte (= 24); The remaining part forms the body, to know the number of bytes in the body do an integer division by 4; The obtained quotient is multiplied by the string 'i' to create the correct format for the unpack method:

struct.unpack("i" * ((len(fileContent) -24) // 4), fileContent[20:-4])

The end byte: struct.unpack("i", fileContent[-4:])

Python >= 3.8

Thanks to the walrus operator (:=) the solution is quite short. We read bytes objects from the file and assign them to the variable byte

with open("myfile", "rb") as f:
    while (byte := f.read(1)):
        # Do stuff with byte.

Python >= 3

In older Python 3 versions, we get have to use a slightly more verbose way:

with open("myfile", "rb") as f:
    byte = f.read(1)
    while byte != b"":
        # Do stuff with byte.
        byte = f.read(1)

Or as benhoyt says, skip the not equal and take advantage of the fact that b"" evaluates to false. This makes the code compatible between 2.6 and 3.x without any changes. It would also save you from changing the condition if you go from byte mode to text or the reverse.

with open("myfile", "rb") as f:
    byte = f.read(1)
    while byte:
        # Do stuff with byte.
        byte = f.read(1)

Python >= 2.5

In Python 2, it's a bit different. Here we don't get bytes objects, but raw characters:

with open("myfile", "rb") as f:
    byte = f.read(1)
    while byte != "":
        # Do stuff with byte.
        byte = f.read(1)

Note that the with statement is not available in versions of Python below 2.5. To use it in v 2.5 you'll need to import it:

from __future__ import with_statement

In 2.6 this is not needed.

Python 2.4 and Earlier

f = open("myfile", "rb")
try:
    byte = f.read(1)
    while byte != "":
        # Do stuff with byte.
        byte = f.read(1)
finally:
    f.close()

for rec in inh: reads one line at a time -- not what you want for a binary file. Read 4 bytes at a time (with a while loop and inh.read(4)) instead (or read everything into memory with a single .read() call, then unpack successive 4-byte slices). The second approach is simplest and most practical as long as the amount of data involved isn't huge:

import struct
with open('test.bin', 'rb') as inh:
    indata = inh.read()
for i in range(0, len(data), 4):
    pos = struct.unpack('i', data[i:i+4])  
    print(pos)  

If you do fear potentially huge amounts of data (which would take more memory than you have available), a simple generator offers an elegant alternative:

import struct
def by4(f):
    rec = 'x'  # placeholder for the `while`
    while rec:
        rec = f.read(4)
        if rec: yield rec           
with open('test.bin', 'rb') as inh:
    for rec in by4(inh):
        pos = struct.unpack('i', rec)  
        print(pos)  

A key advantage to this second approach is that the by4 generator can easily be tweaked (while maintaining the specs: return a binary file's data 4 bytes at a time) to use a different implementation strategy for buffering, all the way to the first approach (read everything then parcel it out) which can be seen as "infinite buffering" and coded:

def by4(f):
    data = inf.read()
    for i in range(0, len(data), 4):
        yield data[i:i+4]

while leaving the "application logic" (what to do with that stream of 4-byte chunks) intact and independent of the I/O layer (which gets encapsulated within the generator).

No, you can't use a TextIOWrapper() object because it'll read from the underlying buffer in larger blocks, not just in lines, so yes, it'll try to decode binary data past than first line. You can't prevent this.

For a single line of text using \n line delimiters, you really don't need to use TextIOWrapper(). Binary files still support line-by-line reading, where file.readline() will give you the binary data up to the next \n byte. Just open the file as binary, and read one line.

Valid UTF-16 data will always have an even length. But because UTF-16 comes in two flavours, big endian and little endian byte orders, you'll need to check how much data was read to see what byte order was used, in order to conditionally read a single byte that should be part of that first line of data. If UTF-16 little-endian was used, you are guaranteed to have read an odd number of bytes, as newlines are encoded to 0a 00 rather than 00 0a and the .readline() call will have left the single 00 byte in the file stream. In that case, just read one more byte and add it to the first line data before decoding:

with open(filename, 'rb') as binfile:
    firstline = binfile.readline()
    if len(firstline) % 2:
        # little-endian UTF-16, add one more byte
        firstline += binfile.read(1)
    text = firstline.decode('utf-16')

    # read binary data from the file

A demo with io.BytesIO() where we first write UTF-16 little-endian data (with the BOM to indicate the byte order for the decoder), with the text followed by two low-surrogate sequences that would cause a UTF-16 decoding error to stand for 'binary data', after which we read the text and data again:

>>> import io, codecs
>>> from pprint import pprint
>>> binfile = io.BytesIO()
>>> utf16le_wrapper = io.TextIOWrapper(binfile, encoding='utf-16-le', write_through=True)
>>> utf16le_wrapper.write('\ufeff')  # write the UTF-16 BOM manually, as the -le and -be variants won't include this
1
>>> utf16le_wrapper.write('The quick brown 🦊 jumps over the lazy 🐕\n')
40
>>> binfile.write(b'\xDF\xFF\xDF\xFF')  # binary data, guaranteed to not decode as UTF-16
4
>>> binfile.flush()  # flush and seek back to start to move to reading
>>> binfile.seek(0)
0
>>> firstline = binfile.readline()  # read that first line
>>> len(firstline) % 2              # confirm we read an odd number of bytes
1
>>> firstline += binfile.read(1)    # add the expected null byte
>>> pprint(firstline)               # peek at the UTF-16 data we read
(b'\xff\xfeT\x00h\x00e\x00 \x00q\x00u\x00i\x00c\x00k\x00 \x00b\x00r\x00o\x00'
 b'w\x00n\x00 \x00>\xd8\x8a\xdd \x00j\x00u\x00m\x00p\x00s\x00 \x00o\x00v\x00'
 b'e\x00r\x00 \x00t\x00h\x00e\x00 \x00l\x00a\x00z\x00y\x00 \x00=\xd8\x15\xdc'
 b'\n\x00')
>>> print(firstline.decode('utf-16'))  # bom included, so the decoder detects LE vs BE
The quick brown 🦊 jumps over the lazy 🐕

>>> binfile.read()
b'\xdf\xff\xdf\xff'

Any alternative implementations that still can use TextIOWrapper() would require an intermediate wrapper to sit between the binary file and the TextIOWrapper() instance to prevent TextIOWrapper() to read too far, and this would get complex fast and would require the wrapper have knowledge of the codec used anyway. For a single line of text, that's just not worth the effort required.