bytes doesn't support item deletion because it's immutable. To "modify" strings and string-like objects you need to take a copy, so to remove olddata[start:end] do:

newdata = olddata[:start] + olddata[end:]

Of course that's a fair amount of copying, not all of which is necessary, so you might prefer to rework your code a bit for performance. You could use bytearray (which is mutable). Or perhaps you could find a way to work through the buffer (using an index or iterating over its elements), instead of needing to shorten it after each step.

You can use regex:

import re

s = '"trackingId":"f<0x85>9\u0004+L<0x9b><0x91>\u001A<0x87>&\u0013i+T"},{"pendingInvitation":false'
print(s)
print(re.sub(r'<0x\w{2}>', '',s))

with output:

"trackingId":"f<0x85>9+L<0x9b><0x91><0x87>&i+T"},{"pendingInvitation":false
"trackingId":"f9+L&i+T"},{"pendingInvitation":false

I have searched for the patten <0x__>, where the __ is any char or digit of length 2.

There are two issues here, one of which is the actual issue, the other is confusing you, but not an actual issue. Firstly:

Your string is a bytes object, ie a string of 8-bit bytes. Python 3 handles this differently from text, which is Unicode. Where do you get the string from? Since you want to treat it as text, you should probably convert it to a str-object, which is used to handle text. This is typically done with the .decode() function, ie:

somestring.decode('UTF-8')

Although calling str() also works:

str(somestring, 'UTF8')

(Note that your decoding might be something else than UTF8)

However, this is not your actual question. Your actual question is how to strip a bytes string. And the asnwer is that you do that the same way as you string a text-string:

somestring.strip()

There is no strip() builtin in either Python 2 or Python 3. There is a strip-function in the string module in Python 2:

from string import strip

But it hasn't been good practice to use that since strings got a strip() method, which is like ten years or so now. So in Python 3 it is gone.

Use bytes.replace to replace the substring with an empty string:

b = b'Today, in the digital age, any type of data, such as text, images, and audio, can be\r\ndigitized, stored indefinitely, and transmitted at high speeds. Notwithstanding these\r\nadvantages, digital data also have a downside. They are easy to access illegally, tamper\r\nwith, and copy for purposes of copyright violation.\r\nThere is therefore a need to hide secret identification inside certain types of digital\r\ndata. This information can be used to prove copyright ownership, to identify attempts\r\nto tamper with sensitive data, and to embed annotations. Storing, hiding, or embedding\r\nsecret information in all types of digital data is one of the tasks of the field of\r\nsteganography.\r\nSteganography is the art and science of data hiding. In contrast with cryptography,\r\nwhich secures data by transforming it into another, unreadable format, steganography\r\nmakes data invisible by hiding (or embedding) them in another piece of data, known\r\nalternatively as the cover, the host, or the carrier. The modified cover, including the\r\nhidden data, is referred to as a stego object. It can be stored or transmitted as a message.\r\nWe can think of cryptography as overt secret writing and of steganography as covert\r\nsecret writing.\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

b = b.replace(b'\x00', b'')
assert b.endswith(b'writing.')

>>> sByte = b'\x00\x81308 921 q53 246 133 137 022 1   0 1 1  1 130 C13 330 0000000199 04002201\n'
>>> sByte[2:]
b'308 921 q53 246 133 137 022 1   0 1 1  1 130 C13 330 0000000199 04002201\n'

See also https://appdividend.com/2022/07/09/python-slice-notation/

The code snippet returns sByte from and including the third byte until the end.

If you wanted to store the variable again you could do this:

>>> sByte = b'\x00\x81308 921 q53 246 133 137 022 1   0 1 1  1 130 C13 330 0000000199 04002201\n'
>>> sByte = sByte[2:]
>>> sByte
b'308 921 q53 246 133 137 022 1   0 1 1  1 130 C13 330 0000000199 04002201\n'

You can use StringIO to read a string like a file

>>> import StringIO
>>> s = 'Hello, World!'
>>> sio = StringIO.StringIO(s)
>>> sio.read(6)
'Hello,'
>>> sio.read()
' World!'

I would also suggest you take a look at the struct module for help with parsing binary data

>>> from struct import *
>>> pack('hhl', 1, 2, 3)
'\x00\x01\x00\x02\x00\x00\x00\x03'
>>> unpack('hhl', '\x00\x01\x00\x02\x00\x00\x00\x03')
(1, 2, 3)

You define the format of the data using format strings, so 'hhl' in the above example is short (2 bytes), short (2 bytes), int (4 bytes). It also supports specifying endianness (byte order) in the format string.

For example if your header format was uint, 4 byte str, uint, uint, ushort, ulong:

>>> import struct
>>> data = ''.join(chr(i) for i in range(128)) * 10
>>> hdr_fmt = 'I4sIIHL'
>>> struct.calcsize(hdr_fmt)
32
>>> struct.unpack_from(hdr_fmt, data, 0)
(50462976, '\x04\x05\x06\x07', 185207048, 252579084, 4368, 2242261671028070680)

If you are dealing with a zero-padded buffer then you can use rstrip to remove trailing \x00s

>>> text = 'Hello\x00\x00\x00\x00'
>>> text.rstrip('\x00')
'Hello'

It removes all \x00 characters at the end of the string but keeps any nulls in the middle. Not suitable for null-terminated strings that may contain random data after the terminator.

If you are dealing with a null-terminated string where the first zero indicates the end of string, but there might be other characters following it, you should use anregen's solution.

>>> text = 'Hello\x00\x24\x4e\x32'
>>> text.split('\x00', 1)[0]
'Hello'

It splits the text at the first zero and returns the slice. It works with strings having no null character too.

EDIT:
Explained rstrip in more detail and provided a correct use case.
Included alternative solution.

This should do the trick:

pw_bytes.decode("utf-8")

According to the Python docs, the b prefix means that your string is a byte string. Specifically:

A prefix of 'b' or 'B' is ignored in Python 2; it indicates that the literal should become a bytes literal in Python 3 (e.g. when code is automatically converted with 2to3). A 'u' or 'b' prefix may be followed by an 'r' prefix.

To convert this to a string without trailing newline and return, and to remove the byte prefix, you would use:

str(b'helloworld\r\n').rstrip('\r\n')

Usually you'd use a filtered version of the object, for example:

In [63]: test
Out[63]: 'hello\x00world'
In [68]: for my_bytes in filter(lambda x: x != b'\x00', test):
   ....:     print(my_bytes)
   ....:
h
e
l
l
o
w
o
r
l
d

Note I used my_bytes instead of bytes, which is a built-in name you'd rather not overwrite.

Similar you can also simply construct a filtered bytes object for further processing:

In [62]: test = b'hello\x00world'
In [63]: test
Out[63]: 'hello\x00world'
In [64]: test_without_nulls = bytes(filter(lambda x: x != b'\x00', test))
In [65]: test_without_nulls
Out[65]: 'helloworld'

I usually use bytes objects as it does not share the interface with strings in python 3. Certainly not byte arrays.

You can use bytes.decode function if you really need to "get rid of b": http://docs.python.org/3.3/library/stdtypes.html#bytes.decode

But it seems from your code that you do not really need to do this, you really need to work with bytes.

You could use ord to extract each character's numeric value, then combine them with simple arithmetic.

Copy>>> a = '\x02'
>>> b = '\x00'
>>> c = ord(a)*256 + ord(b)
>>> c == 0x0200
True
>>> print hex(c)
0x200

To remove these unwanted values from the list you can simply do:

arr = [b'\x01', b'\x02', b'', b'My Value That i Want!']

# Manually remove all unwanted values from the list
arr.remove(b'\x01')
arr.remove(b'\x02')
arr.remove(b'')

print(arr)
# Out:
# [b'My Value That i Want!']

Or you can also do:

from typing import Iterable

def isalphastr(value: Iterable[int]):
    # Check if the value dont have nothing to iterate
    if not value:
        return False

    # Iterate between all caracters in bytes
    for c in value:
        # Check if in the ASCII table they are alpha caracters
        if 126 > c < 32:
            return False
    return True

a = [b'\x01', b'\x02', b'', b'My Value That i Want!']

# Filter my array of bytes
it = filter(lambda x: isalphastr(x), a)

# Convert the iterator to a list
new_filted_array = list(it)

print(new_filted_array)
# Out:
# [b'My Value That i Want!']

In your case you could slice the list and get a chunk of the list, e.g.

a = [b'\x01', b'\x02', b'', b'My Value That i Want!', b'Another value that i want']

# Slice my list to get the last 2 values
new_a = a[3:]

print(new_a)
# Out: 
# [b'My Value That i Want!', b'Another value that i want']

You can use slicing:

teststr[:-1]

is equal to:

b'\x01\x02'

Replacement part also must be a byte string.

chunk = re.sub(b'-----------------------------(.+)--\r\n', b'', chunk)

Example:

>>> chunk = b'-----------------------------5313032314004\r\nContent-Disposition: form-data; name="file"; filename="4.jpg"\r\nContent-Type: image/jpeg\r\n\r\n\xff\xd8\xff'
>>> re.sub(b'-', b'', chunk)
b'5313032314004\r\nContentDisposition: formdata; name="file"; filename="4.jpg"\r\nContentType: image/jpeg\r\n\r\n\xff\xd8\xff'

Try below code line instead of last two lines. Hope it helps:

line=line.decode('utf-8','ignore').encode("utf-8")

Just use a bytearray:

>>> a = bytearray(b'abcdef')
>>> del a[1]
>>> a
bytearray(b'acdef')

It's almost like bytes but mutable:

The bytearray class is a mutable sequence of integers in the range 0 <= x < 256. It has most of the usual methods of mutable sequences, described in Mutable Sequence Types, as well as most methods that the bytes type has, see Bytes and Bytearray Operations.

Something that may help performance wise could be itertools.islice. This will allow you to call str.isprintable() max_width amount of times, as this is a binary file that may not have many \ns it can save a lot of effort.

output = line.decode(codec, "replace")
if max_width:
    print("".join(itertools.islice((c for c in output if c.isprintable()), max_width)))
else:
    print(output)

This on it's own may not help on files that have a lot of \ns. The bottle neck in these file would most likely be the overhead incurred by print. And so it's much faster to build a string to display once. In these cases you would want to use something like:
(Untested code)

def read_data(path):
    with open(path) as f:
        for line in f:
            output = line.decode(codec, "replace")
            if max_width:
                yield "".join(itertools.islice(
                    (c for c in output if c.isprintable()),
                    max_width))
            else:
                yield output

print('\n'.join(read_data(...)))

However the above is not good on machines with limited memory or extremely large files. In these cases you would want to use a buffer and print the buffer when a threshold has been reached.

After PEP 3138 your method to remove non-printables seems to be the correct way.

I am guessing that you mistakenly assigned the above byte string “sentence” to an object of type str. Instead, it needs to be assigned to a byte string object and interpret it as a sequence of UTF-8 bytes. Compare:

b = b'... known as \xe2\x80\x9ccomorbidity\xe2\x80\x9d and ...'
s = b.decode('utf-8')
print(b)
# b'... known as \xe2\x80\x9ccomorbidity\xe2\x80\x9d and ...'
print(s)
# ... known as “comorbidity” and ...

Either way, the issue is unrelated to compression: a lossless compression (such as bzip2) roundtrip never changes the data:

print(bz2.decompress(bz2.compress(b)).decode('utf-8'))
# ... known as “comorbidity” and ...