>>> help(zlib.crc32)
Help on built-in function crc32 in module zlib:

crc32(...)
    crc32(string[, start]) -- Compute a CRC-32 checksum of string.

    An optional starting value can be specified.  The returned checksum is
    a signed integer.
>>> zlib.crc32("".join(chr(random.randrange(0,255)) for _ in xrange(100)))
333158331

EDIT: code that uses the start value 0xFFFF:

>>> text = "".join(chr(random.randrange(0,255)) for _ in xrange(100))

>>> zlib.crc32(text)
-964269250

>>> zlib.crc32(text, 0xFFFF)
2057263175
Answer from Matt Anderson on Stack Overflow
🌐
GeeksforGeeks
geeksforgeeks.org › python › zlib-crc32-in-python
zlib.crc32() in python - GeeksforGeeks
March 23, 2020 - Syntax : zlib.crc32(s) Return : Return the unsigned 32-bit checksum integer.
🌐
Reddit
reddit.com › r/learnpython › faulty behavior zlib.crc32()
r/learnpython on Reddit: Faulty behavior zlib.crc32()
September 20, 2022 -

Hi,

I usually use Bash and the command line to calculate checksums (including crc32). I wanted to learn how to do it in python. I found the zlib module which handles MD5, SHA1 and SHA256 very well and fast!

However there is some problem with zlib.crc32(). For small files it returns the same results as the crc32 command in the terminal. However if I feed zlib.crc32() any file larger than roughly 2GB the results differ. I am not sure how to provide a minimal working example since I cannot attach a 2GB file. But I will show you which commands I run. Maybe there is a mistake I do not see.

In python I do:

import zlib
path2file = path/to/my.file
#The [2:] gets rid of the HEX 0x prefix
crc_python  = hex(zlib.crc32(open(path2file,'rb').read()))[2:]

In the command line I would run:

~$ crc32 path/to/my.file

As mentioned the results of these two commands are the same for smaller files but differ for larger files.

Naturally I could yield the same result as in the terminal using python by:

import subprocess
path2file = path/to/my.file
crc_shell = subprocess.run(f'crc32 "{path2file}"', shell=True, text=True, capture_output=True).stdout.strip()

which kinda defeats the purpose of using python in the first place and is much slower.

Any ideas why crc_python == crc_shell for small files but crc_python != crc_shell for large files?

Thanks!

Discussions

Calculating CRC using Python zlib.crc32 function - Stack Overflow
I created a list of size 100 and populated the array with 8bit data in python using the following code and I want to calculate the CRC value using the zlib.crc32() function. Init_RangenCrc8 = [] f... More on stackoverflow.com
🌐 stackoverflow.com
Calculate CRC32 correctly with Python - Stack Overflow
Note that Python 3 guarantees that binascii.crc32 returns an unsigned value, and 2.6 and 2.7 should be guaranteeing a signed value, so platform differences shouldn't be affecting this. 2019-02-05T10:12:43.223Z+00:00 ... It seems that python is returning an signed integer (hence the negative number), whereas the others are returning an unsigned integer. I have tried using a modulus with 2^32, and it gave the same value as these sites. >>> hex(zlib... More on stackoverflow.com
🌐 stackoverflow.com
C zlib crc32 and Python zlib crc32 doesn't match - Stack Overflow
Im experimenting a bit with crc32 in Python and C but my results won't match. C: #include #include #include #define NUM_BYTES 9 int main(void) { More on stackoverflow.com
🌐 stackoverflow.com
crc - Python: setting generator Polynomial in zlib.crc32 - Stack Overflow
I am trying to use zlib.crc32 to compute CRC in Python. I would like to be able to set the generator polynomial of the CRC, but I cannot find any documentation. So the question is: can this be done... More on stackoverflow.com
🌐 stackoverflow.com
🌐
Beautiful Soup
tedboy.github.io › python_stdlib › generated › generated › zlib.crc32.html
zlib.crc32() — Python Standard Library
zlib.crc32() View page source · zlib.crc32()¶ · crc32(string[, start]) – Compute a CRC-32 checksum of string. An optional starting value can be specified.
🌐
ProgramCreek
programcreek.com › python › example › 1627 › zlib.crc32
Python Examples of zlib.crc32
""" data = [ struct.pack("<4sBBqIIi", b"OggS", self.version, self.__type_flags, self.position, self.serial, self.sequence, 0) ] lacing_data = [] for datum in self.packets: quot, rem = divmod(len(datum), 255) lacing_data.append(b"\xff" * quot + chr_(rem)) lacing_data = b"".join(lacing_data) if not self.complete and lacing_data.endswith(b"\x00"): lacing_data = lacing_data[:-1] data.append(chr_(len(lacing_data))) data.append(lacing_data) data.extend(self.packets) data = b"".join(data) # Python's CRC is swapped relative to Ogg's needs. # crc32 returns uint prior to py2.6 on some platforms, so force uint crc = (~zlib.crc32(data.translate(cdata.bitswap), -1)) & 0xffffffff # Although we're using to_uint_be, this actually makes the CRC # a proper le integer, since Python's CRC is byteswapped.
🌐
TechOverflow
techoverflow.net › 2024 › 08 › 13 › how-to-configure-stm32h7-hardware-crc-to-match-zlib-crc32-in-python
How to configure STM32H7 hardware CRC to match zlib CRC32 in Python | TechOverflow
August 13, 2024 - void TestCRC32() { uint32_t crc = CalculateCRC32((void*)"DEADBEEF", 8); printf("CRC32 of 'DEADBEEF': X", crc); // When testing, always include a non-divisible-by-4 number of bytes crc = CalculateCRC32((void*)"Hello, World!", 13); SendLogMessageF("CRC32 of 'Hello, World!': X", crc); } ... import zlib def calculate_crc32(data: bytes) -> int: return zlib.crc32(data) print(f"CRC32 of 'Hello, World!': {calculate_crc32(b'Hello, World!'):08X}") print(f"CRC32 of 'DEADBEEF': {calculate_crc32(b'DEADBEEF'):08X}")
🌐
GitHub
github.com › enthought › Python-2.7.3 › blob › master › Modules › zlib › crc32.c
Python-2.7.3/Modules/zlib/crc32.c at master · enthought/Python-2.7.3
/* crc32.c -- compute the CRC-32 of a data stream · * Copyright (C) 1995-2005 Mark Adler · * For conditions of distribution and use, see copyright notice in zlib.h · * * Thanks to Rodney Brown <rbrown64@csc.com.au> for his contribution of faster · * CRC methods: exclusive-oring 32 bits of data at a time, and pre-computing ·
Author   enthought
Find elsewhere
Top answer
1 of 4
6

There were problems in your original C and Python code snippets. As for your second C snippet, I haven't tried to compile it, but it's not portable since byte order within an int is platform-dependant. So it will give different results depending on the endianness of the CPU.

One problem, as Serge Ballesta has mentioned, is the difference between {1, 2, 3, 4, 5, 6, 7, 8, 9} and {'1', '2', '3', '4', '5', '6', '7', '8', '9'}. Another problem is that the loop in your original C code didn't actually scan the data, since you didn't use i in the loop, as bav mentioned.

crctest.c

#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <zlib.h>

#define NUM_BYTES 9

// gcc -std=c99 -lz -o crctest test.c

void do_crc(uint8_t *bytes)
{
    uint32_t crc = crc32(0L, Z_NULL, 0);

    for (int i = 0; i < NUM_BYTES; ++i)
    {
        crc = crc32(crc, bytes + i, 1);
    }

    printf("CRC32 value is: %lu\n", crc);
}

int main(void)
{
    uint8_t bytes0[NUM_BYTES] = {1, 2, 3, 4, 5, 6, 7, 8, 9};
    uint8_t bytes1[NUM_BYTES] = {'1', '2', '3', '4', '5', '6', '7', '8', '9'};

    do_crc(bytes0);
    do_crc(bytes1);
}

output

CRC32 value is: 1089448862
CRC32 value is: 3421780262

crctest.py

#! /usr/bin/env python

import zlib

def do_crc(s):
    n = zlib.crc32(s)
    return n + (1<<32) if n < 0 else n

s = b'\x01\x02\x03\x04\x05\x06\x07\x08\x09'
print `s`, do_crc(s)

s = b'123456789'
print `s`, do_crc(s)

output

'\x01\x02\x03\x04\x05\x06\x07\x08\t' 1089448862
'123456789' 3421780262

edit

Here's a better way to handle the conversion in Python:

def do_crc(s):
    n = zlib.crc32(s)
    return n & 0xffffffff

See the answers here for more info on this topic: How to convert signed to unsigned integer in python.

2 of 4
2

According to www.lammertbies.nl that has detailed references on CRC calculation and C routines, the CRC32 of the ASCII string 123456789 in 0xCBF43926, that is 3421780262 as an unsigned 32 integer in decimal form.

That means that your Python computation is correct, but to get same result in C you should write

uint8_t bytes[NUM_BYTES] = {'1', '2', '3', '4', '5', '6', '7', '8', '9'};
uint32_t crc = crc32(0L, Z_NULL, 0);

Alternatively, if what you want is indeed the crc 32 for uint8_t bytes[NUM_BYTES] = {1, 2, 3, 4, 5, 6, 7, 8, 9};, you must use in python 2.x:

s = ''
for i in range(10):
    s += chr(i)
s

outputs : '\x00\x01\x02\x03\x04\x05\x06\x07\x08\t'

then

zlib.crc32(s)

outputs : 1164760902

Nota: in python 3.x, you would have written : s = bytes(range(10))

🌐
Python
bugs.python.org › issue1202
Issue 1202: zlib.crc32() and adler32() return value - Python tracker
This issue tracker has been migrated to GitHub, and is currently read-only. For more information, see the GitHub FAQs in the Python's Developer Guide · This issue has been migrated to GitHub: https://github.com/python/cpython/issues/45543
🌐
Python
docs.python.org › 3 › library › zlib.html
zlib — Compression compatible with gzip
This documentation doesn’t attempt ... the zlib manual for authoritative information. For reading and writing .gz files see the gzip module. The available exception and functions in this module are: ... Exception raised on compression and decompression errors. ... Computes an Adler-32 checksum of data. (An Adler-32 checksum is almost as reliable as a CRC32 but can be ...
Top answer
1 of 1
5

No, the polynomial and other CRC parameters are hard-coded in zlib.

However the CRC-32 generated by those parameters is the CRC that zlib implements. You can just use zlib.crc32.

>>> import crcmod
>>> crc32 = crcmod.mkCrcFun(poly=0x104c11db7, rev=True, initCrc=0, xorOut=0xFFFFFFFF)
>>> print(hex(crc32(b'123456789')))
0xcbf43926
>>> import zlib
>>> print(hex(zlib.crc32(b'123456789')))
0xcbf43926

Update:

From the comments, speed is not important, and they want a different CRC polynomial.

Here is a simple CRC-32 implementation, assuming rev=True, for which you can change or parameterize the polynomial, the initial value, and the final exclusive-or:

def crc32(msg):
    crc = 0xffffffff
    for b in msg:
        crc ^= b
        for _ in range(8):
            crc = (crc >> 1) ^ 0xedb88320 if crc & 1 else crc >> 1
    return crc ^ 0xffffffff

print(hex(crc32(b'123456789')))

Prints: 0xcbf43926.

If you need something faster, then you'd need to generate a table for a byte-wise table-driven CRC implementation.

Update:

As the original poster continues to trickle out tiny bits of information about their actual question in comments (as opposed to simply saying in total what they want to do in the question), we learn that they need to generate a forward CRC as well (rev=False), specifically a CRC-32/MPEG-2. So here is the simple code for that:

def crc32mpeg2(msg):
    crc = 0xffffffff
    for b in msg:
        crc ^= b << 24
        for _ in range(8):
            crc = (crc << 1) ^ 0x04c11db7 if crc & 0x80000000 else crc << 1
    return crc & 0xffffffff

print(hex(crc32mpeg2(b'123456789')))

Prints: 0x376e6e7.

🌐
GitHub
github.com › python › cpython › issues › 105967
crc32 function outputs wrong result for large data on the macOS arm64 platform · Issue #105967 · python/cpython
June 21, 2023 - OS-macstdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error ... The functions zlib.crc32 and binascii.crc32 share the problematic behavior.
Author   python
🌐
Runebook.dev
runebook.dev › en › docs › python › library › zlib › zlib.crc32
Integrity Check 101: Correctly Using and Chaining zlib.crc32() in Python
The zlib. crc32() function in Python's built-in zlib module computes the CRC (Cyclic Redundancy Check) checksum of data
🌐
Blogger
notepad2.blogspot.com › 2016 › 01 › python-calculate-crc32-checksum-for-file.html
My Tech Notes: Python: calculate crc32 checksum for a file
import zlib import sys BUFFER_SIZE=8192 def get_crc32(path): with open(path, 'rb') as f: crc = 0 while True: data = f.read(BUFFER_SIZE) if not data: break crc = zlib.crc32(data, crc) return crc def main(): for f in sys.argv[1:]: crc32 = get_crc32(f) crc32hex = hex(crc32)[2:] print(f'{f}: {crc32hex}') if __name__ == '__main__': main() % python3 crc32.py my-file.dat ./my-file.dat: 780acc19
Top answer
1 of 3
11

I'm using right now zlib.crc32, but for C there is no such library

Um, yes, there is. It's called zlib. zlib is written in C, and it's what Python is using! Hence the name of the class.

You can use the crc32() function in zlib. That implementation is a fair bit faster than others you might find. Read zlib.h for the interface information.

You can compile zlib yourself, or it may already be installed on your system.

Update:

I now see your comment (which should be edited into the question since it is critical to getting the right answer) that you have extremely limited memory. Then you can use this:

static uint32_t crc32(uint32_t crc, unsigned char *buf, size_t len)
{
    int k;

    crc = ~crc;
    while (len--) {
        crc ^= *buf++;
        for (k = 0; k < 8; k++)
            crc = crc & 1 ? (crc >> 1) ^ 0xedb88320 : crc >> 1;
    }
    return ~crc;
}

The crc is initially set to zero.

The use of ~ will give the correct result, since the uint32_t type in stdint.h is assured to be 32 bits.

If you can afford a little more code space, then unrolling the loop will likely speed it up (if the compiler doesn't already do this):

static uint32_t crc32(uint32_t crc, unsigned char *buf, size_t len)
{
    crc = ~crc;
    while (len--) {
        crc ^= *buf++;
        crc = crc & 1 ? (crc >> 1) ^ 0xedb88320 : crc >> 1;
        crc = crc & 1 ? (crc >> 1) ^ 0xedb88320 : crc >> 1;
        crc = crc & 1 ? (crc >> 1) ^ 0xedb88320 : crc >> 1;
        crc = crc & 1 ? (crc >> 1) ^ 0xedb88320 : crc >> 1;
        crc = crc & 1 ? (crc >> 1) ^ 0xedb88320 : crc >> 1;
        crc = crc & 1 ? (crc >> 1) ^ 0xedb88320 : crc >> 1;
        crc = crc & 1 ? (crc >> 1) ^ 0xedb88320 : crc >> 1;
        crc = crc & 1 ? (crc >> 1) ^ 0xedb88320 : crc >> 1;
    }
    return ~crc;
}

You said that you only have 4 KBytes of "memory". Is that just working memory for the program, or does the program have to live there as well? If you have more space in flash for example for the code, then the table can be precomputed and stored with the code. A table-driven CRC will be much faster. The zlib code provides table-driven CRCs that do one byte at a time and four-bytes at a time, requiring respectively a 1Kbyte or 4Kbyte table.

Update 2:

Since you answered in a comment that the 4KBytes are just working memory, then you should use a table-driven CRC. You can simply use the crc32() function in zlib's crc32.c and the table in crc32.h with BYFOUR undefined.

2 of 3
5

C:

UInt32
crc32(UInt32 crc, UInt8 *p, SInt len)
{
  crc = ~crc;
  while (--len >= 0) {
    crc = crc ^ *p++;
    for (SInt i = 8; --i >= 0;) {
      crc = (crc >> 1) ^ (0xedb88320 & -(crc & 1));
    }
  }
  return ~crc;
}

void
crc_unitTest(void)
{
  UInt8 b1[] = { 0, 0, 0, 0, 0, 0, 0, 0 };
  UInt8 b2[] = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 };
  UInt8 b3[] = { 0xff, 0, 0xff, 0, 0xff, 0, 0xff, 0 };
  assert(crc32(0, b1,  8) == 0x6522df69);
  assert(crc32(0, b2, 10) == 0x456cd746);
  assert(crc32(0, b3,  8) == 0xea8c89c0);
}

Python:

def crc32(crc, p, len):
  crc = 0xffffffff & ~crc
  for i in range(len):
    crc = crc ^ p[i]
    for j in range(8):
      crc = (crc >> 1) ^ (0xedb88320 & -(crc & 1))
  return 0xffffffff & ~crc

def unitTest():
  b1 = [ 0, 0, 0, 0, 0, 0, 0, 0 ]
  b2 = [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 ]
  b3 = [ 0xff, 0, 0xff, 0, 0xff, 0, 0xff, 0 ]
  assert(crc32(0, b1,  8) == 0x6522df69)
  assert(crc32(0, b2, 10) == 0x456cd746)
  assert(crc32(0, b3,  8) == 0xea8c89c0)
🌐
YouTube
youtube.com › watch
Using zlib.crc32 in Python: Understanding CRC Generator Polynomials - YouTube
Explore how to work with CRCs using Python's `zlib` library, and learn about defining generator polynomials for CRC calculations without external libraries.-...
Published   March 31, 2025
Views   31
🌐
Python Module of the Week
pymotw.com › 2 › zlib
zlib – Low-level access to GNU zlib compression library - Python Module of the Week
import zlib data = open('lorem.txt', 'r').read() cksum = zlib.adler32(data) print 'Adler32: d' % cksum print ' : d' % zlib.adler32(data, cksum) cksum = zlib.crc32(data) print 'CRC-32 : d' % cksum print ' : d' % zlib.crc32(data, cksum) $ python zlib_checksums.py Adler32: 1865879205 : 118955337 CRC-32 : 1878123957 : -1940264325
🌐
SourceForge
crcmod.sourceforge.net › intro.html
Introduction — crcmod v1.7 documentation
zlib.crc32() function from the zlib module · CRC-32 implementation · Module hashlib · Secure hash and message digest algorithms. Module md5 · RSA’s MD5 message digest algorithm. Module sha · NIST’s secure hash algorithm, SHA. Module hmac · Keyed-hashing for message authentication. Introduction · Guidelines · Dependencies · Python Version ·