Depending on where you want to extract the information from, you can use minecart. It has really robust support for colors and allows easy conversion to RGB. Though you can't input a coordinate and get the color value there, if you are trying to get color information from a shape you could do something like the following:

import minecart
doc = minecart.Document(open("my-doc.pdf", "rb"))
page = doc.get_page(0)
BOX = (.5 * 72,  # left bounding box edge
       9 * 72,   # bottom bounding box edge
       1 * 72,   # right bounding box edge
       10 * 72)  # top bounding box edge
for shape in page.shapes:
    if shape.check_in_bbox(BOX):
        r, g, b = shape.fill.color.as_rgb()
        # do stuff with r, g, b

[Disclaimer: I'm the author of minecart]