The prometheus_client library documentation addresses this case:

Prometheus client libraries presume a threaded model, where metrics are shared across workers. This doesn't work so well for languages such as Python where it's common to have processes rather than threads to handle large workloads.

I won't copy here the explanation (which is geared toward gunicorn) use case but basically, you need to:

• define an env variable with directory to use: since you are using Process yourself, you can set it in the code

os.environ["PROMETHEUS_MULTIPROC_DIR"] = "/path/to/writeable/tmp/"

• each process must have its own collector, register it at start and unregister it at exit:

from prometheus_client import multiprocess

def called_from_process():
    registry = CollectorRegistry()
    multiprocess.MultiProcessCollector(CollectorRegistry())

def process_exit(process):
    if process.pid is not None:
        multiprocess.mark_process_dead(process.pid)

p = Process(target=f)
# f calls called_from_process
p.start()
p.join()
process_exit(process)

See the full documentation about how to handle gauge and any other quirk.

I expect that PROMETHEUS_MULTIPROC_DIRshould be cleaned at startup of your application to handle odd case where the application previous run was not able to do so.

I was figuring out the same thing, and the solution was as simple as you would imagine.

Updated your example code to work:

from multiprocessing import Process
import shutil
import time, os
from prometheus_client import start_http_server, multiprocess, CollectorRegistry, Counter


COUNTER1 = Counter('counter1', 'Incremented by the first child process')
COUNTER2 = Counter('counter2', 'Incremented by the second child process')
COUNTER3 = Counter('counter3', 'Incremented by both child processes')


def f1():
    while True:
        time.sleep(1)
        print("Child process 1")
        COUNTER1.inc()
        COUNTER3.inc()
    

def f2():
    while True:
        time.sleep(1)
        print("Child process 2")
        COUNTER2.inc()
        COUNTER3.inc()


if __name__ == '__main__':
    # ensure variable exists, and ensure defined folder is clean on start
    prome_stats = os.environ["PROMETHEUS_MULTIPROC_DIR"]
    if os.path.exists(prome_stats):
        shutil.rmtree(prome_stats)
    os.mkdir(prome_stats)

    # pass the registry to server
    registry = CollectorRegistry()
    multiprocess.MultiProcessCollector(registry)
    start_http_server(8000, registry=registry)

    p = Process(target=f1, args=())
    a = p.start()
    p2 = Process(target=f2, args=())
    p2.start()

    print("collect")

    while True:
        time.sleep(1)

localhost:8000/metrics
# HELP counter1_total Incremented by the first child process
# TYPE counter1_total counter
counter1_total 9.0
# HELP counter2_total Incremented by the second child process
# TYPE counter2_total counter
counter2_total 9.0
# HELP counter3_total Incremented by both child processes
# TYPE counter3_total counter
counter3_total 18.0

What you would want to do here is start up a separate process just to serve the metrics. Put the app function in https://github.com/prometheus/client_python#multiprocess-mode-gunicorn in an app of its own, and make sure that prometheus_multiproc_dir is the same for both it and the main application.