You are misunderstanding the purpose of sum. It is not performing a sum over time but over the dimensions of your metric.

In your example, assuming there are multiple requests metrics (with a dimension page by exemple), rate(requests[3 sec]) will give you (at some point in time):

{page="A"}  12.4
{page="B"}  1.5
{page="C"}  0 .... (and so on for each metrics requests with different label set)

The sum function will sum the values of the different rates; and sum(rate(requests[3 sec])) will give you only one value:

{}  42.13 <-- the sum of all rate(requests[3s]) values

BONUS: In the case you metric have multiple dimensions (represented by multiple labels in your metric) you can tell sum() to operate on a subset of them: sum(rate(requests[3 sec])) ON(foo)

You'd better expose your counters at 0 on application start, if the other labels (aaa, etc) have a limited set of possible combinations. This way rate() function works correctly at the bottom level and sum() will give you correct results.

If you have to do a rate() of the sum(), read this first:

Note that when combining rate() with an aggregation operator (e.g. sum()) or a function aggregating over time (any function ending in _over_time), always take a rate() first, then aggregate. Otherwise rate() cannot detect counter resets when your target restarts.

If you can tolerate this (or the instances reset counters at the same time), there's a way to work around. Define a recording rule as

record: job:mycounter:sum
expr: sum without(instance) (mycounter)

and then this expression works:

sum(rate(job:mycounter:sum[5m]))

It is likely you have many short-lived time series during the hour, and these time series do not intersect. For example, every time series receives raw samples only for a minute after the registration, and then it is substituted by a new time series. So, you end up with 60 unique time series per each hour. Such a situation is known as high churn rate, and it may lead to increased resource usage (CPU, RAM, disk space and disk IO) at Prometheus side.

When you calculate the sum of increase rates over short durations, then individual time series results do not intersect, so the sum at every point on the graph (or at every query execution timestamp) remains comparable to the individual increase rates per each time series. But when you calculate the sum of rates over hour-long durations, then it becomes bigger than individual per-series rate by significant multiplier, because every point on the graph corresponds to more than one rate() result. So the sum() returns bigger values.

You can check whether this is your case by comparing the results of the following two PromQL queries:

count(some_metrics)

count(last_over_time(some_metrics[1h]))

If some_metrics time series have high churn rate, then the result of the second query (it returns the total number of some_metrics time series over the last hour) will be much bigger than the result of the first query (it returns the number of some_metrics time series over the last few minutes - see these docs).

The reason you want the rate() function in the Prometheus query is so you can see what the average rate was in that time window ([10s] in that doc example).

If instead you are using the overall sum/count then that number will continue to grow and the average won't cover the latest time frame, but instead will be the average of all timing since the service started.

Example:

Imagine you have a timing that takes 1 second each time it is called and it is called about 30 times each minute:

                   Count        Sum        sum/count   sum/count (with increase)
First Minute:      30           30         1           1
After 10 hour:     18,000       18,000     1           1
After 1000 hours:  1,800,000    1,800,000  1           1

So far it looks identical. Now assume that for the last 1 minute all the requests take 10 seconds. Which is 10 times as slow. You would want to know about that last minute

                   Count        Sum        sum/count   sum/count (with increase)
First Minute:      30           300        10          10
After 10 hour:     18,000       18,270     1.015       10
After 1000 hours:  1,800,000    1,800,270  1.00015     10

The rate (or increase) function ensures that it is just using the change in that window for the calculation. As the metric is running for longer period, the large number masks any volatility.

Note: In my example I used the increase function since it is a little easier to reason through. It just reports how much has the counter or sum increased in that window. rate is similar, but just normalizes it to a per/second rate.