Before starting my answer, let me remark that all your questions concern a general distribution , and isn't really related to the fact that .

In point 2, the usual definition is indeed with any test function (it is a trivial definition: ; then the specificity of rely on the fact that with ). However, your proof of this fact is incorrect as are not . Though I trust the result, but you should find a proof of the fact : if satisfies , then .

For point 1, you should require a sign for . More generally, the definition of when is a distribution is A simple reason for that is to notice that if for a locally integrable function, then the above definition is equivalent to . If you didn't require a sign on , then using , you would be lead to point 1!

About your last question: if is a positive (Radon) measure (depending on the setting you work in, you could consider non-positive measure), then it is a positive distribution. The converse is true: if is a positive distribution, then it is of order , and therefore, according to Riesz representation theorem, can be represented with of positive (Radon) measure. The proof of that is a classical and short exercise (start on a compact set so that you don't have issue with the compact support of test functions)

Answer from John Steinbeck on Stack Exchange
Top answer
1 of 3
35

If you want per second results that don't vary, you'll want to GROUP BY time(1s). This will give you accurate perSecond results.

Consider the following example:

Suppose that the value of the counter at each second changes like so

0s → 1s → 2s → 3s → 4s
1  → 2  → 5  → 8  → 11

Depending on how we group the sequence above, we'll see different results.

Consider the case where we group things into 2s buckets.

 0s-2s   →    2s-4s
(5-1)/2  →  (11-5)/2
   2     →      3

versus the 1s buckets

 0s-1s  →  1s-2s  →  2s-3s  →  3s-4s
(2-1)/1 → (5-2)/1 → (8-5)/1 → (11-8)/1
   1    →    3    →    3    →    3

Addressing

So to me, that means that the value at a given point should not change that much when expanding the time view, since the value should be rate of change per unit (1s in my example query above).

The rate of change per unit is a normalizing factor, independent of the GROUP BY time unit. Interpreting our previous example when we change the derivative interval to 2s may offer some insight.

The exact equation is

∆y/(∆x/tu)

Consider the case where we group things into 1s buckets with a derivative interval of 2s. The result we should see is

 0s-1s    →  1s-2s    →  2s-3s    →  3s-4s
2*(2-1)/1 → 2*(5-2)/1 → 2*(8-5)/1 → (11-8)/1
   2      →    6      →    6      →    6

This may seem a bit odd, but if you consider what this says it should make sense. When we specify a derivative interval of 2s what we're asking for is what the 2s rate of change is for the 1s GROUP BY bucket.

If we apply similar reasoning to the case of 2s buckets with a derivative interval of 2s is then

 0s-2s     →    2s-4s
2*(5-1)/2  →  2*(11-5)/2
   4       →      6

What we're asking for here is what the 2s rate of change is for the 2s GROUP BY bucket and in the first interval the 2s rate of change would be 4 and the second interval the 2s rate of change would be 6.

2 of 3
10

@Michael-Desa gives an excellent explanation.

I'd like to augment that answer with a solution to a pretty common metric our company is interested in: "What is the maximum "operation per second" value on a specific measurement field?".

I will use a real-life example from our company.

Scenario Background

We send a lot of data from an RDBMS to redis. When transferring that data, we keep track of 5 counters:

  1. TipTrgUp -> Updates by a business trigger (stored procedure)
  2. TipTrgRm -> Removes by a business trigger (stored procedure)
  3. TipRprUp -> Updates by an unattended auto-repair batch process
  4. TipRprRm -> Removes by an unattended auto-repair batch process
  5. TipDmpUp -> Updates by a bulk-dump process

We made a metrics collector that sends the current state of these counters to InfluxDB, with an interval of 1 second (configurable).

Grafana graph 1: low resolution, no true max ops

Here is the grafana query that is useful, but does not show the true max ops when zoomed out (we know it will go to around 500 ops on a normal business day, when no special dumps or maintenance is taking place - otherwise it goes into the thousands):

SELECT
    non_negative_derivative(max(TipTrgUp),1s) AS "update/TipTrgUp"
   ,non_negative_derivative(max(TipTrgRm),1s) AS "remove/TipTrgRm"
   ,non_negative_derivative(max(TipRprUp),1s) AS "autorepair-up/TipRprUp"
   ,non_negative_derivative(max(TipRprRm),1s) AS "autorepair-rm/TipRprRm"
   ,non_negative_derivative(max(TipDmpUp),1s) AS "dump/TipDmpUp"
FROM "$rp"."redis_flux_-transid-d-s"
WHERE
    host =~ /$servertimeFilter
GROUP BY time($interval),* fill(null)

Sidenotes: $rp is the name of the retention policy, templated in grafana. We use CQ's to downsample to retention policies with a larger duration. Also note the 1s as a derivative parameter: it is needed, since the default is different when using GROUP BY. This can be easily overlooked in the InfluxDB documentation.

The graph, seen by 24 hours looks like this:

If we simply use a resolution of 1s (as suggested by @Michael-Desa), an enormous amount of data is transferred from influxdb to the client. It works reasonably well (about 10 seconds), but too slow for us.

Grafana graph 2: low and high resolution, true max ops, slow performance

We can however use subqueries to add the true maxops to this graph, which is a slight improvement. A lot less data is transferred to the client, but the InfluxDB server has to do a lot of number crunching. Series B (with maxops prepended in the aliases):

SELECT
    max(subTipTrgUp) AS maxopsTipTrgUp
   ,max(subTipTrgRm) AS maxopsTipTrgRm
   ,max(subTipRprUp) AS maxopsRprUp
   ,max(subTipRprRm) AS maxopsTipRprRm
   ,max(subTipDmpUp) AS maxopsTipDmpUp
FROM (
    SELECT
        non_negative_derivative(max(TipTrgUp),1s) AS subTipTrgUp
       ,non_negative_derivative(max(TipTrgRm),1s) AS subTipTrgRm
       ,non_negative_derivative(max(TipRprUp),1s) AS subTipRprUp
       ,non_negative_derivative(max(TipRprRm),1s) AS subTipRprRm
       ,non_negative_derivative(max(TipDmpUp),1s) AS subTipDmpUp
    FROM "$rp"."redis_flux_-transid-d-s"
    WHERE
        host =~ /$servertimeFilter
    GROUP BY time(1s),* fill(null)
)
WHERE $timeFilter
GROUP BY time($interval),* fill(null)

Gives:

Grafana graph 3: low and high resolution, true max ops, high performance, pre-calculate by CQ

Our final solution to these kind of metrics (but only when we need a live view, the subquery approach works fine for ad-hoc graphs) is: use a Continuous Query to pre-calculate the true maxops. We generate CQ's like this:

CREATE CONTINUOUS QUERY "redis_flux_-transid-d-s.maxops.1s"
ON telegraf
BEGIN
    SELECT
        non_negative_derivative(max(TipTrgUp),1s) AS TipTrgUp
       ,non_negative_derivative(max(TipTrgRm),1s) AS TipTrgRm
       ,non_negative_derivative(max(TipRprUp),1s) AS TipRprUp
       ,non_negative_derivative(max(TipRprRm),1s) AS TipRprRm
       ,non_negative_derivative(max(TipDmpUp),1s) AS TipDmpUp
    INTO telegraf.A."redis_flux_-transid-d-s.maxops"
    FROM telegraf.A."redis_flux_-transid-d-s"
    GROUP BY time(1s),*
END

From here on, it's trivial to use these maxops measurements in grafana. When downsampling to an RP with longer retention, we again use max() as the selector function.

Series B (with .maxops appended in the aliases)

SELECT
    max(TipTrgUp) AS "update/TipTrgUp.maxops"
   ,max(TipTrgRm) AS "remove/TipTrgRm.maxops"
   ,max(TipRprUp) as "autorepair-up/TipRprUp.maxops"
   ,max(TipRprRm) as "autorepair-rm/TipRprRm.maxops"
   ,max(TipDmpUp) as "dump/TipDmpUp.maxops"
FROM "$rp"."redis_flux_-transid-d-s.maxops"
WHERE
    host =~ /$servertimeFilter
GROUP BY time($interval),* fill(null)

Gives:

When zoomed in to 1s precision, you can see that the graphs become identical:

Hope this helps, TW

Discussions

Derivative of nulls is not null
System info: influxdb 0.13.0 Steps to reproduce: Use a measurement with some gaps in it (e.g. several hours of no data) Run a derivative or non_negative_derivative over the mean/median/mode(/etc.) ... More on github.com
🌐 github.com
15
August 20, 2016
Is it possible to have a null value at a given point of the derivative of function while having this point not a local maximum or minimum? - Mathematics Stack Exchange
I thought that monotonic function could not have a null derivative ? On wikipedia, it is said that : en.wikipedia.org/wiki/… : "A function f ( x ) is said to be absolutely monotonic over an interval if the derivatives of all orders of f are nonnegative or all nonpositive at all points on the interval" $\endgroup$ ... $\begingroup$ Non negative ... More on math.stackexchange.com
🌐 math.stackexchange.com
Non-negative difference and non-negative derivative aggregate functions
Difference() and Derivative() are great for getting the changes in a variable such as a packet counter. Some variables, however, wrap around or reset to zero (such as after a server reboot). I woul... More on github.com
🌐 github.com
1
March 25, 2015
distribution theory - non negative distributional derivative - Mathematics Stack Exchange
$\begingroup$ At least $f$ can be increasing on null sets. $\endgroup$ ... $\begingroup$ For example, consider $$f(x) = \begin{cases} -x & \text{ if }x \in \mathbb{R}\setminus\mathbb{Q},\\ x & \text{ if }x \in \mathbb{Q}. \end{cases}$$ Its distributional derivative is negative, but $f$ is not decreasing. $\endgroup$ ... 4 What is the intuition behind distributional derivative and why distributional derivative is useful? ... How would you "stun" a spaceship (i.e. incapacitate it nonlethally... More on math.stackexchange.com
🌐 math.stackexchange.com
🌐
GitHub
github.com › influxdata › flux › issues › 2896
nonNegative default value for derivative · Issue #2896 · influxdata/flux
June 10, 2020 - By default, derivative() returns only positive derivative values and replaces negative values with null. But from a few tests I made with Influx 1.8, it seems that nonNegative defaults to false.
Author: influxdata
🌐
GitHub
github.com › influxdata › influxdb › issues › 7185
Derivative of nulls is not null · Issue #7185 · influxdata/influxdb
August 20, 2016 - The derivative function behaves the same as if fill(none) had been used, and suppresses records when its inputs are all null. > SELECT non_negative_derivative(mean("value")) FROM "disk_read" WHERE "host" = 'kirisame' AND "instance" = 'sda' AND ...
Author: influxdata
🌐
InfluxData Documentation
docs.influxdata.com › flux › v0 › stdlib › universe › derivative
derivative() function | Flux Documentation
April 8, 2024 - Use zero (0) as the initial value in the derivative calculation when the subsequent value is less than the previous value and nonNegative is true. Default is false. Input data. Default is piped-forward data (<-). Calculate the non-negative rate of change per second · Calculate the rate of ...
🌐
Stack Exchange
math.stackexchange.com › questions › 4148703 › is-it-possible-to-have-a-null-value-at-a-given-point-of-the-derivative-of-functi
Is it possible to have a null value at a given point of the derivative of function while having this point not a local maximum or minimum? - Mathematics Stack Exchange
I thought that monotonic function could not have a null derivative ? On wikipedia, it is said that : en.wikipedia.org/wiki/… : "A function f ( x ) is said to be absolutely monotonic over an interval if the derivatives of all orders of f are nonnegative or all nonpositive at all points on the interval" $\endgroup$ ... $\begingroup$ Non negative or non positive means $\ge 0$ or $\le 0$, not $>0$ or $<0$. A function has a local extremum at a point if & only if its derivative changes sign at that point.
🌐
GitHub
github.com › influxdata › influxdb › issues › 2073
Non-negative difference and non-negative derivative aggregate functions · Issue #2073 · influxdata/influxdb
March 25, 2015 - Difference() and Derivative() are great for getting the changes in a variable such as a packet counter. Some variables, however, wrap around or reset to zero (such as after a server reboot). I would find it very useful to have NonNegativeDifference() and NonNegativeDerivative(), where a negative change is regarded as a zero or NaN change.
Author: influxdata
Find elsewhere
🌐
Stack Exchange
math.stackexchange.com › questions › 2907336 › non-negative-distributional-derivative
distribution theory - non negative distributional derivative - Mathematics Stack Exchange
$\begingroup$ At least $f$ can be increasing on null sets. $\endgroup$ ... $\begingroup$ For example, consider $$f(x) = \begin{cases} -x & \text{ if }x \in \mathbb{R}\setminus\mathbb{Q},\\ x & \text{ if }x \in \mathbb{Q}. \end{cases}$$ Its distributional derivative is negative, but $f$ is not decreasing. $\endgroup$ ... 4 What is the intuition behind distributional derivative and why distributional derivative is useful? ... How would you "stun" a spaceship (i.e. incapacitate it nonlethally)?
🌐
Grafana
community.grafana.com › time series panel
Issues with non_negative_derivative - Time Series Panel - Grafana Labs Community Forums
May 6, 2020 - Hi, I’m trying to grapha Tx/Rx stream coming from some SNMP monitoring. The values are in Bytes and I understand I need to use a non-negative-derivative with a 1s period to obtain the Up/Down speeds. This seems to work perfectly in InfluxDB with the following query returning the following graph: SELECT NON_NEGATIVE_DERIVATIVE(mean("ifOutOctets"),1s) AS "mean_ifOutOctets", NON_NEGATIVE_DERIVATIVE(mean("ifInOctets"),1s) AS "mean_ifInOctets" FROM "telegraf"."autogen"."ifTable" WHERE time > :das...
🌐
InfluxData Community
community.influxdata.com › t › understanding-non-negative-derivative-calculations › 10300
Understanding non-negative-derivative calculations - InfluxData Community Forums
June 28, 2019 - Hi, I’m trying to understand how a specific calculation is being made in my InfluxDB instance. I’m trying to convert outOctets to bits per second. How is the non_negative_derivative on line 79 being calculated? My data…
🌐
GitHub
github.com › elastic › elasticsearch › issues › 15542
Add non-negative option to derivative aggregation · Issue #15542 · elastic/elasticsearch
December 18, 2015 - The derivative aggregation is great for getting the changes in a variable such as a packet counter. Some variables, however, wrap around or reset to zero (such as after a server reboot). I would find it very useful to have non negative derivative option, where a negative change is regarded as a zero change.
Author: elastic
🌐
Subwiki
calculus.subwiki.org › wiki › Nonnegative_derivative_that_is_not_identically_zero_on_any_interval_implies_increasing
Nonnegative derivative that is not identically zero on any interval implies increasing - Calculus
December 13, 2011 - Thus, the condition of derivative not being zero on any interval can be replaced by the stronger condition that the derivative is zero only at isolated points. The resulting statement is somewhat weaker (i.e., less powerful) because it assumes a stronger hypothesis on the function.
🌐
Statistics How To
statisticshowto.com › home › non negative function
Non Negative Function - Statistics How To
November 10, 2021 - A non negative function has function values equal to or greater than zero (i.e., f(x) ≥ 0). The domain (inputs) of the function can be negative, but the outputs (y-values) must be zero or greater in order for the function to be classified ...
🌐
Subwiki
calculus.subwiki.org › wiki › Increasing_and_differentiable_implies_nonnegative_derivative_that_is_not_identically_zero_on_any_interval
Increasing and differentiable implies nonnegative derivative that is not identically zero on any interval - Calculus
December 18, 2011 - Nonnegative derivative that is not identically zero on any interval implies increasing · Positive derivative implies increasing · Zero derivative implies locally constant · Negative derivative implies decreasing · Local maximum from the left implies left hand derivative is nonnegative if ...