The suggestion in the other answer may be a good step forward. Thank you @vinodk.
Asking many developers on this subject, I finally found a good solution.
The solution is:
- Use Prometheus as a data source of the availability measurements. Prometheus is needed anyway for regular monitoring.
- To prevent that the monitoring data gets lost when a Docker image or node crashes, map the data volume to an external file storage, e.g. S3.
- Use a data visualisation tool that performs a SINGLE but very powerful Prometheus API call.
As an example, this Prometheus API call gives the daily up (or availability) time over a period of a number of days. Take care of using Z/GMT times.
http://prometheus-server/prometheus/api/v1/query_range?query=avg_over_time(up[1h])*100&start=2019-12-15T06:00:00.000Z&end=2019-12-17T16:59:59.000Z&step=1d
What about alternatives?
1) Store the results of Prometheus in a remote database and query it seperately. It's a bit of overkill for just retrieving the availability percentages for a month.
2) You could use the tool 'Telegraph' which has a smaller memory footprint. It can store the data in InfluxDb. I like the Prometheus solution / architecture / standard for monitoring, so this option is not the right one in my case.
3) We use Spring Boot for our applications. Spring Boot Admin is not applicable because it only looks to the current situation and is not persisting the results.
Answer from tm1701 on Stack OverflowThe suggestion in the other answer may be a good step forward. Thank you @vinodk.
Asking many developers on this subject, I finally found a good solution.
The solution is:
- Use Prometheus as a data source of the availability measurements. Prometheus is needed anyway for regular monitoring.
- To prevent that the monitoring data gets lost when a Docker image or node crashes, map the data volume to an external file storage, e.g. S3.
- Use a data visualisation tool that performs a SINGLE but very powerful Prometheus API call.
As an example, this Prometheus API call gives the daily up (or availability) time over a period of a number of days. Take care of using Z/GMT times.
http://prometheus-server/prometheus/api/v1/query_range?query=avg_over_time(up[1h])*100&start=2019-12-15T06:00:00.000Z&end=2019-12-17T16:59:59.000Z&step=1d
What about alternatives?
1) Store the results of Prometheus in a remote database and query it seperately. It's a bit of overkill for just retrieving the availability percentages for a month.
2) You could use the tool 'Telegraph' which has a smaller memory footprint. It can store the data in InfluxDb. I like the Prometheus solution / architecture / standard for monitoring, so this option is not the right one in my case.
3) We use Spring Boot for our applications. Spring Boot Admin is not applicable because it only looks to the current situation and is not persisting the results.
Recording Rules will help you to aggregate and group your ingested metrics on regular basis and their results are ingested into prometheus.
I'm using Grafana for Dashboarding and it works great.
But i'm looking for a Reporting Tool like Tableau, MS Power BI or IBM Cognos Analytics that is easy to use with Prometheus or InfluxDB Data.
Do you have any recommendations?
Build in Datasource Connector for Prometheus would be great, or at least an easy way to get the data into the reporting tool.
Use case: Create monthly (or any other timeframe) pdf reports with Prometheus data, enriched with some other data from SQL sources.
I actually use Grafana to present reports to clients. I give them Viewer accounts and I can aggregate data from both Elasticsearch and Prometheus in the same dashboard/report.
But I've never used those more advanced tools that you mentioned so I have no idea how it compares.
Redash may be able to help you do this, we have ours hooked up to Prometheus and SQL sources
Hi,
I wonder if anyone has a clever way to generate reports from grafana/prometheus. I run several kubernetes cluster and I'd like to generate report to identify long term trends. Object of the report is pods and their use of cpu/memory.
Thanks!