Grafana学习(3)——Introduction to time series

发布时间 2023-11-22 12:15:52作者: 钱塘江畔
Imagine you wanted to know how the temperature outside changes throughout the day. Once every hour, you’d check the thermometer and write down the time along with the current temperature. After a while, you’d have something like this:
Temperature data like this is one example of what we call a time series — a sequence of measurements, ordered in time. Every row in the table represents one individual measurement at a specific time.

想象一下,你想知道室外的温度在一天中是如何变化的。每小时检查一次温度计,记下时间和当前温度。过一段时间,你会得到这样的东西:
像这样的温度数据是我们所说的时间序列的一个例子——一系列按时间顺序排列的测量结果。表中的每一行表示在特定时间的一个单独测量。

Tables are useful when you want to identify individual measurements, but they make it difficult to see the big picture. A more common visualization for time series is the graph, which instead places each measurement along a time axis. Visual representations like the graph make it easier to discover patterns and features of the data that otherwise would be difficult to see.
Temperature data like the one in the example, is far from the only example of a time series. Other examples of time series are:
CPU and memory usage
Sensor data
Stock market index

当你想确定单个测量值时,表格很有用,但它们会让你很难看到全貌。时间序列的一个更常见的可视化是图形,它将每个测量值沿着时间轴放置。像图这样的可视化表示可以更容易地发现数据的模式和特征,否则很难看到这些模式和特征。
温度数据(如示例中的数据)远不是时间序列的唯一示例。时间序列的其他示例包括:

  • CPU和内存使用情况
  • 传感器数据
  • 股票市场指数
While each of these examples are sequences of chronologically ordered measurements, they also share other attributes:
  New data is appended at the end, at regular intervals — for example, hourly at 09:00, 10:00, 11:00, and so on.
  Measurements are seldom updated after they were added — for example, yesterday’s temperature doesn’t change.

虽然这些例子中的每一个都是按时间顺序排列的测量序列,但它们也共享其他属性:

  • 新数据会以固定的间隔附加在末尾,例如,每小时09:00、10:00、11:00,依此类推。
  • 测量值添加后很少更新——例如,昨天的温度没有变化。
Time series are powerful. They help you understand the past by letting you analyze the state of the system at any point in time. Time series could tell you that the server crashed moments after the free disk space went down to zero.

Time series can also help you predict the future, by uncovering trends in your data. If the number of registered users has been increasing monthly by 4% for the past few months, you can predict how big your user base is going to be at the end of the year.

Some time series have patterns that repeat themselves over a known period. For example, the temperature is typically higher during the day, before it dips down at night. By identifying these periodic, or seasonal, time series, you can make confident predictions about the next period. If you know that the system load peaks every day around 18:00, you can add more machines right before.

时间序列非常强大。它们可以让你在任何时间点分析系统的状态,从而帮助你理解过去。时间序列可以告诉你,在可用磁盘空间降至零后,服务器崩溃了。
时间序列还可以通过揭示数据中的趋势来帮助您预测未来。如果在过去的几个月里,注册用户的数量每月增长4%,你可以预测年底你的用户群会有多大。
一些时间序列具有在已知时间段内重复出现的模式。例如,在夜间气温下降之前,白天的气温通常会更高。通过识别这些周期性或季节性的时间序列,你可以对下一个时期做出自信的预测。如果你知道系统负载在每天18:00左右达到峰值,你可以在之前添加更多的机器。

Aggregating time series
Depending on what you’re measuring, the data can vary greatly. What if you wanted to compare periods longer than the interval between measurements? If you’d measure the temperature once every hour, you’d end up with 24 data points per day. To compare the temperature in August over the years, you’d have to combine the 31 times 24 data points into one.
Combining a collection of measurements is called aggregation. There are several ways to aggregate time series data. Here are some common ones:
    Average returns the sum of all values divided by the total number of values.
    Min and Max return the smallest and largest value in the collection.
    Sum returns the sum of all values in the collection.
    Count returns the number of values in the collection.
For example, by aggregating the data in a month, you can determine that August 2017 was, on average, warmer than the year before. Instead, to see which month had the highest temperature, you’d compare the maximum temperature for each month.
How you choose to aggregate your time series data is an important decision and depends on the story you want to tell with your data. It’s common to use different aggregations to visualize the same time series data in different ways.

聚合时间序列
根据测量的内容,数据可能会有很大差异。如果您想比较比测量间隔更长的周期,该怎么办?如果你每小时测量一次温度,你每天会得到24个数据点。要比较多年来8月份的温度,你必须将31乘以24的数据点合并为一个。
组合一组测量值称为聚合。有几种方法可以聚合时间序列数据。以下是一些常见的:

  • Average返回所有值的总和除以值的总数。
  • Min和Max返回集合中的最小值和最大值。
  • Sum返回集合中所有值的总和。
  • Count返回集合中的值的数目。
    例如,通过汇总一个月的数据,您可以确定2017年8月的平均气温比前一年高。相反,为了了解哪个月的温度最高,你可以比较每个月的最高温度。
    如何选择汇总时间序列数据是一个重要的决定,取决于你想用数据讲述的故事。使用不同的聚合以不同的方式可视化相同的时间序列数据是很常见的。
Time series and monitoring
In the IT industry, time series data is often collected to monitor things like infrastructure, hardware, or application events. Machine-generated time series data is typically collected with short intervals, which allows you to react to any unexpected changes, moments after they occur. As a consequence, data accumulates at a rapid pace, making it vital to have a way to store and query data efficiently. As a result, databases optimized for time series data have seen a rise in popularity in recent years.
Time series databases
A time series database (TSDB) is a database explicitly designed for time series data. While it’s possible to use any regular database to store measurements, a TSDB comes with some useful optimizations.
Modern time series databases take advantage of the fact that measurements are only ever appended, and rarely updated or removed. For example, the timestamps for each measurement change very little over time, which results in redundant data being stored.

时间序列和监测
在IT行业中,时间序列数据通常用于监控基础设施、硬件或应用程序事件。机器生成的时间序列数据通常以短时间间隔收集,这使您能够在任何意外变化发生后立即对其做出反应。因此,数据积累速度很快,因此有一种有效存储和查询数据的方法至关重要。因此,近年来,为时间序列数据优化的数据库越来越受欢迎。
时间序列数据库
时间序列数据库(TSDB)是为时间序列数据明确设计的数据库。虽然可以使用任何常规数据库来存储测量值,但TSDB附带了一些有用的优化。
现代时间序列数据库利用了这样一个事实,即测量值只被附加,很少更新或删除。例如,每次测量的时间戳随时间变化很小,这导致存储冗余数据。
Look at this sequence of Unix timestamps: 1572524345, 1572524375, 1572524404, 1572524434, 1572524464
Looking at these timestamps, they all start with 1572524, leading to poor use of disk space. Instead, we could store each subsequent timestamp as the difference, or delta, from the first one:
1572524345, +30, +29, +30, +30
We could even take it a step further, by calculating the deltas of these deltas: 1572524345, +30, -1, +1, +0

If measurements are taken at regular intervals, most of these delta-of-deltas will be 0. Because of optimizations like these, TSDBs use drastically less space than other databases.
Another feature of a TSDB is the ability to filter measurements using tags. Each data point is labeled with a tag that adds context information, such as where the measurement was taken. Here’s an example of the InfluxDB data format that demonstrates how each measurement is stored.
weather,location=us-midwest temperature=82 1465839830100400200
  |    -------------------- --------------  |
  |             |             |             |
  |             |             |             |
+-----------+--------+-+---------+-+---------+
|measurement|,tag_set| |field_set| |timestamp|
+-----------+--------+-+---------+-+---------+

如果以规则的间隔进行测量,那么这些增量中的大多数将为0。由于这样的优化,TSDB使用的空间比其他数据库少得多。
TSDB的另一个功能是能够使用标签过滤测量结果。每个数据点都标记有一个标签,该标签添加了上下文信息,例如测量的位置。下面是InfluxDB数据格式的一个示例,它演示了如何存储每个度量值。
Here are some of the TSDBs supported by Grafana:

  • Graphite
  • InfluxDB
  • Prometheus
Collecting time series data
Now that we have a place to store our time series, how do we actually gather the measurements? To collect time series data, you’d typically install a collector on the device, machine, or instance you want to monitor. Some collectors are made with a specific database in mind, and some support different output destinations.
Here are some examples of collectors:
    collectd
    statsd
    Prometheus exporters
    Telegraf
A collector either pushes data to a database or lets the database pull the data from it. Both methods come with their own set of pros and cons:

收集时间序列数据
既然我们有了一个存储时间序列的地方,我们如何真正收集测量结果?要收集时间序列数据,通常需要在要监视的设备、机器或实例上安装收集器。有些收集器考虑到了特定的数据库,有些收集器支持不同的输出目的地。
收集器要么将数据推送到数据库,要么让数据库从中提取数据。这两种方法都有自己的优缺点:

Pros Cons
Push Easier to replicate data to multiple destinations. The TSDB has no control over how much data gets sent.
Pull Better control of how much data that gets ingested, and its authenticity. Firewalls, VPNs or load balancers can make it hard to access the agents.
Since it would be inefficient to write every measurement to the database, collectors pre-aggregate the data and write to the time series database at regular intervals.

由于将每个测量值写入数据库的效率很低,收集器会预先聚合数据,并定期写入时间序列数据库。