# Downsampling

Scrutiny collects alot of data, that can cause the database to grow unbounded. 

- Smart data
- Smart test data
- Temperature data
- Disk metrics (capacity/usage)
- etc

This data must be accurate in the short term, and is useful for doing trend analysis in the long term.
However, for trend analysis we only need aggregate data, individual data points are not as useful.

Scrutiny will automatically downsample data on a schedule to ensure that the database size stays reasonable, while still
ensuring historical data is present for comparisons.


| Bucket Name | Retention Period | Downsampling Range | Downsampling Aggregation Window | Downsampling Cron | Comments |
| --- | --- | --- | --- | --- | --- |
| `metrics` | 15 days | `-2w -1w` | `1w` | main bucket, weekly on Sunday at 1:00am |
| `metrics_weekly` | 9 weeks | `-2mo -1mo` | `1mo` | monthly on first day of the month at 1:30am
| `metrics_monthly` | 25 months | `-2y -1y` | `1y` | yearly on the first day of the year at 2:00am
| `metrics_yearly` | forever | - | - | - | |


After 5 months, here's how may data points should exist in each bucket for one disk

| Bucket Name | Datapoints | Comments |
| --- | --- | --- |
| `metrics` | 15 | 7 daily datapoints , up to 7 pending data, 1 buffer data point |
| `metrics_weekly` | 9 | 4 aggregated weekly data points, 4 pending datapoints, 1 buffer data point |
| `metrics_monthly` | 3 | 3 aggregated monthly data points | 
| `metrics_yearly` | 0 | |

After 5 years, here's how may data points should exist in each bucket for one disk

| Bucket Name | Datapoints | Comments |
| --- | --- | --- |
| `metrics` | - | - |
| `metrics_weekly` | - | 
| `metrics_monthly` | - |
| `metrics_yearly` | - |