Manage data retention
Stack Serverless
Use the Data retention tab on the Manage stream page to set how long your stream retains data and to get insight into your stream's data ingestion and storage size.
The Data retention page is made up of the following components that can help you determine how long you want your stream to retain data:
- Retention period: The minimum number of days after which the data is deleted
- Source: The origin of the data retention policy.
- Last updated: When data retention was last updated for the selected stream.
- Ingestion: Estimated ingestion per day and month, calculated based on the total size of all data in the stream divided by the stream's age. This is an estimate, and the actual ingestion may vary.
- Total doc count: The total number of documents in the stream.
- Ingestion Rate: Estimated ingestion rate per time bucket. The bucket interval is dynamic and adjusts based on the selected time range. The ingestion rate is calculated using the average document size in the stream multiplied by the number of documents in each bucket. This is an estimate, and the actual ingestion rate may vary.
- Policy summary: Stack The amount of data ingested per phase (hot, warm, cold).
From any stream page, select Edit data retention to change how long your data stream retains data.
The Retention period is the minimum number of days after which the data is deleted. To set data retention to a specific time period:
- Select Edit data retention → Set specific retention days.
- From here, set the period of time you want to retain data for this stream.
To define a global default retention policy, refer to project settings.
Stack
ILM policies let you automate and standardize data retention across streams and other data streams. To have your streams follow an existing policy:
- Select Edit data retention → Use a lifecycle policy.
- Select a pre-defined ILM policy from the list.
You can also create a new ILM policy. Refer to Configure a lifecycle policy for more information.