diff --git a/docs/user-guide/admin/data-tiers/_category_.json b/docs/user-guide/admin/data-tiers/_category_.json new file mode 100644 index 00000000..7c89ad16 --- /dev/null +++ b/docs/user-guide/admin/data-tiers/_category_.json @@ -0,0 +1,8 @@ +{ + "label": "Data Tiers", + "position": 3, + "link": { + "type": "generated-index", + "description": "Logz.io offers different levels of data access and retention: hot for frequently accessed data, warm for intermediate storage, and cold for long-term archival." + } +} diff --git a/docs/user-guide/log-management/cold-tier.md b/docs/user-guide/admin/data-tiers/cold-tier.md similarity index 98% rename from docs/user-guide/log-management/cold-tier.md rename to docs/user-guide/admin/data-tiers/cold-tier.md index 7c365ae3..08097a68 100644 --- a/docs/user-guide/log-management/cold-tier.md +++ b/docs/user-guide/admin/data-tiers/cold-tier.md @@ -1,9 +1,10 @@ --- -sidebar_position: 11 +sidebar_position: 3 title: Cold Tier description: Search your archived data before re-ingesting it with Cold Tier image: https://dytvr9ot2sszz.cloudfront.net/logz-docs/social-assets/docs-social.jpg keywords: [logz.io, cold tier, storage, archive, opensearch dashboards, log analysis, observability] +slug: /user-guide/log-management/cold-tier/ --- diff --git a/docs/user-guide/admin/data-tiers/data-tiers-intro.md b/docs/user-guide/admin/data-tiers/data-tiers-intro.md new file mode 100644 index 00000000..d542aad9 --- /dev/null +++ b/docs/user-guide/admin/data-tiers/data-tiers-intro.md @@ -0,0 +1,61 @@ +--- +sidebar_position: 1 +title: Data Storage Tiers Overview +image: https://dytvr9ot2sszz.cloudfront.net/logz-docs/social-assets/docs-social.jpg +description: Everything you need to know about Logz.io's Warm Tier solution +keywords: [manage, log redaction, admin controls, account administration, access control, warm, warm tier, warm logs, archive] +--- + +Logz.io offers three distinct storage tiers: **Hot**, **Warm**, and **Cold**, designed to optimize data access, retention, and cost. Each tier suits different data usage patterns and needs, allowing you to tailor your storage to specific requirements. + +:::info note +Warm and Cold tier configurations can be set up by contacting the [Sales team](mailto:sales@logz.io). +::: + +## Hot Tier + +The Hot tier is designed for data that requires immediate and frequent access. It's the logs that are ingested daily into your account, providing the fastest retrieval speeds, ideal for recent logs, metrics, or traces actively used in analysis, monitoring, and troubleshooting. + +Hot tier data is available in real-time or near real-time, allowing you to identify, troubleshoot, and query ongoing active monitoring, alerts, and analytics, and is typically accessed within a shorter retention window. + +## Warm Tier + +The Warm tier is designed to handle logs that don’t require frequent access but still need to be readily available for analysis and troubleshooting. By enabling Warm tier data, you can access historical data when necessary, making it a practical choice for mid-term storage. + +Data transitions to the Warm Tier **only after** stored in the Hot Tier. All Hot Tier data moves to the Warm Tier based on the retention period you configure. When enabling the Warm Tier, the minimum retention period for the Hot Tier is set to 4 days. + +Data in the Warm tier remains readily accessible, but with slightly slower retrieval times. Use the Warm tier for data that: + +* Is relevant for medium-term retention and analysis +* Balances cost with retrieval speed, allowing less-frequent access at a lower cost than the Hot tier + +## Cold Tier + +The Cold tier is intended for long-term data storage, optimized for cost over speed. This tier is ideal for archived data that rarely needs access but must be retained for compliance, auditing, or historical records. Data retrieval from the Cold tier may take longer, making it suitable for data that: + +* Requires long-term retention for regulatory compliance or record-keeping +* Is rarely accessed but must remain available if needed + +## Archive & Restore + +Archiving allows you to move rarely accessed data to a more cost-effective, long-term storage solution of logs that don't need to be instantly searchable. When you need to investigate old logs after they have expired from your account, you can restore them to a temporary account with no additional storage costs. + + +When to Use Archiving: + +* When data is no longer actively used but must be retained for compliance, historical analysis, or regulatory requirements. +* To optimize storage costs without permanently deleting data. + +By effectively managing your data lifecycle with archiving and restoring, you can maintain performance and cost-efficiency while ensuring long-term accessibility of critical information. + +[Read more about Logz.io's Archive & Restore options](https://docs.logz.io/docs/user-guide/data-hub/archive-restore/archive-and-restore/). + + + +## Choosing the right tier for your data + +When choosing which tier to store your data in, consider how often you need to access your data and the importance of immediate retrieval. + +The Hot tier is best for data that’s critical for immediate analysis and monitoring. The Warm tier is suitable for data that’s accessed occasionally but remains important for historical insights. The Cold tier is ideal for archived data that must be retained but is rarely accessed. + +By selecting the appropriate storage tier, you can effectively manage data access and costs based on your specific needs. \ No newline at end of file diff --git a/docs/user-guide/admin/data-tiers/warm-tier.md b/docs/user-guide/admin/data-tiers/warm-tier.md new file mode 100644 index 00000000..88e7c950 --- /dev/null +++ b/docs/user-guide/admin/data-tiers/warm-tier.md @@ -0,0 +1,43 @@ +--- +sidebar_position: 2 +title: Warm Tier +image: https://dytvr9ot2sszz.cloudfront.net/logz-docs/social-assets/docs-social.jpg +description: Everything you need to know about Logz.io's Warm Tier solution +keywords: [manage, log redaction, admin controls, account administration, access control, warm, warm tier, warm logs, archive] +--- + +Warm tier is a storage option designed for data that is not frequently accessed but still needs to be readily available for analysis and troubleshooting. It provides a practical solution for mid-term storage ensuring you can retrieve historical data when required. + +:::info note +Warm tier configuration can be set up by contacting the [Sales team](https://logz.io/contact-us/). +::: + +Prerequisites + +Data gets to warm only after it's been to hot. All hot data goes to warm data according to the retention period configured by you. When you enable Warm Tier, the retention minimum for Hot tier is 4 days. + +## Setting Up the Warm Tier + +Once Warm Tier has been enabled on your Logz.io account, you can assign it to one or more of your sub-accounts. To do so, navigate to [Manage Accounts](https://app.logz.io/#/dashboard/settings/manage-accounts), select the sub account you want to configure, choose the desired Warm Tier retention period, and click **Apply** to save your changes. + +warm-tier-button + +Data transitions to the Warm Tier **only after** stored in the Hot Tier. All Hot Tier data moves to the Warm Tier based on the retention period you configure. When enabling the Warm Tier, the minimum retention period for the Hot Tier is set to 4 days. + + +## Accessing Warm Tier Data + +After activation, all sub account users can access Warm Tier data through [Explore](https://app.logz.io/#/dashboard/explore). + +:::tip +When querying Warm Tier data, limit the time range, use filters, and focus on key information to improve speed. Warm Tier data has a 5-minute timeout; you'll be notified if retrieval fails. +::: + +Open the time picker, where indicators show the availability of Warm or Hot data. + +When selecting a date containing Warm Tier data, a notification will inform you that retrieval may take several minutes. To proceed, confirm your date selection by clicking **Apply** in the time picker, then click **Run Query** in the main Explore query bar. This manual step ensures intentional data retrieval, as Warm Tier data may take longer to load and incur additional costs. + + +![warm data activate](https://dytvr9ot2sszz.cloudfront.net/logz-docs/data-tiers/both-times-warm-data.png) + +While waiting for results, you can navigate away from Explore. You’ll receive an in-app notification once the data is ready.