Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support cooldown to external lake table #51649

Open
hoffermei opened this issue Oct 9, 2024 · 2 comments · May be fixed by #45536
Open

Support cooldown to external lake table #51649

hoffermei opened this issue Oct 9, 2024 · 2 comments · May be fixed by #45536

Comments

@hoffermei
Copy link
Contributor

Feature request

Is your feature request related to a problem? Please describe.

As data stored in olap table and the limit of local storage, we offten set a ttl(dynamic.start property) for local partition. When a partition reach it's ttl time, it will be delete by dynamic partition scheduler. If we config a table with big ttl, we must use more machine to store history data.

Another case is we sometimes need do offline analysis, expect to be independent with real time compute.

Describe the solution you'd like

Add a mechanism to cooldown olap table partition to external lake table, then history data could be save to lake table, offline analysis can be do on lake table with independent compute nodes.

Describe alternatives you've considered

Additional context

@hoffermei
Copy link
Contributor Author

pr #45536

hoffermei added a commit to hoffermei/starrocks that referenced this issue Oct 9, 2024
hoffermei added a commit to hoffermei/starrocks that referenced this issue Oct 9, 2024
hoffermei added a commit to hoffermei/starrocks that referenced this issue Oct 9, 2024
hoffermei added a commit to hoffermei/starrocks that referenced this issue Oct 9, 2024
hoffermei added a commit to hoffermei/starrocks that referenced this issue Oct 9, 2024
hoffermei added a commit to hoffermei/starrocks that referenced this issue Oct 9, 2024
hoffermei added a commit to hoffermei/starrocks that referenced this issue Oct 9, 2024
hoffermei added a commit to hoffermei/starrocks that referenced this issue Oct 10, 2024
hoffermei added a commit to hoffermei/starrocks that referenced this issue Oct 10, 2024
hoffermei added a commit to hoffermei/starrocks that referenced this issue Oct 10, 2024
@LiShuMing
Copy link
Contributor

After cooldown, what's the behavior of querying partitions (cooldown) ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants