Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ILM the delete action waits for a TSDS index time/bounds to lapse #100207

Merged
merged 2 commits into from Oct 4, 2023

Conversation

andreidan
Copy link
Contributor

This adds the check-ts-end-time-passed step to the delete action so, for time series indices, we wait until they're not meant to accept new writes anymore before deleting them (i.e. until the time interval that covers the ingestion for now has lapsed)

Relates to #99696

This adds the `check-ts-end-time-passed` step to the delete action so,
for time series indices, we wait until they're not meant to accept new
writes anymore before deleting them (i.e. until the time interval the covers
the ingestion for `now` has lapsed)
@andreidan andreidan added >bug :Data Management/ILM+SLM Index and Snapshot lifecycle management v8.11.0 v8.10.3 labels Oct 3, 2023
@elasticsearchmachine elasticsearchmachine added the Team:Data Management Meta label for data/management team label Oct 3, 2023
@elasticsearchmachine
Copy link
Collaborator

Hi @andreidan, I've created a changelog YAML for you.

@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-data-management (Team:Data Management)

Copy link
Member

@jbaiera jbaiera left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@martijnvg martijnvg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@andreidan andreidan merged commit f32c8f5 into elastic:main Oct 4, 2023
12 checks passed
@andreidan andreidan added the auto-backport-and-merge Automatically create backport pull requests and merge when ready label Oct 4, 2023
andreidan added a commit to andreidan/elasticsearch that referenced this pull request Oct 4, 2023
…astic#100207)

This adds the `check-ts-end-time-passed` step to the delete action so,
for time series indices, we wait until they're not meant to accept new
writes anymore before deleting them (i.e. until the time interval the covers
the ingestion for `now` has lapsed)

(cherry picked from commit f32c8f5)
Signed-off-by: Andrei Dan <andrei.dan@elastic.co>
@andreidan andreidan removed the auto-backport-and-merge Automatically create backport pull requests and merge when ready label Oct 4, 2023
elasticsearchmachine pushed a commit that referenced this pull request Oct 4, 2023
…00207) (#100261)

This adds the `check-ts-end-time-passed` step to the delete action so,
for time series indices, we wait until they're not meant to accept new
writes anymore before deleting them (i.e. until the time interval the covers
the ingestion for `now` has lapsed)

(cherry picked from commit f32c8f5)
Signed-off-by: Andrei Dan <andrei.dan@elastic.co>
@rodrigc
Copy link
Contributor

rodrigc commented Oct 11, 2023

@andreidan for this ILM fix, are there any changes required to the Kibana screens for configuring ILM policies,
or is this fix "under the covers", and not required for configuring ILM policies?

@andreidan
Copy link
Contributor Author

andreidan commented Oct 12, 2023

@rodrigc That's a great question. I don't know of a good place where some additional information about the delays that ILM will incour due to the time series end_time would be helpful.
Perhaps when creating an index template that has index.mode: time_series and an ILM policy (or, in 8.11 onwards a lifecycle object) configured?
@martijnvg what do you think? Is there a more dedicated TSDB screen in Kibana?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :Data Management/ILM+SLM Index and Snapshot lifecycle management Team:Data Management Meta label for data/management team v8.10.3 v8.11.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants