[go: up one dir, main page]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dropping of Iceberg tables, whose metadata is deleted from its S3 storage, taking too much time than expected. #23511

Closed
dmariamgeorge opened this issue Aug 23, 2024 · 2 comments
Labels

Comments

@dmariamgeorge
Copy link
Contributor
dmariamgeorge commented Aug 23, 2024

The SQL statement DROP TABLE <table_name> for Iceberg tables, where the associated table metadata has already been deleted from the S3 storage, is taking significantly longer time than anticipated. The delay in dropping these Iceberg tables, where the metadata has already been deleted from the S3 storage, is due to multiple retries during the process of checking if the table still exists.

Environment

  • Presto version used: v0.289
  • Storage (HDFS/S3/GCS..): S3
  • Data source and connector used: Iceberg

Expected Behavior

If the metadata for the table is missing, it should skip retry, when the FileNotFoundException is thrown. It should be quickly concluded that the table no longer exists and complete the drop operation without excessive retry delays.

Current Behavior

Now even if the FileNotFoundException is thrown around 20 retries are happening, before the table is getting dropped.

Possible Solution

Avoid retries once a FileNotFoundException is thrown.

PR: #23510

Steps to Reproduce

  1. Connect Presto to an S3 storage through an Iceberg connector.
  2. Create a schema, and within the schema create a table.
  3. Now, delete the metadata and the bucket.
  4. Try dropping the associated the table.

Context

Drop table operation is not completing within the expected timeframe, causing potential disruptions or delays in the workflow.

@github-project-automation github-project-automation bot moved this to 🆕 Unprioritized in Bugs and support requests Aug 23, 2024
@dmariamgeorge dmariamgeorge changed the title Dropping of Iceberg/Hive tables, whose metadata is deleted from its S3 storage, taking too much time than expected. Dropping of Iceberg tables, whose metadata is deleted from its S3 storage, taking too much time than expected. Aug 24, 2024
@ZacBlanco
Copy link
Contributor

@dmariamgeorge can you close this issue if the merged PR solves the problem?

@dmariamgeorge
Copy link
Contributor Author

Sure @ZacBlanco. Closing the issue. Fixed with PR: #23510.

@github-project-automation github-project-automation bot moved this from 🆕 Unprioritized to ✅ Done in Bugs and support requests Sep 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Archived in project
Development

No branches or pull requests

2 participants