Skip to content

Commit

Permalink
TruncateTableCommand Logical Command
Browse files Browse the repository at this point in the history
  • Loading branch information
jaceklaskowski committed Mar 21, 2024
1 parent 9a6185d commit c071c93
Show file tree
Hide file tree
Showing 2 changed files with 41 additions and 17 deletions.
12 changes: 12 additions & 0 deletions docs/SessionCatalog.md
Original file line number Diff line number Diff line change
Expand Up @@ -313,3 +313,15 @@ registerFunction[T](
* `CreateFunctionCommand` is executed
* `RefreshFunctionCommand` is executed
* `SessionCatalog` is requested to [resolvePersistentFunctionInternal](#resolvePersistentFunctionInternal)

## Looking Up Table Metadata { #getTableMetadata }

```scala
getTableMetadata(
name: TableIdentifier): CatalogTable
```

`getTableMetadata` [getTableRawMetadata](#getTableRawMetadata) and replaces `CharType` and `VarcharType` field types with `StringType` in the table schema, recursively.

!!! note "`CharType` and `VarcharType` Unsupported"
The Spark SQL query engine does not support char/varchar types yet.
46 changes: 29 additions & 17 deletions docs/logical-operators/TruncateTableCommand.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,35 +17,47 @@ title: TruncateTableCommand

* [ResolveSessionCatalog](../logical-analysis-rules/ResolveSessionCatalog.md) logical resolution rule is executed

<!---
=== [[run]] Executing Logical Command -- `run` Method
## Executing Command { #run }

[source, scala]
----
run(spark: SparkSession): Seq[Row]
----
??? note "RunnableCommand"

NOTE: `run` is part of RunnableCommand.md#run[RunnableCommand Contract] to execute (_run_) a logical command.
```scala
run(
spark: SparkSession): Seq[Row]
```

`run`...FIXME
`run` is part of the [RunnableCommand](RunnableCommand.md#run) abstraction.

`run` throws an `AnalysisException` when executed on [external tables](../CatalogTable.md#tableType):
`run` requests the given [SparkSession](../SparkSession.md) for the [SessionCatalog](../SessionState.md#catalog) (via [SessionState](../SparkSession.md#sessionState)).

```text
Operation not allowed: TRUNCATE TABLE on external tables: [tableIdentWithDB]
```
`run` requests the `SessionCatalog` for the [table metadata](../SessionCatalog.md#getTableMetadata) by the given [tableName](#tableName).

`run` finds the paths of the partitions (if specified) or the main directory of the table and deletes them one by one, recursively.

??? note "Uses Apache Hadoop to Delete Paths"
`run` uses Apache Hadoop's [FileSystem.delete]({{ hadoop.api }}/org/apache/hadoop/fs/FileSystem.html#delete-org.apache.hadoop.fs.Path-boolean-) to delete the directories.

Once deleted, `run` re-creates the paths (so they are empty directories), possibly re-creating their file permissions (based on the [spark.sql.truncateTable.ignorePermissionAcl.enabled](../configuration-properties.md#spark.sql.truncateTable.ignorePermissionAcl.enabled) configuration property).

`run` throws an `AnalysisException` when executed on [views](../CatalogTable.md#tableType):
??? note "Uses Apache Hadoop to Re-Create Paths"
`run` uses Apache Hadoop's [FileSystem.mkdirs]({{ hadoop.api }}/api/org/apache/hadoop/fs/FileSystem.html#mkdirs-org.apache.hadoop.fs.Path-) to re-create the directories.

After deleting the data, `run` requests the [Catalog](../SparkSession.md#catalog) to [refresh the table](../Catalog.md#refreshTable) followed by [updating the table statistics](../CommandUtils.md#updateTableStats).

`run` returns no `Row`s.

### Exceptions

`run` throws an `AnalysisException` when executed on [external tables](../CatalogTable.md#tableType):

```text
Operation not allowed: TRUNCATE TABLE on views: [tableIdentWithDB]
Operation not allowed: TRUNCATE TABLE on external tables: [tableIdentWithDB]
```

`run` throws an `AnalysisException` when executed with <<partitionSpec, TablePartitionSpec>> on [non-partitioned tables](../CatalogTable.md#partitionColumnNames):
`run` throws an `AnalysisException` when executed with the [TablePartitionSpec](#partitionSpec) on [non-partitioned tables](../CatalogTable.md#partitionColumnNames):

```text
Operation not allowed: TRUNCATE TABLE ... PARTITION is not supported for tables that are not partitioned: [tableIdentWithDB]
```

`run` throws an `AnalysisException` when executed with <<partitionSpec, TablePartitionSpec>> with filesource partition disabled or partition metadata not in a Hive metastore.
-->
`run` throws an `AnalysisException` when executed with the [TablePartitionSpec](#partitionSpec) with filesource partition disabled or partition metadata not in a Hive metastore.

0 comments on commit c071c93

Please sign in to comment.