Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AutoMigrate can break clickhouse data if you change a column type #125

Open
SergFilippov opened this issue Aug 15, 2023 · 0 comments
Open
Assignees

Comments

@SergFilippov
Copy link

SergFilippov commented Aug 15, 2023

GORM Playground Link

go-gorm/playground#639

Description

When changing the column type AutoMigrate performs ALTER TABLE MODIFY COLUMN on each field every time migration is started.

If the old type is compatible with the new one (e.g. Int32->Int64), the type is changed and there seems to be no problem. But if not (e.g. from String to Int64) then ALTER TABLE MODIFY COLUMN is executed and a mutation is created which fails with an error:

code: 341, message: Exception happened during execution of mutation 'mutation_2.txt' with part '202308_1_1_0' reason: 'Code: 6. DB::Exception: Cannot parse string 'string value' as Int32: syntax error at begin of string. Note: there are toInt32OrZero and toInt32OrNull functions, which returns zero/NULL instead of throwing exception.: while executing 'FUNCTION _CAST(value :: 0, 'Int32' :: 2) -> _CAST(value, 'Int32') Int32 : 3': (while reading from part /var/lib/clickhouse/store/a35/a3515250-be07-4ea7-a399-d406026db762/202308_1_1_0/ located on disk default of type local): While executing MergeTreeSequentialSource. (CANNOT_PARSE_TEXT) (version 23.7.2.25 (official build))'. This error maybe retryable or not. In case of unretryable error, mutation can be killed with KILL MUTATION query

This actually changes the data type in Clickhouse and breaks data. The query "SELECT * FROM events" gives an error:

Code: 6. DB::Exception: Cannot parse string 'string value' as Int32: syntax error at begin of string. Note: there are toInt32OrZero and toInt32OrNull functions, which returns zero/NULL instead of throwing exception.: while executing 'FUNCTION _CAST(value :: 0, 'Int32' :: 2) -> _CAST(value, 'Int32') Int32 : 3': (while reading from part /var/lib/clickhouse/store/50c/50c405ac-c069-4d7f-b816-ec2aec6ed96a/202308_1_1_0/ located on disk default of type local): While executing MergeTreeInOrder. (CANNOT_PARSE_TEXT) (version 23.7.2.25 (official build))

You can see the example of code in main.go in playground above

Expected behavior

If the old type is incompatible with the new AutoMigrate does not execute the ALTER TABLE MODIFY COLUMN query, the mutation is not created.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants