use tsql-ttds for structure_load (fixes #956)

[jonahgeorge]

This PR modifies structure_load to utilize tsql-ttds to load the structure.sql file generated by structure_dump. With this change, we can remove the gsub to remove GO commands which should allow views to be properly imported.

Fixes #956

[ttilberg]

@jonahgeorge Did you close this for any particular reason? Did you find an alternate solution?

I'm trying to support a join table using a view in my Rails app, and ye ole 'CREATE VIEW' must be the first statement in a query batch. is in my way after fixing the unquoted [key] column name in AR Metadata...

• Did you solve this a different way?
• Did you find other strange issues you had to overcome using SQL Server + structure.sql?
• Did you give up on sqlserver? 🤣

[travismiller]

I ran into the same [key] and first statement in a query batch issues using db/structure.sql. These changes worked for me and this feels like the right change since it uses the same toolset for both dump and load.

[jonahgeorge]

@jonahgeorge Did you close this for any particular reason? Did you find an alternate solution?

I'm trying to support a join table using a view in my Rails app, and ye ole 'CREATE VIEW' must be the first statement in a query batch. is in my way after fixing the unquoted [key] column name in AR Metadata...

• Did you solve this a different way?
• Did you find other strange issues you had to overcome using SQL Server + structure.sql?
• Did you give up on sqlserver? 🤣

It appears I missed this notification way back when 🙈 Sorry about that! I'm operating on a slightly hazy old memory here, so bear with me.

• Did you solve this a different way?
• Did you find other strange issues you had to overcome using SQL Server + structure.sql?

I recall having issues with the format generated by structure_dump here primarily due to the usage of GO statements. I ended up using a structure.sql file generated by DataGrip & then partially hand edited as I could not get any CLI tooling to work for my use-case.

• Did you give up on sqlserver? 🤣

Hah, I did not; however, the current solution is subpar-- We don't redump the structure on migrations and instead rely on only the migration chain.

---

I may have some time in the coming months to take another look at this; especially, due to motivation to move off of my fork for Rails 7.1 support. I'll post any updates on this PR and potentially will re-open it.

[travismiller]

Probably a better place for this, but I'll leave it here for now.

I've also encountered an issue in the dumped structure.sql file where the default NEWID is not being included for UUID primary keys.

[jonahgeorge]

A few recent findings:

My initial attempts at using the schema created by defncopy were unsuccessful. It appears this tool has a handful of issues including limited support for stored procedures, improperly escaped names, etc & I'm not motivated to write C lately 😄

A while back I got around this by using a combination SQL Scripts > Generate DDL to Clipboard feature in Jetbrains DataGrip + config.active_record.dump_schema_after_migration = false but really disliked not having an open source or scriptable way to do this. Additionally, as the structure file is not kept up-to-date with migrations, your test suite execution time may suffer as your migration chain grows.

---

In looking for more recent tools for schema dumping I came across microsoft/mssql-scripter-- this is a thin Python wrapper around the microsoft/sqltoolsservice C# JSONRPC server.

A quick test of commands below (with the patches in this PR & some minimal manual editing of the generated file) applied successfully:

pip install mssql-scripter

mssql-scripter \
  -S localhost \
  -U "user" \
  -P "password" \
  -d "database" \
  --display-progress \
  --enable-toolsservice-logging > db/structure.sql

rails db:create && rails db:schema:load

However, as the migrations table is not populated it causes issues with running tests. I'll have to look at how the Postgres adapter includes data for a subset of tables. Additionally, it took nearly 3 minutes to dump my entire schema (~1,200 objects).

Potentially a path forward for the library would be to depend on one of the projects listed above. Alternatively, I was able to track down the majority of the schema generation logic to the following package but it would be a mountain of work to re-implement in Ruby.

https://github.com/microsoft/sqlmanagementobjects/blob/main/src/Microsoft/SqlServer/Management/SqlScriptPublish

One last alternative could be a Ruby re-write of defncopy. Looking through its implementation, it's primarily parsing the output of sp_help and then constructing creation scripts based on it. I'd imagine to duplicate its current logic in Ruby would be fairly minimal and would be a better foundation for implementing other features.

[Michoels]

The FreeTDS guys have a fix for defncopy on the way: FreeTDS/freetds#429 (comment)