Skip to content
This repository has been archived by the owner on Feb 22, 2023. It is now read-only.

Create SQLAlchemy ORM models for Climsoft CDMS #2

Closed
isedwards opened this issue Jul 19, 2021 · 14 comments · Fixed by #28
Closed

Create SQLAlchemy ORM models for Climsoft CDMS #2

isedwards opened this issue Jul 19, 2021 · 14 comments · Fixed by #28

Comments

@isedwards
Copy link
Contributor

isedwards commented Jul 19, 2021 •
edited
Loading

This issue will begin to add support for the Climsoft CDMS. You will need an instance of MySQL 8+ installed and then to restore the Climsoft 4.1.1 SQL DDL file here.

When the Climsoft software is installed, the SQL DDL creates two databases:

CREATE DATABASE IF NOT EXISTS `mariadb_climsoft_db_v4`
CREATE DATABASE IF NOT EXISTS `mariadb_climsoft_test_db_v4`

The first is used as the actual climate database whilst the second contains some test data that is used for training with tutorials.

We only need to create models for mariadb_climsoft_db_v4. The models should be created in /opencdms/models/climsoft/v4_1_1.py.

Once we have the models, we should try using the get_schema_diff function from opencdms.models to check our new models against the second database schema (mariadb_climsoft_test_db_v4) to see whether it is identical. It's possible that there may be minor differences.

There is one final complication: some of the tables in Climsoft do not have primary keys explicitly specified - this prevents SQLAlchemy/sqlacodegen from creating ORM models (see docs here). The attached PDF describes where primary keys should be specified.

We should discuss our options for working around the primary key issue. We can either:

  • Submit a change request to the Climsoft project to ALTER TABLE and add the missing primary keys, this will then appear in future releases of Climsoft
  • Alter our local copy of the schema to make it appear that primary keys are specified. If my understanding is correct, we should then be able to use our models (that explicitly set the primary keys) to connect to current versions of the database where many primary keys are not explicitly defined, but everything will work correctly.

I'm very happy to discuss this more - either in GitHub comments, messaging or scheduling a call...
Climsoft Data Model Tables and Keys.pdf
@isedwards isedwards transferred this issue from opencdms-components/opencdms-rclimatic Jul 25, 2021
@isedwards isedwards mentioned this issue Aug 1, 2021
Open
@isedwards isedwards changed the title Create SQLAlchemy ORM models for supports CDMSs Create SQLAlchemy ORM models for 3 supported open-source CDMSs Aug 1, 2021
@isedwards isedwards changed the title Create SQLAlchemy ORM models for 3 supported open-source CDMSs Create SQLAlchemy ORM models for Climsoft CDMS Aug 3, 2021
@faysal-ishtiaq faysal-ishtiaq mentioned this issue Aug 18, 2021
Closed
@Shaibujnr
Copy link
Contributor

@isedwards Concerning the primary key issue, I'd rather we go with option 2 at the moment. However I am having difficulty understanding the table in the pdf. It would be nice to have a more detailed explanation. We could also discuss in a call.

@isedwards
Copy link
Contributor Author

@Shaibujnr I have a bit more information on this from climsoft/climsoft-database#2 (comment)

We should start by only creating models for the 24 main tables that are considered to "constitute the relational data model". These 24 tables are listed in the PDF file above. The other tables should be ignored for now.

These 24 tables either have a 'single column' primary key already specified - or they have a unique index on a composite key that we can use as the primary key.

Taking the first two tables from the PDF, these have:

UNIQUE INDEX `obsFinalIdentification` (`recordedFrom`, `describedBy`, `obsDatetime`)
UNIQUE INDEX `obsInitialIdentification` (`recordedFrom`, `describedBy`, `obsDatetime`, `qcStatus`, `acquisitionType`)

We can use @faysal-ishtiaq 's ALTER TABLE commands to add the primary keys for the tables in the PDF that need them.

So...

  1. Run the ALTER TABLE commands on a local copy of the database - just for those of the 24 tables in the PDF file that need a composite primary key added
  2. Find a way to get sqlacodegen to generate models for just the 24 tables in the PDF
  3. Run get_schema_diff() against the automatically generated models for these 24 tables that have had PKs added and paste the diff for us all to see in the comments
  4. Run alembic revision --autogenerate to generate a migration file for the difference between the current database schema and the defined SQLAlchemy models (as here)
  5. On a separate commit, manually change our models so that they match the output of the alembic migration
  6. Run get_schema_diff() again and report any differences
  7. Can any remaining differences be fixed (or explicitly ignored, like we did with ignore_tables in CliDE so that views were not reported)

@Shaibujnr
Copy link
Contributor

Got it @isedwards , I have been able to install mysql locally and run the DDL file, I now have the main database and test database as well.

I am about to create the primary keys for the tables with the unique identifiers. I observed that in the pdf, for table observationfinal, the columns recordedFrom, describedBy, obsDatetime, obsLevel were listed as the unique keys, but on viewing the table properties of my db locally, the columns recordedFrom, describedBy, obsDatetime were the only unique constraints. Is this an error with the pdf or is my db not created properly?

@isedwards
Copy link
Contributor Author

In production, the unique indexes on the Climsoft observationinitial and observationfinal tables are currently missing the obslevel field.

In the case of observationfinal this means that only one 'observation level' can be recorded for each unique [recordedFrom, describedBy, obsDatetime] row and similarly with the observationinitial table.

In practice, I think obslevel is currently not implemented and therefore it always has the default value of NULL (different 'observation levels' are currently captured by having multiple describedBy values that also capture the 'level' in addition to other characteristics).

In OpenCDMS, we should include obslevel in the primary key for correctness. The risk is if the OpenCDMS software was used alonside Climsoft - and OpenCDMS allowed users to record multiple 'observation levels' - then this could lead to the Climsoft software overwritting data because the software does not recognised the obslevel field until a fix is implemented.

Although our model will include obslevel in the primary key, the actual database implementation will not include this in the unique index until a fix is implemented in Climsoft. This means that when OpenCDMS queries an older version of the Climsoft database - the obslevel field will not be indexed. However, I doubt there would be any performance issues given there is only one obslevel per unique row and the value is NULL.

@isedwards
Copy link
Contributor Author

After applying the alter tables commands locally:

ALTER TABLE mariadb_climsoft_db_v4.observationfinal ADD PRIMARY KEY (recordedFrom, describedBy, obsDatetime, obsLevel);
ALTER TABLE mariadb_climsoft_db_v4.observationinitial ADD PRIMARY KEY (recordedFrom, describedBy, obsDatetime, qcStatus,acquisitionType, obsLevel);
ALTER TABLE ...

we should then generate the models with sqlacodegen for the altered version and, given the output of alembic, manually update the models (on a separate commit so we can see the diff). We're then looking for whether:

  1. get_schema_diff() reports no differences with our locally altered database
  2. How to control the output of get_schema_diff() so that it does not report the known difference between our model PKs and the original Climsoft database's unique indexes (whilst ensuring that it continues to report any unexpected differences).

@Shaibujnr
Copy link
Contributor

Shaibujnr commented Sep 5, 2021 •
edited
Loading

I have been able to generate the models for the tables listed in the pdf, all 24 (25 actually) of them. 25 because one of the tables has a relationship to a table not listed in the pdf and therefore sqlacodegen automatically generated that for us. However, I am currently facing a dilemma with the get_schema_diff.

If I run get_schema_diff as is, with only the 24 models defined, against all the tables in the db, it will report all differences from all the tables, including/especially those for which the models have not been generated. Is this the output you'd like to see? It's actually a lot. @isedwards

I have currently updated the get_schema_diff method to accept include_tables and exclude_tables arguments. exclude_tables is exactly the same as ignore_tables. Think of it as blacklisting and whitelisting. exclude_tables will blacklist the tables in the list and therefore not report any operation difference for those tables in the result of get_schema_diff. include_tables on the other hand will ignore every operation not related to the listed tables.

I believe this is the output you'd like to see, am I correct? @isedwards

To describe the issue, i'd first like to describe the output of alembic's compare_metadata. It's simply a list of tuples. The first item in the tuple is a string describing the operation to be performed e.g add_table, remove_table, add_index, remove_index and the second item is the sqlalchemy object for that operation. e.g for the add_table operation, the second item in the tuple will be a Table object that should be added, similarly for remove_table but for operations like add_index the second object in the tuple would be an sqlalchemy Index object that has a table as it's attribute.

Now the issue. To whitelist/blacklist the report i.e constraint the diff to only report differences concerning a specified list of tables like the 24 we have pre-selected. I have to identify what table every operation is targeted at and filter by that table. For operations like add_table, remove_table I can easily pick the second element in the tuple and just filter. But for operations like add_index or remove_index I have to get the table name from the index and then filter. Meaning, the filter logic would be different for every operation group. It's difficult to tell all the operation commands alembic currently supports and the logic to handle them, and therefore there's a high chance we'd get inaccurate results if we don't handle all the possible operations properly.

@Shaibujnr
Copy link
Contributor

This is diff report generated for climsoft without whitelisting or blacklisting.

[   (   'remove_table',
        Table('tm_309052', MetaData(), Column('order', INTEGER(), table=<tm_309052>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x11fe44d90>, for_update=False)), Column('Bufr_Template', VARCHAR(length=50), table=<tm_309052>), Column('CREX_Template', VARCHAR(length=50), table=<tm_309052>), Column('Sequence_Descriptor1', VARCHAR(length=255), table=<tm_309052>), Column('Sequence_Descriptor0', VARCHAR(length=255), table=<tm_309052>), Column('Bufr_Element', VARCHAR(length=255), table=<tm_309052>), Column('Crex_Element', VARCHAR(length=50), table=<tm_309052>), Column('Climsoft_Element', VARCHAR(length=50), table=<tm_309052>), Column('Element_Name', VARCHAR(length=255), table=<tm_309052>), Column('Crex_Unit', VARCHAR(length=25), table=<tm_309052>), Column('Crex_Scale', VARCHAR(length=25), table=<tm_309052>), Column('Crex_DataWidth', VARCHAR(length=25), table=<tm_309052>), Column('Bufr_Unit', VARCHAR(length=255), table=<tm_309052>), Column('Bufr_Scale', INTEGER(), table=<tm_309052>), Column('Bufr_RefValue', BIGINT(), table=<tm_309052>), Column('Bufr_DataWidth_Bits', INTEGER(), table=<tm_309052>), Column('Selected', TINYINT(), table=<tm_309052>), Column('Observation', VARCHAR(length=255), table=<tm_309052>), Column('Crex_Data', VARCHAR(length=30), table=<tm_309052>), Column('Bufr_Data', VARCHAR(length=255), table=<tm_309052>), schema=None)),
    (   'remove_table',
        Table('tm_307086', MetaData(), Column('order', INTEGER(), table=<tm_307086>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x120655160>, for_update=False)), Column('Bufr_Template', VARCHAR(length=50), table=<tm_307086>), Column('Crex_Template', VARCHAR(length=50), table=<tm_307086>), Column('Sequence_Descriptor1', VARCHAR(length=50), table=<tm_307086>), Column('Sequence_Descriptor0', VARCHAR(length=50), table=<tm_307086>), Column('Bufr_Element', VARCHAR(length=50), table=<tm_307086>), Column('Crex_Element', VARCHAR(length=255), table=<tm_307086>), Column('Climsoft_Element', VARCHAR(length=255), table=<tm_307086>), Column('Element_Name', VARCHAR(length=255), table=<tm_307086>), Column('synop_code', VARCHAR(length=255), table=<tm_307086>), Column('unit', VARCHAR(length=255), table=<tm_307086>), Column('scale', VARCHAR(length=255), table=<tm_307086>), Column('width', VARCHAR(length=255), table=<tm_307086>), Column('units_length_scale', VARCHAR(length=255), table=<tm_307086>), Column('data_type', VARCHAR(length=255), table=<tm_307086>), Column('selected', TINYINT(), table=<tm_307086>), Column('observation', VARCHAR(length=255), table=<tm_307086>), Column('Crex_Data', VARCHAR(length=255), table=<tm_307086>), Column('Bufr_Data', VARCHAR(length=50), table=<tm_307086>), schema=None)),
    (   'remove_table',
        Table('tm_307084', MetaData(), Column('order', INTEGER(), table=<tm_307084>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x12065e280>, for_update=False)), Column('Bufr_Template', VARCHAR(length=50), table=<tm_307084>), Column('Crex_Template', VARCHAR(length=50), table=<tm_307084>), Column('Sequence_Descriptor1', VARCHAR(length=50), table=<tm_307084>), Column('Sequence_Descriptor0', VARCHAR(length=50), table=<tm_307084>), Column('Bufr_Element', VARCHAR(length=50), table=<tm_307084>), Column('Crex_Element', VARCHAR(length=255), table=<tm_307084>), Column('Climsoft_Element', VARCHAR(length=255), table=<tm_307084>), Column('Element_Name', VARCHAR(length=255), table=<tm_307084>), Column('synop_code', VARCHAR(length=255), table=<tm_307084>), Column('unit', VARCHAR(length=255), table=<tm_307084>), Column('scale', VARCHAR(length=255), table=<tm_307084>), Column('width', VARCHAR(length=255), table=<tm_307084>), Column('units_length_scale', VARCHAR(length=255), table=<tm_307084>), Column('data_type', VARCHAR(length=255), table=<tm_307084>), Column('selected', TINYINT(), table=<tm_307084>), Column('observation', VARCHAR(length=255), table=<tm_307084>), Column('Crex_Data', VARCHAR(length=255), table=<tm_307084>), Column('Bufr_Data', VARCHAR(length=50), table=<tm_307084>), schema=None)),
    (   'remove_index',
        Index('obsInitialIdentification', Column('StationId', VARCHAR(length=15), table=<qcabslimits>, nullable=False), Column('ElementId', BIGINT(), table=<qcabslimits>), Column('Datetime', DATETIME(), table=<qcabslimits>))),
    (   'remove_table',
        Table('qcabslimits', MetaData(), Column('StationId', VARCHAR(length=15), table=<qcabslimits>, nullable=False), Column('ElementId', BIGINT(), table=<qcabslimits>), Column('Datetime', DATETIME(), table=<qcabslimits>), Column('YYYY', INTEGER(), table=<qcabslimits>), Column('mm', TINYINT(), table=<qcabslimits>), Column('dd', TINYINT(), table=<qcabslimits>), Column('hh', TINYINT(), table=<qcabslimits>), Column('obsValue', VARCHAR(length=10), table=<qcabslimits>), Column('limitValue', VARCHAR(length=10), table=<qcabslimits>), Column('qcStatus', INTEGER(), table=<qcabslimits>), Column('acquisitionType', INTEGER(), table=<qcabslimits>), Column('obsLevel', VARCHAR(length=255), table=<qcabslimits>), Column('capturedBy', VARCHAR(length=255), table=<qcabslimits>), Column('dataForm', VARCHAR(length=255), table=<qcabslimits>), schema=None)),
    (   'remove_table',
        Table('climsoftusers', MetaData(), Column('userName', VARCHAR(length=50), table=<climsoftusers>, primary_key=True, nullable=False), Column('userRole', VARCHAR(length=50), table=<climsoftusers>, nullable=False), schema=None)),
    (   'remove_table',
        Table('seq_month_day_synoptime', MetaData(), Column('mm', INTEGER(), table=<seq_month_day_synoptime>, primary_key=True, nullable=False), Column('dd', INTEGER(), table=<seq_month_day_synoptime>, primary_key=True, nullable=False), Column('hh', INTEGER(), table=<seq_month_day_synoptime>, primary_key=True, nullable=False), schema=None)),
    (   'remove_table',
        Table('seq_month_day', MetaData(), Column('mm', INTEGER(), table=<seq_month_day>, primary_key=True, nullable=False), Column('dd', INTEGER(), table=<seq_month_day>, primary_key=True, nullable=False), schema=None)),
    (   'remove_table',
        Table('qc_interelement_relationship_definition', MetaData(), Column('elementId_1', INTEGER(), table=<qc_interelement_relationship_definition>, primary_key=True, nullable=False), Column('relationship', VARCHAR(length=50), table=<qc_interelement_relationship_definition>, primary_key=True, nullable=False), Column('elementId_2', INTEGER(), table=<qc_interelement_relationship_definition>, primary_key=True, nullable=False), schema=None)),
    (   'remove_table',
        Table('seq_month_day_element', MetaData(), Column('mm', INTEGER(), table=<seq_month_day_element>, nullable=False), Column('dd', INTEGER(), table=<seq_month_day_element>, nullable=False), Column('elementId', INTEGER(), table=<seq_month_day_element>, nullable=False), schema=None)),
    (   'remove_table',
        Table('aws_stations', MetaData(), Column('aws_id', VARCHAR(length=50), table=<aws_stations>, primary_key=True, nullable=False), Column('national_id', VARCHAR(length=50), table=<aws_stations>, primary_key=True, nullable=False), Column('station_name', VARCHAR(length=50), table=<aws_stations>), schema=None)),
    (   'remove_index',
        Index('identification', Column('Cols', INTEGER(), table=<aws_lsi>, nullable=False))),
    (   'remove_table',
        Table('aws_lsi', MetaData(), Column('Cols', INTEGER(), table=<aws_lsi>, nullable=False), Column('Element_abbreviation', VARCHAR(length=20), table=<aws_lsi>), Column('Element_Name', VARCHAR(length=20), table=<aws_lsi>), Column('Element_Details', VARCHAR(length=25), table=<aws_lsi>), Column('Climsoft_Element', VARCHAR(length=6), table=<aws_lsi>), Column('Bufr_Element', VARCHAR(length=6), table=<aws_lsi>), Column('unit', VARCHAR(length=15), table=<aws_lsi>), Column('lower_limit', VARCHAR(length=10), table=<aws_lsi>), Column('upper_limit', VARCHAR(length=10), table=<aws_lsi>), Column('obsv', VARCHAR(length=25), table=<aws_lsi>), schema=None)),
    (   'remove_table',
        Table('tm_307073', MetaData(), Column('order', DOUBLE(asdecimal=True), table=<tm_307073>), Column('Bufr_Template', VARCHAR(length=255), table=<tm_307073>), Column('CREX_Template', VARCHAR(length=255), table=<tm_307073>), Column('Sequence_Descriptor1', VARCHAR(length=255), table=<tm_307073>), Column('Sequence_Descriptor0', VARCHAR(length=255), table=<tm_307073>), Column('Bufr_Element', VARCHAR(length=255), table=<tm_307073>), Column('Crex_Element', VARCHAR(length=255), table=<tm_307073>), Column('Climsoft_Element', VARCHAR(length=255), table=<tm_307073>), Column('Element_Name', VARCHAR(length=255), table=<tm_307073>), Column('Crex_Unit', VARCHAR(length=255), table=<tm_307073>), Column('Crex_Scale', VARCHAR(length=255), table=<tm_307073>), Column('Crex_DataWidth', VARCHAR(length=255), table=<tm_307073>), Column('Bufr_Unit', VARCHAR(length=255), table=<tm_307073>), Column('Bufr_Scale', DOUBLE(asdecimal=True), table=<tm_307073>), Column('Bufr_RefValue', DOUBLE(asdecimal=True), table=<tm_307073>), Column('Bufr_DataWidth_Bits', DOUBLE(asdecimal=True), table=<tm_307073>), Column('Selected', TINYINT(), table=<tm_307073>), Column('Observation', VARCHAR(length=255), table=<tm_307073>), Column('Crex_Data', VARCHAR(length=255), table=<tm_307073>), Column('Bufr_Data', VARCHAR(length=255), table=<tm_307073>), schema=None)),
    (   'remove_table',
        Table('bufr_indicators', MetaData(), Column('Tmplate', VARCHAR(length=50), table=<bufr_indicators>, primary_key=True, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x12069bdc0>, for_update=False)), Column('Msg_Header', VARCHAR(length=50), table=<bufr_indicators>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1206a6160>, for_update=False)), Column('BUFR_Edition', VARCHAR(length=10), table=<bufr_indicators>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1206a6250>, for_update=False)), Column('Originating_Centre', VARCHAR(length=10), table=<bufr_indicators>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1206a6340>, for_update=False)), Column('Originating_SubCentre', VARCHAR(length=10), table=<bufr_indicators>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1206a6430>, for_update=False)), Column('Update_Sequence', VARCHAR(length=10), table=<bufr_indicators>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1206a6520>, for_update=False)), Column('Optional_Section', VARCHAR(length=10), table=<bufr_indicators>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1206a6610>, for_update=False)), Column('Data_Category', VARCHAR(length=10), table=<bufr_indicators>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1206a6700>, for_update=False)), Column('Intenational_Data_SubCategory', VARCHAR(length=10), table=<bufr_indicators>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1206a67f0>, for_update=False)), Column('Local_Data_SubCategory', VARCHAR(length=10), table=<bufr_indicators>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1206a68e0>, for_update=False)), Column('Master_table', VARCHAR(length=10), table=<bufr_indicators>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1206a69d0>, for_update=False)), Column('Local_Table', VARCHAR(length=10), table=<bufr_indicators>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1206a6ac0>, for_update=False)), schema=None)),
    (   'remove_table',
        Table('aws_sasscal1', MetaData(), Column('Cols', INTEGER(), table=<aws_sasscal1>, primary_key=True, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1206ad400>, for_update=False)), Column('Element_Abbreviation', VARCHAR(length=50), table=<aws_sasscal1>), Column('Element_Name', VARCHAR(length=50), table=<aws_sasscal1>), Column('Element_Details', VARCHAR(length=50), table=<aws_sasscal1>), Column('Climsoft_Element', VARCHAR(length=6), table=<aws_sasscal1>), Column('Bufr_Element', VARCHAR(length=6), table=<aws_sasscal1>), Column('unit', VARCHAR(length=15), table=<aws_sasscal1>), Column('lower_limit', VARCHAR(length=50), table=<aws_sasscal1>), Column('upper_limit', VARCHAR(length=50), table=<aws_sasscal1>), Column('obsv', VARCHAR(length=50), table=<aws_sasscal1>), schema=None)),
    (   'remove_table',
        Table('tm_307081', MetaData(), Column('Nos', INTEGER(), table=<tm_307081>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1206b8130>, for_update=False)), Column('Bufr_Template', VARCHAR(length=50), table=<tm_307081>), Column('Crex_Template', VARCHAR(length=50), table=<tm_307081>), Column('Sequence_Descriptor1', VARCHAR(length=50), table=<tm_307081>), Column('Sequence_Descriptor0', VARCHAR(length=50), table=<tm_307081>), Column('Bufr_Element', VARCHAR(length=50), table=<tm_307081>), Column('Crex_Element', VARCHAR(length=255), table=<tm_307081>), Column('Climsoft_Element', VARCHAR(length=255), table=<tm_307081>), Column('Element_Name', VARCHAR(length=255), table=<tm_307081>), Column('synop_code', VARCHAR(length=255), table=<tm_307081>), Column('unit', VARCHAR(length=255), table=<tm_307081>), Column('scale', VARCHAR(length=255), table=<tm_307081>), Column('width', VARCHAR(length=255), table=<tm_307081>), Column('units_length_scale', VARCHAR(length=255), table=<tm_307081>), Column('data_type', VARCHAR(length=255), table=<tm_307081>), Column('selected', TINYINT(), table=<tm_307081>), Column('Observation', VARCHAR(length=255), table=<tm_307081>), Column('Crex_Data', VARCHAR(length=255), table=<tm_307081>), Column('Bufr_Data', VARCHAR(length=50), table=<tm_307081>), schema=None)),
    (   'remove_index',
        Index('identification', Column('No', INTEGER(), table=<aws1>, nullable=False))),
    (   'remove_table',
        Table('aws1', MetaData(), Column('No', INTEGER(), table=<aws1>, nullable=False), Column('Element_abbreviation', TEXT(), table=<aws1>, nullable=False), Column('Element_Name', TEXT(), table=<aws1>, nullable=False), Column('Element_Details', TEXT(), table=<aws1>, nullable=False), Column('Climsoft_Element', TEXT(), table=<aws1>, nullable=False), Column('Bufr_Element', TEXT(), table=<aws1>, nullable=False), Column('unit', TEXT(), table=<aws1>, nullable=False), Column('lower_limit', TEXT(), table=<aws1>, nullable=False), Column('upper_limit', TEXT(), table=<aws1>, nullable=False), Column('obsv', TEXT(), table=<aws1>, nullable=False), schema=None)),
    (   'remove_table',
        Table('tm_307080', MetaData(), Column('order', INTEGER(), table=<tm_307080>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1206c2fd0>, for_update=False)), Column('Bufr_Template', VARCHAR(length=50), table=<tm_307080>), Column('Crex_Template', VARCHAR(length=50), table=<tm_307080>), Column('Sequence_Descriptor1', VARCHAR(length=50), table=<tm_307080>), Column('Sequence_Descriptor0', VARCHAR(length=50), table=<tm_307080>), Column('Bufr_Element', VARCHAR(length=50), table=<tm_307080>), Column('Crex_Element', VARCHAR(length=255), table=<tm_307080>), Column('Climsoft_Element', VARCHAR(length=255), table=<tm_307080>), Column('Element_Name', VARCHAR(length=255), table=<tm_307080>), Column('synop_code', VARCHAR(length=255), table=<tm_307080>), Column('unit', VARCHAR(length=255), table=<tm_307080>), Column('scale', VARCHAR(length=255), table=<tm_307080>), Column('width', VARCHAR(length=255), table=<tm_307080>), Column('units_length_scale', VARCHAR(length=255), table=<tm_307080>), Column('data_type', VARCHAR(length=255), table=<tm_307080>), Column('selected', TINYINT(), table=<tm_307080>), Column('observation', VARCHAR(length=255), table=<tm_307080>), Column('Crex_Data', VARCHAR(length=255), table=<tm_307080>), Column('Bufr_Data', VARCHAR(length=50), table=<tm_307080>), schema=None)),
    (   'remove_table',
        Table('stationnetworkdefinition', MetaData(), Column('networkAbbreviation', VARCHAR(length=255), table=<stationnetworkdefinition>, primary_key=True, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1206d81c0>, for_update=False)), Column('networkFullName', VARCHAR(length=255), table=<stationnetworkdefinition>), schema=None)),
    (   'remove_table',
        Table('seq_year', MetaData(), Column('yyyy', INTEGER(), table=<seq_year>), schema=None)),
    (   'remove_index',
        Index('identification', Column('Cols', INTEGER(), table=<aws_lsi1>, nullable=False))),
    (   'remove_table',
        Table('aws_lsi1', MetaData(), Column('Cols', INTEGER(), table=<aws_lsi1>, nullable=False), Column('Element_abbreviation', VARCHAR(length=20), table=<aws_lsi1>), Column('Element_Name', VARCHAR(length=20), table=<aws_lsi1>), Column('Element_Details', VARCHAR(length=25), table=<aws_lsi1>), Column('Climsoft_Element', VARCHAR(length=6), table=<aws_lsi1>), Column('Bufr_Element', VARCHAR(length=6), table=<aws_lsi1>), Column('unit', VARCHAR(length=15), table=<aws_lsi1>), Column('lower_limit', VARCHAR(length=10), table=<aws_lsi1>), Column('upper_limit', VARCHAR(length=10), table=<aws_lsi1>), Column('obsv', VARCHAR(length=25), table=<aws_lsi1>), schema=None)),
    (   'remove_table',
        Table('seq_month_day_leap_yr', MetaData(), Column('mm', INTEGER(), table=<seq_month_day_leap_yr>, primary_key=True, nullable=False), Column('dd', INTEGER(), table=<seq_month_day_leap_yr>, primary_key=True, nullable=False), schema=None)),
    (   'remove_table',
        Table('tblproducts', MetaData(), Column('productId', VARCHAR(length=10), table=<tblproducts>, primary_key=True, nullable=False), Column('productName', VARCHAR(length=50), table=<tblproducts>), Column('prDetails', VARCHAR(length=50), table=<tblproducts>), Column('prCategory', VARCHAR(length=50), table=<tblproducts>), schema=None)),
    (   'remove_table',
        Table('aws_toa5_mg2', MetaData(), Column('Cols', BIGINT(), table=<aws_toa5_mg2>, primary_key=True, nullable=False), Column('Element_Abbreviation', VARCHAR(length=50), table=<aws_toa5_mg2>), Column('Element_Name', VARCHAR(length=50), table=<aws_toa5_mg2>), Column('Element_Details', VARCHAR(length=50), table=<aws_toa5_mg2>), Column('Climsoft_Element', VARCHAR(length=6), table=<aws_toa5_mg2>), Column('Bufr_Element', VARCHAR(length=6), table=<aws_toa5_mg2>), Column('unit', VARCHAR(length=15), table=<aws_toa5_mg2>), Column('lower_limit', VARCHAR(length=50), table=<aws_toa5_mg2>), Column('upper_limit', VARCHAR(length=50), table=<aws_toa5_mg2>), Column('obsv', VARCHAR(length=50), table=<aws_toa5_mg2>), schema=None)),
    (   'remove_table',
        Table('aws_tahmo', MetaData(), Column('Cols', INTEGER(), table=<aws_tahmo>, primary_key=True, nullable=False), Column('Element_abbreviation', VARCHAR(length=50), table=<aws_tahmo>), Column('Element_Name', VARCHAR(length=50), table=<aws_tahmo>), Column('Element_Details', VARCHAR(length=50), table=<aws_tahmo>), Column('Climsoft_Element', VARCHAR(length=6), table=<aws_tahmo>), Column('Bufr_Element', VARCHAR(length=6), table=<aws_tahmo>), Column('unit', VARCHAR(length=15), table=<aws_tahmo>), Column('lower_limit', VARCHAR(length=50), table=<aws_tahmo>), Column('upper_limit', VARCHAR(length=50), table=<aws_tahmo>), Column('obsv', VARCHAR(length=50), table=<aws_tahmo>), schema=None)),
    (   'remove_table',
        Table('seq_month_day_synoptime_leap_yr', MetaData(), Column('mm', INTEGER(), table=<seq_month_day_synoptime_leap_yr>, primary_key=True, nullable=False), Column('dd', INTEGER(), table=<seq_month_day_synoptime_leap_yr>, primary_key=True, nullable=False), Column('hh', INTEGER(), table=<seq_month_day_synoptime_leap_yr>, primary_key=True, nullable=False), schema=None)),
    (   'remove_table',
        Table('aws_rwanda4', MetaData(), Column('Cols', INTEGER(), table=<aws_rwanda4>, primary_key=True, nullable=False), Column('Element_Name', VARCHAR(length=20), table=<aws_rwanda4>), Column('Element_Abbreviation', VARCHAR(length=20), table=<aws_rwanda4>), Column('Element_Details', VARCHAR(length=25), table=<aws_rwanda4>), Column('Climsoft_Element', VARCHAR(length=6), table=<aws_rwanda4>), Column('Bufr_Element', VARCHAR(length=6), table=<aws_rwanda4>), Column('unit', VARCHAR(length=15), table=<aws_rwanda4>), Column('lower_limit', VARCHAR(length=10), table=<aws_rwanda4>), Column('upper_limit', VARCHAR(length=10), table=<aws_rwanda4>), Column('obsv', VARCHAR(length=25), table=<aws_rwanda4>), schema=None)),
    (   'remove_index',
        Index('identification', Column('Element_Abbreviation', VARCHAR(length=50), table=<aws_toa5_bw1>))),
    (   'remove_table',
        Table('aws_toa5_bw1', MetaData(), Column('Cols', INTEGER(), table=<aws_toa5_bw1>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x12070f610>, for_update=False)), Column('Element_Abbreviation', VARCHAR(length=50), table=<aws_toa5_bw1>), Column('Element_Name', VARCHAR(length=50), table=<aws_toa5_bw1>), Column('Element_Details', VARCHAR(length=50), table=<aws_toa5_bw1>), Column('Climsoft_Element', VARCHAR(length=6), table=<aws_toa5_bw1>), Column('Bufr_Element', VARCHAR(length=6), table=<aws_toa5_bw1>), Column('unit', VARCHAR(length=15), table=<aws_toa5_bw1>), Column('lower_limit', VARCHAR(length=50), table=<aws_toa5_bw1>), Column('upper_limit', VARCHAR(length=50), table=<aws_toa5_bw1>), Column('obsv', VARCHAR(length=50), table=<aws_toa5_bw1>), schema=None)),
    (   'remove_index',
        Index('identification', Column('Cols', INTEGER(), table=<testing_aws>, nullable=False))),
    (   'remove_table',
        Table('testing_aws', MetaData(), Column('Cols', INTEGER(), table=<testing_aws>, nullable=False), Column('Element_abbreviation', VARCHAR(length=20), table=<testing_aws>), Column('Element_Name', VARCHAR(length=20), table=<testing_aws>), Column('Element_Details', VARCHAR(length=25), table=<testing_aws>), Column('Climsoft_Element', VARCHAR(length=6), table=<testing_aws>), Column('Bufr_Element', VARCHAR(length=6), table=<testing_aws>), Column('unit', VARCHAR(length=15), table=<testing_aws>), Column('lower_limit', VARCHAR(length=10), table=<testing_aws>), Column('upper_limit', VARCHAR(length=10), table=<testing_aws>), Column('obsv', VARCHAR(length=25), table=<testing_aws>), schema=None)),
    (   'remove_table',
        Table('seq_month', MetaData(), Column('mm', VARCHAR(length=50), table=<seq_month>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x120722310>, for_update=False)), schema=None)),
    (   'remove_table',
        Table('missing_data', MetaData(), Column('STN_ID', VARCHAR(length=255), table=<missing_data>, primary_key=True, nullable=False), Column('OBS_DATE', DATE(), table=<missing_data>, primary_key=True, nullable=False), Column('ELEM', BIGINT(), table=<missing_data>, primary_key=True, nullable=False), schema=None)),
    (   'remove_index',
        Index('identification', Column('Element_Name', VARCHAR(length=50), table=<aws_rwanda1>))),
    (   'remove_table',
        Table('aws_rwanda1', MetaData(), Column('Cols', INTEGER(), table=<aws_rwanda1>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1207296a0>, for_update=False)), Column('Element_abbreviation', VARCHAR(length=50), table=<aws_rwanda1>), Column('Element_Name', VARCHAR(length=50), table=<aws_rwanda1>), Column('Element_Details', VARCHAR(length=50), table=<aws_rwanda1>), Column('Climsoft_Element', VARCHAR(length=6), table=<aws_rwanda1>), Column('Bufr_Element', VARCHAR(length=6), table=<aws_rwanda1>), Column('unit', VARCHAR(length=15), table=<aws_rwanda1>), Column('lower_limit', VARCHAR(length=50), table=<aws_rwanda1>), Column('upper_limit', VARCHAR(length=50), table=<aws_rwanda1>), Column('obsv', VARCHAR(length=50), table=<aws_rwanda1>), schema=None)),
    (   'remove_index',
        Index('synop_code', Column('Element_Name', VARCHAR(length=255), table=<tm_307089>))),
    (   'remove_index',
        Index('synop_code1', Column('synop_code', VARCHAR(length=50), table=<tm_307089>))),
    (   'remove_table',
        Table('tm_307089', MetaData(), Column('order', INTEGER(), table=<tm_307089>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x120732550>, for_update=False)), Column('Bufr_Template', VARCHAR(length=50), table=<tm_307089>), Column('Crex_Template', VARCHAR(length=50), table=<tm_307089>), Column('Sequence_Descriptor1', VARCHAR(length=10), table=<tm_307089>), Column('Sequence_Descriptor0', VARCHAR(length=10), table=<tm_307089>), Column('Bufr_Element', VARCHAR(length=50), table=<tm_307089>), Column('Crex_Element', VARCHAR(length=255), table=<tm_307089>), Column('Climsoft_Element', VARCHAR(length=50), table=<tm_307089>), Column('Element_Name', VARCHAR(length=255), table=<tm_307089>), Column('synop_code', VARCHAR(length=50), table=<tm_307089>), Column('unit', VARCHAR(length=50), table=<tm_307089>), Column('scale', VARCHAR(length=50), table=<tm_307089>), Column('width', VARCHAR(length=50), table=<tm_307089>), Column('units_length_scale', VARCHAR(length=255), table=<tm_307089>), Column('data_type', VARCHAR(length=255), table=<tm_307089>), Column('selected', TINYINT(), table=<tm_307089>), Column('observation', VARCHAR(length=100), table=<tm_307089>), Column('crex', VARCHAR(length=25), table=<tm_307089>), Column('Crex_Data', VARCHAR(length=255), table=<tm_307089>), Column('Bufr_Data', VARCHAR(length=50), table=<tm_307089>), schema=None)),
    (   'remove_index',
        Index('Identification', Column('Element_Name', VARCHAR(length=50), table=<aws_malawi1>))),
    (   'remove_table',
        Table('aws_malawi1', MetaData(), Column('Cols', INTEGER(), table=<aws_malawi1>, primary_key=True, nullable=False), Column('Element_abbreviation', VARCHAR(length=50), table=<aws_malawi1>), Column('Element_Name', VARCHAR(length=50), table=<aws_malawi1>), Column('Element_Details', VARCHAR(length=50), table=<aws_malawi1>), Column('Climsoft_Element', VARCHAR(length=6), table=<aws_malawi1>), Column('Bufr_Element', VARCHAR(length=6), table=<aws_malawi1>), Column('unit', VARCHAR(length=15), table=<aws_malawi1>), Column('lower_limit', VARCHAR(length=50), table=<aws_malawi1>), Column('upper_limit', VARCHAR(length=50), table=<aws_malawi1>), Column('obsv', VARCHAR(length=50), table=<aws_malawi1>), schema=None)),
    (   'remove_table',
        Table('language_translation', MetaData(), Column('tagID', VARCHAR(length=50), table=<language_translation>, primary_key=True, nullable=False), Column('en', VARCHAR(length=100), table=<language_translation>), Column('fr', VARCHAR(length=100), table=<language_translation>), Column('de', VARCHAR(length=100), table=<language_translation>), Column('pt', VARCHAR(length=100), table=<language_translation>), schema=None)),
    (   'remove_index',
        Index('report', Column('reportClass', VARCHAR(length=255), table=<routinereporttransmission>), Column('reportDatetime', DATETIME(), table=<routinereporttransmission>), Column('reportedFrom', VARCHAR(length=255), table=<routinereporttransmission>))),
    (   'remove_table',
        Table('routinereporttransmission', MetaData(), Column('reportClass', VARCHAR(length=255), ForeignKey('routinereportdefinition.reportClass'), table=<routinereporttransmission>), Column('reportDatetime', DATETIME(), table=<routinereporttransmission>), Column('receivedDatetime', DATETIME(), table=<routinereporttransmission>), Column('reportedFrom', VARCHAR(length=255), ForeignKey('station.stationId'), table=<routinereporttransmission>), schema=None)),
    (   'remove_index',
        Index('identification', Column('Cols', INTEGER(), table=<aws_rwanda_rain>, nullable=False))),
    (   'remove_table',
        Table('aws_rwanda_rain', MetaData(), Column('Cols', INTEGER(), table=<aws_rwanda_rain>, nullable=False), Column('Element_abbreviation', VARCHAR(length=20), table=<aws_rwanda_rain>), Column('Element_Name', VARCHAR(length=20), table=<aws_rwanda_rain>), Column('Element_Details', VARCHAR(length=25), table=<aws_rwanda_rain>), Column('Climsoft_Element', VARCHAR(length=6), table=<aws_rwanda_rain>), Column('Bufr_Element', VARCHAR(length=6), table=<aws_rwanda_rain>), Column('unit', VARCHAR(length=15), table=<aws_rwanda_rain>), Column('lower_limit', VARCHAR(length=10), table=<aws_rwanda_rain>), Column('upper_limit', VARCHAR(length=10), table=<aws_rwanda_rain>), Column('obsv', VARCHAR(length=25), table=<aws_rwanda_rain>), schema=None)),
    (   'remove_index',
        Index('report_code', Column('reportCode', VARCHAR(length=255), table=<routinereportdefinition>))),
    (   'remove_table',
        Table('routinereportdefinition', MetaData(), Column('reportClass', VARCHAR(length=255), table=<routinereportdefinition>, primary_key=True, nullable=False), Column('reportSchedule', VARCHAR(length=255), table=<routinereportdefinition>), Column('reportCode', VARCHAR(length=255), table=<routinereportdefinition>), Column('reportDescription', VARCHAR(length=255), table=<routinereportdefinition>), schema=None)),
    (   'remove_table',
        Table('qc_interelement_2', MetaData(), Column('stationId_2', VARCHAR(length=50), table=<qc_interelement_2>, primary_key=True, nullable=False), Column('elementId_2', INTEGER(), table=<qc_interelement_2>, primary_key=True, nullable=False), Column('obsDatetime_2', DATETIME(), table=<qc_interelement_2>, primary_key=True, nullable=False), Column('obsValue_2', INTEGER(), table=<qc_interelement_2>), Column('qcStatus_2', INTEGER(), table=<qc_interelement_2>), Column('acquisitionType_2', INTEGER(), table=<qc_interelement_2>), Column('obsLevel_2', VARCHAR(length=50), table=<qc_interelement_2>), Column('capturedBy_2', VARCHAR(length=50), table=<qc_interelement_2>), Column('dataForm_2', VARCHAR(length=50), table=<qc_interelement_2>), schema=None)),
    (   'remove_table',
        Table('form_hourlywind', MetaData(), Column('stationId', VARCHAR(length=255), table=<form_hourlywind>, primary_key=True, nullable=False), Column('yyyy', INTEGER(), table=<form_hourlywind>, primary_key=True, nullable=False), Column('mm', INTEGER(), table=<form_hourlywind>, primary_key=True, nullable=False), Column('dd', INTEGER(), table=<form_hourlywind>, primary_key=True, nullable=False), Column('elem_112_00', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_112_01', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_112_02', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_112_03', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_112_04', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_112_05', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_112_06', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_112_07', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_112_08', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_112_09', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_112_10', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_112_11', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_112_12', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_112_13', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_112_14', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_112_15', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_112_16', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_112_17', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_112_18', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_112_19', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_112_20', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_112_21', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_112_22', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_112_23', VARCHAR(length=255), table=<form_hourlywind>), Column('ddflag00', VARCHAR(length=255), table=<form_hourlywind>), Column('ddflag01', VARCHAR(length=255), table=<form_hourlywind>), Column('ddflag02', VARCHAR(length=255), table=<form_hourlywind>), Column('ddflag03', VARCHAR(length=255), table=<form_hourlywind>), Column('ddflag04', VARCHAR(length=255), table=<form_hourlywind>), Column('ddflag05', VARCHAR(length=255), table=<form_hourlywind>), Column('ddflag06', VARCHAR(length=255), table=<form_hourlywind>), Column('ddflag07', VARCHAR(length=255), table=<form_hourlywind>), Column('ddflag08', VARCHAR(length=255), table=<form_hourlywind>), Column('ddflag09', VARCHAR(length=255), table=<form_hourlywind>), Column('ddflag10', VARCHAR(length=255), table=<form_hourlywind>), Column('ddflag11', VARCHAR(length=255), table=<form_hourlywind>), Column('ddflag12', VARCHAR(length=255), table=<form_hourlywind>), Column('ddflag13', VARCHAR(length=255), table=<form_hourlywind>), Column('ddflag14', VARCHAR(length=255), table=<form_hourlywind>), Column('ddflag15', VARCHAR(length=255), table=<form_hourlywind>), Column('ddflag16', VARCHAR(length=255), table=<form_hourlywind>), Column('ddflag17', VARCHAR(length=255), table=<form_hourlywind>), Column('ddflag18', VARCHAR(length=255), table=<form_hourlywind>), Column('ddflag19', VARCHAR(length=255), table=<form_hourlywind>), Column('ddflag20', VARCHAR(length=255), table=<form_hourlywind>), Column('ddflag21', VARCHAR(length=255), table=<form_hourlywind>), Column('ddflag22', VARCHAR(length=255), table=<form_hourlywind>), Column('ddflag23', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_111_00', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_111_01', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_111_02', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_111_03', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_111_04', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_111_05', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_111_06', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_111_07', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_111_08', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_111_09', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_111_10', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_111_11', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_111_12', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_111_13', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_111_14', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_111_15', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_111_16', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_111_17', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_111_18', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_111_19', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_111_20', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_111_21', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_111_22', VARCHAR(length=255), table=<form_hourlywind>), Column('elem_111_23', VARCHAR(length=255), table=<form_hourlywind>), Column('total', VARCHAR(length=50), table=<form_hourlywind>), Column('signature', VARCHAR(length=50), table=<form_hourlywind>), Column('entryDatetime', DATETIME(), table=<form_hourlywind>), schema=None)),
    (   'remove_table',
        Table('tm_307092', MetaData(), Column('order', INTEGER(), table=<tm_307092>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1207a9400>, for_update=False)), Column('Bufr_Template', VARCHAR(length=50), table=<tm_307092>), Column('CREX_Template', VARCHAR(length=50), table=<tm_307092>), Column('Sequence_Descriptor1', VARCHAR(length=255), table=<tm_307092>), Column('Sequence_Descriptor0', VARCHAR(length=255), table=<tm_307092>), Column('Bufr_Element', VARCHAR(length=255), table=<tm_307092>), Column('Crex_Element', VARCHAR(length=50), table=<tm_307092>), Column('Climsoft_Element', VARCHAR(length=50), table=<tm_307092>), Column('Element_Name', VARCHAR(length=255), table=<tm_307092>), Column('Crex_Unit', VARCHAR(length=25), table=<tm_307092>), Column('Crex_Scale', VARCHAR(length=25), table=<tm_307092>), Column('Crex_DataWidth', VARCHAR(length=25), table=<tm_307092>), Column('Bufr_Unit', VARCHAR(length=255), table=<tm_307092>), Column('Bufr_Scale', INTEGER(), table=<tm_307092>), Column('Bufr_RefValue', BIGINT(), table=<tm_307092>), Column('Bufr_DataWidth_Bits', INTEGER(), table=<tm_307092>), Column('Selected', TINYINT(), table=<tm_307092>), Column('Observation', VARCHAR(length=255), table=<tm_307092>), Column('Crex_Data', VARCHAR(length=30), table=<tm_307092>), Column('Bufr_Data', VARCHAR(length=255), table=<tm_307092>), schema=None)),
    (   'remove_table',
        Table('form_hourly', MetaData(), Column('stationId', VARCHAR(length=50), table=<form_hourly>, primary_key=True, nullable=False), Column('elementId', INTEGER(), table=<form_hourly>, primary_key=True, nullable=False), Column('yyyy', INTEGER(), table=<form_hourly>, primary_key=True, nullable=False), Column('mm', INTEGER(), table=<form_hourly>, primary_key=True, nullable=False), Column('dd', INTEGER(), table=<form_hourly>, primary_key=True, nullable=False), Column('hh_00', VARCHAR(length=50), table=<form_hourly>), Column('hh_01', VARCHAR(length=50), table=<form_hourly>), Column('hh_02', VARCHAR(length=50), table=<form_hourly>), Column('hh_03', VARCHAR(length=50), table=<form_hourly>), Column('hh_04', VARCHAR(length=50), table=<form_hourly>), Column('hh_05', VARCHAR(length=50), table=<form_hourly>), Column('hh_06', VARCHAR(length=50), table=<form_hourly>), Column('hh_07', VARCHAR(length=50), table=<form_hourly>), Column('hh_08', VARCHAR(length=50), table=<form_hourly>), Column('hh_09', VARCHAR(length=50), table=<form_hourly>), Column('hh_10', VARCHAR(length=50), table=<form_hourly>), Column('hh_11', VARCHAR(length=50), table=<form_hourly>), Column('hh_12', VARCHAR(length=50), table=<form_hourly>), Column('hh_13', VARCHAR(length=50), table=<form_hourly>), Column('hh_14', VARCHAR(length=50), table=<form_hourly>), Column('hh_15', VARCHAR(length=50), table=<form_hourly>), Column('hh_16', VARCHAR(length=50), table=<form_hourly>), Column('hh_17', VARCHAR(length=50), table=<form_hourly>), Column('hh_18', VARCHAR(length=50), table=<form_hourly>), Column('hh_19', VARCHAR(length=50), table=<form_hourly>), Column('hh_20', VARCHAR(length=50), table=<form_hourly>), Column('hh_21', VARCHAR(length=50), table=<form_hourly>), Column('hh_22', VARCHAR(length=50), table=<form_hourly>), Column('hh_23', VARCHAR(length=50), table=<form_hourly>), Column('flag00', VARCHAR(length=50), table=<form_hourly>), Column('flag01', VARCHAR(length=50), table=<form_hourly>), Column('flag02', VARCHAR(length=50), table=<form_hourly>), Column('flag03', VARCHAR(length=50), table=<form_hourly>), Column('flag04', VARCHAR(length=50), table=<form_hourly>), Column('flag05', VARCHAR(length=50), table=<form_hourly>), Column('flag06', VARCHAR(length=50), table=<form_hourly>), Column('flag07', VARCHAR(length=50), table=<form_hourly>), Column('flag08', VARCHAR(length=50), table=<form_hourly>), Column('flag09', VARCHAR(length=50), table=<form_hourly>), Column('flag10', VARCHAR(length=50), table=<form_hourly>), Column('flag11', VARCHAR(length=50), table=<form_hourly>), Column('flag12', VARCHAR(length=50), table=<form_hourly>), Column('flag13', VARCHAR(length=50), table=<form_hourly>), Column('flag14', VARCHAR(length=50), table=<form_hourly>), Column('flag15', VARCHAR(length=50), table=<form_hourly>), Column('flag16', VARCHAR(length=50), table=<form_hourly>), Column('flag17', VARCHAR(length=50), table=<form_hourly>), Column('flag18', VARCHAR(length=50), table=<form_hourly>), Column('flag19', VARCHAR(length=50), table=<form_hourly>), Column('flag20', VARCHAR(length=50), table=<form_hourly>), Column('flag21', VARCHAR(length=50), table=<form_hourly>), Column('flag22', VARCHAR(length=50), table=<form_hourly>), Column('flag23', VARCHAR(length=50), table=<form_hourly>), Column('total', VARCHAR(length=50), table=<form_hourly>), Column('signature', VARCHAR(length=50), table=<form_hourly>), Column('entryDatetime', DATETIME(), table=<form_hourly>), schema=None)),
    (   'remove_table',
        Table('tm_307091', MetaData(), Column('Rec', INTEGER(), table=<tm_307091>, primary_key=True, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1207d0460>, for_update=False)), Column('Bufr_Template', VARCHAR(length=50), table=<tm_307091>), Column('CREX_Template', VARCHAR(length=50), table=<tm_307091>), Column('Sequence_Descriptor1', VARCHAR(length=255), table=<tm_307091>), Column('Sequence_Descriptor0', VARCHAR(length=255), table=<tm_307091>), Column('Bufr_Element', VARCHAR(length=255), table=<tm_307091>), Column('Crex_Element', VARCHAR(length=50), table=<tm_307091>), Column('Climsoft_Element', VARCHAR(length=50), table=<tm_307091>), Column('Element_Name', VARCHAR(length=255), table=<tm_307091>), Column('Crex_Unit', VARCHAR(length=25), table=<tm_307091>), Column('Crex_Scale', VARCHAR(length=25), table=<tm_307091>), Column('Crex_DataWidth', VARCHAR(length=25), table=<tm_307091>), Column('Bufr_Unit', VARCHAR(length=255), table=<tm_307091>), Column('Bufr_Scale', INTEGER(), table=<tm_307091>), Column('Bufr_RefValue', BIGINT(), table=<tm_307091>), Column('Bufr_DataWidth_Bits', INTEGER(), table=<tm_307091>), Column('Selected', TINYINT(), table=<tm_307091>), Column('Observation', VARCHAR(length=255), table=<tm_307091>), Column('Crex_Data', VARCHAR(length=30), table=<tm_307091>), Column('Bufr_Data', VARCHAR(length=255), table=<tm_307091>), schema=None)),
    (   'remove_table',
        Table('form_hourly_time_selection', MetaData(), Column('hh', INTEGER(), table=<form_hourly_time_selection>, primary_key=True, nullable=False), Column('hh_selection', TINYINT(), table=<form_hourly_time_selection>), schema=None)),
    (   'remove_table',
        Table('seq_leap_year', MetaData(), Column('yyyy', INTEGER(), table=<seq_leap_year>), schema=None)),
    (   'remove_index',
        Index('stationid_alias_identification', Column('idAlias', VARCHAR(length=255), table=<stationidalias>))),
    (   'remove_index',
        Index('stationstationidAlias', Column('belongsTo', VARCHAR(length=255), table=<stationidalias>))),
    (   'remove_table',
        Table('stationidalias', MetaData(), Column('idAlias', VARCHAR(length=255), table=<stationidalias>), Column('refersTo', VARCHAR(length=255), table=<stationidalias>), Column('belongsTo', VARCHAR(length=255), ForeignKey('station.stationId'), table=<stationidalias>), Column('idAliasBeginDate', VARCHAR(length=50), table=<stationidalias>), Column('idAliasEndDate', VARCHAR(length=50), table=<stationidalias>), schema=None)),
    (   'remove_table',
        Table('seq_month_day_element_leap_yr', MetaData(), Column('mm', INTEGER(), table=<seq_month_day_element_leap_yr>, nullable=False), Column('dd', INTEGER(), table=<seq_month_day_element_leap_yr>, nullable=False), Column('elementId', INTEGER(), table=<seq_month_day_element_leap_yr>, nullable=False), schema=None)),
    (   'remove_table',
        Table('aws_mss', MetaData(), Column('ftpId', VARCHAR(length=50), table=<aws_mss>, primary_key=True, nullable=False), Column('inputFolder', VARCHAR(length=20), table=<aws_mss>, nullable=False), Column('userName', VARCHAR(length=15), table=<aws_mss>, nullable=False), Column('ftpMode', VARCHAR(length=10), table=<aws_mss>), Column('password', VARCHAR(length=15), table=<aws_mss>), schema=None)),
    (   'remove_table',
        Table('seq_hour', MetaData(), Column('hh', INTEGER(), table=<seq_hour>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1207f0be0>, for_update=False)), schema=None)),
    (   'remove_table',
        Table('form_daily2', MetaData(), Column('stationId', VARCHAR(length=50), table=<form_daily2>, primary_key=True, nullable=False), Column('elementId', INTEGER(), table=<form_daily2>, primary_key=True, nullable=False), Column('yyyy', INTEGER(), table=<form_daily2>, primary_key=True, nullable=False), Column('mm', INTEGER(), table=<form_daily2>, primary_key=True, nullable=False), Column('hh', INTEGER(), table=<form_daily2>, primary_key=True, nullable=False), Column('day01', VARCHAR(length=45), table=<form_daily2>), Column('day02', VARCHAR(length=45), table=<form_daily2>), Column('day03', VARCHAR(length=45), table=<form_daily2>), Column('day04', VARCHAR(length=45), table=<form_daily2>), Column('day05', VARCHAR(length=45), table=<form_daily2>), Column('day06', VARCHAR(length=45), table=<form_daily2>), Column('day07', VARCHAR(length=45), table=<form_daily2>), Column('day08', VARCHAR(length=45), table=<form_daily2>), Column('day09', VARCHAR(length=45), table=<form_daily2>), Column('day10', VARCHAR(length=45), table=<form_daily2>), Column('day11', VARCHAR(length=45), table=<form_daily2>), Column('day12', VARCHAR(length=45), table=<form_daily2>), Column('day13', VARCHAR(length=45), table=<form_daily2>), Column('day14', VARCHAR(length=45), table=<form_daily2>), Column('day15', VARCHAR(length=45), table=<form_daily2>), Column('day16', VARCHAR(length=45), table=<form_daily2>), Column('day17', VARCHAR(length=45), table=<form_daily2>), Column('day18', VARCHAR(length=45), table=<form_daily2>), Column('day19', VARCHAR(length=45), table=<form_daily2>), Column('day20', VARCHAR(length=45), table=<form_daily2>), Column('day21', VARCHAR(length=45), table=<form_daily2>), Column('day22', VARCHAR(length=45), table=<form_daily2>), Column('day23', VARCHAR(length=45), table=<form_daily2>), Column('day24', VARCHAR(length=45), table=<form_daily2>), Column('day25', VARCHAR(length=45), table=<form_daily2>), Column('day26', VARCHAR(length=45), table=<form_daily2>), Column('day27', VARCHAR(length=45), table=<form_daily2>), Column('day28', VARCHAR(length=45), table=<form_daily2>), Column('day29', VARCHAR(length=45), table=<form_daily2>), Column('day30', VARCHAR(length=45), table=<form_daily2>), Column('day31', VARCHAR(length=45), table=<form_daily2>), Column('flag01', VARCHAR(length=1), table=<form_daily2>), Column('flag02', VARCHAR(length=1), table=<form_daily2>), Column('flag03', VARCHAR(length=1), table=<form_daily2>), Column('flag04', VARCHAR(length=1), table=<form_daily2>), Column('flag05', VARCHAR(length=1), table=<form_daily2>), Column('flag06', VARCHAR(length=1), table=<form_daily2>), Column('flag07', VARCHAR(length=1), table=<form_daily2>), Column('flag08', VARCHAR(length=1), table=<form_daily2>), Column('flag09', VARCHAR(length=1), table=<form_daily2>), Column('flag10', VARCHAR(length=1), table=<form_daily2>), Column('flag11', VARCHAR(length=1), table=<form_daily2>), Column('flag12', VARCHAR(length=1), table=<form_daily2>), Column('flag13', VARCHAR(length=1), table=<form_daily2>), Column('flag14', VARCHAR(length=1), table=<form_daily2>), Column('flag15', VARCHAR(length=1), table=<form_daily2>), Column('flag16', VARCHAR(length=1), table=<form_daily2>), Column('flag17', VARCHAR(length=1), table=<form_daily2>), Column('flag18', VARCHAR(length=1), table=<form_daily2>), Column('flag19', VARCHAR(length=1), table=<form_daily2>), Column('flag20', VARCHAR(length=1), table=<form_daily2>), Column('flag21', VARCHAR(length=1), table=<form_daily2>), Column('flag22', VARCHAR(length=1), table=<form_daily2>), Column('flag23', VARCHAR(length=1), table=<form_daily2>), Column('flag24', VARCHAR(length=1), table=<form_daily2>), Column('flag25', VARCHAR(length=1), table=<form_daily2>), Column('flag26', VARCHAR(length=1), table=<form_daily2>), Column('flag27', VARCHAR(length=1), table=<form_daily2>), Column('flag28', VARCHAR(length=1), table=<form_daily2>), Column('flag29', VARCHAR(length=1), table=<form_daily2>), Column('flag30', VARCHAR(length=1), table=<form_daily2>), Column('flag31', VARCHAR(length=1), table=<form_daily2>), Column('period01', VARCHAR(length=45), table=<form_daily2>), Column('period02', VARCHAR(length=45), table=<form_daily2>), Column('period03', VARCHAR(length=45), table=<form_daily2>), Column('period04', VARCHAR(length=45), table=<form_daily2>), Column('period05', VARCHAR(length=45), table=<form_daily2>), Column('period06', VARCHAR(length=45), table=<form_daily2>), Column('period07', VARCHAR(length=45), table=<form_daily2>), Column('period08', VARCHAR(length=45), table=<form_daily2>), Column('period09', VARCHAR(length=45), table=<form_daily2>), Column('period10', VARCHAR(length=45), table=<form_daily2>), Column('period11', VARCHAR(length=45), table=<form_daily2>), Column('period12', VARCHAR(length=45), table=<form_daily2>), Column('period13', VARCHAR(length=45), table=<form_daily2>), Column('period14', VARCHAR(length=45), table=<form_daily2>), Column('period15', VARCHAR(length=45), table=<form_daily2>), Column('period16', VARCHAR(length=45), table=<form_daily2>), Column('period17', VARCHAR(length=45), table=<form_daily2>), Column('period18', VARCHAR(length=45), table=<form_daily2>), Column('period19', VARCHAR(length=45), table=<form_daily2>), Column('period20', VARCHAR(length=45), table=<form_daily2>), Column('period21', VARCHAR(length=45), table=<form_daily2>), Column('period22', VARCHAR(length=45), table=<form_daily2>), Column('period23', VARCHAR(length=45), table=<form_daily2>), Column('period24', VARCHAR(length=45), table=<form_daily2>), Column('period25', VARCHAR(length=45), table=<form_daily2>), Column('period26', VARCHAR(length=45), table=<form_daily2>), Column('period27', VARCHAR(length=45), table=<form_daily2>), Column('period28', VARCHAR(length=45), table=<form_daily2>), Column('period29', VARCHAR(length=45), table=<form_daily2>), Column('period30', VARCHAR(length=45), table=<form_daily2>), Column('period31', VARCHAR(length=45), table=<form_daily2>), Column('total', VARCHAR(length=45), table=<form_daily2>), Column('signature', VARCHAR(length=45), table=<form_daily2>), Column('entryDatetime', DATETIME(), table=<form_daily2>), Column('temperatureUnits', VARCHAR(length=45), table=<form_daily2>), Column('precipUnits', VARCHAR(length=45), table=<form_daily2>), Column('cloudHeightUnits', VARCHAR(length=45), table=<form_daily2>), Column('visUnits', VARCHAR(length=45), table=<form_daily2>), schema=None)),
    (   'remove_index',
        Index('identification', Column('No', INTEGER(), table=<aws_malawi12>, nullable=False))),
    (   'remove_table',
        Table('aws_malawi12', MetaData(), Column('No', INTEGER(), table=<aws_malawi12>, nullable=False), Column('Element_abbreviation', TEXT(), table=<aws_malawi12>, nullable=False), Column('Element_Name', TEXT(), table=<aws_malawi12>, nullable=False), Column('Element_Details', TEXT(), table=<aws_malawi12>, nullable=False), Column('Climsoft_Element', TEXT(), table=<aws_malawi12>, nullable=False), Column('Bufr_Element', TEXT(), table=<aws_malawi12>, nullable=False), Column('unit', TEXT(), table=<aws_malawi12>, nullable=False), Column('lower_limit', TEXT(), table=<aws_malawi12>, nullable=False), Column('upper_limit', TEXT(), table=<aws_malawi12>, nullable=False), Column('obsv', TEXT(), table=<aws_malawi12>, nullable=False), schema=None)),
    (   'remove_index',
        Index('ftpId_UNIQUE', Column('ftpId', VARCHAR(length=50), table=<aws_basestation>, primary_key=True, nullable=False))),
    (   'remove_table',
        Table('aws_basestation', MetaData(), Column('ftpId', VARCHAR(length=50), table=<aws_basestation>, primary_key=True, nullable=False), Column('inputFolder', VARCHAR(length=20), table=<aws_basestation>, nullable=False), Column('ftpMode', VARCHAR(length=10), table=<aws_basestation>), Column('userName', VARCHAR(length=15), table=<aws_basestation>, nullable=False), Column('password', VARCHAR(length=15), table=<aws_basestation>, nullable=False), schema=None)),
    (   'remove_table',
        Table('aws_elements', MetaData(), Column('aws_element', VARCHAR(length=50), table=<aws_elements>, primary_key=True, nullable=False), Column('climsoft_element', VARCHAR(length=50), table=<aws_elements>, primary_key=True, nullable=False), Column('element_description', VARCHAR(length=50), table=<aws_elements>, nullable=False), schema=None)),
    (   'remove_index',
        Index('identification', Column('Cols', INTEGER(), table=<aws_test>, nullable=False))),
    (   'remove_table',
        Table('aws_test', MetaData(), Column('Cols', INTEGER(), table=<aws_test>, nullable=False), Column('Element_abbreviation', VARCHAR(length=20), table=<aws_test>), Column('Element_Name', VARCHAR(length=20), table=<aws_test>), Column('Element_Details', VARCHAR(length=25), table=<aws_test>), Column('Climsoft_Element', VARCHAR(length=6), table=<aws_test>), Column('Bufr_Element', VARCHAR(length=6), table=<aws_test>), Column('unit', VARCHAR(length=15), table=<aws_test>), Column('lower_limit', VARCHAR(length=10), table=<aws_test>), Column('upper_limit', VARCHAR(length=10), table=<aws_test>), Column('obsv', VARCHAR(length=25), table=<aws_test>), schema=None)),
    (   'remove_table',
        Table('form_monthly', MetaData(), Column('stationId', VARCHAR(length=255), table=<form_monthly>, primary_key=True, nullable=False), Column('elementId', INTEGER(), table=<form_monthly>, primary_key=True, nullable=False), Column('yyyy', INTEGER(), table=<form_monthly>, primary_key=True, nullable=False), Column('mm_01', VARCHAR(length=255), table=<form_monthly>), Column('mm_02', VARCHAR(length=255), table=<form_monthly>), Column('mm_03', VARCHAR(length=255), table=<form_monthly>), Column('mm_04', VARCHAR(length=255), table=<form_monthly>, nullable=False), Column('mm_05', VARCHAR(length=255), table=<form_monthly>), Column('mm_06', VARCHAR(length=255), table=<form_monthly>), Column('mm_07', VARCHAR(length=255), table=<form_monthly>), Column('mm_08', VARCHAR(length=255), table=<form_monthly>), Column('mm_09', VARCHAR(length=255), table=<form_monthly>), Column('mm_10', VARCHAR(length=255), table=<form_monthly>), Column('mm_11', VARCHAR(length=255), table=<form_monthly>), Column('mm_12', VARCHAR(length=255), table=<form_monthly>), Column('flag01', VARCHAR(length=255), table=<form_monthly>), Column('flag02', VARCHAR(length=255), table=<form_monthly>), Column('flag03', VARCHAR(length=255), table=<form_monthly>), Column('flag04', VARCHAR(length=255), table=<form_monthly>), Column('flag05', VARCHAR(length=255), table=<form_monthly>), Column('flag06', VARCHAR(length=255), table=<form_monthly>), Column('flag07', VARCHAR(length=255), table=<form_monthly>), Column('flag08', VARCHAR(length=255), table=<form_monthly>), Column('flag09', VARCHAR(length=255), table=<form_monthly>), Column('flag10', VARCHAR(length=255), table=<form_monthly>), Column('flag11', VARCHAR(length=255), table=<form_monthly>), Column('flag12', VARCHAR(length=255), table=<form_monthly>), Column('period01', VARCHAR(length=255), table=<form_monthly>), Column('period02', VARCHAR(length=255), table=<form_monthly>), Column('period03', VARCHAR(length=255), table=<form_monthly>), Column('period04', VARCHAR(length=255), table=<form_monthly>), Column('period05', VARCHAR(length=255), table=<form_monthly>), Column('period06', VARCHAR(length=255), table=<form_monthly>), Column('period07', VARCHAR(length=255), table=<form_monthly>), Column('period08', VARCHAR(length=255), table=<form_monthly>), Column('period09', VARCHAR(length=255), table=<form_monthly>), Column('period10', VARCHAR(length=255), table=<form_monthly>), Column('period11', VARCHAR(length=255), table=<form_monthly>), Column('period12', VARCHAR(length=255), table=<form_monthly>), Column('signature', VARCHAR(length=50), table=<form_monthly>), Column('entryDatetime', DATETIME(), table=<form_monthly>), schema=None)),
    (   'remove_table',
        Table('ccitt', MetaData(), Column('Characters', VARCHAR(length=25), table=<ccitt>), Column('MostSignificant', INTEGER(), table=<ccitt>), Column('LeastSignificant', INTEGER(), table=<ccitt>), schema=None)),
    (   'remove_index',
        Index('Identification', Column('stationId', VARCHAR(length=10), table=<form_synoptic2_tdcf>, primary_key=True, nullable=False), Column('yyyy', BIGINT(), table=<form_synoptic2_tdcf>, primary_key=True, nullable=False), Column('mm', BIGINT(), table=<form_synoptic2_tdcf>, primary_key=True, nullable=False), Column('dd', BIGINT(), table=<form_synoptic2_tdcf>, primary_key=True, nullable=False), Column('hh', VARCHAR(length=5), table=<form_synoptic2_tdcf>, primary_key=True, nullable=False))),
    (   'remove_table',
        Table('form_synoptic2_tdcf', MetaData(), Column('stationId', VARCHAR(length=10), table=<form_synoptic2_tdcf>, primary_key=True, nullable=False), Column('yyyy', BIGINT(), table=<form_synoptic2_tdcf>, primary_key=True, nullable=False), Column('mm', BIGINT(), table=<form_synoptic2_tdcf>, primary_key=True, nullable=False), Column('dd', BIGINT(), table=<form_synoptic2_tdcf>, primary_key=True, nullable=False), Column('hh', VARCHAR(length=5), table=<form_synoptic2_tdcf>, primary_key=True, nullable=False), Column('106', VARCHAR(length=6), table=<form_synoptic2_tdcf>), Column('107', VARCHAR(length=6), table=<form_synoptic2_tdcf>), Column('399', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('301', VARCHAR(length=8), table=<form_synoptic2_tdcf>), Column('185', VARCHAR(length=6), table=<form_synoptic2_tdcf>), Column('101', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('103', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('105', VARCHAR(length=50), table=<form_synoptic2_tdcf>), Column('110', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('114', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('115', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('168', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('192', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('169', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('170', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('171', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('119', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('116', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('117', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('118', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('123', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('120', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('121', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('122', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('127', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('124', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('125', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('126', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('131', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('128', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('129', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('130', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('167', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('197', VARCHAR(length=50), table=<form_synoptic2_tdcf>), Column('193', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('18', VARCHAR(length=6), table=<form_synoptic2_tdcf>), Column('532', VARCHAR(length=6), table=<form_synoptic2_tdcf>), Column('132', VARCHAR(length=6), table=<form_synoptic2_tdcf>), Column('5', VARCHAR(length=6), table=<form_synoptic2_tdcf>), Column('174', VARCHAR(length=50), table=<form_synoptic2_tdcf>), Column('3', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('2', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('112', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('111', VARCHAR(length=5), table=<form_synoptic2_tdcf>), Column('85', VARCHAR(length=50), table=<form_synoptic2_tdcf>), Column('flag1', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag2', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag3', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag4', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag5', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag6', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag7', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag8', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag9', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag10', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag11', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag12', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag13', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag14', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag15', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag16', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag17', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag18', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag19', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag20', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag21', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag22', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag23', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag24', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag25', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag26', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag27', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag28', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag29', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag30', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag31', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag32', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag33', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag34', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag35', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag36', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag37', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag38', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag39', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag40', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag41', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag42', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag43', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag44', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('flag45', VARCHAR(length=1), table=<form_synoptic2_tdcf>), Column('signature', VARCHAR(length=50), table=<form_synoptic2_tdcf>), Column('entryDatetime', DATETIME(), table=<form_synoptic2_tdcf>), schema=None)),
    (   'remove_table',
        Table('seq_daily_element', MetaData(), Column('seq', BIGINT(), table=<seq_daily_element>, primary_key=True, nullable=False), Column('elementId', BIGINT(), table=<seq_daily_element>, nullable=False), schema=None)),
    (   'remove_table',
        Table('seq_day', MetaData(), Column('dd', VARCHAR(length=50), table=<seq_day>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1208995e0>, for_update=False)), schema=None)),
    (   'remove_table',
        Table('userrecords', MetaData(), Column('username', VARCHAR(length=255), table=<userrecords>, primary_key=True, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x120899fd0>, for_update=False)), Column('recsexpt', INTEGER(), table=<userrecords>), Column('recsdone', VARCHAR(length=255), table=<userrecords>), schema=None)),
    (   'remove_table',
        Table('seq_monthly_element', MetaData(), Column('seq', BIGINT(), table=<seq_monthly_element>, primary_key=True, nullable=False), Column('elementId', BIGINT(), table=<seq_monthly_element>, nullable=False), schema=None)),
    (   'remove_table',
        Table('qc_interelement_1', MetaData(), Column('stationId_1', VARCHAR(length=50), table=<qc_interelement_1>, primary_key=True, nullable=False), Column('elementId_1', INTEGER(), table=<qc_interelement_1>, primary_key=True, nullable=False), Column('obsDatetime_1', DATETIME(), table=<qc_interelement_1>, primary_key=True, nullable=False), Column('obsValue_1', INTEGER(), table=<qc_interelement_1>), Column('qcStatus_1', INTEGER(), table=<qc_interelement_1>), Column('acquisitionType_1', INTEGER(), table=<qc_interelement_1>), Column('obsLevel_1', VARCHAR(length=50), table=<qc_interelement_1>), Column('capturedBy_1', VARCHAR(length=50), table=<qc_interelement_1>), Column('dataForm_1', VARCHAR(length=50), table=<qc_interelement_1>), schema=None)),
    (   'remove_index',
        Index('identification', Column('Cols', INTEGER(), table=<ss>, nullable=False))),
    (   'remove_table',
        Table('ss', MetaData(), Column('Cols', INTEGER(), table=<ss>, nullable=False), Column('Element_abbreviation', VARCHAR(length=20), table=<ss>), Column('Element_Name', VARCHAR(length=20), table=<ss>), Column('Element_Details', VARCHAR(length=25), table=<ss>), Column('Climsoft_Element', VARCHAR(length=6), table=<ss>), Column('Bufr_Element', VARCHAR(length=6), table=<ss>), Column('unit', VARCHAR(length=15), table=<ss>), Column('lower_limit', VARCHAR(length=10), table=<ss>), Column('upper_limit', VARCHAR(length=10), table=<ss>), Column('obsv', VARCHAR(length=25), table=<ss>), schema=None)),
    (   'remove_index',
        Index('identification', Column('Cols', INTEGER(), table=<testing_aws1>, nullable=False))),
    (   'remove_table',
        Table('testing_aws1', MetaData(), Column('Cols', INTEGER(), table=<testing_aws1>, nullable=False), Column('Element_abbreviation', VARCHAR(length=20), table=<testing_aws1>), Column('Element_Name', VARCHAR(length=20), table=<testing_aws1>), Column('Element_Details', VARCHAR(length=25), table=<testing_aws1>), Column('Climsoft_Element', VARCHAR(length=6), table=<testing_aws1>), Column('Bufr_Element', VARCHAR(length=6), table=<testing_aws1>), Column('unit', VARCHAR(length=15), table=<testing_aws1>), Column('lower_limit', VARCHAR(length=10), table=<testing_aws1>), Column('upper_limit', VARCHAR(length=10), table=<testing_aws1>), Column('obsv', VARCHAR(length=25), table=<testing_aws1>), schema=None)),
    (   'remove_table',
        Table('aws_process_parameters', MetaData(), Column('RetrieveInterval', INTEGER(), table=<aws_process_parameters>, primary_key=True, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1208b9c40>, for_update=False)), Column('HourOffset', INTEGER(), table=<aws_process_parameters>, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1208b9fa0>, for_update=False)), Column('RetrievePeriod', INTEGER(), table=<aws_process_parameters>, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1208c20d0>, for_update=False)), Column('RetrieveTimeout', INTEGER(), table=<aws_process_parameters>, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1208c21c0>, for_update=False)), Column('DelinputFile', TINYINT(), table=<aws_process_parameters>, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1208c22b0>, for_update=False)), Column('UTCDiff', TINYINT(display_width=2, unsigned=True, zerofill=True), table=<aws_process_parameters>, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1208c23a0>, for_update=False)), schema=None)),
    (   'remove_index',
        Index('identification', Column('Cols', INTEGER(), table=<aws_rema1>, nullable=False))),
    (   'remove_table',
        Table('aws_rema1', MetaData(), Column('Cols', INTEGER(), table=<aws_rema1>, nullable=False), Column('Element_abbreviation', VARCHAR(length=20), table=<aws_rema1>), Column('Element_Name', VARCHAR(length=20), table=<aws_rema1>), Column('Element_Details', VARCHAR(length=25), table=<aws_rema1>), Column('Climsoft_Element', VARCHAR(length=6), table=<aws_rema1>), Column('Bufr_Element', VARCHAR(length=6), table=<aws_rema1>), Column('unit', VARCHAR(length=15), table=<aws_rema1>), Column('lower_limit', VARCHAR(length=10), table=<aws_rema1>), Column('upper_limit', VARCHAR(length=10), table=<aws_rema1>), Column('obsv', VARCHAR(length=25), table=<aws_rema1>), schema=None)),
    (   'remove_table',
        Table('gaps', MetaData(), Column('Missing_STNID', VARCHAR(length=255), table=<gaps>, primary_key=True, nullable=False), Column('Missing_ELEM', BIGINT(), table=<gaps>, primary_key=True, nullable=False), Column('Missing_Date', DATE(), table=<gaps>, primary_key=True, nullable=False), schema=None)),
    (   'remove_table',
        Table('flagtable', MetaData(), Column('Bufr_Descriptor', VARCHAR(length=6), table=<flagtable>, primary_key=True, nullable=False), Column('Crex_Descriptor', VARCHAR(length=6), table=<flagtable>), Column('Details', VARCHAR(length=255), table=<flagtable>), Column('Widths', INTEGER(), table=<flagtable>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1208d2730>, for_update=False)), Column('Missing', INTEGER(), table=<flagtable>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1208d2820>, for_update=False)), schema=None)),
    (   'remove_table',
        Table('tm_307083', MetaData(), Column('order', INTEGER(), table=<tm_307083>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1208da190>, for_update=False)), Column('Bufr_Template', VARCHAR(length=50), table=<tm_307083>), Column('Crex_Template', VARCHAR(length=50), table=<tm_307083>), Column('Sequence_Descriptor1', VARCHAR(length=50), table=<tm_307083>), Column('Sequence_Descriptor0', VARCHAR(length=50), table=<tm_307083>), Column('Bufr_Element', VARCHAR(length=50), table=<tm_307083>), Column('Crex_Element', VARCHAR(length=255), table=<tm_307083>), Column('Climsoft_Element', VARCHAR(length=255), table=<tm_307083>), Column('Element_Name', VARCHAR(length=255), table=<tm_307083>), Column('synop_code', VARCHAR(length=255), table=<tm_307083>), Column('unit', VARCHAR(length=255), table=<tm_307083>), Column('scale', VARCHAR(length=255), table=<tm_307083>), Column('width', VARCHAR(length=255), table=<tm_307083>), Column('units_length_scale', VARCHAR(length=255), table=<tm_307083>), Column('data_type', VARCHAR(length=255), table=<tm_307083>), Column('selected', TINYINT(), table=<tm_307083>), Column('observation', VARCHAR(length=255), table=<tm_307083>), Column('Crex_Data', VARCHAR(length=255), table=<tm_307083>), Column('Bufr_Data', VARCHAR(length=50), table=<tm_307083>), schema=None)),
    (   'remove_index',
        Index('identification', Column('No', INTEGER(), table=<abc>, nullable=False))),
    (   'remove_table',
        Table('abc', MetaData(), Column('No', INTEGER(), table=<abc>, nullable=False), Column('Element_abbreviation', TEXT(), table=<abc>, nullable=False), Column('Element_Name', TEXT(), table=<abc>, nullable=False), Column('Element_Details', TEXT(), table=<abc>, nullable=False), Column('Climsoft_Element', TEXT(), table=<abc>, nullable=False), Column('Bufr_Element', TEXT(), table=<abc>, nullable=False), Column('unit', TEXT(), table=<abc>, nullable=False), Column('lower_limit', TEXT(), table=<abc>, nullable=False), Column('upper_limit', TEXT(), table=<abc>, nullable=False), Column('obsv', TEXT(), table=<abc>, nullable=False), schema=None)),
    (   'remove_table',
        Table('code_flag', MetaData(), Column('FXY', VARCHAR(length=255), table=<code_flag>, primary_key=True, nullable=False), Column('Fxyyy', VARCHAR(length=50), table=<code_flag>), Column('Description', VARCHAR(length=255), table=<code_flag>), Column('Bufr_DataWidth_Bits', INTEGER(), table=<code_flag>), Column('Crex_DataWidth', VARCHAR(length=25), table=<code_flag>), Column('Bufr_Unit', VARCHAR(length=255), table=<code_flag>), Column('Bufr_Value', VARCHAR(length=50), table=<code_flag>), Column('Crex_Value', VARCHAR(length=10), table=<code_flag>), schema=None)),
    (   'remove_table',
        Table('tdcf_indicators', MetaData(), Column('CREX_Edition', INTEGER(), table=<tdcf_indicators>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1208efc70>, for_update=False)), Column('CREX_Table', INTEGER(), table=<tdcf_indicators>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1208f8070>, for_update=False)), Column('BUFR_Table', INTEGER(), table=<tdcf_indicators>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1208f8160>, for_update=False)), Column('Local_Table', INTEGER(), table=<tdcf_indicators>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1208f8250>, for_update=False)), Column('Data_Category', INTEGER(), table=<tdcf_indicators>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1208f8340>, for_update=False)), Column('Data_SubCategory', INTEGER(), table=<tdcf_indicators>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1208f8430>, for_update=False)), Column('Originating_Centre', INTEGER(), table=<tdcf_indicators>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1208f8520>, for_update=False)), schema=None)),
    (   'remove_table',
        Table('form_agro1', MetaData(), Column('stationId', VARCHAR(length=50), table=<form_agro1>, primary_key=True, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1208f8e20>, for_update=False)), Column('yyyy', INTEGER(), table=<form_agro1>, primary_key=True, nullable=False), Column('mm', INTEGER(), table=<form_agro1>, primary_key=True, nullable=False), Column('dd', INTEGER(), table=<form_agro1>, primary_key=True, nullable=False), Column('Val_Elem101', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem102', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem103', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem105', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem002', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem003', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem099', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem072', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem073', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem074', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem554', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem075', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem076', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem561', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem562', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem563', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem513', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem005', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem504', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem532', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem137', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem018', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem518', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem511', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem512', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem503', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem515', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem564', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem565', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem566', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem531', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem530', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem541', VARCHAR(length=6), table=<form_agro1>), Column('Val_Elem542', VARCHAR(length=6), table=<form_agro1>), Column('Flag101', VARCHAR(length=1), table=<form_agro1>), Column('Flag102', VARCHAR(length=1), table=<form_agro1>), Column('Flag103', VARCHAR(length=1), table=<form_agro1>), Column('Flag105', VARCHAR(length=1), table=<form_agro1>), Column('Flag002', VARCHAR(length=1), table=<form_agro1>), Column('Flag003', VARCHAR(length=1), table=<form_agro1>), Column('Flag099', VARCHAR(length=1), table=<form_agro1>), Column('Flag072', VARCHAR(length=1), table=<form_agro1>), Column('Flag073', VARCHAR(length=1), table=<form_agro1>), Column('Flag074', VARCHAR(length=1), table=<form_agro1>), Column('Flag554', VARCHAR(length=1), table=<form_agro1>), Column('Flag075', VARCHAR(length=1), table=<form_agro1>), Column('Flag076', VARCHAR(length=1), table=<form_agro1>), Column('Flag561', VARCHAR(length=1), table=<form_agro1>), Column('Flag562', VARCHAR(length=1), table=<form_agro1>), Column('Flag563', VARCHAR(length=1), table=<form_agro1>), Column('Flag513', VARCHAR(length=1), table=<form_agro1>), Column('Flag005', VARCHAR(length=1), table=<form_agro1>), Column('Flag504', VARCHAR(length=1), table=<form_agro1>), Column('Flag532', VARCHAR(length=1), table=<form_agro1>), Column('Flag137', VARCHAR(length=1), table=<form_agro1>), Column('Flag018', VARCHAR(length=1), table=<form_agro1>), Column('Flag518', VARCHAR(length=1), table=<form_agro1>), Column('Flag511', VARCHAR(length=1), table=<form_agro1>), Column('Flag512', VARCHAR(length=1), table=<form_agro1>), Column('Flag503', VARCHAR(length=1), table=<form_agro1>), Column('Flag515', VARCHAR(length=1), table=<form_agro1>), Column('Flag564', VARCHAR(length=1), table=<form_agro1>), Column('Flag565', VARCHAR(length=1), table=<form_agro1>), Column('Flag566', VARCHAR(length=1), table=<form_agro1>), Column('Flag531', VARCHAR(length=1), table=<form_agro1>), Column('Flag530', VARCHAR(length=1), table=<form_agro1>), Column('Flag541', VARCHAR(length=1), table=<form_agro1>), Column('Flag542', VARCHAR(length=1), table=<form_agro1>), Column('signature', VARCHAR(length=45), table=<form_agro1>), Column('entryDatetime', DATETIME(), table=<form_agro1>), schema=None)),
    (   'remove_table',
        Table('bufr_crex_master', MetaData(), Column('Bufr_FXY', VARCHAR(length=6), table=<bufr_crex_master>, nullable=False), Column('Crex_Fxxyyy', VARCHAR(length=255), table=<bufr_crex_master>, nullable=False), Column('ElementName', VARCHAR(length=255), table=<bufr_crex_master>, nullable=False), Column('Bufr_Unit', VARCHAR(length=255), table=<bufr_crex_master>, nullable=False), Column('Bufr_Scale', VARCHAR(length=25), table=<bufr_crex_master>, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1209217c0>, for_update=False)), Column('Bufr_RefValue', VARCHAR(length=50), table=<bufr_crex_master>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1209218b0>, for_update=False)), Column('Bufr_DataWidth_Bits', VARCHAR(length=50), table=<bufr_crex_master>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1209219a0>, for_update=False)), Column('Crex_Unit', VARCHAR(length=25), table=<bufr_crex_master>), Column('Crex_Scale', VARCHAR(length=25), table=<bufr_crex_master>, nullable=False), Column('Crex_DataWidth', VARCHAR(length=25), table=<bufr_crex_master>, nullable=False), Column('Observation', VARCHAR(length=255), table=<bufr_crex_master>), Column('Crex_Data', VARCHAR(length=30), table=<bufr_crex_master>), Column('Bufr_Data', VARCHAR(length=255), table=<bufr_crex_master>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x120921b80>, for_update=False)), schema=None)),
    (   'remove_index',
        Index('strID_UNIQUE', Column('strID', INTEGER(), table=<aws_structures>, nullable=False))),
    (   'remove_table',
        Table('aws_structures', MetaData(), Column('strID', INTEGER(), table=<aws_structures>, nullable=False), Column('strName', VARCHAR(length=20), table=<aws_structures>, primary_key=True, nullable=False), Column('data_delimiter', VARCHAR(length=10), table=<aws_structures>, nullable=False), Column('hdrRows', INTEGER(), table=<aws_structures>, nullable=False), Column('txtQualifier', VARCHAR(length=5), table=<aws_structures>), schema=None)),
    (   'remove_table',
        Table('tm_307082', MetaData(), Column('order', INTEGER(), table=<tm_307082>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x120932130>, for_update=False)), Column('Bufr_Template', VARCHAR(length=50), table=<tm_307082>), Column('Crex_Template', VARCHAR(length=50), table=<tm_307082>), Column('Sequence_Descriptor1', VARCHAR(length=50), table=<tm_307082>), Column('Sequence_Descriptor0', VARCHAR(length=50), table=<tm_307082>), Column('Bufr_Element', VARCHAR(length=50), table=<tm_307082>), Column('Crex_Element', VARCHAR(length=255), table=<tm_307082>), Column('Climsoft_Element', VARCHAR(length=255), table=<tm_307082>), Column('Element_Name', VARCHAR(length=255), table=<tm_307082>), Column('synop_code', VARCHAR(length=255), table=<tm_307082>), Column('unit', VARCHAR(length=255), table=<tm_307082>), Column('scale', VARCHAR(length=255), table=<tm_307082>), Column('width', VARCHAR(length=255), table=<tm_307082>), Column('units_length_scale', VARCHAR(length=255), table=<tm_307082>), Column('data_type', VARCHAR(length=255), table=<tm_307082>), Column('selected', TINYINT(), table=<tm_307082>), Column('observation', VARCHAR(length=255), table=<tm_307082>), Column('Crex_Data', VARCHAR(length=255), table=<tm_307082>), Column('Bufr_Data', VARCHAR(length=50), table=<tm_307082>), schema=None)),
    (   'remove_table',
        Table('missing_stats', MetaData(), Column('STN_ID', VARCHAR(length=255), table=<missing_stats>, primary_key=True, nullable=False), Column('ELEM', BIGINT(), table=<missing_stats>, primary_key=True, nullable=False), Column('MISSING', BIGINT(), table=<missing_stats>), Column('Closing_Date', DATE(), table=<missing_stats>, primary_key=True, nullable=False), Column('Opening_Date', DATE(), table=<missing_stats>, primary_key=True, nullable=False), schema=None)),
    (   'remove_index',
        Index('SiteID_UNIQUE', Column('SiteID', VARCHAR(length=20), table=<aws_sites>, primary_key=True, nullable=False))),
    (   'remove_table',
        Table('aws_sites', MetaData(), Column('SiteID', VARCHAR(length=20), table=<aws_sites>, primary_key=True, nullable=False), Column('SiteName', VARCHAR(length=50), table=<aws_sites>), Column('InputFile', VARCHAR(length=255), table=<aws_sites>), Column('FilePrefix', VARCHAR(length=50), table=<aws_sites>), Column('chkPrefix', TINYINT(display_width=1), table=<aws_sites>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x120947130>, for_update=False)), Column('DataStructure', VARCHAR(length=50), table=<aws_sites>), Column('MissingDataFlag', VARCHAR(length=20), table=<aws_sites>), Column('awsServerIP', VARCHAR(length=50), table=<aws_sites>), Column('OperationalStatus', TINYINT(display_width=1, unsigned=True, zerofill=True), table=<aws_sites>, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1209472b0>, for_update=False)), Column('GTSEncode', TINYINT(display_width=1), table=<aws_sites>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1209473a0>, for_update=False)), Column('GTSHeader', VARCHAR(length=20), table=<aws_sites>), schema=None)),
    (   'remove_table',
        Table('form_synoptic_2_ra1', MetaData(), Column('stationId', VARCHAR(length=50), table=<form_synoptic_2_ra1>, primary_key=True, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x120947e50>, for_update=False)), Column('yyyy', INTEGER(), table=<form_synoptic_2_ra1>, primary_key=True, nullable=False), Column('mm', INTEGER(), table=<form_synoptic_2_ra1>, primary_key=True, nullable=False), Column('dd', INTEGER(), table=<form_synoptic_2_ra1>, primary_key=True, nullable=False), Column('hh', INTEGER(), table=<form_synoptic_2_ra1>, primary_key=True, nullable=False), Column('Val_Elem106', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem107', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem400', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem814', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem399', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem301', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem185', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem101', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem102', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem103', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem105', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem192', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem110', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem114', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem112', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem111', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem167', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem197', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem193', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem115', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem168', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem169', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem170', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem171', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem119', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem116', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem117', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem118', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem123', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem120', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem121', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem122', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem127', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem124', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem125', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem126', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem131', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem128', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem129', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem130', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem002', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem003', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem099', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem018', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem084', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem132', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem005', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem174', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Val_Elem046', VARCHAR(length=6), table=<form_synoptic_2_ra1>), Column('Flag106', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag107', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag400', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag814', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag399', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag301', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag185', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag101', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag102', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag103', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag105', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag192', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag110', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag114', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag112', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag111', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag167', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag197', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag193', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag115', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag168', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag169', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag170', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag171', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag119', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag116', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag117', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag118', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag123', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag120', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag121', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag122', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag127', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag124', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag125', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag126', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag131', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag128', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag129', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag130', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag002', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag003', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag099', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag018', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag084', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag132', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag005', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag174', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('Flag046', VARCHAR(length=1), table=<form_synoptic_2_ra1>), Column('signature', VARCHAR(length=45), table=<form_synoptic_2_ra1>), Column('entryDatetime', DATETIME(), table=<form_synoptic_2_ra1>), schema=None)),
    (   'remove_index',
        Index('identification', Column('seq', BIGINT(), table=<seq_element>, primary_key=True, nullable=False))),
    (   'remove_table',
        Table('seq_element', MetaData(), Column('seq', BIGINT(), table=<seq_element>, primary_key=True, nullable=False), Column('element_code', VARCHAR(length=50), table=<seq_element>), schema=None)),
    (   'remove_table',
        Table('bufr_crex_data', MetaData(), Column('nos', INTEGER(), table=<bufr_crex_data>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x120981bb0>, for_update=False)), Column('Bufr_Template', VARCHAR(length=50), table=<bufr_crex_data>), Column('Crex_Template', VARCHAR(length=50), table=<bufr_crex_data>), Column('Sequence_Descriptor1', VARCHAR(length=50), table=<bufr_crex_data>), Column('Sequence_Descriptor0', VARCHAR(length=50), table=<bufr_crex_data>), Column('Bufr_Element', VARCHAR(length=50), table=<bufr_crex_data>), Column('Crex_Element', VARCHAR(length=255), table=<bufr_crex_data>), Column('Climsoft_Element', VARCHAR(length=255), table=<bufr_crex_data>), Column('Element_Name', VARCHAR(length=255), table=<bufr_crex_data>), Column('Crex_Unit', VARCHAR(length=25), table=<bufr_crex_data>), Column('Crex_Scale', VARCHAR(length=25), table=<bufr_crex_data>, nullable=False), Column('Crex_DataWidth', VARCHAR(length=25), table=<bufr_crex_data>, nullable=False), Column('Bufr_Unit', VARCHAR(length=255), table=<bufr_crex_data>, nullable=False), Column('Bufr_Scale', VARCHAR(length=25), table=<bufr_crex_data>, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x12098b370>, for_update=False)), Column('Bufr_RefValue', VARCHAR(length=50), table=<bufr_crex_data>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x12098b460>, for_update=False)), Column('Bufr_DataWidth_Bits', VARCHAR(length=50), table=<bufr_crex_data>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x12098b550>, for_update=False)), Column('selected', TINYINT(), table=<bufr_crex_data>), Column('Observation', VARCHAR(length=255), table=<bufr_crex_data>), Column('Crex_Data', VARCHAR(length=30), table=<bufr_crex_data>), Column('Bufr_Data', VARCHAR(length=255), table=<bufr_crex_data>, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x12098b6d0>, for_update=False)), schema=None)),
    (   'add_index',
        Index('ix_faultresolution_associatedWith', Column('associatedWith', BigInteger(), table=<faultresolution>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('code', Column('instrumentId', VARCHAR(length=255), table=<instrument>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_instrument_installedAt', Column('installedAt', String(length=255), table=<instrument>))),
    (   'add_index',
        Index('ix_instrument_instrumentId', Column('instrumentId', String(length=255), table=<instrument>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('report_id', Column('reportId', BIGINT(), table=<instrumentfaultreport>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_instrumentfaultreport_reportId', Column('reportId', BigInteger(), table=<instrumentfaultreport>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_instrumentfaultreport_reportedFrom', Column('reportedFrom', String(length=255), table=<instrumentfaultreport>))),
    (   'add_index',
        Index('ix_instrumentinspection_performedAt', Column('performedAt', String(length=255), table=<instrumentinspection>))),
    (   'remove_index',
        Index('elementCode', Column('elementId', BIGINT(), table=<obselement>, primary_key=True, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x12069b820>, for_update=False)))),
    (   'add_index',
        Index('ix_obselement_elementId', Column('elementId', BigInteger(), table=<obselement>, primary_key=True, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x11fd03eb0>, for_update=False)))),
    (   'remove_index',
        Index('obsElementObservationInitial', Column('describedBy', BIGINT(), table=<observationfinal>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('stationObservationInitial', Column('recordedFrom', VARCHAR(length=255), table=<observationfinal>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_observationfinal_describedBy', Column('describedBy', BigInteger(), table=<observationfinal>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_observationfinal_recordedFrom', Column('recordedFrom', String(length=255), table=<observationfinal>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('obsElementObservationInitial', Column('describedBy', BIGINT(), table=<observationinitial>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('stationObservationInitial', Column('recordedFrom', VARCHAR(length=255), table=<observationinitial>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_observationinitial_describedBy', Column('describedBy', BigInteger(), table=<observationinitial>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_observationinitial_recordedFrom', Column('recordedFrom', String(length=255), table=<observationinitial>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('scheduleClassIdeification', Column('scheduleClass', VARCHAR(length=255), table=<obsscheduleclass>, primary_key=True, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1206d8a00>, for_update=False)))),
    (   'add_index',
        Index('ix_obsscheduleclass_refersTo', Column('refersTo', String(length=255), table=<obsscheduleclass>))),
    (   'add_index',
        Index('ix_obsscheduleclass_scheduleClass', Column('scheduleClass', String(length=255), table=<obsscheduleclass>, primary_key=True, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x11fd90910>, for_update=False)))),
    (   'add_index',
        Index('ix_paperarchive_classifiedInto', Column('classifiedInto', String(length=50), table=<paperarchive>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('paperarchivedef', Column('formId', VARCHAR(length=50), table=<paperarchivedefinition>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_paperarchivedefinition_formId', Column('formId', String(length=50), table=<paperarchivedefinition>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('physicalFeatureidentification_idx', Column('classifiedInto', VARCHAR(length=255), table=<physicalfeature>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('stationfeature', Column('associatedWith', VARCHAR(length=255), table=<physicalfeature>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_physicalfeature_associatedWith', Column('associatedWith', String(length=255), table=<physicalfeature>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_physicalfeature_classifiedInto', Column('classifiedInto', String(length=255), table=<physicalfeature>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('stationFeatureClass', Column('featureClass', VARCHAR(length=255), table=<physicalfeatureclass>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_physicalfeatureclass_featureClass', Column('featureClass', String(length=255), table=<physicalfeatureclass>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_physicalfeatureclass_refersTo', Column('refersTo', String(length=255), table=<physicalfeatureclass>))),
    (   'remove_index',
        Index('StationStationId', Column('stationId', VARCHAR(length=255), table=<station>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_station_stationId', Column('stationId', String(length=255), table=<station>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('obsElementobservationInitial', Column('describedBy', BIGINT(), table=<stationelement>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('stationobservationInitial', Column('recordedFrom', VARCHAR(length=255), table=<stationelement>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_stationelement_describedBy', Column('describedBy', BigInteger(), table=<stationelement>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_stationelement_recordedFrom', Column('recordedFrom', String(length=255), table=<stationelement>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_stationelement_recordedWith', Column('recordedWith', String(length=255), table=<stationelement>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_stationelement_scheduledFor', Column('scheduledFor', String(length=255), table=<stationelement>))),
    (   'remove_index',
        Index('stationQualifierIdentification', Column('belongsTo', VARCHAR(length=255), table=<stationqualifier>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_stationqualifier_belongsTo', Column('belongsTo', String(length=255), table=<stationqualifier>, primary_key=True, nullable=False)))]

@Shaibujnr
Copy link
Contributor

This is the output when the report is whitelisted to only the defined models.

[   (   'remove_index',
        Index('report_code', Column('reportCode', VARCHAR(length=255), table=<routinereportdefinition>))),
    (   'remove_index',
        Index('identification', Column('Cols', INTEGER(), table=<aws_test>, nullable=False))),
    (   'remove_index',
        Index('SiteID_UNIQUE', Column('SiteID', VARCHAR(length=20), table=<aws_sites>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('stationid_alias_identification', Column('idAlias', VARCHAR(length=255), table=<stationidalias>))),
    (   'remove_index',
        Index('stationstationidAlias', Column('belongsTo', VARCHAR(length=255), table=<stationidalias>))),
    (   'remove_index',
        Index('identification', Column('Element_Abbreviation', VARCHAR(length=50), table=<aws_toa5_bw1>))),
    (   'remove_index',
        Index('synop_code', Column('Element_Name', VARCHAR(length=255), table=<tm_307089>))),
    (   'remove_index',
        Index('synop_code1', Column('synop_code', VARCHAR(length=50), table=<tm_307089>))),
    (   'remove_index',
        Index('identification', Column('No', INTEGER(), table=<aws1>, nullable=False))),
    (   'remove_index',
        Index('ftpId_UNIQUE', Column('ftpId', VARCHAR(length=50), table=<aws_basestation>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('obsInitialIdentification', Column('StationId', VARCHAR(length=15), table=<qcabslimits>, nullable=False), Column('ElementId', BIGINT(), table=<qcabslimits>), Column('Datetime', DATETIME(), table=<qcabslimits>))),
    (   'remove_index',
        Index('identification', Column('Cols', INTEGER(), table=<aws_lsi1>, nullable=False))),
    (   'remove_index',
        Index('identification', Column('seq', BIGINT(), table=<seq_element>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('identification', Column('Cols', INTEGER(), table=<aws_lsi>, nullable=False))),
    (   'remove_index',
        Index('identification', Column('Element_Name', VARCHAR(length=50), table=<aws_rwanda1>))),
    (   'remove_index',
        Index('identification', Column('Cols', INTEGER(), table=<testing_aws1>, nullable=False))),
    (   'remove_index',
        Index('identification', Column('Cols', INTEGER(), table=<aws_rwanda_rain>, nullable=False))),
    (   'remove_index',
        Index('Identification', Column('Element_Name', VARCHAR(length=50), table=<aws_malawi1>))),
    (   'remove_index',
        Index('strID_UNIQUE', Column('strID', INTEGER(), table=<aws_structures>, nullable=False))),
    (   'remove_index',
        Index('identification', Column('Cols', INTEGER(), table=<ss>, nullable=False))),
    (   'remove_index',
        Index('identification', Column('No', INTEGER(), table=<abc>, nullable=False))),
    (   'remove_index',
        Index('identification', Column('No', INTEGER(), table=<aws_malawi12>, nullable=False))),
    (   'remove_index',
        Index('identification', Column('Cols', INTEGER(), table=<testing_aws>, nullable=False))),
    (   'remove_index',
        Index('Identification', Column('stationId', VARCHAR(length=10), table=<form_synoptic2_tdcf>, primary_key=True, nullable=False), Column('yyyy', BIGINT(), table=<form_synoptic2_tdcf>, primary_key=True, nullable=False), Column('mm', BIGINT(), table=<form_synoptic2_tdcf>, primary_key=True, nullable=False), Column('dd', BIGINT(), table=<form_synoptic2_tdcf>, primary_key=True, nullable=False), Column('hh', VARCHAR(length=5), table=<form_synoptic2_tdcf>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('report', Column('reportClass', VARCHAR(length=255), table=<routinereporttransmission>), Column('reportDatetime', DATETIME(), table=<routinereporttransmission>), Column('reportedFrom', VARCHAR(length=255), table=<routinereporttransmission>))),
    (   'remove_index',
        Index('identification', Column('Cols', INTEGER(), table=<aws_rema1>, nullable=False))),
    (   'add_index',
        Index('ix_faultresolution_associatedWith', Column('associatedWith', BigInteger(), table=<faultresolution>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('code', Column('instrumentId', VARCHAR(length=255), table=<instrument>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_instrument_installedAt', Column('installedAt', String(length=255), table=<instrument>))),
    (   'add_index',
        Index('ix_instrument_instrumentId', Column('instrumentId', String(length=255), table=<instrument>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('report_id', Column('reportId', BIGINT(), table=<instrumentfaultreport>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_instrumentfaultreport_reportId', Column('reportId', BigInteger(), table=<instrumentfaultreport>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_instrumentfaultreport_reportedFrom', Column('reportedFrom', String(length=255), table=<instrumentfaultreport>))),
    (   'add_index',
        Index('ix_instrumentinspection_performedAt', Column('performedAt', String(length=255), table=<instrumentinspection>))),
    (   'remove_index',
        Index('elementCode', Column('elementId', BIGINT(), table=<obselement>, primary_key=True, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x114b07dc0>, for_update=False)))),
    (   'add_index',
        Index('ix_obselement_elementId', Column('elementId', BigInteger(), table=<obselement>, primary_key=True, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x114032ee0>, for_update=False)))),
    (   'remove_index',
        Index('obsElementObservationInitial', Column('describedBy', BIGINT(), table=<observationfinal>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('stationObservationInitial', Column('recordedFrom', VARCHAR(length=255), table=<observationfinal>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_observationfinal_describedBy', Column('describedBy', BigInteger(), table=<observationfinal>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_observationfinal_recordedFrom', Column('recordedFrom', String(length=255), table=<observationfinal>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('obsElementObservationInitial', Column('describedBy', BIGINT(), table=<observationinitial>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('stationObservationInitial', Column('recordedFrom', VARCHAR(length=255), table=<observationinitial>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_observationinitial_describedBy', Column('describedBy', BigInteger(), table=<observationinitial>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_observationinitial_recordedFrom', Column('recordedFrom', String(length=255), table=<observationinitial>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('scheduleClassIdeification', Column('scheduleClass', VARCHAR(length=255), table=<obsscheduleclass>, primary_key=True, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x114b41b20>, for_update=False)))),
    (   'add_index',
        Index('ix_obsscheduleclass_refersTo', Column('refersTo', String(length=255), table=<obsscheduleclass>))),
    (   'add_index',
        Index('ix_obsscheduleclass_scheduleClass', Column('scheduleClass', String(length=255), table=<obsscheduleclass>, primary_key=True, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x1140bf940>, for_update=False)))),
    (   'add_index',
        Index('ix_paperarchive_classifiedInto', Column('classifiedInto', String(length=50), table=<paperarchive>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('paperarchivedef', Column('formId', VARCHAR(length=50), table=<paperarchivedefinition>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_paperarchivedefinition_formId', Column('formId', String(length=50), table=<paperarchivedefinition>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('physicalFeatureidentification_idx', Column('classifiedInto', VARCHAR(length=255), table=<physicalfeature>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('stationfeature', Column('associatedWith', VARCHAR(length=255), table=<physicalfeature>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_physicalfeature_associatedWith', Column('associatedWith', String(length=255), table=<physicalfeature>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_physicalfeature_classifiedInto', Column('classifiedInto', String(length=255), table=<physicalfeature>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('stationFeatureClass', Column('featureClass', VARCHAR(length=255), table=<physicalfeatureclass>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_physicalfeatureclass_featureClass', Column('featureClass', String(length=255), table=<physicalfeatureclass>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_physicalfeatureclass_refersTo', Column('refersTo', String(length=255), table=<physicalfeatureclass>))),
    (   'remove_index',
        Index('StationStationId', Column('stationId', VARCHAR(length=255), table=<station>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_station_stationId', Column('stationId', String(length=255), table=<station>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('obsElementobservationInitial', Column('describedBy', BIGINT(), table=<stationelement>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('stationobservationInitial', Column('recordedFrom', VARCHAR(length=255), table=<stationelement>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_stationelement_describedBy', Column('describedBy', BigInteger(), table=<stationelement>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_stationelement_recordedFrom', Column('recordedFrom', String(length=255), table=<stationelement>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_stationelement_recordedWith', Column('recordedWith', String(length=255), table=<stationelement>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_stationelement_scheduledFor', Column('scheduledFor', String(length=255), table=<stationelement>))),
    (   'remove_index',
        Index('stationQualifierIdentification', Column('belongsTo', VARCHAR(length=255), table=<stationqualifier>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_stationqualifier_belongsTo', Column('belongsTo', String(length=255), table=<stationqualifier>, primary_key=True, nullable=False)))]

@Shaibujnr
Copy link
Contributor

The whitelist algorithm basically, takes the second item in the list, checks if it's a Table object then checks the name of the Table object against the list of the tables and filters accordingly.

For the second output, it looks like all we have left is add_index and remove_index operations. I can easily extend the algorithm to check for index objects and then check the table name of the index objects then filter accordingly too. And for every operation we encounter we can keep extending the algorithm to support it.

So let me know what you think @isedwards

@isedwards
Copy link
Contributor Author

Yes, it makes sense. Let's add a capability to ignore the add_index and remove_index operations for the tables that we're not interested in.

Let's refer to these 25 tables as climsoft/v4_1_1_core.py.

Once we have an empty result from get_schema_diff() for these 25 tables when comparing to the database schema then it may make sense to try and see whether we can represent more of the tables with a larger complete set of models called climsoft/v4_1_1.

@Shaibujnr
Copy link
Contributor

output after whitelisting indexes too.

[   (   'add_index',
        Index('ix_faultresolution_associatedWith', Column('associatedWith', BigInteger(), table=<faultresolution>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('code', Column('instrumentId', VARCHAR(length=255), table=<instrument>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_instrument_installedAt', Column('installedAt', String(length=255), table=<instrument>))),
    (   'add_index',
        Index('ix_instrument_instrumentId', Column('instrumentId', String(length=255), table=<instrument>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('report_id', Column('reportId', BIGINT(), table=<instrumentfaultreport>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_instrumentfaultreport_reportId', Column('reportId', BigInteger(), table=<instrumentfaultreport>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_instrumentfaultreport_reportedFrom', Column('reportedFrom', String(length=255), table=<instrumentfaultreport>))),
    (   'add_index',
        Index('ix_instrumentinspection_performedAt', Column('performedAt', String(length=255), table=<instrumentinspection>))),
    (   'remove_index',
        Index('elementCode', Column('elementId', BIGINT(), table=<obselement>, primary_key=True, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x11d2b69d0>, for_update=False)))),
    (   'add_index',
        Index('ix_obselement_elementId', Column('elementId', BigInteger(), table=<obselement>, primary_key=True, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x11c99f100>, for_update=False)))),
    (   'remove_index',
        Index('obsElementObservationInitial', Column('describedBy', BIGINT(), table=<observationfinal>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('stationObservationInitial', Column('recordedFrom', VARCHAR(length=255), table=<observationfinal>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_observationfinal_describedBy', Column('describedBy', BigInteger(), table=<observationfinal>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_observationfinal_recordedFrom', Column('recordedFrom', String(length=255), table=<observationfinal>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('obsElementObservationInitial', Column('describedBy', BIGINT(), table=<observationinitial>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('stationObservationInitial', Column('recordedFrom', VARCHAR(length=255), table=<observationinitial>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_observationinitial_describedBy', Column('describedBy', BigInteger(), table=<observationinitial>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_observationinitial_recordedFrom', Column('recordedFrom', String(length=255), table=<observationinitial>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('scheduleClassIdeification', Column('scheduleClass', VARCHAR(length=255), table=<obsscheduleclass>, primary_key=True, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x11d308640>, for_update=False)))),
    (   'add_index',
        Index('ix_obsscheduleclass_refersTo', Column('refersTo', String(length=255), table=<obsscheduleclass>))),
    (   'add_index',
        Index('ix_obsscheduleclass_scheduleClass', Column('scheduleClass', String(length=255), table=<obsscheduleclass>, primary_key=True, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x11ca0ee80>, for_update=False)))),
    (   'add_index',
        Index('ix_paperarchive_classifiedInto', Column('classifiedInto', String(length=50), table=<paperarchive>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('paperarchivedef', Column('formId', VARCHAR(length=50), table=<paperarchivedefinition>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_paperarchivedefinition_formId', Column('formId', String(length=50), table=<paperarchivedefinition>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('physicalFeatureidentification_idx', Column('classifiedInto', VARCHAR(length=255), table=<physicalfeature>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('stationfeature', Column('associatedWith', VARCHAR(length=255), table=<physicalfeature>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_physicalfeature_associatedWith', Column('associatedWith', String(length=255), table=<physicalfeature>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_physicalfeature_classifiedInto', Column('classifiedInto', String(length=255), table=<physicalfeature>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('stationFeatureClass', Column('featureClass', VARCHAR(length=255), table=<physicalfeatureclass>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_physicalfeatureclass_featureClass', Column('featureClass', String(length=255), table=<physicalfeatureclass>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_physicalfeatureclass_refersTo', Column('refersTo', String(length=255), table=<physicalfeatureclass>))),
    (   'remove_index',
        Index('StationStationId', Column('stationId', VARCHAR(length=255), table=<station>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_station_stationId', Column('stationId', String(length=255), table=<station>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('obsElementobservationInitial', Column('describedBy', BIGINT(), table=<stationelement>, primary_key=True, nullable=False))),
    (   'remove_index',
        Index('stationobservationInitial', Column('recordedFrom', VARCHAR(length=255), table=<stationelement>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_stationelement_describedBy', Column('describedBy', BigInteger(), table=<stationelement>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_stationelement_recordedFrom', Column('recordedFrom', String(length=255), table=<stationelement>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_stationelement_recordedWith', Column('recordedWith', String(length=255), table=<stationelement>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_stationelement_scheduledFor', Column('scheduledFor', String(length=255), table=<stationelement>))),
    (   'remove_index',
        Index('stationQualifierIdentification', Column('belongsTo', VARCHAR(length=255), table=<stationqualifier>, primary_key=True, nullable=False))),
    (   'add_index',
        Index('ix_stationqualifier_belongsTo', Column('belongsTo', String(length=255), table=<stationqualifier>, primary_key=True, nullable=False)))]

@Shaibujnr
Copy link
Contributor

After generating the migration script with alembic revision --autogenerate I have manually updated the models to align more with the database. Right now, get_schema_diff returns [] for the core tables, however the alembic migration script still detects the below.

def upgrade():
    # ### commands auto generated by Alembic - please adjust! ###
    op.alter_column('featuregeographicalposition', 'latitude',
               existing_type=mysql.DOUBLE(precision=11, scale=6, asdecimal=True),
               type_=sa.Float(precision=11, asdecimal=True),
               existing_nullable=True)
    op.alter_column('featuregeographicalposition', 'longitude',
               existing_type=mysql.DOUBLE(precision=11, scale=6, asdecimal=True),
               type_=sa.Float(precision=11, asdecimal=True),
               existing_nullable=True)
    op.alter_column('station', 'latitude',
               existing_type=mysql.DOUBLE(precision=11, scale=6, asdecimal=True),
               type_=sa.Float(precision=11, asdecimal=True),
               existing_nullable=True)
    op.alter_column('station', 'longitude',
               existing_type=mysql.DOUBLE(precision=11, scale=6, asdecimal=True),
               type_=sa.Float(precision=11, asdecimal=True),
               existing_nullable=True)
    op.alter_column('stationlocationhistory', 'latitude',
               existing_type=mysql.DOUBLE(precision=11, scale=6, asdecimal=True),
               type_=sa.Float(precision=11, asdecimal=True),
               existing_nullable=True)
    op.alter_column('stationlocationhistory', 'longitude',
               existing_type=mysql.DOUBLE(precision=11, scale=6, asdecimal=True),
               type_=sa.Float(precision=11, asdecimal=True),
               existing_nullable=True)
    # ### end Alembic commands ###

For some reason, compare_metadata doesn't output alter_column statements. These columns are of type double in the database and that type is only specific to mysql. SQLACodegen generated the columns as float.

how do we resolve this? @isedwards

@isedwards
Copy link
Contributor Author

Thank you @Shaibujnr - it's odd that compare_metadata doesn't pick this up - I thought alembic used compare_metadata in order to produce_migrations (autogenerate api).

It looks like the best solution would be to manually update the output of sqlacodegen to use mysql.DOUBLE (again in a separate commit to make it easier to document and reference the changes)? Presumably get_schema_diff would still return [] if we do this but the model would more accurately represent the database.

@Shaibujnr Shaibujnr mentioned this issue Sep 8, 2021
Merged
@Shaibujnr
Copy link
Contributor

@isedwards https://github.com/opencdms/pyopencdms/pull/28

@isedwards isedwards mentioned this issue Sep 16, 2021
Merged
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Feat/climsoft models Shaibujnr/pyopencdms
2 participants
@isedwards @Shaibujnr