Data Integration Demo

Data Integration for Data with Schema Changes

1. Attribute Renaming
2. Cell Value Reformatting
3. Key Expansion
4. Key Order Changing

Table Cell Position Prediction Using Sequence/Language Model

1. Column (attribute) prediction
2. Key index prediction
3. Aggregation mode prediction (only apply to covid-19 dataset)

Dataset

Covid-19 scenario

• Covid-19 dataset https://github.com/CSSEGISandData/COVID-19

• Google Mobility Report dataset https://www.google.com/covid19/mobility/

Machine log scenario

• Linux log data
• macOS log data
• Android log data

Model

• Pretrained Word Embedding (fastText) + Bi-LSTM
• Pretrained Word Tokenizer (BERT Tokenizer) + Encoder-only Transfomer

Usage

Dataset:

Github Repository

Google Drive

Download dataset:

import gdown
gdown.download('https://drive.google.com/drive/folders/19oLAKktjI0uk8v4lcdBTnRBTyqN-tGeR', output=None, quiet=False)

Run with Google Colab or local Jupyter Notebook.