DeepStochLog-LM is a neuro-symbolic framework that combines grammar, logic, probabilities, and language models. By writing a DeepStochLog-LM program, one can train a language model with the given background knowledge. We show the power of DeepStochLog-LM in providing the validity guarantee for the text-to-SQL task. It is an extension of DeepStochLog. For more information, consult the NeSy2024 paper Valid Text-to-SQL Generation with Unification-based DeepStochLog.
sudo apt-add-repository ppa:swi-prolog/stable
sudo apt-get update
sudo apt-get install swi-prolog
conda env create -f deepstochlog-lm.yml
Download the Spider dataset from here, and put it under "data/".
- For task 1
python src/process.py --task 1
- For task 2
python src/process.py --task 2
- For setting 1, Bert as selection switcher
python src/task1/task1.py --ss_type "bert"
- For setting 2, T5 as selection switcher
python src/task1/task1.py
- Train the language models with data in "data/lms_task2" and "src/task2/train_lms.ipynb". Download the saved checkpoints and put them under "models/".
- Generate outputs
python src/task2/task2.py
- Evaluation
output/evaluate.bat
Outputs from vanilla T5-small baseline are generated with "data/train_t5_baseline.json", "data/test_t5_baseline.json", and "src/task2/train_lms.ipynb". DAIL-SQL, DIN-SQL, Graphix-T5, and C3 putputs are from their orginal repos. T5-small+CFGs outputs are generated by "src/task2/task2.py" with context_sensitive = False.
The paper is also accepted to NeSy24. Please cite that version of the paper when the proceedings are out.
We would like to acknowledge the following sources for their contributions to this project:
- Spider for the dataset and the official evaluation matrices
- DeepStochLog for its implementation of NDCGs