Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SQLite support via a MySQL parser #9

Merged
merged 109 commits into from
Feb 23, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
109 commits
Select commit Hold shift + click to select a range
2de6628
Draft lexer-based SQL translation layer
adamziel Feb 8, 2023
46b0621
Add a note about lexer
adamziel Feb 8, 2023
41ca55d
Remove dev artifacts
adamziel Feb 8, 2023
1d9c695
Add support for more query types
adamziel Feb 8, 2023
39d0c89
phpcbf
aristath Feb 9, 2023
1fd0d14
fix a fatal error
aristath Feb 9, 2023
4b47e76
Add the standalone Lexer object
aristath Feb 9, 2023
53ee424
Use constants & static vars from WP_SQLite_Lexer
aristath Feb 9, 2023
e8a1bc4
WordPress Coding Standards
aristath Feb 9, 2023
8fa6227
no need for else after return
aristath Feb 9, 2023
b0e642a
Merged all the query translation logic into the Translator class
adamziel Feb 9, 2023
cbfe844
Bring over field_types_translation and mysql_php_date_formats to the …
adamziel Feb 9, 2023
eb8e4de
minor tweaks
aristath Feb 9, 2023
dcf0243
Consolidate the translation logic and sketch some unit tests
adamziel Feb 9, 2023
6317df8
Converge all translators in the SQLiteTranslationResult class
adamziel Feb 9, 2023
a6ec488
MVP WordPress integration – Installation, frontend, wp-admin seem to …
adamziel Feb 9, 2023
03390ad
Adjust a few details to get the unit suite to boot
adamziel Feb 9, 2023
c543d27
coding standards
aristath Feb 10, 2023
5e0afe8
bugfix
aristath Feb 10, 2023
30819c6
cleanup + restructure
aristath Feb 10, 2023
54ce5ef
add $options to consume_all
aristath Feb 10, 2023
86482b1
add some docs
aristath Feb 10, 2023
38464ea
Remove the WP_SQLite_Query object
aristath Feb 10, 2023
8a795f6
Record num rows for insert queries
adamziel Feb 10, 2023
6c6239e
Support for TRANCATE queries
adamziel Feb 10, 2023
4ff78af
Use WP_SQLite_Translator::get_query_object instead of SQLiteQuery
adamziel Feb 10, 2023
5f7fb23
Remove the WP_SQLite_Translation_Result class
aristath Feb 10, 2023
c998a3a
Merge branch 'main' into lexer-translation
aristath Feb 10, 2023
b4ee88a
Support more query types – WooCommerce is now installing and reportin…
adamziel Feb 10, 2023
6389246
Fix a typo
adamziel Feb 10, 2023
79a16a3
Implement REGEX via a PHP callback
adamziel Feb 10, 2023
2d5a64b
Refactor the lexer
aristath Feb 13, 2023
d5961f7
Documentation + Coding-standards + readability tweaks
aristath Feb 14, 2023
fad76d7
Remove parser dependency in CREATE TABLE queries
adamziel Feb 14, 2023
1e1629d
CS fixes & other small tweaks
aristath Feb 15, 2023
345e643
Add VARIABLES
aristath Feb 15, 2023
a30f5b5
Merge branch 'main' into lexer-translation
aristath Feb 16, 2023
84a04b7
Introduce WP_SQLite_Crosscheck_DB – a DB class that runs all the quer…
adamziel Feb 16, 2023
b8b94ce
Fix a few more discrepancies between sqlite and mysql
adamziel Feb 16, 2023
26faba7
Support counting rows per table
adamziel Feb 17, 2023
1651ea6
Crosscheck mode: Restore SQLite's Autoincreement values after rollback
adamziel Feb 17, 2023
7987cce
Remove unused code
adamziel Feb 17, 2023
fe718f4
Fix unit tests
adamziel Feb 17, 2023
c9eeb76
Major API cleanup
adamziel Feb 17, 2023
250a21c
Plug in SQLite's regexp function, not regexpp
adamziel Feb 17, 2023
4d864d3
API Cleanup
adamziel Feb 17, 2023
0749584
Resolve lexer bug where it doesn't correctly calculate the length of …
adamziel Feb 17, 2023
444c5a8
Support compound ALTER queries
adamziel Feb 17, 2023
58425bf
Remove the sql queries log
adamziel Feb 17, 2023
4bc8995
Retry on SQLite BUSY error, stringify COUNT(*) results
adamziel Feb 17, 2023
e8b1a17
Reverse the argument order in REGEXP handler and add unit tests
adamziel Feb 18, 2023
5b6a0c3
Stringify fetch data
adamziel Feb 18, 2023
3bb4b9b
Fix a bug in the field() function implementation
adamziel Feb 18, 2023
582837e
Improve handling of YEAR(), MONTH(), etc
adamziel Feb 18, 2023
190d2bb
Translate DATE_ADD to DATETIME instead of DATE
adamziel Feb 18, 2023
ee5a9bb
Store the number of affected rows as an integer, not as a string
adamziel Feb 18, 2023
60f76ac
Migrate date handling to user defined functions
adamziel Feb 18, 2023
124df49
Support case-insensitive text field lookups
adamziel Feb 18, 2023
d33fd36
Add geom data type support
adamziel Feb 18, 2023
4a1d62e
Support for MySQL data format with a timezone baked in
adamziel Feb 18, 2023
4bf2135
Implement get_col_charset with a hardcoded utf8mb4 encoding
adamziel Feb 18, 2023
66fb606
Support 24 hour format
adamziel Feb 18, 2023
a8d1cda
Support arbitrary ON DUPLICATE KEY queries
adamziel Feb 18, 2023
ab80f7b
Add support for REGEXP BINARY
adamziel Feb 18, 2023
1ce224c
Support inserting "\f" and "\v" as a literal string, not as a form fe…
adamziel Feb 18, 2023
d5255de
Extend debug info in slqite pdo engine
adamziel Feb 18, 2023
8ba62bb
Support truncating invalid dates
adamziel Feb 18, 2023
29d73e1
Support MySQL's WEEKDAY() function
adamziel Feb 18, 2023
b7a8502
Increase the quality of date support
adamziel Feb 18, 2023
e73c601
Add support for date lookup based on comparing date_format() to floats
adamziel Feb 18, 2023
6954055
Support for ON DUPLICATE KEY with a composite pk
adamziel Feb 19, 2023
dd9b222
Brush up compatibility with class wpdb
adamziel Feb 19, 2023
2d20a2d
Improve readability of the console-logged errors during the test run
adamziel Feb 19, 2023
6e012ce
Translate DESCRIBE queries
adamziel Feb 19, 2023
db6e8b3
ALTER TABLE CHANGE COLUMN support
adamziel Feb 20, 2023
6ad37f1
Fix one last failing dbDelta test
adamziel Feb 20, 2023
bce3338
Migrate a number of SQLiteTranslator tests to SQLiteEngine tests
adamziel Feb 20, 2023
c60a29f
Fix up dayofweek() return value
adamziel Feb 20, 2023
1544eeb
Add methods docs & fix coding-standards issues
aristath Feb 20, 2023
7bedd3f
Support for SELECT FROM DUAL
adamziel Feb 20, 2023
578a766
more docs & CS fixes
aristath Feb 21, 2023
758a058
Remove the PDO_Driver class
aristath Feb 21, 2023
77d8c85
moved files + CS/Docs fixes
aristath Feb 21, 2023
24eba06
minor formatting tweaks
aristath Feb 21, 2023
f1bcc08
The sqlite_regexp method doesn't seem to be used anywhere
aristath Feb 21, 2023
adf1820
Add phpcs:ignore comments for builtin methods
aristath Feb 21, 2023
38b2804
Remove unused methods from the WP_SQLite_Tokens_List object
aristath Feb 21, 2023
ddc7b0c
copy/paste error
aristath Feb 21, 2023
025b43a
WP_SQLite_Tokens_List doesn't need to implement ArrayAccess
aristath Feb 21, 2023
94f524a
revert comments deletion
aristath Feb 21, 2023
d93383c
Docs fix - props @adamziel
aristath Feb 21, 2023
ee45851
Avoid using list() to assign vars
aristath Feb 21, 2023
5297093
Inline the process_results() method of the SQLite engine class
adamziel Feb 21, 2023
3efdc7c
Remove the private _results variable
adamziel Feb 21, 2023
6411003
move require_once calls to db.php
aristath Feb 21, 2023
fcd5a54
Merge remote-tracking branch 'WordPress/lexer-translation' into lexer…
aristath Feb 21, 2023
f93fb03
CS
aristath Feb 21, 2023
dc18b26
add files to .gitattributes
aristath Feb 22, 2023
203f0e8
Merge PDO_Engine & Translator classes
aristath Feb 22, 2023
a8dbf21
copy/paste error
aristath Feb 22, 2023
f5a6a5e
Update wp-includes/sqlite/class-wp-sqlite-translator.php
aristath Feb 22, 2023
c7fb01e
Update wp-includes/sqlite/class-wp-sqlite-db.php
aristath Feb 22, 2023
11c3a9a
fix comments formatting
aristath Feb 22, 2023
12747f4
Move files
aristath Feb 22, 2023
7e1af11
Up the version & update the readme file
aristath Feb 22, 2023
fd0cba9
Merge Tokens_List in Lexer
aristath Feb 22, 2023
7aa61de
Simpler solution for WP_SQLite_Object_Array
aristath Feb 23, 2023
3b9043d
Add more keywords for MariaDB
aristath Feb 23, 2023
ecc760d
inline docs for crosscheck & cleanup
aristath Feb 23, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@
.gitignore export-ignore
composer.json export-ignore
phpcs.xml.dist export-ignore
tests/*.php export-ignore
4 changes: 3 additions & 1 deletion load.php
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,14 @@
* Plugin Name: SQLite Database Integration
* Description: SQLite database driver drop-in. (based on SQLite Integration by Kojima Toshiyasu)
* Author: WordPress Performance Team
* Version: 1.0.3
* Version: 2.0
* Requires PHP: 5.6
* Textdomain: sqlite-database-integration
*
* This project is based on the original work of Kojima Toshiyasu and his SQLite Integration plugin,
* and the work of Evan Mattson and his WP SQLite DB plugin - See https://github.com/aaemnnosttv/wp-sqlite-db
*
* @package wp-sqlite-integration
*/

define( 'SQLITE_MAIN_FILE', __FILE__ );
Expand Down
1 change: 1 addition & 0 deletions phpcs.xml.dist
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
<file>.</file>

<rule ref="WordPress-Core"/>
<rule ref="WordPress-Docs"/>
<rule ref="WordPress.CodeAnalysis.EmptyStatement"/>

<!-- Directories and third party library exclusions. -->
Expand Down
7 changes: 6 additions & 1 deletion readme.txt
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Contributors: wordpressdotorg, aristath
Requires at least: 6.0
Tested up to: 6.1
Requires PHP: 5.6
Stable tag: 1.0.3
Stable tag: 2.0
License: GPLv2 or later
License URI: https://www.gnu.org/licenses/gpl-2.0.html
Tags: performance, database
Expand All @@ -15,11 +15,16 @@ SQLite-integration plugin from the WordPress Performance Team.

The SQLite plugin is a community, feature plugin. The intent is to allow testing an SQLite integration with WordPress and gather feedback, with the goal of eventually landing it in WordPress core.

This project is based on the original work of Kojima Toshiyasu and his SQLite Integration plugin, and the work of Evan Mattson and his WP SQLite DB plugin - See https://github.com/aaemnnosttv/wp-sqlite-db.

It also includes code from the PHPMyAdmin project (specifically parts of the PHPMyAdmin/sql-parser library), licensed under the GPL v2 or later. More info on the PHPMyAdmin/sql-parser library can be found on [GitHub](https://github.com/phpmyadmin/sql-parser).

== Frequently Asked Questions ==

= What is the purpose of this plugin? =

The primary purpose of the SQLite plugin is to allow testing the use of an SQLite database, with the goal to eventually land in WordPress core.

You can read the original proposal on the [Make blog](https://make.wordpress.org/core/2022/09/12/lets-make-wordpress-officially-support-sqlite/), as well as the [call for testing](https://make.wordpress.org/core/2022/12/20/help-us-test-the-sqlite-implementation/) for more context and useful information.

= Can I use this plugin on my production site? =
Expand Down
85 changes: 85 additions & 0 deletions tests/QueryRewriterTests.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
<?php

require __DIR__ . '/class-wp-sqlite-query-rewriter.php';
require __DIR__ . '/class-wp-sqlite-lexer.php';

use PHPUnit\Framework\TestCase;

class WP_SQLite_Query_RewriterTests extends TestCase {


public function testConsume() {
$r = new WP_SQLite_Query_Rewriter(
array(
WP_SQLite_Lexer::get_token( ' ', WP_SQLite_Lexer::TYPE_DELIMITER ),
WP_SQLite_Lexer::get_token( 'int', WP_SQLite_Lexer::TYPE_KEYWORD, WP_SQLite_Lexer::FLAG_KEYWORD_DATA_TYPE ),
WP_SQLite_Lexer::get_token( ' ', WP_SQLite_Lexer::TYPE_DELIMITER ),
WP_SQLite_Lexer::get_token( 'DATE_ADD', WP_SQLite_Lexer::TYPE_KEYWORD, WP_SQLite_Lexer::FLAG_KEYWORD_FUNCTION ),
WP_SQLite_Lexer::get_token( ' ', WP_SQLite_Lexer::TYPE_DELIMITER ),
WP_SQLite_Lexer::get_token( 'SET', WP_SQLite_Lexer::TYPE_KEYWORD ),
WP_SQLite_Lexer::get_token( ' ', WP_SQLite_Lexer::TYPE_WHITESPACE ),
)
);
$this->assertEquals(
'int',
$r->consume(
array(
'type' => WP_SQLite_Lexer::TYPE_KEYWORD,
'flags' => WP_SQLite_Lexer::FLAG_KEYWORD_DATA_TYPE,
)
)->value
);
$this->assertEquals(
'DATE_ADD',
$r->consume(
array(
'type' => WP_SQLite_Lexer::TYPE_KEYWORD,
'flags' => WP_SQLite_Lexer::FLAG_KEYWORD_FUNCTION,
)
)->value
);
}
public function testSkip() {
$r = new WP_SQLite_Query_Rewriter(
array(
WP_SQLite_Lexer::get_token( 'DO', WP_SQLite_Lexer::TYPE_KEYWORD ),
WP_SQLite_Lexer::get_token( ' ', WP_SQLite_Lexer::TYPE_WHITESPACE ),
WP_SQLite_Lexer::get_token( 'UPDATE', WP_SQLite_Lexer::TYPE_KEYWORD ),
WP_SQLite_Lexer::get_token( ' ', WP_SQLite_Lexer::TYPE_WHITESPACE ),
WP_SQLite_Lexer::get_token( 'SET', WP_SQLite_Lexer::TYPE_KEYWORD ),
WP_SQLite_Lexer::get_token( ' ', WP_SQLite_Lexer::TYPE_WHITESPACE ),
)
);
$this->assertEquals(
'DO',
$r->skip()->value
);
$this->assertEquals(
'UPDATE',
$r->skip()->value
);
}

public function skip_over() {
$r = new WP_SQLite_Query_Rewriter(
array(
WP_SQLite_Lexer::get_token( 'DO', WP_SQLite_Lexer::TYPE_KEYWORD ),
WP_SQLite_Lexer::get_token( ' ', WP_SQLite_Lexer::TYPE_WHITESPACE ),
WP_SQLite_Lexer::get_token( 'UPDATE', WP_SQLite_Lexer::TYPE_KEYWORD ),
WP_SQLite_Lexer::get_token( ' ', WP_SQLite_Lexer::TYPE_WHITESPACE ),
WP_SQLite_Lexer::get_token( 'SET', WP_SQLite_Lexer::TYPE_KEYWORD ),
WP_SQLite_Lexer::get_token( ' ', WP_SQLite_Lexer::TYPE_WHITESPACE ),
)
);
$buffer = $r->skip_over(
array(
'values' => array( 'UPDATE' ),
)
);
$this->assertCount( 3, $buffer );
$this->assertEquals( 'DO', $buffer[0]->value );
$this->assertEquals( ' ', $buffer[1]->value );
$this->assertEquals( 'UPDATE', $buffer[2]->value );
}

}
Loading