Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rusqlite #457

Closed
wants to merge 2,323 commits into from
Closed

rusqlite #457

wants to merge 2,323 commits into from

Conversation

MarinPostma
Copy link
Contributor

@MarinPostma MarinPostma commented Oct 16, 2023

No description provided.

gwenn and others added 30 commits November 11, 2022 15:08
Intoduce one_column for tests
Flushing the buffer on each new entry makes iotrace
much more useful for debugging interactive sessions.
We don't use Fossil nor plan to, so drop the config.
The distinction will make it abundantly clear that the user
is running libSQL.
If it fails, it should not break the build.
Currently, the runtime of choice is wasmtime
and its libwasmtime.so library, available at
https://github.com/bytecodealliance/wasmtime/releases
From now on, configuring with ./configure --enable-wasm-runtime
turns on WebAssembly integration.
The function can be used to run abitrary WebAssembly
functions. It takes the WebAssembly module body
and the function name in user data, and forwards
all the arguments to this function as is.
The final ABI for these functions still remains to be specified,
but for now the following assumptions hold true:
 - integers are passed as is
 - doubles are passed as is
 - strings are passed as a pointer to the following structure:
     [1 byte with value SQLITE_TEXT (3)][data]
 - blobs are going to be passed as a pointer to:
     [1 byte with value SQLITE_BLOB (4)][4 bytes of size][blob]
   ... but are not supported yet
 - nulls are passed as a pointer to:
     [1 byte with value SQLITE_NULL (5)]
   ... but are not supported yet
In addition to looking up all of the built-in and user-defined
functions, they can now also be stored in a special
libsql_wasm_func_table table. This table's schema is
  name text PRIMARY KEY, body text
, where `body` is expected to be valid WebAssembly code
represented in .wat - WebAssembly Text Format.
The WebAssembly module compiled from this source code
should export a function with name `name`, which can then
be called as if it was a regular user-defined function.

The implementation is deeply experimental, as there is no cache
of compiled programs whatsoever, so running these functions
is heavy and involves recompiling the source each time.
The routines will later be called when parsing CREATE FUNCTION
and DROP FUNCTION statements, which are more or less syntactic
sugar over inserting an entry in the libsql_wasm_func_table table
(or deleting from it, accordingly).
The helper routine which compiles WebAssembly
and registers the wasm function dynamically is now split
into two parts - one will be used when parsing to validate
the output WebAssembly.
The routine creates the libsql_wasm_func_table table,
responsible for storing WebAssembly source code for dynamically
added Wasm functions.
It will be used to drop functions via the DROP FUNCTION statement.
The table will be created on startup in order to allow
registering Wasm functions dynamically.
The new experimental syntax loosely follows SQL's CREATE FUNCTION.
It still misses OR REPLACE keywords which would allow overriding
an already existing function.
The suite is wrapped in a feature flag, because user-defined
functions need to be compiled opt-in into libSQL.
The new command runs Rust test with udf feature enabled,
which assumes that libSQL was compiled with --enable-wasm-runtime.
Previous dynamic lookup of Wasm function was lazy and only performed
on its first use - this is redundant, and the logic is much clearer
when the functions are initialized on startup + when they're registered
dynamically.
The newly covered cases also check operations on strings,
blobs and null.
This document will serve as an entrypoint for various extensions
added to libSQL and not necessarily compatible with SQLite.
Eventually it might grow to become a separate directory.
Before the fix, single quotes were not properly loaded
from the database.
If .init_wasm_func_table is the initial call to the shell,
call open_db() first to initialize the connection.
gwenn and others added 29 commits April 17, 2023 19:46
Use track_caller for panicking methods
Force linking against system sqlite libs
fix compilation for target wasm32-wasi
Add SQLITE_MAX_COLUMN compile-time option
Following the mechanism already present for picking sqlite3
or sqlcipher, libsql backend is added, guarded by a new opt-in
feature - bundled-libsql.
The call is idempotent and for best user experience, it's just injected
into connection initialization.
Modeled after open_with_flags_and_vfs, now users are also able
to open a connection with custom WAl methods via open_with_flags_and_wal.

Technically these parameters should be moved to a map or a config struct
to avoid an exponential explosion, but for now, open_with_flags_and_vfs_and_wal
is simply not available.
The dependency is only used with `libsql-wasm` backend.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.