Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Long string optimization for string column parsing in JSON reader #13803

Merged
merged 63 commits into from
Sep 20, 2023
Merged

Long string optimization for string column parsing in JSON reader #13803

merged 63 commits into from
Sep 20, 2023

Commits on Aug 2, 2023

  1. warp per string parsing of string columns (unicode)

    @karthikeyann
    karthikeyann committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    91742af View commit details
    Browse the repository at this point in the history

  2. remove dependency of data_casting.cuh in write_json.cu

    @karthikeyann
    karthikeyann committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    51e70e2 View commit details
    Browse the repository at this point in the history

  3. cleanup

    @karthikeyann
    karthikeyann committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    318b4a3 View commit details
    Browse the repository at this point in the history

Commits on Aug 7, 2023

  1. try load balancing with global counter for string index

    @karthikeyann
    karthikeyann committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    363d5ab View commit details
    Browse the repository at this point in the history

  2. fix intra-warp divergence issue with cub::WarpScan stuck

    @karthikeyann
    karthikeyann committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    58e0d6c View commit details
    Browse the repository at this point in the history

  3. remove unnecessary WarpReduce, reduce shmem usage

    @karthikeyann
    karthikeyann committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    c0edf8f View commit details
    Browse the repository at this point in the history

  4. cleanup comments, unused code

    @karthikeyann
    karthikeyann committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    086dfa9 View commit details
    Browse the repository at this point in the history

Commits on Aug 11, 2023

  1. add block per string algorithm

    @karthikeyann
    karthikeyann committed Aug 11, 2023
    Configuration menu
    Copy the full SHA
    0aa2c0e View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2023

  1. cleanup, kernel name

    @karthikeyann
    karthikeyann committed Aug 16, 2023
    Configuration menu
    Copy the full SHA
    57ea056 View commit details
    Browse the repository at this point in the history

Commits on Aug 17, 2023

  1. add BLOCK_SIZE to block kernel

    @karthikeyann
    karthikeyann committed Aug 17, 2023
    Configuration menu
    Copy the full SHA
    7e4cfd2 View commit details
    Browse the repository at this point in the history

  2. clean up, add constants

    @karthikeyann
    karthikeyann committed Aug 17, 2023
    Configuration menu
    Copy the full SHA
    6622460 View commit details
    Browse the repository at this point in the history

  3. add long string json test

    @karthikeyann
    karthikeyann committed Aug 17, 2023
    Configuration menu
    Copy the full SHA
    efe7898 View commit details
    Browse the repository at this point in the history

  4. remove debug prints

    @karthikeyann
    karthikeyann committed Aug 17, 2023
    Configuration menu
    Copy the full SHA
    589e0a3 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'branch-23.10' of github.com:rapidsai/cudf into enh-json… 

    …_string_perf
    @karthikeyann
    karthikeyann committed Aug 17, 2023
    Configuration menu
    Copy the full SHA
    4f8e413 View commit details
    Browse the repository at this point in the history

  6. comment

    @karthikeyann
    karthikeyann committed Aug 17, 2023
    Configuration menu
    Copy the full SHA
    d3dc8cf View commit details
    Browse the repository at this point in the history

  7. style fix, add constants

    @karthikeyann
    karthikeyann committed Aug 17, 2023
    Configuration menu
    Copy the full SHA
    e17589e View commit details
    Browse the repository at this point in the history

Commits on Aug 19, 2023

  1. Merge branch 'branch-23.10' of github.com:rapidsai/cudf into enh-json… 

    …_string_perf
    @karthikeyann
    karthikeyann committed Aug 19, 2023
    Configuration menu
    Copy the full SHA
    21fe6c3 View commit details
    Browse the repository at this point in the history

Commits on Aug 25, 2023

  1. unified kernel for warp and block

    @karthikeyann
    karthikeyann committed Aug 25, 2023
    Configuration menu
    Copy the full SHA
    631528a View commit details
    Browse the repository at this point in the history

  2. remove duplicate block kernel, cleanup names

    @karthikeyann
    karthikeyann committed Aug 25, 2023
    Configuration menu
    Copy the full SHA
    6fe5afa View commit details
    Browse the repository at this point in the history

  3. Merge branch 'branch-23.10' of github.com:rapidsai/cudf into enh-json… 

    …_string_perf
    @karthikeyann
    karthikeyann committed Aug 25, 2023
    Configuration menu
    Copy the full SHA
    3f613c8 View commit details
    Browse the repository at this point in the history

Commits on Aug 27, 2023

  1. address review comments

    @karthikeyann
    karthikeyann committed Aug 27, 2023
    Configuration menu
    Copy the full SHA
    d3a35b1 View commit details
    Browse the repository at this point in the history

Commits on Aug 28, 2023

  1. cleanup infer_data_type signature

    @karthikeyann
    karthikeyann committed Aug 28, 2023
    Configuration menu
    Copy the full SHA
    658d8ba View commit details
    Browse the repository at this point in the history

  2. Rename type_inference.cuh to .cu file

    @karthikeyann
    karthikeyann committed Aug 28, 2023
    Configuration menu
    Copy the full SHA
    3e9a88c View commit details
    Browse the repository at this point in the history

  3. Cleanup parse_data signature

    @karthikeyann
    karthikeyann committed Aug 28, 2023
    Configuration menu
    Copy the full SHA
    1838341 View commit details
    Browse the repository at this point in the history

  4. rename data_casting.cuh to .cu

    @karthikeyann
    karthikeyann committed Aug 28, 2023
    Configuration menu
    Copy the full SHA
    efe9712 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'branch-23.10' of github.com:rapidsai/cudf into enh-json… 

    …_string_perf
    @karthikeyann
    karthikeyann committed Aug 28, 2023
    Configuration menu
    Copy the full SHA
    86c4519 View commit details
    Browse the repository at this point in the history

Commits on Aug 29, 2023

  1. move get_escaped_char to parsing_utils.cuh

    @karthikeyann
    karthikeyann committed Aug 29, 2023
    Configuration menu
    Copy the full SHA
    1d86996 View commit details
    Browse the repository at this point in the history

  2. last backslash errored bug fix 

    add unit tests with unicode to use both kernels
    @karthikeyann
    karthikeyann committed Aug 29, 2023
    Configuration menu
    Copy the full SHA
    e11452f View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2023

  1. Merge branch 'branch-23.10' into enh-json_string_perf

    @vuule
    vuule authored Aug 30, 2023
    Configuration menu
    Copy the full SHA
    21e56e5 View commit details
    Browse the repository at this point in the history

  2. address review comments, update docs

    @karthikeyann
    karthikeyann committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    9426dda View commit details
    Browse the repository at this point in the history

Commits on Aug 31, 2023

  1. add complex test cases

    @karthikeyann
    karthikeyann committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    b481d7c View commit details
    Browse the repository at this point in the history

  2. add is escaping backslash lookback

    @karthikeyann
    karthikeyann committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    a5acda6 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'branch-23.10' of github.com:rapidsai/cudf into enh-json… 

    …_string_perf
    @karthikeyann
    karthikeyann committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    b661d95 View commit details
    Browse the repository at this point in the history

  4. call thread kernel for small size, adjust sizes

    @karthikeyann
    karthikeyann committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    4a3941f View commit details
    Browse the repository at this point in the history

Commits on Sep 1, 2023

  1. address review comments

    @karthikeyann
    karthikeyann committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    d227dad View commit details
    Browse the repository at this point in the history

  2. Merge branch 'branch-23.10' of github.com:rapidsai/cudf into enh-json… 

    …_string_perf
    @karthikeyann
    karthikeyann committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    c9802c0 View commit details
    Browse the repository at this point in the history

Commits on Sep 4, 2023

  1. address review comments, fix 2 data hazards

    @karthikeyann
    karthikeyann committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    bcade0f View commit details
    Browse the repository at this point in the history

  2. Merge branch 'branch-23.10' of github.com:rapidsai/cudf into enh-json… 

    …_string_perf
    @karthikeyann
    karthikeyann committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    431e1ec View commit details
    Browse the repository at this point in the history

  3. update comments

    @karthikeyann
    karthikeyann committed Sep 4, 2023
    Configuration menu
    Copy the full SHA
    4d1e048 View commit details
    Browse the repository at this point in the history

Commits on Sep 5, 2023

  1. using bitfields for state_table, no local mem

    @karthikeyann
    karthikeyann committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    896141d View commit details
    Browse the repository at this point in the history

Commits on Sep 8, 2023

  1. Merge branch 'branch-23.10' into enh-json_string_perf

    @vuule
    vuule authored Sep 8, 2023
    Configuration menu
    Copy the full SHA
    38f8253 View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2023

  1. review comments syncthreads()

    @karthikeyann
    karthikeyann committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    7095aab View commit details
    Browse the repository at this point in the history

  2. address review comments syncthreads()

    @karthikeyann
    karthikeyann committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    0d723e0 View commit details
    Browse the repository at this point in the history

  3. fix consts, zero size column, roundup

    @karthikeyann
    karthikeyann committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    3dce8d9 View commit details
    Browse the repository at this point in the history

  4. optimzie single character write case, also fixes direct unicode bug

    @karthikeyann
    karthikeyann committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    8de7d76 View commit details
    Browse the repository at this point in the history

  5. add unit test JsonReaderTest.ErrorStrings

    @karthikeyann
    karthikeyann committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    cb0e0ba View commit details
    Browse the repository at this point in the history

  6. Merge branch 'enh-json_string_perf' of github.com:karthikeyann/cudf i… 

    …nto enh-json_string_perf
    @karthikeyann
    karthikeyann committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    7f3a534 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'branch-23.10' into enh-json_string_perf

    @karthikeyann
    karthikeyann authored Sep 11, 2023
    Configuration menu
    Copy the full SHA
    7ee241a View commit details
    Browse the repository at this point in the history

  8. Merge branches 'enh-json_string_perf' and 'enh-json_string_perf' of g… 

    …ithub.com:karthikeyann/cudf into enh-json_string_perf
    @karthikeyann
    karthikeyann committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    7460997 View commit details
    Browse the repository at this point in the history

  9. add comment

    @karthikeyann
    karthikeyann committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    4b46027 View commit details
    Browse the repository at this point in the history

  10. update comments

    @karthikeyann
    karthikeyann committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    51390fd View commit details
    Browse the repository at this point in the history

  11. adjust kernel string limits

    @karthikeyann
    karthikeyann committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    d7bb5ac View commit details
    Browse the repository at this point in the history

  12. Merge branch 'branch-23.10' into enh-json_string_perf

    @karthikeyann
    karthikeyann authored Sep 11, 2023
    Configuration menu
    Copy the full SHA
    8007ec3 View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2023

  1. reorg json type test code

    @karthikeyann
    karthikeyann committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    d0c8612 View commit details
    Browse the repository at this point in the history

  2. add error cases for parse_data

    @karthikeyann
    karthikeyann committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    56d7fb6 View commit details
    Browse the repository at this point in the history

  3. address review comments (vuule)

    @karthikeyann
    karthikeyann committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    d088e8e View commit details
    Browse the repository at this point in the history

  4. Merge branch 'branch-23.10' of github.com:rapidsai/cudf into enh-json… 

    …_string_perf
    @karthikeyann
    karthikeyann committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    eabb7a8 View commit details
    Browse the repository at this point in the history

  5. fix review comments, remove nvtx ranges

    @karthikeyann
    karthikeyann committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    79b4f38 View commit details
    Browse the repository at this point in the history

  6. fix unit test cases nullability

    @karthikeyann
    karthikeyann committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    403a374 View commit details
    Browse the repository at this point in the history

Commits on Sep 16, 2023

  1. Merge branch 'branch-23.10' into enh-json_string_perf

    @vuule
    vuule authored Sep 16, 2023
    Configuration menu
    Copy the full SHA
    af334fc View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2023

  1. address review comments, split code for string type

    @karthikeyann
    karthikeyann committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    72d23fb View commit details
    Browse the repository at this point in the history

  2. Merge branch 'branch-23.10' of github.com:rapidsai/cudf into enh-json… 

    …_string_perf
    @karthikeyann
    karthikeyann committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    c8e1f69 View commit details
    Browse the repository at this point in the history

  3. add comments, style fix

    @karthikeyann
    karthikeyann committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    d0a5e23 View commit details
    Browse the repository at this point in the history