Issue installing pysqlite3 on Windows

[Poiuy7312]

When using the command poetry install
fatal error C1083: Cannot open include file: 'sqlite3.h': No such file or directory
ERROR: Failed building wheel for pysqlite3
Failed to build pysqlite3
ERROR: Could not build wheels for pysqlite3, which is required to install pyproject.toml-based projects

When using pip install pysqlite3

Collecting pysqlite3
Using cached pysqlite3-0.5.1.tar.gz (40 kB)
Preparing metadata (setup.py) ... done
Building wheels for collected packages: pysqlite3
Building wheel for pysqlite3 (setup.py) ... error
error: subprocess-exited-with-error

I've tried looking for solutions however most of them aren't for Windows

• Ones that I've tried are updating my build tools
• updating python from 3.11.1 to 3.11.5
• Tried installing sqlite3 from the actual website to get the files it says are missing

[simojo]

@Poiuy7312 what package manager do you use?

[Poiuy7312]

Its when using poetry and pip

…

On Wed, Aug 30, 2023 at 9:50 AM Simon Jones ***@***.***> wrote: @Poiuy7312 <https://github.com/Poiuy7312> what package manager do you use? — Reply to this email directly, view it on GitHub <#38 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A2ZWPMDSBJCHDSDJEUETFOLXX5ALJANCNFSM6AAAAAA4EQOQHU> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[gkapfham]

Hi @Poiuy7312, can you please share more details about:

• What commands you typed
• What operating system you are using
• What other error messages you are seeing on your screen
• What steps you have already taken to resolve this issue

It is worth noting that the error you are documenting is a common one for people who are using Windows to install programs that use a SQLite 3 database. Here is an example:

coleifer/sqlite-vtfunc#10

Although it is only a guess, I predict that this problem is appearing when you try to install the Datasette dependency for chasten. Is that the case? If so, please make sure that you check the detailed documentation for Datasette that is available from its web site:

https://datasette.io/

[Poiuy7312]

Running Chasten on Windows with out pysqlite3

Configure

Running the following command should get a similar output
I already ran configure before which is why --force was used.

chasten configure create --force

💫 chasten: Analyze the AST of Python Source Code
🔗 GitHub: https://github.com/gkapfham/chasten
✨ Created configuration directory and file(s) in C:\Users\Preston\AppData\Local\ChastenedTeam\chasten

Analyze

After That Running the analyze command for chasten outputted this

chasten analyze lazytracker --config C:\Users\Preston\AppData\Local\ChastenedTeam\chasten --search-path subject-data\lazytracker\ --save-directory subject-data\lazytracker --save

💫 chasten: Analyze the AST of Python Source Code
🔗 GitHub: https://github.com/gkapfham/chasten

✨ Configuration directory: C:\Users\Preston\AppData\Local\ChastenedTeam\chasten

✨ Validated C:\Users\Preston\AppData\Local\ChastenedTeam\chasten/config.yml? Yes
✨ Validated C:\Users\Preston\AppData\Local\ChastenedTeam\chasten/checks.yml? Yes

✨ Analyzing Python source code in: C:\Users\Preston\CMPSC-203\ChastenProject\chasten\subject-data\lazytracker

🎉 Performing 5 check(s):

  ✓ id: 'C001', name: 'class-definition', pattern: './/ClassDef', min=1, max=10
    • C:\Users\Preston\CMPSC-203\ChastenProject\chasten\subject-data\lazytracker\lazytracker\lazytracker.py - 1 matches
  ✗ id: 'F001', name: 'all-function-definition', pattern: './/FunctionDef', min=1, max=10
    • C:\Users\Preston\CMPSC-203\ChastenProject\chasten\subject-data\lazytracker\lazytracker\lazytracker.py - 6 matches
    • C:\Users\Preston\CMPSC-203\ChastenProject\chasten\subject-data\lazytracker\lazytracker\tracked.py - 3 matches
    • C:\Users\Preston\CMPSC-203\ChastenProject\chasten\subject-data\lazytracker\tests\test_lazytracker.py - 6 matches
    • C:\Users\Preston\CMPSC-203\ChastenProject\chasten\subject-data\lazytracker\tests\test_tracked.py - 2 matches
  ✗ id: 'F002', name: 'non-test-function-definition', pattern: './/FunctionDef[not(contains(@name, "test_"))]', min=1,
max=10
    • C:\Users\Preston\CMPSC-203\ChastenProject\chasten\subject-data\lazytracker\lazytracker\lazytracker.py - 6 matches
    • C:\Users\Preston\CMPSC-203\ChastenProject\chasten\subject-data\lazytracker\lazytracker\tracked.py - 3 matches
    • C:\Users\Preston\CMPSC-203\ChastenProject\chasten\subject-data\lazytracker\tests\test_lazytracker.py - 2 matches
  ✓ id: 'CL001', name: 'single-nested-if', pattern: './/FunctionDef/body//If', min=1, max=10
    • C:\Users\Preston\CMPSC-203\ChastenProject\chasten\subject-data\lazytracker\lazytracker\lazytracker.py - 1 matches
    • C:\Users\Preston\CMPSC-203\ChastenProject\chasten\subject-data\lazytracker\lazytracker\tracked.py - 7 matches
  ✗ id: 'CL002', name: 'double-nested-if', pattern: './/FunctionDef/body//If[ancestor::If and not(parent::orelse)]',
min=1, max=10

✨ Saved the file 'chasten-results-lazytracker-20230906091042-50b52c21ad9c4c6bbf6d777b9d3b567f.json'

😓 At least one check did not pass.

Integrate

Problems start to arise when running Integrate Im currently unsure if this is a pysqlite3 issue or a broader problem

chasten integrate all-programs C:\Users\Preston\CMPSC-203\ChastenProject\chasten\subject-data\lazytracker\chasten-results-lazytracker-20230905150837-770ab3a3e3f3498dbacca10fabaedb05.json -s subject-data/

💫 chasten: Analyze the AST of Python Source Code
🔗 GitHub: https://github.com/gkapfham/chasten

✨ Combining data file(s) in:

• Directory: C:\Users\Preston\CMPSC-203\ChastenProject\chasten\subject-data\lazytracker
  • File: 'chasten-results-lazytracker-20230905150837-770ab3a3e3f3498dbacca10fabaedb05.json'

✨ Saved the file 'chasten-integrated-results-all-programs-20230906091937-bbff3ed2af874e65b4f74e4faa9a1e55.json'
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ C:\Users\Preston\CMPSC-203\ChastenProject\chasten\chasten\main.py:742 in integrate               │
│                                                                                                  │
│   739 │   │   output.console.print(f"\n:sparkles: Saved the file '{combined_json_file_name}'")   │
│   740 │   # "flatten" (i.e., "un-nest") the now-saved combined JSON file using flatterer         │
│   741 │   # create the SQLite3 database and then configure the database for use in datasett      │
│ ❱ 742 │   combined_flattened_directory = filesystem.write_flattened_csv_and_database(            │
│   743 │   │   combined_json_file_name,                                                           │
│   744 │   │   output_directory,                                                                  │
│   745 │   │   project,                                                                           │
│                                                                                                  │
│ C:\Users\Preston\CMPSC-203\ChastenProject\chasten\chasten\filesystem.py:280 in                   │
│ write_flattened_csv_and_database                                                                 │
│                                                                                                  │
│   277 │   # create a view that combines all of the data                                          │
│   278 │   database.create_chasten_view(database_file_name_str)                                   │
│   279 │   # enable full-text search in the SQLite3 database                                      │
│ ❱ 280 │   database.enable_full_text_search(database_file_name_str)                               │
│   281 │   # return the name of the directory that contains the flattened CSV files               │
│   282 │   return flattened_output_directory_str                                                  │
│   283                                                                                            │
│                                                                                                  │
│ C:\Users\Preston\CMPSC-203\ChastenProject\chasten\chasten\database.py:66 in                      │
│ enable_full_text_search                                                                          │
│                                                                                                  │
│    63 │   │   ]                                                                                  │
│    64 │   )                                                                                      │
│    65 │   # enable full-text search on the sources database table                                │
│ ❱  66 │   database["sources"].enable_fts(                                                        │
│    67 │   │   [                                                                                  │
│    68 │   │   │   "filename",                                                                    │
│    69 │   │   │   "filelines",                                                                   │
│                                                                                                  │
│ C:\Users\Preston\AppData\Local\pypoetry\Cache\virtualenvs\chasten-Ti8Qy4ta-py3.11\Lib\site-packa │
│ ges\sqlite_utils\db.py:2358 in enable_fts                                                        │
│                                                                                                  │
│   2355 │   │   │   self.disable_fts()                                                            │
│   2356 │   │                                                                                     │
│   2357 │   │   self.db.executescript(create_fts_sql)                                             │
│ ❱ 2358 │   │   self.populate_fts(columns)                                                        │
│   2359 │   │                                                                                     │
│   2360 │   │   if create_triggers:                                                               │
│   2361 │   │   │   old_cols = ", ".join("old.[{}]".format(c) for c in columns)                   │
│                                                                                                  │
│ C:\Users\Preston\AppData\Local\pypoetry\Cache\virtualenvs\chasten-Ti8Qy4ta-py3.11\Lib\site-packa │
│ ges\sqlite_utils\db.py:2408 in populate_fts                                                      │
│                                                                                                  │
│   2405 │   │   │   │   table=self.name, columns=", ".join("[{}]".format(c) for c in columns)     │
│   2406 │   │   │   )                                                                             │
│   2407 │   │   )                                                                                 │
│ ❱ 2408 │   │   self.db.executescript(sql)                                                        │
│   2409 │   │   return self                                                                       │
│   2410 │                                                                                         │
│   2411 │   def disable_fts(self) -> "Table":                                                     │
│                                                                                                  │
│ C:\Users\Preston\AppData\Local\pypoetry\Cache\virtualenvs\chasten-Ti8Qy4ta-py3.11\Lib\site-packa │
│ ges\sqlite_utils\db.py:524 in executescript                                                      │
│                                                                                                  │
│    521 │   │   """                                                                               │
│    522 │   │   if self._tracer:                                                                  │
│    523 │   │   │   self._tracer(sql, None)                                                       │
│ ❱  524 │   │   return self.conn.executescript(sql)                                               │
│    525 │                                                                                         │
│    526 │   def table(self, table_name: str, **kwargs) -> Union["Table", "View"]:                 │
│    527 │   │   """                                                                               │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
OperationalError: no such column: filelines

Datasette-serve

Running datasette-serve also doesn't work

chasten datasette-serve C:\Users\Preston\CMPSC-203\ChastenProject\chasten\subject-data\chasten-flattened-csvs-sqlite-db-all-programs-20230905151458-42dbbaac257447cfa602106ce0d5a93d\chasten.db --port 8001

💫 chasten: Analyze the AST of Python Source Code
🔗 GitHub: https://github.com/gkapfham/chasten

✨ Starting a local datasette instance:
   • Database: '...
subject-data\chasten-flattened-csvs-sqlite-db-all-programs-20230905151458-42dbbaac257447cfa602106ce0d5a93d\chasten.db'
   • Metadata: 'None'
   • Port: 8001

✨ Details for datasette startup:
   • Venv: 'C:\Users\Preston\AppData\Local\pypoetry\Cache\virtualenvs\chasten-Ti8Qy4ta-py3.11'
   • Cannot find: 'C:\Users\Preston\AppData\Local\pypoetry\Cache\virtualenvs\chasten-Ti8Qy4ta-py3.11/bin/datasette'

🤷 Was not able to find '{executable_name}'

[boulais01]

This issue is being worked on from this branch: https://github.com/AstuteSource/chasten/tree/pysqlite3-removal-test-branch

[gkapfham]

Hi @boulais01 and @Poiuy7312, can you give us a status update here?

[gkapfham]

Hi @Poiuy7312, the issue that you are pointing out with datasette serve might be related to a separate issue associated with finding an executable in a virtual environment on Windows.

Can you please confirm whether you were running this after using the version that was installed through pipx? (Side note: there is a way to do an editable installation of a project in a GitHub repository with pipx and that can help to test the experience of someone using the project instead of developing the project).

Finally, it is super-important that we make sure that each of these commands works on all operating systems. Right now we only have evidence that some of them work on Windows and some do not. It would be helpful if we could collect data about how well all of these commands work on all three of the major operating systems that we support.

[Poiuy7312]

@gkapfham

Can you please confirm whether you were running this after using the version that was installed through pipx? (Side note: there is a way to do an editable installation of a project in a GitHub repository with pipx and that can help to test the experience of someone using the project instead of developing the project).

I have not used pipx to install as I'm not sure on how I can edit pipx like I can with a poetry install. Can I get an explanation on how to do that.

[gkapfham]

Hi @Poiuy7312, I will try to share more details at a later time. You should also work with the staff software engineer on our team to learn more about how to do an editable installation of a Python project that is managed by poetry.

Ultimately, it is important that we take two perspectives: (1) developers who are going to add features and/or bug fixes and/or documentation to the tool and (2) people who are going to install the tool through the use of either pip or pipx.

[VitalJoseph]

@Poiuy7312 did you ever get the datasette issue resolved?

[Poiuy7312]

@Poiuy7312 did you ever get the datasette issue resolved?

@VitalJoseph Yes I did it's Issues #51 and PR #52