Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

I used to compile it in windows but still an error presenting #1809

Open
mablue opened this issue Nov 21, 2024 · 3 comments
Open

I used to compile it in windows but still an error presenting #1809

mablue opened this issue Nov 21, 2024 · 3 comments

Comments

@mablue
Copy link

mablue commented Nov 21, 2024

mab@DESKTOP-1SOLNEE MINGW64 ~
$ cd ~/tmp/icefall/egs/ljspeech/TTS/
mab@DESKTOP-1SOLNEE MINGW64 ~/tmp/icefall/egs/ljspeech/TTS (master)
$ ./prepare.sh
2024-11-21 18:19:47 (prepare.sh:25:main) dl_dir: /c/Users/mab/tmp/icefall/egs/ljspeech/TTS/download
2024-11-21 18:19:47 (prepare.sh:28:main) Stage -1: build monotonic_align lib
running build_ext
building 'core' extension
error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/

I installed all microsoft visual studio tools
but still I have errors

mab@DESKTOP-1SOLNEE MINGW64 ~/tmp/icefall/egs/ljspeech/TTS (master)
$ ./prepare.sh
2024-11-21 18:43:06 (prepare.sh:25:main) dl_dir: /c/Users/mab/tmp/icefall/egs/ljspeech/TTS/download
2024-11-21 18:43:06 (prepare.sh:28:main) Stage -1: build monotonic_align lib
running build_ext
building 'core' extension
creating build
creating build\temp.win-amd64-cpython-311
creating build\temp.win-amd64-cpython-311\Release
"C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.42.34433\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD "-IC:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2544.0_x64__qbz5n2kfra8p0\include" "-IC:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2544.0_x64__qbz5n2kfra8p0\Include" -IC:\Users\mab\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\numpy\_core\include "-IC:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.42.34433\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.42.34433\ATLMFC\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" /Tccore.c /Fobuild\temp.win-amd64-cpython-311\Release\core.obj
core.c
core.c(17705): warning C4244: '=': conversion from 'Py_ssize_t' to 'int', possible loss of data
creating C:\Users\mab\tmp\icefall\egs\ljspeech\TTS\vits\monotonic_align\build\lib.win-amd64-cpython-311
"C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.42.34433\bin\HostX86\x64\link.exe" /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO "/LIBPATH:C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2544.0_x64__qbz5n2kfra8p0\libs" "/LIBPATH:C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2544.0_x64__qbz5n2kfra8p0" "/LIBPATH:C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2544.0_x64__qbz5n2kfra8p0\PCbuild\amd64" "/LIBPATH:C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.42.34433\ATLMFC\lib\x64" "/LIBPATH:C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.42.34433\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.22621.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64" /EXPORT:PyInit_core build\temp.win-amd64-cpython-311\Release\core.obj /OUT:build\lib.win-amd64-cpython-311\core.cp311-win_amd64.pyd /IMPLIB:build\temp.win-amd64-cpython-311\Release\core.cp311-win_amd64.lib
   Creating library build\temp.win-amd64-cpython-311\Release\core.cp311-win_amd64.lib and object build\temp.win-amd64-cpython-311\Release\core.cp311-win_amd64.exp
Generating code
Finished generating code
copying build\lib.win-amd64-cpython-311\core.cp311-win_amd64.pyd ->
~/tmp/icefall/egs/ljspeech/TTS/matcha/monotonic_align ~/tmp/icefall/egs/ljspeech/TTS
C:\Users\mab\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\Cython\Compiler\Main.py:381: FutureWarning: Cython directive 'language_level' not set, using '3str' for now (Py3). This has changed from earlier releases! File: C:\Users\mab\tmp\icefall\egs\ljspeech\TTS\matcha\monotonic_align\core.pyx
  tree = Parsing.p_module(s, pxd, full_module_name)
performance hint: core.pyx:13:5: Exception check on 'maximum_path_each' will always require the GIL to be acquired.
Possible solutions:
        1. Declare 'maximum_path_each' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
        2. Use an 'int' return type on 'maximum_path_each' to allow an error code to be returned.
performance hint: core.pyx:44:6: Exception check on 'maximum_path_c' will always require the GIL to be acquired.
Possible solutions:
        1. Declare 'maximum_path_c' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
        2. Use an 'int' return type on 'maximum_path_c' to allow an error code to be returned.
performance hint: core.pyx:49:21: Exception check after calling 'maximum_path_each' will always require the GIL to be acquired.
Possible solutions:
        1. Declare 'maximum_path_each' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
        2. Use an 'int' return type on 'maximum_path_each' to allow an error code to be returned.
Compiling core.pyx because it changed.
[1/1] Cythonizing core.pyx
running build
running build_ext
building 'matcha.monotonic_align.core' extension
creating build
creating build\temp.win-amd64-cpython-311
creating build\temp.win-amd64-cpython-311\Release
"C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.42.34433\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Users\mab\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\numpy\_core\include "-IC:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2544.0_x64__qbz5n2kfra8p0\include" "-IC:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2544.0_x64__qbz5n2kfra8p0\Include" "-IC:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.42.34433\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.42.34433\ATLMFC\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" /Tccore.c /Fobuild\temp.win-amd64-cpython-311\Release\core.obj
core.c
C:\Users\mab\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\numpy\_core\include\numpy\npy_1_7_deprecated_api.h(14) : Warning Msg: Using deprecated NumPy API, disable it with #define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
core.c(19968): warning C4244: '=': conversion from 'Py_ssize_t' to 'int', possible loss of data
creating C:\Users\mab\tmp\icefall\egs\ljspeech\TTS\matcha\monotonic_align\build\lib.win-amd64-cpython-311
creating C:\Users\mab\tmp\icefall\egs\ljspeech\TTS\matcha\monotonic_align\build\lib.win-amd64-cpython-311\matcha
creating C:\Users\mab\tmp\icefall\egs\ljspeech\TTS\matcha\monotonic_align\build\lib.win-amd64-cpython-311\matcha\monotonic_align
"C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.42.34433\bin\HostX86\x64\link.exe" /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO "/LIBPATH:C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2544.0_x64__qbz5n2kfra8p0\libs" "/LIBPATH:C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2544.0_x64__qbz5n2kfra8p0" "/LIBPATH:C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2544.0_x64__qbz5n2kfra8p0\PCbuild\amd64" "/LIBPATH:C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.42.34433\ATLMFC\lib\x64" "/LIBPATH:C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.42.34433\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.22621.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64" /EXPORT:PyInit_core build\temp.win-amd64-cpython-311\Release\core.obj /OUT:build\lib.win-amd64-cpython-311\matcha\monotonic_align\core.cp311-win_amd64.pyd /IMPLIB:build\temp.win-amd64-cpython-311\Release\core.cp311-win_amd64.lib
   Creating library build\temp.win-amd64-cpython-311\Release\core.cp311-win_amd64.lib and object build\temp.win-amd64-cpython-311\Release\core.cp311-win_amd64.exp
Generating code
Finished generating code
mv: cannot stat 'build/lib.*/matcha/monotonic_align/core.*.so': No such file or directory
@csukuangfj
Copy link
Collaborator

The code assumes you are using Linux. Please change the code for Windows.

@mablue
Copy link
Author

mablue commented Nov 22, 2024

opss!! its hard. I will use some chatgpt

@mablue
Copy link
Author

mablue commented Nov 22, 2024

prepare.ps1 content generated by chatgpt

# Set the environment variable for protocol buffers to use Python
$env:PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION = "python"

# Set strict error handling (equivalent to 'set -eou pipefail')
$ErrorActionPreference = "Stop"

# Initialize stage and stop_stage variables
$stage = -1
$stop_stage = 100

# Define download directory
$dl_dir = Get-Location

# Include the parse_options script if it exists (adjust path as necessary)
. "../../../icefall/shared/parse_options.sh" 

# Create the data directory if it doesn't exist
if (-not (Test-Path "data")) {
    New-Item -ItemType Directory -Force -Path "data"
}

# Logging function
function Log($message) {
    $scriptName = [System.IO.Path]::GetFileName($MyInvocation.MyCommand.Path)
    $time = (Get-Date).ToString("yyyy-MM-dd HH:mm:ss")
    Write-Host "$time ($scriptName) $message"
}

Log "dl_dir: $dl_dir"

# Stage -1: Build monotonic_align lib for vits
if ($stage -le -1 -and $stop_stage -ge -1) {
    Log "Stage -1: build monotonic_align lib"
    if (-not (Test-Path "vits/monotonic_align/build")) {
        Set-Location "vits/monotonic_align"
        python3 setup.py build_ext --inplace
        Set-Location $dl_dir
    } else {
        Log "monotonic_align lib for vits already built"
    }

    if (-not (Test-Path "./matcha/monotonic_align/core.cpython-38-x86_64-linux-gnu.so")) {
        Set-Location "matcha/monotonic_align"
        python3 setup.py build
        
        # Use Move-Item for PowerShell to move files
        Move-Item -Path "build/lib.*/matcha/monotonic_align/core.*.so" -Destination .
        
        # Clean up build files
        Remove-Item -Recurse -Force "build"
        Remove-Item -Force "core.c"
        
        # List files to verify
        Get-ChildItem -File | Format-List
        Set-Location $dl_dir
    } else {
        Log "monotonic_align lib for matcha-tts already built"
    }
}

# Stage 0: Download data (similar to existing script)
if ($stage -le 0 -and $stop_stage -ge 0) {
    Log "Stage 0: Download data"
    if (-not (Test-Path "$dl_dir/LJSpeech-1.1")) {
        lhotse download ljspeech $dl_dir
    }
}

# Stage 1: Prepare LJSpeech manifest
if ($stage -le 1 -and $stop_stage -ge 1) {
    Log "Stage 1: Prepare LJSpeech manifest"
    if (-not (Test-Path "data/manifests/.ljspeech.done")) {
        lhotse prepare ljspeech "$dl_dir/LJSpeech-1.1" "data/manifests"
        New-Item -ItemType File -Force -Path "data/manifests/.ljspeech.done"
    }
}

# Stage 2: Compute spectrogram for LJSpeech
if ($stage -le 2 -and $stop_stage -ge 2) {
    Log "Stage 2: Compute spectrogram for LJSpeech"
    if (-not (Test-Path "data/spectrogram/.ljspeech.done")) {
        & "./local/compute_spectrogram_ljspeech.py"
        New-Item -ItemType File -Force -Path "data/spectrogram/.ljspeech.done"
    }

    if (-not (Test-Path "data/spectrogram/.ljspeech-validated.done")) {
        Log "Validating data/spectrogram for LJSpeech"
        python3 ./local/validate_manifest.py "data/spectrogram/ljspeech_cuts_all.jsonl.gz"
        New-Item -ItemType File -Force -Path "data/spectrogram/.ljspeech-validated.done"
    }
}

# Stage 3: Prepare phoneme tokens for LJSpeech
if ($stage -le 3 -and $stop_stage -ge 3) {
    Log "Stage 3: Prepare phoneme tokens for LJSpeech"
    if (-not (Test-Path "data/spectrogram/.ljspeech_with_token.done")) {
        & "./local/prepare_tokens_ljspeech.py" --in-out-dir "./data/spectrogram"
        Move-Item -Path "data/spectrogram/ljspeech_cuts_with_tokens_all.jsonl.gz" -Destination "data/spectrogram/ljspeech_cuts_all.jsonl.gz"
        New-Item -ItemType File -Force -Path "data/spectrogram/.ljspeech_with_token.done"
    }
}

# Stage 4: Split the LJSpeech cuts into train, valid and test sets
if ($stage -le 4 -and $stop_stage -ge 4) {
    Log "Stage 4: Split the LJSpeech cuts into train, valid and test sets"
    if (-not (Test-Path "data/spectrogram/.ljspeech_split.done")) {
        lhotse subset --last 600 "data/spectrogram/ljspeech_cuts_all.jsonl.gz" "data/spectrogram/ljspeech_cuts_validtest.jsonl.gz"
        lhotse subset --first 100 "data/spectrogram/ljspeech_cuts_validtest.jsonl.gz" "data/spectrogram/ljspeech_cuts_valid.jsonl.gz"
        lhotse subset --last 500 "data/spectrogram/ljspeech_cuts_validtest.jsonl.gz" "data/spectrogram/ljspeech_cuts_test.jsonl.gz"

        Remove-Item "data/spectrogram/ljspeech_cuts_validtest.jsonl.gz"

        $n = ((gunzip -c "data/spectrogram/ljspeech_cuts_all.jsonl.gz" | measure -Line).Lines - 600)
        lhotse subset --first $n "data/spectrogram/ljspeech_cuts_all.jsonl.gz" "data/spectrogram/ljspeech_cuts_train.jsonl.gz"
        
        New-Item -ItemType File -Force -Path "data/spectrogram/.ljspeech_split.done"
    }
}

# Stage 5: Generate token file
if ($stage -le 5 -and $stop_stage -ge 5) {
    Log "Stage 5: Generate token file"
    if (-not (Test-Path "data/tokens.txt")) {
        & "./local/prepare_token_file.py" --tokens "data/tokens.txt"
    }
}

# Stage 6: Generate fbank (used by ./matcha)
if ($stage -le 6 -and $stop_stage -ge 6) {
    Log "Stage 6: Generate fbank"
    if (-not (Test-Path "data/fbank/.ljspeech.done")) {
        & "./local/compute_fbank_ljspeech.py"
        New-Item -ItemType File -Force -Path "data/fbank/.ljspeech.done"
    }

    if (-not (Test-Path "data/fbank/.ljspeech-validated.done")) {
        Log "Validating data/fbank for LJSpeech"
        python3 ./local/validate_manifest.py "data/fbank/ljspeech_cuts_all.jsonl.gz"
        New-Item -ItemType File -Force -Path "data/fbank/.ljspeech-validated.done"
    }
}

# Stage 7: Prepare phoneme tokens for LJSpeech
if ($stage -le 7 -and $stop_stage -ge 7) {
    Log "Stage 7: Prepare phoneme tokens for LJSpeech"
    if (-not (Test-Path "data/fbank/.ljspeech_with_token.done")) {
        & "./local/prepare_tokens_ljspeech.py" --in-out-dir "./data/fbank"
        Move-Item -Path "data/fbank/ljspeech_cuts_with_tokens_all.jsonl.gz" -Destination "data/fbank/ljspeech_cuts_all.jsonl.gz"
        New-Item -ItemType File -Force -Path "data/fbank/.ljspeech_with_token.done"
    }
}

# Stage 8: Split the LJSpeech cuts into train, valid and test sets
if ($stage -le 8 -and $stop_stage -ge 8) {
    Log "Stage 8: Split the LJSpeech cuts into train, valid and test sets"
    if (-not (Test-Path "data/fbank/.ljspeech_split.done")) {
        lhotse subset --last 600 "data/fbank/ljspeech_cuts_all.jsonl.gz" "data/fbank/ljspeech_cuts_validtest.jsonl.gz"
        lhotse subset --first 100 "data/fbank/ljspeech_cuts_validtest.jsonl.gz" "data/fbank/ljspeech_cuts_valid.jsonl.gz"
        lhotse subset --last 500 "data/fbank/ljspeech_cuts_validtest.jsonl.gz" "data/fbank/ljspeech_cuts_test.jsonl.gz"

        Remove-Item "data/fbank/ljspeech_cuts_validtest.jsonl.gz"

        $n = ((gunzip -c "data/fbank/ljspeech_cuts_all.jsonl.gz" | measure -Line).Lines - 600)
        lhotse subset --first $n "data/fbank/ljspeech_cuts_all.jsonl.gz" "data/fbank/ljspeech_cuts_train.jsonl.gz"
        
        New-Item -ItemType File -Force -Path "data/fbank/.ljspeech_split.done"
    }
}

# Stage 9: Compute fbank mean and std
if ($stage -le 9 -and $stop_stage -ge 9) {
    Log "Stage 9: Compute fbank mean and std"
    if (-not (Test-Path "data/fbank/cmvn.json")) {
        & "./local/compute_fbank_statistics.py" "data/fbank/ljspeech_cuts_train.jsonl.gz" "data/fbank/cmvn.json"
    }
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants