Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Alphapeptdeep Integration #3123

Open
wants to merge 19 commits into
base: master
Choose a base branch
from
Open

Alphapeptdeep Integration #3123

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
c4e7c59
Rename PythonInstaller to PythonInstallerLegacyDlg
xgwang-uw Aug 13, 2024
dc612f9
Add PythonInstaller that supports virtual environment
xgwang-uw Aug 16, 2024
d77d35e
Update to add OneOf dependency with PackageReference
xgwang-uw Aug 21, 2024
9d6cab0
Add RuntimeIdentifier win to Skyline.csproj to fix nuget restore error
xgwang-uw Aug 20, 2024
888627c
Fix scaling issue and enable locatable for PythonInstallerDlg form
xgwang-uw Aug 21, 2024
d06c394
Fix AutoScaleDimensions value in PythonInstallerDlg.resx
xgwang-uw Aug 21, 2024
3b12574
Add more verbose logging if "CheckForMissingResources" fails.
Aug 22, 2024
1e5eb21
Fix CodeInspection test by moving PythonInstaller strings from ToolsU…
Aug 22, 2024
f2d97e9
Added OneOf.dll to Product-template.wxs
chambm Aug 22, 2024
7812cb5
Fix PythonInstallerTest by consuming strings from ToolsResources
xgwang-uw Aug 26, 2024
5c487ab
Setup python virtual environment and implement building library with …
xgwang-uw Jul 11, 2024
eade5f9
Remove unused exception vars
xgwang-uw Aug 27, 2024
e0a85e1
Revert "Setup python virtual environment and implement building libra…
xgwang-uw Aug 27, 2024
15930d7
Setup python virtual environment and add empty AlphapeptdeepLibraryBu…
xgwang-uw Aug 27, 2024
dab7240
Add top level logic for BuildLibrary in AlphapeptdeepLibrarybuilder
xgwang-uw Aug 28, 2024
494b457
AlphapeptdeepLibraryBuilder - prepare input file for alphapeptdeep fr…
xgwang-uw Sep 3, 2024
622c47a
AlphapeptdeepLibraryBuilder - prepare settings.yaml file and execute …
xgwang-uw Sep 4, 2024
668f811
AlphapeptdeepLibraryBuilder - transform alphapeptdeep output spectral…
xgwang-uw Sep 5, 2024
2043f45
AlphapeptdeepLibraryBuilder - build transformed alphapeptdeep output …
xgwang-uw Sep 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions pwiz_tools/Skyline/Executables/Installer/Product-template.wxs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -237,6 +237,9 @@
<Component Id="NHibernate.dll">
<File Source="$(var.Skyline.TargetDir)/NHibernate.dll" KeyPath="yes"/>
</Component>
<Component Id="OneOf.dll">
<File Source="$(var.Skyline.TargetDir)/OneOf.dll" KeyPath="yes"/>
</Component>
<Component Id="PrmPasefScheduler.dll">
<File Source="$(var.Skyline.TargetDir)/PrmPasefScheduler.dll" KeyPath="yes"/>
</Component>
Expand Down
405 changes: 405 additions & 0 deletions pwiz_tools/Skyline/Model/AlphaPeptDeep/AlphaPeptDeepLibraryBuilder.cs
View file
Open in desktop

Large diffs are not rendered by default.

13 changes: 13 additions & 0 deletions pwiz_tools/Skyline/Model/ModifiedSequence.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
using System.Globalization;
using System.Linq;
using System.Text;
using log4net.Util;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why log4net?

using pwiz.Common.Collections;
using pwiz.Common.SystemUtil;
using pwiz.Skyline.Model.DocSettings;
Expand Down Expand Up @@ -211,7 +212,19 @@ private string FormatSelf(Func<IEnumerable<Modification>, string> modFormatter)
return result.ToString();
}

// TODO:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like old code that was aimed at a generalized solution for a specific case that requires very few modifications. This code should be reverted.

// 1. create mapper in AlphapeptdeepLibraryBuilder to map from unimode to hardcoded text
// 2. do the formatting in AlphapeptdeepLibraryBuilder instead of here
public string FormatFullNamesByPattern(string pattern)
{
return Format(mods => string.Concat(Enumerable.Select(mods, mod => Bracket(FormatFullNameByPattern(mod, pattern)))));
}

private string FormatFullNameByPattern(Modification mod, string pattern)
{
Assume.IsTrue(!string.IsNullOrEmpty(mod.Name), @"The full modification name [mod.Name] is required but does not exist.");
return mod.Name; // TODO(xgwang): modify
}

public override string ToString()
{
Expand Down
714 changes: 714 additions & 0 deletions pwiz_tools/Skyline/Model/Tools/PythonInstaller.cs
View file
Open in desktop

Large diffs are not rendered by default.

385 changes: 383 additions & 2 deletions pwiz_tools/Skyline/Model/Tools/ToolsResources.designer.cs
View file
Open in desktop

Large diffs are not rendered by default.

120 changes: 120 additions & 0 deletions pwiz_tools/Skyline/Model/Tools/ToolsResources.resx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -244,4 +244,124 @@
<data name="ToolMacros_listArguments_This_tool_requires_an_Active_Replicate_Name_to_run" xml:space="preserve">
<value>This tool requires an Active Replicate Name to run</value>
</data>
<data name="PythonInstaller_GetPackages_Installing_Packages" xml:space="preserve">
<value>Installing Packages</value>
</data>
<data name="PythonInstaller_GetPip_Installing_Pip" xml:space="preserve">
<value>Installing Pip</value>
</data>
<data name="PythonInstaller_GetPython_Installing_Python" xml:space="preserve">
<value>Installing Python</value>
</data>
<data name="PythonInstaller_InstallPackages_Install" xml:space="preserve">
<value>Install</value>
</data>
<data name="PythonInstaller_Download_failed__Check_your_network_connection_or_contact_Skyline_team_for_help_" xml:space="preserve">
<value>Download failed. Check your network connection or contact Skyline team for help.</value>
</data>
<data name="PythonInstaller_GetPythonVirtualEnvironmentTask_Downloading_Python_embeddable_package" xml:space="preserve">
<value>Downloading Python embeddable package</value>
</data>
<data name="PythonInstaller_GetPythonVirtualEnvironmentTask_Failed_to_download_Python_embeddable_package" xml:space="preserve">
<value>Failed to download Python embeddable package</value>
</data>
<data name="PythonInstaller_GetPythonVirtualEnvironmentTask_Unzipping_Python_embeddable_package" xml:space="preserve">
<value>Unzipping Python embeddable package</value>
</data>
<data name="PythonInstaller_GetPythonVirtualEnvironmentTask_Failed_to_unzip_Python_embeddable_package" xml:space="preserve">
<value>Failed to unzip Python embeddable package</value>
</data>
<data name="PythonInstaller_GetPythonVirtualEnvironmentTask_Enabling_search_path_in_Python_embeddable_package" xml:space="preserve">
<value>Enabling search path in Python embeddable package</value>
</data>
<data name="PythonInstaller_GetPythonVirtualEnvironmentTask_Failed_to_enable_search_path_in_Python_embeddable_package" xml:space="preserve">
<value>Failed to enable search path in Python embeddable package</value>
</data>
<data name="PythonInstaller_GetPythonVirtualEnvironmentTask_Downloading_the_get_pip_py_script" xml:space="preserve">
<value>Downloading the get pip py script</value>
</data>
<data name="PythonInstaller_GetPythonVirtualEnvironmentTask_Failed_to_download_the_get_pip_py_script" xml:space="preserve">
<value>Failed to download the get pip py script</value>
</data>
<data name="PythonInstaller_GetPythonVirtualEnvironmentTask_Failed_to_run_the_get_pip_py_script" xml:space="preserve">
<value>Failed to run the get pip py script</value>
</data>
<data name="PythonInstaller_GetPythonVirtualEnvironmentTask_Failed_to_create_virtual_environment__0_" xml:space="preserve">
<value>Failed to create virtual environment {0}</value>
</data>
<data name="PythonInstaller_GetPythonVirtualEnvironmentTask_Installing_Python_packages_in_virtual_environment__0_" xml:space="preserve">
<value>Installing Python packages in virtual environment {0}</value>
</data>
<data name="PythonInstaller_GetPythonVirtualEnvironmentTask_Failed_to_install_Python_packages_in_virtual_environment__0_" xml:space="preserve">
<value>Failed to install Python packages in virtual environment {0}</value>
</data>
<data name="PythonInstaller_Failed_to_execute_command____0__" xml:space="preserve">
<value>Failed to execute command: [{0}]</value>
</data>
<data name="PythonInstaller__0__Running_command____1____2__" xml:space="preserve">
<value>{0} Running command: [{1}] {2} </value>
</data>
<data name="PythonInstaller_PipInstall__0__This_sometimes_could_take_3_5_minutes__Please_be_patient___1__" xml:space="preserve">
<value>{0} This sometimes could take 3-5 minutes. Please be patient. {1} </value>
</data>
<data name="PythonInstallerUnsupportedTaskException_Task_with_action_type__0__is_not_supported_by_PythonInstaller_yet" xml:space="preserve">
<value>Task with action type {0} is not supported by PythonInstaller yet</value>
</data>
<data name="PythonInstallerUnsupportedTaskNameException_Task_with_task_name__0__is_not_supported_by_PythonInstaller_yet" xml:space="preserve">
<value>Task with task name {0} is not supported by PythonInstaller yet</value>
</data>
<data name="PythonInstaller_GetPythonTask_Downloading_Python_embeddable_package" xml:space="preserve">
<value>Downloading Python embeddable package</value>
</data>
<data name="PythonInstaller_GetPythonTask_Failed_to_download_Python_embeddable_package" xml:space="preserve">
<value>Failed to download Python embeddable package</value>
</data>
<data name="PythonInstaller_GetPythonTask_Unzipping_Python_embeddable_package" xml:space="preserve">
<value>Unzipping Python embeddable package</value>
</data>
<data name="PythonInstaller_GetPythonTask_Failed_to_unzip_Python_embeddable_package" xml:space="preserve">
<value>Failed to unzip Python embeddable package</value>
</data>
<data name="PythonInstaller_GetPythonTask_Enabling_search_path_in_Python_embeddable_package" xml:space="preserve">
<value>Enabling search path in Python embeddable package</value>
</data>
<data name="PythonInstaller_GetPythonTask_Failed_to_enable_search_path_in_Python_embeddable_package" xml:space="preserve">
<value>Failed to enable search path in Python embeddable package</value>
</data>
<data name="PythonInstaller_GetPythonTask_Downloading_the_get_pip_py_script" xml:space="preserve">
<value>Downloading the get pip py script</value>
</data>
<data name="PythonInstaller_GetPythonTask_Failed_to_download_the_get_pip_py_script" xml:space="preserve">
<value>Failed to download the get pip py script</value>
</data>
<data name="PythonInstaller_GetPythonTask_Running_the_get_pip_py_script" xml:space="preserve">
<value>Running the get pip py script</value>
</data>
<data name="PythonInstaller_GetPythonTask_Failed_to_run_the_get_pip_py_script" xml:space="preserve">
<value>Failed to run the get pip py script</value>
</data>
<data name="PythonInstaller_GetPythonTask_Running_pip_install__0_" xml:space="preserve">
<value>Running pip install {0}</value>
</data>
<data name="PythonInstaller_GetPythonTask_Failed_to_run_pip_install__0_" xml:space="preserve">
<value>Failed to run pip install {0}</value>
</data>
<data name="PythonInstaller_GetPythonTask_Creating_virtual_environment__0_" xml:space="preserve">
<value>Creating virtual environment {0}</value>
</data>
<data name="PythonInstaller_GetPythonTask_Failed_to_create_virtual_environment__0_" xml:space="preserve">
<value>Failed to create virtual environment {0}</value>
</data>
<data name="PythonInstaller_GetPythonTask_Installing_Python_packages_in_virtual_environment__0_" xml:space="preserve">
<value>Installing Python packages in virtual environment {0}</value>
</data>
<data name="PythonInstaller_GetPythonTask_Failed_to_install_Python_packages_in_virtual_environment__0_" xml:space="preserve">
<value>Failed to install Python packages in virtual environment {0}</value>
</data>
<data name="PythonInstaller_EnableSearchPathInPythonEmbeddablePackage_Found_0_or_more_than_one_files_with__pth_extension__this_is_unexpected" xml:space="preserve">
<value>Found 0 or more than one files with _pth extension, this is unexpected</value>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe make the second clause, "expecting one and only one." to clarify. It took me a second to get this.

</data>
<data name="PythonInstallerTaskValidator_ValidatePipInstallPackages_Failed_to_parse_package_name_and_version_from_entry___0__" xml:space="preserve">
<value>Failed to parse package name and version from entry [{0}]</value>
</data>
</root>
23 changes: 9 additions & 14 deletions pwiz_tools/Skyline/SettingsUI/BuildLibraryDlg.Designer.cs
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

45 changes: 41 additions & 4 deletions pwiz_tools/Skyline/SettingsUI/BuildLibraryDlg.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
*/
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Windows.Forms;
Expand All @@ -28,14 +29,17 @@
using pwiz.Skyline.Controls;
using pwiz.Skyline.FileUI.PeptideSearch;
using pwiz.Skyline.Model;
using pwiz.Skyline.Model.AlphaPeptDeep;
using pwiz.Skyline.Model.DocSettings;
using pwiz.Skyline.Model.Find;
using pwiz.Skyline.Model.Irt;
using pwiz.Skyline.Model.Lib;
using pwiz.Skyline.Model.Koina;
using pwiz.Skyline.Properties;
using pwiz.Skyline.ToolsUI;
using pwiz.Skyline.Util;
using pwiz.Skyline.Util.Extensions;
using pwiz.Skyline.Model.Tools;

namespace pwiz.Skyline.SettingsUI
{
Expand Down Expand Up @@ -84,13 +88,20 @@ public class PropertiesPage : IFormView { }
public class FilesPage : IFormView { }
public class LearningPage : IFormView { }

private const string PYTHON = @"Python";
private const string ALPHAPEPTDEEP_PYTHON_VERSION = @"3.12.4";
private const string ALPHAPEPTDEEP = @"alphapeptdeep";


private static readonly IFormView[] TAB_PAGES =
{
new PropertiesPage(), new FilesPage(), new LearningPage(),
};

private bool IsAlphaEnabled => false; // TODO: Implement and enable
private bool IsAlphaEnabled => true;
private bool IsCarafeEnabled => false; // TODO: Implement and enable
private string AlphapeptdeepPythonVirtualEnvironmentDir =>
PythonInstallerUtil.GetPythonVirtualEnvironmentScriptsDir(ALPHAPEPTDEEP_PYTHON_VERSION, ALPHAPEPTDEEP);

public enum DataSourcePages { files, alpha, carafe, koina }
public enum LearningOptions { none, libraries, document }
Expand Down Expand Up @@ -258,9 +269,9 @@ private bool ValidateBuilder(bool validateInputFiles)
}
else if (radioAlphaSource.Checked)
{
// TODO: Replace with working AlphaPeptDeep implementation
if (!CreateKoinaBuilder(name, outputPath))
if (!SetupPythonEnvironmentForAlpha())
return false;
Builder = new AlphapeptdeepLibraryBuilder(name, outputPath, AlphapeptdeepPythonVirtualEnvironmentDir, _documentUiContainer.DocumentUI);
}
else if (radioCarafeSource.Checked)
{
Expand All @@ -286,7 +297,7 @@ private bool ValidateBuilder(bool validateInputFiles)
// TODO: Probably need to validate that all the libraries can be loaded into memory with progress UI
}

// TODO: Create CarafeLibraryBuilder class with everything necessary to build a library
// TODO: Create AlphapeptdeepLibraryBuilder class with everything necessary to build a library
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like the original text is still appropriate.

if (!CreateKoinaBuilder(name, outputPath))
return false;
}
Expand Down Expand Up @@ -374,6 +385,32 @@ private bool CreateKoinaBuilder(string name, string outputPath, int nce = 27)
return true;
}

private bool SetupPythonEnvironmentForAlpha()
{
var programPathContainer = new ProgramPathContainer(PYTHON, ALPHAPEPTDEEP_PYTHON_VERSION);
var packages = new List<PythonPackage>()
{
new PythonPackage {Name = @"peptdeep", Version = null },
// We manually set numpy to the latest version before 2.0 because of a backward incompatibility issue
// See details for tracking issue in AlphaPeptDeep repo: https://github.com/MannLabs/alphapeptdeep/issues/190
// TODO: delete the following line after the issue above is resolved
new PythonPackage {Name = @"numpy", Version = @"1.26.4" }
};
var pythonInstaller = new PythonInstaller(programPathContainer, packages, new TextBoxStreamWriterHelper(),
new PythonInstallerTaskValidator(), ALPHAPEPTDEEP);
if (pythonInstaller.IsPythonVirtualEnvironmentReady())
{
return true;
}

using var dlg = new PythonInstallerDlg(pythonInstaller);
if (dlg.ShowDialog(this) == DialogResult.Cancel)
{
return false;
}
return true;
}

private void textName_TextChanged(object sender, EventArgs e)
{
string name = textName.Text;
Expand Down
Loading