Package manager

[aarroyoc]

Most modern development environments have a package manager, useful to install libraries in a convenient way and keeping track of dependencies and updates. Having a good package manager story improves the the productivity of the developers and makes te system itself more attractive.

Some examples of package managers are:

• Cargo (for Rust)
• Gem (for Ruby)
• npm (for JavaScript)
• Pip (for Python)

Even in Prolog this concept is not new as we already have SWI-Prolog packs.

Here are some questions I have:

• Do we really need a package manager? I think yes, but maybe there are some other opinions
• Should we develop a package manager compatible with different Prologs or just Scryer?
• Should we just adapt SWI-Prolog packs?
• Can we use some generic package manager like Nix for example?
• Should Scryer recognize folders with package manager metadata to adjust paths of the modules?

[triska]

Related issue: #202

The ability to quickly install packages would be great! Emacs and Rust are good examples where a good package manager has greatly simplified the installation of packages that cover a wide range of applications, and I think it would be very nice to have that for Scryer Prolog too!

It would be ideal if the innovative implementation methods of Scryer Prolog, most notably its compact internal string representation, could be advertised by packages and even the package manager in such a way that other Prolog systems will also adopt them!

[pmoura]

I'm implementing a package manager for Logtalk that's expected to support all Prolog backends (see LogtalkDotOrg/logtalk3#118).

[aarroyoc]

That is very similar to what I was thinking!

[pmoura]

For running on Scryer Prolog, the shell/1-2 predicates found in most Prolog systems will need to be implemented. Is that something where you can help?

[aarroyoc]

It doesn't seem too complicated. I'm sure there's a cross-platform Rust way to do it. I'll take a look at it as soon as possible

[aarroyoc]

A first version here: #1052

[pmoura]

@aarroyoc With a non-pushed change to Logtalk's os library to use your implementation of the shell/1-2 predicates:

$ scryerlgt 

Logtalk 3.51.0-b04
Copyright (c) 1998-2021 Paulo Moura
...

?- {packs(loader)}.
...
% Initializing/checking packs directory structure...
% Verifying availability of the required shell utilities...
% Loading existing registry and pack specifications...
...
Packs manager ready. For basic help, type: packs::help.
...
   true
;  ...
?- packs::help.
Common queries using the packs tool main predicates:

    List registries:            registries::list.
    Add a new registry:         registries::add(Registry, URL).
    Print registry information: registries::describe(Registry).

    List available packs:       packs::available.
    List installed packs:       packs::installed.

    Print pack information:     packs::describe(Pack).
                                packs::describe(Registry, Pack).

    Install a pack:             packs::install(Pack).
                                packs::install(Registry, Pack).
                                packs::install(Registry, Pack, Version).

    Update a pack:              packs::update(Pack).
    Delete a pack:              packs::delete(Pack).

Consult the Handbook and the tool API documentation for details.
   true.
?- registries::list.
% Registries:
% - 'MagicLogtalkPacks'
% - 'logtalk-packs'
   true.
?- registries::describe('logtalk-packs').
% 
% Registry:    'logtalk-packs'
% Description: Official Logtalk packs registry
% Home:        https://github.com/LogtalkDotOrg/logtalk-packs
% Cloning URL: https://github.com/LogtalkDotOrg/logtalk-packs.git
% Archive URL: https://github.com/LogtalkDotOrg/logtalk-packs/archive/main.zip
% 
   true.
?- packs::available.
% Available packs:
% MagicLogtalkPacks:
% - swidict
% logtalk-packs:
% - lflat
   true.
?- packs::installed.
% Installed packs:
%   'logtalk-packs'::lflat@2:1:0
%   'MagicLogtalkPacks'::swidict@1:1:1
   true.
?- packs::describe(lflat).
% 
% Registry:    'logtalk-packs'
% Pack:        lflat
% Description: L-FLAT - Logtalk Formal Language and Automata Toolkit
% License:     MIT
% Home:        https://github.com/l-flat/lflat
% Versions:
%   2:1:0 (stable; installed)
%     Dependencies: [logtalk>=3:36:0]
%     Portability: all
%   2:0:2 (stable)
%     Dependencies: [logtalk>=3:36:0]
%     Portability: all
% 
   true.
?- packs::pin(lflat).
   true
;  ...
?- packs::describe(lflat).
% 
% Registry:    'logtalk-packs'
% Pack:        lflat
% Description: L-FLAT - Logtalk Formal Language and Automata Toolkit
% License:     MIT
% Home:        https://github.com/l-flat/lflat
% Versions:
%   2:1:0 (stable; installed; pinned)
%     Dependencies: [logtalk>=3:36:0]
%     Portability: all
%   2:0:2 (stable)
%     Dependencies: [logtalk>=3:36:0]
%     Portability: all
% 
   true.
?- packs::delete(lflat).
!     Cannot delete pinned pack: lflat
false.
?- packs::unpin(lflat).
   true.
?- packs::delete(lflat).
% Deleting pack: 'logtalk-packs'::lflat
% Pack deleted
   true.
?- packs::installed.
% Installed packs:
%   'MagicLogtalkPacks'::swidict@1:1:1
   true.
?- packs::install('logtalk-packs', lflat, 2:0:2).
% Installing pack: 'logtalk-packs'::lflat@2:0:2
% Pack installed
   true
;  ...
?- packs::installed.
% Installed packs:
%   'logtalk-packs'::lflat@2:0:2
%   'MagicLogtalkPacks'::swidict@1:1:1
   true.
?- packs::update(lflat).
% Updating pack: 'logtalk-packs'::lflat (2:0:2 to 2:1:0)
% Pack updated
   true
;  ...
?- packs::installed.
% Installed packs:
%   'logtalk-packs'::lflat@2:1:0
%   'MagicLogtalkPacks'::swidict@1:1:1
   true.

[pmoura]

@aarroyoc Do you want to setup your own packs registry with e.g. advent-of-code-2020 and postgresql-prolog for experimentation? See the packs tool documentation for details.

[aarroyoc]

Yes of course. I would like to try it for postgresql-prolog (I don't see the point for the advent repo). I'll try to follow it as soon as possible.

[aarroyoc]

What should be the good way to load the content of the packs? How can I access the installed packs and call a predicate? Should I manually use LOGTALKPACKS env?

[pmoura]

Some Prolog systems copied Quintus Prolog file_search_path/2 dynamic, multifile, hook predicate. For these systems, a solution would be to add a fact to this predicate for each installed Prolog pack. The second argument, the path, is usually expanded and can contain environment variables. No idea if there are any plans for Scryer Prolog to add this mechanism (to be clear, I'm not advocating it as it's handy but problematic and often misused).

For now, in the specific case of Scryer Prolog, using the value of the LOGTALKPACKS environment variable should work. A pack foo from registry reg will be installed in the $LOGTALKPACKS/packs/reg/foo directory.

Related, in Logtalk's os library code for Scryer Prolog, I'm using the following auxiliary predicate to go from a path to an expanded path:

expand_path_chars(Path, ExpandedPathChars) :-
	atom_chars(Path, PathChars),
	{path_segments(PathChars, [Segment| Segments])},
	(	['$'| EnvVar] = Segment ->
		{getenv(EnvVar, Value)},
		{path_segments(PathCharsExpanded, [Value| Segments])}
	;	PathCharsExpanded = PathChars
	),
	(	{path_canonical(PathCharsExpanded, ExpandedPathChars)} ->
		true
	;	PathCharsExpanded = ['/'| _] ->
		ExpandedPathChars = PathCharsExpanded
	;	{working_directory(Current, Current)},
		{append(Current, ['/'| PathCharsExpanded], ExpandedPathChars)}
	).

But path expansion functionality really belongs in a Scryer Prolog own library. It's basic operating-system interoperability functionality that's natively provided by most Prolog systems.

P.S. The development of the packs tool was moved to the master branch.

[pmoura]

Logtalk 3.51.0, released earlier today, includes a beta version of the packs tool. I tested its functionality with the current Scryer Prolog git version and found no issues specific to Scryer.

[aarroyoc]

I've submitted #1063 to add absolute_file_name/2, which is a native option. Also, I was facing a similar problem with that in my code, so it's better to have it implemented