Segment fault/Bus error from CreateHomalgMatrixFromSparseString in Singular

[HereAround]

I create a large (56.426 x 48.018) spare matrix over the rationals in Singular. For convenience, the entries are saved as a list in the following .txt file:

entries.txt

Now I issue the following code, to crease the corresponding matrix:

LoadPackage( "RingsForHomalg" );
rationals := HomalgFieldOfRationalsInSingular();
Read( "entries.txt" );
matrix := CreateHomalgMatrixFromSparseString( String( entries ), 56426, 48018, rationals );

On Plesken, this leads to the following error message:

Error, the external CAS Singular (running with PID 406990) returned the following error:
Singular : signal 11 (v: 4122):
current line:>>homalg_variable_5[38059,42739]=1;<<
Segment fault/Bus error occurred at 7f11f1c2ce68 because of 10a82 (r:1579015574)
please inform the authors
trying to restart...

at /opt/gap/gap_pkg/pkg/homalg_project/HomalgToCAS/gap/homalgSendBlocking.gi:812 called from
homalgSendBlocking( [ M, "[", c, r, "]=", s ], "need_command", HOMALG_IO.Pictograms.SetMatElm ); at /opt/gap/gap_pkg/pkg/homalg_project/RingsForHomalg/gap/Singular.gi:2113 called from
SetMatElm( M, Int( a[1] ), Int( a[2] ), a[3], R ); at /opt/gap/gap_pkg/pkg/homalg_project/MatricesForHomalg/gap/HomalgMatrix.gi:2139 called from
f( x ); at /opt/gap/gap-stable-4.11/lib/coll.gi:2189 called from
Perform( s, function ( a )
SetMatElm( M, Int( a[1] ), Int( a[2] ), a[3], R );
return;
end ); at /opt/gap/gap_pkg/pkg/homalg_project/MatricesForHomalg/gap/HomalgMatrix.gi:2139 called from
<function "CreateHomalgMatrixFromSparseString constructor for homalg matrices">( )
called from read-eval loop at Error.gi:14
you can 'quit;' to quit to outer loop, or
you can 'return;' to continue

Any suggestion on how this can be fixed?

[mohamed-barakat]

This is a Singular problem. You can use

gap> homalgIOMode("f");

(before creating the ring) to instruct homalg to dump the communication in a Singular-readable ifle.

[HereAround]

Thanks for the suggestion. I have just tried this on Plesken:

LoadPackage( "RingsForHomalg" );
homalgIOMode("f");
rationals := HomalgFieldOfRationalsInSingular();
Read( "entries.txt" );
matrix := CreateHomalgMatrixFromSparseString( String( entries ), 56426, 48018, rationals );

Unfortunately, this leads to the same error as above.

[mohamed-barakat]

Yes, but you now have the Singular-dump :) So now you can take the dump with the needed file entries.txt and open an issue on Singular/Sources.

[mohamed-barakat]

What is the state of this issue?

[HereAround]

Andreas Steenpass suggested, that this is likely linked to insufficient RAM. Moreover, it seems possible to save RAM by changing the code supplied to Singular by the CASForHomalg-Interface.

Whilst I believe that this is likely to fix the issue, I have unfortunately not yet been able to investigate in more detail for time reasons.

[mohamed-barakat]

Good, then we leave it open until you report on it.

[HereAround]

Let me comment more on the above. I have just tried to run the above code again, and it is working now. Still, it typically requires about 40GB of RAM. Hence, as suggested above, this code will likely fail in case there is fewer RAM available.

• To reduce the memory consumption by Singular, @steenpass suggested the following: "Auf meinem Rechner läuft der Code, den Du in dem Issue auf Github gepostet hast, tatsächlich durch, schreibt aber fast den ganzen Swap voll und braucht insgesamt gut 40 GB. Das kannst Du schon auf die Hälfte reduzieren, wenn Du einfach
  matrix mat[48018][56426];
  anstelle von
  matrix mat[48018][56426] = 0;
  schreibst. Das wird trotzdem mit Nullen initialisiert ". Maybe we can remove the "=0" at the interface to Singular?
• In general, I think it would be good to increase the support for sparse matrices.
• One software (Gauss over finite fields) which recently caught my attention is Spasms (https://github.com/cbouilla/spasm). A first interface to this software is provided via
  • https://github.com/homalg-project/SheafCohomologyOnToricVarieties/tree/master/SpasmInterface
  • https://github.com/homalg-project/SheafCohomologyOnToricVarieties/tree/master/SparseMatrices

Maybe we can discuss, if all of this can be resolved uniformly by a package "SparseMatricesForHomalg", which provides SparseMatrices interfaces to Singular, Magma, Spasms... Or (if not already happened?) we extend the existing MatricesForHomalgPackage by this functionality.

[mohamed-barakat]

• To reduce the memory consumption by Singular, @steenpass suggested the following: "Auf meinem Rechner läuft der Code, den Du in dem Issue auf Github gepostet hast, tatsächlich durch, schreibt aber fast den ganzen Swap voll und braucht insgesamt gut 40 GB. Das kannst Du schon auf die Hälfte reduzieren, wenn Du einfach
  matrix mat[48018][56426];
  anstelle von
  matrix mat[48018][56426] = 0;
  schreibst. Das wird trotzdem mit Nullen initialisiert ". Maybe we can remove the "=0" at the interface to Singular?

This is not as easy as it seems since homalgSendBlocking supports specifying the LHS and the RHS of = but does not support creating an assignment without an =.

[mohamed-barakat]

• In general, I think it would be good to increase the support for sparse matrices.

• One software (Gauss over finite fields) which recently caught my attention is Spasms (https://github.com/cbouilla/spasm). A first interface to this software is provided via

  • https://github.com/homalg-project/SheafCohomologyOnToricVarieties/tree/master/SpasmInterface
  • https://github.com/homalg-project/SheafCohomologyOnToricVarieties/tree/master/SparseMatrices

Maybe we can discuss, if all of this can be resolved uniformly by a package "SparseMatricesForHomalg", which provides SparseMatrices interfaces to Singular, Magma, Spasms... Or (if not already happened?) we extend the existing MatricesForHomalgPackage by this functionality.

Yes, this would be great but should happen in coordination with the new matrixobj interface in GAP. On the long run, I would like to outsource the handling of matrices/sparse-matrices to GAP/Julia/Oscar.

[HereAround]

• To reduce the memory consumption by Singular, @steenpass suggested the following: "Auf meinem Rechner läuft der Code, den Du in dem Issue auf Github gepostet hast, tatsächlich durch, schreibt aber fast den ganzen Swap voll und braucht insgesamt gut 40 GB. Das kannst Du schon auf die Hälfte reduzieren, wenn Du einfach
  matrix mat[48018][56426];
  anstelle von
  matrix mat[48018][56426] = 0;
  schreibst. Das wird trotzdem mit Nullen initialisiert ". Maybe we can remove the "=0" at the interface to Singular?

This is not as easy as it seems since homalgSendBlocking supports specifying the LHS and the RHS of = but does not support creating an assignment without an =.

I suppose a specialized/additional `homalgSendBlocking'-method, which initializes objects with zeros, is too involved (e.g. for architectural reasons)?

[HereAround]

• In general, I think it would be good to increase the support for sparse matrices.

• One software (Gauss over finite fields) which recently caught my attention is Spasms (https://github.com/cbouilla/spasm). A first interface to this software is provided via

  • https://github.com/homalg-project/SheafCohomologyOnToricVarieties/tree/master/SpasmInterface
  • https://github.com/homalg-project/SheafCohomologyOnToricVarieties/tree/master/SparseMatrices

Maybe we can discuss, if all of this can be resolved uniformly by a package "SparseMatricesForHomalg", which provides SparseMatrices interfaces to Singular, Magma, Spasms... Or (if not already happened?) we extend the existing MatricesForHomalgPackage by this functionality.

Yes, this would be great but should happen in coordination with the new matrixobj interface in GAP. On the long run, I would like to outsource the handling of matrices/sparse-matrices to GAP/Julia/Oscar.

I agree. Best to solve this once in one place.

[mohamed-barakat]

I suppose a specialized/additional `homalgSendBlocking'-method, which initializes objects with zeros, is too involved (e.g. for architectural reasons)?

homallgSendBlocking is a global function that does the complex analysis internally, which would be far too complicated for GAP's simplistic method selection. If this a really a pressing serious issue for you I can invest some time and resolve it for you.

[HereAround]

Thank you Mohamed. No, I think the current support for matrices is sufficient (and has been for years). Let us consider this a feature request.