Builtin Persistency

[aarroyoc]

I've seen that SWI-Prolog and Trealla have a persistance directive that allows to save on disk predicates.

To me it seems a very elegant solution for small databases that need to be persisted between restarts (like SQLite in other environments).

• Are there any pitfalls to this approach of persistence in contrast to a more manual way?
• Should it be provided by a library or directly by the system? Maybe if it's provided by the system it can handle crashes and errors better to not corrupt the data
• What options should be available to the developer? Location of the real db file? Allow different predicates to go to different db files?

[triska]

I agree that persistency would be a good feature, especially for database-oriented use cases. In fact, just recently a company asked me about how to implement persistency with Scryer Prolog for their own database use case, so there is definitely also commercial demand for this.

The question I have is: Is there a more fundamental feature that can also be used to implement persistency? Personally, I think a mechanism that allows us to store and quickly load compiled Prolog programs could be a case of a more general mechanism that would also be useful to implement persistency. Suppose I can quickly "dump" a compiled Prolog program in such a way that all the generated instructions, all the generated indices etc. are stored with it, and I can later quickly reload the program from disk without the need to regenerate all indices etc. Would this help? Certainly, it would be useful for other cases too: For instance, it will likely significantly reduce the startup time when loading larger libraries such as library(clpz), since Scryer could ship with such "precompiled" versions of its own libraries.

[infogulch]

How are facts and compiled predicates represented and accessed? Ideally that system could be overridden with one that implements persistence, then everything would be persisted transparently including programs.

[EricGT]

FYI for the OP and others interested.

As one who uses SWI-Prolog with library(persistency) regularly and having also written a good portion of the SWI-Prolog Wiki entry: SWI-Prolog connecting to PostgreSQL via ODBC and also having used SWI-Prolog Quick load files I can tell you that there are various pros and cons to each.

If want your data stored with integrity during processing to survive something like a power outage then I would go with the SQL database.

If you want your generated facts available after halt and then restart and can afford to lose the data with a power loss then I would go with library(persistency).

Quick Load Files (QLF) are nice if you have gigabytes of data and want to load it fast (ref) but this is not meant for interim updates.

Also library(persistency) requires that you also give a type to each value when saving. The type is only used for consistency checking and is not stored with the data. See: persistent/1

Another option that has not been mentioned in this topic but has been noted on the SWI-Prolog forum is to use the low level C interfaces and hook up to an existing NO-SQL database such as Redis for which SWI-Prolog has a client: SWI-Prolog client for redis

Or even lower and hook up with Apache Lucene

HTH

---

EDIT

Forgot to mention: Logica: language of Big Data. Not an exact fit for the topic but in the ball park enough that if you don't know where you are and where you are going then it might be an answer to your problem.

[aarroyoc]

The Redis protocol is also implemented in Logtalk. While it doesn't list Scryer as supported yet, I don't think it will be too difficult to make it available: https://logtalk.org/docs/redis_0.html