Performance problems on save

[rdegraaf]

I wrote a script to take a list of balances from Fidelity and reconcile the corresponding accounts in Gnucash. There are a few hundred accounts and a few thousand transactions. Loading the database and Fidelity data, cross-referencing everything, and marking transactions as reconciled takes a couple seconds. Saving the database takes >15 minutes.

I re-ran my script with cProfile; out of 1340 seconds total run-time, it spent 1337 in book.save. That function was called 695 times because I chose to save after processing every account for UEX reasons: a few hundred operations that take 1-2 seconds each is preferable to a single operation that takes >15 minutes because I can print status messages showing that my program is still making progress. I've attached my script (which you probably won't be able to run without my Gnucash database and financial data, which I'm not going to share) and my cProfile output. If you'd like me to run some other profiler or analyis, just let me know what to do.

reconcile.py
reconcile-cProfile.csv

[sdementen]

I am not actively maintaining the project.
If you can have a complete reproducible example (with fake data), I can have a look with a debugger.

[Wm-X]

@sdementen I think this is probably obscure use of piecash, gnc was never intended for trading and reconciling is not a time specific task, looking at the code, I'd guess piecash had little to do with any delays.

But I'm not an expert at python and could be wrong.