Is python output from not pdf output also cached?

[dflvunoooooo]

If I use a custum preamble and create a pgf or tex file, is that file also cached or is it generate every time? And is the process of that file cached?

[tobiasBora]

I'm not sure to understand the question, do you mean with python or latex? Do you mean if you add to the latex preset add to preamble or if you use \runHereAndAddToPreamble? If so, this does not change the python codes in any way, so it will use the cached value, but it will, as expected, recompile the latex images.

More precisely, in that case, the library will, in memory, generate the content of the .tex file and compute its hash (no way to avoid it but it is fairly quick), also based on the dependencies and the compilation command. The hash determines the name of the file, so then I check if the source file has been created. If not, we create that file. Then we check if a .pdf with the same name exists: if so we don't run any compilation command and we directly run the include command, using the cached pdf. If not, we run the compilation command, using by default shell escape, to generate this file once for all.

If you want to avoid recompiling all latex images if you just change a single macro, look at the commands to forward automatically a macro, it is way more chirurgical when recompiling.

[dflvunoooooo]

I was wondering, because for gnuplot there is an extra option to cache the from gnuplot generated tikz file. I had to do this

add to preset={gnuplot}{
        tikz terminal/.append style={
            cache tikz,	
        }
},

And I wondered if the with python generated file is cached as well.

If I undetstad you correctly it is checked if a pdf file exists. But what about other files. If the python output is a pfg or tex file. Will that be chacked too? Or is the difference here, that in gnuplot -> tikz the tikz file is not saved as a file, whereas in python it is actually saved as a file?

[tobiasBora]

I was wondering, because for gnuplot there is an extra option to cache the from gnuplot generated tikz file. I had to do this

Ah, I understand better now. So if the python generates a pdf, no need to add cache tikz since it is a pdf. If the python generates a .pgf that you \input, then the .pgf will be cached (so python/shell escape is needed only once), but unless you cache tikz pictures by default, the generation of the tikz picture itself will not be cached (so if you have many images, it might take a bit of time). If you also want to cache the tikz pictures, then you basically need to add cache tikz in your preset, or to enable it globally using \cacheTikz (actually, this will not work as they use pgfpicture instead of tikzpicture, see my fix below).

If I undetstad you correctly it is checked if a pdf file exists. But what about other files. If the python output is a pfg or tex file. Will that be chacked too?

Even if other files (e.g. .pgf or -to-input.tex) are created, the library only checks for the existence of the .pdf to know if the compilation succeeded, so that's why we must create a .pdf even if we don't use it. In the default python preset, this file is actually automatically created at the very end (check the source code of the files, you will see), unless an error occurred earlier. That's why you don't need to do it yourself.

Or is the difference here, that in gnuplot -> tikz the tikz file is not saved as a file, whereas in python it is actually saved as a file?

In both cases, a tikz file is created from the python/gnuplot code. What might not be created is the file containing the image (pdf): this second file is only created if cache tikz or \cacheTikz is used. You can test it yourself: just clean the cache by removing robustExternalize, and create a single picture + compile. Then, go to the robustExternalize folder: if you see two .tex files (one should contain python code, one latex code), you cache both the python -> tikz and the tikz -> pdf. If you see a single .tex file, you only cache python -> tikz. You can also open the other files to see what they contain, i might be more instructive than my explanations ;-)

[dflvunoooooo]

I see. Thank you for the detailed answer! So only for tikz it is a special behavior.

Great work you are doing here! :)

[tobiasBora]

Actually, I just realized that I told you something wrong: the pgf backend uses pgfpicture instead of tikzpicture, and I don't cache pgfpicture automatically (also because a normal user rarely uses pgfpicture directly. But it is fairly easy to cache this as well, just type:

\cacheEnvironment{pgfpicture}{tikz, add to preamble={\makeatletter}}

(note that the \makeatletter is needed as pgfpicture relies on this being set to true)

Another solution is to use instead a different \input (this avoids to cache all pgfpictures at the same time, like many we don't want to cache pgfpictures from other packages). For that, change the custom include command to:

custom include command={\evalPlaceholder{\cacheMe[tikz]{\input{__ROBEXT_OUTPUT_PREFIX__.pgf}}}},

(You might also like this example I gave here https://tex.stackexchange.com/a/710138/116348)

[dflvunoooooo]

Ah, great! Maybe I imagined this, but it seems to me it is compiling faster now, with the second compilation. Thanks for clarifying.