diff --git a/dev/frompackage/basic_use/index.html b/dev/frompackage/basic_use/index.html index 252cca6..12cdbe2 100644 --- a/dev/frompackage/basic_use/index.html +++ b/dev/frompackage/basic_use/index.html @@ -2,4 +2,4 @@ Basic Use · PlutoDevMacros.jl

Basic Use

@fromparent import_block
 @frompackage target import_block

The @frompackage macro takes a local Package (derived from the target path), loads it as a submodule of the current Pluto workspace and then process the various import/using statements inside import_block to extract varables/functions from the local Package into the notebook workspace.

Note

@fromparent is simply a convenience synthax that uses the calling notebook file as target. More details on how the target path is processed are given below.

Due to the equivalence in functionality between @frompackage and @fromparent, the rest of the documentation will only refer to @frompackage for convenience.

When changes to the code of the local Package are made, the cell containing the call to @frompackage can be re-executed to reload the most recent version of the module. Thanks to Pluto's reactivity, this will automatically trigger re-execution of all cells that use functionality defined in the local package loaded by the macro.

When testing functionalities of the local package using statements in the notebook cells, this reactivity simplifies the workflow as one does not need to manually re-run tests. At the same time, re-loading the full package module at each statement (similar to what Revise does) would most likely generate a significant overhead.

For this reason the reload of local code is only triggered manually within @frompackage and happens only when manually re-running the cell containing the macro call. This process is simplified by the reload button explained below.

target path

The first argument to @frompackage (target) has to be an AbstractString (or a @raw_str) containing the path (either absolute or relative to the file calling the macro) that points to a local Package (the path can be to any file or subfolder within the Package folder).

The main module of the package identified by the target path will be used as the module to process and load within the calling notebook

import_block

The second argument to the macro is supposed to be either a single using/import statement, or multiple statements wrapped inside a begin...end block.

These statements are used to select which parts of the loaded Package module have to be evaluated and which of its variables have te be imported within the notebook scope. Most of these import statements are only relevant when called within Pluto, so @frompackage simply avoid loading the target Package and deletes these import statements in most cases when called oustide of Pluto. There is a specific type of import statement (relative import) that is relevant and applicable also outside of Pluto, so this kind of statement is maintained in the macro output even outside of Pluto.

The macro respects the differentiation between using and import as in normal Julia, so statements containing using Module without any variable name specifier will import all the exported names of Module.

Reload Button

When called within Pluto, the @frompackage macro also creates a convenient button that can be used to re-execute the cell calling the macro to reload the Package code due to a change in the source files.

This button can also be used to quickly navigate to the position of the cell containing the macro by using Ctrl+Click.
The reload button will change appearance (getting a red border) when the macrocall incur in a runtime error either due to incorrect import statement (like if a relative import is used without a proper target) or due to an error encountered when loading the package code.

Here is a short video showing the reload button. The window on the left has opened the specific_imports1.jl notebook, while the one on the right has the specific_imports2.jl one.

Both are included in the TestPackage using for tests and defined in test/TestPackage/src/TestPackage.jl

+ diff --git a/dev/frompackage/custom_settings/index.html b/dev/frompackage/custom_settings/index.html index d7635ad..6b5757c 100644 --- a/dev/frompackage/custom_settings/index.html +++ b/dev/frompackage/custom_settings/index.html @@ -8,4 +8,4 @@ setting2 = val2 end import * -end

Only assignments are supported inside the @settings block and only primitive values can be used as val, i.e. anything that is parsed as an Expr as macro argument is not a valid val.

These are the supported settings:

SHOULD_PREPEND_LOAD_PATH

Bool value that defaults to false.

This specifies whether the active environment added to the LOAD_PATH by the macro should go to the end or to the front of the LOAD_PATH. In some cases it can be useful to have the custom environment at the beginning of the LOAD_PATH for the purpose of locating packages.

+end

Only assignments are supported inside the @settings block and only primitive values can be used as val, i.e. anything that is parsed as an Expr as macro argument is not a valid val.

These are the supported settings:

SHOULD_PREPEND_LOAD_PATH

Bool value that defaults to false.

This specifies whether the active environment added to the LOAD_PATH by the macro should go to the end or to the front of the LOAD_PATH. In some cases it can be useful to have the custom environment at the beginning of the LOAD_PATH for the purpose of locating packages.

diff --git a/dev/frompackage/import_statements/index.html b/dev/frompackage/import_statements/index.html index 856da9a..498b8da 100644 --- a/dev/frompackage/import_statements/index.html +++ b/dev/frompackage/import_statements/index.html @@ -5,4 +5,4 @@ end

This modification is necessary when trying to use @frompackage in combination with the Pluto PkgManager, as explained in Issue 10.

These kind of statements (import/using from Dependencies) are also supported both inside and outside Pluto. Outside of Pluto, only direct dependencies are supported (as that is how julia works) which means that the example code above will effectively translate to using BenchmarkTools both inside and outside of Pluto if BenchmarkTools is a direct dependency.

Note

These kind of statements can not be used in combination with the catch-all imported name (*).

Imports from package dependencies are useful when trying to combine @frompackage with the integrated Pluto PkgManager. In this case, is preferable to keep in the Pluto notebook environment just the packages that are not also part of the target/loaded Package environment, and load the eventual packages that are also in the environment of the loaded Package directly from within the @frompackage import_block.

Doing so minimizes the risk of having issues caused by versions collision between dependencies that are shared both by the notebook environment and the loaded Package environment. Combining the use of @frompackage with the Pluto PkgManager is a very experimental feature that comes with significant caveats. Please read the related section at the end of this README

Re-export using names with Catch-All

Prior to version v0.7.3 of PlutoDevMacros, the catch-all import would not allow to automatically import names in the scope of the parent/package module that were added via using statements. See Issue 11 for more details on the reasons.

This meant that to effectively have access to all of the names in the parent/package scope, the statement @fromparent import * would not be sufficient. If for example you had using Base64 at the top of your module, to effectively have access to also the names of the Base64 stdlib, you would have to call the macro as follows:

@fromparent begin
     import *
     using >.Base64
-end

Since v0.8.0 of PlutoDevMacros, the automatic inclusion of names exposed via using statements is the default behavior when doing catch-all imports.

Note

Since this behavior now brings by default many more names into scope, a new functionality in 0.8 is a check that names in the target packages do not clash with existing names in the Pluto module that is calling @frompackage. Once a clash is identified (i.e. two different objects with the same name are defined in both the Pluto module and the target package), the specific clashed name will not be brought into scope by @frompackage and a warning will be issued.

You can see this warning for example in the test notebook at test/TestUsingNames/test_notebook2.jl, where the name :clash_name is explicitly given to different values both in the target and calling modules, and results in the following warning: Warning Example Image

If one wants to revert back to the previous version where only names effectively defined within the target module (or explicitly imported with import OtherPackage: x) would be brought into the Pluto module's scope, it is sufficient to prepend the @exclude_using macro to the catch-all import statement like so:

@fromparent @exclude_using import *

This statement can be used alone or coupled with other valid import statements within a begin ... end block.

+end

Since v0.8.0 of PlutoDevMacros, the automatic inclusion of names exposed via using statements is the default behavior when doing catch-all imports.

Note

Since this behavior now brings by default many more names into scope, a new functionality in 0.8 is a check that names in the target packages do not clash with existing names in the Pluto module that is calling @frompackage. Once a clash is identified (i.e. two different objects with the same name are defined in both the Pluto module and the target package), the specific clashed name will not be brought into scope by @frompackage and a warning will be issued.

You can see this warning for example in the test notebook at test/TestUsingNames/test_notebook2.jl, where the name :clash_name is explicitly given to different values both in the target and calling modules, and results in the following warning: Warning Example Image

If one wants to revert back to the previous version where only names effectively defined within the target module (or explicitly imported with import OtherPackage: x) would be brought into the Pluto module's scope, it is sufficient to prepend the @exclude_using macro to the catch-all import statement like so:

@fromparent @exclude_using import *

This statement can be used alone or coupled with other valid import statements within a begin ... end block.

diff --git a/dev/frompackage/introduction/index.html b/dev/frompackage/introduction/index.html index f6300cb..9466bad 100644 --- a/dev/frompackage/introduction/index.html +++ b/dev/frompackage/introduction/index.html @@ -1,2 +1,2 @@ -Introduction · PlutoDevMacros.jl

Introduction

The main functionality provided by PlutoDevMacros is the possibility of loading local packages within a Pluto notebook without requiring to activate a local environment (and thus deactivate the Pluto package manager).

This functionality is implemented inside the FromPackage submodule of PlutoDevMacros and accessed throught the @fromparent or @frompackage macros.

Note

Both @fromparent and @frompackage are exported by PlutoDevMacros itself without requiring to explicitly use FromPackage.

While the @fromparent macro was initially developed in order to facilitate creating Julia packages using Pluto notebooks as building blocks, in its current implementation it helps a lot with prototyping and testing during local package development, even when creating normal packages not relying on Pluto notebooks as building blocks.

This macro allows in fact to load the module of a local package within a running Pluto notebook and permits to easily reload the local code upon request similar to a Revise-based workflow but with a few notable advantages:

  • Package code can be re-evaluated correctly without requiring a julia restart even when re-defining structs or constants
  • Local code reload, triggered manually via a floating button in the Pluto notebook, automatically triggers execution of all dependent cells, simplifying the process of testing changes of code on specific runtime paths
  • Possibilty of adding packages to the notebook environment which are not dependencies of the local package, very useful for testing plotting or benchmarking of the local package code without having to put the related packages in either the global or package-local environment
  • Support for the package extensions functionality added in julia 1.9, which together with the point on notebook environment above simplify the testing and development of extensions on the local package under development.

More details on the synthax and functionality of these macros is given in the following sections.

+Introduction · PlutoDevMacros.jl

Introduction

The main functionality provided by PlutoDevMacros is the possibility of loading local packages within a Pluto notebook without requiring to activate a local environment (and thus deactivate the Pluto package manager).

This functionality is implemented inside the FromPackage submodule of PlutoDevMacros and accessed throught the @fromparent or @frompackage macros.

Note

Both @fromparent and @frompackage are exported by PlutoDevMacros itself without requiring to explicitly use FromPackage.

While the @fromparent macro was initially developed in order to facilitate creating Julia packages using Pluto notebooks as building blocks, in its current implementation it helps a lot with prototyping and testing during local package development, even when creating normal packages not relying on Pluto notebooks as building blocks.

This macro allows in fact to load the module of a local package within a running Pluto notebook and permits to easily reload the local code upon request similar to a Revise-based workflow but with a few notable advantages:

  • Package code can be re-evaluated correctly without requiring a julia restart even when re-defining structs or constants
  • Local code reload, triggered manually via a floating button in the Pluto notebook, automatically triggers execution of all dependent cells, simplifying the process of testing changes of code on specific runtime paths
  • Possibilty of adding packages to the notebook environment which are not dependencies of the local package, very useful for testing plotting or benchmarking of the local package code without having to put the related packages in either the global or package-local environment
  • Support for the package extensions functionality added in julia 1.9, which together with the point on notebook environment above simplify the testing and development of extensions on the local package under development.

More details on the synthax and functionality of these macros is given in the following sections.

diff --git a/dev/frompackage/package_extensions/index.html b/dev/frompackage/package_extensions/index.html index 4891f1c..57b659c 100644 --- a/dev/frompackage/package_extensions/index.html +++ b/dev/frompackage/package_extensions/index.html @@ -13,4 +13,4 @@ [extensions] PlutoPlotlyExt = "PlutoPlotly" -Magic = "Example"

We can see that this package defines two extensions PlutoPlotlyExt and Magic that depend on the PlutoPlotly and Example packages respectively.

When the TestDirectExtension module is loaded within a Pluto notebook using the @frompackage or @fromparent macro, the module is not associated to a package UUID so julia does not know that the Magic extension has to be loaded whenever the Example package is loaded into the notebook.

To still support loading the extension code in this situation, the macro will eval the Magic module definition within the TestDirectExtension module if the Example package is defined within the notebook environment. This allows prototyping and testing package extensions during their development exploiting the @frompackage macro.

Check the relevant example notebook located at test/TestDirectExtension/test_extension.jl for more clarity.

Indirect Extensions

Indirect extensions are mostly handled correctly by julia directly, but some issues may arise when a package added to the notebook environment triggers the load of an extension of a package loaded as a direct dependency within the import_block.

If the notebook package triggering the extension is loaded after the direct dependency has already been loaded by the @frompackage macro, an extension compilation error is generated because the direct dependency is not in the LOAD_PATH (See the Use with PlutoPkg section)

The way to solve this issue is to simply reload the local code by using the re-executing the cell containing the macro call. This will trigger a call to Base.retry_load_extensions.

+Magic = "Example"

We can see that this package defines two extensions PlutoPlotlyExt and Magic that depend on the PlutoPlotly and Example packages respectively.

When the TestDirectExtension module is loaded within a Pluto notebook using the @frompackage or @fromparent macro, the module is not associated to a package UUID so julia does not know that the Magic extension has to be loaded whenever the Example package is loaded into the notebook.

To still support loading the extension code in this situation, the macro will eval the Magic module definition within the TestDirectExtension module if the Example package is defined within the notebook environment. This allows prototyping and testing package extensions during their development exploiting the @frompackage macro.

Check the relevant example notebook located at test/TestDirectExtension/test_extension.jl for more clarity.

Indirect Extensions

Indirect extensions are mostly handled correctly by julia directly, but some issues may arise when a package added to the notebook environment triggers the load of an extension of a package loaded as a direct dependency within the import_block.

If the notebook package triggering the extension is loaded after the direct dependency has already been loaded by the @frompackage macro, an extension compilation error is generated because the direct dependency is not in the LOAD_PATH (See the Use with PlutoPkg section)

The way to solve this issue is to simply reload the local code by using the re-executing the cell containing the macro call. This will trigger a call to Base.retry_load_extensions.

diff --git a/dev/frompackage/skipping_parts/index.html b/dev/frompackage/skipping_parts/index.html index 111d5b5..5bfebd5 100644 --- a/dev/frompackage/skipping_parts/index.html +++ b/dev/frompackage/skipping_parts/index.html @@ -7,4 +7,4 @@ "22-23" # This skips from line 21 to 22 in the main file, including extrema. "test_macro1.jl:::28-10000" # This skips parts of test_macro1.jl end -end

The output of the notebook is also pasted here for reference:

image

+end

The output of the notebook is also pasted here for reference:

image

diff --git a/dev/frompackage/use_with_plutopkg/index.html b/dev/frompackage/use_with_plutopkg/index.html index d9c37df..6f77a9a 100644 --- a/dev/frompackage/use_with_plutopkg/index.html +++ b/dev/frompackage/use_with_plutopkg/index.html @@ -1,2 +1,2 @@ -Use with PlutoPkg · PlutoDevMacros.jl

Use with PlutoPkg

The macro evaluates the code of the local target package within the Pluto notebook workspace. For this to work, the notebook needs to have access to all the packages that are loaded within the target package.

Normally, this is achieved by either:

  • Adding the dependencies directly to the notebook environment
  • Activating the environment of the local package within the notebook

The first option risk polluting the notebook environment with a lot of packages that are not directly used within the notebook, while the second option deactivate the integrate PlutoPkg which handles package dependencies.

To address these issues, the macro currently adds the target package environment to the LOAD_PATH during package code evaluation in the notebook workspace.

This approach gives the flexibility of loading arbitrary local package code without requiring to modify the notebook environment itself.

Note

For this to work, the environment of the local target package needs to be instantiated. The macro will actually errors if this is not the case.

The macro tries to catch all possible exceptions that are thrown either during macro compilation or during the resulting expression evaluation (using a try catch) to correctly clean LOAD_PATH after the macro is executed.

This approach may cause issues in case the notebook and the package environment share some dependencies at different version. In this case, the one that was loaded first is the actual version used within the notebook (and within the Package module when loaded in the notebook).

The macro adds the local package environment at the second position in the LOAD_PATH (so after the notebook environment). This should minimize the potential issues as the notebook environment is parsed first to find the packages. This does not prevent the case when a package (for example DataFrames) that is only used by the loaded package, is also added to the notebook after the target Package has been loaded. In this case, the version of DataFrames used by the notebook will be the version loaded by Package, and not the one installed in the notebook environment. Upon restarting the notebook, the situation will flip. Since now DataFrames is in the notebook environment, the notebook version will be loaded both in the notebook and in the Package module, potentially causing issues with the PackageCode if it was depending on a different version of DataFrames.

Due to the issues just mentioned, use the macro knowing that it might break if you want to use the Pluto PkgManager without manually adding all depending packages to the notebook environment.

+Use with PlutoPkg · PlutoDevMacros.jl

Use with PlutoPkg

The macro evaluates the code of the local target package within the Pluto notebook workspace. For this to work, the notebook needs to have access to all the packages that are loaded within the target package.

Normally, this is achieved by either:

  • Adding the dependencies directly to the notebook environment
  • Activating the environment of the local package within the notebook

The first option risk polluting the notebook environment with a lot of packages that are not directly used within the notebook, while the second option deactivate the integrate PlutoPkg which handles package dependencies.

To address these issues, the macro currently adds the target package environment to the LOAD_PATH during package code evaluation in the notebook workspace.

This approach gives the flexibility of loading arbitrary local package code without requiring to modify the notebook environment itself.

Note

For this to work, the environment of the local target package needs to be instantiated. The macro will actually errors if this is not the case.

The macro tries to catch all possible exceptions that are thrown either during macro compilation or during the resulting expression evaluation (using a try catch) to correctly clean LOAD_PATH after the macro is executed.

This approach may cause issues in case the notebook and the package environment share some dependencies at different version. In this case, the one that was loaded first is the actual version used within the notebook (and within the Package module when loaded in the notebook).

The macro adds the local package environment at the second position in the LOAD_PATH (so after the notebook environment). This should minimize the potential issues as the notebook environment is parsed first to find the packages. This does not prevent the case when a package (for example DataFrames) that is only used by the loaded package, is also added to the notebook after the target Package has been loaded. In this case, the version of DataFrames used by the notebook will be the version loaded by Package, and not the one installed in the notebook environment. Upon restarting the notebook, the situation will flip. Since now DataFrames is in the notebook environment, the notebook version will be loaded both in the notebook and in the Package module, potentially causing issues with the PackageCode if it was depending on a different version of DataFrames.

Due to the issues just mentioned, use the macro knowing that it might break if you want to use the Pluto PkgManager without manually adding all depending packages to the notebook environment.

diff --git a/dev/index.html b/dev/index.html index 0726f9f..6ab0e22 100644 --- a/dev/index.html +++ b/dev/index.html @@ -1,2 +1,2 @@ -Home · PlutoDevMacros.jl
+Home · PlutoDevMacros.jl
diff --git a/dev/other_functions/index.html b/dev/other_functions/index.html index 2306386..c4339ac 100644 --- a/dev/other_functions/index.html +++ b/dev/other_functions/index.html @@ -2,8 +2,8 @@ Other Exports · PlutoDevMacros.jl

Other Functions

This package also exports some additional convenience macros for simplifying package development aided by Pluto notebooks.

Additionally, the non-exported function PlutoDevMacros.hide_this_log can be used for sending javascript code through logs to Pluto and hide the corresponding log from view (without stopping the javascript code to execute)

Utilities Macros

PlutoDevMacros.@addmethodMacro
@addmethod func(args...;kwargs...) = body
 @addmethod function func(args...;kwargs...) 
 	body
-end

This simple macro modifies a function definition expression (only when called from a Pluto notebook) to prepend the name of the module defining the function (here called DefiningModule) to the method definition.

So the code

@addmethod func(args...;kwargs...) = something

is simply translated to

DefiningModule.func(args...;kwargs...) = something

when called from a Pluto notebook, and to:

func(args...;kwargs...) = something

when called outside of Pluto.

This is useful to avoid multiple definition errors inside Pluto but has the caveat that defining a method with @addmethod does not trigger a reactive run of all cells that call the modified function. This also mean that when removing the cell with the @addmethod call, the actual method added to the DefiningModule will not be automatically erased by Pluto and will still accessible until it is not overwritten with another method with the same signature.

This is easy to fix in the case of adding methods to modules loaded with @frompackage/@fromparent as reloading the module is sufficient to remove the hanging method.

See this video for an example:

See also: @frompackage, @fromparent

source
PlutoDevMacros.@only_in_nbMacro
only_in_nb(ex)

Executes the expression ex only if the macro is called from a running Pluto instance and ran directly from the source notebook file.

This is more strict than PlutoHooks.@skip_as_script as including a notebook with @skip_as_script ex from another notebook would still execute ex.
@only_in_nb ex instead only evaluates ex if the calling notebook is the original source notebook file.

See also: @only_out_nb. PlutoHooks.@skip_as_script.

source
PlutoDevMacros.@current_pluto_cell_idMacro
@current_pluto_cell_id()

Returns the cell_id (as a string) of the cell where the macro is called. If not ran from within the pluto notebook containing the call, returns an empty string

source
PlutoDevMacros.@current_pluto_notebook_fileMacro
@current_pluto_notebook_file()

Returns the path of the notebook file of the cell where the macro is called. If not ran from within the pluto notebook containing the call, returns an empty string

source

Utilities Functions

PlutoDevMacros.hide_this_logFunction
hide_this_log(html_content::AbstractString = "")
+end

This simple macro modifies a function definition expression (only when called from a Pluto notebook) to prepend the name of the module defining the function (here called DefiningModule) to the method definition.

So the code

@addmethod func(args...;kwargs...) = something

is simply translated to

DefiningModule.func(args...;kwargs...) = something

when called from a Pluto notebook, and to:

func(args...;kwargs...) = something

when called outside of Pluto.

This is useful to avoid multiple definition errors inside Pluto but has the caveat that defining a method with @addmethod does not trigger a reactive run of all cells that call the modified function. This also mean that when removing the cell with the @addmethod call, the actual method added to the DefiningModule will not be automatically erased by Pluto and will still accessible until it is not overwritten with another method with the same signature.

This is easy to fix in the case of adding methods to modules loaded with @frompackage/@fromparent as reloading the module is sufficient to remove the hanging method.

See this video for an example:

See also: @frompackage, @fromparent

source
PlutoDevMacros.@only_in_nbMacro
only_in_nb(ex)

Executes the expression ex only if the macro is called from a running Pluto instance and ran directly from the source notebook file.

This is more strict than PlutoHooks.@skip_as_script as including a notebook with @skip_as_script ex from another notebook would still execute ex.
@only_in_nb ex instead only evaluates ex if the calling notebook is the original source notebook file.

See also: @only_out_nb. PlutoHooks.@skip_as_script.

source
PlutoDevMacros.@current_pluto_cell_idMacro
@current_pluto_cell_id()

Returns the cell_id (as a string) of the cell where the macro is called. If not ran from within the pluto notebook containing the call, returns an empty string

source
PlutoDevMacros.@current_pluto_notebook_fileMacro
@current_pluto_notebook_file()

Returns the path of the notebook file of the cell where the macro is called. If not ran from within the pluto notebook containing the call, returns an empty string

source

Utilities Functions

PlutoDevMacros.hide_this_logFunction
hide_this_log(html_content::AbstractString = "")
 hide_this_log(html::Docs.HTML)

Simple function that returns a Docs.HTML object which when sent to Pluto logs with e.g. @info (or any other loggin macro), will hide the log from view.

It is mostly intended to send some javascript to the logs for execution but avoid having an empty log box hanging around below the cell. The output of this function contains a script which will hide the specific log that contains it, and if no other logs are present, will also hide the log container in Pluto.

The function also optionally accept some content that will be inserted just before the script for hiding the log. The custom input can be provided as an AbstractString, or directly as an HTML object of type Docs.HTML, in which case it will simply extract the HTML contents as String from the contents field. The provided content will be directly interpreted as HTML, meaning that any script will have to be surrounded by <script> tags.

This function is used inside PlutoDevMacros to create the reload @fromparent button also via logs for redundancy.

Example

Suppose you are inside a Pluto notebook and you want to execute some custom javascript from a cell, but for some reason you don't want to have the javascript to be the final output of your cell. You can exploit logs for this (assuming you didn't disable logging for the cell in question).

The following snippet can be inserted in a cell to send a custom message on the javascript developer console while still having a non-javascript cell output.

julia_output = let
     @info PlutoDevMacros.hide_this_log(html"<script>console.log('message')</script>")
     3
-end

Which will correctly send the message to the console even if the cell output is not the javascript script: hide_this_log example gif

source
+end

Which will correctly send the message to the console even if the cell output is not the javascript script: hide_this_log example gif

source diff --git a/dev/search/index.html b/dev/search/index.html index 1b30f24..db6fd2d 100644 --- a/dev/search/index.html +++ b/dev/search/index.html @@ -1,2 +1,2 @@ -Search · PlutoDevMacros.jl

Loading search...

    +Search · PlutoDevMacros.jl

    Loading search...