Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Precompile times #241

Open
jpfairbanks opened this issue Jun 8, 2024 · 1 comment
Open

Precompile times #241

jpfairbanks opened this issue Jun 8, 2024 · 1 comment
Assignees

Comments

@jpfairbanks
Copy link
Member

I think we can use the techniques in this post to bring down Decapodes precompile times

https://sciml.ai/news/2022/09/21/compile_time/#starting_the_process_profiling_why_ordinarydiffeq_first_solve_time_was_30_second_compilation

SnoopCompile should also help find type inference failures, which were the cause of the major performance problems in ACSets.jl that were biting Decapodes.

@GeorgeR227
Copy link
Collaborator

I'm running some benchmarks and I think Decapodes by itself precompiles fairly quickly, only 7.2ms. Overall compilation time is a it over 2 seconds. The initial loading time should have also been heavily improved by #216. However I'm still looking at compilation times for our larger functions like gensim.

image
julia> @time_imports using Decapodes
      0.5 ms  Reexport
      0.6 ms  Compat
      0.4 ms  Compat  CompatLinearAlgebraExt
      0.4 ms  StructEquality
      6.3 ms  Preferences
      0.5 ms  PrecompileTools
      0.8 ms  StaticArraysCore
    115.2 ms  StaticArrays
     49.6 ms  MLStyle
      0.6 ms  AlgebraicInterfaces
      0.4 ms  DataValueInterfaces
      0.7 ms  DataAPI
      0.4 ms  IteratorInterfaceExtensions
      0.4 ms  TableTraits
      6.5 ms  Tables
      4.9 ms  StringManipulation
      8.1 ms  Crayons
      0.9 ms  LaTeXStrings
     60.8 ms  PrettyTables
      5.4 ms  MacroTools
      0.6 ms  CompTime
      3.5 ms  StructTypes
               ┌ 0.0 ms Parsers.__init__() 
     28.7 ms  Parsers 41.74% compilation time
     21.7 ms  JSON3
     18.5 ms  DataStructures
      1.5 ms  Combinatorics
      1.0 ms  Permutations
     28.8 ms  ACSets
               ┌ 14.6 ms SuiteSparse_jll.__init__() 
     15.4 ms  SuiteSparse_jll
               ┌ 6.2 ms SparseArrays.CHOLMOD.__init__() 52.06% compilation time
    112.0 ms  SparseArrays 2.87% compilation time
      0.8 ms  Statistics
      0.4 ms  StaticArrays  StaticArraysStatisticsExt
     12.5 ms  FixedPointNumbers
               ┌ 9.5 ms ColorTypes.__init__() 99.89% compilation time (100% recompilation)
     27.4 ms  ColorTypes 34.63% compilation time (100% recompilation)
     25.2 ms  Colors
    153.6 ms  GATlab
      0.8 ms  JLLWrappers
               ┌ 5.4 ms Zlib_jll.__init__() 
      6.1 ms  Zlib_jll
               ┌ 4.5 ms Libiconv_jll.__init__() 
      5.4 ms  Libiconv_jll
               ┌ 2.0 ms XML2_jll.__init__() 
      2.8 ms  XML2_jll
      2.2 ms  LightXML
     14.1 ms  IterTools
      2.1 ms  Measures
               ┌ 0.0 ms Requires.__init__() 
      0.8 ms  Requires
      6.3 ms  JSON
               ┌ 0.0 ms Compose.__init__() 
      7.8 ms  Compose
      0.8 ms  PrettyPrint
      0.7 ms  NameResolution
      2.5 ms  JuliaVariables
      6.9 ms  GeneralizedGenerated
     66.0 ms  Catlab
     10.8 ms  DiagrammaticEquations
     43.6 ms  FillArrays
      2.5 ms  FillArrays  FillArraysSparseArraysExt
      0.8 ms  FillArrays  FillArraysStatisticsExt
    136.0 ms  ArrayLayouts
      1.0 ms  ArrayLayouts  ArrayLayoutsSparseArraysExt
     47.0 ms  LazyArrays
      0.8 ms  LazyArrays  LazyArraysStaticArraysExt
      0.8 ms  ConstructionBase
      0.7 ms  ConstructionBase  ConstructionBaseStaticArraysExt
     16.7 ms  StructArrays
      1.0 ms  StructArrays  StructArraysStaticArraysExt
      0.9 ms  StructArrays  StructArraysSparseArraysExt
      1.9 ms  Extents
      5.1 ms  GeoInterface
               ┌ 134.2 ms EarCut_jll.__init__() 97.00% compilation time (100% recompilation)
    135.2 ms  EarCut_jll 96.30% compilation time (100% recompilation)
     95.8 ms  GeometryBasics
     12.7 ms  DataMigrations
               ┌ 0.0 ms FileIO.__init__() 
     28.2 ms  FileIO
      5.1 ms  MeshIO
     10.3 ms  CombinatorialSpaces
      5.7 ms  Krylov
     12.5 ms  ChainRulesCore
      0.9 ms  StaticArrays  StaticArraysChainRulesCoreExt
      0.8 ms  ChainRulesCore  ChainRulesCoreSparseArraysExt
      0.7 ms  SuiteSparse
      1.5 ms  ArrayInterface
      0.9 ms  ArrayInterface  ArrayInterfaceStaticArraysCoreExt
      0.7 ms  IfElse
     12.3 ms  Static
     29.0 ms  StaticArrayInterface
      1.5 ms  StaticArrayInterface  StaticArrayInterfaceStaticArraysExt
      2.3 ms  Functors
      0.8 ms  PackageExtensionCompat
               ┌ 0.0 ms ComponentArrays.__init__() 
     42.8 ms  ComponentArrays
      0.9 ms  ComponentArrays  ComponentArraysConstructionBaseExt
      4.4 ms  IrrationalConstants
      1.0 ms  DiffRules
      1.2 ms  DiffResults
               ┌ 10.5 ms OpenLibm_jll.__init__() 
     11.4 ms  OpenLibm_jll
      0.9 ms  NaNMath
               ┌ 0.0 ms DocStringExtensions.__init__() 
      1.9 ms  DocStringExtensions
      0.8 ms  LogExpFunctions
      0.8 ms  LogExpFunctions  LogExpFunctionsChainRulesCoreExt
               ┌ 23.1 ms CompilerSupportLibraries_jll.__init__() 12.03% compilation time
     24.0 ms  CompilerSupportLibraries_jll 11.60% compilation time
               ┌ 1.8 ms OpenSpecFun_jll.__init__() 
      2.9 ms  OpenSpecFun_jll
      4.9 ms  SpecialFunctions
      1.0 ms  SpecialFunctions  SpecialFunctionsChainRulesCoreExt
      0.7 ms  CommonSubexpressions
     28.9 ms  ForwardDiff
      1.1 ms  ForwardDiff  ForwardDiffStaticArraysExt
      0.8 ms  Adapt
      0.7 ms  StructArrays  StructArraysAdaptExt
      0.7 ms  ComponentArrays  ComponentArraysAdaptExt
      0.8 ms  Adapt  AdaptStaticArraysExt
      1.6 ms  PreallocationTools
      7.3 ms  Decapodes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants