Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doctest fails to allocate memory #271

Open
ocharles opened this issue Aug 6, 2020 · 10 comments
Open

doctest fails to allocate memory #271

ocharles opened this issue Aug 6, 2020 · 10 comments

Comments

@ocharles
Copy link

ocharles commented Aug 6, 2020

I'm using cabal-doctest and the following whopper of a command line is built:

$ doctest -i -i/home/ollie/work/circuithub/dist-newstyle/build/x86_64-linux/ghc-8.10.1/quoting-1.0.0/noopt/build/autogen -i/home/ollie/work/circuithub/dist-newstyle/build/x86_64-linux/ghc-8.10.1/quoting-1.0.0/noopt/build -i/home/ollie/work/circuithub/quoting/lib -package-env=- -hide-all-packages -no-user-package-db -package-db=/home/ollie/.cabal/store/ghc-8.10.1/package.db -package-db=/home/ollie/work/circuithub/dist-newstyle/packagedb/ghc-8.10.1 -package-db=/home/ollie/work/circuithub/dist-newstyle/build/x86_64-linux/ghc-8.10.1/quoting-1.0.0/noopt/package.conf.inplace -optP-include -optP/home/ollie/work/circuithub/dist-newstyle/build/x86_64-linux/ghc-8.10.1/quoting-1.0.0/noopt/build/autogen/cabal_macros.h -package-id=access-0.1.3-G4qeORg2IZ1GafQapKbA4B -package-id=adjunctions-4.4-3L855q73vEAQrHLCIrS0R -package-id=aeson-1.4.6.0-87MK9QIk6oMATdW4zwzTKX -package-id=base-4.14.0.0 -package-id=cassava-0.5.2.0-JcEwJBq6aYkrpZqgq3KPD -package-id=ch-model-0.1.0.0-inplace -package-id=ch-persistence-1.0-inplace -package-id=ch-servant-auth-1.0-inplace -package-id=circuithub-prelude-0.0.28-inplace -package-id=containers-0.6.2.1 -package-id=distributive-0.6.2-93n9twDTAloDDPaChLm5Hz -package-id=errors-2.3.0-9kOFLev5kiKLBSXK2tckls -package-id=generic-data-0.7.0.0-LEqvUGHM185DAaGF4RnsUI -package-id=logging-effect-1.3.10-8TOaXbUcSLeCxR67bNwCic -package-id=monoid-extras-0.5.1-LWyGpM1ZekmY3RmHcjPE1 -package-id=mtl-2.2.2 -package-id=opaleye-0.6.7004.2-3it6copuwOcL2cQHw7KtjU -package-id=postgresql-simple-0.6.2-BgvijHjeuFiC9yQsfWCDfX -package-id=prettyprinter-1.6.1-JYR0EmFtzfSIqvhE5QZWUw -package-id=rel8-0.1.0.0-8Hbiy9W133v7zpq9u12by7 -package-id=safe-decimal-0.2.0.0-xGHgc1q1sC8OixP6W4A52 -package-id=safe-money-0.9-IxlBNzRdZ728NvqpyR72zs -package-id=scientific-0.3.6.2-2JBA4boE6mq1raXLRr9Zpv -package-id=semialign-1.1-KwcTtoVfizZCsqIQASTcjy -package-id=semigroupoids-5.3.4-HlGah1tiXXN6YRVaskvnCK -package-id=servant-0.17-7nuWjWjgb4pBFYGPLrkjyx -package-id=servant-server-0.17-2TIRB2Fl1AN1AYAEOXqWQ1 -package-id=simple-effects-0.13.0.0-2gEBk4k6Nlt5D2X7ZxntcD -package-id=text-1.2.3.2 -package-id=these-1.1-AB0Dj8DOCcj6tnMeXDKj1G -package-id=time-1.9.3 -package-id=transformers-0.5.6.2 -package-id=units-2.4.1.2-HIwSZJjLV3a5YTFcDqfdo9 -package-id=units-defs-2.1.0.1-1SEMyvkhkcHJUCfGXLZVWc -package-id=unliftio-0.2.12-F6uxEi4gcaQFq5kk7ZQgGp -package-id=validation-1.1-DFjR9MdFrYL9xmDiFUozEi -package-id=witherable-0.3.5-IyPz4Gv0Jt2ibDuZg0u0F -package-id=base-compat-0.10.5-Gk0Pu7SQIuC58a2X5uclr -package-id=doctest-0.17-5Ex9GqTAwmrNlmqPGdCek -package=quoting-1.0.0 CircuitHub.Query.Quoting CircuitHub.Quoting CircuitHub.Quoting.AssemblyPricing.Composite CircuitHub.Quoting.AssemblyPricing.Composite.Features CircuitHub.Quoting.AssemblyPricing.Composite.Offer CircuitHub.Quoting.AssemblyPricing.Picofactory CircuitHub.Quoting.AssemblyPricing.Picofactory.Features CircuitHub.Quoting.AssemblyPricing.Picofactory2 CircuitHub.Quoting.AssemblyPricing.Picofactory2.Features CircuitHub.Quoting.AssemblyPricing.Regression CircuitHub.Quoting.AssemblyPricing.Regression.Features CircuitHub.Quoting.AssemblyPricing.Regression.Offer CircuitHub.Quoting.AssemblyPricing.Worthington CircuitHub.Quoting.Auth.Effect CircuitHub.Quoting.Auth.Effect.PostgreSQL CircuitHub.Quoting.Auth.Named CircuitHub.Quoting.Auth.Owns CircuitHub.Quoting.Auth.Proven CircuitHub.Quoting.Auth.RevisionReadAccess CircuitHub.Quoting.Auth.Some CircuitHub.Quoting.Auth.UserIsAdmin CircuitHub.Quoting.BoardPricing.Composite CircuitHub.Quoting.BoardPricing.Composite.Features CircuitHub.Quoting.BoardPricing.Composite.Offer CircuitHub.Quoting.BoardPricing.Manual CircuitHub.Quoting.BoardPricing.Manual.Features CircuitHub.Quoting.BoardPricing.Manual.Offer CircuitHub.Quoting.BoardPricing.OSHPark CircuitHub.Quoting.BoardPricing.OSHPark.Features CircuitHub.Quoting.BoardPricing.OSHPark.LayerCount CircuitHub.Quoting.BoardPricing.OSHPark.Offer CircuitHub.Quoting.BoardPricing.OSHPark.PricingOption CircuitHub.Quoting.BoardPricing.OSHPark.Shipping CircuitHub.Quoting.BoardPricing.RoyalCircuits CircuitHub.Quoting.BoardPricing.RoyalCircuits.Features CircuitHub.Quoting.BoardPricing.RoyalCircuits.LayerCount CircuitHub.Quoting.BoardPricing.RoyalCircuits.Offer CircuitHub.Quoting.BoardPricing.RoyalCircuits.SolderMaskColor CircuitHub.Quoting.BoardPricing.Storm CircuitHub.Quoting.BoardPricing.Storm.Config CircuitHub.Quoting.BoardPricing.Storm.Features CircuitHub.Quoting.BoardPricing.Storm.LayerCount CircuitHub.Quoting.BoardPricing.Storm.Offer CircuitHub.Quoting.CompositeQuoter CircuitHub.Quoting.CompositeQuoter.ExternalWork CircuitHub.Quoting.CompositeQuoter.Features CircuitHub.Quoting.LeadTime CircuitHub.Quoting.Markup CircuitHub.Quoting.PartPricing CircuitHub.Quoting.PartPricing.BomFeatures CircuitHub.Quoting.PartPricing.Features CircuitHub.Quoting.PartPricing.SourceOffer CircuitHub.Quoting.PartPricing.SourceOffer.Consigned CircuitHub.Quoting.PartPricing.SourceOffer.Purchase CircuitHub.Quoting.PartPricing.SourceSelection CircuitHub.Quoting.PartPricing.SubstituteFeatures CircuitHub.Quoting.Projects.Solution CircuitHub.Quoting.Projects.Solution.BOM CircuitHub.Quoting.Projects.Solution.BOM.Excess CircuitHub.Quoting.Projects.Solution.Flag CircuitHub.Quoting.Quantity.Area CircuitHub.Quoting.Quantity.Density CircuitHub.Quoting.Quantity.Dollar CircuitHub.Quoting.Quantity.Length CircuitHub.Quoting.Quantity.Mass CircuitHub.Quoting.Quantity.NumberOf CircuitHub.Quoting.Quantity.Time CircuitHub.Quoting.Quantity.Volume CircuitHub.Quoting.Queries CircuitHub.Quoting.Queries.Features CircuitHub.Quoting.Queries.Quote CircuitHub.Quoting.Queries.QuoteAlreadyExists CircuitHub.Quoting.Queries.WorthingtonCustomer CircuitHub.Quoting.Quote CircuitHub.Quoting.Quote.CompleteOffer CircuitHub.Quoting.Quote.CompleteOffer.BomItem CircuitHub.Quoting.Quote.CustomerParams CircuitHub.Quoting.Quote.Offer CircuitHub.Quoting.Quote.Offer.BomItem CircuitHub.Quoting.QuoteOwner CircuitHub.Quoting.Server.API CircuitHub.Quoting.Server.API.Response.Quote CircuitHub.Quoting.Server.Handlers CircuitHub.Quoting.Server.ServerState Control.Concurrent.Deduplicate Data.MonotoneMap Data.Ratio.Extra Numeric.Decimal.Extra
doctest: mmap 131072 bytes at (nil): Cannot allocate memory
doctest: Try specifying an address with +RTS -xm<addr> -RTS
doctest: internal error: m32_allocator_init: Failed to map
    (GHC version 8.10.1 for x86_64_unknown_linux)
    Please report this as a GHC bug:  https://www.haskell.org/ghc/reportabug

If I call ghci with these exact same arguments, it works - so I guess this is something specific to doctest.

@quasicomputational
Copy link
Collaborator

Is there a handy way of trying to reproduce this locally? e.g., I assume that it's not just a long command line that's the trigger, otherwise lens would probably be being bitten by that.

@ocharles
Copy link
Author

I'm afraid I don't have a minimal repro. My guess is that it's more the number of packages being loaded, rather than the length of the command line. I can get doctest to run if I run it on a single module, but only for some modules - other single module invocations produce the same error. It was the end of the day when I reported this issue, but I'll dig into it in more detail now.

@ocharles
Copy link
Author

Ok, I weirdly found the problem. Here's the diff that I need to stop doctest crashing...

@@ -57,12 +57,6 @@ import qualified Data.MonotoneMap as MonotoneMap
 import Data.Ratio.Extra
 
 
-{-# ann module "HLint: ignore Evaluate" #-}
-
-
-{-# ann module "HLint: ignore Use let" #-}
-
-
 -- | Burden rates per role in $/hour
 rates :: Durations Rational
 rates =
``

That's right, it seems that module annotations are the source of the problem!

@ocharles
Copy link
Author

Ok, I might be slightly wrong there. The diff above is from a different branch I'm working on, but it doesn't fix master. The good news is I have a minimal repro on our master branch. The following single module is enough to crash doctest. You will need a dependency on units:

{-# language TemplateHaskell #-}
{-# language TypeFamilies #-}

module Data.Units.US.Money where

-- units
import Data.Metrology
import Data.Metrology.TH


declareDimension "Cost"


declareCanonicalUnit "Dollar" [t| Cost |] ( Just "USD" )


type instance DefaultUnitOfDim Cost =
  Dollar


declareDerivedUnit "Cent" [t| Dollar |] ( 1 / 100 ) ( Just "cent" )

@ocharles
Copy link
Author

It seems to be related to actually starting the TH interpreter. If I compile this module with -ddump-splices and inline all TH, I get:

{-# language TypeFamilies #-}

module Data.Units.US.Money where

-- units
import Data.Metrology


data Cost = Cost
instance Dimension Cost


data Dollar = Dollar
instance Unit Dollar where
  type BaseUnit Dollar = Canonical
  type DimOfUnit Dollar = Cost
instance Show Dollar where
  show _ = "USD"


type instance DefaultUnitOfDim Cost =
  Dollar


data Cent = Cent
instance Unit Cent where
  type BaseUnit Cent = Dollar
  conversionRatio _ = 1.0e-2
instance Show Cent where
  show _ = "cent"

Which doctest can happily evaluate.

@ocharles
Copy link
Author

Ok, final update - the ann module stuff also has to removed on the master branch. Perhaps these are two different problems that just manifest in the same failure. But TL;DR:

  • Removing Template Haskell from Data.Units.US.Money and
  • Removing ANN pragmas

Allows doctest to run on this package.

quasicomputational pushed a commit to quasicomputational/doctest-271 that referenced this issue Aug 20, 2020
@quasicomputational
Copy link
Collaborator

I had a go at reproducing this in https://github.com/quasicomputational/doctest-271 but cabal v2-test works fine with GHC 8.10.2.

If you've still got the reproducer handy, can you see if haddock works on it? doctest and haddock are actually quite similar: if haddock works, then it's some kind of doctest-specific bug, but if haddock fails it may ultimately be a GHC issue.

@ocharles
Copy link
Author

ocharles commented Sep 4, 2020

I unfortunately don't have a reproducible case around anymore. I'll close this for now. Hopefully the trail above may help anyone else who runs into this!

@ocharles ocharles closed this as completed Sep 4, 2020
@qnikst
Copy link

qnikst commented Jan 3, 2021

Seems I have this issue on ghc-8.10.3 on Linux (works on macOS). Is there any fix I can try to recipe how to debug it?

It's a huge project with TH and seems that not all the packages with TH fails, so I'm not sure if I can extract the minimal case.

@quasicomputational
Copy link
Collaborator

Try haddocking the project as above, and also manually going in with GHCi and trying some of the doctest examples. If either of those break, it's fundamentally going to be a GHC issue. Otherwise, I'm not sure there's much for it except lopping off leaf modules or removing doctests and seeing if things still break.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants