Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't exclude .git on the slug #470

Open
deis-admin opened this issue Jan 19, 2017 · 13 comments
Open

Don't exclude .git on the slug #470

deis-admin opened this issue Jan 19, 2017 · 13 comments

Comments

@deis-admin
Copy link

From @crigor on June 25, 2015 23:0

Is there a reason the builder still excludes .git? https://github.com/deis/deis/blob/3cc3baf0fe7084bfe11b1dad47dc377879a14a2d/builder/rootfs/usr/local/src/slugbuilder/builder/build.sh#L155

When using a github url as a dependency on an elixir app, the .git dir is needed. The app worked fine on heroku but failed to start on deis.

Copied from original issue: deis/deis#3924

@deis-admin
Copy link
Author

From @mboersma on June 25, 2015 23:39

I don't know about "still"; the builder has excluded .git at least since v1.0 and I think the rationale was that it wasn't needed in the slug.

worked fine on heroku but failed to start on deis.

That's not good. Is there an example app & BUILDPACK_URL to test this with?

@deis-admin
Copy link
Author

From @bacongobbler on June 25, 2015 23:44

While we don't have a public example app to test this just yet (we'll probably need to create one), but the buildpack used in question was https://github.com/HashNuke/heroku-buildpack-elixir. I created a fork so we could expose the root problem, but from the logs it didn't show anything obvious. In /app/deps/ecto, the .git directory existed in Heroku but not in Deis. Essentially the app's dependency list had something similar to

gem 'foo', git: 'git://github.com/bacongobbler/foo'

Only for Elixir, which uses mix.

@deis-admin
Copy link
Author

From @fbjork on June 26, 2015 0:12

Here's the line that causes the error from mix:

https://github.com/elixir-lang/elixir/blob/master/lib/mix/lib/mix/scm/git.ex#L38-L41

@deis-admin
Copy link
Author

From @fbjork on June 26, 2015 0:15

I'll see if I can create a public repo to reproduce it. Meanwhile if you create a Phoenix app and add the following reps in the mix.exs file you'll get the error due to the ecto github dependency:

 defp deps do
    [{:phoenix, "~> 0.13.1"},
     {:phoenix_ecto, "~> 0.5"},
     {:ecto, github: "elixir-lang/ecto", override: true},
     {:postgrex, ">= 0.0.0"},
     {:phoenix_html, "~> 1.0"},
     {:phoenix_live_reload, "~> 0.4", only: :dev},
     {:cowboy, "~> 1.0"}]
  end

@deis-admin
Copy link
Author

From @bacongobbler on June 26, 2015 1:5

thanks @fbjork for the link! This should be an easy fix :)

@deis-admin
Copy link
Author

From @crigor on June 30, 2015 7:52

I created 2 phoenix apps that shows this issue. I used https://github.com/HashNuke/heroku-buildpack-elixir as the buildpack.

https://github.com/crigor/phoenix3924 deployed to http://italic-universe.4b79da4.my.ey.io/ returns 200.
https://github.com/crigor/phoenix3924-broken deployed to http://jiggly-lambskin.4b79da4.my.ey.io/ returns 502.

The only difference is using ecto from github on the broken app crigor/phoenix3924-broken@9e3c6b0.

The error from journalctl -u jiggly-lambskin_v3.web.1 is

Jun 30 07:40:15 ip-10-165-66-107.ec2.internal sh[5026]: Unchecked dependencies for environment prod:
Jun 30 07:40:15 ip-10-165-66-107.ec2.internal sh[5026]: * ecto (git://github.com/elixir-lang/ecto.git)
Jun 30 07:40:15 ip-10-165-66-107.ec2.internal sh[5026]: the dependency is not available, run `mix deps.get`
Jun 30 07:40:15 ip-10-165-66-107.ec2.internal sh[5026]: ** (Mix) Can't continue due to errors on dependencies

which is similar to what @fbjork reported on the EY ticket.

Ping me if you want to log in to the cluster to check the containers.

@deis-admin
Copy link
Author

From @mboersma on July 2, 2015 17:45

@crigor thanks for setting up the test apps! I've been testing changes to the --exclude flag but the second app continues to fail. I'm going to remove the flag entirely next and test again as a sanity check. I'll report here or submit a PR when I've pinned it down.

@deis-admin
Copy link
Author

From @fbjork on October 14, 2015 0:33

@crigor @mboersma Any update on this? Is it fixed?

@deis-admin
Copy link
Author

From @mboersma on October 14, 2015 15:0

@fbjork the builder itself has been completely rewritten in Go, but I think this essential behavior is the same. I apologize for not following up--I'll try again to test the above-mentioned changes with the new builder and see if that's a reasonable change.

@deis-admin
Copy link
Author

From @fbjork on October 16, 2015 17:53

@mboersma I just tried this again in Deis 1.10.1 and now I'm getting a new error:

-----> Fetching app dependencies with mix 
* Getting newrelic ([email protected]:wooga/newrelic-erlang.git) 
Cloning into '/tmp/build/deps/newrelic'... 
remote: ost key verification failed. 
fatal: Could not read from remote repository.

** (Mix) Command `git clone --no-checkout --progress "[email protected]:wooga/newrelic-erlang.git" "/tmp/build/deps/newrelic"` failed 
Please make sure you have the correct access rights 
and the repository exists. 

@deis-admin
Copy link
Author

From @fbjork on October 16, 2015 18:42

@mboersma actually, I take that back. I switched to https instead of ssh to fetch the git dependencies. It's back to the same issue:

2015-10-16T18:39:19UTC feed-service-fbjork[web.1]: Unchecked dependencies for environment prod:
2015-10-16T18:39:19UTC feed-service-fbjork[web.1]: * newrelic (https://github.com/wooga/newrelic-erlang.git)
2015-10-16T18:39:19UTC feed-service-fbjork[web.1]:   the dependency is not available, run `mix deps.get`
2015-10-16T18:39:19UTC feed-service-fbjork[web.1]: ** (Mix) Can't continue due to errors on dependencies
2015-10-16T18:39:30UTC feed-service-fbjork[web.1]: Unchecked dependencies for environment prod:
2015-10-16T18:39:30UTC feed-service-fbjork[web.1]: * newrelic (https://github.com/wooga/newrelic-erlang.git)
2015-10-16T18:39:30UTC feed-service-fbjork[web.1]:   the dependency is not available, run `mix deps.get`
2015-10-16T18:39:30UTC feed-service-fbjork[web.1]: ** (Mix) Can't continue due to errors on dependencies
2015-10-16T18:39:40UTC feed-service-fbjork[web.1]: Unchecked dependencies for environment prod:
2015-10-16T18:39:40UTC feed-service-fbjork[web.1]: * newrelic (https://github.com/wooga/newrelic-erlang.git)
2015-10-16T18:39:40UTC feed-service-fbjork[web.1]:   the dependency is not available, run `mix deps.get`
2015-10-16T18:39:40UTC feed-service-fbjork[web.1]: ** (Mix) Can't continue due to errors on dependencies

@deis-admin
Copy link
Author

From @mboersma on October 16, 2015 19:8

@fbjork I tried this with the broken example app that @crigor had worked up. With Deis master (basically the same as v1.11.1), the git push goes fine but at runtime I see the error in the app unit:

$ deis create -b  https://github.com/HashNuke/heroku-buildpack-elixir
Creating Application... done, created molten-sailfish
$ git push deis master
Counting objects: 78, done.
...
-----> Fetching custom buildpack
-----> elixir app detected
-----> Checking Erlang and Elixir versions
       WARNING: elixir_buildpack.config wasn't found in the app
       Using default config from Elixir buildpack
       Will use the following versions:
       * Stack cedar-14
       * Erlang 17.5
       * Elixir 1.0.4
       Will export the following config vars:
       * Config vars DATABASE_URL
       * MIX_ENV=prod
-----> Stack changed, will rebuild
-----> Fetching Erlang 17.5
...
-----> Fetching app dependencies with mix
       * Getting ecto (git://github.com/elixir-lang/ecto.git)
       Cloning into '/tmp/build/deps/ecto'...
       remote: Counting objects: 21377, done.
remote: remote: Compressing objects: 100% (213/213), done.
remote: remote: Total 21377 (delta 103), reused 0 (delta 0), pack-reused 21160
remote: Receiving objects: 100% (21377/21377), 5.17 MiB | 3.31 MiB/s, done.
remote: Resolving deltas: 100% (12026/12026), done.
       Checking connectivity... done.
       Running dependency resolution
       * Getting phoenix (Hex package)
       Checking package (https://s3.amazonaws.com/s3.hex.pm/tarballs/phoenix-0.14.0.tar)
       Fetched package
       Unpacked package tarball (/app/.hex/packages/phoenix-0.14.0.tar)
       * Getting phoenix_ecto (Hex package)
       Checking package (https://s3.amazonaws.com/s3.hex.pm/tarballs/phoenix_ecto-0.5.0.tar)
       Fetched package
       Unpacked package tarball (/app/.hex/packages/phoenix_ecto-0.5.0.tar)
...
-----> Copying hex from /app/.mix/archives/hex.ez
-----> Compiling
       ==> poolboy (compile)
       Compiled src/poolboy_worker.erl
...
       ==> ecto
       Compiled lib/ecto.ex
       Compiled lib/ecto/adapter.ex
...
       Compiled lib/ecto/datetime.ex
       Generated ecto app
       ==> cowlib (compile)
...
       Default process types for elixir -> web
-----> Compiled slug size is 65M

-----> Building Docker image
remote: Sending build context to Docker daemon 67.87 MB
...
Successfully built 00fc84adaf10
-----> Pushing image to private registry

-----> Launching... 
       done, molten-sailfish:v3 deployed to Deis

       http://molten-sailfish.local3.deisapp.com

       To learn more, use `deis help` or visit http://deis.io

To ssh://[email protected]:2222/molten-sailfish.git
 * [new branch]      master -> master

But it doesn't actually start, and the fleet unit Deis created says this:

Oct 16 19:03:48 deis-01 sh[14958]: 0be0b2cefa6d: Download complete
Oct 16 19:03:48 deis-01 sh[14958]: Status: Image is up to date for 172.17.8.100:5000/molten-sailfish:v3
Oct 16 19:03:48 deis-01 systemd[1]: Started molten-sailfish_v3.web.1.
Oct 16 19:03:50 deis-01 sh[14985]: Unchecked dependencies for environment prod:
Oct 16 19:03:50 deis-01 sh[14985]: * ecto (git://github.com/elixir-lang/ecto.git)
Oct 16 19:03:50 deis-01 sh[14985]: the dependency is not available, run `mix deps.get`
Oct 16 19:03:50 deis-01 sh[14985]: ** (Mix) Can't continue due to errors on dependencies
Oct 16 19:03:51 deis-01 systemd[1]: molten-sailfish_v3.web.1.service: Main process exited, code=exited, status=1/FAILURE
Oct 16 19:03:51 deis-01 sh[15148]: Error response from daemon: No such container: molten-sailfish_v3.web.1
Oct 16 19:03:51 deis-01 sh[15148]: time="2015-10-16T19:03:51Z" level="fatal" msg="Error: failed to stop one or more containers"
Oct 16 19:03:51 deis-01 systemd[1]: molten-sailfish_v3.web.1.service: Control process exited, code=exited status=1
Oct 16 19:03:51 deis-01 systemd[1]: molten-sailfish_v3.web.1.service: Unit entered failed state.
Oct 16 19:03:51 deis-01 systemd[1]: molten-sailfish_v3.web.1.service: Failed with result 'exit-code'.

I'll keep testing this and see what I can figure out.

@deis-admin
Copy link
Author

From @mboersma on October 16, 2015 21:18

After removing the --exclude flag from deis-builder's tar commands, the git push still succeeds and the actual app fails, in a different way:

Oct 16 21:17:05 deis-03 sh[8309]: 0d560c6d506f: Download complete
Oct 16 21:17:05 deis-03 sh[8309]: Status: Image is up to date for 172.17.8.101:5000/ginger-traverse:v2
Oct 16 21:17:05 deis-03 systemd[1]: Started ginger-traverse_v2.web.1.
Oct 16 21:17:05 deis-03 sh[8368]: bash: mix: command not found
Oct 16 21:17:05 deis-03 systemd[1]: ginger-traverse_v2.web.1.service: Main process exited, code=exited, status=127/n/a
Oct 16 21:17:05 deis-03 docker[8458]: Error response from daemon: no such id: ginger-traverse_v2.web.1
Oct 16 21:17:05 deis-03 docker[8458]: Error: failed to stop containers: [ginger-traverse_v2.web.1]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants