8000 kic entrypoint: retryable update-alternatives, move earlier into execution by tstromberg · Pull Request #8826 · kubernetes/minikube · GitHub
[go: up one dir, main page]

Skip to content

kic entrypoint: retryable update-alternatives, move earlier into execution - #8826

Closed
tstromberg wants to merge 3 commits into
kubernetes:masterfrom
tstromberg:longer-container-retry
Closed

tstromberg wants to merge 3 commits into
kubernetes:masterfrom
tstromberg:longer-container-retry

Conversation

@tstromberg
@tstromberg tstromberg commented Jul 24, 2020 •
Copy link
Copy Markdown
Contributor

Makes #8814 easier to debug - and may actually just resolve it.

Let me know if you want a separate PR to add a pristine copy of the entrypoint.

@tstromberg
tstromberg requested a review from medyagh July 24, 2020 15:59
@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 24, 2020
@k8s-ci-robot
Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tstromberg

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot
k8s-ci-robot requested a review from blueelvis July 24, 2020 15:59
@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 24, 2020
@tstromberg tstromberg changed the title entrypoint: retryable update-alternatives, move earlier into execution kic entrypoint: retryable update-alternatives, move earlier into execution Jul 24, 2020
Comment thread hack/images/kicbase.Dockerfile Resolved
@TravisBuddy
Copy link
Copy Markdown

Travis tests have failed

Hey @tstromberg,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

1st Build

View build log

make test
which go-bindata || GO111MODULE=off GOBIN="/home/travis/gopath/bin" go get github.com/jteeuwen/go-bindata/...
PATH="/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.14.6.linux.amd64/bin:/home/travis/bin:/home/travis/bin:/home/travis/.local/bin:/usr/local/lib/jvm/openjdk11/bin:/opt/pyenv/shims:/home/travis/.phpenv/shims:/home/travis/perl5/perlbrew/bin:/home/travis/.nvm/versions/node/v8.12.0/bin:/home/travis/.rvm/gems/ruby-2.5.3/bin:/home/travis/.rvm/gems/ruby-2.5.3@global/bin:/home/travis/.rvm/rubies/ruby-2.5.3/bin:/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.11.1.linux.amd64/bin:/usr/local/maven-3.6.3/bin:/usr/local/cmake-3.12.4/bin:/usr/local/clang-7.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/home/travis/.rvm/bin:/home/travis/.phpenv/bin:/opt/pyenv/bin:/home/travis/.yarn/bin:/home/travis/gopath/bin" go-bindata -nomemcopy -o pkg/minikube/assets/assets.go -pkg assets deploy/addons/...
gofmt -s -w pkg/minikube/assets/assets.go
which go-bindata || GO111MODULE=off GOBIN="/home/travis/gopath/bin" go get github.com/jteeuwen/go-bindata/...
/home/travis/gopath/bin/go-bindata
PATH="/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.14.6.linux.amd64/bin:/home/travis/bin:/home/travis/bin:/home/travis/.local/bin:/usr/local/lib/jvm/openjdk11/bin:/opt/pyenv/shims:/home/travis/.phpenv/shims:/home/travis/perl5/perlbrew/bin:/home/travis/.nvm/versions/node/v8.12.0/bin:/home/travis/.rvm/gems/ruby-2.5.3/bin:/home/travis/.rvm/gems/ruby-2.5.3@global/bin:/home/travis/.rvm/rubies/ruby-2.5.3/bin:/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.11.1.linux.amd64/bin:/usr/local/maven-3.6.3/bin:/usr/local/cmake-3.12.4/bin:/usr/local/clang-7.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/home/travis/.rvm/bin:/home/travis/.phpenv/bin:/opt/pyenv/bin:/home/travis/.yarn/bin:/home/travis/gopath/bin" go-bindata -nomemcopy -o pkg/minikube/translate/translations.go -pkg translate translations/...
gofmt -s -w pkg/minikube/translate/translations.go
MINIKUBE_LDFLAGS="-X k8s.io/minikube/pkg/version.version=v1.12.1 -X k8s.io/minikube/pkg/version.isoVersion=v1.12.0 -X k8s.io/minikube/pkg/version.isoPath=minikube/iso -X k8s.io/minikube/pkg/version.gitCommitID="c3c26bc27041a8861084d477c39451af42980ac0"" ./test.sh
= make lint =============================================================
golangci/golangci-lint info checking GitHub for tag 'v1.26.0'
golangci/golangci-lint info found version: 1.26.0 for v1.26.0/linux/amd64
golangci/golangci-lint info installed out/linters/golangci-lint
ok
= go mod ================================================================
ok
= boilerplate ===========================================================
boilerplate missing: /home/travis/gopath/src/github.com/kubernetes/minikube/test/stress/stress.sh
consider running: /home/travis/gopath/src/github.com/kubernetes/minikube/hack/boilerplate/fix.sh
Makefile:274: recipe for target 'test' failed
make: *** [test] Error 8
TravisBuddy Request Identifier: ed037a00-cdee-11ea-98fb-f5dcae6af814

@TravisBuddy
Copy link
Copy Markdown

Travis tests have failed

Hey @tstromberg,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

1st Build

Vi 8000 ew build log

make test
which go-bindata || GO111MODULE=off GOBIN="/home/travis/gopath/bin" go get github.com/jteeuwen/go-bindata/...
PATH="/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.14.6.linux.amd64/bin:/home/travis/bin:/home/travis/bin:/home/travis/.local/bin:/usr/local/lib/jvm/openjdk11/bin:/opt/pyenv/shims:/home/travis/.phpenv/shims:/home/travis/perl5/perlbrew/bin:/home/travis/.nvm/versions/node/v8.12.0/bin:/home/travis/.rvm/gems/ruby-2.5.3/bin:/home/travis/.rvm/gems/ruby-2.5.3@global/bin:/home/travis/.rvm/rubies/ruby-2.5.3/bin:/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.11.1.linux.amd64/bin:/usr/local/maven-3.6.3/bin:/usr/local/cmake-3.12.4/bin:/usr/local/clang-7.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/home/travis/.rvm/bin:/home/travis/.phpenv/bin:/opt/pyenv/bin:/home/travis/.yarn/bin:/home/travis/gopath/bin" go-bindata -nomemcopy -o pkg/minikube/assets/assets.go -pkg assets deploy/addons/...
gofmt -s -w pkg/minikube/assets/assets.go
which go-bindata || GO111MODULE=off GOBIN="/home/travis/gopath/bin" go get github.com/jteeuwen/go-bindata/...
/home/travis/gopath/bin/go-bindata
PATH="/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.14.6.linux.amd64/bin:/home/travis/bin:/home/travis/bin:/home/travis/.local/bin:/usr/local/lib/jvm/openjdk11/bin:/opt/pyenv/shims:/home/travis/.phpenv/shims:/home/travis/perl5/perlbrew/bin:/home/travis/.nvm/versions/node/v8.12.0/bin:/home/travis/.rvm/gems/ruby-2.5.3/bin:/home/travis/.rvm/gems/ruby-2.5.3@global/bin:/home/travis/.rvm/rubies/ruby-2.5.3/bin:/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.11.1.linux.amd64/bin:/usr/local/maven-3.6.3/bin:/usr/local/cmake-3.12.4/bin:/usr/local/clang-7.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/home/travis/.rvm/bin:/home/travis/.phpenv/bin:/opt/pyenv/bin:/home/travis/.yarn/bin:/home/travis/gopath/bin" go-bindata -nomemcopy -o pkg/minikube/translate/translations.go -pkg translate translations/...
gofmt -s -w pkg/minikube/translate/translations.go
MINIKUBE_LDFLAGS="-X k8s.io/minikube/pkg/version.version=v1.12.1 -X k8s.io/minikube/pkg/version.isoVersion=v1.12.0 -X k8s.io/minikube/pkg/version.isoPath=minikube/iso -X k8s.io/minikube/pkg/version.gitCommitID="c3c26bc27041a8861084d477c39451af42980ac0"" ./test.sh
= make lint =============================================================
golangci/golangci-lint info checking GitHub for tag 'v1.26.0'
golangci/golangci-lint info found version: 1.26.0 for v1.26.0/linux/amd64
golangci/golangci-lint info installed out/linters/golangci-lint
ok
= go mod ================================================================
ok
= boilerplate ===========================================================
boilerplate missing: /home/travis/gopath/src/github.com/kubernetes/minikube/test/stress/stress.sh
consider running: /home/travis/gopath/src/github.com/kubernetes/minikube/hack/boilerplate/fix.sh
Makefile:274: recipe for target 'test' failed
make: *** [test] Error 8
TravisBuddy Request Identifier: e1f004b0-cdf0-11ea-98fb-f5dcae6af814

@medyagh
medyagh commented Jul 25, 2020 •
Copy link
Copy Markdown
Member

@tstromberg 28 failures with this new image
full log can be seen in the artifact that u can download from https://github.com/kubernetes/minikube/pull/8826/checks?check_run_id=908356949

attached is the text fail log
testout.txt

---------------- 28 Failures :( ----------------------------
[
  "TestOffline/group/docker",
  "TestOffline/group/crio",
  "TestOffline/group/containerd",
  "TestErrorSpam",
  "TestJSONOutput/serial/DistinctCurrentSteps",
  "TestJSONOutput/serial/IncreasingCurrentSteps",
  "TestMissingContainerUpgrade",
  "TestNetworkPlugins/group/calico/Start",
  "TestNetworkPlugins/group/custom-weave/Start",
  "TestNetworkPlugins/group/kindnet/Start",
  "TestNetworkPlugins/group/auto/DNS",
  "TestNetworkPlugins/group/enable-default-cni/DNS",
  "TestNetworkPlugins/group/kubenet/DNS",
  "TestNetworkPlugins/group/bridge/DNS",
  "TestStartStop/group/crio/serial/FirstStart",
  "TestStartStop/group/containerd/serial/FirstStart",
  "TestStartStop/group/crio/serial/DeployApp",
  "TestStartStop/group/containerd/serial/DeployApp",
  "TestStartStop/group/crio/serial/SecondStart",
  "TestStartStop/group/containerd/serial/SecondStart",
  "TestStartStop/group/crio/serial/UserAppExistsAfterStop",
  "TestStartStop/group/containerd/serial/UserAppExistsAfterStop",
  "TestStartStop/group/crio/serial/AddonExistsAfterStop",
  "TestStartStop/group/containerd/serial/AddonExistsAfterStop",
  "TestStartStop/group/crio/serial/VerifyKubernetesImages",
  "TestStartStop/group/crio/serial/Pause",
  "TestStartStop/group/containerd/serial/VerifyKubernetesImages",
  "TestStartStop/group/containerd/serial/Pause"
]
-------------------------------------------------------
*** 28 Failed ***

I also noticed the following tests take a lot of time:

Failed Test | Duration
Test-- | Duration--
TestNetworkPlugins/group/custom-weave/Start | 1711.46
TestNetworkPlugins/group/kindnet/Start | 1709.2
TestNetworkPlugins/group/calico/Start | 1703.91

@medyagh
medyagh commented Jul 25, 2020 •
Copy link
Copy Markdown
Member

for cio first start:

failed to set bridge addr: could not add IP address to "cni0": permission denied

for containerd I see this error


I0724 22:19:37.408877  335425 exit.go:58] WithError(failed to start node)=startup failed: wait 6m0s for node: waiting for apps_running: expected k8s-apps: missing components: kube-dns called from:
	goroutine 1 [running]:
	runtime/debug.Stack(0x0, 0x0, 0x100000000000000)
		/home/jenkins/actions-runner/_work/_tool/go/1.14.6/x64/src/runtime/debug/stack.go:24 +0x9d
	k8s.io/minikube/pkg/minikube/exit.WithError(0x1ba7c56, 0x14, 0x1ebf200, 0xc00090f1e0)
		/home/jenkins/actions-runner/_work/minikube/minikube/pkg/minikube/exit/exit.go:58 +0x34
	k8s.io/minikube/cmd/minikube/cmd.runStart(0x2cd0820, 0xc0003f6620, 0x2, 0xe)
		/home/jenkins/actions-runner/_work/minikube/minikube/cmd/minikube/cmd/start.go:206 +0x505
	github.com/spf13/cobra.(*Command).execute(0x2cd0820, 0xc0003f6540, 0xe, 0xe, 0x2cd0820, 0xc0003f6540)
		/home/jenkins/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:846 +0x29d
	github.com/spf13/cobra.(*Command).ExecuteC(0x2ccf860, 0x0, 0x1, 0xc000a1a5f0)
		/home/jenkins/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:950 +0x349
	github.com/spf13/cobra.(*Command).Execute(...)
		/home/jenkins/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:887
	k8s.io/minikube/cmd/minikube/cmd.Execute()
		/home/jenkins/actions-runner/_work/minikube/minikube/cmd/minikube/cmd/root.go:106 +0x72c
	main.main()
		/home/jenkins/actions-runner/_work/minikube/minikube/cmd/minikube/main.go:71 +0x11f
	W0724 22:19:37.409094  335425 out.go:249] failed to start node: startup failed: wait 6m0s for node: waiting for apps_running: expected k8s-apps: missing components: kube-dns
	* 
	X failed to start node: startup failed: wait 6m0s for node: waiting for apps_running: expected k8s-apps: missing components: kube-dns
	* 
	* minikube is exiting due to an error. If the above message is not useful, open an issue:
	  - https://github.com/kubernetes/minikube/issues/new/choose
** /stderr **

@medyagh
medyagh commented Jul 25, 2020
Copy link
Copy Markdown
Member

for containerd first start I see

I0724 22:19:37.408877  335425 exit.go:58] WithError(failed to start node)=startup failed: wait 6m0s for node: waiting for apps_running: expected k8s-apps: missing components: kube-dns called from:
	goroutine 1 [running]:
	runtime/debug.Stack(0x0, 0x0, 0x100000000000000)
		/home/jenkins/actions-runner/_work/_tool/go/1.14.6/x64/src/runtime/debug/stack.go:24 +0x9d
	k8s.io/minikube/pkg/minikube/exit.WithError(0x1ba7c56, 0x14, 0x1ebf200, 0xc00090f1e0)
		/home/jenkins/actions-runner/_work/minikube/minikube/pkg/minikube/exit/exit.go:58 +0x34
	k8s.io/minikube/cmd/minikube/cmd.runStart(0x2cd0820, 0xc0003f6620, 0x2, 0xe)
		/home/jenkins/actions-runner/_work/minikube/minikube/cmd/minikube/cmd/start.go:206 +0x505
	github.com/spf13/cobra.(*Command).execute(0x2cd0820, 0xc0003f6540, 0xe, 0xe, 0x2cd0820, 0xc0003f6540)
		/home/jenkins/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:846 +0x29d
	github.com/spf13/cobra.(*Command).ExecuteC(0x2ccf860, 0x0, 0x1, 0xc000a1a5f0)
		/home/jenkins/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:950 +0x349
	github.com/spf13/cobra.(*Command).Execute(...)
		/home/jenkins/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:887
	k8s.io/minikube/cmd/minikube/cmd.Execute()
		/home/jenkins/actions-runner/_work/minikube/minikube/cmd/minikube/cmd/root.go:106 +0x72c
	main.main()
		/home/jenkins/actions-runner/_work/minikube/minikube/cmd/minikube/main.go:71 +0x11f
	W0724 22:19:37.409094  335425 out.go:249] failed to start node: startup failed: wait 6m0s for node: waiting for apps_running: expected k8s-apps: missing components: kube-dns
	* 
	X failed to start node: startup failed: wait 6m0s for node: waiting for apps_running: expected k8s-apps: missing components: kube-dns
	* 
	* minikube is exiting due to an error. If the above message is not useful, open an issue:
	  - https://github.com/kubernetes/minikube/issues/new/choose
** /stderr **

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

0