8000 make sure worker nodes actually join control plane on restart by sharifelgamal · Pull Request #9476 · kubernetes/minikube · GitHub
[go: up one dir, main page]

Skip to content

make sure worker nodes actually join control plane on restart - #9476

Merged
sharifelgamal merged 4 commits into
kubernetes:masterfrom
sharifelgamal:restart-worker
Oct 16, 2020
Merged

sharifelgamal merged 4 commits into
kubernetes:masterfrom
sharifelgamal:restart-worker

Conversation

@sharifelgamal
Copy link
Copy Markdown
Contributor

This should fix #9456

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Oct 15, 2020
@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 15, 2020
} else {
return errors.Wrapf(err, "cmd failed: %s\n%+v\n", joinCmd, out.Output())
}
return errors.Wrapf(err, "cmd failed: %s\n%+v\n", joinCmd, out.Output())
@medyagh medyagh Oct 16, 2020 •
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do the same check for the error message like before but this time retry and also let user know we are retrying

already exists in the cluster. You must delete the existing Node or change the name of this new joining Node

if that error is there, we will klog.Info (so the IDE users who usually dont like to wait without any logs, know we are retrying

if rr.Output() has that ^^ error, {
klog.Info("retrying to re-register worker node to the controlplane.... ")
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, RunCmd already includes the rr.Output in the error message,
return errors.Wrapf(err, "join cmd") is enough

} else {
return errors.Wrapf(err, "cmd failed: %s\n%+v\n", joinCmd, out.Output())
}
return errors.Wrapf(err, "cmd failed: %s\n%+v\n", joinCmd, out.Output())
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, RunCmd already includes the rr.Output in the error message,
return errors.Wrapf(err, "join cmd") is enough

@medyagh medyagh left a comment
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets also remove the manual time.Sleep in our test
minikube should wait enough for the kubelet thing itself so users dont have to do that.
https://github.com/medyagh/minikube/blob/aad43a2e43317540ef6167c17715a4dd449bcbaa/test/integration/multinode_test.go#L231

@sharifelgamal
Copy link
Copy Markdown
Contributor Author

/ok-to-test

@k8s-ci-robot k8s-ci-robot added the ok-to-test Indicates a non-member PR verified by an org member that is safe to test. label Oct 16, 2020
@kubernetes kubernetes deleted a comment from minikube-pr-bot Oct 16, 2020
@minikube-pr-bot
Copy link
Copy Markdown

kvm2 Driver
Times for minikube: 61.6s 61.7s 60.1s
Average time for minikube: 61.2s

Times for Minikube (PR 9476): 60.3s 62.0s 59.8s
Average time for Minikube (PR 9476): 60.7s

Averages Time Per Log

Details
+--------------------------------+----------+--------------------+
|              LOG               | MINIKUBE | MINIKUBE (PR 9476) |
+--------------------------------+----------+--------------------+
| * minikube v1.14.0 on Debian   | 0.1s     | 0.0s               |
|                           9.11 |          |                    |
| * Using the kvm2 driver based  | 0.0s     | 0.0s               |
| on user configuration          |          |                    |
| * Starting control plane node  | 0.0s     | 0.0s               |
| minikube in cluster minikube   |          |                    |
| * Creating kvm2 VM (CPUs=2,    | 35.8s    | 35.7s              |
| Memory=3700MB, Disk=20000MB)   |          |                    |
| ...                            |          |                    |
| * Preparing Kubernetes v1.19.2 | 23.1s    | 22.9s              |
| on Docker 19.03.12 ...         |          |                    |
| * Verifying Kubernetes         | 1.6s     | 1.6s               |
| components...                  |          |                    |
| * Enabled addons:              | 0.1s     | 0.3s               |
| default-storageclass,          |          |                    |
| storage-provisioner            |          |                    |
|                                | 0.0s     | 0.0s               |
| * Want kubectl v1.19.2? Try    |          |                    |
| 'minikube kubectl -- get pods  |          |                    |
| -A'                            |          |                    |
| * Done! kubectl is now         |          |                    |
| configured to use "minikube"   |          |                    |
| by default                     |          |                    |
+--------------------------------+----------+--------------------+

docker Driver
Times for minikube: 29.9s 29.2s 28.9s
Average time for minikube: 29.3s

Times for Minikube (PR 9476): 28.5s 31.1s 31.5s
Average time for Minikube (PR 9476): 30.4s

Averages Time Per Log

Details
+--------------------------------+----------+--------------------+
|              LOG               | MINIKUBE | MINIKUBE (PR 9476) |
+--------------------------------+----------+--------------------+
| * minikube v1.14.0 on Debian   | 0.2s     | 0.2s               |
|                           9.11 |          |                    |
| * Using the docker driver      | 0.1s     | 0.1s               |
| based on user configuration    |          |                    |
| * Starting control plane node  | 0.1s     | 0.1s               |
| minikube in cluster minikube   |          |                    |
| * Creating docker container    | 9.2s     | 9.0s               |
| (CPUs=2, Memory=3700MB) ...    |          |                    |
| * Preparing Kubernetes v1.19.2 | 18.6s    | 19.9s              |
| on Docker 19.03.8 ...          |          |                    |
| * Verifying Kubernetes         | 1.1s     | 1.1s               |
| components...                  |          |                    |
| * Enabled addons:              | 0.1s     | 0.1s               |
| storage-provisioner,           |          |                    |
| default-storageclass           |          |                    |
|                                | 0.0s     | 0.0s               |
| * Want kubectl v1.19.2? Try    |          |                    |
| 'minikube kubectl -- get pods  |          |                    |
| -A'                            |          |                    |
| * Done! kubectl is now         |          |                    |
| configured to use "minikube"   |          |                    |
| by default                     |          |                    |
+--------------------------------+----------+--------------------+

@k8s-ci-robot
Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: medyagh, sharifelgamal

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [medyagh,sharifelgamal]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@codecov-io
Copy link
Copy Markdown

Codecov Report

Merging #9476 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #9476   +/-   ##
=======================================
  Coverage   29.05%   29.05%           
=======================================
  Files         171      171           
  Lines       10443    10443           
=======================================
  Hits         3034     3034           
  Misses       6987     6987           
  Partials      422      422           

@minikube-pr-bot
Copy link
Copy Markdown

kvm2 Driver
Times for minikube: 62.0s 63.9s 60.2s
Average time for minikube: 62.0s

Times for Minikube (PR 9476): 59.6s 60.7s 61.3s
Average time for Minikube (PR 9476): 60.5s

Averages Time Per Log

Details
+--------------------------------+----------+--------------------+
|              LOG               | MINIKUBE | MINIKUBE (PR 9476) |
+--------------------------------+----------+--------------------+
| * minikube v1.14.0 on Debian   | 0.1s     | 0.0s               |
|                           9.11 |          |                    |
| * Using the kvm2 driver based  | 0.0s     | 0.0s               |
| on user configuration          |          |                    |
| * Starting control plane node  | 0.0s     | 0.0s               |
| minikube in cluster minikube   |          |                    |
| * Creating kvm2 VM (CPUs=2,    | 37.0s    | 35.7s              |
| Memory=3700MB, Disk=20000MB)   |          |                    |
| ...                            |          |                    |
| * Preparing Kubernetes v1.19.2 | 22.8s    | 23.1s              |
| on Docker 19.03.12 ...         |          |                    |
| * Verifying Kubernetes         | 1.6s     | 1.5s               |
| components...                  |          |                    |
| * Enabled addons:              | 0.8s     | 0.1s               |
| default-storageclass,          |          |                    |
| storage-provisioner            |          |                    |
|                                | 0.0s     | 0.0s               |
| * Want kubectl v1.19.2? Try    |          |                    |
| 'minikube kubectl -- get pods  |          |                    |
| -A'                            |          |                    |
| * Done! kubectl is now         |          |                    |
| configured to use "minikube"   |          |                    |
| by default                     |          |                    |
+--------------------------------+----------+--------------------+

docker Driver
Times for minikube: 29.4s 28.6s 27.5s
Average time for minikube: 28.5s

Times for Minikube (PR 9476): 29.4s 29.9s 28.9s
Average time for Minikube (PR 9476): 29.4s

Averages Time Per Log

Details
+--------------------------------+----------+--------------------+
|              LOG               | MINIKUBE | MINIKUBE (PR 9476) |
+--------------------------------+----------+--------------------+
| * minikube v1.14.0 on Debian   | 0.2s     | 0.2s               |
|                           9.11 |          |                    |
| * Using the docker driver      | 0.1s     | 0.1s               |
| based on user configuration    |          |                    |
| * Starting control plane node  | 0.1s     | 0.1s               |
| minikube in cluster minikube   |          |                    |
| * Creating docker container    | 9.2s     | 8.8s               |
| (CPUs=2, Memory=3700MB) ...    |          |                    |
| * Preparing Kubernetes v1.19.2 | 17.9s    | 19.2s              |
| on Docker 19.03.8 ...          |          |                    |
| * Verifying Kubernetes         | 0.9s     | 1.0s               |
| components...                  |          |                    |
| * Enabled addons:              | 0.1s     | 0.1s               |
| storage-provisioner,           |          |                    |
| default-storageclass           |          |                    |
|                                | 0.0s     | 0.0s               |
| * Want kubectl v1.19.2? Try    |          |                    |
| 'minikube kubectl -- get pods  |          |                    |
| -A'                            |          |                    |
| * Done! kubectl is now         |          |                    |
| configured to use "minikube"   |          |                    |
| by default                     |          |                    |
+--------------------------------+----------+--------------------+

@sharifelgamal
sharifelgamal merged commit a282354 into kubernetes:master Oct 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FailedTest: TestMultiNode/serial/RestartMultiNode kubelet not running after restart

5 participants

0