8000 Don't run the extraction tar container for podman by afbjorklund · Pull Request #8057 · kubernetes/minikube · GitHub
[go: up one dir, main page]

Skip to content

Don't run the extraction tar container for podman - #8057

Merged
medyagh merged 1 commit into
kubernetes:masterfrom
afbjorklund:podman-tar
May 11, 2020
Merged

medyagh merged 1 commit into
kubernetes:masterfrom
afbjorklund:podman-tar

Conversation

@afbjorklund
Copy link
Copy Markdown
Collaborator

The preloaded images will be extracted anyway, using ssh.

But if creating them on the volume before the container is
booted, means that /var will not be fully copied over to it.
And without /var/lib/dpkg and others, kicbase will not boot.
So skip the parallel extraction for podman, do it afterwards.

Probably shouldn't mount all of /var, but just a sub-set...

Closes #8056

The preloaded images will be extracted anyway, using ssh.

But if creating them on the volume before the container is
booted, means that /var will not be fully copied over to it.
And without /var/lib/dpkg and others, kicbase will not boot.
So skip the parallel extraction for podman, do it afterwards.

Probably shouldn't mount all of /var, but just a sub-set...
@afbjorklund
afbjorklund requested a review from medyagh May 9, 2020 16:40
@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels May 9, 2020
@k8s-ci-robot
Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: afbjorklund

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 9, 2020
@codecov-io
Copy link
Copy Markdown

Codecov Report

Merging #8057 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #8057   +/-   ##
=======================================
  Coverage   35.45%   35.45%           
=======================================
  Files         146      146           
  Lines        9284     9284           
=======================================
  Hits         3292     3292           
  Misses       5592     5592           
  Partials      400      400           

@medyagh medyagh left a comment •
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, I believe for podman the volume is just a folder on your system. If the host system had lz4 installed we could do it with lz4 in parallel. But probably that would be too much depending on user tools.

@medyagh
medyagh commented May 9, 2020
Copy link
Copy Markdown
Member

/ok-to-test

@k8s-ci-robot k8s-ci-robot added the ok-to-test Indicates a non-member PR verified by an org member that is safe to test. label May 9, 2020
@minikube-pr-bot
Copy link
Copy Markdown

kvm2 Driver
docker Driver

@afbjorklund
Copy link
Copy Markdown
Collaborator Author

Makes sense, I believe for podman the volume is just a folder on your system. If the host system had lz4 installed we could do it with lz4 in parallel. But probably that would be too much depending on user tools.

You could always go back to gzip, or even to plain .tar if that was the only problem. Or perhaps bundle lz4 with minikube, like we did for lzma when it was new and xz hadn't taken over yet.

@afbjorklund
Copy link
Copy Markdown
Collaborator Author

Makes sense, I believe for podman the volume is just a folder on your system.

It's a folder on Docker too, the main difference is in timing when it copies the files over.

But otherwise it is quite similar, there are only small differences in paths and timestamps:

 docker volume inspect docker
[
    {
        "CreatedAt": "2020-05-09T14:36:50+02:00",
        "Driver": "local",
        "Labels": {
            "created_by.minikube.sigs.k8s.io": "true",
            "name.minikube.sigs.k8s.io": "docker"
        },
        "Mountpoint": "/var/lib/docker/volumes/docker/_data",
        "Name": "docker",
        "Options": {},
        "Scope": "local"
    }
]
 sudo podman volume inspect podman
[
     {
          "Name": "podman",
          "Driver": "local",
          "Mountpoint": "/var/lib/containers/storage/volumes/podman/_data",
          "CreatedAt": "2020-05-09T14:35:14.431826651+02:00",
          "Labels": {
               "created_by.minikube.sigs.k8s.io": "true",
               "name.minikube.sigs.k8s.io": "podman"
          },
          "Scope": "local",
          "Options": {
               
          }
     }
]

So we have 5M of various junk under /var in the ubuntu image.
And that gets copied, from the image storage to the volume storage...

 docker run -it ubuntu:19.10 sh -c "du -hxcs /var/*"
4.0K	/var/backups
1.5M	/var/cache
3.7M	/var/lib
4.0K	/var/local
0	/var/lock
252K	/var/log
4.0K	/var/mail
4.0K	/var/opt
0	/var/run
4.0K	/var/spool
4.0K	/var/tmp
5.4M	total
 sudo ls /var/lib/docker/volumes/docker/_data
backups  cache	lib  local  lock  log  mail  opt  run  spool  tmp
 sudo ls /var/lib/containers/storage/volumes/podman/_data
backups  cache	lib  local  lock  log  mail  opt  run  spool  tmp

But we only actually use a couple of those, like #8056 (comment)

The rest just gets duplicated (from the image), for no particular reason.

@afbjorklund
afbjorklund commented May 10, 2020 •
Copy link
Copy Markdown
Collaborator Author

My solution going forward would be to maybe keep the /var volume (for compatibiity when upgrading), but to only mount a subset of the existing folders. Like we are doing with the ISO, to make them same.

But will open a new issue for that.

I don't think the storage-provisioner works in KIC either, since we are missing those mounts.
And it will be weird to have them under "var", so better move it (var) to a volume subdirectory.

Another issue (storage-provisoner)

@medyagh
medyagh merged commit d2ae2b3 into kubernetes:master May 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

No 5039 ne yet

Development

Successfully merging this pull request may close these issues.

Podman driver crashes after timeout and volume recreate

5 participants

0