• ustcmirror-images
  • Table Of Content
  • Introduction
  • Quick Start
  • Configuration
    • Volumes
    • Common Configuration Parameters(AKA environment variables)
    • aptsync
    • apt-sync
    • archvsync
    • crates-io-index
    • debian-cd
    • docker-ce
    • fedora
    • freebsd-pkg
    • freebsd-ports
    • ghcup
    • github-release
    • gitsync
    • google-repo
    • gsutil-rsync
    • hackage
    • homebrew-bottles
    • julia-storage
    • lftpsync
    • misc
    • nix-channels
    • pypi
    • rclone
    • rsync
    • rubygems
    • stackage
    • tsumugu
    • winget-source
    • yukina
    • yum-sync
  • License
  • Contributing

These images are designed for mirroring remote directories/repositories in a consistent and portable way. They are used by ustcmirror (yuki).

docker run --rm \
    -e LOG_ROTATE_CYCLE='5' \
    -e RSYNC_HOST='rsync.alpinelinux.org' \
    -e RSYNC_PATH='alpine/' \
    -e RSYNC_MAXDELETE='10000' \
    -v /var/repos/alpine:/data \
    -v /var/sync-logs/alpine:/log \
    ustcmirror/rsync:latest

• /data: The mount point of the repository on the host. You can refer to it as environment variable TO in your program.
• /log: The mount point of the host directory that save logs. You can refer to it as environment variable LOG in your program.

Notes: The following mirror.list:

deb-i386 https://apt.dockerproject.org/repo debian-jessie main
deb-amd64 https://apt.dockerproject.org/repo debian-jessie main
deb-armhf https://apt.dockerproject.org/repo raspbian-jessie main testing

is equivalent to the following parameters:

APTSYNC_URL='https://apt.dockerproject.org/repo'
APTSYNC_DISTS='debian-jessie|main|i386 amd64:raspbian-jessie|main testing|armhf'

It is almost the same as aptsync. Except that APTSYNC_DISTS accepts 4 parameters (rather than 3) for every item.

Consider use apt-sync when the upstream replaces packages in-place, as aptsync will simply ignore if there're any changes in existed packages.

A.K.A. ftpsync

archvsync respects the env vars used in ftpsync

A dedicated script to sync https://github.com/rust-lang/crates.io-index.

debian-cd accepts the same parameters specified in debian-cd-mirror.conf

docker-ce accepts following parameters:

fedora-quick-mirror

See dist conf for parameters meaning.

Note: This image is not in use now, as quick-fedora-mirror has some mysterious bugs when being used.

Notice: BIND_ADDRESS is only added for curl in freebsd-ports. Make sure that github.com is accessible under default network settings.

ghcup does not have outstanding configuration options.

To specified the repo list to sync, you can:

• Read-only bind mount a YAML file into the container at /repos.yaml. See the example.
• Pass the YAML-format repo list string as REPOS env.

A script for syncing projects (especially AOSP) using Google's repo tool.

A new solution to sync Julia general registry (using StorageMirrorServer.jl). No parameters needed.

Download seperate, small files inconvenient for other sync containers.

Example of DOWNLOAD_LINKS:

brew-install.sh https://github.com/Homebrew/install/raw/HEAD/install.sh
rustup-install.sh https://sh.rustup.rs/

Sync PyPI with official mirror client bandersnatch. Note that this image is not actively maintained. It's suggested to use shadowmire instead.

ref:

• rclone environment variables
• rclone-sync manual

Shadowmire syncs PyPI (or plain HTTP(S) PyPI mirrors using Shadowmire) with a lightweight and easy approach.

Stackage doesn't need to specify upstream.

An alternative HTTP(S) syncing tool, replacing rclone and lftp in some cases. See usage.

A handy tool to sync pre-indexed Windows Package Manager (aka. WinGet) sources.

yukina analyses given nginx log, and maintains binary blobs (which does not modify once exist) state under given size limit. Usually this shall be used with another sync container that only downloads index files.

Note that you shall bind necessary nginx log to /nginx-log/ when syncing.

yum-sync tries to imitate the parameters of aptsync, and it supports the following substitution rule for YUMSYNC_URL and <reponame> & <download_dir> in YUMSYNC_DISTS:

• @{arch}: Architecture (x86_64, armhf, ...)
• @{os_ver}: OS version (6-8, ...)
• @{comp}: The <component> in YUMSYNC_DISTS

yum-sync.py is modified to get the same directory structure as upstream when syncing. And <reponame> should be named the same as the directory containing repodata dir.

Notes:

The following repo configuration:

baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el$releasever-$basearch

translates to:

YUMSYNC_URL='https://packages.cloud.google.com/yum/repos/kubernetes-el@{os_ver}-@{arch}'
YUMSYNC_DISTS='6-7|kubernetes|x86_64,aarch64,armhfp,ppc64le,s390x|kubernetes-el@{os_ver}-@{arch}|/yum/repos/kubernetes-el@{os_ver}-@{arch}'

And the following:

[mysql80-community]
name=MySQL 8.0 Community Server
baseurl=http://repo.mysql.com/yum/mysql-8.0-community/el/$releasever/$basearch/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-mysql

[mysql57-community]
name=MySQL 5.7 Community Server
baseurl=http://repo.mysql.com/yum/mysql-5.7-community/el/$releasever/$basearch/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-mysql

translates to:

YUMSYNC_URL='https://repo.mysql.com/yum/@{comp}/el/@{os_ver}/@{arch}/'
YUMSYNC_DISTS='6-8|mysql-8.0-community,mysql-5.7-community|aarch64,i386,x86_64|@{arch}|/yum/@{comp}/el/@{os_ver}/@{arch}/'

Specially, contents of folder apt-sync, yum-sync, github-release, docker-ce and the generated container image from them are under GPLv3 license, as it uses code from https://github.com/tuna/tunasync-scripts.

Other contents are under MIT license.

See CONTRIBUTING.md.