How to Archive Old Fedora Releases

The Fedora download servers contain terabytes of data, and to allow for mirrors to not have to take all of that data, infrastructure regularly moves data of end of lifed releases (from /pub/fedora/linux) to the archives section (/pub/archive/fedora/linux)

Steps Involved

log into batcave01.iad2.fedoraproject.org and ssh to bodhi-backend01
```
$ sudo -i ssh [email protected]
# su - ftpsync
```
Then change into the releases directory.
```
$ cd /pub/fedora/linux/releases
```
Check to see that the target directory doesn’t already exist.
```
$ ls /pub/archive/fedora/linux/releases/
```
If the target directory does not already exist, do a recursive link copy of the tree you want to the target
```
$ cp -lvpnr 21 /pub/archive/fedora/linux/releases/21
```
Common errors which might occur at this time is finding that the NFS partition is not mounted read-write. Other errors can occur where the NFS is read-write, but ftpsync does not have rights to copy a file. These need to be dealt with individually.

Another error which happens when I script things is that you find out that everything got copied into /pub/archive/fedora/linux/releases/21/21 even though it shouldn’t.
If the target directory already exists, then we need to do a recursive rsync to update any changes in the trees since the previous copy.
```
$ rsync -avAXSHP --delete ./21/ /pub/archive/fedora/linux/releases/21/
```

We now do the updates and updates/testing in similar ways.

$ cd ../updates/
$ cp -lpnr 21 /pub/archive/fedora/linux/updates/21
$ cd testing
$ cp -lpnr 21 /pub/archive/fedora/linux/updates/testing/21

Alternative if this is a later refresh of an older copy.

$ cd ../updates/
$ rsync -avAXSHP 21/ /pub/archive/fedora/linux/updates/21/
$ cd testing
$ rsync -avAXSHP 21/ /pub/archive/fedora/linux/updates/testing/21/

Do the same with fedora-secondary.

$ cd /pub/fedora-secondary/releases/
$ cp -lpnr 21 /pub/archive/fedora-secondary/releases/21
$ cd ../updates/
$ cp -lpnr 21 /pub/archive/fedora-secondary/updates/21
$ cd testing
$ cp -lpnr 21 /pub/archive/fedora-secondary/updates/testing/21

$ cd /pub/fedora-secondary/releases/
$ rsync -avSAXHP --delete 21/ /pub/archive/fedora-secondary/releases/21/
$ cd ../updates/
$ rsync -avSAXHP --delete 21/ /pub/archive/fedora-secondary/updates/21/
$ cd testing
$ rsync -avSAXHP --delete 21/ /pub/archive/fedora-secondary/updates/testing/21/

Announce to the mirror list this has been done and that in 2 weeks you will move the old trees to archives.

In two weeks, run the following playbook from batcave:

$ rbac-playbook -v /srv/web/infra/ansible/playbooks/manual/mirrormanager/move-to-archive.yml --extra-vars="product='Fedora' version='38'"

This will start a job named move-to-archive-fedora-38 on Openshift. The playbook will output two commands to view the job’s logs and to cancel it if needed. You can run those commands on os-control01 or on any host logged in to OpenShift. Alternatively, you can follow the job’s execution on the Openshift WebUI and cancel it from there. The playbook will wait for the job to complete.
Note that the Fedora Codecs will not be archived, because we currently don’t do that. This category is explicitely excluded in the MirrorManager command itself.
If there are problems, the postgres DB may have issues and so you need to get a DBA to update the backend to fix items.

Wait an hour or so then you can remove the files from the main tree.

$ sudo -i ssh bodhi-backend01.iad2.fedoraproject.org
$ cd /pub/fedora/linux
$ cd releases/21
$ ls # make sure you have stuff here
$ rm -rf *
$ ln ../20/README .
$ cd ../../updates/21
$ ls #make sure you have stuff here
$ rm -rf *
$ ln ../20/README .
$ cd ../testing/21
$ ls # make sure you have stuff here
$ rm -rf *
$ ln ../20/README .

$ cd /pub/fedora-secondary/releases
$ cd 21
$ ls # make sure you have stuff here
$ rm -rf *
$ ln ../20/README .
$ cd ../../updates/21
$ ls # make sure you have stuff here
$ rm -rf *
$ ln ../20/README .
$ cd ../testing/21
$ ls # make sure you have stuff here
$ rm -rf *
$ ln ../20/README .

This should complete the archiving.

Want to help? Learn how to contribute to Fedora Docs ›