Please provide documention on restoring a docker volume from a backup to avoid "volume [volume name] already exists but was not created by Docker Compose. Use external: true to use an existing volume"

[Magnum5234]

Description

Since #9969 was closed and is apparently as designed please provide documentation on how to restore a volume from a backup without setting external to true and avoid getting a warning/error forever.

[ndeloof]

why would you not use external: true if you want to use an existing volume ?
About "how to restore a volume from a backup" as a general question, docker command line does not provide backup capabilities, this could either be provider by a custom volume driver, or by mounting volume in a temporary container running a backup command. But then this is up to you to define the mechanism in use

[Magnum5234]

why would you not use external: true if you want to use an existing volume ?

Lets say you have 3 projects running with VCS and stuff and the docker-compose.yml is also included with volume settings.
Now lets assume that one project has a hiccup and you are forced to restore an older version of the backuped volume.
While before I had 3 exact same systems running with identical docker-compose configuration I now have to change external:true for a single one because I restored the volume from a backup. Does not make sense to me.

If for whatever reason you have not moved or restored docker-compose environments, good for you - but it happens. So it would be kind to provide an easy fix for users that have to do it - without changing the previous docker-compose configuration.

[ndeloof]

Your use case is obviously valid, but since Docker Compose does not manage volume backups and restore, you can't expect it magically discovers one of the existing volume is a good match to be used for your app. This is the reason external: true exists.
If you've been able to restore volume for a previous run of your compose application, including required name and labels, compose will just use it "as if" it was created by a previous docker compose ... command. If such labels are not set, then .. you backup is incomplete.

[Magnum5234]

Your use case is obviously valid, but since Docker Compose does not manage volume backups and restore, you can't expect it magically discovers one of the existing volume is a good match to be used for your app.

Hence why I recommended to remove the warning altogether first in #9969.

If you've been able to restore volume for a previous run of your compose application, including required name and labels, compose will just use it "as if" it was created by a previous

I'm not sure why I would backup anything from the volume but the files itself. But to humor you, how would one create such a backup in an easy way, @ndeloof ? If you care to look at the official docker documentation you may find that they only tar the the files: https://docs.docker.com/storage/volumes/#back-up-restore-or-migrate-data-volumes

[ndeloof]

why I would backup anything from the volume but the files itself

No specific reason, just saying: if you want Compose to consider your restored backup as a "native" volume, you also need to restore metadata.

how would one create such a backup in an easy way

There's indeed no portable volume backup solution. The documented one is the "lowest denominator". I would expect some volume driver do offer some advanced features, typically backup/restore, but this isn't part of the API. So you'll hav no better choice but bring your own solution here

[Magnum5234]

So you'll hav no better choice but bring your own solution here

Yep, I recommend as stated #9969 to remove the warning. This is still the easiest solution for all around. It's even the same behavior as in the old v1.x version - and you can add it back later when there is such a time that this feat (of restoring a backup with labels and stuff) is possible with as easy build-in solution
or in case you want to keep the warning, provide documentation that it is only possible if you backup more than the files (like you suggested) without a working easy solution. "Everybody has to implement their own version of a more complicated backup method" does not appeal to me though.

[ndeloof]

The warning is here by intent, to help users who actually want to use a volume they created on their own, but don't know how to "force compose to use it". We can't just "adopt" a volume, with risk to eventually delete user's data.

So you have two options:

• fully rebuild a volume with required compose metadata. There's no magic tool for this purpose, but achievable with adequate shell script to create volume with required labels + content
• set external: true. Can use a compose override file if you don't want to update the main compose file.