4/12/2023 0 Comments Docker convoy plugin![]() ![]() There are few large files rather than many small files. Throughput in the system is measured rather in minutes per task and not in tasks per second. I need no replication (files are only relevant for 30-60 mins and after that they're no longer needed anyways) and I specifically don't want a central storage. However, workers will mostly (90-95% of times) access files which are local to their host. In other words: Each host will have the same "network volume" mounted in some place, and it contains some files which are actually stored on the current host (these files are primarily relevant), and some files which are stored on another host. To achieve this, I need the files to be stored in a shared volume which the workers in other hosts can transparently use to access files stored on another host. However sometimes, a host might be overburdened and I want workers in another host to take over the remaining processing steps.Primarily, workers on each host will only work on files that are already stored locally. Each step generates some large file in the range of 1-10GB. There are several steps to processing a task.That means each host runs all workers needed to make the system work, and ideally one task that enters the system runs completely on the host that first started it. I have a Docker Swarm with many hosts, but each host is essentially self-contained and looks no different than any other host.Trying to pull quay.io/libpod/busybox:latest.I'm looking for a distributed file system solution/network file system which can be used in the following scenario: Reload and use it from another podman]# podman volume podman]# podman run -it -v test_othernode:/test quay.io/libpod/busybox More details:Ĭreate from one ~]# podman volume create -driver convoy test_othernode Set up two host using one volume plugin convoy and use podman volume reload to get the volume created in another host, the volume can be get and used as expected. The volume create/delete for this plugin will be sent to the driver directly. If the same volume is reported by two plugins, the remove of the volume has to be done by specifying the driver.Ī mechanism to discover the volumes from the plugin via podman volume refresh/reloadĪ option to bypass the database for a plugin. Even in case of docker the behavior is indeterministic, if two drivers report the same volume. Presence of same volume name in two different plugin can be an invalid configuration or a corner case and the behavior can be defined and handled accordingly. Not supporting this use case would make it difficult for migration from docker to podman in those deployments. This is a must have requirement for deployment in the clustered environment. Podman should detect all the volumes seen by the volume plugin driver.ĭocker supports this configuration, where in volume available in the plugin are detected and listed. Podman can detect only volumes created locally using podman create. The podman running on the other node, does not detect the volume, using same custom volume driver. Create volume using custom volume driver, on shared storage, on one node of a two node cluster. Version-Release number of selected component (if applicable):ġ. This affects deployment of podman in clustered environment where the volume is created on shared storage from one node, is not detected by podman running on the other node. If volume is created out of band in the volume plugin, the volume is not detected by podman, unlike docker. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |