1

advise which library is suitable for my problem. I need a lot of workers that will run on different machines, they will execute tasks, but each worker has a resource (for example, from 0 to 100) and after the task is completed, its resource is reduced, and when it becomes 0 worker is excluded from dispatching(later it may become 100 and again ready to get tasks) Is there such functionality(or possibility to write it) in any task-queueing library so that the next task gets to the worker with the maximum resource value?

1

Celery is arguably one of most popular frameworks of the many python based task queues available. Search for others on pypi.

As with regards to your worker resource terminology, that's a bit confusing by it seems like you are speaking something along the lines of limiting the max tasks per worker or autoscaling. Could you possibly clarify that part of your question further ?

  • By resources, I mean that a worker can perform, for example, no more than 10 tasks per hour, and I want to distribute tasks between workers so that the next task not arrives to first idle one but who has completed the tasks so far less than other (i.e., has the maximum resource in my terminology) – CatcherStat Apr 25 at 21:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.