RabbitMQ: create queues durable for 4.x compat

[kba]

Transient, non-exclusive queues (ie. queue created with durable=False and exclusive=False are deprecated as of 4.0 and removed as of 4.3. of RabbitMQ.

We have upgraded to RabbitMQ 4.3.1 for our deployment, therefore fixing this in core is now urgent for us.

The solution I went with is to just make the queues durable. That means the queues survive all workers for a processor being disconnected. They do reattach cleanly though, once this is set up.

The only issue I found so far is that I did need to delete the old queues manually, otherwise the different durable settings will lead to errors and somehow the non-durable queues did not automatically get deleted after shutting down the workers. But that could be a timing issue.

Since I am not a RabbitMQ expert, I'm not sure whether this might cause different problems or whether we need to configure some TTL mechanism. So I would appreciate feedback by those who know better e.g. @MehmedGIT

[kba]

Also, if the queues are persistent now, would it not make sense to also make the messages persistent? I.e. in RMQPublisher.publish_to_queue:

  properties = BasicProperties(
      app_id="ocrd_network default app id",
      content_type="application/json",
+     delivery_mode=2,
      headers=headers
)

[MehmedGIT]

Looks good for now.

The only issue I found so far is that I did need to delete the old queues manually

The queue_delete and queue_purge methods inside connector.py should be utilized wherever appropriate to delete the queues on restarts (when desired). I need to check exactly how to implement that so it's less painful when we want fresh restarts, e.g., when doing manual tests.

otherwise the different durable settings will lead to errors and somehow the non-durable queues did not automatically get deleted after shutting down the workers. But that could be a timing issue.

I think that's rather related to the feature flags which are enabled by default. However, I'm not sure how to fix that yet.