Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HealthCheck endpoint configurability #21

Open
rudigiesler opened this issue Aug 31, 2020 · 1 comment
Open

HealthCheck endpoint configurability #21

rudigiesler opened this issue Aug 31, 2020 · 1 comment

Comments

@rudigiesler
Copy link
Collaborator

Currently, the healthcheck endpoint checks both rabbitmq and redis, and this will be fetched from CELERY_BROKER_URL.

But in practice, only one of these two will be used.

Currently, this means that one of the checks will always be failing. This is an issue, because the endpoint returns a 500 error if any of the checks are failing, so we can't use any automated checking tools that will just check the status code.

I think there are 2 ways to go about this:

  1. Have separate REDIS_URL and RABBITMQ_URL configs, and choose the BROKER_URL from one of them (preferring one over the other if both are supplied), and only add a healthcheck for each of those if they're supplied.
  2. Have separate BROKER_URL, REDIS_URL, and RABBITMQ_URL, and use the celery healthcheck to check the BROKER_URL, and the redis and rabbitmq healthchecks to check the others, also only adding the healthchecks if the config values are supplied.

I think the issue for 2 is that if the celery worker is down, we don't want to be alerting or restarting the django web process, because there's nothing wrong with it, the issue is with the celery worker. But we still want to be able to alert if Django cannot contact the message broker. For that reason I'm leaning towards 1.

@cnsr
Copy link
Contributor

cnsr commented Aug 31, 2020

I think adding either of the healthcheck options to INSTALLED_APPS in dev and production settings with respectable URLs will be the cleanest solution

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants