Skip to content

Implement database listeners and make database notify on jobs change #879

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 47 commits into
base: main
Choose a base branch
from

Conversation

ylei-tsubame
Copy link
Collaborator

@ylei-tsubame ylei-tsubame commented Mar 31, 2025

This PR contains the foundation work for #801.

Changes:

  • The database will notify when the jobs table gets an insert or update change.
  • The database perl module will have the basic methods for managing the database listeners which can receive and react to the notify calls from the database.
  • anvil-daemon will react to notify calls triggered by the jobs table.
  • anvil-access-module will use Proc::Simple as a replacement for fork() to keep the code base cleaner.

Additional thoughts:
The functions provided in anvil-access-module to manage database listeners may not be needed since the listener-related logic was moved to the database perl module. But it's kept in place for convenience.

If more control is required, try using the methods directly with JSON as parameters, like: x Database->add_listener '{"param1":"value1",...}'

Got confused, ^ is actually required. Can't write perl subroutines in javascript.

Update 1:
Some changes broke the communication between the striker-ui-api and its child anvil-access-module. Added a transitional patch which will be changed in the next PR.

Update 2:
Removed patch from update 1 because it's not actually solving the problem. The communication between striker-ui-api and its child anvil-access-module will still break (or mismatch) sometimes. Logs show DBI::dbh->clone() is stuck and causes the response to never be sent.

Update 3:
Made a log enhancement for update 2, which shouldn't have fixed the problem in update 2, but I'm having trouble reproducing the problem now... 😭

Update 4:
The problem from update 2 persists; although not always reproducible. It may be caused by the listeners' high-res sleep?

@ylei-tsubame ylei-tsubame requested a review from digimer March 31, 2025 18:23
@ylei-tsubame
Copy link
Collaborator Author

retest this please

@ylei-tsubame ylei-tsubame marked this pull request as ready for review April 1, 2025 05:07
@digimer
Copy link
Member

digimer commented Apr 1, 2025

retest this please

2 similar comments
@digimer
Copy link
Member

digimer commented Apr 1, 2025

retest this please

@digimer
Copy link
Member

digimer commented Apr 1, 2025

retest this please

@ylei-tsubame ylei-tsubame force-pushed the issues/801-event-driven-jobs branch from 0fc0b17 to b7143c6 Compare April 1, 2025 23:42
@ylei-tsubame ylei-tsubame marked this pull request as draft April 2, 2025 02:49
digimer
digimer previously approved these changes Apr 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants