Skip to content

libsuricata: add rate_filter callback - v1 #13086

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from

Conversation

jasonish
Copy link
Member

Mainly to start some conversation around this requested feature.

From ticket https://redmine.openinfosecfoundation.org/issues/7673:

If a ratefilter kicks in, have a callback that allows to customise the action

More comments in-line.

Use the BIT_U8 macros for packet alert flags and rename
PACKET_ALERT_RATE_FILTER_MODIFIED to
PACKET_ALERT_FLAG_RATE_FILTER_MODIFIED for consistency.
@jasonish jasonish requested review from victorjulien and a team as code owners April 23, 2025 16:57
src/decode.h Outdated
Comment on lines 267 to 277
/** user private flag 0 */
PACKET_ALERT_FLAG_USER0 = BIT_U8(12),

/** user private flag 1 */
PACKET_ALERT_FLAG_USER1 = BIT_U8(13),

/** user private flag 2 */
PACKET_ALERT_FLAG_USER2 = BIT_U8(14),

/** user private flag 3 */
PACKET_ALERT_FLAG_USER3 = BIT_U8(15),
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the POC shown to me, based on an older version of Suricata there were still some bits free user for the user to add. The idea here is to reserve some specifically for users (library users in this scope).

uint8_t flags;
uint16_t flags;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There was a hole of 3 bytes here according to pahole, now there is a 1 byte hole.

src/detect.h Outdated
Comment on lines 1138 to 1142
/* user provided rate filter callbacks. */
SCDetectRateFilterFunc rate_filter_callback;

/* use provided data to be passed to rate_filter_callback. */
void *rate_filter_callback_arg;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just allow one callback for now, I'm not sure if a list of callbacks really makes sense here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can expand this later?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, probably even in non-breaking ways if DetectEngineCtx is treated as an opaque data structure. And of course a library user could just fan it out as needed.

@jasonish
Copy link
Member Author

Ping @amirabell.

@jasonish
Copy link
Member Author

Some other thoughts:

  • 8.0 considers tenant ID in rate filtering, 7.0 didn't. Is this even needed?
  • However, library users could implement a completely different ideas of tenancy
  • Would an additional user provided 'tenant' identifier be useful?

But if there is some other logic/lookups that are not built into Suricata, then a callback makes sense.

@jasonish jasonish force-pushed the ratefilter-callback/v1 branch from c550e9e to 87a8739 Compare April 23, 2025 17:24
Copy link

codecov bot commented Apr 23, 2025

Codecov Report

Attention: Patch coverage is 82.60870% with 4 lines in your changes missing coverage. Please review.

Project coverage is 82.61%. Comparing base (f301cd3) to head (f4880a8).

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #13086      +/-   ##
==========================================
- Coverage   82.78%   82.61%   -0.17%     
==========================================
  Files         985      985              
  Lines      272547   272554       +7     
==========================================
- Hits       225617   225178     -439     
- Misses      46930    47376     +446     
Flag Coverage Δ
fuzzcorpus 60.51% <60.86%> (-0.28%) ⬇️
livemode 18.96% <17.39%> (+0.02%) ⬆️
pcap 44.65% <34.78%> (-0.01%) ⬇️
suricata-verify 64.76% <82.60%> (-0.01%) ⬇️
unittests 58.11% <56.52%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jasonish jasonish force-pushed the ratefilter-callback/v1 branch from 87a8739 to f4880a8 Compare April 23, 2025 20:52
@suricata-qa
Copy link

Information: QA ran without warnings.

Pipeline 25767

@suricata-qa
Copy link

Information: QA ran without warnings.

Pipeline 25773

Expand to a uint16_t as there is a library use case to have some
user private flags, so reserve the last 4 flags for library use.

We were already at the max for a u8, so this opens up 4 more for our
use.

As #define's like this can't be logically grouped into an enum, try
Doxygen documentation groups, which create a group just of these flag
values and documents them together on a page.
As the callback is added to the current detection engine, make sure its
copied to the new detection engine on reload.

Ticket: OISF#7673
@jasonish jasonish force-pushed the ratefilter-callback/v1 branch from f4880a8 to 6afc16d Compare April 24, 2025 14:27
@suricata-qa
Copy link

Information: QA ran without warnings.

Pipeline 25776

@jasonish
Copy link
Member Author

Replaced by #13121

@jasonish jasonish closed this Apr 30, 2025
@jasonish jasonish deleted the ratefilter-callback/v1 branch May 21, 2025 21:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants