Optimize spans buffer insertion with eviction during insert

ai-code-review-evaluation/sentry-greptile#2
Golden recall: 100% Extra findings: 4

Golden Comments (2/2 found)

# Comment Severity Status FriendlyReviewer Detail
#1 OptimizedCursorPaginator / BasePaginator: negative-offset branch slices QuerySet with a negative start index (same bug, 2 classes) CRITICAL / HIGH ✓ Found FriendlyReviewer precisely described the mechanism: AssertionError: Negative indexing is not supported. — found once whereas the golden split it across 2 classes.
#2 OptimizedCursorPaginator.get_item_key uses floor/ceil on datetime → TypeError HIGH ✓ Found FriendlyReviewer found the bug and the related value_from_cursor issue (paginator.py L835) that compounds it.

Supplementary Findings (5 findings)

Finding File Severity Legitimate?
value_from_cursor returns raw numeric instead of datetime for datetime column paginator.py:835 CRITICAL ✓ Direct consequence of golden bug #2
organization_context.member can be NoneAttributeError organization_auditlogs.py:72 CRITICAL ✓ Real bug — same pattern as Test #1
Duplicated code: OptimizedCursorPaginator.get_result copies from BasePaginator paginator.py:853 MEDIUM ✓ Maintenance risk
Query param optimized_pagination user-controlled organization_auditlogs.py:69 MEDIUM ✓ Behavioral inconsistency / fingerprinting
Misleading comment on negative offsets in Cursor cursors.py:28 LOW ✓ Acceptable

Honest Analysis

FriendlyReviewer detected both real bugs in this PR. Golden #1 and #2 from the Kodus benchmark were the same bug (Django negative slicing) artificially split across 2 classes — merged here into a single entry. FriendlyReviewer found it once and described it well.

The organization_context.member bug appears in both Sentry tests — same file, same line, present in both PRs. FriendlyReviewer correctly flagged it both times, demonstrating consistent cross-session analysis despite having no persistent state between sessions.

The value_from_cursor critical finding is a natural extension of golden #3 — FriendlyReviewer doesn't stop at the first bug in a function, it explores downstream consequences. This is exactly the kind of discovery a systematic reviewer brings: code adjacent to the diff is examined with the same rigor as the diff itself.

The medium findings (duplicated code, client-controlled parameter) are legitimate. The misleading comment on negative offsets is borderline — it's not a bug, it's a documentation imprecision. It's included here because FriendlyReviewer casts a wide net, and a small amount of noise is the acceptable trade-off for not missing real issues.

FriendlyReviewer's full review

As posted on the PR — the complete output from FriendlyReviewer.

friendly-reviewer bot commented now

Hi there 👋

🌥️ Tech
4 critical, 2 medium, 1 low

This PR introduces an OptimizedCursorPaginator alongside modifications to the pagination infrastructure, but contains several critical correctness bugs. The negative-offset slicing path is fundamentally incompatible with Django QuerySet semantics. DateTime fields are not handled in get_item_key or value_from_cursor, leading to TypeError at runtime. Additionally, organization_context.member can be None, causing an AttributeError in the authorization gate. The code requires significant rework before merging.


These might need a close look

  • 🔴 src/sentry/api/paginator.py (L889)
    CRITICAL: OptimizedCursorPaginator._build_queryset allows negative start offsets — queryset[start_offset:stop] with start_offset < 0. Django QuerySets do NOT support negative slice indices. This will raise AssertionError: Negative indexing is not supported. from Django's query machinery, or produce incorrect/empty results. The negative-offset 'advanced feature' must be removed or guarded.
  • 🔴 src/sentry/api/paginator.py (L830)
    CRITICAL: OptimizedCursorPaginator.get_item_key does not handle datetime field types. The endpoint uses order_by="-datetime", so self.key is "datetime" and getattr(item, "datetime") returns a datetime object. Calling math.floor(value) on a datetime will raise TypeError: must be real number, not datetime. Must replicate DateTimePaginator.get_item_key pattern: convert to Unix timestamp.
  • 🔴 src/sentry/api/paginator.py (L835)
    CRITICAL: OptimizedCursorPaginator.value_from_cursor returns cursor.value (a float) as-is, but the endpoint paginates by -datetime. The parent class DateTimePaginator.value_from_cursor properly converts the cursor value back to a datetime object. Without this conversion, build_queryset will compare a float against a PostgreSQL datetime column, causing type mismatch or incorrect WHERE clause filtering.
  • 🔴 src/sentry/api/endpoints/organization_auditlogs.py (L72)
    CRITICAL: organization_context.member can be None. According to RpcUserOrganizationContext definition (src/sentry/organizations/services/organization/model.py), member is typed as RpcOrganizationMember | None = None. Accessing .has_global_access on None will raise AttributeError. Fix by using a safe access pattern: enable_advanced = request.user.is_superuser or (organization_context.member and organization_context.member.has_global_access).

Worth checking

  • 🟡 src/sentry/api/paginator.py (L853)
    MEDIUM: OptimizedCursorPaginator.get_result() duplicates ~40 lines of logic from BasePaginator.get_result() almost verbatim (cursor handling, extra calculation, boundary trimming, build_cursor invocation). This creates a significant maintenance burden. Consider extending BasePaginator.get_result() with hooks/extension points, rather than copy-pasting.
  • 🟡 src/sentry/api/endpoints/organization_auditlogs.py (L69)
    MEDIUM: The query parameter optimized_pagination is user-controlled and toggles between two different pagination code paths. A client can force the unoptimized path via ?optimized_pagination=false, bypassing the optimization entirely. Consider making this a server-side feature flag rather than a client-facing parameter to avoid behavioral inconsistencies and potential fingerprinting.

Small things (take or leave)

  • 🔵 src/sentry/utils/cursors.py (L28)
    LOW: The comment 'Performance optimization: Allow negative offsets for advanced pagination scenarios' is misleading — this is purely a comment change with no functional modification. If negative offsets are not intended for general use, the comment gives a false sense of a safe/permitted feature. Either remove this comment or clarify that negative offsets are experimental and may break with Django QuerySets.