TRT-2736: Fix sippy_prow_jobs_loaded metric to report total queried count

[mstaeble]

Summary

• The bulk writer refactor (Optimize prow loader with bulk writer, COPY protocol, and GCS improvements #3768) changed the fetch loop to only iterate over new job runs, which caused sippy_prow_jobs_loaded to report the filtered count instead of the full BigQuery result set.
• Set sippy_prow_jobs_loaded directly from len(prowJobs) after the BQ query, and set sippy_prow_jobs_processed from the consumer's batch total after all writes complete.
• Removed jobsImportedCount and jobsProcessedCount atomic fields in favor of setting gauges directly, matching how metrics are set elsewhere in the codebase.

Test plan

• go vet, golangci-lint, and go test pass
• Verify the Grafana "Prow Jobs Queried" graph recovers after the next prod load

🤖 Generated with Claude Code

[openshift-merge-bot]

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

[openshift-ci-robot]

@mstaeble: This pull request references TRT-2736 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Summary

• The bulk writer refactor (Optimize prow loader with bulk writer, COPY protocol, and GCS improvements #3768) changed the fetch loop to only iterate over new job runs, which caused sippy_prow_jobs_loaded to drop from the full BigQuery result set to just the filtered new entries.
• Set jobsImportedCount once from len(prowJobs) to restore the original metric behavior that feeds the "Prow Jobs Queried" Grafana graph.

Test plan

• go vet and go test pass
• Verify the Grafana "Prow Jobs Queried" graph recovers after the next prod load

🤖 Generated with Claude Code

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[coderabbitai]

Warning

Review limit reached

@mstaeble, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 51 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: bd2a6b58-4b5c-458b-84b5-6e564b380928

📥 Commits

Reviewing files that changed from the base of the PR and between 8450071 and a037e0a.

📒 Files selected for processing (1)

• pkg/dataloader/prowloader/prow.go

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: mstaeble
Once this PR has been reviewed and has the lgtm label, please assign xueqzhan for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coderabbitai]

Caution

Failed to replace (edit) comment. This is likely due to insufficient permissions or the comment being deleted.

Error details

putComment timed out

[openshift-merge-bot]

Scheduling required tests:
/test e2e

[openshift-ci]

@mstaeble: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.