Skip to content

FEAT Add JailBreakV-28K dataset#1372

Open
apoorvdarshan wants to merge 1 commit intoAzure:mainfrom
apoorvdarshan:feat/add-jailbreakv-28k-dataset
Open

FEAT Add JailBreakV-28K dataset#1372
apoorvdarshan wants to merge 1 commit intoAzure:mainfrom
apoorvdarshan:feat/add-jailbreakv-28k-dataset

Conversation

@apoorvdarshan
Copy link

Summary

  • Adds the JailBreakV-28K dataset loader (_JailbreakVDataset) to pyrit.datasets
  • Supports both JailBreakV_28K (28,300 jailbreak prompts) and RedTeam_2K (2,000 red team queries) configs
  • Defaults to the mini_JailBreakV_28K split (280 rows) for lightweight usage
  • Sets harm_categories from the dataset's policy column (16 safety policy categories)

Closes #1007

Test plan

  • Unit tests for initialization defaults and custom configs/splits
  • Unit tests for fetching with both JailBreakV_28K and RedTeam_2K configs
  • Unit tests for skipping empty prompts, empty policy handling, and error on empty dataset
  • Unit tests for optional metadata fields

References

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FEAT Add JailbreakV_28K dataset

1 participant

Comments