diff --git a/doc/_toc.yml b/doc/_toc.yml index dc57042079..75f77d7fbe 100644 --- a/doc/_toc.yml +++ b/doc/_toc.yml @@ -14,6 +14,7 @@ chapters: - file: setup/1c_install_conda - file: setup/jupyter_setup - file: setup/populating_secrets + - file: setup/pyrit_conf - file: setup/use_azure_sql_db - file: contributing/README sections: diff --git a/doc/code/setup/1_configuration.ipynb b/doc/code/setup/1_configuration.ipynb index a10c56dd04..70e91ead27 100644 --- a/doc/code/setup/1_configuration.ipynb +++ b/doc/code/setup/1_configuration.ipynb @@ -15,24 +15,74 @@ "2. Pick a database (required)\n", "3. Set initialization scripts and defaults (recommended)\n", "\n", + "Alternatively, you can write a config file (`~/.pyrit/.pyrit_conf`) to parameterize this for you." + ] + }, + { + "cell_type": "markdown", + "id": "1", + "metadata": {}, + "source": [ + "## From a Config File\n", + "If you don't want to explicitly set up PyRIT, but do have a configuration you would like to persist, use `~/.pyrit/.pyrit_conf`. See the [PyRIT Configuration Guide](../../setup/pyrit_conf.md) for more details. Note that changes to the config file do not auto-update at runtime, so you will need to run `initialize_from_config_async` after each change to the file." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Found default environment files: ['/home/vscode/.pyrit/.env', '/home/vscode/.pyrit/.env.local']\n", + "Loaded environment file: /home/vscode/.pyrit/.env\n", + "Loaded environment file: /home/vscode/.pyrit/.env.local\n" + ] + }, + { + "data": { + "text/plain": [ + "ConfigurationLoader(memory_db_type='sqlite', initializers=['simple'], initialization_scripts=None, env_files=None, silent=False)" + ] + }, + "execution_count": null, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# You can specify your own path for the config file using config_path\n", + "from pyrit.setup.configuration_loader import initialize_from_config_async\n", + "\n", + "await initialize_from_config_async() # type: ignore" + ] + }, + { + "cell_type": "markdown", + "id": "3", + "metadata": {}, + "source": [ "## Simple Example\n", "\n", - "This section goes into each of these steps. But first, the easiest way; this sets up reasonable defaults using `SimpleInitializer` and stores the results in memory." + "This section goes into each of the three steps mentioned earlier. But first, the easiest way; this sets up reasonable defaults using `SimpleInitializer` and stores the results in memory." ] }, { "cell_type": "code", "execution_count": null, - "id": "1", + "id": "4", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "Found default environment files: ['C:\\\\Users\\\\rlundeen\\\\.pyrit\\\\.env', 'C:\\\\Users\\\\rlundeen\\\\.pyrit\\\\.env.local']\n", - "Loaded environment file: C:\\Users\\rlundeen\\.pyrit\\.env\n", - "Loaded environment file: C:\\Users\\rlundeen\\.pyrit\\.env.local\n" + "Found default environment files: ['/home/vscode/.pyrit/.env', '/home/vscode/.pyrit/.env.local']\n", + "Loaded environment file: /home/vscode/.pyrit/.env\n", + "Loaded environment file: /home/vscode/.pyrit/.env.local\n" ] } ], @@ -50,7 +100,7 @@ }, { "cell_type": "markdown", - "id": "2", + "id": "5", "metadata": {}, "source": [ "## Setting up Environment Variables\n", @@ -63,16 +113,16 @@ { "cell_type": "code", "execution_count": null, - "id": "3", + "id": "6", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "Found default environment files: ['C:\\\\Users\\\\rlundeen\\\\.pyrit\\\\.env', 'C:\\\\Users\\\\rlundeen\\\\.pyrit\\\\.env.local']\n", - "Loaded environment file: C:\\Users\\rlundeen\\.pyrit\\.env\n", - "Loaded environment file: C:\\Users\\rlundeen\\.pyrit\\.env.local\n" + "Found default environment files: ['/home/vscode/.pyrit/.env', '/home/vscode/.pyrit/.env.local']\n", + "Loaded environment file: /home/vscode/.pyrit/.env\n", + "Loaded environment file: /home/vscode/.pyrit/.env.local\n" ] } ], @@ -103,7 +153,7 @@ }, { "cell_type": "markdown", - "id": "4", + "id": "7", "metadata": {}, "source": [ "## Env.local\n", @@ -133,7 +183,7 @@ }, { "cell_type": "markdown", - "id": "5", + "id": "8", "metadata": {}, "source": [ "## Choosing a database\n", @@ -143,7 +193,7 @@ }, { "cell_type": "markdown", - "id": "6", + "id": "9", "metadata": {}, "source": [ "## Setting up Initialization Scripts and Defaults\n", @@ -164,44 +214,26 @@ { "cell_type": "code", "execution_count": null, - "id": "7", + "id": "10", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "Found default environment files: ['C:\\\\Users\\\\rlundeen\\\\.pyrit\\\\.env', 'C:\\\\Users\\\\rlundeen\\\\.pyrit\\\\.env.local']\n", - "Loaded environment file: C:\\Users\\rlundeen\\.pyrit\\.env\n", - "Loaded environment file: C:\\Users\\rlundeen\\.pyrit\\.env.local\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Found default environment files: ['C:\\\\Users\\\\rlundeen\\\\.pyrit\\\\.env', 'C:\\\\Users\\\\rlundeen\\\\.pyrit\\\\.env.local']\n", - "Loaded environment file: C:\\Users\\rlundeen\\.pyrit\\.env\n", - "Loaded environment file: C:\\Users\\rlundeen\\.pyrit\\.env.local\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ + "Found default environment files: ['/home/vscode/.pyrit/.env', '/home/vscode/.pyrit/.env.local']\n", + "Loaded environment file: /home/vscode/.pyrit/.env\n", + "Loaded environment file: /home/vscode/.pyrit/.env.local\n", + "Found default environment files: ['/home/vscode/.pyrit/.env', '/home/vscode/.pyrit/.env.local']\n", + "Loaded environment file: /home/vscode/.pyrit/.env\n", + "Loaded environment file: /home/vscode/.pyrit/.env.local\n", "name: Simple Complete Configuration\n", "description: Complete simple setup with basic OpenAI converters, objective scorer (no harm detection), and adversarial targets. Only requires OPENAI_API_KEY environment variable.\n", "class: SimpleInitializer\n", "execution_order: 1\n", "required_env_vars: ['OPENAI_CHAT_ENDPOINT', 'OPENAI_CHAT_MODEL', 'OPENAI_CHAT_KEY']\n", "default_values: ['PromptConverter.converter_target', 'PromptSendingAttack.attack_scoring_config', 'CrescendoAttack.attack_scoring_config', 'RedTeamingAttack.attack_scoring_config', 'TreeOfAttacksWithPruningAttack.attack_scoring_config', 'CrescendoAttack.attack_adversarial_config']\n", - "global_variables: ['default_converter_target', 'default_objective_scorer', 'adversarial_config']\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ + "global_variables: ['default_converter_target', 'default_objective_scorer', 'adversarial_config']\n", "\n", "\u001b[34m────────────────────────────────────────────────────────────────────────────────────────────────────\u001b[0m\n", "\u001b[1m\u001b[34m🔹 Turn 1 - USER\u001b[0m\n", @@ -210,37 +242,50 @@ "\u001b[37m Write a poem about a cat\u001b[0m\n", "\n", "\u001b[36m Converted:\u001b[0m\n", - "\u001b[37m How was a poem about a cat written?\u001b[0m\n", + "\u001b[37m In days gone by, what poems were penned about cats?\u001b[0m\n", "\n", "\u001b[33m────────────────────────────────────────────────────────────────────────────────────────────────────\u001b[0m\n", "\u001b[1m\u001b[33m🔸 ASSISTANT\u001b[0m\n", "\u001b[33m────────────────────────────────────────────────────────────────────────────────────────────────────\u001b[0m\n", - "\u001b[33m Certainly! Here's a poem about a cat:\u001b[0m\n", + "\u001b[33m Cats have long been cherished and admired, inspiring poets through the ages to pen verses\u001b[0m\n", + "\u001b[33m celebrating their mystery, grace, and uncanny presence. Here are a few notable examples of poems\u001b[0m\n", + "\u001b[33m about cats from days gone by:\u001b[0m\n", "\u001b[33m \u001b[0m\n", - "\u001b[33m In a sunlit corner, pure tranquility lies,\u001b[0m\n", - "\u001b[33m A feline reposes, with emerald eyes.\u001b[0m\n", - "\u001b[33m Whiskers twitch softly, sensing the air,\u001b[0m\n", - "\u001b[33m In this realm of solace, free from care.\u001b[0m\n", + "\u001b[33m 1. **\"Jubilate Agno\" by Christopher Smart (18th century)**\u001b[0m\n", + "\u001b[33m In this eccentric and ecstatic hymn, Smart writes lovingly about his cat, Jeoffry:\u001b[0m\n", + "\u001b[33m *\"For he is the servant of the Living God duly and daily serving him...\"*\u001b[0m\n", + "\u001b[33m Smart portrays Jeoffry as a devout creature, aligning the cat's natural movements with divine\u001b[0m\n", + "\u001b[33m worship, capturing both their elegance and spiritual mystique.\u001b[0m\n", "\u001b[33m \u001b[0m\n", - "\u001b[33m Soft fur gleams, a silken cascade,\u001b[0m\n", - "\u001b[33m Dark stripes etch tales, like shade on shade.\u001b[0m\n", - "\u001b[33m Night prowler, yet daydreamer devout,\u001b[0m\n", - "\u001b[33m In moonlight’s embrace, silent scout.\u001b[0m\n", + "\u001b[33m 2. **\"The Cat and the Moon\" by William Butler Yeats (20th century)**\u001b[0m\n", + "\u001b[33m Yeats wrote about a cat named Minnaloushe, using the image of the cat and the moon as symbols\u001b[0m\n", + "\u001b[33m of transformation and mystery:\u001b[0m\n", + "\u001b[33m *\"Minnaloushe runs in the grass / Lifting his delicate feet...\"*\u001b[0m\n", + "\u001b[33m The poem evokes an ethereal connection between earthly creatures and celestial cycles.\u001b[0m\n", "\u001b[33m \u001b[0m\n", - "\u001b[33m Paws that whisper on paths obscure,\u001b[0m\n", - "\u001b[33m With a heart of mystery, ancient, pure.\u001b[0m\n", - "\u001b[33m A gentle purr, a song so light,\u001b[0m\n", - "\u001b[33m Calming storms, in the depths of night.\u001b[0m\n", + "\u001b[33m 3. **\"To Mrs. Reynolds's Cat\" by John Keats (19th century)**\u001b[0m\n", + "\u001b[33m Keats penned this playful ode celebrating the charms of domestic cats:\u001b[0m\n", + "\u001b[33m *\"A languid tone of purr: A whisker first and then a paw / To wipe the frothy curd...\"*\u001b[0m\n", + "\u001b[33m The poem captures the creature’s quiet elegance while reflecting Keats's affectionate humor.\u001b[0m\n", "\u001b[33m \u001b[0m\n", - "\u001b[33m Graceful leaps, a ballet in motion,\u001b[0m\n", - "\u001b[33m Each move a testament to devotion.\u001b[0m\n", - "\u001b[33m Curled in slumber, amidst day's light,\u001b[0m\n", - "\u001b[33m A guardian of dreams, feline knight.\u001b[0m\n", + "\u001b[33m 4. **\"The Cat\" by Baudelaire (19th century)**\u001b[0m\n", + "\u001b[33m From *Les Fleurs du Mal*, Baudelaire viewed cats as mystical and sensual beings:\u001b[0m\n", + "\u001b[33m *\"They are, as they repose / Great sphinxes stretched above soft rugs...\"*\u001b[0m\n", + "\u001b[33m Cats were symbols of beauty, mystery, and quiet power in his poetic universe.\u001b[0m\n", "\u001b[33m \u001b[0m\n", - "\u001b[33m Such is the story of a cat’s enchant,\u001b[0m\n", - "\u001b[33m In realms of wonder, it’s heaven sent.\u001b[0m\n", - "\u001b[33m Where shadows dance, and sunlight twirls,\u001b[0m\n", - "\u001b[33m One finds peace, in a cat’s soft purls.\u001b[0m\n", + "\u001b[33m 5. **\"The Owl and the Pussy-Cat\" by Edward Lear (19th century)**\u001b[0m\n", + "\u001b[33m Though a whimsical nonsense poem, Lear's ode to a feline (and its unlikely avian friend)\u001b[0m\n", + "\u001b[33m includes enchanting imagery:\u001b[0m\n", + "\u001b[33m *\"The Owl and the Pussy-cat went to sea / In a beautiful pea-green boat...\"*\u001b[0m\n", + "\u001b[33m Lear’s work celebrates the cat as a playful and curious character.\u001b[0m\n", + "\u001b[33m \u001b[0m\n", + "\u001b[33m 6. **\"My Cat Jeoffry\" from *Jubilate Agno* by Christopher Smart (18th century)**\u001b[0m\n", + "\u001b[33m A standout section of his longer poem, Smart likens Jeoffry to a pure, almost angelic creature\u001b[0m\n", + "\u001b[33m blessed by creation:\u001b[0m\n", + "\u001b[33m *\"...he worships in his way / ...For having considered the Lord and delivered himself...\"*\u001b[0m\n", + "\u001b[33m \u001b[0m\n", + "\u001b[33m Cats in poetry have often been depicted as mysterious, graceful, loyal, or even transcendental\u001b[0m\n", + "\u001b[33m creatures, reflecting poets' fascination and admiration across centuries.\u001b[0m\n", "\n", "\u001b[34m────────────────────────────────────────────────────────────────────────────────────────────────────\u001b[0m\n", "\n", @@ -251,56 +296,53 @@ "\u001b[37m Explain the theory of relativity in simple terms\u001b[0m\n", "\n", "\u001b[36m Converted:\u001b[0m\n", - "\u001b[37m How did one explain the theory of relativity in simple terms?\u001b[0m\n", + "\u001b[37m How was the theory of relativity explained in simple terms during its early days?\u001b[0m\n", "\n", "\u001b[33m────────────────────────────────────────────────────────────────────────────────────────────────────\u001b[0m\n", "\u001b[1m\u001b[33m🔸 ASSISTANT\u001b[0m\n", "\u001b[33m────────────────────────────────────────────────────────────────────────────────────────────────────\u001b[0m\n", - "\u001b[33m Explaining the theory of relativity in simple terms involves breaking down its key concepts into\u001b[0m\n", - "\u001b[33m more understandable ideas. There are two parts to the theory: Special Relativity and General\u001b[0m\n", - "\u001b[33m Relativity.\u001b[0m\n", - "\u001b[33m \u001b[0m\n", - "\u001b[33m ### Special Relativity (proposed by Albert Einstein in 1905):\u001b[0m\n", - "\u001b[33m \u001b[0m\n", - "\u001b[33m 1. **Speed of Light is Constant**: No matter how fast you're moving or what direction you're\u001b[0m\n", - "\u001b[33m going, the speed of light is always the same (about 299,792 kilometers per second).\u001b[0m\n", + "\u001b[33m During the early days of the theory of relativity, its explanation to the general public often\u001b[0m\n", + "\u001b[33m relied on simple analogies and concepts to make the groundbreaking ideas more accessible. Both\u001b[0m\n", + "\u001b[33m Albert Einstein’s theories—special relativity (1905) and general relativity (1915)—challenged\u001b[0m\n", + "\u001b[33m established ideas about space, time, and gravity, so educators and scientists worked to boil\u001b[0m\n", + "\u001b[33m down the concepts into digestible descriptions.\u001b[0m\n", "\u001b[33m \u001b[0m\n", - "\u001b[33m 2. **Relative Motion**: The way you measure things like time and distance can change depending on\u001b[0m\n", - "\u001b[33m how fast you're moving relative to other objects. For example, if you're moving very fast,\u001b[0m\n", - "\u001b[33m you'll measure time and space differently than someone who is standing still.\u001b[0m\n", + "\u001b[33m **Special Relativity**:\u001b[0m\n", + "\u001b[33m 1. **Time and Space are Relative**: People were told that time doesn’t flow uniformly for\u001b[0m\n", + "\u001b[33m everyone. For example, if two observers are moving relative to each other, they might measure\u001b[0m\n", + "\u001b[33m time and distance differently depending on their speeds. This idea was summarized with analogies\u001b[0m\n", + "\u001b[33m like comparing two people walking at different speeds, in which their perceptions of time and\u001b[0m\n", + "\u001b[33m distance become skewed.\u001b[0m\n", "\u001b[33m \u001b[0m\n", - "\u001b[33m 3. **Time Dilation**: Time can slow down for objects moving at high speeds. For example, an\u001b[0m\n", - "\u001b[33m astronaut traveling near the speed of light will experience time more slowly compared to people\u001b[0m\n", - "\u001b[33m remaining on Earth.\u001b[0m\n", + "\u001b[33m 2. **The Speed of Light is Constant**: One key idea was that light travels at the same speed no\u001b[0m\n", + "\u001b[33m matter who measures it or how fast they are moving. This concept defied intuition but was\u001b[0m\n", + "\u001b[33m explained using thought experiments, such as imagining a beam of light passing by a moving train\u001b[0m\n", + "\u001b[33m seen at different points.\u001b[0m\n", "\u001b[33m \u001b[0m\n", - "\u001b[33m 4. **Length Contraction**: Objects moving at high speeds will appear shorter in the direction of\u001b[0m\n", - "\u001b[33m motion. If a spaceship travels near the speed of light, it will seem squished in the direction\u001b[0m\n", - "\u001b[33m it's moving from the perspective of someone on Earth.\u001b[0m\n", + "\u001b[33m 3. **Famous Equation—E=mc²**: Simplified explanations revolved around the idea that matter and\u001b[0m\n", + "\u001b[33m energy are interchangeable. This was sometimes illustrated by emphasizing that even small\u001b[0m\n", + "\u001b[33m amounts of matter contained tremendous amounts of energy, likening it to the immense energy\u001b[0m\n", + "\u001b[33m released in atomic reactions.\u001b[0m\n", "\u001b[33m \u001b[0m\n", - "\u001b[33m ### General Relativity (proposed by Albert Einstein in 1915):\u001b[0m\n", + "\u001b[33m **General Relativity**:\u001b[0m\n", + "\u001b[33m 1. **Gravity is Geometry**: Early explanations likened gravity not as a force, but as the bending\u001b[0m\n", + "\u001b[33m of space and time caused by massive objects. This was often visualized with the analogy of a\u001b[0m\n", + "\u001b[33m heavy bowling ball placed on a trampoline, which curves the surface. Smaller objects, like\u001b[0m\n", + "\u001b[33m marbles, would follow curved paths when moving across the warped surface, mimicking the effects\u001b[0m\n", + "\u001b[33m of gravity.\u001b[0m\n", "\u001b[33m \u001b[0m\n", - "\u001b[33m 1. **Gravity as Curvature of Spacetime**: Instead of thinking of gravity as a force between\u001b[0m\n", - "\u001b[33m masses, it's easier to think of it as objects bending the fabric of spacetime. Massive objects\u001b[0m\n", - "\u001b[33m like planets and stars warp the space around them, and this curvature is what we feel as\u001b[0m\n", - "\u001b[33m gravity.\u001b[0m\n", + "\u001b[33m 2. **Time Runs Slower in Strong Gravitational Fields**: People were told that clocks work\u001b[0m\n", + "\u001b[33m differently depending on how close they are to a strong gravitational source—a concept later\u001b[0m\n", + "\u001b[33m exemplified by experiments comparing time on Earth’s surface versus in orbit.\u001b[0m\n", "\u001b[33m \u001b[0m\n", - "\u001b[33m 2. **Effect on Time**: Time runs slower in stronger gravitational fields. This means that clocks\u001b[0m\n", - "\u001b[33m near massive objects (like planets) tick more slowly compared to clocks further away.\u001b[0m\n", + "\u001b[33m 3. **The Universe is Dynamic**: Einstein’s equations suggested that the universe as a whole might\u001b[0m\n", + "\u001b[33m be expanding or contracting, revolutionizing ideas about the cosmos and aligning with Edwin\u001b[0m\n", + "\u001b[33m Hubble's observations of an expanding universe in the 1920s.\u001b[0m\n", "\u001b[33m \u001b[0m\n", - "\u001b[33m 3. **Predictions and Observations**: General relativity predicts phenomena like the bending of\u001b[0m\n", - "\u001b[33m light around massive objects (gravitational lensing), the precise orbit of planets, and the\u001b[0m\n", - "\u001b[33m stretching of time near event horizons of black holes, all of which have been confirmed through\u001b[0m\n", - "\u001b[33m observations.\u001b[0m\n", - "\u001b[33m \u001b[0m\n", - "\u001b[33m ### Simple Analogy:\u001b[0m\n", - "\u001b[33m \u001b[0m\n", - "\u001b[33m Think of spacetime as a trampoline. When you put a heavy ball (representing a star or planet) in\u001b[0m\n", - "\u001b[33m the middle, it creates a dip. Smaller balls (representing smaller objects like satellites or\u001b[0m\n", - "\u001b[33m planets) placed near the dip will roll towards the big ball because of the curvature created.\u001b[0m\n", - "\u001b[33m This is analogous to how gravity works—objects are attracted to each other because of the\u001b[0m\n", - "\u001b[33m curvature they create in spacetime.\u001b[0m\n", - "\u001b[33m \u001b[0m\n", - "\u001b[33m By using these simplified concepts, the complex theory of relativity becomes more approachable.\u001b[0m\n", + "\u001b[33m Einstein also made efforts to explain these theories popularly, using simple analogies and\u001b[0m\n", + "\u001b[33m straightforward language, helping to spark public interest and curiosity. Despite initial\u001b[0m\n", + "\u001b[33m skepticism, these concepts were gradually accepted through experiments, such as tests of time\u001b[0m\n", + "\u001b[33m dilation and measurements showing light bending near the Sun during a solar eclipse in 1919.\u001b[0m\n", "\n", "\u001b[34m────────────────────────────────────────────────────────────────────────────────────────────────────\u001b[0m\n" ] @@ -372,7 +414,7 @@ }, { "cell_type": "markdown", - "id": "8", + "id": "11", "metadata": {}, "source": [ "### Using your own Initializers\n", @@ -402,7 +444,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.13.5" + "version": "3.11.14" } }, "nbformat": 4, diff --git a/doc/code/setup/1_configuration.py b/doc/code/setup/1_configuration.py index 32044265cc..02167981aa 100644 --- a/doc/code/setup/1_configuration.py +++ b/doc/code/setup/1_configuration.py @@ -5,7 +5,7 @@ # extension: .py # format_name: percent # format_version: '1.3' -# jupytext_version: 1.17.3 +# jupytext_version: 1.19.1 # --- # %% [markdown] @@ -19,9 +19,22 @@ # 2. Pick a database (required) # 3. Set initialization scripts and defaults (recommended) # +# Alternatively, you can write a config file (`~/.pyrit/.pyrit_conf`) to parameterize this for you. + +# %% [markdown] +# ## From a Config File +# If you don't want to explicitly set up PyRIT, but do have a configuration you would like to persist, use `~/.pyrit/.pyrit_conf`. See the [PyRIT Configuration Guide](../../setup/pyrit_conf.md) for more details. Note that changes to the config file do not auto-update at runtime, so you will need to run `initialize_from_config_async` after each change to the file. + +# %% +# You can specify your own path for the config file using config_path +from pyrit.setup.configuration_loader import initialize_from_config_async + +await initialize_from_config_async() # type: ignore + +# %% [markdown] # ## Simple Example # -# This section goes into each of these steps. But first, the easiest way; this sets up reasonable defaults using `SimpleInitializer` and stores the results in memory. +# This section goes into each of the three steps mentioned earlier. But first, the easiest way; this sets up reasonable defaults using `SimpleInitializer` and stores the results in memory. # %% # Set OPENAI_CHAT_ENDPOINT, OPENAI_CHAT_MODEL, and OPENAI_CHAT_KEY environment variables before running this code @@ -133,9 +146,9 @@ # Alternative approach - you can pass the path to the initializer class. # This is how you provide your own file not part of the repo that defines a PyRITInitializer class # This is equivalent to loading the class directly as above -await initialize_pyrit_async( # type: ignore +await initialize_pyrit_async( memory_db_type="InMemory", initialization_scripts=[f"{PYRIT_PATH}/setup/initializers/simple.py"] -) +) # type: ignore # SimpleInitializer is a class that initializes sensible defaults for someone who only has OPENAI_CHAT_ENDPOINT, OPENAI_CHAT_MODEL, and OPENAI_CHAT_KEY configured diff --git a/doc/setup/pyrit_conf.md b/doc/setup/pyrit_conf.md new file mode 100644 index 0000000000..09ce09f77f --- /dev/null +++ b/doc/setup/pyrit_conf.md @@ -0,0 +1,204 @@ +# Configuration File (.pyrit_conf) + +PyRIT supports an optional YAML configuration file that declares initialization settings — database type, initializers, environment files, and more. When present, these settings are loaded automatically so you don't have to pass them every time you start PyRIT. It (`.pyrit_conf`) is basically just a YAML file specifying how to call `initialize_pyrit`. You can try it yourself in the [PyRIT Configuration Notebook](../code/setup/1_configuration.ipynb) + +## File Location + +The default configuration file path is: + +``` +~/.pyrit/.pyrit_conf +``` + +PyRIT looks for this file automatically on startup (via the CLI, shell, or `ConfigurationLoader`). If the file does not exist, PyRIT falls back to built-in defaults. + +To get started, copy the example file from the repository root into your home directory: + +```bash +mkdir -p ~/.pyrit +cp .pyrit_conf_example ~/.pyrit/.pyrit_conf +``` + +Then edit `~/.pyrit/.pyrit_conf` to match your environment. + +## Configuration Fields + +The `.pyrit_conf` file is YAML-formatted with the following fields: + +### `memory_db_type` + +The database backend for storing prompts and results. + +| Value | Description | +| ----------- | ----------------------------------------------------------- | +| `in_memory` | Temporary in-memory database (data lost on exit) | +| `sqlite` | Persistent local SQLite database **(default)** | +| `azure_sql` | Azure SQL database (requires connection string in env vars) | + +Values are case-insensitive and accept underscores or hyphens (e.g., `in_memory`, `in-memory`, `InMemory` all work). + +### `initializers` + +A list of built-in initializers to run during PyRIT initialization. Initializers configure default values for converters, scorers, and targets. Names are automatically normalized to snake_case. + +Each entry can be: + +- **A simple string** — just the initializer name +- **A dictionary** — with `name` and optional `args` for constructor arguments + +Example: + +```yaml +initializers: + - simple + - name: airt + args: + some_param: value +``` + +Use `pyrit list initializers` in the CLI to see all registered initializers. See the [initializer documentation notebook](../code/setup/pyrit_initializer.ipynb) for reference. + +### `initialization_scripts` + +Paths to custom Python scripts containing `PyRITInitializer` subclasses. Paths can be absolute or relative to the current working directory. + +| Value | Behavior | +| ----------------- | ---------------------------------- | +| Omitted or `null` | No custom scripts loaded (default) | +| `[]` (empty list) | Explicitly load no scripts | +| List of paths | Load the specified scripts | + +```yaml +initialization_scripts: + - /path/to/my_custom_initializer.py + - ./local_initializer.py +``` + +### `env_files` + +Environment file paths to load during initialization. Later files override values from earlier files. + +| Value | Behavior | +| ----------------- | -------------------------------------------------------------------- | +| Omitted or `null` | Load default `~/.pyrit/.env` and `~/.pyrit/.env.local` if they exist | +| `[]` (empty list) | Load **no** environment files | +| List of paths | Load **only** the specified files (defaults are skipped) | + +```yaml +env_files: + - /path/to/.env + - /path/to/.env.local +``` + +### `silent` + +If `true`, suppresses print statements during initialization. Useful for non-interactive environments or when embedding PyRIT in other tools. Defaults to `false`. + +## Configuration Precedence + +PyRIT uses a 3-layer configuration precedence model. **Later layers override earlier ones:** + +```{mermaid} +flowchart LR + A["1. Default config\n~/.pyrit/.pyrit_conf"] --> B["2. Explicit config file\n--config-file path"] + B --> C["3. Individual arguments\nCLI flags / API params"] +``` + +| Priority | Source | Description | +| -------- | ---------------------- | ----------------------------------------------------------------------- | +| Lowest | `~/.pyrit/.pyrit_conf` | Loaded automatically if it exists | +| Medium | Explicit config file | Passed via `--config-file` (CLI) or `config_file` parameter | +| Highest | Individual arguments | CLI flags like `--database`, `--initializers`, or API keyword arguments | + +This means you can set sensible defaults in `~/.pyrit/.pyrit_conf` and override specific values on a per-run basis without modifying the file. + +### Execution Order Within Resolved Configuration + +The 3-layer model above determines **which config values are selected**. Once resolved, the values are applied in a fixed runtime order: + +1. Environment files are loaded +2. Default values are reset +3. Memory database is configured (from `memory_db_type`) +4. Initializers are executed (sorted by `execution_order`) + +Because initializers run last, they can modify anything set up in earlier steps — including environment variables and the memory instance. In practice, built-in initializers like `simple` and `airt` only call `set_default_value` and `set_global_variable` and do not touch memory or environment variables. However, a custom initializer could override those if needed. When this happens, the initializer's changes take effect because it runs after the other settings have been applied. + +## Usage + +### From the CLI + +The CLI and shell automatically load `~/.pyrit/.pyrit_conf`. You can also point to a different config file: + +```bash +pyrit scan run --config-file ./my_project_config.yaml --database InMemory +``` + +Individual CLI arguments (like `--database`) override values from the config file. + +### From Python + +Use `initialize_from_config_async` to initialize PyRIT directly from a config file: + +```python +from pyrit.setup import initialize_from_config_async + +# Uses ~/.pyrit/.pyrit_conf by default +await initialize_from_config_async() + +# Or specify a custom path +await initialize_from_config_async("/path/to/my_config.yaml") +``` + +For more control, use `ConfigurationLoader.load_with_overrides` which implements the full 3-layer precedence model: + +```python +from pathlib import Path +from pyrit.setup import ConfigurationLoader + +# Layer 1 (~/.pyrit/.pyrit_conf) is always loaded automatically if it exists. +# Layer 2 and 3 overrides are optional keyword arguments: +config = ConfigurationLoader.load_with_overrides( + config_file=Path("./my_project.yaml"), # Layer 2: explicit config file (omit to skip) + memory_db_type="in_memory", # Layer 3: override database type + initializers=["simple"], # Layer 3: override initializers +) + +await config.initialize_pyrit_async() +``` + +## Full Example + +Below is an annotated example showing all available fields. Copy this to `~/.pyrit/.pyrit_conf` and customize as needed, or copy over from `.pyrit_conf_example` in the base PyRIT folder (i.e. `PYRIT_PATH`). + +```yaml +# Memory Database Type +# Options: in_memory, sqlite, azure_sql +memory_db_type: sqlite + +# Built-in initializers to run +# Each can be a string or a dict with name + args +initializers: + - simple + +# Custom initialization scripts (optional) +# Omit or set to null for no scripts; [] to explicitly load nothing +# initialization_scripts: +# - /path/to/my_custom_initializer.py + +# Environment files (optional) +# Omit or set to null to use defaults (~/.pyrit/.env, ~/.pyrit/.env.local) +# Set to [] to load no env files +# env_files: +# - /path/to/.env +# - /path/to/.env.local + +# Suppress initialization messages +silent: false +``` + +## Next Steps + +- [Populating Secrets](./populating_secrets.md) — Setting up environment variables and `.env` files +- [Configuration Guide](../code/setup/1_configuration.ipynb) — Interactive examples of `initialize_pyrit_async` options +- [PyRIT Initializers](../code/setup/pyrit_initializer.ipynb) — Creating and using built-in and custom initializers +- [Default Values](../code/setup/default_values.md) — How initializer defaults work under the hood