JIF_extraction

A toolkit to build a CJEU judgments dataset, select an extraction model, and run Judicial Interpretative Formulas (JIF) extraction. The entry point is main.py, which interactively guides you through dataset preparation, model selection, and extraction.

For reviewing purposes:

• All evaluation metrics are available and computable in the metrics.ipynb file
• All extraction results are available in the out/ folder

Key Features

• Dataset builder for CJEU judgments (downloaded from metadata).
• Model selection module to compare/choose an extraction model.
• Extraction runner with configurable model and prompting strategy.

Project Structure

JIF_extraction/
├─ out/                     # Outputs produced by extraction or experiments
├─ res/                     # Resources (metadata CSV, optional local data)
├─ src/                     # Project source code (modules and utilities)
├─ .gitignore
├─ main.py                  # Entry point (interactive CLI)
├─ metrics.ipynb            # Notebook for metrics/analysis
├─ README.md                # This file
└─ requirements.txt         # Python dependencies

Requirements

• Python 3.10+
• pip
• Poe API key

Install dependencies:

pip install -r requirements.txt

Data and Paths

• The script expects a metadata CSV at: res/CJEU_complete.csv (configurable via METADATA_PATH in main.py).
• The script checks for a dataset folder at ../../res/CJEU_judgments relative to main.py. If not found, it will offer to download/build the dataset.
  • You can change this location in the code to fit your environment, or create the expected directory and place judgments there.

Tip: If your project root contains res/, you may prefer to adjust the code to use a project-local path like res/CJEU_judgments for convenience.

Usage

Run the interactive workflow:

python main.py

To access the Poe APIs, create a credentials.py file in the utils folder with your API key and base_url.

API_KEY = "YOUR_API_KEY" #from your poe subscription 
base_url = "https://api.poe.garden"

You will be prompted to:

1. Download/build the dataset if it’s not present.
2. Run the Model Selection module.
3. Run the Extraction module with a specified model and prompt (default shown in the prompt).

The default extraction prompt suggests:

• Model: claude3_7_no_system
• Prompt: few_shot_def

You can modify these in the script or extend the modules in src/ to support more models/prompts.

Outputs

• Extraction results and artifacts are typically written under out/ (and/or within module-defined paths).
• Intermediate data and logs may also be placed there depending on module configuration.

Working With Notebooks

• metrics.ipynb contains analyses/evaluation routines to produce the final, published metrics. Open it in Jupyter:

jupyter notebook metrics.ipynb

Configuration

• Metadata path: edit METADATA_PATH in main.py (defaults to res/CJEU_complete.csv).
• Dataset directory: update the path checks in main.py if you prefer a different location than ../../res/CJEU_judgments.
• Model and prompt: adjust the variables in main.py or the corresponding module functions.

Troubleshooting

• If the dataset is not detected:
  • Ensure the judgments directory matches the expected path or update the path in main.py.
  • Ensure the metadata CSV exists at res/CJEU_complete.csv (or update METADATA_PATH).
• If imports fail:
  • Run from the project root so Python can resolve src.* imports.
  • Verify dependencies are installed via pip install -r requirements.txt.