Skip to content

About

AI-Enabled Phishing Detection and Real-Time Email and URL Security System

Resources

Stars

2 stars

Watchers

0 watching

Forks

Repository files navigation

PhishGuard: AI-Enabled Phishing Detection for Email, URLs and Web Pages

PhishGuard is a Chrome extension backed by a small local server. It checks Gmail emails, links before they open and web pages after they load, and shows a clear verdict (Safe / Suspicious / Phishing) with plain-language reasons. All scoring runs on your own computer: no email text, URL or page content is sent anywhere.

Detector Model Test F1 Test set
Email text fine-tuned DistilBERT (runs as ONNX int8) 0.992 11,132 emails
URL (before the page loads) character CNN + 22 URL features 0.934 30,146 URLs from unseen domains
Web page (after it loads) MLP + XGBoost on 53 page features, combined with the URL score by a logistic stacker 0.973 9,607 later-dated pages

Typical scan time on a laptop CPU: email 42 ms, URL 12 ms, page 121 ms (p95).


Contents

  1. How it works
  2. Repository layout
  3. Requirements
  4. Quick start: run PhishGuard with the trained models
  5. Get the dataset
  6. Train the models yourself
  7. Test and evaluate
  8. API reference
  9. Dashboard
  10. Settings and configuration
  11. Security and privacy
  12. Troubleshooting
  13. Datasets and references

1. How it works

 Chrome extension (Manifest V3)                         Local server (FastAPI, 127.0.0.1:8765)
 ─────────────────────────────                          ──────────────────────────────────────
 Gmail content script  ── subject, sender, body, links ─▶ /scan/email  DistilBERT (ONNX int8)
 Background worker     ── every URL before it opens ────▶ /scan/url    allowlist + char-CNN
 Page content script   ── HTML after the page loads ────▶ /scan/page   MLP + XGBoost + URL → stacker
                                                                │
        banner / badge / warning page  ◀── verdict + score + reasons ◀── rules + thresholds
  1. The extension sends what the user sees to the local server.
  2. The server cleans the input exactly as it was cleaned during training.
  3. A model gives a raw score between 0 and 1.
  4. Rules adjust the score using extra evidence (fake link text, trusted sender, shared hosting sites, very short emails).
  5. Two thresholds per detector (amber and red) turn the score into Safe, Suspicious or Phishing. The thresholds are chosen on validation data from a false-positive budget (1% red, 2% amber at the default sensitivity). The user can pick low, medium or high sensitivity.
  6. The extension shows a banner in Gmail, a badge on its icon (OK, !, !!, OFF) or a warning page before a risky site loads.

Popular domains (Tranco top 10,000, minus shared hosting sites such as github.io or docs.google.com) are answered from an allowlist without running a model.


2. Repository layout

aiphishingextension/
├─ phishguard/        shared Python package, used by training AND the server
│                     (text cleaning, URL normalisation, 53 page features, models, scorers, fusion, reasons)
├─ server/            FastAPI app (app.py, inference.py, schemas.py, history.py) + tests/
├─ training/          train_email_baseline · train_email_transformer · train_url · train_page · train_fusion · export_onnx
├─ data/
│  ├─ scripts/        download and build the datasets
│  ├─ allowlist_top10k.txt  brands.json  user_content_hosts.txt   (small, included)
│  ├─ raw/            original downloads          (NOT in the repo, see section 5)
│  └─ processed/      ready-to-train parquet files (NOT in the repo, see section 5)
├─ models/            trained models              (NOT in the repo, see section 4)
├─ extension/         Chrome extension source (React + Vite); extension/dist = ready-to-load build
├─ dashboard/         React dashboard source; dashboard/dist is served at /dashboard/
├─ eval/              latency benchmark, fresh-phishing check, end-to-end extension test, report
├─ demo/              safe local demo pages and sample emails
├─ scripts/           setup.ps1 · start_server.ps1 · install_autostart.ps1 · train_all.ps1 · train_recent_cpu.ps1
├─ requirements-server.txt   packages needed to RUN the server
└─ requirements-train.txt    packages needed to build data and TRAIN (includes the server ones)

Label convention everywhere: 1 = phishing, 0 = legitimate.


3. Requirements

What Needed for Notes
Windows 10/11, Linux or macOS everything the .ps1 helper scripts are Windows-only; every step also has a plain python -m ... command
Python 3.12 or 3.13 (64-bit) server and training
Google Chrome or Microsoft Edge the extension
Node.js 20+ and npm only if you change the extension or dashboard code extension/dist and dashboard/dist are already built
NVIDIA GPU with CUDA 12.x only for training training on CPU works but is much slower
Disk ~3 GB to run, ~8 GB to train

4. Quick start: run PhishGuard with the trained models

4.1 Get the code

git clone https://github.com/<your-account>/<your-repo>.git
cd <your-repo>

4.2 Create the Python environment

Windows (PowerShell)

py -3.12 -m venv .venv
.\.venv\Scripts\python.exe -m pip install --upgrade pip
.\.venv\Scripts\python.exe -m pip install torch --index-url https://download.pytorch.org/whl/cpu
.\.venv\Scripts\python.exe -m pip install -r requirements-server.txt

Linux / macOS

python3 -m venv .venv
.venv/bin/python -m pip install --upgrade pip
.venv/bin/python -m pip install torch --index-url https://download.pytorch.org/whl/cpu
.venv/bin/python -m pip install -r requirements-server.txt

The CPU build of PyTorch is enough to run the server. On Windows, scripts\setup.ps1 -ServerOnly does all of the above.

4.3 Download the trained models

Download phishguard-models-v2.zip from the project's Releases page: https://github.com/AYUS2307/PhishGuard/releases

Unzip it in the project folder so that you get:

models/
├─ email_distilbert_onnx_int8/   email model used by the server (ONNX int8)
├─ email_distilbert/             same model in PyTorch format (only needed for retraining / evaluation)
├─ url_charcnn.pt
├─ page_mlp.pt
├─ page_xgb.json
├─ fusion.json
├─ thresholds.json
└─ metrics.json

4.4 Start the server

# Windows
powershell -ExecutionPolicy Bypass -File scripts\start_server.ps1
# any OS
.venv/bin/python -m uvicorn server.app:app --host 127.0.0.1 --port 8765

Wait for Application startup complete, keep the window open, then check:

Optional (Windows): start the server automatically at login with powershell -ExecutionPolicy Bypass -File scripts\install_autostart.ps1 (remove with -Remove).

4.5 Load the extension

  1. Open chrome://extensions (or edge://extensions).
  2. Turn on Developer mode.
  3. Click Load unpacked and select the extension/dist folder.
  4. Pin the PhishGuard icon. The popup should say the server is connected. OFF on the badge means the server is not running.

4.6 Try it safely

cd demo\sites
..\..\.venv\Scripts\python.exe -m http.server 8001
  1. Right-click the PhishGuard icon → Options → turn on Demo mode (needed to scan localhost pages).
  2. Open http://localhost:8001/sbi_kyc.html, microsoft_login.html or parcel_fee.html → red warning.
  3. Open http://localhost:8001/recipe_blog.html or college_news.html → no warning.
  4. Open Gmail: every opened email gets a banner. Sample emails to paste into the dashboard's Try it tab are in demo/emails.json.

5. Get the dataset

The datasets are too large for GitHub. Download phishguard-data-v2.zip from:

Dataset link: https://drive.google.com/file/d/1z--mmrVI8uWnWSzx4JQaf5froVvV-dAQ/view?usp=sharing

Unzip it in the project folder. It creates:

data/processed/
├─ emails.parquet            recent emails (public mailing lists + Nazario + phishing_pot), split by date
├─ emails_older.parquet      older mixed corpus (CEAS-08, Enron, Ling, Nazario, Nigerian Fraud, SpamAssassin + HF set)
├─ emails_kuladeep.parquet   LLM-generated emails (test only, never trained on)
├─ emails_pot.parquet        honeypot phishing emails (test only)
├─ urls.parquet              300,000 URLs, split by domain
├─ url_model_features.npy    cached 22 URL features for urls.parquet
├─ pages.parquet             54,630 pages as 53 features, split by domain and time
├─ tranco_test_domains.txt   legitimate domains kept for testing
├─ fresh_legit_urls.txt      fresh legitimate deep links (false-alarm check)
└─ bench_pages.parquet       pages used by the latency benchmark
data/raw/fresh/openphish_YYYYMMDD.txt   live phishing URLs (test only)
File Rows Phishing / legitimate
emails.parquet 17,160 6,864 / 10,296
emails_older.parquet 85,569 39,887 / 45,682
emails_kuladeep.parquet 9,127 6,000 / 3,127
urls.parquet 300,000 150,000 / 150,000
pages.parquet 54,630 22,904 / 31,726

Every parquet file has a label column (1 = phishing) and a split column (train / val / test), so the same split is used by everyone. You can also rebuild everything from the original sources (section 6.3).


6. Train the models yourself

Install the training packages first (use the CUDA build of PyTorch if you have an NVIDIA GPU):

.\.venv\Scripts\python.exe -m pip install torch --index-url https://download.pytorch.org/whl/cu124
.\.venv\Scripts\python.exe -m pip install -r requirements-train.txt
.\.venv\Scripts\python.exe -c "import torch; print('CUDA:', torch.cuda.is_available())"

On Windows, scripts\setup.ps1 does this and also builds the extension and dashboard.

All commands below are run from the project folder with the virtual environment's Python (.\.venv\Scripts\python.exe on Windows, .venv/bin/python elsewhere). Shown here as python.

6.1 Train from the provided dataset (recommended)

With the dataset from section 5 unzipped:

# Command What it does Time (laptop RTX GPU)
1 python -m training.train_email_baseline TF-IDF + logistic regression baseline ~3 min
2 python -m training.train_email_transformer --combined fine-tunes DistilBERT on the older + recent emails (the released recipe), keeps the best checkpoint ~45 min
3 python -m training.train_url --epochs 15 character CNN (+ XGBoost baseline), best on validation is kept ~18 min
4 python -m training.train_page page MLP + XGBoost (+ logistic regression / random forest baselines) ~2 min
5 python -m training.train_fusion page stacker, email combiner and all thresholds → models/fusion.json, models/thresholds.json ~2 min
6 python -m training.export_onnx DistilBERT → ONNX int8 for the server, with a parity check ~5 min
7 python -m eval.run_all writes docs/report/metrics.md and checks the targets seconds

Then start the server (section 4.4); it loads the new models from models/.

Settings used for the released models

Model Settings
Email distilbert-base-uncased, max 256 tokens, AdamW lr 2e-5, batch 32 (16 × 2), 6% warm-up, weight decay 0.01, fp16, 4 epochs (best: 3)
URL char embedding 32, Conv1D widths 3/5/7 × 128 filters, 22 extra features, AdamW lr 2e-3, one-cycle, batch 512, 15 epochs (best: 12)
Page MLP 53 → 256 → 128 → 1, BatchNorm + dropout 0.3, AdamW lr 1e-3, cosine, early stopping (best epoch 40)
Page XGBoost depth 7, learning rate 0.03, subsample 0.8, early stopping (843 trees kept)
Stacker logistic regression on the URL, MLP and XGBoost logits, fitted on validation pages

Note 1: email training data. --combined reproduces the released model's data: the older corpus (emails_older.parquet) plus the recent emails (emails.parquet) counted twice, with every email that appears in a test set removed (94,163 training, 4,715 validation and 11,132 test emails). It also sets 4 epochs, 6% warm-up and two validation runs per epoch. Mixing the two periods made the model far more robust on unseen mail (false alarms on an unseen corpus fell from 25% to 0.74%). Add --dry-run to check the data sizes without training. Without --combined, the script trains on the recent emails only.

Note 2: email thresholds. train_fusion picks every threshold from the false-positive budget. The email model is very confident, so this gives email lines of 0.05 (amber) and 0.5 (red). For real inboxes we set them by hand to 0.5 / 0.9 in models/thresholds.json (high sensitivity 0.3 / 0.8, low 0.7 / 0.95). Change them back after retraining if you want the same behaviour.

Useful options: --bs 8 --grad-accum 4 if the GPU runs out of memory; --max-len 128 --subsample 20000 --epochs 2 for a quick CPU-only run.

6.2 Train without a GPU

powershell -ExecutionPolicy Bypass -File scripts\train_recent_cpu.ps1 -SkipDownload runs the whole pipeline on CPU with smaller settings (shorter emails, fewer rows and epochs). Expect lower email accuracy than the released model.

6.3 Rebuild the dataset from the original sources

Only needed if you want fresh data. The scripts download public data, clean it, remove duplicates and make the splits. Kaggle files need a login, so download them by hand first:

Kaggle dataset Put the files in
Phishing Email Dataset data/raw/emails/naser/ (CEAS_08.csv, Enron.csv, Ling.csv, Nazario.csv, Nigerian_Fraud.csv, SpamAssasin.csv)
Phishing and Legitimate Emails (LLM-generated) data/raw/emails/kuladeep/ (optional, test only)

Then:

# Command What it does
1 python -m data.scripts.download_all Hugging Face emails, PhreshPhish pages, PhiUSIIL, Tranco, phishing_pot, OpenPhish feed
2 python -m data.scripts.download_recent recent emails (public mailing lists, Nazario) and active phishing URLs
3 python -m data.scripts.build_allowlist Tranco → data/allowlist_top10k.txt
4 python -m data.scripts.build_email older email corpus → clean, deduplicate, split
5 python -m data.scripts.build_email_recent recent emails → split by date
6 python -m data.scripts.crawl_legit_pages legitimate pages from Tranco sites
7 python -m data.scripts.build_page PhreshPhish + crawled pages → 53 features, split by domain and time
8 python -m data.scripts.crawl_legit_urls legitimate deep links from Tranco sites
9 python -m data.scripts.build_url merges the URL sources, max 50 per domain, split by domain

build_email and build_email_recent both write emails.parquet; rename the output of build_email to emails_older.parquet before running build_email_recent (the --combined training option reads it under that name). After step 9, continue with section 6.1. scripts\train_all.ps1 runs the original (older-data) pipeline end to end.

Data-quality rules applied by the scripts

  • Duplicates are removed before splitting; any training email whose text appears in a test set is dropped.
  • Emails: HTML → text, URLs → [URL], years → [YEAR], dataset fingerprints (Enron routing lines, forwarded headers, list footers) removed, subject and body joined as subject | body.
  • URLs: lower-cased, scheme and www. removed, at most 50 URLs per domain per class, split by registrable domain.
  • Pages: split by domain, and the test pages are later in time than the training pages.

7. Test and evaluate

python -m pytest server/tests -q                       # unit and API tests (37 tests)
python -m eval.bench_latency                           # p50 / p95 per endpoint (server must be running)
python -m eval.fetch_fresh                             # live OpenPhish catch rate + fresh legitimate links
python -m eval.e2e_extension --browser msedge          # loads the real extension in Edge/Chrome (server running)
python -m eval.run_all                                 # metrics report → docs/report/metrics.md

The end-to-end test uses a mocked Gmail page and local demo pages only; it never opens a real phishing site.


8. API reference

Base URL http://127.0.0.1:8765, interactive docs at /docs. Every scan request may include "sensitivity": "low" | "medium" | "high" (default medium).

Method and path Body Response
GET /health status, model versions, thresholds
POST /scan/url {url} verdict, score, allowlisted, reasons, latency
POST /scan/page {url, title, html} verdict, score, URL / MLP / XGBoost scores, reasons, latency
POST /scan/email {subject, sender_name, sender_email, body_text, links:[{href, text}]} verdict, score, text score, per-link results, reasons, latency
GET /allowlist/check?url= allowlisted, reason
POST /feedback · GET /feedback {kind, target, label} "trust" / "mark as phishing" decisions
GET /metrics · GET /history · GET /stats · DELETE /history data for the dashboard

Limits (HTTP 413): email body > 50,000 characters, HTML > 1.5 MB, > 200 links. Invalid fields → 422.

curl -s -X POST http://127.0.0.1:8765/scan/url -H "content-type: application/json" -d "{\"url\": \"http://paypa1-secure-login.xyz/verify\"}"

9. Dashboard

Open http://127.0.0.1:8765/dashboard/ while the server is running.

Tab Shows
Overview scans per day by verdict, latency, latest warnings
Try it test any email, URL or HTML against the models (not logged)
Models & metrics F1 comparison, thresholds, all metric tables
History local scan log with filters and a clear button

To work on the dashboard: cd dashboard && npm install && npm run dev (calls are proxied to the server).


10. Settings and configuration

Extension options (right-click the icon → Options): on/off, block risky sites before they load, sensitivity (low / medium / high), trusted sites and senders, server URL, keep history, demo mode.

Server environment variables

Variable Default Meaning
PHISHGUARD_BACKEND onnx pytorch uses the PyTorch email model instead of ONNX int8
PHISHGUARD_DEVICE cpu cuda runs the PyTorch email model on the GPU

Files you can edit: models/thresholds.json (amber and red lines per detector and sensitivity), data/brands.json (brand names and their official domains), data/user_content_hosts.txt (shared hosting sites that are never allowlisted).

Rebuilding the extension or dashboard after code changes:

cd extension && npm install && npm run build      # then press the reload button in chrome://extensions
cd dashboard && npm install && npm run build

11. Security and privacy

  • The server listens on 127.0.0.1 only; a host-header check blocks DNS-rebinding, and CORS allows only the extension.
  • Request size limits and bounded regular expressions protect against hostile input.
  • The extension shows email and page text as plain text inside a Shadow DOM, never as raw HTML.
  • History keeps only verdict, score, host or sender domain and latency. It can be turned off and cleared.
  • If the server is down, the extension shows OFF and never blocks browsing.
  • Test phishing only with the local demo pages, or in a separate browser profile with no logged-in accounts.

12. Troubleshooting

Problem Fix
Badge shows OFF Start the server (section 4.4) and check /health.
/health shows a model as missing models/ is incomplete: unzip the models release again or retrain that model.
No banner in Gmail Reload the Gmail tab after (re)loading the extension.
Nothing happens on localhost pages Turn on Demo mode in the extension options.
torch.cuda.is_available() is False Install the CUDA build of PyTorch (.../whl/cu124) and update the NVIDIA driver.
CUDA out of memory python -m training.train_email_transformer --bs 8 --grad-accum 4
py not found (Windows) Use python instead of py.
PowerShell blocks the scripts Run them with powershell -ExecutionPolicy Bypass -File ... as shown.
Extension changes not visible cd extension && npm run build, then reload the extension and the tab.

13. Datasets and references

Datasets (each used under its own licence, for research and education)

Papers behind the models

  • Vaswani et al., Attention Is All You Need, NeurIPS 2017. arXiv:1706.03762
  • Sanh et al., DistilBERT, a distilled version of BERT, 2019. arXiv:1910.01108
  • Zhang, Zhao and LeCun, Character-level Convolutional Networks for Text Classification, NeurIPS 2015. arXiv:1509.01626
  • Le et al., URLNet: Learning a URL Representation with Deep Learning for Malicious URL Detection, 2018. arXiv:1802.03162
  • Chen and Guestrin, XGBoost: A Scalable Tree Boosting System, KDD 2016. arXiv:1603.02754
  • Wolpert, Stacked Generalization, Neural Networks, 1992
  • Sundararajan et al., Axiomatic Attribution for Deep Networks (Integrated Gradients), ICML 2017. arXiv:1703.01365
  • Lundberg et al., Consistent Individualized Feature Attribution for Tree Ensembles (TreeSHAP), 2018. arXiv:1802.03888

B.Tech. project, Computer Science and Engineering Department, MNNIT Allahabad.

About

AI-Enabled Phishing Detection and Real-Time Email and URL Security System

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages