CS URL Checker — Machine Learning URL Risk Classifier

A cybersecurity and machine-learning project that classifies URLs as benign or potentially malicious using lexical URL features and a Random Forest classifier.

Overview

CS URL Checker demonstrates how machine learning can identify suspicious URL patterns without opening the website.

The notebook:

1. Loads a labelled URL dataset.
2. Extracts lexical features from each URL.
3. Converts URL categories into benign and malicious classes.
4. Splits the dataset into training and testing sets.
5. Trains a Random Forest classifier.
6. Evaluates the model.
7. Accepts URLs interactively and returns a prediction with a risk score.

This is an educational project and should not be treated as a complete phishing-protection system.

Classification

The dataset categories are converted into two classes:

Original category	Model class
Benign	0 — Benign
Phishing	1 — Malicious
Malware	1 — Malicious
Defacement	1 — Malicious

Extracted Features

The model currently uses the following URL characteristics:

Feature	Description
url_length	Total number of characters in the URL
has_ip	Whether an IPv4-like address appears in the URL
num_digits	Number of numeric characters
num_special	Count of ., -, ?, & and = characters
https	Whether the URL uses the HTTPS scheme
domain_length	Length of the extracted domain name

These features describe URL structure only. They do not prove that a website is safe or malicious.

Tech Stack

• Python
• Jupyter Notebook
• pandas
• NumPy
• scikit-learn
• tldextract
• Regular expressions
• Python urllib.parse

Repository Structure

CS_URL-Checker/
├── CS_URL CHECKER.ipynb
├── LICENSE
└── README.md

The dataset is not included in the repository and must be downloaded separately.

Dataset Requirements

The notebook expects a file named:

malicious_phish.csv

Place it in the repository root, beside the notebook.

The CSV must contain these columns:

Column	Description
url	Website URL
type	benign, phishing, malware or defacement

Example:

url,type
https://example.com,benign
http://suspicious-example.test/login,phishing

Only use a dataset you are authorized to download and process. A commonly used source is the Malicious URLs Dataset available through Kaggle.

Installation

Clone the repository:

git clone https://github.com/Ankit-ku-panda/CS_URL-Checker.git
cd CS_URL-Checker

Create a virtual environment:

Windows PowerShell

python -m venv .venv
.venv\Scripts\Activate.ps1

Linux or macOS

python3 -m venv .venv
source .venv/bin/activate

Install the required packages:

python -m pip install --upgrade pip
pip install jupyter pandas numpy scikit-learn tldextract

Run the Notebook

Start Jupyter Notebook:

jupyter notebook

Open:

CS_URL CHECKER.ipynb

Select Run → Run All Cells.

When training is complete, enter a URL in the interactive cell:

Enter a website URL (or type 'exit'):

Enter exit to stop the input loop.

Example Usage

Enter a website URL (or type 'exit'): https://example.com

Analyzing URL: https://example.com
----------------------------------
SAFE: This website appears legitimate
Risk Score: <model-generated value> % dangerous

The exact result depends on the dataset, train/test split and trained model.

Model Workflow

Labelled dataset
      ↓
URL feature extraction
      ↓
Train/test split
      ↓
Random Forest training
      ↓
Accuracy and classification report
      ↓
Interactive URL prediction

Evaluation

The notebook prints:

• Accuracy score
• Precision
• Recall
• F1-score
• Support for each class

For more reproducible results, set a fixed random state:

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.2,
    random_state=42,
    stratify=y
)

model = RandomForestClassifier(
    random_state=42
)

Accuracy alone may be misleading when the classes are imbalanced. Recall for the malicious class and false-negative rate are especially important for a security classifier.

Security and Privacy

• The current notebook analyzes URL text locally.
• It does not intentionally open or visit submitted URLs.
• It does not download website content.
• It does not perform DNS, WHOIS or reputation lookups.
• Avoid clicking suspicious URLs used for testing.
• Do not include private URLs containing tokens, passwords or personal information.
• Treat every result as an estimate, not a security guarantee.

Current Limitations

• Lexical features alone cannot confirm whether a website is malicious.
• HTTPS does not guarantee that a site is trustworthy.
• The IPv4 regular expression is basic and may match invalid addresses.
• URL-shortening services and newly registered domains may be difficult to classify.
• The model can produce false positives and false negatives.
• The displayed probability is model confidence, not a calibrated real-world danger percentage.
• The notebook catches all prediction errors with a broad except, which can hide useful debugging information.
• Model quality depends heavily on dataset quality, freshness and class balance.

Future Improvements

• Add domain age and WHOIS features
• Add DNS and certificate analysis
• Integrate reputation services such as Google Safe Browsing or VirusTotal
• Detect shortened and encoded URLs
• Add character n-gram or deep-learning models
• Calibrate the probability score
• Add cross-validation and confusion-matrix visualization
• Save and load the trained model
• Build a command-line interface
• Build a browser extension or web interface
• Add automated tests and a reproducible training pipeline

Responsible Use

This project is intended for:

• Cybersecurity education
• Machine-learning experimentation
• Defensive URL analysis
• Authorized research

Do not use it to collect credentials, impersonate websites or support malicious activity.

Contributing

Contributions are welcome.

1. Fork the repository.

2. Create a branch:

   git checkout -b feature/your-feature

3. Commit your changes:

   git commit -m "Add your feature"

4. Push your branch:

   git push origin feature/your-feature

5. Open a pull request.

For bugs or suggestions, use the repository’s Issues page.

Author

Ankit Kumar Panda

• GitHub: Ankit-ku-panda
• Repository: CS_URL-Checker

License

This project is distributed under the MIT License.

---

If this project helped you, consider giving the repository a star.