Assignment 1

[tutoringjedi]

What changes are you trying to make? (e.g. Adding or removing code, refactoring existing code, adding reports)

Completing Assignment 1 and the three accompanying labs for the Deep Learning module:

• assignment_1.ipynb: built a linear baseline model and a CNN on Fashion-MNIST, ran controlled experiments varying the number of convolutional filters and testing dropout on/off, then trained a final tuned model combining the best settings from those experiments.
• lab_1.ipynb: built a feedforward network in Keras on the digits dataset, ran optimizer experiments (learning rate, momentum, Adam), and tested different weight initialization schemes (very small, very large, and all-zero) to see their effect on training.
• lab_2.ipynb: implemented backpropagation from scratch in NumPy — softmax, negative log-likelihood, a logistic regression model, and a 1-hidden-layer neural network — then extended it with a from-scratch 2-hidden-layer network and ran a hyperparameter search over learning rate and hidden layer size.
• lab_3.ipynb: built a Keras-based recommender system on the MovieLens 100k dataset using user/item embeddings, first as a simple dot-product model and then as an improved MLP-based model (concatenated embeddings + Dense layers + Dropout) to reduce test MAE.

What did you learn from the changes you have made?

• How initialization scale interacts with activation choice: both very small and very large initial weights cause vanishing gradients in a tanh/sigmoid network — small weights barely move the gradient, while large weights saturate the activation — and all-zero initialization fails completely due to neuron symmetry that no optimizer (not even Adam) can break.
• Why the softmax + cross-entropy gradient (y_pred - y_true) specifically requires a softmax output layer, and doesn't generalize to sigmoid outputs.
• That a single hidden layer was already enough capacity for the small digits dataset — adding a second hidden layer made optimization harder (smaller gradients through an extra sigmoid layer) without buying any extra accuracy.
• That more model capacity (more Conv2D filters, more layers) doesn't linearly translate to better performance, and needs to be paired with regularization (dropout) to actually generalize better rather than just overfit.

Was there another approach you were thinking about making? If so, what approach(es) were you thinking of?

For the recommender system, I considered adding user/item bias terms to the dot-product model (a classic matrix-factorization-with-biases approach) as a lighter-weight alternative to the full MLP redesign. I went with the MLP version since it directly addresses the "add another layer"/"add dropout" exercise prompts, but the bias-term approach would be worth comparing in a follow-up.

Were there any challenges? If so, what issue(s) did you face? How did you overcome it?

• Had a TensorFlow/kernel mismatch in my local Jupyter environment (Anaconda vs. a project .venv), which I resolved by making sure the notebook kernel pointed at the same environment where TensorFlow was actually installed.
• In lab_2, the NeuralNet class's forward_output comment said to apply a sigmoid, but the provided grad_loss gradient formula only holds for a softmax output — resolved by using softmax there and adding a comment explaining the mismatch.
• Ran into a NameError/unclosed-parenthesis issue in a couple of the optimizer experiment cells in lab_1 from a missing closing ) — fixed by balancing the parens.

A reference to a related issue in your repository (if applicable)

N/A

Checklist

• I can confirm that my changes are working as intended

[github-actions]

Hello, thank you for your contribution. If you are a participant, please close this pull request and open it in your own forked repository instead of here. Please read the instructions on your onboarding Assignment Submission Guide more carefully. If you are not a participant, please give us up to 72 hours to review your PR. Alternatively, you can reach out to us directly to expedite the review process.