Lung Cancer Prediction
CNN Approach

Introduction

CT scans are a primary tool in radiology for identifying lung abnormalities. Lung cancer is one of the highest-mortality diseases globally — classified into SCLC (Small Cell Lung Cancer) and NSCLC (Non-Small Cell Lung Cancer), with further subtypes including adenocarcinoma in situ (ISA), squamous cell cancer (SCC), and invasive adenocarcinoma (IA).

This project trained two CNN architectures to classify CT scans into three categories: Normal, Benign, and Malignant — automating a task that currently depends heavily on experienced radiologists.

The Dataset

Data comes from the IQ-OTH/NCCD dataset, collected at an oncology hospital in Iraq in 2019 over 3 months. It contains:

• 1,190 images from 110 patients
• 40 patients diagnosed as Malignant
• 15 patients diagnosed as Benign
• 55 patients with Normal lungs
• Image resolution: 512×512 px

Noise was present in the form of CT machine artifacts and patient clothing/blankets visible in some scans — addressed through preprocessing transformations.

Approach & Data Engineering

Images were scaled to 244×244 px. Two preprocessing strategies were tested:

• Batch 1 — saturation +100%, brightness −100% to suppress noise and emphasize lung tissue
• Batch 2 — original images, no transformation

Data was split into training, validation, and test sets using a random split. Data augmentation (rotation, saturation, etc.) was available as a fallback but not needed given the dataset size.

Metrics Strategy

Recall was chosen as the primary metric — in a medical context, false negatives (missing cancer) are far more costly than false positives. Priorities:

1. Malignant cancer recall — highest priority
2. Benign cancer recall
3. Normal recall

Model 1

Architecture: convolutional + pooling layers followed by fully connected layers. Batch 1 (saturation/brightness transformed) outperformed Batch 2 on validation — the preprocessing effectively removed noise and emphasized lung features.

Model 2

A deeper architecture designed to extract features purely from raw images via deep learning, without relying on engineered transformations:

• 5 convolutional layers + 5 pooling layers + 2 fully connected layers
• First 2 conv layers: kernel size 3 (fine feature extraction)
• Last 3 conv layers: kernel size 5 (advanced features, larger receptive field)
• Optimizer: RMSprop
• Loss function: Sparse cross-entropy
• Train/test split: 80/20

Result: 100% accuracy — 10 randomly selected images from each category were correctly classified by the model.

Discussion & Takeaways

• Preprocessing matters. Saturation + brightness transformations in Model 1 significantly improved generalization by removing noise artifacts from CT equipment and patient clothing.
• Deeper = better (when done right). Model 2's deeper architecture with larger kernel sizes in later layers captured finer details, reaching perfect accuracy on the test set.
• Recall over accuracy for medical AI. A model with lower overall accuracy but high malignant recall is preferable — missing a cancer diagnosis has life-or-death consequences.
• Balanced data still needs noise handling. Even with a reasonably balanced dataset, artifact noise in CT images required active treatment to prevent the model from learning spurious patterns.

Conclusion

Both models demonstrated that deep learning is a viable path for automated lung cancer detection from CT scans. Model 2's architecture achieved perfect classification on the handpicked test set. Future work would validate on larger, more diverse datasets and explore additional augmentation techniques to improve robustness across different scanning conditions and patient demographics.