Pocket Data Science IV: Tackling Kaggle MNIST on Android with Antigravity CLI — From Zero-Parameter Baselines to Top 47% Subspace SVM

All data loading, model fitting, validation, and Kaggle submissions were executed entirely on a consumer smartphone:

• Host Architecture: Android 14 running Termux with a Debian userspace via PRoot Distro on 64-bit ARM (aarch64).
• Agentic CLI: Google Antigravity CLI (agy) pair-programming in bash, managing background tasks, and verifying data pipelines.
• Environment: Python 3.14 with pandas, numpy, scikit-learn, and the official kaggle CLI.
• Constraints: Operating within smartphone RAM and thermal envelopes requires algorithmic efficiency (e.g., avoiding raw O(N²) distance calculations across 42k × 28k × 784 floating-point matrices).

Across Day 1, we utilized all 5 available daily submissions to establish clear empirical milestones against the 862 active teams on the public leaderboard:

Before training parameterized models, baseline tests define the floor of predictability:

• Empirical Random Guessing (10.14%): Drawing randomly according to class frequencies in the training set matches the theoretical expectation for 10 balanced classes (~10%).
• Constant Majority Class (11.40%): Digit 1 is slightly more frequent than others (4,684 of 42,000 samples, or 11.15%). Predicting 1 for all 28,000 test images yielded 11.40% on Kaggle.
• Nearest Centroid Classifier (81.45%): By simply calculating the element-wise arithmetic mean of all 4,200 training images per digit class, we obtain 10 average “ghost templates”. Classifying each test image to the template with highest cosine similarity requires zero parameter tuning or gradient updates, yet hits 81.45% accuracy, demonstrating the geometric separation already inherent in raw pixel alignments.

Moving to machine learning algorithms revealed the threshold between linear and non-linear classification in 784 dimensions:

• Multinomial Softmax Regression (92.09%): Training a linear multinomial logistic regression classifier via L-BFGS sets 10 separating hyperplanes. It achieves 92.09%, illustrating the limit of linear separability without feature expansion or spatial convolutions.
• Dimensionality Reduction via PCA: Full k-Nearest Neighbors on raw pixels is prohibitive on phone CPUs (over 1.1 billion pairwise Euclidean distance calculations). By applying Principal Component Analysis (PCA) to compress 784 pixels to 55 components (preserving 84.0% variance), inference dropped to just 8 seconds.
• Neighborhood + Tree Ensembling: Combining distance-weighted k-NN (k = 5) with an Extra Trees Classifier (150 trees) on raw pixel partitions raised validation accuracy to 97.96% and public leaderboard accuracy to 97.58%, vaulting past 235 teams to Rank #571.

To push beyond classical decision boundaries without exceeding smartphone hardware constraints, we designed Experiment 06: a tri-blend ensemble pairing a 235k-parameter Deep Multi-Layer Perceptron (MLP(256, 128)) with our distance-weighted k-NN(5) and ExtraTrees(150) models.

On a 5,000-sample stratified holdout, the Tri-Blend achieved 98.34% accuracy (+0.38% over Exp 05), cutting classification error by 18.6%. Here is the exact training and inference computational profile measured directly on our 8-core ARM64 Android device:

• The Dual-Axis 126,000-Sample Breakthrough: In Experiment 10, tripling the training partition to 126,000 samples by combining both horizontal (±1px on the X-axis) and vertical (±1px on the Y-axis) translational jitter provided complete 2D shift-invariance. Fitting the entire 126,000-sample pipeline (PCA + RBF-SVM) took 3.4 minutes on phone CPU and drove our Kaggle score to 0.98792 (Rank #406 / Top 47.05%, beating 457 teams)!
• The Latent Space Regularization Win: Projecting images onto a 55-dimensional orthogonal subspace did not just accelerate training—it acted as an optimal low-pass filter against boundary pixel noise, achieving 98.60% validation accuracy versus 98.24% on uncompressed 784 pixels while fitting 17× faster (7.2s vs 148.6s).
• The Dual Meta-Ensemble Synergy: In Experiment 08, weighting the calibrated RBF-SVM at 90% and Deep MLP at 10% demonstrated how soft neural representations resolve ambiguous edge cases without disrupting primary maximum-margin decision planes.

With Experiment 10 scoring 0.98792, we officially breached the Top 50% cutoff (0.98739, Rank #431) to claim Rank #406 out of 863 active teams worldwide, outperforming 457 competitive teams.

• Zero GPU Resources: The entire progression—from empirical random guessing (0.10139) to classical ensembles (0.97578) and 126k dual-axis augmented subspace SVM (0.98792)—was engineered, trained, and submitted entirely on an Android smartphone CPU via Termux and Antigravity CLI.
• Looking Ahead to Day 3: With all 5 daily submissions successfully exhausted for Day 2, our next frontier is custom lightweight Convolutional Neural Networks (LeNet-5 & Modern Compact ResNets) built directly on PyTorch ARM64 to break into the Top 20%.