PROBLEM
Testing capstone models online incurred high latencies and token bills.
BACK TO PORTFOLIO
AI INFRASTRUCTURE2026
GEMMA LOCAL API SERVER
A professional Python package hosting google/gemma-4-E2B-it locally with FastAPI, Hugging Face Transformers, and CUDA acceleration. Custom built for testing capstone features offline.
USERS
Capstone Dev Team
LAUNCH TIME
1 week
IMPACT
Offline testing setup
PERFORMANCE
Quantised 4-bit CUDA
Mini Case Study
BUILD
Configured an E2B FastAPI microservice to load quantization configurations locally.
RESULT
Achieved seamless offline test cycles for the entire capstone team.
Tech Stack
PythonFastAPIGemma-4CUDAPyTorchHugging Face