Switched from GPT-4 to a smaller open model for my text classifier last month

I was running a sentiment analysis tool for my local bookstore's reviews using GPT-4 and burning through $80 a month on API calls. Swapped to a fine-tuned Llama 3 8B on a rented GPU for $15 total and it actually beat GPT-4 on accuracy by 2 percent. Has anyone else found big savings by moving to smaller models for specific tasks?

3 comments

3 Comments

wade25016d ago

Switched my spam filter to a tiny DistilBERT model and it barely costs anything now lol

blakestone16d ago

Sadly, those tiny models still miss the clever spam that fools simpler keyword filters.

kimfisher16d ago

Oh man, I had a coworker who spent weeks training this tiny model on old Nigerian prince emails (you know the ones) and it still flagged a friend's birthday invitation as spam. Made me wonder if the clever spam is just getting smarter because we're training these things on the same old tricks.