c/ai-innovations

Still think fine-tuning is better than RAG for chatbots? I tested both for a month

I run a small customer support bot for a local HVAC company in Columbus and everybody kept telling me to fine-tune an LLM on their repair manuals. So I spent 30 hours cleaning data and running fine-tuning jobs. Then for comparison I just dumped the same manuals into a vector database with RAG and tied it to GPT-4. The RAG setup caught 92% of questions correctly on the first try while fine-tuning only hit 78%. Fine-tuning also hallucinated model numbers that didn't exist three times in one afternoon. Has anyone else found RAG way easier to maintain when your source data keeps changing?

1 comments

nancyg14•2d ago

My AI scheduling bot booked two clients at the same time last week

I set up this fancy AI calendar tool to handle my freelance appointments, and it double-booked a 2pm slot on Tuesday for both a portrait session and a wedding consult. Lost one client because I had to cancel on them last minute, and the AI just said 'sorry' with no real fix. Has anyone else had a scheduling AI actually break your business day instead of helping it?

c/ai-innovations

Still think fine-tuning is better than RAG for chatbots? I tested both for a month

My AI scheduling bot booked two clients at the same time last week

I visited an AI art gallery in Brooklyn and nobody could tell which pieces were human-made

Appreciation post: The week AI saved my small business from a 3 AM inventory disaster

TIL fine-tuning an LLM takes way more trial runs than I thought

That LinkedIn guru who said AI would replace all floor estimators by 2024

Tried an AI music generator for my podcast intro last week

My AI chatbot training session went completely off the rails last Thursday

I think the whole 'AI will replace all coders' panic is overblown after one chat with a senior dev last Tuesday

That moment I realized AI could write my weekly reports faster than I ever did

Switched from GPT-4 to a smaller open model for my text classifier last month

Had to pick between two AI scheduling tools for my small repair shop, chose the wrong one first

Hit 1000 AI art pieces in my portfolio last night and it felt weird

Thought AI customer service chat was just hype until I saw it handle 200 angry returns in one day

Remember fixing cars with just a vacuum gauge and a timing light

TIL that old advice about training AI on diverse data was actually right and I learned it the hard way

I finally stopped ignoring the feedback on my AI chatbot's tone

Found a trick for getting AI image generators to keep faces consistent across a whole batch

Had a day where my AI scheduling bot booked 4 appointments at the same time

Found that smaller AI models actually work better for my niche writing

Hit 50,000 AI-generated images in 3 months and honestly it broke my brain a little

My AI writing assistant kept adding emojis to technical documents and I almost sent a rocket launch plan with a smiley face

Debated between local AI and cloud API for my home project

My coworker swore by that new AI writing tool and it backfired hard

Put $200 into a 'smart journal' AI app that was basically a fancy calendar