Loading…
4
Warning: figured local LLMs were useless on my 8GB laptop, until llama.cpp proved me wrong last Tuesday
I kept telling people you need a $2k GPU for any real local AI, then I ran a 7B model with llama.cpp on my 3 year old ThinkPad and it handled a full page of text in about 9 seconds, so now I owe my buddy Dave an apology for that whole argument at his cookout.
1 comments
Log in to join the discussion
Log In1 Comment
susan_adams25d ago
Totally get the cookout thing, been there with my brother in law who swears you need a 4090 for everything. Tried llama.cpp on my old XPS last month and yeah, it just chugs along fine once you get the settings right. Poor Dave probably didn't deserve that one lol.
1