Newshot

Ask HN: Did you buy hardware for Local AI?

Original text(via HN AI)
Ask HN: Did you buy hardware for Local AI?
2 points by ptitlaby 2 hours ago │ hide │ past │ favorite │ 1 comment
With the release of some new platforms having 128GB or more of unified memories, I am wondering if I should spend some money on those. I am looking for feedback and stories of people who bought hardware for local-first AI. - Did you identify a sweet spot in model size, and therefore hardware sizing? - Did you continue to use cloud / frontier models?

I am looking for feedback and stories of people who bought hardware for local-first AI.

  • Did you identify a sweet spot in model size, and therefore hardware sizing?
  • Did you continue to use cloud / frontier models?
Aurornis 2 hours ago [–] Primary question is: What are your goals and what will you use it for? I have a 128GB platform that I use for local LLMs. It's fun, but even when I'm pushing the limits of what it can hold, I have to be honest that it's not a replacement for a $20 Claude account. To put it in perspective, the best results I've been able to get out of my local system are still less than the mid-tier models from OpenAI and Anthropic (Terra or Sonnet). If privacy is your goal then that doesn't matter. A local LLM setup delivers that. If saving money is your goal, enjoy one of the frontier model subscription plans and wait for hardware prices to come down again. Before you buy anything, you should determine the best model that would run on that hardware. Then spend $20 on OpenRouter using that model for the hardest task you can think you would use it for. Then remember that the version of that model you run locally will be quantized, so quality will be even worse than that. For some people do some work, Qwen3.8-27B is all they need, which opens up a lot of options. For complex work, the gap between the open models that fit on consumer hardware and even the mid-tier models from a $20/month plan is large. You should aim to understand this gap and how it relates to your work before you spend $3000 or more on hardware for it. replyAurornis 2 hours ago [–] Primary question is: What are your goals and what will you use it for? I have a 128GB platform that I use for local LLMs. It's fun, but even when I'm pushing the limits of what it can hold, I have to be honest that it's not a replacement for a $20 Claude account. To put it in perspective, the best results I've been able to get out of my local system are still less than the mid-tier models from OpenAI and Anthropic (Terra or Sonnet). If privacy is your goal then that doesn't matter. A local LLM setup delivers that. If saving money is your goal, enjoy one of the frontier model subscription plans and wait for hardware prices to come down again. Before you buy anything, you should determine the best model that would run on that hardware. Then spend $20 on OpenRouter using that model for the hardest task you can think you would use it for. Then remember that the version of that model you run locally will be quantized, so quality will be even worse than that. For some people do some work, Qwen3.8-27B is all they need, which opens up a lot of options. For complex work, the gap between the open models that fit on consumer hardware and even the mid-tier models from a $20/month plan is large. You should aim to understand this gap and how it relates to your work before you spend $3000 or more on hardware for it. reply
Aurornis 2 hours ago [–] Primary question is: What are your goals and what will you use it for? I have a 128GB platform that I use for local LLMs. It's fun, but even when I'm pushing the limits of what it can hold, I have to be honest that it's not a replacement for a $20 Claude account. To put it in perspective, the best results I've been able to get out of my local system are still less than the mid-tier models from OpenAI and Anthropic (Terra or Sonnet). If privacy is your goal then that doesn't matter. A local LLM setup delivers that. If saving money is your goal, enjoy one of the frontier model subscription plans and wait for hardware prices to come down again. Before you buy anything, you should determine the best model that would run on that hardware. Then spend $20 on OpenRouter using that model for the hardest task you can think you would use it for. Then remember that the version of that model you run locally will be quantized, so quality will be even worse than that. For some people do some work, Qwen3.8-27B is all they need, which opens up a lot of options. For complex work, the gap between the open models that fit on consumer hardware and even the mid-tier models from a $20/month plan is large. You should aim to understand this gap and how it relates to your work before you spend $3000 or more on hardware for it. reply

I have a 128GB platform that I use for local LLMs. It's fun, but even when I'm pushing the limits of what it can hold, I have to be honest that it's not a replacement for a $20 Claude account. To put it in perspective, the best results I've been able to get out of my local system are still less than the mid-tier models from OpenAI and Anthropic (Terra or Sonnet).

If privacy is your goal then that doesn't matter. A local LLM setup delivers that. If saving money is your goal, enjoy one of the frontier model subscription plans and wait for hardware prices to come down again.

Before you buy anything, you should determine the best model that would run on that hardware. Then spend $20 on OpenRouter using that model for the hardest task you can think you would use it for. Then remember that the version of that model you run locally will be quantized, so quality will be even worse than that.

For some people do some work, Qwen3.8-27B is all they need, which opens up a lot of options. For complex work, the gap between the open models that fit on consumer hardware and even the mid-tier models from a $20/month plan is large. You should aim to understand this gap and how it relates to your work before you spend $3000 or more on hardware for it.

The text above was extracted automatically from the source page and may miss images, tables or code blocks. The original is authoritative.

Published by HN AI
This site only summarises and aggregates; copyright stays with the original author. Headlines and summaries follow the source — where they differ, the original wins.
Read the original ↗