https://hotzbehofsits.com/tags/classification/Recent content in Classification on Christian Hotz-BehofsitsHugoen-usTue, 28 Jul 2026 00:00:00 +0000https://hotzbehofsits.com/post/local_llm_inference/Tue, 28 Jul 2026 00:00:00 +0000https://hotzbehofsits.com/post/local_llm_inference/<p>In a recent project I had 1,500 tweets and a codebook with 16 categories. There were three ways to get labels: code them by hand, send them to a commercial API, or run an open-weight model on our own GPU. I went with the third one, and this post is what I wish someone had told me before I started.</p> <p>The reasons for staying local are not ideological. First, the data never leaves the machine. Second, I have already the hardware and there is no per-token cost, so you can iterate on the prompt fifty times instead of five, and prompt iteration is where the accuracy comes from. Third, a pinned model version at <code>temperature=0</code> is reproducible, which turns the classification step into something you can ship in a replication package rather than a black box you apologize for in a footnote.</p>