Blog on Christian Hotz-Behofsits

Running a Zero-Shot Classifier on Your Own GPU

Tue, 28 Jul 2026 00:00:00 +0000

In a recent project I had 1,500 tweets and a codebook with 16 categories. There were three ways to get labels: code them by hand, send them to a commercial API, or run an open-weight model on our own GPU. I went with the third one, and this post is what I wish someone had told me before I started.

The reasons for staying local are not ideological. First, the data never leaves the machine. Second, I have already the hardware and there is no per-token cost, so you can iterate on the prompt fifty times instead of five, and prompt iteration is where the accuracy comes from. Third, a pinned model version at temperature=0 is reproducible, which turns the classification step into something you can ship in a replication package rather than a black box you apologize for in a footnote.

DiD & friends: An overview of tools and papers

Thu, 29 Jun 2023 00:00:00 +0000

Last updated: July 2026.

This post provides an up-to-date overview of Difference-In-Difference tools, papers, and other links. What is important here is not the timeliness but the relevance. While there are new methods daily, only a few are used in practice (or demanded by reviewers). The idea is that not only the advantages of each method are given here, but also the disadvantages. Unfortunately, disadvantages of individual methods (e.g. scalability to larger data sets) are rarely communicated.

Applying Cosine Similarity on a Large Scale

Fri, 30 Sep 2022 00:00:00 +0000

I wrote my first blog post about cosine similarity back in 2019 when the pandemic was out of sight, and most marketing people were unaware of “representational learning.” However, times have changed, and nowadays, many quantitative marketing papers apply word2vec, prod2vec, or similar approaches.

Thanks to gensim, training such a model is straightforward. For example, one can learn the representations of tracks within a few lines of python code:

from gensim.models import Word2Vec
from gensim.models.word2vec import LineSentence

model = Word2Vec(
 sentences=LineSentence('./data/tmp_isrc_training.txt'), 
 vector_size=50, window=99, epochs=5, min_count=1, workers=8,
 sg=1, negative=10
)

Multi-hot encoding in JAX

Fri, 19 Aug 2022 00:00:00 +0000

JAX offers a range of practical functions, also for data preparation. One of these is jax.nn.one_hot, which performs a classic one-hot encoding. Unfortunately, I was not able to find a suitable multi-hot equivalent for multi-label applications. However, it is also quite easy to implement the functionality directly in JAX:

import jax
import jax.numpy as jnp
from functools import partial

@partial(jax.jit, static_argnames=("num_classes"))
def multi_hot(labels, num_classes: int):
 return jnp.take(jnp.eye(num_classes), jnp.array(labels), axis=0).sum(axis=0)

Multinomial Logistic Regression in JAX

Fri, 29 Jul 2022 00:00:00 +0000

Classifications are a classic machine learning problem we can tackle using logistic regression. If we distinguish between more than two classes, we call it a multinomial logistic regression. In this post, I will show how this can be done using JAX based on the well-known Fisher’s Iris dataset (every R user should be familiar with this one).

First, we have to load the required libraries and load the data. Since this is a classification, we have a set of predictors (aka. features) and a label for each sample.

SQL Multi-Substring Search in Yandex Clickhouse

Sat, 16 Jan 2021 10:27:14 +0100

Nowadays, also analytical databases are more frequently used to search for substrings or patterns in character strings. Thus, most databases provide a like operator or more sophisticated regex functions. However, especially long regex patterns or many of them can be challenging. In a recent project, I had to search for not one, but for 150 substrings at the same time. Thankfully, my database of choice (Yandex Clickhouse) offers several possibilities to handle this:

Bigquery: Least but not NULL

Thu, 14 Jan 2021 12:42:57 +0100

Let’s start with a simple SQL statement that returns the smallest value from each of several columns (similar to min only not across observations, but columns):

SELECT LEAST(date1, date2, date3) as min_date FROM tbl

In BigQuery it works nicely till you encounter NULL values, then it will always pick the NULL. Unfortunately it does not provide a IGNORE NULLS option such as ARRAY_AGG or others do. However, similar behaviour can be achieved with the following UDF:

Merge large CSV files with headers

Sat, 09 Jan 2021 13:54:18 +0100

If you have ever worked with classic data warehousing tools, you may already know the problem: The CSV export splits the output into different files, often with one file containing no more than 1 million observations. However, for analysis purposes, this format is not always optimal and must be merged into a single file. In bash this is not a big problem with cat, but only if there is no header present. The BigQuery CSV table export, for example, adds a header to each CSV file and this must be taken into account when merging.

Parallel JSONLines in R

Wed, 06 Jan 2021 12:37:52 +0100

JSON is becoming more and more widely used in data lakes, especially as a replacement for CSV. In particular, JSONLines where one JSON object is stored per line is to be mentioned here. Although the format has some disadvantages (i.e., overhead, supports only few data types), the advantages mostly outweigh them. For example, the individual lines are easier to read by humans and the format is much more standardized. In addition, each line has its own JSON object that can be processed independently. The overhead caused by the repeating field names can be well managed with compression (e.g., gzip, zstd, brotli).

Interpolate missing values in SQL

Sun, 24 May 2020 12:37:24 +0200

In this post I demonstrate how missing values can be interpolated in SQL. The code is written for BigQuery and I provide a user defined function, however, similar solutions may exist for other databases. Although, SQL may not be the first choice to do such tasks, it is possible. Before one can replace missing values, you have to have missing values. At least in my applications, often the whole rows are missing and not just selective fields. In those cases, try to insert the rows before.

Learning Emoji Representations from Observed Usage

Tue, 05 Nov 2019 21:02:51 +0100

Nowadays it is hard to imagine daily communication without emojis like 😄, 😬 or ❤️. These cute pictograms are not only ideal for expressing emotions, they are also standardized. This makes them ideal for analysis. However, before classical analyses such as clustering can be performed, a numerical representation is required. A simple method would be one-hot-encoding, which would be obvious considering the very limited vocabulary (there are only a few thousand emojis). However, embeddings can not only provide more memory friendly representations, but also those that capture a meaning.

Bigquery & Embeddings

Sun, 27 Oct 2019 08:48:29 +0100

math: true

One can argue if it is wise to store embeddings directly in bigquery or calculate the similarities in SQL. For sure, in some cases a library (e.g. gensim) or approximations (e.g. Facebook faiss) are more appropriate. However, in our setting we wanted to use BigQuery. Therefore, arrays are used to store the word vectors and I created SQL functions to calculate pairwise cosine similarities.

One downside of using BigQuery arrays to store vectors is that there is no guarantee that the ordering of the components is preserved. However, in most situations the order will not change.

Recommender Systems in R

Sat, 26 Oct 2019 20:59:29 +0200

A former WU-member, Michael Hahsler, created a really nice package called recommenderlab, which allows you to build collaborative filtering systems. But before you can use it, you have to install all required packages:

install.packages(c("recommenderlab", "dplyr", "readr"))

… and load them:

library(recommenderlab)
library(tibble)
library(dplyr)
library(readr)

I asked my students to answer some questions. One question was about their favourite TV series, which is a good example for a recommender system (Netflix for example does pretty much the same). So they filled out this survey and I generated a table by adding a new row for each student and a new column for each TV show. I put a 1 in the cell if the user likes the show and nothing if not. You can import such a table (CSV-format) using the following command (all columns are by default parsed as integers except user, which is a string). The file itself came from a course survey and is not published here, so you will have to substitute your own:

Using Embeddings in R

Thu, 08 Aug 2019 21:24:40 +0200

There exist different file formats to store distributed vector or word representations also known as embeddings. However, one of the most convenient ways is to use the text format used by the original word2vec implementation. In this format, each row starts with a label (an item of the vocabulary) followed by the vector components. Furthermore, each field is separated by an ordinary space. The following function processes such *.vec-files and can be used to load them directly into R:

Big Data Analysis in Digital Marketing Research

Mon, 12 Jun 2017 11:00:53 +0100

In the last decades, the increasing availability of broadband internet and the accompanying digitalisation has had a lasting effect on many industries. For example, television and radio are more and more replaced by music and video streaming services.

Nowadays, it is even common to purchase accounting systems, databases or the whole IT-infrastructure as a service. Those trends are challenges for both managers and marketing departments and thus current topics for digital marketing researchers. At the same time, data storage got cheaper and cheaper and firms started to gather every piece of data they could catch in hopes of gilding them one day. Although, data was already available it took a decade till managers had recognised the real value of it and now there is a real demand for data driven decision support. Thus, big data is not an empty phrase anymore, data is available and its analysis is both feasible and reasonable. Nevertheless, mills are slowly working in research and it takes time for innovations in IT (i.e. big data analysis) to arrive at marketing research.