Clickhouse on Christian Hotz-Behofsits

Applying Cosine Similarity on a Large Scale

Fri, 30 Sep 2022 00:00:00 +0000

I wrote my first blog post about cosine similarity back in 2019 when the pandemic was out of sight, and most marketing people were unaware of “representational learning.” However, times have changed, and nowadays, many quantitative marketing papers apply word2vec, prod2vec, or similar approaches.

Thanks to gensim, training such a model is straightforward. For example, one can learn the representations of tracks within a few lines of python code:

from gensim.models import Word2Vec
from gensim.models.word2vec import LineSentence

model = Word2Vec(
 sentences=LineSentence('./data/tmp_isrc_training.txt'), 
 vector_size=50, window=99, epochs=5, min_count=1, workers=8,
 sg=1, negative=10
)

SQL Multi-Substring Search in Yandex Clickhouse

Sat, 16 Jan 2021 10:27:14 +0100

Nowadays, also analytical databases are more frequently used to search for substrings or patterns in character strings. Thus, most databases provide a like operator or more sophisticated regex functions. However, especially long regex patterns or many of them can be challenging. In a recent project, I had to search for not one, but for 150 substrings at the same time. Thankfully, my database of choice (Yandex Clickhouse) offers several possibilities to handle this:

Big Data Analysis in Digital Marketing Research

Mon, 12 Jun 2017 11:00:53 +0100

In the last decades, the increasing availability of broadband internet and the accompanying digitalisation has had a lasting effect on many industries. For example, television and radio are more and more replaced by music and video streaming services.

Nowadays, it is even common to purchase accounting systems, databases or the whole IT-infrastructure as a service. Those trends are challenges for both managers and marketing departments and thus current topics for digital marketing researchers. At the same time, data storage got cheaper and cheaper and firms started to gather every piece of data they could catch in hopes of gilding them one day. Although, data was already available it took a decade till managers had recognised the real value of it and now there is a real demand for data driven decision support. Thus, big data is not an empty phrase anymore, data is available and its analysis is both feasible and reasonable. Nevertheless, mills are slowly working in research and it takes time for innovations in IT (i.e. big data analysis) to arrive at marketing research.