Embeddings on Christian Hotz-Behofsits

Learning Emoji Representations from Observed Usage

Tue, 05 Nov 2019 21:02:51 +0100

Nowadays it is hard to imagine daily communication without emojis like 😄, 😬 or ❤️. These cute pictograms are not only ideal for expressing emotions, they are also standardized. This makes them ideal for analysis. However, before classical analyses such as clustering can be performed, a numerical representation is required. A simple method would be one-hot-encoding, which would be obvious considering the very limited vocabulary (there are only a few thousand emojis). However, embeddings can not only provide more memory friendly representations, but also those that capture a meaning.

Bigquery & Embeddings

Sun, 27 Oct 2019 08:48:29 +0100

math: true

One can argue if it is wise to store embeddings directly in bigquery or calculate the similarities in SQL. For sure, in some cases a library (e.g. gensim) or approximations (e.g. Facebook faiss) are more appropriate. However, in our setting we wanted to use BigQuery. Therefore, arrays are used to store the word vectors and I created SQL functions to calculate pairwise cosine similarities.

One downside of using BigQuery arrays to store vectors is that there is no guarantee that the ordering of the components is preserved. However, in most situations the order will not change.

Using Embeddings in R

Thu, 08 Aug 2019 21:24:40 +0200

There exist different file formats to store distributed vector or word representations also known as embeddings. However, one of the most convenient ways is to use the text format used by the original word2vec implementation. In this format, each row starts with a label (an item of the vocabulary) followed by the vector components. Furthermore, each field is separated by an ordinary space. The following function processes such *.vec-files and can be used to load them directly into R: