R on Christian Hotz-Behofsits

Parallel JSONLines in R

Wed, 06 Jan 2021 12:37:52 +0100

JSON is becoming more and more widely used in data lakes, especially as a replacement for CSV. In particular, JSONLines where one JSON object is stored per line is to be mentioned here. Although the format has some disadvantages (i.e., overhead, supports only few data types), the advantages mostly outweigh them. For example, the individual lines are easier to read by humans and the format is much more standardized. In addition, each line has its own JSON object that can be processed independently. The overhead caused by the repeating field names can be well managed with compression (e.g., gzip, zstd, brotli).

Learning Emoji Representations from Observed Usage

Tue, 05 Nov 2019 21:02:51 +0100

Nowadays it is hard to imagine daily communication without emojis like 😄, 😬 or ❤️. These cute pictograms are not only ideal for expressing emotions, they are also standardized. This makes them ideal for analysis. However, before classical analyses such as clustering can be performed, a numerical representation is required. A simple method would be one-hot-encoding, which would be obvious considering the very limited vocabulary (there are only a few thousand emojis). However, embeddings can not only provide more memory friendly representations, but also those that capture a meaning.

Recommender Systems in R

Sat, 26 Oct 2019 20:59:29 +0200

A former WU-member, Michael Hahsler, created a really nice package called recommenderlab, which allows you to build collaborative filtering systems. But before you can use it, you have to install all required packages:

install.packages(c("recommenderlab", "dplyr", "readr"))

… and load them:

library(recommenderlab)
library(tibble)
library(dplyr)
library(readr)

I asked my students to answer some questions. One question was about their favourite TV series, which is a good example for a recommender system (Netflix for example does pretty much the same). So they filled out this survey and I generated a table by adding a new row for each student and a new column for each TV show. I put a 1 in the cell if the user likes the show and nothing if not. You can import such a table (CSV-format) using the following command (all columns are by default parsed as integers except user, which is a string). The file itself came from a course survey and is not published here, so you will have to substitute your own:

Using Embeddings in R

Thu, 08 Aug 2019 21:24:40 +0200

There exist different file formats to store distributed vector or word representations also known as embeddings. However, one of the most convenient ways is to use the text format used by the original word2vec implementation. In this format, each row starts with a label (an item of the vocabulary) followed by the vector components. Furthermore, each field is separated by an ordinary space. The following function processes such *.vec-files and can be used to load them directly into R: