How NSD and immutability enabled microdata.no - a revolutionary platform for research on register data

Ørnulf Risnes

Short talk - in English

Norway has a large number of registers on individuals that have been established for administrative and statistical purposes. These registers represent a unique and valuable resource for research on welfare and society.

Traditionally, getting approvals to do research on these data has been complicated and time-consuming.

The new platform microdata.no reduces approval time from 9 months to 0 days, and makes these valuable data available for a much wider community of researchers.

Microdata.no was released in 2018 after a 5 year collaboration project between NSD and Statistics Norway (SSB). Researchers analyze data through a privacy-preserving web-IDE inspired by Jupyter notebooks and other widely used statistical tools.

The platform is built with ClojureScript, Clojure, Node, Python, Go and Datomic - and immutability is the cornerstone of the architecture.

In this short talk, we will demonstrate how NSD designed microdata.no around immutable data structures, and how this design decision enabled the development of a novel and globally unique platform for safe research on personal data.

We will discuss how we designed and implemented a new domain specific language (DSL) for analyses of temporal register data, how we execute the DSL with functional reactive programming, and how we use immutable data structures and databases to enforce complete statelessness and simplicity throughout the platform.