Archaeology specific BERT models for English, German, and Dutch

Abstract

This short paper describes a collection of BERT models for the archaeology domain. We took existing

language specific BERT models in English, German, and Dutch, and further pre-trained them with

archaeology specific training data. We then took each of these three archaeology specific models and

fine-tuned them for Named Entity Recognition.