Spark NLP: natural language understanding at scale

Kocaman, V.; Talby, D. (2021)

Article / Letter to editor

Spark NLP is a Natural Language Processing (NLP) library built on top of Apache Spark ML. It provides simple, performant & accurate NLP annotations for machine learning pipelines that can scale easily in a distributed environment. Spark NLP comes with 1100+ pretrained pipelines and models in more than 192+ languages. It supports nearly all the NLP tasks and modules that can be used seamlessly in a cluster. Downloaded more than 2.7 million times and experiencing 9x growth since January 2020, Spark NLP is used by 54% of healthcare organizations as the world’s most widely used NLP library in the enterprise.

• Spark
• Natural language processing
• Deep learning
• Tensorflow
• Cluster

• All authors: Kocaman, V.; Talby, D.
• Date: 2021-05-01
• Journal: Software Impacts
• Volume: 8
• DOI: doi:10.1016/j.simpa.2021.100058
• Link: https://www.sciencedirect.com/science/article/pii/S2665963821000063