eScriptorium

A Digital Text Production Pipeline for Print and Handwritten Texts using machine learning techniques.

Automatic
Transcription

Apply OCR/HTR to images of printed and handwritten documents using shared open models.

Manual
Transcription

Make use of an ergonomic user interface leveraging modern browser technology to edit segmentations and transcriptions.

Train
Models

Create new models or finetune existing ones to improve automatic recognition.

Import/
Export

Import and export models and texts transcriptions in a variety of formats. Access data through a full REST API.

What can you do with it?

The primary tasks of eScriptorium are

• To automatically convert images of manuscripts, books or inscriptions in any script, language or writing direction into machine readable text, including the geometrical information about where on the image the texts are located.
• Establish the reading order
• Distinguish different types of text (e.g. the running headers, titles or interlinear glosses from the main text).

Through automatic layout segmentation the computer establishes the locations and different types of text. Automatic text recognition (ATR) transcribes text lines into machine readable and searchable text you can use e.g. in a word processor like Microsoft word. However, the greatest advantage in the present method is the preservation of the link between the image zones for regions, lines, words or even letters and the transcription. They enable users to create new forms of digital editions or perform automatic paleographical analyses.

You can define your own ontology and train segmentation and recognition models to adapt the machine to your material - and to your conventions of how to segment or transcribe. These models can be shared, exported and imported. You can finetune existing models and considerably reduce the amount of training material to adapt to a new document type, script or language. This is called transfer learning. In addition to different scripts, languages and layouts, documents have different reading orders, e.g. between poetic and prose texts. In the near future eScriptorium will therefore also include trainable reading order detection. Some users also simply profit from the highly ergonomic interface of eScriptorium to transcribe texts manually without automatic means.

eScriptorium uses cutting edge machine learning to perform the automatic tasks. In supervised machine learning the computer learns by example. Unsupervised pretraining is used to reduce the amount of training material. Automatic text to text alignment can help you to create training material from existing e-texts in lightning speed up to self-supervised learning as in the ACDC pipeline.

What can’t you do with it?

eScriptorium is a web based software, not a service. In order to run it you need access to an existing instance or if you have some technical knowledge you can install it locally. eScriptorium can be installed in Linux and Mac or on WSL on Microsoft Windows on individual computers up to multi-server multi-gpu clusters. For the training of segmentation models, a gpu with sufficient memory is indispensable. For the training of recognition models, a gpu is highly recommended.

philosophy

Since the beginning, our philosophy has been to create an open source product for all parts of the UI and the AI, to enable the sharing of models and to promote the sharing of ground truth. Working at an institution famous for its philologist specialized in all kinds of script systems world wide, we planned it to be able to deal with as many scripts, languages and writing directions as possible, right-to-left, left-to-right, top-down.

History

eScriptorium started in December 2019 in the frame of the Scripta-PSL project’s Digital Humanities axis codirected by Peter Stokes and Daniel Stökl Ben Ezra with the lead engineers Robin Tissot (UI and full stack) and Benjamin Kiessling (AI) under the leadership of the EPHE-PSL with important contributions of ALMAnaCH (Inria), and since 2020 also the Universities of Maryland and North-Eastern.

After its initial stage, eScriptorium and kraken were funded by the H2020 InfraDev Resilience, the DIM STCN, ANR EquipEx Biblissima+, and the ERC SyG MiDRASH on the EPHE side, LectauRep and ATRIUM on the Inria side and openITI, NEH “Automatic Collation for Diversifying Corpora: Improving Handwritten Text Recognition (HTR) for Arabic-script Manuscripts” and ERC Kitab on the UMaryland/North-Eastern side. Outside funding made it possible to engage the SMEs Teklia and Performant for a lot of code contributions. Additional contributions come from Princeton University.

list of instances

eScriptorium is running on many small and big instances around the planet. Not all of them are known to us. Our main instance is manuscriptologIA (short msIA) at the mesoPSL cluster at the Observatory of Paris. Other big instances are Cremma at the INRIA, Paris, simorgh (U Maryland), the University Library of Mannheim and the National Library of Israel (Jerusalem).