frak models in ocrd resmgr

[jbarth-ubhd]

I've compared these frak models:

ocrd: https://ub-backup.bib.uni-mannheim.de/~stweil/tesstrain/frak2021/tessdata_best/frak2021-0.905.traineddata from ocrd resmgr

ubma: https://ub-backup.bib.uni-mannheim.de/~stweil/tesstrain/frak2021/tessdata_fast/frak2021_1.069.traineddata from https://ocr-bw.bib.uni-mannheim.de/faq/

size & md5sum:

-rw-rw-r-- 1 jb jb 3421140 Mär 27  2021 ocrd--frak2021-0.905.traineddata
234e8bb819042f615576bd01aa2419fd  ocrd--frak2021-0.905.traineddata
-rw-rw-r-- 1 jb jb 5060763 Dez  9  2021 ubma--frak2021_1.069.traineddata
9405b1603db21cb066e4e7614a405dd4  ubma--frak2021_1.069.traineddata

content after combine_tessdata -u x.traineddata aa :

jb@nuc:~/models$ LC_ALL=C ls -lh ocrd ubma
ocrd:
total 3.3M
-rw-rw-r-- 1 jb jb 3.3M Dec 21 12:18 aa.lstm
-rw-rw-r-- 1 jb jb 2.8K Dec 21 12:18 aa.lstm-recoder
-rw-rw-r-- 1 jb jb  22K Dec 21 12:18 aa.lstm-unicharset
-rw-rw-r-- 1 jb jb   30 Dec 21 12:18 aa.version
-rw-rw-r-- 1 jb jb  345 Dec 21 12:18 extr.log

ubma:
total 4.9M
-rw-rw-r-- 1 jb jb 432K Dec 21 12:18 aa.lstm
-rw-rw-r-- 1 jb jb 6.3K Dec 21 12:18 aa.lstm-number-dawg
-rw-rw-r-- 1 jb jb 4.5K Dec 21 12:18 aa.lstm-punc-dawg
-rw-rw-r-- 1 jb jb 2.8K Dec 21 12:18 aa.lstm-recoder
-rw-rw-r-- 1 jb jb  22K Dec 21 12:18 aa.lstm-unicharset
-rw-rw-r-- 1 jb jb 4.4M Dec 21 12:18 aa.lstm-word-dawg
-rw-rw-r-- 1 jb jb   30 Dec 21 12:18 aa.version
-rw-rw-r-- 1 jb jb  553 Dec 21 12:18 extr.log

ubma is with .lstm-word-dawg, ocrd is without.

ocrd is 3.3M lstm size, ubma is 432k lstm size.

shouldn't ocrd use the ubma file for fraktur/gothic?

[stweil]

Which one of the two models is "better", and how did you compare them?

[jbarth-ubhd]

Comparison in sense of "check if the model files have the same content".

[bertsky]

That's strange indeed. It's not to be expected from the vanilla tesstrain rules (even the fast variant just does ConvertToInt). And the concrete wordlist looks very awkward (contains 400k fullforms, nearly half of which are made of strange punctuation characters indicative of absent tokenisation, and the actual tokens are clearly scraped off the web, not historic at all). I would understand if the wordlist from deu or frk is used in frak2021, but that's not the case at all.

@stweil can you explain?

[stweil]

frak2021_1.069.traineddata was made from the original training result, but with additional components like wordlist, number und punctuation hints (frak2021_1.069.lstm-word-dawg, frak2021_1.069.lstm-number-dawg, frak2021_1.069.lstm-punc-dawg). Those additional components are based on the components from a Tesseract standard model (as far as I remember on Fraktur.traineddata, but I'd have to check). Sort the word list before comparing it with other word lists.

Because of the additional components the file frak2021_1.069.traineddata is larger.

Typically models with a (ideally domain specific) wordlist can achieve slightly higher recognition rates, but sometimes it can also lead to OCR results which differ from the printed text.

And yes, this word list contains a lot of entries which should be removed. That's inherited from all standard Tesseract word lists.

[bertsky]

Those additional components are based on the components from a Tesseract standard model (as far as I remember on Fraktur.traineddata, but I'd have to check)

No, the latter word list is about twice the size, also with texts from the web, but contains none of these strange words with punctuation (non-tokenised), and does contain ſ, which yours does not.

Of course it would be preferable to have a standard dictionary for (say) 18th century German. We could export the fullforms from DTA lexdb, for example. (But this must be accompanied by a thorough evaluation.)

Regardless, the word list in that model files looks exceptionally bad (much worse than the Tesseract word lists) and should be improved.

[bertsky]

Of course it would be preferable to have a standard dictionary for (say) 18th century German. We could export the fullforms from DTA lexdb, for example. (But this must be accompanied by a thorough evaluation.)

I have now distilled a list of full forms, capping at different frequencies, respectively:

• >10: 314248 words
• >50: 100516 words
• >100: 60403 words

I filtered by part-of-speech, removing punctuation, numbers and non-words (XY):

select trim(u,'"') from csv where f > 100 and p != "$(" and p != "$," and p != "$." and p != "FM.xy" and p != "CARD" and p != "XY";

Furthermore, I removed those entries which have not been properly tokenised (indicated by leading punctuation) or are merely numbers (but still do not get p=CARD):

grep -v -e '^[[:punct:]]' -e '^[[:digit:][:punct:]]*$'

The quality is very good!

Maybe I'll also recompose the number and punc DAWGs for the additional historic patterns (e.g. ⸗ instead of hyphen, solidus instead of comma) and remove the contemporary ones (€ sign etc).

I will try to use this with frak2021, but also GT4HistOCR and others.

I guess I'll do some recognition experiments and evaluation before publishing the modified models.

[bertsky]

• 10: 314248 words

• 50: 100516 words

• 100: 60403 words

I will try to use this with frak2021, but also GT4HistOCR and others.

Done: see

• https://github.com/bertsky/dta-lexdb-applications/releases/download/v0.1/frak2021_dta10.traineddata
• https://github.com/bertsky/dta-lexdb-applications/releases/download/v0.1/frak2021_dta50.traineddata
• https://github.com/bertsky/dta-lexdb-applications/releases/download/v0.1/frak2021_dta100.traineddata
• https://github.com/bertsky/dta-lexdb-applications/releases/download/v0.1/GT4HistOCR_dta10.traineddata
• etc.

[stweil]

In my tests frak2021 is much better than GT4HistOCR, so using it with GT4HistOCR might not be worth the efforts.
It would be more interesting to use it with german_print.

[bertsky]

In my tests frak2021 is much better than GT4HistOCR, so using it with GT4HistOCR might not be worth the efforts.
It would be more interesting to use it with german_print.

sure, that's why it's among the models I build the dict into – see full list of assets

Some evaluation (which material which model whether dict or not and which cap freq preferable) will follow.

[jbarth-ubhd]

Here my small tool for checking the wordlists of .traineddata files:

https://gist.github.com/jbarth-ubhd/8d5ceb4035bf2d89700117a311209f20

[jbarth-ubhd]

@bertsky : but frak2021_dta10+100 do not contain »ſ«:

AMBIGIOUS (EXCERPT): 1sten A/ AP. As. AZ. Basalt- Bauers- Besitz- Bietsch- c. cas. Centralbl. Chrysost. cl. Corn. dial. Diener. Ding- Dinge. Ebd. Eigen- eigentl. Eisen. euch. Eurip. fgm. FML. fundam. g1 Gebiets- Geitz- Generals- G.n GOtts Griseb. Haubt- haus- HErre hsg. inst. Jahrbb. Jungfrau- k. Kg. Kiefer- Lactant. lap. legit. Loose. Magdalenen- Mai- Mehl= Namen. nat. neu- NJmb Normal- O1 Pall. pan. Pfand- Pfl. proc. Reb- redet. Rev. Rhodigin. Rich. Roman- Sc. Schulen. Schweine- Sed. SEin SJndt Spargel- Spitz- Strom. Syllog. Trauben- Trav. Trias- Trift- VIEUSSENS. VVilliam Wach- W.-B. wohl- Wolf. XCVII. y2 Ztg. zwei-

264677 lines
0.00 % lines with »ſ«
0.64 % lines all-UPPERCASE
3.51 % lines ambigious

[bertsky]

Indeed – something went wrong. Thanks @jbarth-ubhd, I'll investigate!

[bertsky]

Ok, I found the problem. See new release.

346632 lines
16.37 % lines with »ſ«
0.19 % lines all-UPPERCASE
132.80 % lines ambigious

What's with the > 100% BTW?

[jbarth-ubhd]

>100% is because I've inspected only every 1/0.003th word, to keep output compact and multiply the count - I'll have a look at this.

[jbarth-ubhd]

Just inspected frak2021_dta50.traineddata.

Ambigious:

welchẽ   (not NFC) Welchs     weñ      (not NFC) Weñ      (not NFC) wenigen

a lot of spaces after words(?). And not NFC (double counting, my bug.)

Spaces were not in frak2021_dta10/100 I've downloaded till Jan 30 11:55.