Skip to content

Bump wikipedia2vec to solve unicode error

AKhatun requested to merge fix-unicode-error into main

The Unicode errors in zhwiki and fywiki are solved with the new version of wikipedia2vec. Few other packages had to be updated to resolve intermediate errors.

Ran a few languages to check performance:

wiki precision recall
bnwiki 0.7205778717406625 0.2928261180850556
simplewiki 0.7849090635803397 0.4323563503891373
fywiki 0.8259000303990968 0.4566365731847868
zhwiki 0.8196185286103542 0.0440551861507367

simplewiki and bnwiki results remain same, so this is good. zhwiki and fywiki now run successfully and have good results.

Merge request reports