Commit Graph

2 Commits

Author SHA1 Message Date
473334c3db Refactor tokenization 2020-08-29 14:50:16 +02:00
71678b2ec6 Extract preprocessing from tokenizer 2020-08-29 11:25:10 +02:00