unicode conversion is buggy

[eromoe]

I think this problem cause python stop working #47 , so open a separated issue.

example:

unique_text: https://pastebin.com/n2i280i8

    import datrie
    text = htmls_2_text(input_dir)
    unique_text = ''.join(set(text))
    trie = datrie.Trie(unique_text )
    trie['今天天气真好'] = 111
    trie['今天好'] = 222
    trie['今天'] = 444
    
    print(trie.items())
    
    [('今义', 444), ('今义义傲兢于', 111), ('今义于', 222)]

got wrong word .

---

I tried to locate the error:

Error

u = ''.join(set('今天天气真好' + unique_text[:400]))

got [('今天', 444), ('今天天气I好', 111), ('今天好', 222)]

Correct

u = ''.join(set('今天天气真好' + unique_text[:396]))
u = ''.join(set('今天天气真好' + unique_text[:396]+unique_text[398:400]))
u = ''.join(set('今天天气真好' + unique_text[396:400]))

all correct.

[Cherrymelon]

i meet same bug,unicode chinese character map wrong character