3 points | by seccode 12 hours ago
3 comments
This works by text substitution, I imagine that you could get similar or better gains using zstd custom dictionaries.
This is a direct comparison to zstd with a dictionary actually
Methodology for comparison: train zstd dictionary on enwik9. Then build my dictionary as most common words in enwik9. Mine does 13% better because of the way I discovered how you can generate dictionary replacement symbols
This works by text substitution, I imagine that you could get similar or better gains using zstd custom dictionaries.
This is a direct comparison to zstd with a dictionary actually
Methodology for comparison: train zstd dictionary on enwik9. Then build my dictionary as most common words in enwik9. Mine does 13% better because of the way I discovered how you can generate dictionary replacement symbols