comparison to other search algorithms:
vs string search:
- beats string search with better performance and lower system requirements
- looses in exact matching. tokens have no ordering, tokenization may loose some information.
vs semantic search:
- semantic search is based on language intepretation, ie it utilizes intepretative bias in ranking results.
- tokenized bitwise search is unordered exact match algorithm and thus does not have any interpretative bias.