key - construct a unique number for a string in java -


We need to read / write more than 10 million wire in a file. Apart from this, we do not want to duplicate the file, as soon as the wire is read after reading the wire, we can not keep it in memory.

We can not use hash codes because duplication can be missed because of the collision in the code of humankind. Two other methods found in my Googling:

Use a message digest algorithm like 1.MD5 - but it can be very expensive to calculate and store.

2. Use a checksum algorithm. [I'm not sure if it produces a unique key for string - anyone can confirm it)

Is there any other approach available? Thank you.

If you are fine with the risk of collision, you can use some hash function As you suggest, and rely on hash.

Another alternative, possibly with a large memory footprint, (special type of tree), strings are already recognized.


Update: One more option, one will have to use. However, it is still dependent on hashing, but can be adjusted to be an inevitable small possibility of confrontation.


Comments

Popular posts from this blog

windows - Heroku throws SQLITE3 Read only exception -

lex - Building a lexical Analyzer in Java -

python - rename keys in a dictionary -