I have a list of around 2 billion distinct numbers stored in memory for some computation. Currently, every time I need to add a new entry I have to search the entire list for a potential duplicate.
I have 18 digit, 2 billion numbers to be stored in memory. Some of the numbers are repeated. So, I want to load new number in the memory and ignore if the number already exists. Right now, I append the number in the list if it does not exist already.
This process of linear scan of the already existing number is taking long time. What can I do to get around this performance problem?
Aucun commentaire:
Enregistrer un commentaire