Unique
UniqueHash
If the input does not fit in memory
next(): Eliminate duplicates in each
partition (one at a time, on demand):
- Read the partition into memory.
- Use an in-memory hash table (as when the entire table fit in memory).
- Return rows out of the hash table.
- When all rows returned, go on to the next partition.
|