Remove All Duplicates in an Array Like a Pro in JS

In JavaScript, removing duplicate elements from an array can be achieved through variety of methods, including the higher order array methods such as reduce(), filter(), or with a straightforward for loop. However, an even more easier approach exists by using the inherent Set object.

Get all unique values in an array

A Set cannot contain duplicate values and can be easily initialized from the values of an array. Then, since it is iterable in itself, we can use the spread operator (...) to convert it back to an array of just the unique values easily.

const uniqueElements = arr => [...new Set(arr)];

uniqueElements([1, 2, 2, 3, 4, 4, 5]); // output: [1, 2, 3, 4, 5]

Check whether an array contains duplicates

The Set object lacks of a length property; however, it does have a size property instead. This attribute can be used to determine whether an array contains duplicates.

const duplicatesExist = arr => arr.length !== new Set(arr).size;

duplicatesExist([1, 2, 2, 3, 4, 4, 5]); // true
duplicatesExist([1, 2, 3, 4, 5]); // false

For the truthy check enjoyers, we can check whether all the values of an array are distinct by inverting the condition.

const isAllDistinct = arr => arr.length === new Set(arr).size;

isAllDistinct([1, 2, 2, 3, 4, 4, 5]); // false
isAllDistinct([1, 2, 3, 4, 5]); // true

Filter array items that are not duplicated

To keep only unique values within an array, we can use the filter() method. Since duplicated elements must exist in multiple indices, we can use indexOf() and lastIndexOf() to validate this. For arrays with numerous duplicates, preemptively creating a Set from the array could enhance performance.

const filterUnique = arr =>
  [...new Set(arr)].filter(i => arr.indexOf(i) === arr.lastIndexOf(i));

filterUnique([1, 2, 2, 3, 4, 4, 5]); // [1, 3, 5]

For the falsy enjoyers, we can also do the opposite, and remove all unique values. In this case the two indices have to be the same.

const removeUnique = arr =>
  [...new Set(arr)].filter(i => arr.indexOf(i) !== arr.lastIndexOf(i));

removeUnique([1, 2, 2, 3, 4, 4, 5]); // [2, 4]

Recap

Set is a developer experience booster object when it comes to deal with array with non-unique elements.

• You can get all unique values easily without dealing with filter() and reduce() callbacks (they are also very powerful methods too).
• Spread operator also available on Set object and you can behave it like an array with small differences like length become size in Set case etc.
• You can unite forces of array manipulator functions and Set to come up with clever solutions to custom cases like filtering unique values, removing unique values etc.

Sources:

MDN

Thank you for reading this far. See you in the next chapter :)