Remove Duplicate Words from a Given Sentence in Java

In this article, we will learn to remove duplicate words from a given sentence in Java. The first method uses Java Streams to remove duplicates and join the words back into a sentence using the distinct() and Collectors.joining() methods. The second method uses a Set to automatically filter out duplicate words and a StringJoiner to combine the words back into a sentence. Both methods will give a sentence with only unique words.

Different approaches

Following are the different approaches to remove all duplicate words from a given sentence ?

• Using Java Streams
• Using Set
  

Using Java Streams

Following are the steps to remove all duplicate words from a given sentence using Java Streams ?

• First, we will import the Arrays from the java.util package and Collectors from java.util.stream package.
  
• After that, we will define a sentence as a string that contains duplicate words.
  
• We will use the split() method to divide the sentence into individual words.
  
• After that we will apply distinct() to filter out the duplicate words.
  
• Join the distinct words back into a sentence using the joining() method.
  
• Print the final sentence without duplicates.
  

Example

Below is the Java program to remove all duplicate words from a given sentence Java Streams ?

import java.util.Arrays;
import java.util.stream.Collectors;
public class Demo{
   public static void main(String[] args){
      String my_str = "This is a is sample a sample only.";
      my_str = Arrays.stream(my_str.split("\s+")).distinct().collect(Collectors.joining(" "));
      System.out.println(my_str);
   }
}

Output

This is a sample only.

Using Set

Following are the steps to remove all duplicate words from a given sentence using Set ?

• First, we will import the required classes like HashSet and StringJoiner from java.util package.
  
• After that, we will define a sentence containing duplicate words.
  
• We will split the sentence into words using the split() method and store each word in a Set to automatically remove duplicates.
• We will use StringJoiner to join the distinct words back into a sentence.
  
• At the end, print the final sentence without duplicates.

Example

Below is the Java program to remove all duplicate words from a given sentence using Set ?

import java.util.HashSet;
import java.util.Set;
import java.util.StringJoiner;
public class RemoveDuplicates {
    public static void main(String[] args) {
        String my_str = "This is a is sample a sample only.";
        String[] words = my_str.split("\s+");       
        Set uniqueWords = new HashSet<>();
        StringJoiner result = new StringJoiner(" ");        
        for (String word : words) {
            if (uniqueWords.add(word)) {
                result.add(word);
            }
        }        
        System.out.println(result.toString());
    }
}

Output

This is a sample only.