security

Calculate the CRC Sum of a file

With this example we are going to demonstrate how to calculate the CRC Sum of a file. The Cyclic Redundancy Check is a good way to detect errors and changes to data. In short, in order to calculate the CRC-32 Sum of a file we implemented four different methods :

  • The checksumInputStream(String filepath) method creates a FileInputStream by opening a connection to the file.
  • The checksumBufferedInputStream(String filepath) method creates a BufferedInputStream to save the input stream.
  • The checksumRandomAccessFile(String filepath) method creates a RandomAccessFile to read from the file. The mode argument is set for reading only.
  • The checksumMappedFile(String filepath) method uses a MappedByteBuffer to map the file directly into memory in a Read-only mode, through its FileChannel.
  • In all methods a new CRC32 Object is created. The CRC32 Object is updated for every byte read from each file. The methods are invoked in a main() method and return the CRC32 value of the same file.

Note that for each method implementation time is also calculated, as shown in the output.
 
Let’s take a look at the code snippets and the result output that follow:

001
002
003
004
005
006
007
008
009
010
011
012
013
014
015
016
017
018
019
020
021
022
023
024
025
026
027
028
029
030
031
032
033
034
035
036
037
038
039
040
041
042
043
044
045
046
047
048
049
050
051
052
053
054
055
056
057
058
059
060
061
062
063
064
065
066
067
068
069
070
071
072
073
074
075
076
077
078
079
080
081
082
083
084
085
086
087
088
089
090
091
092
093
094
095
096
097
098
099
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
package com.javacodegeeks.snippets.core;
 
import java.io.BufferedInputStream;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.RandomAccessFile;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
import java.util.zip.CRC32;
 
public class Main {
 
    public static long checksumInputStream(String filepath) throws IOException {
 
  InputStream inputStreamn = new FileInputStream(filepath);
 
  CRC32 crc = new CRC32();
 
 
  int cnt;
 
   
 
  while ((cnt = inputStreamn.read()) != -1) {
 
 
crc.update(cnt);
 
  }
 
  return crc.getValue();
    }
 
    public static long checksumBufferedInputStream(String filepath) throws IOException {
 
  InputStream inputStream = new BufferedInputStream(new FileInputStream(filepath));
 
   
 
  CRC32 crc = new CRC32();
 
 
  int cnt;
 
   
 
  while ((cnt = inputStream.read()) != -1) {
 
 
crc.update(cnt);
 
  }
 
  return crc.getValue();
    }
 
    public static long checksumRandomAccessFile(String filepath) throws IOException {
 
  RandomAccessFile randAccfile = new RandomAccessFile(filepath, "r");
 
  long length = randAccfile.length();
 
  CRC32 crc = new CRC32();
 
 
  for (long i = 0; i < length; i++) {
 
 
randAccfile.seek(i);
 
 
int cnt = randAccfile.readByte();
 
 
crc.update(cnt);
 
  }
 
  return crc.getValue();
    }
 
    public static long checksumMappedFile(String filepath) throws IOException {
 
  FileInputStream inputStream = new FileInputStream(filepath);
 
  FileChannel fileChannel = inputStream.getChannel();
 
 
  int len = (int) fileChannel.size();
 
 
  MappedByteBuffer buffer = fileChannel.map(FileChannel.MapMode.READ_ONLY, 0, len);
 
 
  CRC32 crc = new CRC32();
 
 
  for (int cnt = 0; cnt < len; cnt++) {
 
 
 
 
 
int i = buffer.get(cnt);
 
 
crc.update(i);
 
  }
 
  return crc.getValue();
    }
 
    public static void main(String[] args) throws IOException {
 
 
  String filepath = "C:/Users/nikos7/Desktop/output.txt";
 
 
  System.out.println("Input Strea method:");
 
 
  long start_timer = System.currentTimeMillis();
 
  long crc = checksumInputStream(filepath);
 
  long end_timer = System.currentTimeMillis();
 
 
  System.out.println(Long.toHexString(crc));
 
  System.out.println((end_timer - start_timer) + " ms");
 
  System.out.println("///////////////////////////////////////////////////////////");
 
 
  System.out.println("Buffered Input Stream method:");
 
  start_timer = System.currentTimeMillis();
 
  crc = checksumBufferedInputStream(filepath);
 
  end_timer = System.currentTimeMillis();
 
  System.out.println(Long.toHexString(crc));
 
  System.out.println((end_timer - start_timer) + " ms");
 
 
  System.out.println("///////////////////////////////////////////////////////////");
 
 
  System.out.println("Random Access File method:");
 
  start_timer = System.currentTimeMillis();
 
  crc = checksumRandomAccessFile(filepath);
 
  end_timer = System.currentTimeMillis();
 
  System.out.println(Long.toHexString(crc));
 
  System.out.println((end_timer - start_timer) + " ms");
 
 
  System.out.println("///////////////////////////////////////////////////////////");
 
 
  System.out.println("Mapped File method:");
 
  start_timer = System.currentTimeMillis();
 
  crc = checksumMappedFile(filepath);
 
  end_timer = System.currentTimeMillis();
 
  System.out.println(Long.toHexString(crc));
 
  System.out.println((end_timer - start_timer) + " ms");
    }
}

Output:

Input Strea method:
94ccd63e
75 ms
///////////////////////////////////////////////////////////
Buffered Input Stream method:
94ccd63e
5 ms
///////////////////////////////////////////////////////////
Random Access File method:
94ccd63e
99 ms
///////////////////////////////////////////////////////////
Mapped File method:
94ccd63e
8 ms

 
This was an example of calculating the CRC Sum of a file in Java.

Byron Kiourtzoglou

Byron is a master software engineer working in the IT and Telecom domains. He is an applications developer in a wide variety of applications/services. He is currently acting as the team leader and technical architect for a proprietary service creation and integration platform for both the IT and Telecom industries in addition to a in-house big data real-time analytics solution. He is always fascinated by SOA, middleware services and mobile development. Byron is co-founder and Executive Editor at Java Code Geeks.
Subscribe
Notify of
guest


This site uses Akismet to reduce spam. Learn how your comment data is processed.

2 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Jentec
Jentec
6 years ago

on big files like 1GB mapped is the fasted way
thank you for post

///////////////////////////////////////////////////////////
Buffered Input Stream method:
3ef92509
29903 ms
///////////////////////////////////////////////////////////
Mapped File method:
3ef92509
2949 ms

mohamed
mohamed
2 years ago
seems that the file is still locked after process, i added inputstream.close(), after ChecksumBufferedInputStream
Back to top button