Menu

Source Merge Request #33: Enhancement (merged)

Merging...

Merged

Something went wrong. Please, merge manually

Checking if merge is possible...

Something went wrong. Please, merge manually

Cai Yi wants to merge 2 commits from /u/cy2008/opencsv/ to master, 2024-09-02

1.Replace java.text.SimpleDateFormat to org.apache.commons.lang3.time.DateFormatUtils.format to reduce instance creation
2.Add ResultSetHelper ability to CSVParserWriter when using CSVWriterBuilder
3.Configurable null default in ResultSetHelperService

Commit Date  
[853ba0] (HEADmaster) by caiyi caiyi

Add ResultSetHelper ability to CSVParserWriter when using CSVWriterBuilder

2022-08-16 07:28:37 Tree
[9a073a] by caiyi caiyi

Configurable null default in ResultSetHelperService

(cherry picked from commit 96fd0d2fd7928889d798a974f5a91c5b755f3ee9)

2022-08-12 00:47:47 Tree

Discussion

  • Scott Conway

    Scott Conway - 2022-08-13

    Hello Cai

    The ResultSetHelperService the changes look good and I will merge that in a future date.

    For the change in CSVWriterBuilder please create a test that will fail or error if your code is not there. This will provide good documentation as to why your change is necessary and prevent it from being removed if the code was ever refactored in the future.

    Now for the SimpleDateFormat vs DateFormatUtils I wrote a test (attached at the end) and got some very interesting results. Depending on the order in which the test (and stress test portion) were run the SimpleDateFormat was at times way faster and other times way slower than DateFormatUtils!

    Basically with the different orders I was seeing SimpleDateFormat take between 420-950 ms to run (your times may be different depending on the machine you run it on) and the DateFormatUtils was a more consistent 600-650ms. Because of the consistency (not the speed because it sometimes was not there) I will definitely merge that in as well.

    Here was my test code - let me know if I did something wrong. The stress system is currently last instead of first because I wanted to check the times without the stress test.

    import org.apache.commons.lang3.time.DateFormatUtils;
    
    import org.junit.jupiter.api.*;
    
    import java.text.SimpleDateFormat;
    import java.util.ArrayList;
    import java.util.Date;
    import java.util.List;
    import java.util.Random;
    
    import static org.junit.Assert.assertEquals;
    import static org.junit.Assert.assertNotNull;
    
    @TestMethodOrder(MethodOrderer.OrderAnnotation.class)
    public class SimpeDateFormatVsDateFormatUtilsTest {
        private static final int LOOP_SIZE = 1000000;
        private static final int ARRAY_SIZE = 100;
        private static final String DEFAULT_DATE_FORMAT = "dd-MMM-yyyy";
    
        private static List<Date> testDates = new ArrayList<>(ARRAY_SIZE);
    
        @BeforeAll
        public static void init() {
            System.out.println("before all method called.");
            Random random = new Random();
            for (int i = 0; i < ARRAY_SIZE; i++) {
                Date date = new Date(random.nextLong());
                testDates.add(date);
            }
            System.out.println("arraysize: " + testDates.size());
        }
    
        @BeforeEach
        public void stressTheSystem() {
            DateFormatUtils dateFormat = new DateFormatUtils();
    
            for (int i = 0; i < 1000; i++) {
                String dateString = dateFormat.format(testDates.get(i%ARRAY_SIZE), "mm-dd-yyyy");
            }
        }
    
        @Order(1)
        @Test
        public void didInitializationHappen() {
            assertEquals(ARRAY_SIZE, testDates.size());
        }
    
        @Order(20)
        @Test
        public void stressSystem() {
            SimpleDateFormat dateFormat = new SimpleDateFormat(DEFAULT_DATE_FORMAT);
            DateFormatUtils dateFormat2 = new DateFormatUtils();
    
            for (int i = 0; i < LOOP_SIZE*2; i++) {
                String dateString = dateFormat.format(testDates.get(i%ARRAY_SIZE));
                assertNotNull(dateString);
                dateString = dateFormat2.format(testDates.get(i%ARRAY_SIZE), DEFAULT_DATE_FORMAT);
                assertNotNull(dateString);
            }
        }
    
        @Order(4)
        @RepeatedTest(5)
        @Test
        public void simpleDateFormatTest() {
            SimpleDateFormat dateFormat = new SimpleDateFormat(DEFAULT_DATE_FORMAT);
    
            for (int i = 0; i < LOOP_SIZE; i++) {
                String dateString = dateFormat.format(testDates.get(i%ARRAY_SIZE));
                assertNotNull(dateString);
            }
        }
    
    
        @Order(3)
        @RepeatedTest(5)
        @Test
        public void dateFormatUtilsTest() {
            DateFormatUtils dateFormat = new DateFormatUtils();
    
            for (int i = 0; i < LOOP_SIZE; i++) {
                String dateString = dateFormat.format(testDates.get(i%ARRAY_SIZE), DEFAULT_DATE_FORMAT);
                assertNotNull(dateString);
            }
        }
    
    }
    
     
    • Cai Yi

      Cai Yi - 2022-08-15

      Add ResultSetHelper ability to CSVParserWriter when using CSVWriterBuilder

      When using CSVWriterBuilder.createCSVParserWriter to create ICSVWriter, property custom resultSetHelper in CSVWriterBuilder will be ignored, and new default ResultSetHelper created by AbstractCSVWriter.resultService will be used in ICSVWriter, then date format and number format are always default format, custom format will not work

      Replacement of SimpleDateFormat

      I am considered about memory usage when too many rows are writed, Mybe a property simpleDateFormat in ResultSetHelperService is a good idea raher than every creation when getColumnValue calls. I will test it later.

       

      Last edit: Cai Yi 2022-08-15
  • Scott Conway

    Scott Conway - 2024-09-02
    • Status: open --> merged
     

Log in to post a comment.

Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.