Building low latency java applications with ehcache
21 likesâ˘4,214 views
The document discusses building low-latency Java applications using Ehcache, highlighting the importance of in-memory computing and persistence strategies to enhance application performance. It outlines the architecture of Ehcache, its configurations, and API usage to reduce database reliance, network traffic, and CPU usage through optimized memory utilization. Additionally, it covers distributed applications with Terracotta server arrays for scalability and consistency options, providing practical examples and recent developments in the technology.
Building low latency java applications with ehcache
- 1. Building Low Latency Java Applications with Ehcache Dhruv Kumar Software Engineer, Terracotta Inc. [email protected]
- 15. 15 Trend Summary 15 §ď§âŻ Storage is cheap and increases exponentially §ď§âŻ Most transfer rates increase exponentially except disk throughput. â⯠Gap between disk capacity and transfer rate is increasing §ď§âŻ Accessing disk is very slow: transferring 100GB @ 100 MB/s takes 1000s, or approx. 17 minutes. â⯠Will get worse, as 512GB/node RAM will be common in 2 years. â⯠Faster to access Network Attached Memory! â⯠SSDs still not common due to unfavorable price performance
- 16. 16 In-Memory Computing Opportunity 16 §ď§âŻ 256 GB RAM * 4 nodes = 1 TB In-Memory Cluster §ď§âŻ 1 TB can hold many large datasets: 1 billion users @ 1 KB each §ď§âŻ Number of CPU cores double every 24 months: memory/core increases exponentially §ď§âŻ Bottomline: judiciously using RAM is key.
- 18. 18 Terracotta Server Array Scale with Data and Processing Needs 18 BigMemory App Server Ehcache App Server Ehcache Traditional 3-Tier Architecture Scale Vertically Scale Horizontally Elastic Scale In the Cloud App Server Ehcache Quartz App Server Ehcache Quartz Database Database Database Database App Server Ehcache App Server Ehcache BigMemory Terracotta Server Array BigMemory Increase Data in Memory Reduce Database Reliance
- 19. 19 Hello Ehcache! Database Traditional 3-Tier Architecture App Server Ehcache §ď§âŻ Reduction in database reliance â⯠Store repetitively used data and access them in memory speeds §ď§âŻ Reduction of new objects created â⯠Stored objects will be reused constantly thus reducing the creation of new objects §ď§âŻ Less network traffic â⯠Stored data is already at the application layer so no need to make a network call to get it §ď§âŻ Reduce CPU usage â⯠Objects in cache are already in the format you need for use, no additional marshaling needed.
- 20. 20 Example Configuration Configuration via Ehcache.xml 20 Example Configuration <ehcache>! ! <cache name=âUserPreferencesCache"! maxElementsInMemory="10000"! timeToIdleSeconds="300â! ! ! !memoryStoreEvictionPolicy="LRUâ/>! ! <cache name=âShoppingCartCache"! maxElementsInMemory=â2500"! timeToLiveSeconds=â6000â! ! ! !memoryStoreEvictionPolicy=âLFU"/>! ! </ehcache>! 20
- 21. 21 Simple API 21 Example Code public Object testCache(String key) throws Exception {! !CacheManager cacheManager = ! ! !new CacheManager( â<path to my ehcache.xml>â);! !Cache myCache = cacheManager.getCache("MyCache");! !Object value;! ! !! !Element element = myCache.get(key);! !if (element == null) {! ! !value = "go get it from somewhere like DB or service, ! ! ! etc";! ! !myCache.put(new Element(key, value));! !} else {! ! !value = (Object) element.getValue();! !}! ! !! !return value;! }!
- 22. 22 Ehcache Search §ď§âŻ Intuitive, full-featured Search API §ď§âŻ Fast, efficient implementation §ď§âŻ Make cache searchable, then specify attributes 22 <cache name=âpeopleCacheâ> <searchable> <searchAttribute name="age" expression="value.getAge()"/> <searchAttribute name="gender" expression="value.getGender()"/> </searchable> </cache> Example: Search for 32-year-old males results = cache.createQuery().includeKeys() .addCriteria(age.eq(32)).and (gender.eq("male")) .execute(); Example Configuration and Code
- 23. 23 Ehcache Search Using SQL-like Syntax §ď§âŻ Can also search Ehcache using a SQL like syntax §ď§âŻ String Queries mapped to Ehcache Search API 23 Example: Search for 32-year-old males results = cache.createQuery().includeKeys() .addCriteria(age.eq(32)).and (gender.eq("male")) .execute(); // or using Ehcache Query Language QueryManager qm = (new QueryManagerBuilder()).addEhcache(peopleCache).build(); String statement = âselect age, gender from peopleCache where age = 32 and gender =âmaleâ;â results = qm.createQuery(statement).end().execute(); Example Configuration and Code
- 24. 24 Ehcache Persistence §ď§âŻ Persist actions in a log-append store (Fast Restart Store). §ď§âŻ Each mutate operation is recorded. Optionally synchronous. §ď§âŻ On restart, Ehcache entries and Index data are reconstructed from the log. 24 Example ConfigurationExample Configuration <ehcache>! <cache name=âUserPreferencesCache"! maxElementsInMemory="10000"! timeToIdleSeconds="300â! ! ! persistenceStrategy=âlocalRestartableâ>! </ehcache>!
- 25. 25 In-memory Data and Speed 25 Milliseconds App Response Time Microseconds App Response Time Memory 90% of Data in Database Database 90% of Data in Memory
- 26. 26 Storing Data away from Java Heap (BigMemory) Off-heap memory allocator. Uses NIO DirectByteBuffers. Bypasses Java Garbage Collection. Only Serializable keys and values can be placed in Off-heap. Pure Java implementation compatible with all popular JVMs Requires no changes to application code, JVM or OS 26 Database Vertical Scaling App Server Ehcache BigMemory
- 27. 27 BigMemory Configuration <ehcache> <cache name= UserPreferencesCache" maxBytesOnHeap=â512M" timeToIdleSeconds="300 !memoryStoreEvictionPolicy="LRU !overflowToOffHeap="true ! !maxBytesOffHeap= 30G"/> <cache name= ShoppingCartCache" maxElementsInMemory= 2500" timeToLiveSeconds= 6000 !memoryStoreEvictionPolicy="LFU"/> </ehcache>! 27 Example Configuration
- 28. 28 Testing Offheap Performance 28 â˘âŻPublic URL: http://svn.terracotta.org/svn/forge/offHeap-test/ â˘âŻOur Configuration: Six 4-core Xeon CPUs @ 2.93 GHz, 128 GB RAM, RHEL 5.1, Sun JDK 1.6.0_21 in 64 bit mode. â˘âŻLoad the cache and then run 50 threads doing 90% Read (get() call) calls and 10 % Writes (put() call) â˘âŻRun in Heap and Offheap, and compare GC duration, latency, throughput.
- 29. 29 Offheap Perf Test: Garbage Collection Pauses 29
- 30. 30 Offheap Perf Test: Maximum Latency 30
- 31. 31 Offheap Perf Test: Mean Latency 31
- 34. 3434 Distributed Ehcache using Terracotta Server Array §ď§âŻ Stateful Server Array §ď§âŻ Synchronous TCP based connections to clients §ď§âŻ Highly Available using Master Slave replication §ď§âŻ Linear Scale Out Characteristics §ď§âŻ Persistent §ď§âŻ Various consistency optionsDatabase Scale Out App Server Ehcache Terracotta Server Array BigMemory App Server Ehcache
- 35. 35 Terracotta Server Array 35 Terracotta ServerArray Stripe Pure Java cache server on commodity HW Transactional updated mirror for high availability Stripe Commodity Server Application Terracotta Driver Stripe Stripe Stripe Stripe Commodity Server Application Terracotta Driver Commodity Server Application Terracotta Driver TCP §ď§âŻDurability §ď§âŻMirroring §ď§âŻStriping §ď§âŻDeveloper Console §ď§âŻPlug-in Monitoring §ď§âŻOperations Center Commodity Server Disk Active Server Commodity Server Disk BigMemory Mirror BigMemory
- 36. 36 Example Configuration Configuring Ehcache to use Terracotta 36 <ehcache> !<terracottaConfig url="someserver:9510"/> !<cache name= UserPreferencesCache ! !maxElementsInMemory="10000 ! !timeToIdleSeconds= 300 /> !<cache name= ShoppingCartCache" ! !maxElementsInMemory= 25000" ! !timeToLiveSeconds= 6000 />! !<terracotta /> </ehcache>! Simply change two lines of configuration
- 37. 37 Tiered Storage In-Memory Tiered Data Storage 37 Terracotta Heap Heap Store BigMemory Off-Heap Store 2,000,000+ 1,000,000 100,000 2 1,000 10,000+ Speed (TPS) 1,000s Size (GB) External Data Source (e.g., Database) Terracotta Offheap
- 38. 38 Example Configuration Consistency options in TSA 38 Strongly Consistent Fully Transactional Eventually Consistent More Consistency More Performance <cache name=âUserPreferencesCache" maxElementsInMemory="10000" timeToLiveSeconds="300â> <terracotta consistency=âeventual"/> </cache> <cache name=âShoppingCartCache" maxElementsInMemory=â5000" timeToIdleSeconds=â6000â> <terracotta consistency=âstrong"/> </cache> Example Configuration
- 39. 39 Terracotta Server Array Scale with Data and Processing Needs 39 BigMemory App Server Ehcache App Server Ehcache Traditional 3-Tier Architecture Scale Up Scale Out Elastic Scale In the Cloud App Server Ehcache Quartz App Server Ehcache Quartz Database Database Database Database App Server Ehcache App Server Ehcache BigMemory Terracotta Server Array BigMemory Increase Data in Memory Reduce Database Reliance
- 40. 40 Recent Developments 40 â˘âŻJSR107 â˘âŻHadoop Integration: One way (EhcacheOutputFormat) â˘âŻNew Management and Monitoring