Articles on this Page
- 06/06/12--07:33: _Doing more as a Pro...
- 08/11/12--03:58: _All about Hibernate...
- 09/08/12--23:25: _Java Memory Profili...
- 10/14/12--01:10: _Top 7 tips for succ...
- 11/18/12--03:23: _Using Jasper Report...
- 01/13/13--01:02: _11 Online Learning ...
- 02/24/13--05:21: _5 Things a Java Dev...
- 03/20/13--00:28: _A 3 Step Guide to G...
- 04/21/13--00:43: _Package your classe...
- 04/29/13--07:24: _NoSQL Distilled : B...
- 05/23/13--07:00: _GOF Design Patterns...
- 09/21/13--22:25: _How can Agile help ...
- 10/09/13--07:54: _On the way to a blo...
- 10/22/13--08:42: _A MindMap for Java ...
- 12/18/13--04:05: _Does the View in da...
- 01/02/14--05:17: _Why is tomcat a Web...
- 02/23/14--00:47: _4 simple steps to m...
- 03/01/14--08:13: _TalkNotes - The sto...
- 03/24/14--06:49: _Integration Testing...
- 05/04/14--00:43: _EXIN Cloud Computin...
- 07/18/14--01:27: _Identifying the ski...
- 01/24/16--22:23: _Application Securit...
- 04/18/16--01:55: _Do you need microse...
- 08/01/16--03:32: _An approach to help...
- 11/03/18--09:45: _Practical communica...
- 06/06/12--07:33: Doing more as a Programmer
- 08/11/12--03:58: All about Hibernate Second Level Cache
- 09/08/12--23:25: Java Memory Profiling Simplified
- 11/18/12--03:23: Using Jasper Reports to create reports in Java
- 01/13/13--01:02: 11 Online Learning websites that you should check out
- 02/24/13--05:21: 5 Things a Java Developer consider start doing this year
- 03/20/13--00:28: A 3 Step Guide to Getting Started with NoSQL
- 04/21/13--00:43: Package your classes by Feature and not by Layers
- 04/29/13--07:24: NoSQL Distilled : Book Review
- 05/23/13--07:00: GOF Design Patterns Notes on Trello!
- 09/21/13--22:25: How can Agile help you in clearing the technical debt?
- 10/09/13--07:54: On the way to a blogger - My Journey with this blog so far
- 10/22/13--08:42: A MindMap for Java Developer Interviews
- 12/18/13--04:05: Does the View in database reduce the query performance?
- 01/02/14--05:17: Why is tomcat a Webserver and not an Application Server
- 02/23/14--00:47: 4 simple steps to migrate legacy projects from Ant to Maven
- 03/01/14--08:13: TalkNotes - The story of SonarQube told to a DevOps Engineer
- 05/04/14--00:43: EXIN Cloud Computing Foundation Exam Review
- 07/18/14--01:27: Identifying the skills gap for a Software Developer
- 01/24/16--22:23: Application Security for Java Developers
- 04/18/16--01:55: Do you need microservices architecture?
- 08/01/16--03:32: An approach to help developers write meaningful tests
- 11/03/18--09:45: Practical communication strategies for software architects
I have attached the presentation here, it is available on SlideShare.
I had fun presenting it in my office, hope you will enjoy it too.
Recently I have experimented with hibernate cache. In this post I would like share my experience and point out some of the details of Hibernate Second Level Cache. On the way I will direct you to some articles that helped me implement the cache. Let's get started from the ground.
Caching in hibernateCaching functionality is designed to reduces the amount of necessary database access. When the objects are cached they resides in memory. You have the flexibility to limit the usage of memory and store the items in disk storage.The implementation will depend on the underlying cache manager. There are various flavors of caching available, but is better to cache non-transactional and read-only data.
Hibernate provides 3 types of caching.
1. Session CacheThe session cache caches object within the current session. It is enabled by default in Hibernate. Read more about Session Cache. Objects in the session cache resides in the same memory location.
2. Second Level Cache
The second level cache is responsible for caching objects across sessions. When this is turned on, objects will be first searched in cache and if they are not found, a database query will be fired. Read here on how to implement Second Level Cache. Second level cache will be used when the objects are loaded using their primary key. This includes fetching of associations. In case of second level cache the objects are constructed and hence all of them will reside in different memory locations.
3. Query Cache
Query Cache is used to cache the results of a query. Read here on how to implement query cache.When the query cache is turned on, the results of the query are stored against the combination query and parameters. Every time the query is fired the cache manager checks for the combination of parameters and query. If the results are found in the cache they are returned other wise a database transaction is initiated. As you can see, it is not a good idea to cache a query if it has number of parameters or a single parameter can take number of values. For each of this combination the results are stored in the memory. This can lead to extensive memory usage.
Finally, here is a list of good articles written on this topic,
1. Speed Up Your Hibernate Applications with Second-Level Caching
2. Hibernate: Truly Understanding the Second-Level and Query Caches
3. EhCache Integration with Spring and Hibernate. Step by Step Tutorial
4. Configuring Ehcache with hibernate
One of the best thing in java is that the developers are not required to handle the memory allocations when the objects are created. The JVM does that for us. Most of the time we just need the outer layer knowledge of heap memory and garbage collector. I will share some really interesting investigations that I had. I is a huge topic and I am writing from a point of view of a web application developer, what all minimum I thought we should understand about it.
There are number of good tools available to profile the java applications some of them are,
1. Your Kit Java Profiler
3. Eclipse MAT
4. Visual VM
Out of these Your Kit and JProfilers needs licences and others are free to use products. We are going to use VisualVM. It is a simple yet powerful tool and comes bundled inside the JDK. It has power list of plugins that you can download and use. To start using VisualVM, go to your <JDK_HOME>\bin and run the jvisualvm.exe. I found the below articles it useful to get going.
1.Profiling With VisualVM
2.VisualVM performance tuning tool
3.How to Get VisualVM to Profile JBoss Without Crashing
Since we are talking about memory here make sure you install the Visual GC plugin on the VisualVM as told in this article.
Setting the stage - JVM Memory Structure
The JVM Memory is divided in to 3 parts as shown in the below image. In our applications we are concerned about the Heap Memory. We can input this value to JVM using the parameters,
-Xmx<size> - to set the maximum Java heap size
-Xms<size> - to set the initial Java heap size
|JVM Memory Structure|
The non-Heap memory stores per-class structures such as runtime constant pool, field and method data, and the code for methods and constructors, as well as interned Strings.
Here is a nice article with more details on the JVM memory sizes. Read Javin's article on JVM Heap space here.
The one common confusion is about the stack memory and heap memory. This is well explained here.Stack values only exist within the scope of the function they are created in. Once it returns, they are discarded. Java only stores primitives on the stack. This keeps the stack small and helps keeping individual stack frames small, thus allowing more nested calls. Objects are created on the heap, and only references (which in turn are primitives) are passed around on the stack.
Now, Lets get real. Given below the image from Visual GC, a plugin inside the VisualVM as told earlier. We see many graphs here a detailed decription of the output is available here.
The Game begins - What happens when the application runs
When the objects are created, they reside inside the Eden. When the Garbage collector(GC) runs, if the object is dead (means they are no active references) it is flushed out otherwise they are moved to S1(Survivor Space 1) or S2. This is called a GC cycle. Internal GM algorithm decides the frequency of the GC cycle. Eden + S1 + S2 section of Heap memory is called as Young generation. Objects that survives a fixed number of GC cycles are moved in to Old Gen space. Most number of java objects die as infant and never reach Old Gen. This typically includes local variables which are flushed after the methods are executed.
The frequency of GC cycles inside the Old Gen is much lesser than Young Gen. Typical examples of Old Gen objects are singletons, cached objects and other application wide used data.
When things do not go as per the plan
In a typical application there will be less variation inside the Old Gen space. If the Old Gen space grows linearly with time even after the GC cycle that would lead to a OutOfMemoryError. This might be a indication of a memory leak inside the code. However we might need to use a profiler to find out the exact reason for the same. Here is a Dzon article on some of the Causes of Java EE Enterprise Performance Problems.
These are the basic building blocks of how JVM memory is organized and reacts when the application is executed. From this point there are lots of topics including tuning the memory parameters and garbage collector. I will add some of the useful resources related to this.
1. Java Performance Tuning, Profiling, and Memory Management
2. InfoQ Presentation : Diagnosing Web Application OutOfMemoryErrors
3. InfoQ Presentation : Everything I Ever Learned about JVM Performance Tuning @twitter
4. InfoQ Presentation : Extreme Performance Java
5. Java theory and practice: Garbage collection and performance
1. Be honest and don't bluff
Last week I was trying to create a report using Jasper. In this post I will document some of the resources and links so that it will be useful for any one looking for similar information. I will cover life cycle of Jasper reports, examples and Dynamic Jasper.
As in the image the life cycle has 3 distinct phases,
1. Designing the Report
I have found it really hard to find a working example of Jasper report. But it is right there inside the package shipment!. Once you have downloaded the Jasper Library go to demo\samples, you will find a lot of sample programs. Many of these needs a working HSQL DB connection, to activate it go to demo\hsqldb and start the server. Every folder has a readme.txt file which will help you in understanding how to run it. All the examples can be executed using ant tasks.
Here is a list of few other sources.
DynamicJasper (DJ) is an open source free library that hides the complexity of Jasper Reports, it helps developers to save time when designing simple/medium complexity reports generating the layout of the report elements automatically.
The project homepage provides lots of examples and code snippets on how to use the library. I have been using it for some time and it is a pretty stable replacement for the JRXML file.While using dynamic jasper the report design is coded in Java. Which means every time the report is compiled, filled and exported. By using dynamic jasper we are replacing the first step in the above mentioned jasper life cycle. Even with dynamic jasper you need the jasper library and other dependent files.
3. Khan Academy
5. Code Academy
6. Code School
Related articles on the web:
I have been looking in to NoSQL databases for few months and would like to share my experience with it. This is a post might help you if you indent to start learning about the NoSQL Databases. I would try to link the resources which I found useful here.
Step 1: What is NoSQL?
NoSQL DEFINITION: Next Generation Databases mostly addressing some of the points: being non-relational, distributed, open-source and horizontally scalable. The original intention has been modern web-scale databases. The movement began early 2009 and is growing rapidly. Often more characteristics apply such as: schema-free, easy replication support, simple API, eventually consistent / BASE (not ACID), a huge amount of data and more. So the misleading term "nosql" (the community now translates it mostly with "not only sql").
As seen on NoSQL-Database.org.
If you likes to view the slides, then the below presentation by Tobias Lindaaker on slideshare might inspire you. He gives similar ideas.
MongoDB has an online course MongoDB for Java Developers which is really useful if you are interested in trying out things.
Step 2: How and for what are NoSQL used in Real world?
Once you have some idea, try to find the usage patterns. The above presentations will give lot of information on how these systems are used. You could go through the below links, which explains how specific business problems are solved using NoSQL. This is important because we could easily relate the case studies and get more insights into the capabilities of these systems.
1. MongoDB Customers page.
2. Powerd By Haddop
3. Neo4J Customers Page
Step 3 : Find Usage Patterns that you could work on!
Once you have reached this point, you should try and implement the concepts. Look back at the application that you are working on and see if there is a need for an alternative data store. Do you store Product recommendations? Do you have issues with heterogeneous data? Can your application compromise ACID model for scalability? Do you store XML files or Images on you relational DB? These are some of the questions that you could ask. This way you could determine if there is a serious need of a investigation for a alternative persistence mechanisms. This is in no way means removing the RDBMS completely but moving to a polygot structure of data stores.
If there is no opportunity to try out these concepts in your work, you could create your own test projects and implement them. This way you would encounter problems and will learn from them.
Hope it could be useful for others as well.
Defining Technical DebtAs per Wikipedia, "Technical debt metaphor referring to the eventual consequences of poor or evolving software architecture and software development within a code base". Technical debt is most understood when it is compared with Cost of Change of software. More Debt means the cost of introducing a change in your system is more.
Technical Debt in Agile ContextThe below principle from the list of agile principles was the best one I could find which refers to clearing technical debt "Continuous attention to technical excellence and good design enhances agility". In agile projects working software has a lot of emphasis but this in no way implies that the intrinsic value of software can be compromised. As we push for more and more features it is important that we take time to look back at the health of the application. So, make make sure that your agile projects do not become Fragile as it grows.
The second techniques that we have tried to minimize technical debt is to negotiate with the product owner and take him on board in the technical improvements that we wanted to do. You have to Use the opportunity of Change requests to clean up the functionality. We had a module in our application with lots of bugs. There was a major change request which was planned on that module. After much discussions we identified that it would be much easier to rebuild the module than fixing the issues. Once we rebuild the change request was 75% easier to implement compared to the old code.
Note:- There are many ways to define and minimize technical debt. I have discussed few ways that we have successfully tried. You can go through the above links to find more information and choose the best methods that suits you.
XMind gives a nice listing of the map. You can find the map here. Here is Image which you can download and use.
Finally here is a old fashioned tabbed content list which is easier to copy paste.
Interface - Abstract Class
IS-A vs HAS-A Relationships
Aggregation vs Composition
Method overloading vs Method Overloading
Compile time vs Runtime
Shallow copy vs Deep Copy
Map, List and Set
Equals - Hashcode
Legacy - Synchronized Classes
Stack vs Heap Memory
JRE, JVM, JDK
Checked Vs Unchecked Exceptions
Exception handling best practices
try, catch, finally, throw, throws
String - StringBuffer - String Builder
SAX Based & DOM Based
JAXB - Java API for XML Binding
Packaging the Applications
SOAP, WSDL Webservices basics
Contract first vs
RESTful and its advantages
This is a work in progress and I hope to refine it further. Let me know if you have any comments.
Any third party jars can be added to the maven repository as told here. If you are using very old versions of jar files some of them may not be available in maven repository.Here you can either try upgrading to newer versions or prepare local install as told before. Once you have added all the dependencies try building the application. Watch out for any major issues.
Maven is a build tool. This means your WAR should not change. So, in the last step we compare both versions and make sure they are the same. Make sure you are on top of all the differences. Also, compare the jar files generated by maven and your existing files, Sync them by
Most examples of connecting to database in spring is done using DriverManagerDataSource. If you don't read the documentation properly then you are going to miss a very important point.
1. Server (Container) managed connection pool (Using JNDI)
2. Application managed connection pool
Below user guides can help you configure this.
The below articles speaks about the general guidelines and best practices in configuring the connection pools.
3. How to decide the max number of connections
4. Monitoring the number of active connections in SQL Server 2008
Note:- All the text in italics are copied from the spring documentation of the DriverManagerDataSource.
The principles of Cloud Computing. This chapter deals with definitions, types of clouds (Public, Private and Hybrid) and cloud services (IAAS, PAAS, SAAS).
Most contents in the section are from the The NIST Definition of Cloud Computing paper. Other topics include The Evolution Toward Cloud Computing, Cloud Computing Architectures and Benefits and Limitations of Cloud Computing. The part about Virtualization and its role in the raise of Could computing was quite interesting for me.
Using the Cloud. This part is about accessing the cloud and mobility in the cloud.
This module covers the topics Overview of Accessing the Cloud, How Cloud Computing Can Support Business Processes and Service Providers Using the Cloud.
Security and Compliance. Is about the risks of cloud computing and the measures you can take
This module covers the paper Top Threats to Cloud Computing prepared by the Cloud Security Alliance under the Security Risks and Mitigating Measures title. Managing Identity and Privacy section deals with Triple-A authentication and various aspects of identity management.
Implementing and managing Cloud Computing. You learn about local cloud networks and how to support the use of cloud computing
This module includes the topics Building a Local Cloud Environment and Managing Cloud Services. There is a lot of focus on managing cloud services and related governance frameworks.
Evaluation of Cloud Computing. Examples of the subjects here are cost aspects, (dis)advantages and SLA’s.
This module speaks about the business case for cloud computing. For example cost implications to an organization evaluating the cloud services in terms of capex and opex. Forming the Service level requirements and agreements.
The text in italics are taken from the official exam page.
Written with StackEdit.
|B. Section||C||D. These things happens with You||E. Your Action Plans|
Understanding what to do
|What to do? (40%)||
1. You have missed some of the requirements.
2. You hear others say "This feature was not supposed to work like this"
3. Your completed work gets re-opened during QA or User Testing.
|-Improve your domain knowledge.
-Ask more questions to your PO so that you can impove your understanding of the requirements.
-Push for improved requirements documentation.
-Spend more time in testing your features.
-Listen to sprint demos to get the overview of all the new features added.
Knowledge of Frameworks,
Design patterns, practices and principles
|How to do it - Your skills to do it
(20 - 30 %)
1. You don't know where to start with when you have to implement a new feature
2. You don't know if a similar functionality already exists in the application or not
3. You don't completely understand the frameworks in the application and how they are used
|-Pair program with an experienced developer to learn how he approaches a problem.
-Learn more about the frameworks used in your app.
-Try creating sample applications using them.
-Identify the patterns and principles used in your app and try to use them.
|3||Problem solving, Analytical, Debugging skills||
How to do it - Your Ability to do it
(10 - 20 %)
1. You face difficulties when it comes to writing algorithms
2. You are weak in debugging and finding issues in the code
|-See if you can apply some known patterns to solve the problem.|
|4||Communicating with your code||How well you did it?
How easily somebody can understand how ?
1. Your code is not up to the standards or frequently ignores code quality.
2. You don't have enough code coverage
3. You can't write a quality documentation
|-Use tools like sonar to asses the quality of your code.
-Spend more time in refactoring and improving the code quality.
|5||Communicating about your work||
How well can you communicate about your work
1. Your don't follow the process in the team.
2. Your check-in comments are not useful.
3. Your team don't know what you are working on.
|-Understand and adhere to the team policies. If you feel that there is somethings wrong, communicate and get it clarified.
1. Authentication and AuthorizationWhen it comes to security the most fundamental concepts are Authentication and Authorization. Unless you have a strong reason you should be following a widely accepted framework for this purpose. We have Java EE Authentication and Spring Security to help us out in this context. I have worked with spring security in the past and it can be customized to suite your specific needs.
2. Security in the Web LayerIn our application stack the web layer is most vulnarable to attacks. We have may established standard practices and detection mechanisms to minimize these risks. OWASP Top 10 list is a must have check point for security checks. The Open Web Application Security Project (OWASP) mission is to make software security visible, so that individuals and organizations worldwide can make informed decisions about true software security risks.
3. API SecurityWith the rise of mobile applications and stronger browsers expressing functionalities using the API is more popular day by day. We need to follow the same security practices for the web layer. All the API requests should be authenticated and we should use the principle of least privilege. I found the presentation from Greg Patton in the AppSec EU15 titled The API Assessment Primer is a great start for API security validations. Two major points focused in his talk are,
Do not expose any operations that are not needed
Do not expose any data that is not required
Which is in line with the basic security principle of giving least privilege by default.
To authenticate the services, we can create simple token-based API authentication mechanism based OAuth2 standards. If the services expose any sensitive data, it is better to use https so that man-in-the-middle attacks can be avoided.
Other Useful Reference Materials
|Tests||Naming convention||Runs at||When to use||Exec Time|
|Unit Test||Ends with Test||Every build||Rule based implementations where the logic can be tested in isolation||Few Milliseconds|
|App Level Tests||Ends with TestApp||Every build / Nightly builds (Teams choice)||Tests the service layers in connection with others. Frees you from creation of mock objects. Application context is loaded in the tests.||Few Seconds|
|Integration Test||Ends with TestIntg||Runs on demand when a special profile is used in build.||All the above + Use when you need to connect to external points like DB, web services etc..||Depends on the integration points.|
|Manually Running Tests||Ends withTestIntgManual||Manually running tests, Used debugging a specific problem locally||All the above - Can't be automated.||Depends on the integration points.|
Here is a video recording of my session titled Practical communication strategies for software architects on Bangalore software architect meetup.
The session covers communication ideas for various stages and to different stakeholders in a project scenario.
Have a look at the video recording of the session