J2EE project dangers!

In my various tenures as developer, senior developer, and architect, I have seen the good, the bad, and the ugly when it comes to enterprise Java projects! When I ask myself what makes one project succeed and another fail, it is difficult to come up with the perfect answer, as success proves difficult to define for all software projects. J2EE projects are no exception. Instead, projects succeed or fail by varying degrees. In this article I aim to take the top 10 dangers that affect the success of an enterprise Java project and showcase them for you, the reader.

Some of these dangers simply slow down the project, some are false trails, and still others can outright ruin any chance of success. However, all are avoidable with good preparation, knowledge about the journey ahead, and local guides who know the terrain.

This article is simple in structure; I will cover each hazard as follows:

• Danger name: One-liner outlining the hazard
• Project phase: The project phase where the hazard occurs
• Project phase(s) affected: In a lot of cases, hazards can have a knock-on effect on later project phases
• Symptoms: Symptoms associated with this hazard
• Solution: Ways to avoid the hazard altogether and how to minimize its effects on your project
• Notes: Points I wish to impart that relate to the hazard, but don’t fit into the previous categories

As noted above, we’ll examine each danger in the context of an enterprise Java project, along with its important phases. The project phases cover:

• Vendor Selection: The process of picking the best possible mix of tools before you start your J2EE project — from the application server right down to your brand of coffee.
• Design: In between a rigorous waterfall methodology and an approach of “code it and see,” lies my take on design: I do enough design so that I can move comfortably into development. I consider my design phase complete when I know exactly what I am building and how I will build it. Moreover, I use design templates to make sure I have asked all the right questions of myself and my proposed solution before moving into development. However, I’m not afraid to code in this phase; sometimes it’s the only way to answer a question on say, performance or modularity.
• Development: The project phase where the amount of work done in earlier phases will show. A good choice of tools coupled with a good design doesn’t always mean a super-smooth development, but it sure helps!
• Stabilization/Load Testing: In this phase, the system architect and project manager will impose a feature freeze and focus on build quality, as well as ensure that the system’s vital statistics — number of concurrent users, failover scenarios, and so on — can be met. However, quality and performance should not be ignored until this phase. Indeed, you can’t write poor-quality or slow code and leave it until stabilization to fix.
• Live: This is not really a project phase, it’s a date set in stone. This phase is all about preparation. It’s where the ghosts of past mistakes will come back to haunt your project, from bad design and development to a poor choice of vendors.

Figure 1 illustrates the project phases most affected by the different causes (and in particular the knock-on effects).

Well then, without further ado, let’s dive right into the top 10!

Danger 1: Not understanding Java, not understanding EJB, not understanding J2EE

Right, I’m going to break this one into three subelements for easier analysis.

Description: Not understanding Java

Project phase:

Development

Project phase(s) affected:

Design, Stabilization, Live

System characteristics affected:

Maintainability, scalability, performance

Symptoms:

• Reimplementing functionality and classes already contained in the JDK core APIs
• Not knowing what any or all of the following are and what they do (this list represents just a sample of topics):
  • Garbage collector (train, generational, incremental, synchronous, asynchronous)
  • When objects can be garbage collected — dangling references
  • Inheritance mechanisms used (and their tradeoffs) in Java
  • Method over-riding and over-loading
  • Why java.lang.String (substitute your favorite class here!) proves bad for performance
  • Pass-by reference semantics of Java (versus pass-by value semantics in EJB)
  • Using == versus implementing the equals() method for nonprimitives
  • How Java schedules threads on different platforms (for example, pre-emptive or not)
  • Green threads versus native threads
  • Hotspot (and why old performance tuning techniques negate Hotspot optimizations)
  • The JIT and when good JITs go bad (unset JAVA_COMPILER and your code runs just fine, etc.)
  • The Collections API
  • RMI

Solution:

You need to improve your knowledge of Java, especially its strengths and weaknesses. Java exists far beyond just the language. It’s equally important to understand the platform (JDK and tools). Concretely, you should become certified as a Java programmer if you aren’t already — you’ll be amazed how much you didn’t know. Even better, do it as part of a group and push one another. It’s also more fun this way. Further, set up a mailing list devoted to Java technology and keep it going! (Every company I have worked with has these lists, most of which are on life-support due to inactivity.) Learn from your peer developers — they’re your best resource.

Notes:

If you or other members of your team do not understand the programming language and the platform, how can you hope to build a successful enterprise Java application? Strong Java programmers take to EJB and the J2EE like ducks to water. In contrast, poor or inexperienced programmers will construct poor-quality J2EE applications.

Description: Not understanding EJB

Project phase:

Design

Project phase(s) affected:

Development, Stabilization

System characteristics affected:

Maintenance

Symptoms:

• EJBs that work when they are first called but never thereafter (especially stateless session beans that are returned to the ready pool)
• Nonreusable EJBs
• Not knowing for what the developer is responsible, compared with what the container provides
• EJBs that do not correspond to the specification (fire threads, load native libraries, attempt to perform I/O, and so on)

Solution:

To improve your EJB knowledge, take a weekend and read the EJB specification (the 1.1 version is 314 pages long). Then read the 2.0 specification (524 pages!) to get a feel for what the 1.1 specification didn’t address and where the 2.0 specification will take you. Concentrate on the parts of the specification that tell you, the application developer, what are legal actions in an EJB. Sections 18.1 and 18.2 are good places to start.

Notes:

Don’t look at the EJB world through your vendor’s eyes. Make sure you know the difference between the specifications underpinning the EJB model and a particular take on them. This will also ensure that you can transfer your skills to other vendors (or versions) as needed.

Description: Not understanding J2EE

Project phase:

Design

Project phase(s) affected:

Development

System characteristics affected:

Maintenance, scalability, performance

Symptoms:

• The “Everything is an EJB” design
• Manual transaction management instead of using the container-provided mechanisms
• Custom security implementations — the J2EE platform has probably the most complete and integrated security architecture in enterprise computing, from presentation right through to the back end; it is rarely used to its full capabilities

Solution:

Learn J2EE’s key components and what advantages and disadvantages each component brings to the table. Cover each service in turn; knowledge equals power here.

Notes:

Only knowledge can fix these problems. Good Java developers make good EJB developers, who in turn are ideally positioned to become J2EE gurus. The more Java and J2EE knowledge you possess, the better you will be at design and implementation. Things will start to slot into place for you at design time.

Danger 2: Over-engineering (to EJB or not to EJB)

Project phase:

Design

Project phase(s) affected:

Development

System characteristics affected:

Maintenance, scalability, performance

Symptoms:

• Oversized EJBs
• Developers who can’t explain what their EJBs do and the relationships among them
• Nonreusable EJBs, components, or services when they should be
• EJBs starting new transactions when an existing transaction will do
• Data isolation levels set too high (in an attempt to be safe)

Solution:

The solution for over-engineering comes straight from the extreme programming (XP) methodology: design and code the bare minimum to meet the requirements from scoping, nothing more. While you need to be aware of the future requirements, for example future average load requirements or behavior of the system at peak loading times, don’t try to second-guess what the system will need to look like in the future. In addition, the J2EE platform defines characteristics such as scalability and failover as tasks the server infrastructure should handle for you.

With minimal systems, composed of small components designed to do one thing and do it well, the reuse level improves, as does system stability. Moreover, your system’s maintainability strengthens and future requirements can be added much more easily.

Notes:

In addition to the solutions listed above, employ design patterns — they significantly improve your system design. The EJB model itself uses design patterns extensively. For example, the

Home

interface of every EJB is an example of a Finder and Factory pattern. The remote interface of an EJB acts as a Proxy for the actual bean implementation and is central to the ability of the container to intercept calls and provide services such as transparent load balancing. Ignore the value of design patterns at your peril.

Another danger that I continually warn against: using EJB for the sake of it. Not only might some parts of your application be modeled as EJBs when they shouldn’t be, your entire application might use EJBs for no measurable gain. This is over-engineering taken to extremes, but I have seen perfectly good servlet and JavaBean applications ripped apart, redesigned, and implemented using EJBs for no good technical reasons.

Danger 3: Not separating presentation logic from business logic

Project phase:

Design

Project phase(s) affected:

Development

System characteristics affected:

Maintainability, extensibility, performance

Symptoms:

• Large and unwieldy JSPs
• You find yourself editing JSPs when business logic changes
• A change in display requirements forces you to edit and redeploy EJBs and other backend components

Solution:

The J2EE platform gives you the opportunity to separate presentation logic from navigation and control, and finally from business logic. It’s called the Model 2 architecture (see Resources for a good article). If you have already fallen into this trap, a stiff dose of refactoring is called for. You should at least have thin vertical slices that are for the most part self-contained (that is, how I order a widget is a separate slice from how I change my username or password). Use this implicit organization of your system to refactor in stages.

Notes:

Using a consistent design in conjunction with a UI framework (taglibs for example) will also help ensure that you avoid logic separation problems on your project. Don’t bother building another GUI framework for your own needs, there are too many good implementations readily available. Evaluate each in turn and adopt the framework that best fits your needs.

Danger 4: Not deploying where you develop

Project phase:

Development

Project phase(s) affected:

Stabilization, Parallel, Live

System characteristics affected:

Your sanity

Symptoms:

• Multiday or weeklong transitions to live systems
• The risk involved in going live is substantial, with many unknowns and major usage scenarios not tested
• Data in live systems not the same as data in development or stabilization setups
• Inability to run builds on developer machines
• Application behavior is not the same in the development, stabilization, and production environments

Solution:

The solution to Danger 4 begins with duplicating the production environment faithfully in your development environment. Develop on the exact same setup as that on which you plan to go to live — don’t develop on JDK 1.3 and Red Hat Linux when you plan to go live on JDK 1.2.2 and Solaris 7. Further, don’t develop on one application server and go live on another. Also, get a snapshot of data from the production database and use it for testing, don’t rely on artificially created data. If production data is sensitive, desensitize it and load it up. Unexpected production data will break:

• Data validation rules
• Tested system behavior
• Contracts between system components (EJB-EJB and EJB-database in particular)

Worst of all, each of these will result in exceptions, null pointers, and behavior that you have never seen before.

Notes:

Developers frequently leave security until stabilization (“Yeah, the screens work, now let’s add in the user validation stuff.”). Avoid this trap by devoting the same amount of time to implementing security as you do to business logic.

Going live is a complicated process, as fraught with political issues as it is with technical ones. You will run into unexpected problems; that’s what live is all about. Development and stabilization environments give you the room to make many mistakes and find these problems before going live! Use it, as that will remove most of the pain and risk from taking your application live.

The more projects you get under your belt, the more you will already know what works and what doesn’t. Keep an engineering notebook for yourself and your peers. Your goal should be to never make the same mistake twice in going live.

Danger 5: Choosing the wrong vendors

Project phase:

Vendor Selection

Project phase(s) affected:

Design, Development, Stabilization/Load Testing, Live

System characteristics affected:

Scalability, performance, maintainability, stability

Symptoms:

• Developers spending more time wrestling with tools than productively using them
• Significant system redesign required in order to work around known or unknown bugs in the implementation
• Poor or zero integration between different tools (application servers and IDEs, IDEs and debuggers, source control and build tools, the list goes on)
• For IDEs, debuggers, etc., developers simply forsaking them in favor of their own favorite tools

Solution:

To avoid Danger 5, you need a good vendor selection process. Danger 10 is applicable here too.

For IDEs, the only way to really evaluate one is to use it. The only way to evaluate a J2EE implementation is to build a proof of concept that touches the features you are betting your architecture on. Indeed, you do not want to find a bug in the clustering implementation when you have already spent three months in development and invested in specific training.

What if, halfway through development, you have trouble with your toolset? Well, some tools prove more important than others. If your chosen application server inadequately meets your needs, you need to suck it up and change the specification. If the IDE stinks, set minimal levels around coding standards (tabs versus spaces and so on) and let the developers pick whatever makes them the most productive.

Notes:

Knowing the best vendor or tool for a particular task is not a “do once” job. You need to continually evaluate the market. For example, I have used four different IDEs in the past 12 months, depending on my application server, platform, and whether or not I was coding EJBs.

Danger 6: Not knowing your vendors

Project phase:

Vendor Selection

Project phase(s) affected:

All phases after Vendor Selection — Design, Development, Stabilization/Load Testing, Live

System characteristics affected:

Maintainability, scalability, performance

Symptoms:

• Development takes 33 percent longer than the worst estimate
• Developers reinvent the wheel when the vendor or implementation provides the required functionality out of the box

Solution:

To avoid the dangers resulting from not knowing your vendors, subscribe to all the vendor-supplied support resources you can find such as email lists, news groups, and release notes (especially those with a list of bug fixes); you will get invaluable information.

Once you’ve picked your vendors, invest in training as soon as possible, ideally well before the project kicks off. Next, build a quick proof of concept to break the team in gently. Build a couple of EJBs and deploy them, then call to them from your presentation-layer technology (Swing GUI, JSPs, and so on). If you try to construct your development environment while concurrently trying to meet a project goal, the environment setup will always play second fiddle. Indeed, I have seen projects where there was never a build process in place because, “We didn’t have time.” That became even more profound when the team had to stay until 11:00 each night just to get the application together for a release to development. Therefore, take the time up front to nail this stuff down — it will save huge amounts of time down the road. To those who say, “Our schedule doesn’t give us the time,” I reply, “Your schedule doesn’t give you the time to not do it.”

Notes:

How transferable are vendor-specific skills in the J2EE world? Let’s look at a concrete example of how two vendors — IBM and BEA Systems Inc. — that support EJB 1.1 produce dissimilar application servers. Indeed, how alike are BEA WebLogic 5.1 and IBM WebSphere 3.5?

1. BEA WebLogic is configured and administered in a completely different manner than IBM WebSphere.
2. IBM adopted an all-GUI environment philosophy with WebSphere. In contrast, BEA provided a whole host of command line utilities with WebLogic.
3. IBM WebSphere uses IIOP to communicate and throws CORBA exceptions that are visible to the programmer; WebLogic has no CORBA infrastructure at all, and uses the t3 protocol by default.
4. WebSphere is joined at the hip with Visual Age, while WebLogic is more IDE-agnostic. Indeed, you can develop for WebLogic using almost any IDE.

These are just a few points of difference; many more exist. Bottom line: most vendors are as alike as chalk and cheese. Being an expert on one application server does not mean you are an expert on all. The arguments above apply to anything: IDEs, debuggers, build tools, configuration management, and so on. Having experience with a particular tool is a good thing that positions you to evaluate a competitor. However, it does not mean you can transition seamlessly from one product to another. As such, allow yourself the time required to be skilled with your tools.

Danger 7: Not designing for scalability or performance

Project phase:

Design

Project phase(s) affected:

Development, Load Testing, Live

System characteristics affected:

Scalability, performance, maintainability

Symptoms:

• Unacceptably slow systems (that are not amenable to refactoring)
• Systems that make heavy use of server-side state and cannot take full advantage of vendor clustering technology

Solution:

With Danger 7, close in on your performance and scalability requirements in the scoping phase — know what numbers you need to hit before you move into development. If you know you need 50 transactions per second and your all-entity bean design will provide only 40, then you need to look at alternative solutions such as stored procedures, batch processes, or reworked OLTP aspects of your system.

Involve your vendor as much as possible in the scaling requirements — they should know the strong and weak characteristics of their product and help you accordingly.

Notes:

Not designing for scalability or performance is often seen as a conflict to Danger 2 (over-engineering). In fact, they complement each other. My solution to Danger 2 advocates building only what is absolutely necessary. By addressing performance and scalability, you set an upper bound on what is necessary.

If you identify massive scalability as a key requirement, then you will first specify an application server with great clustering support, probably with a transactional cache underneath for performance. Further, you will engineer business objects as EJBs in order to take advantage of the server architecture. XP won’t have a problem with this, you are still building only what is absolutely necessary.

I view this line of thinking as a method of providing checks and balances. While I want the simplest possible system, that system needs to provide the functionality and behavior required by the client.

Danger 8: Antiquated development processes

Project phase:

Development

Project phase(s) affected:

Stabilization, Live

System characteristics affected:

Maintenance, code quality

Symptoms:

• A project plan that looks suspiciously like the waterfall method: “First we design the system from scratch, then we sit down and code it in one huge chunk of time.”
• Because a build process doesn’t exist, builds are a nightmare.
• Build days equal lost development days because nothing else gets done.
• Components are not adequately tested before integration. Indeed, integration testing means taking two unstable components, strapping them together, then looking at the stack traces.

Solution:

A good software methodology will save your life. I’ve already mentioned XP; the Resources section below contains a link to the site. There you will find XP covered in great detail.

Notes:

I can’t believe that I ever managed without JUnit for unit testing and Ant for building — two free tools that underpin the XP methodology. For more on JUnit and Ant, see Resources.

Danger 9: Failure to employ frameworks

Project phase:

Development

Project phase(s) affected:

Development, Stabilization, Live

System characteristics affected:

Maintenance, extensibility, code quality

Symptoms:

• Bugs in core libraries that are used multiple times in code.
• No set logging standards — so system output is impossible to read or parse with scripts.
• Bad and inconsistent exception handling. At some sites I have seen, end users are exposed to low-level system errors, for example sending back a SQLException stack trace when a user tries to check out their shopping cart. What is the user going to do? Ring up the database administrator and request a fix on a primary key constraint?

The following tasks are handled in a myriad of ways by developers and should be among the first targets of any framework:

• Logging
• Exception handling
• Getting a connection to a resource (database, naming service, and so on)
• Building JSP pages
• Data validation

Solution:

I am a huge believer in heavily used, lightweight frameworks. In fact, my first JavaWorld article — “Frameworks Save the Day” — discussed frameworks in enterprise Java environments. If you are already in development, you will still reap huge dividends by adopting a framework now. You will experience some pain in reworking tasks such as exception handling and logging, but in the long run it is time and money well spent.

Notes:

Think of different levels of reuse when it comes to frameworks and component-based development in general. The first level, plumbing, has a reuse factor of 0.9 or higher, which means 90 percent of projects will use it. The more specific the service, the lower the reuse factor. That is, I might build an accounting service to manage resource usage and expect to use it on 50 percent of projects. But for those projects that need it — boy, are the developers ever glad it’s there!

Danger 10: Basing project plans and designs on marketing blurb, not on technical fact

Note: Danger 10 didn’t appear on my list, until I realized that many more people than I thought harbor serious misunderstandings about enterprise Java, particularly new arrivals to the field.

Project phase:

All phases are affected, with Vendor Selection particularly influenced

Project phase(s) affected:

All phases are affected

System characteristics affected:

Maintenance, extensibility, design quality, code quality

Symptoms:

• Technical decisions taken lightly because EJBs are designed to be portable
• Vendor selection performed without a “hands-on” trial of the product
• Needing to switch tools during a project lifecycle

Solution:

Don’t believe anyone with a vested interest outside of your project. This means: don’t believe vendors (unless you know them already) and don’t believe white papers paid for by vendors. If you want some real-world advice on application servers, check out the reviews links in Resources. Going further, download the tool you want to evaluate, roll up your sleeves, and prototype with it. Run through the examples provided (any good vendor will have these).

To sum up, selecting the right vendor and toolset for your project takes time, time you probably don’t have. Narrow down your selection to three to four choices, then take them for a drive. Build a weeklong proof of concept using your initial choices of application server, IDE, build process, and so on. Be comfortable that you can do what the development plan calls for with the tools you have.

Notes:

If you are short on J2EE expertise, that will hurt you most in the initial stages of your project. Decisions made at the beginning produce huge knock-on impacts on the overall success of the project. A good J2EE consultant will be able to run a great vendor selection process and leave you in great shape moving into design and development.

Just 10 dangers to worry about?

Ten is an arbitrary number that serves as a cutoff point — numerous other dangers exist. Indeed, I know of a couple more myself. But even so, if you address the ones listed here, I guarantee you that your project will be in excellent shape and positioned to succeed.

If there is one thing that I urge you to take away from reading this article it is this: There is no substitute for experience and planning. If you don’t have experience, get it. Don’t bet on on-the-go training for yourself or your team during a project. Be proactive and source it before development, ideally before design commences. Seed your team with Java and J2EE mentors to act as guides for the project’s overall direction and make sure that they grow other, less experienced team members along the way.

Finally, the more I wrote, the more I realized I also want to talk about:

• Social aspects of software engineering
• Unit testing versus integration testing (when is testing done?)
• Design patterns
• Exception handling

Alas, I just don’t have room. Perhaps another article soon. Anyway, good luck!

Conclusion

Well, there it is. The top 10 dangers that, taken together, are responsible for most if not all of the problems you will face or have already encountered in enterprise Java development. I am certain that other pitfalls await you on your journey, but I believe that I have covered the main ones. Just to recap, below are the top 10 dangers ordered according to priority:

1. Not understanding Java, not understanding EJB, not understanding J2EE
2. Over-engineering (to EJB or not to EJB) (design patterns)
3. Not separating presentation logic from business logic
4. Not deploying where you develop
5. Choosing the wrong vendor(s)
6. Not knowing your vendor(s)
7. Not designing for scalability or performance
8. Antiquated development processes
9. Not using frameworks
10. Basing project plans and designs on marketing blurb, not technical fact

Humphrey Sheil, a Sun-certified J2EE architect
and programmer, has worked on over 20 enterprise Java projects in
Europe and in the United States, from initial scoping right through
to live. Humphrey is a technical architect at Teogas Systems in Dublin,
Ireland.

Source: www.infoworld.com