COST ACCOUNTING

A mid-market Engineer-to-Order manufacturer in the aerospace and defense sector retained the services of TEC to help select an Enterprise Resource Planning System. This system is intended to replace disparate applications that lacked the functionality and integration to support the company's Lean Manufacturing initiatives.

One of the most important steps in TEC's selection methodology is the Scripted Scenarios sessions. This step requires vendors that make the short list to validate their functionality with respect to the client's business requirements through product demonstrations (To learn more about TEC's selection methodology see An Overview Of The Knowledge Based Selection Process). The sessions require the vendor to spend over 15 hours of demonstration time showing the client how their software can be used to address specific business issues. Flexibility, Ease of Use/Navigation and Process Fit were also evaluated. Four vendors made the short list:

The Scripted Scenarios allowed TEC to gain a detailed understanding of the strengths and weaknesses of each product. ERGO 2001, TEC's decision support tool, was used to record and analyze the results of the scenarios (For more information on ERGO 2001 see ERGO 2001 IT Evaluation Tool).

About this note: This is a two part note. Part 1 includes the Business Scenarios Model and Results. Part 2 contains the Qualitative Assessments and Analysis, and User Recommendations.

Business Scenario Model

TEC worked in conjunction with the client to develop over 450 specific functional software requirements resulting from business issues (scenarios) organized into a hierarchical tree. The high level illustration of the business scenarios model below indicates how the scenarios were organized.

Figure 1. Business Scenarios Model Tree.

Hierarchical functional requirements exist within each of the categories under Business Scenarios. The client weighs the importance of each of these categories as it relates to their business and scores each vendor on their ability to demonstrate the product's functionality. The weights appear below.

Figure 2. Business Scenarios Model Weights.

Results

Figure 3 illustrates the overall percent match for each of the vendors within Business Scenarios. Percent match is an optimal decision method which measures the compatibility of the detailed expectations of the decision maker, and compares it to ERGO 2001's detailed analysis of the options, using pattern matching statistical loss function approaches. It is usually a non-linear method of comparison.

Figure 3. Business Scenarios Results.

Figure 3 indicates that IFS took the overall lead in Business Scenarios by a slim margin over SAP. IFS also fared well in Product Ease of Use/Navigation, Process Fit and Flexibility. Figure 4 illustrates the results in Product Ease of Use/Navigation. The client generally felt that both IFS' and JD Edwards' products were intuitive and easy to use. JD Edwards' screens mimic the windows explorer environment and IFS uses customizable, instinctive right mouse click menu options. The client was impressed with the breadth and depth of SAP, but was dismayed by the cumbersome screen navigation and perceived steep learning curve. TEC felt that Oracle could have scored higher in this area, but the demonstration did not run smoothly and may have impacted client scores.

Figure 4. Product Ease of Use/Navigation Results

One of the key differentiators during the demonstration was the vendors' fit with the client's processes. This criterion, Process Fit, accounted for 13.5% of the model's overall weight. Figure 5 shows that IFS was the strongest in demonstrating the best match to the client's processes while Oracle and SAP were virtually tied for second place. JD Edwards' was not as strong due to a very weak fit in Product Repair and Customer Service. This criterion was dominated by Oracle, while JD Edwards showed the best fit with regards to Production Material Management and Cost Accounting. IFS led the pack in Product Definition, Supplier Definition and Procurement, Customer Definition and Proposal, Sales Order Processing, and System Overhauls and Upgrades. All vendors scored equally well in Goods Receipt & Handling, Goods Inspection, and Goods Shipment & Delivery.

Figure 5. Process Fit

To assess flexibility, TEC developed the criteria in Figure 6 with the client. These factors measured the perceived cost and risk associated with the implementation and modification of each vendor's proposed solution.

Figure 6. Flexibility Model Tree

Figure 7 illustrates the results in Flexibility. Both JD Edwards and IFS scored well in Flexibility primarily because both vendors focus on reduced implementation time and adaptable reporting and workflow features. Furthermore, IFS sells applications as a set of integrated, loosely coupled components. Implementation can be easily phased by choosing to implement a few components at a time. Oracle and SAP were perceived by the client as being rigid, monolithic applications that have a number of interdependencies amongst modules, thus requiring significantly more implementation time.

Figure 7. Flexibility Results

Although IFS had the overall highest score, it did not score the highest in each section. Figure 8 is a radar diagram that reveals how well each vendor did among the Business Scenarios sections as well as Product Ease of Use/Navigation and Flexibility.

Figure 8. Radar Diagram of Vendor Scores

The radar diagram indicates that all four vendors have very comparable products with regard to functionality. What particularly differentiated the vendors were Product Ease of Use/Navigation, Process Fit and Flexibility. These areas of the model can significantly impact the final selection decision depending on the importance assigned by the client. These three criteria were significant differentiators in the decision because they carried a combined weight of 36.5% in the model.

See Part 2 of this note for the Qualitative Assessments and Analysis of the vendors, and User Recommendations.

Sidebar Information

The graphs shown have been generated using the TEC's patented decision analysis software ERGO (formerlyTESS), which uses the Multi-Attribute Utility theory (MAU), Analytic Hierarchy Process (AHP) and TEC's patented decision science to compare vendors and products relative to one another in a statistically valid model.