2.58 Conclusion – We found that overall progress on the department’s efforts to place more attention on planning large IT projects, managing the necessary transition, and involving all key stakeholders has been unsatisfactory given that two of the four projects, the Global Case Management System and the Expenditure Management Information System, have significant governance issues (Exhibit 2.4).

Exhibit 2.4 - Recommendation 3.61 (from November 2006 Chapter 3 Large Information Technology Projects) : Government departments and agencies should improve their external quality reviews of IT projects.  Senior departmental executives should review the key decision documents that are produced to support the IT project and ensure the analysis is thorough and supportable before signing off the submission.  

Progress – Satisfactory – IRC

Nil response

Progress made on this recommendation was satisfactory for CRA (IRC).

2.84 (formerly 2.85) Conclusion – The Secretariat has met its commitments and developed and published business guidance to be used by departments in preparing their business cases (see paragraph 2.26 and Exhibit 2.1) . However, we found that three of the five projects we examined in this audit – GCMS, IRC, and Secure Channel – continue to have deficiencies regarding identification and measurement of benefits (Exhibit 2.5).

2.85 (formerly 2.86) Recommendation – When departments and agencies develop a business case, they should include a benefits measurement plan that

o   contains specific quantifiable benefits including clear benchmarks,

o   assigns responsibilities for delivering identified benefits, and

o   includes a process for measuring and reporting benefits.

Agreed. The Agency believes that since the end of the audit (October 31, 2010) it has addressed the recommendations in the following manner:  As noted by the Auditor General, a benefits measurement plan for IRC Phase 1 had been completed by the CRA in 2009. The benefits measurement plan was further refined in 2010 to reflect annual gains to be achieved, benchmark information, a data collection strategy including when benefits would start to be realized, and responsibility for delivery of the identified benefits.  The 2010 revised plan was recently approved and work is ongoing in respect of measuring and reporting on the benefits.

In fact, the approval process in place at the CRA requires a benefits measurement plan be prepared as part of business cases for large IT-enabled projects which is an established best practice.

As noted in the Agency’s response, work is ongoing in respect of measuring and reporting on the benefits of IRC Phase 1 and includes target dates for the realization of these benefits as follows:

• Technology Transition to Business Processes – Ongoing through 2011/12
• Benefits Measurement Cycle – Ongoing through 2012/13
• Benefits Measurement Report – 2013/14

2.94 (formerly 2.95) Conclusion – We found that for three of the four projects included in this portion of the audit, progress on improving organizational capacity was satisfactory.  EMIS did not access organizational capacity as required (Exhibit 2.6).

Exhibit 2.6 – Recommendation 3.98 (from November 2006 Chapter 3 Large Information Technology Projects) :

At the start of a project, departments and agencies  should clearly demonstrate that their organization is ready to accept the business transformation that comes with the project and has the capability and commitment to successfully deliver the project. Specifically, departments and agencies should analyze  their track record in

• completing projects of similar size and complexity;
• completing skills appraisals and plans to address shortfalls;
• planning for business preparation, transition, and operational phases; and
• considering key stakeholders buy-in and commitment.

Progress -  Satisfactory – IRC

Nil response

Progress made on this recommendation was satisfactory for CRA (IRC).

2.106 (formerly 2.107) Conclusion – We found that overall progress on the implementation of risk management for the projects we reviewed has been satisfactory, although progress is unsatisfactory for the Temporary Resident Biometrics (Exhibit 2.7).

Exhibit 2.7 – Recommendation 3.111 (from November 2006 Chapter 3 Large Information Technology Projects) : Departments and agencies should

• comply with the Treasury Board Secretariat project management guidance,
• closely manage the project risks,
• monitor key success factors, and
• regularly report to management any significant internal or external event that may prevent the expected project benefits or result in cost overruns or in the project not being delivered on time.  

Progress -  Satisfactory – IRC

Nil response

Progress made on this recommendation was satisfactory for CRA (IRC).