Infusing Project Management techniques into a Results-based Management approach leads to better accountability

Infusing Project Management techniques into a Results-based Management approach leads to better accountability

Marje Aksli 

Two Canadian projects, G-8 Summit in 2002 and resettlement of Syrian Refugees in 2016, exemplify how international development or public policy objectives can be met more effectively and efficiently when a results-based-management (RBM) approach is supported by using project management (PM) tools.  

In short, could government programs, currently running on RBM, be made more effective and efficient if they were managed using PM principles, as well?

The Development Assistance Committee at the Organization for Economic Cooperation and Development (OECD DAC) has outlined five criteria for evaluating projects and programs: relevance, effectiveness, efficiency, sustainability, and impact. Setting the latter two aside, for now, we discuss the multiple sets of tools to adequately meet the relevance, effectiveness and efficiency criteria. 

RBM and PM 

The most widely used management tool in international development, and in public service, is Results-Based-Management, or RBM, which connects the project outputs to desired outcomes in a measurable way. For example, we assume that if a country accepts and resettles tens of thousands of refugees from the Middle East - a project output in the Operation Syrian Refugees - suffering is alleviated, which is the desired outcome. Similarly, coordinating G-8 Summits – project output – would, in theory, lead to an increased global visibility and influence for Canada – the desired outcome. Project relevance is thus established with the logical and causal connection from outputs to outcomes through RBM. 

Setting relevance aside, the managers also need to ensure projects are implemented effectively and efficiently as per OECD requirements. This entails achieving project outputs effectively (within the project scope) with scarce inputs or tax dollars (efficiently). Though RBM is focused on achieving higher level results (outcomes), it provides no tools to manage project outputs in a cost-effective and timely manner. In effect, RBM is not designed to provide the guidance required for achieving effectiveness and financial efficiency – two of the main DAC evaluation criteria.

The systemic managerial approach, specified in the Project Management Body of Knowledge (PMBoK) does provide a logical normative system for managing projects effectively and efficiently. Several lessons learned sections of evaluation reports indicate that using PM tools would help to control project scope, which relates to effectiveness, and cost, which explains the efficiency of a project. PMBoK principles could also assist with stakeholder management, a crucial aspect in a project’s success. Still, the public sector and organizations involved in program development seem to overlook these obvious benefits because they have yet to practice PM tools to the same extent as RBM and deliverology.

For better effectiveness and efficiency in tax payer funded programming, results-based management and project management tools should be used in combination because they are complementary by design. RBM ensures that project outputs are connected to project outcomes through logical and causal connections (relevance) and PM tools ensure the management of projects is focused on scope and budget (effectiveness and efficiency).  

Public servants and NGO programming personnel would benefit from extensive training in PM techniques and they should be encouraged to use them. A change in management styles through PM training would also provide a more business-oriented approach to development or to public sector programming, one that mitigates risks through a judicious use of resources. 

By breaking down project components into a 2D Matrix relating knowledge areas, such as project cost, essential human resources, project duration, scope, risk, stakeholders, and quality, with project phases or life cycles, PM tools lay out a set of manageable tasks in a coherent process. In the project initiation phase, the PM tools instruct managers to work on two knowledge areas only: to specify a project goal and to identify stakeholders based on an assessment of needs. In the proceeding planning phase, managers have to engage with many additional project knowledge areas. The project description is explicitly stated in the planning phase and it becomes the project scope. The work needed to achieve this scope, and only this scope, is then broken down into manageable work packages or activities to which cost is assigned, human resources allocated, duration stipulated; risk level mitigated and quality identified. Only then can the manager know the total cost, the duration, and risks of the project. Similarly, PM tools offer guidance for three consequent phases: execution, monitoring and controlling, and for the closing phases. 

Lessons to be learned 

Comparing project closure or evaluation reports to the PMBoK norms reveals several short comings. The biggest deviations between PM theory (the norm) and management practice in Canadian government departments result from omitting, or shortening, the initiation phase and from blending the planning phase with the execution phase. This leads to confusion in many knowledge areas and severely impacts both project scope and cost management. The final report of Kananaskis Summit from 2002, for example, did not distinguish between words such as ‘planning’ and ‘managing’. The two words were used interchangeably in the meaning ‘organizing’, which seems to indicate that the implementation team did not understand how to sequence or prioritize activities in distinct project phases. This became acutely apparent in the time and scope management areas. Additions to the project scope expanded until late in the project execution phase, which led to ballooning budget and unmet commitments. 

The merged planning and execution phases resulted in several planning documents not being completed, which consequently hindered the controlling of project scope and led to difficulties in estimating the exact costs and necessary human resources. For example, the report regrets that environmental impacts were not addressed at the Summit and suggests, correctly, that these requirements should have been included with the scope requirements in the planning phase. Promises were made about the, so called, legacy projects to First Nations – the most important stakeholders, and event co-hosts, yet these add-on projects were not included in the original project scope and were therefore not part of the plan and were consequently not delivered because of insufficient funds. 

Similarly, the Operation Syrian Refugees (OSR) in 2016, which aimed to resettle 25.000 refugees between November 2015 and February 2016, suffered under the lack of interdepartmental coordination during the initiation and planning phases. This led to significant consultation gaps between the government departments, resulting in unclear responsibilities and competing priorities between different government departments. For example, the planning model used by the lead department IRCC did not sufficiently address the significant efforts required to set up operations abroad during the operation’s earlier phases. The activities abroad were not well planned, consequently not assigned costs, durations, human resources, or risk ratings for and were thus managed poorly. Managers cannot execute activities, which are not planned. The recently published IRCC Rapid Impact Evaluation Report of the Syrian Refugee Initiative adds that there was also a lack of sufficient planning and forward-looking preparations regarding the resettlement and early integration phases in the first few weeks post arrival in Canada. The report admits that ‘the expedited nature of the initiative made it challenging for the settlement sector to effectively plan for the delivery of in-Canada services’, and that ‘the quick pace was difficult on refugees’. 

In short – both projects suffered under the lack of proper planning. 

But the largest disparity between project management norms and actual practice relates to efficiency, i.e. how the total cost of projects is estimated in the Canadian public service. Both, the OSR and the G-8 Summit reports reveal that cost planning is conducted through “guess-estimations” by a budget office as opposed to rigorous activity-based costing by a project team. Budgets are often division-based instead of activity-based. Projects with budgets estimated in this manner cannot be as efficient as those projects which total cost has been assessed based on individual activities. 

Although the cited managerial aim in organizing a G-8 Summit in 2002 was to ‘deliver the best possible value from the available mix of resources’, the project closure report was not able to critically assess whether the over 100M$ for a 2-day event is indeed an efficient way to advance Canada’s policy goals in G-8. Canadian experience of hosting four G-8 meetings previously would offer enough material to draft a blueprint Work Breakdown Structure for the next occasion, which would allow controlling scope, durations, costs and necessary human resources. As the closure report revealed, the lack of proper planning indeed led to inadequately composed project scope, late scope additions, overlapping and conflicting requirements, which were poorly communicated to the main stakeholders. One project goal - to make ‘Canadians proud’ – was not measurable or actionable. 

These examples demonstrate how faulty project planning makes it near impossible to assess whether the project was indeed a success. Despite that, the Summit was considered ‘unprecedently successful’ based on only one concern of cohosts – to be ‘conflict-free’. 

In the case of OSR, the Government of Canada’s aim to resettle 25,000 Syrian refugees between November 2015 and February 2016, was a political commitment made during the election campaign in response to the ongoing humanitarian crisis facing the Middle East and Europe. It became a major issue during the federal election season in 2015 after it was revealed that the family of a drowned Syrian toddler had hoped to come to Canada. All parties presented their competing visions several months before the election about how they would respond to the crisis. The Liberal leader Justin Trudeau presented the most ambitious plan in terms of volumes and timelines, saying that a Liberal government would bring in up to 25,000 refugees from Syria by January 2016, as CBC reported. His vision did not specify the expected cost, government’s capacity to deliver on the commitment or the experience of impacted refugees. The management of this project was thus not thought out and no clear criteria were provided for estimating project’s success rate. 

Accordingly, the IRCC Rapid Impact Evaluation report[1] on OSR concluded that one major area which should be taken into account to help ensure successful resettlement and settlement results includes “the need for end-to-end planning”. Yet, despite these shortcomings, the lessons learned portion of the report declares that “the whole of government initiative was a great success in many regards”. 

Conclusion 

As illustrated above in the two distinct projects, undertaken a decade apart, by following PM managerial logic in initiating, planning, executing and monitoring these unique and temporary endeavors with a goal to achieve a specific deliverable would have avoided several problems with effectiveness and efficiency. Both projects suffered under poorly stated and communicated project scopes, under scope creep through add-ons, which led to poor effectiveness, and poorly managed costs (inefficiency), and partially unsatisfied stakeholders. 

The Kananaskis report reveals that the main stakeholders the G-8 Summit who the project intended to make proud were cooperating reluctantly because the stakeholder identification and outreach were not conducted properly in the initiation and planning phase, respectively. OSR, which was meant to alleviate refugees’ fate was suffering under unrealistic timelines and overwhelming scope and led to at times frustrated refugees. 

The PMBoK offers a systemic approach to managing endeavors that are temporary, unique and have a clear beginning and an end, including measurable guidelines for creating progress reports and closing all tasks. PM tools can thus offer increased efficiency and effectiveness in delivering outputs. 

-xxx-