Infusing
Project Management techniques into a Results-based Management approach leads to
better accountability
Marje
Aksli
Two
Canadian projects, G-8 Summit in 2002 and resettlement of Syrian Refugees in
2016, exemplify how international development or public policy objectives can
be met more effectively and efficiently when a results-based-management (RBM)
approach is supported by using project management (PM) tools.
In
short, could government programs, currently running on RBM, be made more effective
and efficient if they were managed using PM principles, as well?
The
Development Assistance Committee at the Organization for Economic Cooperation
and Development (OECD DAC) has outlined five criteria for evaluating projects
and programs: relevance, effectiveness, efficiency, sustainability, and impact.
Setting the latter two aside, for now, we discuss the multiple sets of tools to
adequately meet the relevance, effectiveness and efficiency criteria.
RBM
and PM
The
most widely used management tool in international development, and in public
service, is Results-Based-Management, or RBM, which connects the project
outputs to desired outcomes in a measurable way. For example, we assume that if
a country accepts and resettles tens of thousands of refugees from the Middle
East - a project output in the Operation Syrian Refugees - suffering is
alleviated, which is the desired outcome. Similarly, coordinating G-8 Summits –
project output – would, in theory, lead to an increased global visibility and
influence for Canada – the desired outcome. Project relevance is thus
established with the logical and causal connection from outputs to outcomes
through RBM.
Setting
relevance aside, the managers also need to ensure projects are implemented
effectively and efficiently as per OECD requirements. This entails achieving
project outputs effectively (within the project scope) with scarce inputs or
tax dollars (efficiently). Though RBM is focused on achieving higher level
results (outcomes), it provides no tools to manage project outputs in a
cost-effective and timely manner. In effect, RBM is not designed to provide the
guidance required for achieving effectiveness and financial efficiency – two of
the main DAC evaluation criteria.
The
systemic managerial approach, specified in the Project Management Body of
Knowledge (PMBoK) does provide a logical normative system for managing projects
effectively and efficiently. Several lessons learned sections of evaluation
reports indicate that using PM tools would help to control project scope, which
relates to effectiveness, and cost, which explains the efficiency of a project.
PMBoK principles could also assist with stakeholder management, a crucial
aspect in a project’s success. Still, the public sector and organizations involved
in program development seem to overlook these obvious benefits because they
have yet to practice PM tools to the same extent as RBM and deliverology.
For
better effectiveness and efficiency in tax payer funded programming,
results-based management and project management tools should be used in
combination because they are complementary by design. RBM ensures that project
outputs are connected to project outcomes through logical and causal
connections (relevance) and PM tools ensure the management of projects is
focused on scope and budget (effectiveness and efficiency).
Public
servants and NGO programming personnel would benefit from extensive training in
PM techniques and they should be encouraged to use them. A change in management
styles through PM training would also provide a more business-oriented approach
to development or to public sector programming, one that mitigates risks
through a judicious use of resources.
By
breaking down project components into a 2D Matrix relating knowledge areas,
such as project cost, essential human resources, project duration, scope, risk,
stakeholders, and quality, with project phases or life cycles, PM tools lay out
a set of manageable tasks in a coherent process. In the project initiation phase, the PM tools instruct managers
to work on two knowledge areas only: to specify a project goal and to identify
stakeholders based on an assessment of needs. In the proceeding planning phase, managers have
to engage with many additional project knowledge areas. The project description
is explicitly stated in the planning phase and it becomes the project scope.
The work needed to achieve this scope, and only this scope, is then broken down
into manageable work packages or activities to which cost is assigned, human resources
allocated, duration stipulated; risk level mitigated and quality identified.
Only then can the manager know the total cost, the duration, and risks of the
project. Similarly, PM tools offer guidance for three consequent phases:
execution, monitoring and controlling, and for the closing phases.
Lessons
to be learned
Comparing
project closure or evaluation reports to the PMBoK norms reveals several short
comings. The biggest deviations between PM theory (the norm) and management
practice in Canadian government departments result from omitting, or
shortening, the initiation phase and from blending the planning phase with the
execution phase. This leads to confusion in many knowledge areas and severely
impacts both project scope and cost management. The final report of Kananaskis
Summit from 2002, for example, did not distinguish between words such as
‘planning’ and ‘managing’. The two words were used interchangeably in the
meaning ‘organizing’, which seems to indicate that the implementation team did
not understand how to sequence or prioritize activities in distinct project
phases. This became acutely apparent in the time and scope management areas.
Additions to the project scope expanded until late in the project execution
phase, which led to ballooning budget and unmet commitments.
The
merged planning and execution phases resulted in several planning documents not
being completed, which consequently hindered the controlling of project scope
and led to difficulties in estimating the exact costs and necessary human
resources. For example, the report regrets that environmental impacts were not
addressed at the Summit and suggests, correctly, that these requirements should
have been included with the scope requirements in the planning phase. Promises
were made about the, so called, legacy projects to First Nations – the most
important stakeholders, and event co-hosts, yet these add-on projects were not
included in the original project scope and were therefore not part of the plan
and were consequently not delivered because of insufficient funds.
Similarly,
the Operation Syrian Refugees (OSR) in 2016, which aimed to resettle 25.000
refugees between November 2015 and February 2016, suffered under the lack of
interdepartmental coordination during the initiation and planning phases. This
led to significant consultation gaps between the government departments,
resulting in unclear responsibilities and competing priorities between
different government departments. For example, the planning model used by the
lead department IRCC did not sufficiently address the significant efforts
required to set up operations abroad during the operation’s earlier phases. The
activities abroad were not well planned, consequently not assigned costs,
durations, human resources, or risk ratings for and were thus managed poorly.
Managers cannot execute activities, which are not planned. The recently
published IRCC Rapid Impact Evaluation Report of the Syrian Refugee Initiative
adds that there was also a lack of sufficient planning and forward-looking
preparations regarding the resettlement and early integration phases in the
first few weeks post arrival in Canada. The report admits that ‘the expedited
nature of the initiative made it challenging for the settlement sector to effectively
plan for the delivery of in-Canada services’, and that ‘the quick pace was
difficult on refugees’.
In
short – both projects suffered under the lack of proper planning.
But
the largest disparity between project management norms and actual practice
relates to efficiency, i.e. how the total cost of projects is estimated in the
Canadian public service. Both, the OSR and the G-8 Summit reports reveal that
cost planning is conducted through “guess-estimations” by a budget office as
opposed to rigorous activity-based costing by a project team. Budgets are often
division-based instead of activity-based. Projects with budgets estimated in
this manner cannot be as efficient as those projects which total cost has been
assessed based on individual activities.
Although
the cited managerial aim in organizing a G-8 Summit in 2002 was to ‘deliver the
best possible value from the available mix of resources’, the project closure
report was not able to critically assess whether the over 100M$ for a 2-day event
is indeed an efficient way to advance Canada’s policy goals in G-8. Canadian
experience of hosting four G-8 meetings previously would offer enough material
to draft a blueprint Work Breakdown Structure for the next occasion, which
would allow controlling scope, durations, costs and necessary human resources.
As the closure report revealed, the lack of proper planning indeed led to
inadequately composed project scope, late scope additions, overlapping and
conflicting requirements, which were poorly communicated to the main
stakeholders. One project goal - to make ‘Canadians proud’ – was not measurable
or actionable.
These
examples demonstrate how faulty project planning makes it near impossible to
assess whether the project was indeed a success. Despite that, the Summit was
considered ‘unprecedently successful’ based on only one concern of cohosts – to
be ‘conflict-free’.
In
the case of OSR, the Government of Canada’s aim to resettle 25,000 Syrian
refugees between November 2015 and February 2016, was a political commitment
made during the election campaign in response to the ongoing humanitarian
crisis facing the Middle East and Europe. It became a major issue during the
federal election season in 2015 after it was revealed that the family of a
drowned Syrian toddler had hoped to come to Canada. All parties presented their
competing visions several months before the election about how they would
respond to the crisis. The Liberal leader Justin Trudeau presented the most
ambitious plan in terms of volumes and timelines, saying that a Liberal
government would bring in up to 25,000 refugees from Syria by January 2016, as
CBC reported. His vision did not specify the expected cost, government’s
capacity to deliver on the commitment or the experience of impacted refugees.
The management of this project was thus not thought out and no clear criteria
were provided for estimating project’s success rate.
Accordingly,
the IRCC Rapid Impact Evaluation report[1] on
OSR concluded that one major area which should be taken into account to help
ensure successful resettlement and settlement results includes “the need for
end-to-end planning”. Yet, despite these shortcomings, the lessons learned portion
of the report declares that “the whole of government initiative was a great
success in many regards”.
Conclusion
As
illustrated above in the two distinct projects, undertaken a decade apart, by
following PM managerial logic in initiating, planning, executing and monitoring
these unique and temporary endeavors with a goal to achieve a specific
deliverable would have avoided several problems with effectiveness and
efficiency. Both projects suffered under poorly stated and communicated project
scopes, under scope creep through add-ons, which led to poor effectiveness, and
poorly managed costs (inefficiency), and partially unsatisfied stakeholders.
The
Kananaskis report reveals that the main stakeholders the G-8 Summit who the
project intended to make proud were cooperating reluctantly because the
stakeholder identification and outreach were not conducted properly in the
initiation and planning phase, respectively. OSR, which was meant to alleviate
refugees’ fate was suffering under unrealistic timelines and overwhelming scope
and led to at times frustrated refugees.
The
PMBoK offers a systemic approach to managing endeavors that are temporary,
unique and have a clear beginning and an end, including measurable guidelines
for creating progress reports and closing all tasks. PM tools can thus offer
increased efficiency and effectiveness in delivering outputs.
-xxx-
