Lessons for Incumbents from GAO Decisions on 4 Rebid Protests
Guest Blog from Nigel Thacker, Managing Director at Rebidding Solutions
More Lessons from Lost Rebids
It’s not often we get the chance to see the details of another company’s lost rebid, what they think about the loss and an analysis of how the customer made the decision. In fact in many cases we don’t always get to see these lessons from our own business!
But in the USA there is an established process where losing bidders can protest against a lost bid (or rebid) decision, and one which results in publicly published summaries of the circumstances of the procurement, the reason for protest – and the analysis of why that protest is upheld or denied.
The United States Government Accountability Office (GAO) is one of the bodies responsible for (amongst other things) reviewing and making decisions on bid protests filed regarding procurements made by federal government agencies. Protests are usually filed by losing bidders and the GAO publishes summaries of the protests and their decisions. They often make interesting reading. Not just because they provide useful illustrations of how Federal Acquisition Regulations (FAR) work, but because they can illustrate the mistakes real bidders make (most bid protests are denied) when putting in their bids.
We’ve previously published a paper reviewing 13 such rebid (or as they are called in the USA, recompete) protests that failed and the lessons from these for other incumbents.. You can access it here. This new paper looks at four more recent protests and the lessons we can learn from them.
Many of the mistakes are not specific to FAR but are general mistakes from which incumbents anywhere can learn. The examples we have picked out are those where we see similar mistakes made by bidders outside the US as well: so if you are an incumbent anywhere, these real life examples are still relevant to you.
It’s also interesting to note that many of the companies protesting are very experienced and capable bidding organisations. For us that illustrates something we often see: knowing the principles and processes of successful rebidding in theory isn’t always the same as putting them into practice under the pressure of a real rebid.
We won’t cover all the detail of the protest and decision criteria and have only picked out those elements relevant to the specific lessons we see in the examples. If you want to follow up and read the full decisions go to www.gao.gov and search for the decision using the Reference we give for each example.
Contract Performance Counts in your Rebid
Performance on the existing contract does count. Even though rebids are an opportunity to tell the customer all will be better next time, the customer will remember, or have access to, your performance throughout the contract. In our survey of procurers over 80% said contract performance has an impact on their rebid decision. Even in Government contracts where evaluators have to be objective to what is in the rebid document the US government has access to Contract Performance Assessment Reports (CPARS) and can set past performance as an evaluation criteria. And the UK government is starting to use a similar approach to be able to take past performance (on other contracts as well as your incumbent contract being rebid) into account.
Lessons from lost rebids
GAO decision dated:September 5, 2014
Erickson Helicopters Inc, the incumbent, protested the award of a contract for helicopter transport services for the U.S. Transportation Command to AAR Airlift Group Inc.
Erickson protested on a number of areas (some technical regarding the competitor’s helicopter’s capabilities) all of which were rejected by the GAO. The area which is of most interest to us was a protest regarding the evaluation that Erickson received for their past performance. They received a score of ‘Limited Confidence’, based on the customer’s evaluation of a number of other contracts, but also (and with more weight given by the customer) reports of performance (called CPARS – Contractor Performance Assessment Reports) on the existing contract being rebid. The first two reports marked Erickson’s performance as being ‘unsatisfactory’ for the first 9 months of the contract. The third, whilst noting some improvement, still showed performance as ‘marginal’ and that Erickson still ‘failed to meet the Operational Readiness Requirement at any time and was unable to perform all auxiliary requirements’
Erickson’s protest argued that their latest report from the contract showed that they had improved, but that this final report was not taken into account by the evaluators, and that Erickson had included in the narrative of their bid a section telling the customer about this improvement.
However the GAO state:
“Here, with regard to the agency’s failure to consider the latest CPARS for the incumbent contract, the agency explains that this report was not available to the government when the agency completed its past performance evaluation. Given that there is no general requirement that an agency continue to seek updated performance information once its past performance evaluation is complete, we find nothing improper with the agency’s failure to obtain and consider the latest CPARS”
And perhaps more tellingly for other incumbents:
“Although Erickson believes that consideration of narrative information in its revised proposal would have tempered the agency’s negative views of its incumbent performance, the record establishes that the agency’s past performance evaluation focused on third-party assessments of the offerors’ performance, as opposed to self-interested performance assessments presented by offerors in their proposals. As noted by our Office, common sense dictates that an offeror’s self- assessment regarding the quality of its past performance is, by its nature, of less value as compared to the disinterested assessment provided by third parties.”
“In fact, the agency viewed the solicitation as not allowing offerors “to mitigate poor past performance with unsubstantiated, self-serving statements regarding their ability to improve performance in the future.” “
Erickson’s protest was denied
GAO decision dated:August 8th 2014
APT, the incumbent, protested the award of a contract for advisory and assistance services for the Department of Defense, Missile Defense Agency (MDA) to a.i. solutions. The relevant part of their protest for our purposes was that APT protested their rating in the evaluation of ‘satisfactory’ for past performance.
From the GAO report:
“MDA’s quality rating of APT’s performance on the incumbent task order was based on the agency’s assessment of APT’s PPIs (Past Performance Information sheets), APT’s CPAR (Contractor Performance Assessment Reports) ratings, APT’s MAR (Monthly Assessment Report) ratings, and interviews with agency personnel.“
APT protested that the scores received for their performance on the contract should have led to a higher score in the rebid:
“APT argues that it was unreasonable for the agency to assign its proposal a satisfactory quality rating, given that the overall ratings it achieved on the various CPAR and MAR were exceptional or very good. APT asserts that its CPAR and MAR ratings should have resulted in at least a very good past performance quality rating.”
However, GAOs review says:
“The record reflects that the agency found APT’s CPAR and MAR to have quality ratings of exceptional, very good, and satisfactory. Based upon the CPAR and MAR, the agency identified two issues of concern: (1) difficulties with planning and managing costs, and (2) a five month delay in filling a vacant position for a MDA program. These concerns led the agency to conduct interviews with MDA representatives.”
“The evaluators interviewed five MDA representatives to gain further insight into the severity and causes of the identified issues. Based on these interviews, the agency concluded that although APT eventually met all technical requirements, it had difficulty planning and managing resources on several occasions that led to missing or delayed coverage of critical mission assurance activities, and that the nature and scope of the problems, and the length of time it took APT to resolve these issues, was a relevant part of its past performance record. For this reason, the MDA found APT’s performance to be consistent with the solicitation’s definition of a satisfactory quality rating.”
“Based upon our review of the past performance information and the agency’s evaluation, we find no basis to object to MDA’s evaluation of APT’s past performance. First, the solicitation’s evaluation criteria for the past performance factor did not obligate the agency to assign the same quality rating that was reflected in an offeror’s CPARs and MARs. Rather, the RTOP (Request for Task Order Proposals) permitted the agency to consider the offeror’s CPARs and MARs along with other data in arriving at the task order quality rating. Moreover, the solicitation stated that the agency’s quality rating would be based upon the PPQ definitions, not the definitions in the CPAR or MAR. Second, the record here reflects that the evaluators found that–despite APT’s record of very good and exceptional performance–the quality of its performance was more consistent with the PPQ rating of satisfactory because of the nature and scope of the problems in its past performance record and the length of time it took APT to resolve the issues.”
The GAO denied the protest.
Discussion and lessons
These two examples raise slightly different aspects about the impact of contract performance on your chances of winning your rebid. Erickson’s performance during the contract would seem to have been less than perfect, at least during the earlier period. Usually, customers will appreciate if your performance improves during the contract period. In fact we recommend that incumbents always look to improve their performance during the contract. But, if it takes you most of the contract to get up to an acceptable level of performance against the customer’s criteria, they will remember and as in this case may well penalise you for this early non-performance. And more subtly it may impact on how the customer views your promises for the future contract. After all, we would guess that Erickson did not in their original bid say they would underperform for the majority of the contract. More likely they said they would be able to deliver. This would be more difficult to say believably in your rebid if your history on the existing contract didn’t meet your promises last time around.
APTs example seems slightly different. Although we don’t have the details of how or against what specific criteria the customer chose to measure their performance in their bid evaluation (PPQ above) methodology, it would seem to be different to how they were being measured during the contract. That might not seem fair to some. But the reality is often that the customer, in looking forward to the next contract, will have different criteria and levels of performance in mind for what they want and expect from the winner in the future than they did from the incumbent of the existing contract. When that is the case, even good performance against the ‘old’ measures might not be good enough for the future. Or it might be that over the period of the contract the customer’s
priorities have changed and they are looking at completely different criteria to what was being measured during the contract as part of the KPIs or SLAs. One of the lessons for incumbents here is not to rely only on performance against the KPIs set in the original bid. You need to keep up to date with the changes in customer priorities and, if needed, measure new things – or existing things against new levels of acceptable performance. Leaving this to the customer (often an operational customer who may just be focused on compliance with the spec) isn’t enough. You need to take a proactive approach. For more on this take a look at our article on measuring what really matters.
They can’t deliver it for that price!
This is often the first reaction of a losing incumbent when they find out they have lost the contract to a lower priced competitor. Sometimes it is true. But more often the incumbent finds that their competitor has found a way to deliver at a significantly lower price – and still make a profit. Even if the competitor doesn’t manage this, you have still lost. And it is rare for the customer to come back out to retender the new contract early. We included several examples of this type of argument failing to impress the GAO in our initial paper of 13 example rebid loses. Here is another more recent example
GAO decision dated:October 3rd 2014
Protestor:Arrington Dixon & Associates, Inc
Arrington Dixon & Associates (ADA) protested the award of a contract by the Department of the Interior for call centre services to Cognosante LLC. Amongst a number of protests by ADA (the incumbent) they challenged the agency’s evaluation of the price submitted by Cognosante.
“ADA first contends that the agency failed to properly evaluate the realism of the awardees’ proposed prices because they are substantially below the rates that the protester proposed, which ADA states were based on its incumbent contract. The protester alleges that the awardees’ prices reflect a lack of understanding of the RFQ requirements, and that the awardees must have proposed unrealistically low labor rates, unrealistically low hours, or deficient staffing mixes.”
However, after working through the detail of how the agency calculated the relative prices given by the different bidders (of which ADAs were the highest of all 4 bidders) the GAO stated that
“While ADA asserts that, as the incumbent, it has historical experience performing the same requirements and its proposed price reflects its experience performing this effort for the predecessor contract, this does not provide a sufficient basis to show the agency’s price realism analysis here was unreasonable. The fact that an offeror’s price is below the IGCE (Independent Government Cost Estimate) or the incumbent’s rates does not require the agency to conclude that the price was unrealistically low.”
Based on this and rejections of the other protest elements the protest was denied.
Discussion and lessons
Incumbent experience and knowledge can be a double edged sword. Used correctly it can help you understand the customer’s needs, create a solution that fits the customer’s requirements and show in your rebid that you really understand the details and have the best solution, and price for your customer.
However, there is often a point where what the incumbent thinks is ‘knowledge’ can actually be ‘assumption’. That can often be the case when looking at how much it will cost to deliver the new contract. Too often the incumbent looks at their own costs – both in terms of cost per item, or member of staff, and the required volume of these items required to deliver the solution – and assumes that these are the lowest costs and volumes that can possibly deliver the contract.
In the US, where staff protections such as TUPE don’t apply, this can often relate to the salaries of staff. As a contract progresses staff experience grows and so too can their wages. By the end of the contract staff might be more experienced and more highly paid than the customer specifies for the new contract. Competitors, following the specification, can bid at lower wages and offer existing staff these wages to stay – or replace them with other people at the lower salary quoted (assuming they can find them – and they usually can). Incumbents too often bid at the rates they are paying their existing (perhaps now too experienced vs the spec) staff and find they are more expensive.
But even where TUPE does apply, the number and roles of staff required can be ‘assumed’ to be higher than competitors taking a fresh look at the customer’s new specification see are required. And of course that also applies to all other aspects of costs. The volume, type and unit cost of all elements of a contract can be ‘known’ by the incumbent for the existing contract. But are they the best available for the new contract, are the costs at the level the market can buy, or have they crept up during the contract above what the market and current best practice require? In other words, is what is ‘known’ actually a flawed assumption when looking forward at the new contract?
Plus of course the assumptions made about the costs of the existing contract are only valid to the extent that the customer is actually asking for the same scope and specification for the next contract. There can be a tendency for incumbents to be blind (or at least have slightly blurred vision) when acknowledging and addressing the changes the customer is looking for. Because the existing provision is seen as a strength, any move away from that is seen as reducing this strength. This can mean changes to the customer’s needs, either underlying or even those clearly set out in the specification, are ignored or downplayed in favour of retaining the existing solution which may have delivered well for the existing or past expressed needs.
This ‘inertia’ from the existence of the present delivery model and cost base often leads to incumbents failing to make the changes they need to be competitive vs competitor’s costs and delivery models, and to favour existing models, sometimes adapted, over a clear and objective search for the best model and lowest costs that are more likely to win.
When a competitor with a lower price is chosen as the winner, the reaction is often incredulity and sometimes as in this case a protest. Usually such protests fail. And equally usually despite the incumbent believing that ‘they can’t deliver for that price’ the competitor actually can, because they have found a better model to deliver the customer’s needs, and have pushed for the lowest available costs in delivering their solution, not the assumed to be lowest costs presently being paid.
As baseball coach Yogi Berra is quoted as saying, “The future ain’t what it used to be”
We don’t need a transition plan, we’re the incumbent
It’s true that incumbents often require less of a transition to move from their existing solution to the new solution proposed, as they already have assets, staff, processes and systems in place. This can be a big strength, reducing risk for the customer as well as the time and cost of getting the new solution in place. But too often incumbents turn this strength into a weakness by assuming the customer knows this (or assuming they will get points just for saying it), and not taking the transition section of their submission seriously.
GAO decision dated:October 10th 2014
Protestor:SunGard Data Systems, Inc.
SunGard protested the award of a contract for mainframe computer services for the Bureau of Labor Statistics to Data Management Associates.
Amongst SunGard’s protests was the fact that the customer should not have evaluated their transition plan as merely ‘acceptable’ when the winners was evaluated as ‘outstanding’
With regard to the transition approach evaluation subfactor, the RFP stated that an offeror’s narrative should describe its transition approach, including the risks associated with transition and a description of the specific tasks associated with phasing in the new contract.
As the protest document states:
‘SunGard challenges its “acceptable” rating under the transition approach evaluation subfactor of the technical capability factor. SunGard asserts that, as the incumbent contractor, the firm did not need a transition plan, and it was unreasonable for the agency to conclude that the firm had only a “minimal understanding” of the requirements.”’
However the GAO stated generally that:
“When an RFP asks firms to provide a discussion of their approach, schedule, transition plan, or the like, the agency evaluation may reasonably consider the level of detail provided in the requested discussion. We have repeatedly held, where a firm merely indicates that it will meet a particular requirement, thereby discussing only the end results, but provides little or no detail about how it plans to meet or exceed the requirement, the agency may reasonably downgrade the proposal. Ultimately, an agency’s evaluation is dependent upon information furnished in a proposal, and it is the firm’s burden to submit an adequately written proposal for the agency to evaluate.”
And more specifically:
“In response to the solicitation’s transition approach requirement, SunGard’s proposal provided, “[a]s the current Contractor, no Transition Plan is anticipated.” The agency’s evaluation of SunGard transition approach reached the following conclusion:
As for the transition approach, while the offeror is the incumbent and generally would not need a robust transition plan, the offeror did not address any of the criteria requested in the solicitation. Also, this solicitation has significantly reduced and simplified requirements from the current mainframe services contract, but the offeror did not address how they would incorporate these changes as they transition onto the new contract.
Since SunGard’s proposal did not demonstrate its understanding or provide details of its transition approach, and instead, simply stated that it need not propose a transition plan because it was the incumbent, we think the agency reasonably rated SunGard’s transition approach as “acceptable,” rather than assessing it some higher rating.”
The GAO denied SunGard’s protest
Discussion and lessons
On one level the lesson from this is, as the GAO state, to answer in full the questions set. But for implementation, we believe there are additional lessons for incumbents.
It can be tempting to spend less time on a transition plan if you believe there is little to do. Although as with the above scenario, and the previous lesson above that, there is always the danger that the incumbent underplays the amount of change actually required even by themselves to address changes in the specification.
But assuming you as the incumbent are alive to the changes being asked for, you are still likely to be in a position of relative strength regarding transition. As we said in the introduction to this example, your transition is likely to be lower risk, shorter, and less costly than those of your competitors, because you are already present on the contract, delivering the existing solution. To win however you need to make the most of this, not just pass over it with a simple statement as SunGard appear to have done above (and we see other incumbents do too often). Even if you think your transition will be minimal, fully answer all the customer’s questions. Put in a full plan for the actions you will need to take, and resource the transition properly.
We’ve seen some incumbents make the most of this strength by putting in a ‘shadow’ transition plan against their own showing all the actions that a competitor would need to take to implement the contract and comparing this to their own. (In doing this they take a fair view of what would need to be done, rather than exaggerating the work, or assuming competitor incompetence- they base it on what they would do if this was a new win for themselves). This both illustrates that the incumbent knows how a full implementation would work, and highlights the specific differences to their own transition requirements – putting evidence to claims of lower risk, less time and less cost. It also allows the incumbent to show all the risks of a full (competitors) implementation vs their own, showing how many of these risks are mitigated by the incumbent. And by doing so ‘ghosting’ the competitors, and giving the evaluator a template to test competitors’ responses against if they choose to do so, potentially showing up areas that competitors may not have covered adequately.
As the incumbent you do potentially have a real strength in your transition – but as with other strengths, only if you make the most of it and don’t just assume the customer understands the strength without you telling them and will give you the marks in the evaluation based on that understanding.