NSF Rebaseline Review

Answers to questions from day 1

Overall questions on the BOEs:

Travel: Travel is standardized according to the Key Assumptions Document. There is a travel analysis, done by travel for the project by UW, to support the standard amount. (It is in the docushare folder – we can go through it if you want). Travel is not escalated in the out years. We had not planned to reanalyze travel but could consider doing it ~ yearly and bundling up any changes in a change request.
(travel analysis is here: https://docushare.icecube.wisc.edu/dsweb/Get/Document-90304/D_04_IC_Upgrade_TravelAnalysis.pdf )

PQ costs: this is a conservative estimate, and again we have standardized it across the project. PQ costs vary depending on the individual. An additional allowance of $250 for additional ECW gear is allowed for drillers / deployers.

BOE Questions and Answers

1.1
Q: Is 28 seats the number for all the L2/L3 CAMS plus the PMO (from invoice review)?   Did you escalate the outyear costs?
A: For SmartSheet 28 seats are for entire WPAC. Upgrade is assigned about half of the yearly cost.The license costs are not escalated in out years.
 

1.2
 
1.2.1 BOE / 7.2 M&S / 1.2.1 MandS Table
 
Q: Show an example of the extrapolation.  Is this extrapolated from Y1 and Y2 costs or prior?  How are the costs updated to PY5 costs and then are they escalated going forward?
 
A: Example of FY23 loading/trucking extrapolation:
 



 
At the time of estimation it was assumed that the trucking market would stabilize after pandemic cost increases. It appears now that this may not have been a good assumption. No inflation was added to the total of $8500 in shipping costs for FY24 & FY25.
 
 
1.2.1 BOE / 7.3 Travel
 
Q: Show example of the extrapolation from actuals.  What is the change from actuals and the basis for the change? 
 
A: Extrapolation from actuals is carried out by analyzing past domestic, international and South Pole deployment actual costs. See:
https://docushare.icecube.wisc.edu/dsweb/Get/Document-90304/D_04_IC_Upgrade_TravelAnalysis.pdf
 
Q: How many folks go to the international meeting and why is that the proper number? 
 
A: One person (L2 or drill manager) go to collaboration meetings. They have complementary qualifications and one is sufficient.
 
Q: Why is 5 the proper number of staff for the annual ASC meeting?
 
A: The following work directly with ASC and need to attend the meeting: Implementation L2, Sys Engr, Drill Manager, Safety and quality manager, installation logistics
 
Q: Walk through a development example of the international and domestic travel rates as they are developed from past actuals? 
 
A: See travel analysis document linked above
 
Q: How are these rates updated as new actual costs are received?  How are they escalated?
 
A: The travel rates are not updated with new actuals.
 
 
1.2.1 BOE / 8.2 Summary of Labor Resources / 1.2.1.2 Labor Hours Table
 
Q: Why is Expert opinion BOE used when this is the third phase of IceCube?  Why not analogy?  SE hours support stair steps down between LPY5 and 6 and 7 by 33% and another 55% between LPY7 and 8.  What is the likelihood that that will occur?  What kind of schedule delay is covered by the existing continency?   Same question on Mgt step down of 33% and 62% and SE support of 19% and 34% respectively.
 
A: Upgrade drill engineering roles are a different staffing model from Gen1 (multiple part time engineers at PSL vs. a bin of full time engineers working directly for the project), and have therefore been estimated from bottom up using expert opinion.  The step down from PY5 to PY6/PY7 is because the system engineer is covered by 1.2.8 Field Seasons during on-ice work. In PY5, there is no field season so SE is covered the entire year in 1.2.1 (at part time), and in PY6 and PY7 the SE LOE is carried for only 8 of 12 months in 1.2.1, the other 4 months in 1.2.8 Field Seasons).  There is another step down in PY8 because it has a field season but is also a partial project year.  The same applies to the Drill Manager position as well.
 
Q: Where does each FTE mentioned above go if not charging to Icecube?  Are there other active non-IceCube projects they can charge to?
 
A: Yes, there are other non-IceCube projects at PSL they can charge to.
 
 
1.2.1 BOE / 8.2 Summary of Labor Resources / 1.2.1.3 Labor Hours Table
 
Q: Same question as prior labor area: Why expert opinion if this is the third phase of an IceCube? Isn’t Analogy more appropriate?
 
A: Same answer as above, different staffing model than Gen1 and bottoms-up estimated based on yearly work tasks.
 

1.3

1.3.3 PDOM
Q: There were two Pos/invoice included.  Are the other items below supported with current quotes?
A: The other items are not supported by current quotes. They are based on engineering estimates and on previous experience from Gen1
 
Q: Is this supported by the 2-23-22 quote of $1556.629? show how this is supported.
A: The difference is due to overhead cost for a stencil amortized over quantity.
 
Q: Is Michigan State your subrecipent?  Do you have a current quote or are these already in stock?
A: The PCAs are in stock and $610/PCA is the final price. MSU made the purchase and supplied PCAs to DOM facilities.

1.4 CPT
1.4.1 Downhole Cable Assemblies
Q: Where is the support for the prices in this paragraph?  Has the price or delivery been volatile? Are these items in house or are there contingency to cover any likely volatility?
A: Support for prices estimated based on past purchases is included as attachments at the end of BOE document; others are SME estimates.  All components are custom items from suppliers specializing in subsea instrumentation so detailed price histories are unavailable, but prices and delivery schedules are believed to be volatile.  Contingency is assigned to each item according to its classification under the table on pp. 13-14 of the Key Assumptions document.
 
1.4.4 CPT Central Infrastructure
Q: Please provide vendor quote
A: Vendor quotes are included as attachments at the end of the BOE document.  White Rabbit equipment is shown on pp. 6-7 and Mean Well power supplies on p. 8.
 

1.5 
Q: How does a project wide standard domestic and international rates work with a subrecipient like Alabama? Does it provide accurate costs?  
A: We use the common travel estimate from the key assumptions document, as is standard practice for distributed grants in IceCube.
 
Q: Was Alabama subrecipient to prior IceCube builds?  If so why not use analogy?
A: Yes, Alabama was a subrecipient of awards from the MREFC construction grant and the Maintenance and Operations awards. The nature of work is unique to ICU calibration so we believe expert opinion to be more appropriate.
 
What is the likelihood and impact of any schedule delay? Can Alabama start later without incurring standing army costs (are there other projects to charge to)?  
Q: The main cost is from the postdocs who are hired in the last two years of the project to manage the integrated calibration effort and delivery of initial calibration constants, therefore a delay in the project would delay the hire of the postdocs and there is no standing army cost associated with this. There is a smaller standing cost associated with the L2 manager’s partial summer salary, this is a minor cost impact. 
 

1.6
 
1.6.1.4.3.11 - SPAT equipment:  $7500. ” Why expert opinion? Hasn’t this been done before? “   This equipment supplies hardware for a small test stand for use at Pole, including hardware toolbox of electrical tools, meters, and laptop for test management.  While we did test all Gen1 DOMs on-ice at Pole prior to deployment, this equipment is out of date, and the mini-FieldHub interface is different than the Gen1 StringHub interfaces.  This cost is estimated from the SPAT testing expert shopping list for COTS items, plus a few mini-FieldHubs and cables recycled from testing setups.
 
1.6.4.0 - SPS computing: $12000 “ Please show extrapolation from actuals ” . 1.6 BOE includes a Dell PO from a recent computing purchase for use at pole.  Nearly identical equipment is expected for Upgrade (Qty:2)
 
Labor question:  “What would be the likely impact of integration delays?  Can Staff work on non-IceCube effort?   Software integration/effort often can over-run.  What is the contingency plans?  Can this effort be fast-tracked if needed?”
 
Integration efforts are already substantially advanced, and can continue to be performed on small numbers of modules, with scaling to full system mockups with simulated data sources.  All developers working on online and in-OM software efforts are also IceCube M&O support folks, and already naturally change from support of IceCube Gen1 systems to integration of Upgrade instrumentation, so can easily pivot based on short-term needs.  Given our experience in this area, we’re confident in our estimated efforts, but contingency plans include prioritizing development effort from other M&O developers to bolster effort, and delaying delivery of some parts of system functionality to a following software release. If particular parts of the software are needed earlier, these can be fast tracked while potentially delaying other subsystem work.

3: Assess the appropriateness of the major level of effort components of the WBS, especially in the project office, as a proportion of the total budget needed to complete the project. 

C4: Are full time equivalent levels of labor well documented and justified, and is the labor mix appropriate?  

Questions for the project: 

·   What is Vivian O’Dell’s time commitment to the project (she is not on the Sr. Personnel on the NSF Budget Sheets nor on the PMO BOE)? What is her time frame of commitment (FY23 only? Or more) Vivian O’Dell (PD) is currently supported by UW Institutional Funding; she will remain as PD for Upgrade through the end of the Upgrade.  

·   What is the PIs PMO function at .08 FTE/Yr on BOE 1.1? Is his time charged to other IceCube tasks? PI ensures scientific goals of the project are achieved; interacts regularly with PD, PM, and other PMO staff and NSF/IPT; time is also charged to IceCube M&O / institutional support. 

·   What are the functions of Doug Cowen and Greg Sullivan for just a week and a half a year (ramping down to a week in FY26)? These persons identified are faculty institutional leaders supported primarily by institutional funds. The amount of support in the project budget is not necessarily representative of the actual effort expended.

 

·   Is Fevzi’s ramp down from full time FY23, .53 FTE in FY24, .40 in FY25, and .24 in FY26 dependent on achieving certain milestones?  What if they are delayed?  How can a project be adequately managed with a half time or less PM in its last three years? Most of the instrumentation and drill systems will have shipped by FY24. Remaining project activities will be concentrated around the polar seasons and will be low activity during the remaining portion of the year. We feel fractional PM involvement is appropriate.

·   What is the function of the .40 FTE Finance Person?  Isn’t finance an indirect function? OMB circular A-21 allows financial/administrative management that directly supports funded activity to be costed as direct labor. The Financial Manager provides support for sub-award management and collection and analysis of actual costs from UW, PSL, and from the sub-award institutions.  

·   Is the project controls person on board?  Are they also in charge of the schedule integration?  Does the project PM requirements justify a ramp down from full time FTE (FY23) to .5 FTE (FY24-25), .33FTE (FY26)?  Doesn’t FY24 have major field activity that may require PM support?  No PC not on board following loss of personnel in Jan 2022; they are in charge of schedule integration. The ramp down comes because most Northern Hemisphere tasks are wrapping up - scheduling activity is primarily on-ice which is planned by personnel other than the Project Controls expert.  

·   Show past experience from the analogy supporting .20 FTE logistics support?  Why does logistics support drop to .10 FTE in FY24 even though it is the year that has the most logistics?  Justification includes (a) support in final seasons will move from 1.1 to 1.2 (on-ice installation); (b) the final year only includes 7 months instead of 12. The logistics execution ramps up in FY24+ but the planning is most heavy during FY23 (cargo master tracker support).

Back to top