51	TECH27	1.2.10	Failure of a harness or rigging element that would result in partial loss of string, unknown during deployment	String Partial Loss	1/15/2021	1/25/2022		Low	Very Low	Very Low	High	Low	Low	Moderate	Installation Mgr	FS3	Mitigate	Operator Error/Equipment Failure/Hardware Failure	Set robust mechanical requirements (design and proof test) for each element (harness, pressure vessels, rigging pieces, yale grips)		Low	Very low	Low	Moderate	Low	Low	Moderate	No cost or schedule slip as this risk assumes installation team does not know this. Loss of performance only.	$0				0			FS3 end
52	TECH28	1.2.10	Failure of a harness or rigging element that would result in total loss of string (string is stuck during drop, or deployment cannot continue and undeployment cannot happen, instrumentation is abandoned)	String Total Loss	1/15/2021	1/25/2022		Low	High	Very High	Very High	Moderate	Moderate	Moderate	Installation Mgr	FS3	Mitigate	Operator Error/Equipment Failure/Hardware Failure	Set robust mechanical requirements (design and proof test) for each element (harness, pressure vessels, rigging pieces, yale grips). Contingency time accounted for each installation step in hole lifetime calculation and prescribed drill charts.		Low	Very low	Low	Moderate	Low	Low	Moderate	Mitigation greatly reduces probability. String could not be replaced so no cost or schedule risk, only loss of performance.	$0				0			FS3 end
53	TECH29	1.2.10	Dust Logger winch failure	Dust Logger Winch	12/13/2018	1/25/2022		Low	Low	Low	Moderate	Low	Low	Moderate	Installation Mgr	FS3	Mitigate	Technical problems or Operator Error	Test Winch and Dust Logger Comms as soon as they are delivered to Pole, and at tower once electrical connections are established. If maulfunction happens prior to Dust Logger run start: establish Abort Time Prior to Start, postpone Dust Logger Run to next (last hole).		Low	Low	Low	Low	Low	Low	Low	Last hole will need to be made larger to accomodate dust logger run. low estimate of 110 gal (difference between a deep cold ream hole and a hot + log hole) => 110*25.45= $2800. High estimate 10% fuel (800 gal) to account for possible re-ordering of holes.	$2,800	$2,800		$20,360	1	0	1	FS3 end
54	TECH30	1.2.10	String Installation Winch Failure - The TU20 winch fails during installation operations. TU-20 needs to be swapped out	TU-20 Failure	12/13/2018	1/25/2022		Low	Low	Moderate	Low	Low	Moderate	Low	Installation Mgr	FS3	Mitigate	Technical problems or Operator Error	Inspect and repair both TU-20's to full operation. Create Deployment Cable Swap-Out Procedure.		Low	Low	Low	Low	Low	Low	Low	1-day delay - longer season (short crew)	$ - 0				1	0	1	FS3 end
55			1.3 Deep Ice Sensor Modules
56
57	TECH31	1.3	If one of the planned sensor types does not pass FDR as scheduled the project schedule could be severely disrupted	OM FDR	8/13/2018			Low	High	Moderate	Low	Moderate	Moderate	Low	OM Lead	Until FDR, April 2022	Mitigate	FDR fails	Design and plan the production capacity of all production facilities with sufficient flexibility to be able to fall back to a different OM type		Low	Low	Low	Very Low	Low	Low	Low	Off NSF budget	$ - 0		$0		26	8	52
58	TECH32	1.3.1	If bubbles remain in the mDOM Optical Gel, the optical sensitivity of the mDOM might worse than expected	mDOM Bubbles		1/27/2022		Moderate	Low	Low	Moderate	Moderate	Moderate	Moderate	mDOM Lead	Until FDR, April 2022	Mitigate	Design does not meet requirements	Optimize surface of support structure (use MJF, chemically smoothing, remove sharp edges) Implement bubbles in simulations and investigate impact on physics Gel-pad mDOM: Will delay the start of series production by several months		Low	Low	Low	Low	Low	Low	Low	Off NSF budget	$ - 0		$0		4	0	8
59	EXT9	1.3.1	If electronics parts for the mDOM Mainboard cannot be procured in time, the mDOM production and testing will be delayed	mDOM Electronics Supply Chain		1/27/2022		High	Moderate	Low	Low	High	Moderate	Moderate	OM Lead	Until procurement	Mitigate	External	Continuously monitor market throughout collaboration and centrally collect information of availability of critical components Place orders / publish calls for tender as early as possible Develop plans for series integration of half-mDOMs, add in mainboards as late as possible		High	Moderate	Low	Low	High	Moderate	Moderate	Air shipping to New Zealand might be required if parts are sufficiently delayed. Off project cost. Critical path schedule delay would be zero if air shipping is utilized as long as parts show up by mid-2023.	$ - 0		$0		4	0	52
60	EXT10	1.3.1	If suppliers don't have sufficient capacity to 3D-print mDOM PMT Support Structures, the mDOM production and testing will be delayed	mDOM Support Structure Supply Chain		1/27/2022		Moderate	Moderate	Low	Low	Moderate	Moderate	Moderate	OM Lead	Until procurement	Mitigate	External	Foresee production of PMT Support Structures in several batches that can be tendered to different manfufacturers		Low	Low	Low	Low	Low	Low	Low	Air shipping to New Zealand might be required if parts are sufficiently delayed. Off project cost. Critical path schedule delay would be zero if air shipping is utilized as long as parts show up by mid-2023.	$ - 0		$0		4	0	26
61	TECH33	1.3.1	If mDOM series production does not reach the projected rate of 9 mDOMs / site / week, the shipping date for the mDOMs for Strings 87+88 will be missed	mDOM Production Yield		1/27/2022		Low	Moderate	Low	Low	Moderate	Low	Low	mDOM Lead	Until OM shipping	Mitigate	Technical problems	Produce mDOM test batches at DESY and MSU as early as possible to excercise integration and FAT procedures see also risk ORG5		Low	Low	Low	Low	Low	Low	Low	Air shipping to New Zealand might be required if parts are sufficiently delayed. Off project cost. Critical path schedule delay would be zero if air shipping is utilized as long as parts show up by mid-2023.	$ - 0		$0		4	0	26
62	EXT11	1.3	If integrated sensors have to be stored for too long before deployment, the vacuum seal might leak and re-sealing might become necessary	Sensor Long-Term Storage		1/27/2022		Low	High	Low	Low	Moderate	Low	Low	OM Lead	Until deployment	Mitigate		Regularly monitor the inside pressure of a sample of stored sensors Store sensors such that they are accessible for monitoring (not McM?)		Low	Low	Low	Very low	Low	Low	Low	If stored sensors are accessible for regular monitoring, the problem can be detected early and sensors can be un-sealed and re-sealed at a rate of about 5 sensors / day. Off project cost	$ - 0		$0		8	4	12
63	TECH34	1.3.2	If during D-Egg FAT a high failure rate or design flaw is found, reworking D-Eggs will lead to significant cost and schedule slippage	D-Egg FAT Yield		1/27/2022		Low	Low	Low	Moderate	Low	Low	Moderate	D-Egg Lead	Until FAT	Mitigate		Commence D-Egg FAT as early as possible Work closely with D-Egg team on executing and interpreting FAT		Low	Very low	Very low	Low	Low	Low	Low	Even a minor re-work of all (300) sealed D-Eggs is a major undertaking, requiring un-sealing, re-sealing and re-testing all modules. Off project cost	$ - 0		$ - 0		26	13	52
64	EXT12	1.3.5	If one of the Special Devices is not delivered, the string design must be updated and verified	Special Device Cancellation	1/15/2022	1/27/2022		Very High	Low	Low	Very low	Moderate	Moderate	Moderate	OM Lead, Deployment Lead	Until deployment readiness review	Mitigate		Start early developing contingency plans how to replace or bypass missing Special Devices during deployment		Moderate	Very low	Very low	Very low	Low	Low	Low	Can be managed in close to real time, perhaps a couple of days schedule slip on the ice or slight risk of degraded performance	$ - 0		$ - 0		1	0	1
65	TECH35	1.3.3	Because PDOM pressure vessels are re-furbished from Gen1, handling and resealing with new PCA may introduce subtle flaws that could cause PV flooding after deployment.	PDOM PV leak	1/15/2022	1/27/2022		Low	Moderate	Moderate	Low	Moderate	Moderate	Low	PDOM Lead	Until after freeze-in	Mitigate	No comms with deployed PDOM	Inspection of sealing surfaces, comparison of Gen1 and Gen2 PCA sealing surfaces, High Pressure testing.		Low	Low	Moderate	Low	Low	Moderate	Low	If detected before deployment of remaining strings, a work stoppage for analysis of the problem might be required Costed as a week delay. Should not happen if PDOM design verification is properly done	$ 70,000.00	$ 20,000.00	$ 72,500.00	$ 125,000.00	1	0	2
66	EXT18	1.3.1	If mDOM planned main board electronics parts remain unavailable, a few mainboard redesign will be required: this includes hardware design effort (off of the NSF budget) and software & firmware effort (on NSF budget).	mDOM redesign	4/6/2022	4/6/2022		High	Moderate	High	Low	High	High	Moderate	mDOM lead	Calendar 2022	Mitigate	Parts delivery delays/unavailable parts	Begin planning and design work ahead of risk trigger on the new mainboard design		High	Moderate	High	Low	Moderate	Moderate	Moderate	On the NSF budget would need 1 FTE-year of software effort and 1/2 FTE-year of firmware work.	$ 240,000.00	$ 160,000.00		$ 320,000.00	2	0	4
67	TECH36	1.3	Because NPX is very dry, sensor electronics could be damaged by ESD during NPX testing or deployment	xDOM Electronics ESD Damage	1/15/2022	1/27/2022		Low	Low	Moderate	Low	Low	Moderate	Low	OM Lead, Deployment Lead	Until deployment	Mitigate	Modules failures, procedure errors during deployment	Employ appropriate internal ESD protection circuitry in the xDOM and ICM. Ensure that ESD mitigation is used on floors, people, and equipment at NPX. Ensure charge neutralization is emplloyed by procedure and equipment during NPX testing. Ensure that charge neutralization is present between electrical cables, TOS and xDOMs as they are being deployed.		Very Low	Low	Moderate	Low	Low	Low	Low	If process problems with ESD at South Pole, a work stoppage for renewing procedures might be required. Costed as a week delay.	$ 70,000.00	$ 20,000.00	$ 72,500.00	$ 125,000.00	1	0	2
68			1.4 Comms, Power, and Timing (CPT) Distribution System
69	TECH37	1.4.1.1	Because the MCA prototype has not completed mechanical testing, we may need to revise the main cable design, which will require additional design and testing effort and delay the start of production.	Main cable design iteration	1/27/2022	3/1/2022		Low	High	Low	Very low	Moderate	Low	Low	CPT Lead	June 2022 (PSL crescent test)	Accept (near-future)	Unsatisfactory prototype test	N/A		Low	High	Low	Very Low	Moderate	Low	Low	Schedule impact depends on magnitude of changes required -- an additional prototype cycle could take 1/2 year, while minor changes could be completed in a few weeks. Cost impact limited to on-project engineering labor, likely ~1 FTE-month	$ 15,000	$0		$30,000	12	6	26
70	TECH38	1.4.1	Because the MCA prototype has not completed mechanical testing, we may need to add a separate co-deployed strength member to the Downhole Cable Assembly, which will increase deployment complexity and require additional equipment.	Strength member needed	1/27/2022	3/1/2022		Moderate	High	Low	Very low	High	Moderate	Low	CPT Lead, Deill Lead, Deployment Lead	June 2022 (PSL crescent test)	Mitigate	Unsatisfactory prototype test	Begin preparing design for co-deployed rope and test prototype in advance of cable tests		Moderate	Moderate	Low	Very Low	Moderate	Moderate	Low	Estimated cost of ropes ($18k/string) is largely offset by reduced need for YaleGrips (approx. 60 omitted per string @ $270 each). Cost of additional winch etc. unknown. Time required for design of second winch system unknown. Rough guesses inserted.	$ 25,000	$10,000		$60,000	4	2	12
71	TECH39	1.4.1	Because the MCA prototype has not completed mechanical testing, we may need to redesign the string hardware and deployment procedures, which will require additional design effort, prototyping, and equipment testing.	Deployment procedures	1/27/2022	3/1/2022		Low	Moderate	Very Low	Very Low	Moderate	Low	Low	Deployment Lead	June 2022 (PSL crescent test)	Accept (near-future)	Unsatisfactory prototype test	N/A		Low	Moderate	Very Low	Very Low	Moderate	Low	Low	Design work plus prototyping/testing	$ 5,000	$2,500		$10,000	6	4	12
72	TECH40	1.4.1.1	Because the MCA prototype has not completed mechanical testing, we may need to select an alternate Main Cable supplier, which will necessitate project support for cost and/or impact communications and timing performance	Alternate cable	1/27/2022	3/1/2022		Low	Very high	Very high	Moderate	Moderate	Moderate	Moderate	CPT Lead	June 2022 (PSL crescent test)	Mitigate	Unsatisfactory prototype test	Pursue development of alternate suppliers, procure cable samples where feasible		Very Low	High	Very High	Moderate	Low	Moderate	Low	Likely to be delays in switching to new main cable supplier, most vendors are less responsive than Hexatronic.	$ 1,500,000	$300,000		$3,500,000	20	4	30
73	TECH41	1.4.1.1	Because a MCA prototype has not yet been built, the overall diameter of the cable may be larger than estimated and not fit on the currently planned shipping/deployment spool, which will require redesign of the spool, cable, and/or deployment procedure and equipment	Cable too large	1/27/2022	3/1/2022		Very low	Moderate	Low	Very Low	Low	Low	Low	Drill Lead	Until first MCA is complete (Aug 2023)	Accept	Prototype cable diameter is too large. Breakouts are too large	Minimize cable diameter to preserve room on spool for breakout connections		Very Low	Moderate	Low	Very Low	Low	Low	Low	Design work, larger spools, potentially different equipment.	$ 20,000	$10,000		$40,000	8	4	16
74	TECH42	1.4.1.1	Because a MCA prototype has not yet been built, the overall weight of the cable may be larger than estimated and exceed the limit of the currently planned shipping pallet and/or aircraft, which will require redesign of the cable, spool, or shipping pallet.	Cable too heavy	1/27/2022	3/1/2022		Very low	Moderate	Low	Very low	Low	Low	Low	CPT Lead	April 2022 (production of prototype cable)	Accept	Prototype cable is too heavy	N/A		Very Low	Moderate	Low	Very Low	Low	Low	Low	Possible redesign of shipping pallet, requiring additional liaison with ASC and drill lead	$ 30,000	$7,500		$45,000	16	4	24
75	TECH43	1.4.1.2	Because a BCA supplier has not been identified, we may need to make significant design changes, which will increase costs and could delay production of other cable elements	BCA design iteration	1/27/2022	3/1/2022		Low	High	Moderate	Very low	Moderate	Moderate	Low	CPT Lead	Aug 2022 (start of prototype production)	Mitigate	Issues during preliminary design	Engage with potential suppliers as early as possible		Low	High	Low	Very low	Moderate	Low	Low	Restarted design phase could take several months, require several FTE-month of engineering effort	$ 25,000	$5,000		$75,000	8	2	26
76	TECH44	1.4.2	Because of connector supply chain disruptions, we may not be able to ship the SCAs or SJBs on the scheduled vessel, which will require them to be shipped ComSur the following year or eliminate the option of staging at McMurdo the season before installation.	SCA delay	1/27/2022	3/1/2022		Very low	Low	Low	Very low	Low	Low	Low	CPT Lead	Until connectors are received	Mitigate	Connector delivery delayed beyond Aug 2022	Order connectors as early as possible (order placed May 2021). Complete first cables using Gen1 leftover connectors to validate assembly procedures.		Very Low	Low	Moderate	Very low	Low	Low	Low	Cost of Comsur shipping estimated based on past shipments of large cables. Schedule impact relates to cable shipping date	$ 7,500	$6,000		$15,000	4	1	24
77	EXT13	1.4.3	Because of electronics supply chain disruptions, there may be delays in production of rev1 and/or rev2 FieldHubs, which would reduce the number of prototyping cycles available before we need to enter production.	FieldHub delay	1/27/2022	3/1/2022		Moderate	High	Very Low	very low	High	Low	Low	FieldHub Lead	June 2023 (start of FieldHub production)	Mitigate	Delays in production of rev1/2 FieldHubs	Finalize selection of FPGA and other long-lead components as early as possible, order as soon as practicable. Eliminate prototyping cycles rather than delaying production (FieldHub design is already quite mature)		Moderate	Very low	Very low	Low	Low	Low	Moderate	Elimination of prototyping cycle removes schedule impact at cost of slightly elevated technical risk	$ - 0	$0		$0	0	0	0
78	EXT14	1.4.4.1	Because the vendor identified for the timing and communications hardware used in the FieldHub was recently acquired, the interface to the hardware may change or the product may be retired, which will require additional hardware development effort to change to a different timing node.	White Rabbit supplier	1/27/2022	3/1/2022		Low	High	Low	very low	Moderate	Low	Low	CPT Infrastructure Lead	Spring 2023	Mitigate	Substantial changes to baseline timing solution	Identify alternate suppliers or alternate timing fanout designs that can be implemented using parts currently on hand		Low	moderate	Low	Very low	Moderate	Low	Low	Alternative designs identified. Additional effort of 1±1 FTE-month required for design.	$ 15,000	$0		$30,000	4	0	8
79	ORG9	1.4.4.2	Because of increased mDOM and DEgg power consumption and remaining uncertainties around maximum device power consumption, the power distribution system may need to be redesigned for a higher DC voltage, which will require additional development effort.	DOM power limit	1/27/2022	3/1/2022		High	High	Low	Very low	High	Moderate	Low	CPT Infrastructure Lead	April 2022	Redesign	final DOM power consumption measurements	Due to high risk probability, terminate work on baseline design and switch to new design with lower power density		very high	Very low	Low	Very low	Moderate	Moderate	Moderate	Additional capEx cost of $40k. Reduced labor required for power monitoring system, some additional labor for power fanout (~40 hrs). Approximately one month schedule delay for timing system and FieldHubs	$ 50,000	$40,000		$65,000	4	3	8
80	TECH45	1.4.3	Because MCA procurement is not finalized, we may have insufficient bandwidth to transmit all data to ICL using the planned cables and comms protocol, which will require additional software or firmware engineering and may reduce science capabilities.	Comms bandwidth	8/18/2018	3/1/2022		Low	Low	Moderate	Low	Low	Moderate	Low	Software Lead	Until cable procurement is finalized	Mitigate	Inability to identify supplier for cables with planned comms quality	Design protocol for efficiency and robustness, deploy all elements to NTS for integration testing, begin firmware development early if needed; Consider In-OM hit reductions		Low	Very low	Moderate	Low	Low	Moderate	Low	Add engineering effort - 1 FTE yr - to develop better communications protocol. Development schedule permits completion before deployment	$200,000	$150,000		$300,000	0	0	26
81	TECH52	1.4.1.1	Because MCA breakout terminations are a custom solution, the costs of breakout installation may exceed the MSU commitment, requiring project support for MCA costs	Breakout costs	3/28/2022			High	Very low	High	Very low	Low	High	Low	CPT Lead	Until MCA procurement is finalized: PY4 & PY5	Mitigate	Final bids for breakout installation exceed available MSU funds	Provide cable samples to multiple potential vendors, explore intial designs and negotiate cost reductions with multiple vendors		High	Low	High	Very low	Moderate	High	Low	High cost estimate based on initial bids from two suppliers (JDR and HGS), cost exposure reflects some assumed cost reductions. Schedule risk reflects possibility of extended design phase	$400,000	$0		$750,000	0	0	24
82	TECH46	1.4.1.4	Because the planned YaleGrips have not been demonstrated to work with the planned MCA in a pull test, we may need to increase the YaleGrip tail lengths, which will increase costs	YaleGrip size	2/15/2022	3/1/2022		Low	Very low	Moderate	Very low	Low	Moderate	Low	CPT lead	Until final cable design review	Watch	Unsatisfactory pull test with prototype cable	N/A		Low	very low	Moderate	Very low	Low	Moderate	Low	Replace 461 of YG 944506ET20 quoted @$240 with YG 944507ET22 quoted @$410. Possible longer lead time for custom extended tail grips	$78,000	$75,000		$85,000	0	0	12
83	EXT15	1.4.1.1	Because of supply chain disruptions, we may face delays in production of prototypes and production Main Cables, which will require alternate shipping methods and/or eliminate the option of staging the MCAs at McMurdo the season before deployment.	Cable delay	1/15/2021	3/1/2022		Moderate	High	High	Very low	High	High	Low	CPT Lead	PY4 & PY5	Mitigate	Production is delayed	Offer procurement assistance for long-lead items. Invesitgate and qualify alternative suppliers		Low	High	High	Low	Moderate	Moderate	Low	Prototypes for qualifying alt. suppliers ~$75k each. Long-lead procurement assistance risks ~$100k. Few-month scale delays still possible	$200,000	$50,000	$225,000	$400,000	12	0	40	2021		$180,000	$180,000	$180,000	$0	$0
84			1.5 Characterization and Calibration System
85	EXT16	1.5.2.6	Mini Mainboard may be delayed due to components availailability/supply chain issues	MMB delay	1/27/2022	2/28/2022		Moderate	High	Moderate	High	High	Moderate	High	1.5 L2 Lead Williams	2022, calibration module DVT and production	Mitigate	Parts delivery delays/unavailable parts	try and obtain components separately		Low	High	Moderate	High	Moderate	Moderate	Moderate	Some components have been ordered. FPGAs are a concern. Some vendors have been suggesting shopping for parts in China in person	$40,000	$20,000		$60,000	26	8	52
86	EXT17	1.5.3.5	Dust Logger winch cannot be borrowed from IDP, or IDP is unable to fix it, and a new winch needs to be procured	Dust Logger Winch Procurement	1/27/2022	2/28/2022		Moderate	Low	Moderate	Moderate	Moderate	Moderate	Moderate	1.5 L2 Lead Williams	2022 discussions with IDP, 2024 testing	Early communications	Approval delay at IDP, lack of engineering support at IDP	Working with IDP to secure the loan of this equipment.		Low	Low	Moderate	Moderate	Low	Moderate	Moderate	Risk is retired after testing winch with logger (Spring/Summer 2024)	$145,000	$140,000		$160,000	8	6	10
87			1.6 M&O Data Systems integration
88	TECH47	1.6.1.1	If optical sensor or calibration device software interfaces specifications are delayed will impact DAQ software development	DAQ software interface delays	8/17/2018			Low	Moderate	Low	Very low	Moderate	Low	Low	1.6 L2 lead	Sensor software delivery milestones	Better communications	Delivery delays	Work closely with Sensor and Calibration devices L2s to ensure software interfaces are developed and adhered to. Interface specification workshop		Low	Low	Low	Very low	Low	Low	Low	Minor cost impact, mostly delays to schedule. Estimate 20 week possible delay, and 0.1 FTE*yr	$20,000	$0	$20,000	$40,000	20	0	30
89	TECH48	1.6.1.1	Because the hit rate from OM devices exceeds the rate available in the DAQ trigger system, and triggers/hits could potentially be missed and physics events missed.	DAQ trigger too high	8/17/2018	1/25/2022 (increased probability)		Moderate	Low	Moderate	Moderate	Moderate	Moderate	Moderate	1.6 L2 lead	DAQ software testing	Mitigate	DAQ trigger testing failures	Refactor DAQ trigger algorithms to allow for higher rates, including developing noise rejection methods or look to reduce rate of trigger primitives from modules via cuts		Very low	Low	Moderate	Low	Low	Low	Low	Would require potential refactoring of DAQ architecture to support additional rates. If realized, could be an 0.5FTE*yr and 20 week delay	$80,000	$40,000	$80,000	$120,000	20	10	30
90	TECH49	1.6.1.1	If the per-PMT noise rates in mDOMs exceed design specifications, the in-OM data processing and noise mitigation algorithms may not have enough in-OM resources and data could potentially be lost.	mDOM noise rates overwhelm MCU	1/25/2022			Moderate	Low	Low	Moderate	Moderate	Moderate	Moderate	1.6 L2 lead	integrated mDOM testing	Mitigate	MCU can't keep up with rate of mDOM PMTs	Use FPGA firmware resources to offload MCU; develop more CPU efficient feature extraction algorithms		Very low	Low	Low	Low	Low	Low	Low	Minimal cost impacts or schedule impacts: these mitigations are already planned, just need to retune and understand physics impact	$10,000	$0	$7,500	$15,000	6	0	10
91	TECH50	1.6.1.1	If an in-ice device could be identified as the wrong type at boot up, improper hardware configuration could be used, leading to damage of components.	MCU SW Config ID failure	1/25/2022			Low	Very Low	Low	Low	Low	Low	Low	1.6 L2 lead	FAT testing of modules	Mitigate	Module type incorrectly ID by HW selection and hardware systems potentially damaged	Ensure robust testing of all mainboards as early as possible to ID issues with this key circuit; implement additional checks against backup IDs in scratch nvram		Very low	Low	Low	Low	Low	Low	Low	Addiitional internal software safety checks might be needed. 0.25 FTE*yr, 12 weeks	$40,000	$20,000	$40,000	$60,000	6	3	9
92	TECH51	1.6.1.1	If the total data rate from all connected devices exceeds the total available processing bandwidth, the FieldHub will be unable to keep up with data transfer from all connected devices and data could be lost.	FieldHub software unable to keep up	1/25/2022			Low	Low	Low	Low	Low	Low	Low	1.6 L2 lead	FieldHub full scale testing at NTS	Mitigate	Fully loaded field hub can't process all hits from ~100 connected DOMs	Scale tests on pre-production field hub units, investigate non-USB transfer mechanisms, parallelism in SBC design.		Very low	Low	Low	Low	Low	Low	Low	Potentially need to investigate alternative data transfer methods internally, or additional parallelism. Work generally off-Upgrade funding, but potential if M&O resources overwehlmed.	$25,000	$0	$20,000	$40,000	20	5	30
93			Opportunities
94	OPP1	1.1	If contributed drillers (from collaborating institutions) are provided as in-kind contributions, the project can save seasonal driller labor costs.	Contributed drillers	3/8/2022			Moderate	Very low	High	Very low	Low	High	Low	1.1 PM	Ahead of FS1	Opportunity	Volunteers from the collaboration	Will advertise it widely within the collaboration. Fixed FS years will help significantly.		Moderate	Very low	High	Very low	Low	High	Low	Contributed drillers (up to 10 person*seasons) replace $50000 in direct costs.	$250,000	$100,000		$500,000	0	0	0
95
96			Retirement home + Deleted risks
97	RET1	1.6.4	If there are delays in design of the IDF, computing infrastructure equipment ordering and shipping could be delayed - retire	IDF late design	8/17/2018	7/4/2019	7/4/2019	Low	Low	Moderate	Very low	Low	Moderate	Low	1.6 L2 lead	IDF facility design delivery	Mitigate	IDF Design delays	Separate network/computing design elements in the IDF design process, preventing delays in other areas from largely impacting 1.6		Low	Low	Low	Very low	Low	Low	Low	Minor cost impact. The IFD is no longer being used in the IceCube Upgrade Project.			$0
98	RET2	1.6.1.1	If optical sensor or calibration device software interfaces specifications are delayed will impact DAQ software development Includes ICM Firmware Development. This includes the delivery of the CPT infrastructure. (Combined Risks TECH15 and TECH16) COVID-19	DAQ software interface delays Risks: 3, 10	8/17/2018		1/25/2022	Low	Moderate	Very low	Very low	high	Low	Low	1.6 L2 lead	Sensor software delivery milestones	Better communications	Delivery delays	Work closely with Sensor and Calibration devices L2s to ensure software interfaces are developed and adhered to. Interface specification workshop. Prioritize remote development efforts, such as FAT software readiness using networked main boards.		high	moderate	low	low	High	Moderate	Moderate
99	RET3	1.6.1	If there are delays in delivery of CPT infrastructure for testing (Field Hubs, SJBs, NTS) then the development of Online Systems will be delayed	CPT milestone delays	8/17/2018		1/25/2022	Low	Moderate	Very low	Very low	Moderate	Low	Low	1.6 L2 lead	CPT Hardware delivery date	Better communications	Delivery delays	Work closely with CPT L2 and L3s to ensure at least partially hardware delivery for development and testing		Low	Low	Very low	Very low	Low	Low	Low	Minor			$0
100	RET4	1.5.1.1	If there are delays in implementation of common calibration database for production calibration, calibration constants may not be stored in easily usable condition	Production calibration database	8/17/2018		2/28/2022	Low	Low	Very low	Moderate	Low	Low	Moderate	1.5.1 L3 Lead Williams	Module Production	Better communications	Deliverable delay in 1.5.1.1	Coordinate early between module production sites and 1.5.1 L3 lead but add software labor if needed in PY4-PY5.		Low	Low	Very low	Very low	Low	Low	Low	2 months software developer effort

Back to top

---