The Assembly, Integration and Verification (AIV) work package represents one of nine key elements that will make up the SKA1 Telescope. Whereas the other eight elements are tasked with designing key components of the SKA1 Telescope, the AIV element is tasked to perform all necessary planning to integrate these key components into a telescope system that meets the engineering (Level-1) requirements.
The SKA1 (SKA Phase 1) Telescope Reference Baseline consists of SKA1-mid, which will be located in South Africa, and SKA1-low, which will be located in Australia. SKA1-mid consists of approximately 200 dishes, of which 64 dishes are from the MeerKAT precursor telescope and the remaining dishes are a new design which is currently in the design phase. SKA1-low will consist out of approximately 512 SKA1-low Stations, which will include a total of approximately 130,000 individual low-frequency antennas.
The member organisations of the AIV Consortium are SARAO, CSIRO and ASTRON, with SARAO leading the consortium. All three member organisations have significant experience in building radio telescopes, and therefore have a vast amount of integration and verification know-how that is benefitting the AIV work packages.
The derivation of Test Procedures for the SKA1 Telescope
The AIV Consortium is currently busy with its last major delivery during the Pre Construction Stage of the SKA1 project, namely the writing of Test Procedures; more specifically, the writing of high-level Test Procedures. During the Construction Phase these high-level Test Procedures will provide a running start to the integration and verification team.
Arriving at Test Procedures is not as simple as writing a test plan for every single system (Level-1) requirement. This would lead to inefficiencies during the execution phase of these Test Procedures, since similar system requirements can often be verified by a single (slightly elaborated) Test Procedure. More importantly, this approach does not consider the sequence in which these Test Procedures would be executed, nor does it consider the early mitigation of risk by executing some of these Test Procedures early during construction.
Figure1 below shows a schematic that illustrates some of the key steps that were executed by the AIV Consortium in order to arrive at Test Procedures.
At the top of the “pyramid” are the system requirements. In order to mitigate verification risks as early as possible, a Roll-Out Plan was written for both SKA1-mid and SKA1-low. The Roll-Out Plan describes a sequential roll-out of Telescope functionality within Array Assemblies and therefore forms the basis for the delivery of products and for the planning of integration and verification activities. It also lists some of the key engineering goals for each Array Assembly, as well as for the back-end system that will be qualified in the Integration Test Facility (ITF). Radio Telescopes lend themselves to a sequential roll-out, and this characteristic is fully utilised in the integration and verification approach. The philosophy of “integrate early and often” underpins all of the work conducted by the AIV Consortium.
A separate delivery of the AIV Consortium is the derivation of Verification Requirements. These are high-level descriptions of how to verify the system requirements. Similar system requirements may be verified by a single (slightly elaborated) Verification Requirement. During this stage of the process the team or entity responsible for executing the verification is also identified.
Verification Requirements are grouped into Verification Events. At the highest level, these Verification Events are dictated by the Roll-Out Plan, i.e. every Array Assembly (as well as the ITF Qualification Event) is a Verification Event. Figure2 illustrates how these Verification Events are aligned within a tree structure. Verification Requirements are fulfilled at Verification Events that are at the lowest level of this tree structure.
Once the Verification Requirements and the Verification Events are defined, the Integration & Verification Plan can be written. The I&V Plan provides a structured framework, in which all integration and verification activities will be carried out in a coordinated manner. It identifies both integration events and verification events. Such a plan can be used to apply project management tools to manage the integration and verification activities during the Construction Phase.
Finally, the high-level Test Procedures can be defined. Every Verification Requirement is further elaborated in a Test Procedure, providing more detail with regard to the test setup, test method and pass/fail criteria. Every Verification Event gives rise to a single Test Procedure document, which contains all of the Test Procedures associated with the Verification Requirements that are contained in that Verification Event.
The generation of the Test Procedure documents is automated by using a systems engineering tool called CORE. This makes it possible to automatically include the full traceability within each Test Procedure document with regard to:
- The originating system requirement
- The derived Verification Requirement
- The Test Procedure executing the Verification Requirement
It should be noted that some Test Procedures will show no traceability to a system requirement. These tests are referred to as Commissioning Tests (or System Tests) and are seen to be needed to commission the system, but they do not directly verify a system requirement. They are primarily derived from experience gained from integrating and commissioning Precursor telescopes. Both types of Test Procedure are grouped into Verification Events.
Furthermore, in addition to listing the system requirements number and name within the Test Procedure document, the system requirements full description and rationale are also provided. This information is helpful when the Test Procedures are developed further during the Construction Phase.
Test Procedures for SKA1-MID
Test Procedures for SKA1-mid are developed according to two approaches:
- Firstly, in a top-down approach, Test Procedures are expanded from Verification Requirements, which provide a high-level description of how to verify the Telescope’s system requirements. These tests are termed “Verification Tests”.
- Secondly, in a bottom-up approach, based on Precursor experience, the Test Procedures that were useful in commissioning the MeerKAT Telescope are applied and re-written for SKA1-mid. These tests are termed “Commissioning Tests”.
All of these tests are executed within Verification Events at the various stages of SKA1-mid construction: at the ITF and at Array Assemblies 1 to 4, depending on the available functionality of the Telescope at hand. In many cases a system requirement can be partially verified at an early stage (e.g. at Array Assembly1) and only fully verified at a later stage (e.g. at Array Assembly4).
The linking of System Requirements, Verification Requirements, Verification Events, Verification Tests, Commissioning Tests and the generation of Test Procedure documents is maintained in CORE, which is a Model-Based Systems Engineering tool.
Figure3 shows an example of the Verification Event and Test Procedure structure and linkage from CORE.
An example of how the schedule of Verification Events and Test Procedures is maintained within MSProject is shown in Figure4. Verification Tests are shown in black font and Commissioning Tests are shown in blue font. The yellow highlighting in the first column indicates Test Procedures that are executed for the first time. Initial commissioning and verification testing often takes a lot longer than initially anticipated, with a lot of iteration as bugs are fixed, test procedures are written and verification scripts are optimised.
Test Procedures for SKA1-LOW
The development of Test Procedures for SKA1-low builds upon the extensive work already done by the AIV Consortium regarding Verification Requirements and Integration and Verification Plans. The Test Procedures are derived from these two documents and further refine and detail the test methods already described at a high level in the Verification Requirements document. The test procedures capture the way in which the AIV Team will test the SKA1-low system at different stages of its deployment (the Array Assembly Milestones) and verify the System Requirements.
For such a complicated system as the SKA, a great deal of liaison is required with system domain experts. The intention of the AIV team at the end of the Pre-construction phase is to capture as much of this specialist information as possible, to be used as a starting point for refining the test procedures during construction.
For the ITF and the four Array Assemblies (AA1 – AA4) there will be about 400 test procedures for SKA1-low. Some are relatively straightforward (such as power consumption and power cycle tests), whereas others are far more complex (such as Continuum/Spectral Line Imaging and Pulsar Timing/Pulsar Search). The implementation of these tests will require an AIV team with a comprehensive set of skills (both in the engineering and astronomy fields); a team with the ability to determine system faults and performance from the behaviour of the SKA1-low system during testing.
The AIV testing activity is a key part of SKA1-low construction, sandwiched between the physical installation and implementation of system hardware and software at the MRO site and the handing-over of a completed telescope system to the SKA Operations Team and the radio astronomy community.
The work being done on the development of SKA1-low Test Procedures has a couple of valuable spin-offs:
Firstly, the creation of Test Procedures has allowed the identification of durations, resources and dependencies for each test case. That has led to an overall Assembly, Integration and Verification schedule within a hierarchy of Verification Events. The AIV Consortium has indeed created a schedule for SKA1-low AIV and has provided this to the SKAO as an input for construction planning. The critical path for SKA1-low construction and key risks to the schedule have also been identified and communicated by the AIV Consortium.
Secondly, the development of Test Procedures supports the creation of detailed resource estimates for Assembly, Integration and Verification of SKA1-low. Using a bottom-up approach, and by aggregating the effort of individual tasks, the total amount of effort required for SKA1-low AIV can be accurately determined.
The combination of these two spin-offs provides a clear picture of SKA1-low AIV resourcing over time, including the particular skills needed at different points in time. This analysis supports strategic HR management of AIV activities throughout the construction phase of the SKA1. This kind of planning is iterative, with information becoming more detailed and accurate with each attempt. The iterations of forward planning, scheduling and resourcing within AIV is likely to continue until the end of SKA1 construction.
On Friday, 13th July 2018, South Africa’s 64-Dish MeerKAT Radio Telescope was officially inaugurated by Deputy President David Mabuza. This event has been covered on the SKA website here.
Towards the end of SKA1-mid construction, MeerKAT will be integrated into the SKA1-mid array. In order to ensure that all of the necessary integration planning has been completed, and to understand the remaining integration risks, the SKA Office has decided to conduct a MeerKAT Integration CDR, which is scheduled in the week of 22 October 2018.
Experience with other radio telescopes has consistently shown that the roll-out activities and AIV work scope is often underestimated, even at component level, and often causes delays in deployment, due to re-engineering and retrofitting of components. This may significantly increase the total cost of the system.
Many issues that are discovered during “downstream” integration and verification are the result of “upstream” neglect. Early in the project, during the design stage, science requirements need to be accurately translated to Element-level requirements, and interfaces between products need to be accurately defined.
Report provided by the AIV consortium