The Assembly, Integration and Verification (AIV) work package represents one of nine key elements that will make up the SKA1 Telescope. Whereas the other eight elements are tasked with designing key components of the SKA1 Telescope, the AIV element is tasked to perform all necessary planning to integrate these key components into a telescope system that meets the engineering (Level-1) requirements.
The SKA1 Telescope will consist of SKA1-mid, which will be located in South Africa, and SKA1-low, which will be located in Australia. SKA1-mid will consist out of approximately 133 SKA1-mid Dishes, plus a further 64 MeerKAT Precursor Dishes. The AIV work package therefore also includes the planning for integrating the MeerKAT Precursor into the SKA1-mid Telescope. SKA1-low will consist out of 512 SKA1-low Stations, which will include a total of approximately 125,000 individual low-frequency antennas.
The member organisations of the AIV Consortium are SKASA, CSIRO and ASTRON, with SKASA leading the consortium. All three member organisations have significant experience in building radio telescopes, and therefore have a vast amount of integration and verification know-how that is benefitting the AIV work package.
AIV Consortium Face-to-Face Meeting at Jodrell Bank
The AIV Consortium had a F2F Meeting at Jodrell Bank during the week of 20 February 2017. The primary objective of the meeting was to focus on Verification Requirements, which are high-level descriptions of how to verify a specified requirement. Many of the L1 Requirements are highly specialised, and developing their verification procedures requires domain knowledge that is outside of the AIV Consortium’s expertise. Having the F2F Meeting at the SKA Office made it easier to communicate with project scientists and engineers regarding these issues, and indeed good progress was made during that time.
Apart from working on Verification Requirements, the AIV team also had discussions with the SKA Office regarding the AIV Risk Register, the Product Hand-over Checklist document, the Integration Test Facility (ITF), the software hand-over process, procurement and product hand-over, the overlap between system verification and science operations, as well as issues regarding the integration of the MeerKAT Precursor into SKA1-mid.
Peter Hekman, Nico Ebbendorf, Adam MacLeod, Donald Gammon, Michael Hayes, Richard Lord.
The Verification Model
The December 2016 publication of the SKA eNewsletter introduced the Verification Model, which showed the relationship between L1 Requirements, Verification Requirements, Verification Events, Test Procedures, Test Configurations and Test Equipment.
This publication describes the many-to-one relationships between these elements of the Verification Model. Referring to Figure2 below:
- One Verification Requirement may verify more than one Level-1 Requirement.
- One Verification Requirement may be fulfilled at more than one Verification Event. For example, a Verification Requirement may be partially fulfilled at an event associated with Array Assembly1, 2, 3 and4.
- One Verification Event may fulfil more than one Verification Requirement.
- One Verification Event may employ more than one Test Procedure.
- One Test Configuration may be formed by more than one Test Equipment.
- It is, however, not common for one Verification Event to employ more than one Test Configuration.
Planning the verification of the Telescope system begins by writing a high-level description of how each requirement will be verified. The AIV Consortium has completed the first draft of this work, together with help from domain experts from other consortia and from the SKA Office. The internally-reviewed document will be released at the end of March 2017, where after it will be further reviewed by external reviewers in the weeks and months to follow.
Verification planning also needs to consider at what level of system integration the verification can be executed. The roll-out plans define five levels of system integration:
- System ITF
- Array Assembly 1, 2, 3 and 4
Each Verification Requirement is either partially or fully verified at each of these high-level roll-out milestones. Most of the verification work will be executed by the AIV contractor during construction, but many requirements are also verified by product contractors, and some are verified by the Science Verification Team.
Design Qualification versus Product Verification
The difference between design qualification and product verification is often a matter of definition.
Qualification is associated with design and is therefore important during the design phase, i.e. the Pre-Construction Phase. Product verification or product acceptance is associated with manufactured products and is therefore important during the Construction Phase. Both design qualification and product acceptance are often regarded as verification activities.
All Design Consortia are expected to deliver qualification results at their respective Element CDR, which provide confidence to the SKA Office that their design will meet their L2 Requirements, which can be traced to the L1 Requirements. The qualification results are obtained either by analysis (i.e.paper work) or by test (i.e. a prototype has been built). If the design qualification is incomplete at Element CDR, it is expected that the affected Design Consortia also deliver a Qualification Plan, which describes how and when the design qualification will be completed (e.g. at the System ITF).
Once a product has been manufactured (this includes software/firmware products), it needs to be verified against its requirements (e.g. L2 Requirements). One distinguishing feature between design qualification and product verification is therefore that qualification testing is usually performed on a prototype, whereas verification testing is usually performed on the final product.
Design qualification and product verification are performed at all levels of the system:
- Level-2 Design Qualification should be completed at Element CDR, although it is likely that some of it will be completed at the System ITF.
- Level-1 Design Qualification, i.e. system-level design qualification, will be partially performed with the system installed in the System ITF and partially with Array Assembly 1.
- Level-2 Product Verification is expected to be performed at the contractor’s factory as part of factory acceptance. Such products may be shipped to the System ITF or to site, where final acceptance testing will be performed by the product contractor after installation, as part of the hand-over process.
- Level-1 Verification of the Telescope system is performed in a staged manner with the four Array Assemblies on site. A significant amount of verification testing can also be performed with the system installed in the System ITF.
In addition to the qualification results, all Design Consortia are expected to deliver a Verification Plan at their respective Element CDR. This Verification Plan should contain the following:
- A high-level description of how each of the product’s L2 Requirements will be verified. These descriptions are not as detailed as a test procedure.
- At what level of system integration the verification will be performed. Verification testing could happen at the contractor’s factory using simulators or emulators, or it could happen at the System ITF, where the product can be interfaced with other products.
Experience with other radio telescopes has consistently shown that the roll-out activities and AIV work scope is often under-estimated, even at component level, and often causes delays in deployment, due to re-engineering and retrofitting of components. This may significantly increase the total cost of the system.
Many issues that are discovered during “downstream” integration and verification are the result of “upstream” neglect. Early in the project, during the design stage, science requirements need to be accurately translated to Element-level requirements, and interfaces between products need to be accurately defined.
Another major challenge is that the software/firmware dominated Elements (CSP, SDP and TM) need to deliver systems with basic functionality very soon after tenders have been awarded.