ASKAP science data processing
The SKA precursor telescopes are on the front line of big data astrophysics. With multiple beams, baselines and frequency channels, ASKAP is already generating visibility data of order 100 TB per night and we must now demonstrate that this torrent of raw data can be converted into high-quality images in a reasonable timeframe.
ASKAP’s Science Data Processing team has developed a suite of software tools designed to work efficiently on the parallel architectures found in modern supercomputers, but until now we have only been able to test this software on simulated data sets.
With the commissioning of ASKAP’s latest 28-antenna sub-array, we now have access to real data approaching the size and complexity of the full-scale telescope. In order to prepare for the first round of ASKAP survey observations, the SDP team are taking on a number of “data challenges”, designed to gain experience with real-world visibilities and guide the optimisation of the software. The first of these challenges concluded two weeks ago.
As expected, we found that serial tasks associated with flagging and calibration were a major bottleneck. These can be tackled in parallel by dividing the raw data into smaller slices in line with the required access patterns. We also found that not all processing nodes should be treated equally – in order to make large images, we will need to allocate more memory to specific tasks responsible for gathering information from the rest of the workers. Experience gained during these data challenges will be incorporated into the tender process for a new astronomy supercomputer at the Pawsey centre, to be delivered in 2020.
Of course, creating images is only the first stage in the science data pipeline – there are also many lessons to learn in adapting existing astronomy work-flows to cope with extremely large image file sizes. ASKAP and its fellow precursors will provide valuable experience for future SKA regional centre developments.
Commissioning a telescope as complicated as ASKAP requires a great deal of flexibility. Although the list of required tasks can be defined in advance, some activities will block all other work and there will always be unforeseen challenges. In the case of ASKAP, we have been balancing the need to incorporate more antennas into the array, with ongoing implementation of key control system features. Changing too many things at once makes it difficult to track down problems, but doing everything one step at a time makes progress very slow.
Recently, we took a giant step forward with the integration of 28 out of 36 antennas into a functional array for the first time. This occurred sooner than expected, while other aspects of the control system were being finalised in the laboratory. Previously, the largest number of antennas integrated into the array at once was 16.
The first thing the team noticed was how much better our calibration and imaging software performed with the extra sensitivity and UV coverage of the larger array. Preliminary images of a calibrator source show a dynamic range of 830:000 to 1, which is extremely encouraging and suggests that ASKAP will be capable of making very high quality images over its wide field of view.
We have been observing several science fields with the expanded array over the last few months and hope to have more images to share in the next update! Until you can catch all the ASKAP commissioning news here on the Australia Telescope National Facility website, where Dr Aidan Hotan posts a monthly ASKAP Commissioning Update.
New magnetic field map demonstrates one of ASKAP’s major science goals
Magnetic fields are thought to play a key role in many astrophysical processes and can be found in different objects with an astonishing range of strength. Powerful magnetic fields govern the jets of emission that give pulsars their lighthouse-like beams, but weak magnetic fields in the ionised interstellar medium may also influence star formation and gas dynamics on very large scales.
Measuring the strength and direction of magnetic fields is an observational challenge that a wide-field telescope like ASKAP is well suited for. One of ASKAP’s 8 survey science projects, POSSUM, aims to use Faraday rotation of background radio sources to probe foreground magnetic field structures, creating magnetic maps of unprecedented resolution.
POSSUM researchers have used early science observations and a preliminary calibration technique to create the most detailed image of the magnetic field structure in one of the giant radio lobes of the galaxy Centaurus A. This image has been published in Galaxies. The complicated structures that can be seen will require years of additional research to understand. With the help of new calibration methods and additional antennas, we will soon have even more detailed magnetic maps from the full ASKAP telescope.
A paper https://www.mdpi.com/2075-4434/6/4/127
More on ASKAP Science Papers
ASKAP survey science teams are producing an ever-growing list of science papers, with several recently attracting some serious attention in the Australian and international media.
The dispersion-brightness relation for fast radio bursts from a wide-field survey, describing the first 20 fast radio bursts (FRBs) detected by ASKAP, was published in Nature on 10 October, 2018. This result has bought together researchers from several institutions, with Dr Ryan Shannon, Swinburne University and the OzGrav Centre of Excellence, Dr Jean-Pierre Macquart, Curtin University node of ICRAR, and CSIRO’s Dr Keith Bannister, leading the team.
The team reports that none of the ASKAP 20 FRBs repeated during follow-up observations and the sample includes both the nearest and the most energetic bursts detected so far. The survey demonstrates a relationship between burst dispersion and brightness and that the high-fluence bursts are the nearby analogues of the more distant events found in higher-sensitivity, narrower-field surveys.
A partner media release about the paper was distributed by ICRAR, Swinburne/OzGrav and CSIRO. The story generated 639 articles in 56 countries, reaching millions of readers worldwide. This was coupled with extensive social media reach.
Another collaborative ASKAP campaign, with CSIRO, the Australian National University (ANU) and Nature Astronomy, included the publication of Dr Naomi McClure-Griffiths’ paper, Cold gas outflows from the Small Magellanic Cloud traced with ASKAP, in the November issue of Nature Astronomy, with ASKAP featured on the front cover. Naomi’s work on the SMC with ASKAP, attracted attention prior to this, with the release in November 2017 of the most detailed image of the SMC, using just 16 of ASKAP’s 36 antennas.
You can find a list of publications about ASKAP or reporting results from ASKAP observations here.
For real-time alerts about ASKAP publications and news, follow us on Twitter @CSIRO_ATNF
ASKAP Community Workshop
CSIRO hosted an ASKAP Community Workshop on Wednesday Oct 24 and Thursday Oct 25, 2018, to review progress through the Early Science program and to facilitate planning of ASKAP Pilot Surveys in 2019. The workshop included updates from CSIRO staff on the technical issues of relevance for planning Pilot Surveys, and updates from each of the Survey Science Teams (SSTs).
A detailed draft proposal on the Pilot Survey Plan (led by Dr Aidan Hotan, ASKAP Project Scientist) was circulated before the meeting and laid the foundation for much of the discussion. Each SST presented a concept for their pilot survey and these will be taken forward through the ASKAP Commissioning and Early Science Group and Early Science Forum.
A primary concern of SSTs is the capability of the Galaxy computer at the Pawsey Supercomputing Centre, which is currently being pushed to its limits. A key part of the proposed Pilot Survey Plan is that ATNF take control of the pipeline imaging processing and manage the limited Galaxy resources more efficiently. Galaxy will be replaced in 2020 but until that time, we must make the best use of available resources.
Another matter raised at the workshop was the idea of “observatory-managed” projects, including a possible shallow all-sky survey and a possible collaboration with MeerKAT on a common reference field (dubbed “ground truth” by Jim Condon). The discussion here was less conclusive but working groups have been identified to examine the specific proposals. Dr Fernando Camilo, Project Scientist at the South African Radio Astronomy Observatory (SARAO), attended the meeting and offered many useful insights from a MeerKAT perspective.
We had about 50 participants and it was a constructive and positive event.
Report provided by Annabelle Young, CSIRO