The Frontier Fields Implementation and Science Data Products Team has been developing a data processing pipeline aimed at producing a full set of high-level science products as the data arrive for each cluster. These products will include fast-turnaround mosaics produced every week, as well as full-depth combined mosaics produced after each epoch is complete.
Each of the Frontier Field clusters consists of two targets (the main cluster field and the parallel “blank” field), which will be observed over two epochs, for a total of 140 orbits per cluster. During the first epoch, one of the targets will be observed with ACS and the other with WFC3; the second epoch generally takes place 6 months later, enabling the spacecraft orientation to be flipped so that the ACS and WFC3 coverage can be swapped on the two targets. This thereby provides full ACS and WFC3 coverage, across the full suite of our selected filters, on both the main cluster target and the parallel “blank” field target once both epochs are complete.
During the course of each epoch, which will generally extend between 40 to 70 days, the arriving data will be continually processed by the Science Data Products Team to enable data products to be delivered to the community on a rapid timescale, using the automated pipelines running on a set of dedicated machines at STScI.
During an epoch, as the data from each visit are obtained from the spacecraft and arrive on disk, they will be organized by an initial set of pipeline scripts that will perform the following steps:
- Carry out an initial calibration on all the ACS and WFC3 exposures in the visit, using CALACS and CALWF3, to correct for a range of effects including:
- bias and dark current subtraction (including bias de-striping for ACS)
- flatfield correction
- electronic gain calibration
- non-linearity correction
- CTE correction (for ACS only)
- up-the-ramp cosmic ray rejection (for WFC3/IR only)
- photometric calibration
- As the exposures for each visit are calibrated, they will then be made available to the team for data quality inspection and verification, in particular to identify any images with potential issues or complications that may need more special treatment (for example, images that are affected by persistence or strongly time-variable sky).
- Once all the exposures pass the initial quality assessment, the next steps in the pipelines take care of correcting their geometric distortion and creating combined images using AstroDrizzle, as well as aligning exposures relative to one another using Tweakreg, and carrying out a first-pass cosmic ray rejection. These pipeline steps will also generate additional products that can be inspected to verify the accuracy of the alignment, the quality of the initial cosmic ray rejection, and identify any issues that may require reprocessing.
The resulting image products from this first-pass stage of the pipeline will be delivered to the community on a weekly basis, constituting the “v0.5″-level data release, which will include cumulative depth images where the new data are combined with previous exposures to provide the deepest available images, on a weekly timescale, as each epoch proceeds.
Once each epoch is complete for a given cluster target, all the exposures from that epoch will be reprocessed more fully through the “v1.0″ pipelines, in order to further improve the calibration, bad pixel rejection, and astrometric alignment, using information from all the exposures obtained during a given epoch. Specifically, this will include the following steps:
- repeat the recalibration of exposures using more up-to-date reference files, and improving the rejection of bad pixels and cosmic rays by comparing all exposures in a given filter obtained during the epoch.
- for ACS data, add a new additional step of “self-calibration” (described in more detail in a previous blog post by Jay Anderson), which improves the calibration by solving for a custom dark using the science data itself. This approach does a better job of identifying and correcting for warm pixels that are CTE-trailed out of existence on the standard pipeline darks which have zero background.
- for WFC3 data, further refine the flagging and removal of pixels affected by persistence and other detector defects, as well as more accurately recalibrating images that may be affected by time-variable sky or other observational anomalies.
- for both cameras, exposures in all filters will be combined with other existing observations in order to produce the deepest images of these fields in all filters, for ACS and WFC3.
The products from this pipeline will constitute the v1.0 mosaics for the Frontier Fields clusters, in all the filters that were used to observe them in both ACS and WFC3. It is expected that these will also achieve the full depths envisioned for these fields in all the filters.
All the mosaics will be distortion-corrected, combined, and registered to a common pixel scale to enable science to be carried out with them as-is. The products will all be archived and distributed to the community through the STScI MAST pages for the Frontier Fields, accessible at: http://archive.stsci.edu/prepds/frontier/.
Anton Koekemoer (Frontier Fields Science Data Products Lead)
and the Frontier Fields Implementation Science Data Products Team