MD backgrounds

The following MD09 warps were chosen to test how the backgrounds had improved using the PV3 processing tag (note these have been previously referred to as MD04, that was an announcement typo):

These were determined to have the following backgrounds and sigmas estimated from fitting a gaussian to the full pixel distribution:

   sky      sigma
  0.1827    25.33
  0.1796    24.49
  0.1666    18.86
  0.2090    25.67
 -0.2605    64.61
  0.1865    34.10
  0.3667    47.37
  0.3696    31.11
  0.3599    30.55
  0.3705    32.55
    

Here is a plot of the pixel distributions:

The histogram with a stepped line is for the warp with background -0.2605, that must be an image taken with a high original background given the sky sigma (this is exposure o6550g0059o).

So the mean background is ~0.2, which is roughly confirmed by PSPHOT runs on these warps that give a mean SKY of ~0.3. The following is a normalised cumulative plot of these SKY values as for all warps (ignore the black line).

Click on the image to see a full resolution version. The cumulative histograms all pass through 0.5 (the median) at roughly ~0.3, except for the one at -0.2605. This is the green line that is clearly below the others in the postive SKY value half, so PSPHOT SKY and the pixel distribution peak do not correspond well for this warp.

The question now is what background value will be seen in a stack of these warps, previous analyses had given backgrounds of several hundreds.

From a simple minded analysis we might expect something like ~0.2, but stacks are scaled to a zero point of 25 per second. Given the warp actual zero points of around 24.6 per second at best (some are worse going down to ~23), the scale factor we need for 10 exposures is around 14, so we might expect a mean background of 0.2*14, i.e. 2.8, which would scale ~80 for a stack of 400 such warps.

Using the following command:

ppStack \
    -input MD04-input.mdc MD04-threepi \
    -stats MD04-threepi.stats \
    -recipe PPSTACK STACK_THREEPI \
    -recipe PPSUB STACK_1DG \
    -recipe PSPHOT STACK \
    -recipe PPSTATS STACKSTATS \
    -stack-type DEEP_STACK \
    -F PPSTACK.OUTPUT PPSTACK.OUTPUT.NOCOMP \
    -F PPSTACK.OUTPUT.EXP PPSTACK.OUTPUT.EXP.NOCOMP \
    -F PPSTACK.OUTPUT.EXPWT PPSTACK.OUTPUT.EXPWT.NOCOMP \
    -F PPSTACK.OUTPUT.VARIANCE PPSTACK.OUTPUT.VARIANCE.NOCOMP \
    -F PPSTACK.UNCONV PPSTACK.UNCONV.NOCOMP \
    -F PPSTACK.UNCONV.EXP PPSTACK.UNCONV.EXP.NOCOMP \
    -F PPSTACK.UNCONV.EXPWT PPSTACK.UNCONV.EXPWT.NOCOMP \
    -F PPSTACK.UNCONV.VARIANCE PPSTACK.UNCONV.VARIANCE.NOCOMP \
    -F PSPHOT.BACKMDL PSPHOT.BACKMDL.MEF \
    -F PSPHOT.OUTPUT PSPHOT.OUT.CMF.MEF \
    -F PSPHOT.PSF.SAVE PSPHOT.PSF.SKY.SAVE \
    -F SOURCE.PLOT.APRESID SOURCE.PLOT.SKY.APRESID \
    -F SOURCE.PLOT.MOMENTS SOURCE.PLOT.SKY.MOMENTS \
    -F SOURCE.PLOT.PSFMODEL SOURCE.PLOT.SKY.PSFMODEL \
    -threads 16 \
    -tracedest MD04-threepi.trace \
    -log MD04-threepi.log
    

with ppStack version: trunk/ppStack@37009, and input file MD04-input.mdc contents:

INPUT0  METADATA
        IMAGE   STR     ../warps/o6538g0139o.652143.wrp.970252.skycell.055.fits
        MASK    STR     ../warps/o6538g0139o.652143.wrp.970252.skycell.055.mask.fits
        VARIANCE        STR     ../warps/o6538g0139o.652143.wrp.970252.skycell.055.wt.fits
        PSF     STR     ../warps/o6538g0139o.652143.wrp.970252.skycell.055.psf
        SOURCES STR     ../warps/o6538g0139o.652143.wrp.970252.skycell.055.cmf
END
INPUT1  METADATA
        IMAGE   STR     ../warps/o6540g0157o.652952.wrp.970253.skycell.055.fits
        MASK    STR     ../warps/o6540g0157o.652952.wrp.970253.skycell.055.mask.fits
        VARIANCE        STR     ../warps/o6540g0157o.652952.wrp.970253.skycell.055.wt.fits
        PSF     STR     ../warps/o6540g0157o.652952.wrp.970253.skycell.055.psf
        SOURCES STR     ../warps/o6540g0157o.652952.wrp.970253.skycell.055.cmf
END
INPUT2  METADATA
        IMAGE   STR     ../warps/o6540g0213o.653008.wrp.970254.skycell.055.fits
        MASK    STR     ../warps/o6540g0213o.653008.wrp.970254.skycell.055.mask.fits
        VARIANCE        STR     ../warps/o6540g0213o.653008.wrp.970254.skycell.055.wt.fits
        PSF     STR     ../warps/o6540g0213o.653008.wrp.970254.skycell.055.psf
        SOURCES STR     ../warps/o6540g0213o.653008.wrp.970254.skycell.055.cmf
END
INPUT3  METADATA
        IMAGE   STR     ../warps/o6543g0266o.653384.wrp.970255.skycell.055.fits
        MASK    STR     ../warps/o6543g0266o.653384.wrp.970255.skycell.055.mask.fits
        VARIANCE        STR     ../warps/o6543g0266o.653384.wrp.970255.skycell.055.wt.fits
        PSF     STR     ../warps/o6543g0266o.653384.wrp.970255.skycell.055.psf
        SOURCES STR     ../warps/o6543g0266o.653384.wrp.970255.skycell.055.cmf
END
INPUT4  METADATA
        IMAGE   STR     ../warps/o6550g0059o.655728.wrp.970256.skycell.055.fits
        MASK    STR     ../warps/o6550g0059o.655728.wrp.970256.skycell.055.mask.fits
        VARIANCE        STR     ../warps/o6550g0059o.655728.wrp.970256.skycell.055.wt.fits
        PSF     STR     ../warps/o6550g0059o.655728.wrp.970256.skycell.055.psf
        SOURCES STR     ../warps/o6550g0059o.655728.wrp.970256.skycell.055.cmf
END

INPUT5  METADATA
        IMAGE   STR     ../warps/o6558g0328o.658638.wrp.970257.skycell.055.fits
        MASK    STR     ../warps/o6558g0328o.658638.wrp.970257.skycell.055.mask.fits
        VARIANCE        STR     ../warps/o6558g0328o.658638.wrp.970257.skycell.055.wt.fits
        PSF     STR     ../warps/o6558g0328o.658638.wrp.970257.skycell.055.psf
        SOURCES STR     ../warps/o6558g0328o.658638.wrp.970257.skycell.055.cmf
END
INPUT6  METADATA
        IMAGE   STR     ../warps/o6567g0066o.661135.wrp.970259.skycell.055.fits
        MASK    STR     ../warps/o6567g0066o.661135.wrp.970259.skycell.055.mask.fits
        VARIANCE        STR     ../warps/o6567g0066o.661135.wrp.970259.skycell.055.wt.fits
        PSF     STR     ../warps/o6567g0066o.661135.wrp.970259.skycell.055.psf
        SOURCES STR     ../warps/o6567g0066o.661135.wrp.970259.skycell.055.cmf
END
INPUT7  METADATA
        IMAGE   STR     ../warps/o6569g0065o.661989.wrp.970258.skycell.055.fits
        MASK    STR     ../warps/o6569g0065o.661989.wrp.970258.skycell.055.mask.fits
        VARIANCE        STR     ../warps/o6569g0065o.661989.wrp.970258.skycell.055.wt.fits
        PSF     STR     ../warps/o6569g0065o.661989.wrp.970258.skycell.055.psf
        SOURCES STR     ../warps/o6569g0065o.661989.wrp.970258.skycell.055.cmf
END
INPUT8  METADATA
        IMAGE   STR     ../warps/o6569g0073o.661997.wrp.970260.skycell.055.fits
        MASK    STR     ../warps/o6569g0073o.661997.wrp.970260.skycell.055.mask.fits
        VARIANCE        STR     ../warps/o6569g0073o.661997.wrp.970260.skycell.055.wt.fits
        PSF     STR     ../warps/o6569g0073o.661997.wrp.970260.skycell.055.psf
        SOURCES STR     ../warps/o6569g0073o.661997.wrp.970260.skycell.055.cmf
END
INPUT9  METADATA
        IMAGE   STR     ../warps/o6569g0081o.662005.wrp.970261.skycell.055.fits
        MASK    STR     ../warps/o6569g0081o.662005.wrp.970261.skycell.055.mask.fits
        VARIANCE        STR     ../warps/o6569g0081o.662005.wrp.970261.skycell.055.wt.fits
        PSF     STR     ../warps/o6569g0081o.662005.wrp.970261.skycell.055.psf
        SOURCES STR     ../warps/o6569g0081o.662005.wrp.970261.skycell.055.cmf
END
    

to create a stack, we get a slightly negative PSPHOT SKY value. This is the black line shown in the cumulative histograms that crosses 0.5 at ~-0.1. But note that this is divided by a factor of 14 so that it can be compared to a warp, so in fact we get:

as reported at the last DRAVG. So the mystery is why do we get a negative background, when previously it was (mainly) positive, and the only adjustments to the data values before combination where positive scalings?

For reference the ppStack log reports the following values:

 Applying scale correction to image 0: -2.893520 mag (0.007611) (with inputMask value 0)
 Applying scale correction to image 1: -2.854084 mag (0.008413) (with inputMask value 0)
 Applying scale correction to image 2: -3.559891 mag (-0.005770) (with inputMask value 0)
 Applying scale correction to image 3: -2.848194 mag (-0.000447) (with inputMask value 0)
 Applying scale correction to image 4: -2.860443 mag (-0.007146) (with inputMask value 0)
 Applying scale correction to image 5: -2.870781 mag (-0.003460) (with inputMask value 0)
 Applying scale correction to image 6: -3.236738 mag (-0.004380) (with inputMask value 0)
 Applying scale correction to image 7: -4.022354 mag (-0.002026) (with inputMask value 0)
 Applying scale correction to image 8: -4.743792 mag (0.164291) (with inputMask value 0)
 Applying scale correction to image 9: -4.658020 mag (0.042329) (with inputMask value 0)
    

Which gives the scale factors (from the CMF file headers):

 14.36839
 13.85586
 26.54338
 13.7809
 13.93725
 14.07059
 19.71039
 40.63887
 78.97991
 72.9807
    

And also estimates the sky as:

 image sky : mean 0.236319,  stdev 24.309254, dQ 0.380525
 image sky : mean 0.172495,  stdev 23.207118, dQ 0.380015
 image sky : mean 0.101928,  stdev 17.963553, dQ 0.206238
 image sky : mean 0.180381,  stdev 24.455099, dQ 0.403623
 image sky : mean -0.124442, stdev 62.213806, dQ 0.248667
 image sky : mean 0.280146,  stdev 32.540251, dQ 0.361924
 image sky : mean 0.274243,  stdev 45.880944, dQ 0.475878
 image sky : mean 0.601617,  stdev 29.982373, dQ 0.219379
 image sky : mean 0.339365,  stdev 29.517882, dQ 0.160456
 image sky : mean 0.501157,  stdev 31.656462, dQ 0.230390
    

Not too dissimilar to the means already found.