Skyrect parameters

 as for skyrect parameters that are important... there are too many
  that became important ... and some are mildly degenerate [rather
  counter-intuitively since they really do orthogonal things but
  in the end can end up compensating for each other in certain events].

(1) -skyint XXX:  specifies the size of the chunk (in x) that gets worked
on. this is a memory issue. if you are working on a really long slit, like
800 rows or something, then the libraries don't like doing a B-spline
fit to an 800x2048 image and you have to break the problem down into
smaller chunks. for most multi-slit work, you can do skyint=2048 and
have it do the whole slit in one go. the default is 200 so the sky
subtraction is done in smaller sections. this does not affect the
execution time at all as the problem scales linearly with dimension
in "x".

(2) -dkx XXX: default = 1.0. this sets the knot spacing (uniform spacing).
when the B-spline is constructed, dkx is the knot spacing in the wavelength
direction in units of image pixels. for really undersampled data you want
to go with dkx=0.75 or even dkx=0.5 (certainly dkx<1). interestingly there
can be a bit of a tradeoff here with -skycut but the reason is a bit too
nutty for me to explain simply in a short email given my current state of
mind. if you have smaller knot spacings you have more knots and it takes
more cpu time to create the b-spline. if there is no line curvature and
the sky lines are perfectly oriented along columns then you certainly
don't want knots spaced less than 1 pixel apart because you would have
no information on the sky spectrum at sub-pixel positions. then you
would just go with dkx=1. so the choice of dkx kind of goes with how
tilted the lines are. there is an automated selection of knots of you
choose dkx=0 but i am not sure i managed to write it well enough for
me to be satisfied with what it does... so i wouldn't use it.

(2b) -ncut XXX: this is probably not necessary for ldss2 data. if ncut >0
then the code chooses knot locations around sky features that have
signficance greater than something like ncut*sigma. this is handy for
data where there are very few sky features and you know the sky is
mostly smooth featureless continuum.

(3) -skyplot:  a logical toggle and if invoked you get pgplot windows
showing the flagging of useful pixels for the fitting of the B-spline.
worth having up to help guide the choice of -skycut (or -check).

(4) -check XXX: half-width of the window over which the discrepant pixel
search is performed. before fitting the b-spline, all pixels in
a slit (in the skyint interval) are sorted in order of x_r (the
rectified x coordinate) and a sliding window is passed over that
sorted list to find a "robust" univariate spectrum of the sky.
this smoothed spectrum is used in the search for bad pixels (cosmic rays,
bright obejcts) and the default window is 0 [no search for bad pixels].
running with -check 5 or 8 or 3 does give some interestingly different
results... and it is worth playing with that along with...

(5) -skycut XXX: the number of sigma outside of which the bad pixels are
flagged and excluded from the fit of the b-spline. go with 5 or 8 or 10...
and use -skyplot and see what happens.

(6) -spass 1: you can also use -check 0 and skip the rejection of pixels at the
front end. you can have several passes through the fitting. with -spass 1
you only get one pass through the fit of the b-spline. with spass>1 you
get a sort of iterative fitting where subsequent passes through the fit
of the b-spline you reject pixels that are > skycut*sigma discrepant
from the fit. you can use both front-end and back-end rejection of
pixels.... with check>0 or spass>1 it all takes longer...

(7) -sky FILE: specify sky apertures. i'm not enthusiastic about this but
it is a nice old-fashioned way to go. i'm still holding out for a faster
(but so far fast is not what i'm getting, with -skycut, -check, and -spass)
way to get good sky subtraciton without having to specify sky apertures...
since you may want to impose more into the object continuum at the
locations of bright sky lines.... and thus have wavelength-dependent
sky apertures... so i may end up re-working the automated continuum
tracing down to some S/N threshold, generating of a mask image etc...
but haven't done that in the context of skyrect (yet).

(8) -goofy: don't use this. it is a scheme to increase the spatial
knot density but i don't remember if i got it to work well. this was
to help deal with fringing...

(9a) -kx 3:  this is the order of the b-spline in the x direction.
larger kx increases cpu time and often a higher order isn't warrented.
kx=3 is usually best but sometimes [maybe even with ldss2 data?] kx=1
works great. this can be traded off with the knot spacing as if you
have smaller knot spacing you can get by with a b-spline that is
only 1st order between knots.

(9b) -ky 1:  this is the order of the b-spline in the spatial direction.
if you have long-ish slits go with ky=3 or for really long ones use ky=5.
if your slits are *short* and you think you have flat-fielded okay try
out ky=0 and see if a univariate B-spline works okay for your data/slit.

(10) -x0 XXX -x1 XXX: if you only want to do the sky fit between certain
columns.

(11) -b XXX:  don't include the XXX pixels nearest the slit ends [as
defined by the stuff in the header]. -b1 and -b2 are used to define
different boundaries for bottm and top of slits.


   whenever i create a new task or script [like skyrect!] i always start
   with zero parameters. then suddenly i find that there are things that
   are different about different datasets and i have to create a new
   parameter. sadly skyrect has way too many... in part because i haven't
   had time to make it very intelligent about knot locations, in part
   because i don't want to slow it down more than it needs to be, and
   in part because bivariate B-splines are still a bit new to me... sadly.

   all of this means that even i, yes even i, will take one or two passes
   through an image to ensure that the sky subtraction is as good as
   i can get it. often one set of parameters works okay for >75% of the
   slits and then maybe i tweak a couple of the slits to improve things.

   it annoys me greatly that two seemingly identical slitlets will
   "require" slightly different parameters to provide equally good
   rectifications, or eqaully good wavelength calibrations, or equally
   good sky subtraction. sometimes things just go a little non-linear and
   i have no idea why some data are better behaved than others...

   oh yeah: one more parameter:

(12) -pct XXX:  30 is the default. when doing the sliding window
to get a "robust" sky spectrum for doing the exclusion of bax pixels
before the fit. this is the percentile used. a median would be -pct 50.
but the median can be contaminated by object flux if the object covers
a large fraction of the sli length. if the slits are much much longer
than the object width then try out -pct 40. if you use -skyplot you
can see in the lower window the effects of changing -pct. The reason
why I use -pct instead of something like a biweight location estimator
[which *would be the best thing to do*] is that a slitlet can contain
>10000 pixels and you already have to sort them into a long 1D array
and then in the sliding window you are doing >10000 sorts of (2*check+1)
pixels  and a biweight computation would slow things down too much. so
i compromised on this and a 30th percentile-smoothing is usually
pretty darn okay.

  anyway, this is where things are at with skyrect right now. with
  each dataset i crunch, things improve or get tweaked and it is
  still pretty early days for this script [it is only 3 months old].

  as for comparisons with IRAF:   LDSS2 data are reasonably sampled so,
  sadly, the big gain of doing skyrect first over rectifying first
  is going to be small. also LDSS2 data do the worst of all the
  spectroscopic data i've evcer dumped through my scripts --- grism
  data are always tough because the distortions can be crazy... and
  ldss2 data show a bit of line curvature because the distortions are
  so crazy [grisms in general give you zero line curvature --- the lines
  are supposed to be straight] and so you can't really restrict the
  rectifications to -x 1 -y 1 and get great results with ldss2 data.
  [you should see the ldss2 long slit data i have --- i have to go to
  crazy crazy high orders for the rectifications...]

  you can always just do "unrect" on the data [without having run skyrect]
  and then run easyrect on the rectified frames to have things go
  swiftly... since the traditional route is less cpu-intensive anyway.