# Getting Started¶

## Installing the Pipeline¶

The ssos pipeline is available from the Python Package Index via [1]

$pip install ssos  Alternatively, you can clone the GitHub Repository or download the zip file. If not installing with pip, the package dependencies can be installed by running the following line within the package directory $ pip install -r requirements.txt


or by using the setup script:

$[sudo] python setup.py install  Make sure that the astrOmatic binaries sex, scamp, and swarp are in your PATH shell variable. Try it with e.g. $ sex --version
SExtractor version 2.25.0
$scamp --version SCAMP version 2.7.8$ swarp --version
SWarp version 2.38.1


After installing, you can verify that the install worked by entering ssos from anywhere in your system. It should give you the version and some basic instructions.

## Pipeline Setting Files¶

The default default.ssos file can be found here. It is a plain ASCII, designed very similar to the configuration files of SExtractor and SCAMP in order to make the user feel right at home. Short descriptions and default values of the parameters are below, for more detailed descriptions refer to the Pipeline page.

Note

The default configuration file default.ssos and the auxiliary SExtractor, SCAMP, and SWARP setup files are included in the python package and can be copied to the current working directory using the ssos -d or ssos --default syntax.

Parameter Values Examples Description
SCI_EXTENSION mixed 1 | 2 | 1,2 | all Index of science extension of FITS images. For details, see SExtractor.
WEIGHT_IMAGES mixed False | /tmp/weights Absolute path to weight images for SExtractor run. [2] If False, SExtractor runs with settings according to ssos.sex file.
DATE-OBS string DATE-OBS / DATE FITS header keyword for observation date in ISOT or MJD format
FILTER string FILTER FITS header keyword for observation filter/band
EXPTIME string EXPTIME / TEXP / EXP FITS header keyword for exposure time in seconds

SEX_CONFIG string semp/sso.sex SExtractor configuration file for source detection in the survey images. For details, see SExtractor.
SEX_PARAMS string semp/sso.param SExtractor output parameter for source detection in the survey images. For details, see SExtractor.
SEX_FILTER string semp/gauss_2.5_5x5 .conv SExtractor convolution filter file for source detection in the survey images. For details, see SExtractor and the Guide to SExtractor.
SEX_NNW string semp/sso.nnw SExtractor neural network for galaxy-star differentiation. For details, see SExtractor and the Guide to SExtractor.

SCAMP_CONFIG string semp/sso.scamp SCAMP configuration file to link source detections at different epochs, see SCAMP.
REMOVE_REF_SOURCES bool True | False Remove source detections close to reference catalogue sources, see SCAMP.

SWARP_CONFIG string semp/sso.swarp SWARP configuration file for creation of cutout images of SSO candidates, see Optional Analyses.

FILTER_DETEC bool True | False Turn filter based on number of detections on or off. See Filter Chain.
DETECTIONS integer 1,2 | 1,2,3,4 | 1,5 Sources with this number of detections are rejected. See Filter Chain.
FILTER_PM bool True | False Turn filter based on proper motion values on or off. See Filter Chain.
PM_LOW float
Lower limit on proper motion of sources. See Filter Chain.
PM_UP float
Upper limit on proper motion of sources. See Filter Chain.
PM_SNR float
Lower limit on signal-to-noise ratio of proper motion of sources. See Filter Chain.
FILTER_PIXEL bool True | False Turn filter based on pixel positions on or off. See Filter Chain.
DELTA_PIXEL float
Minimum number of pixel the centre position of the source has to shift by over all exposures in X and Y. See Filter Chain.
FILTER_MOTION bool True | False Turn filter based on linearity of motion on or off. See Filter Chain.
IDENTIFY_OUTLIER bool True | False Identify outliers in epoch-space and treat their motion separately. See Filter Chain.
OUTLIER_THRESHOLD float
Threshold in Median Absolute Deviations for identification of outlier. See Filter Chain.
R_SQU_M float 0.95 Lower limit of R-Squared goodness-of-fit parameter for linear motion fit. Must be between 0 and 1. See Filter Chain.
FILTER_TRAIL bool True | False Turn filter based on constant trail parameters on or off. See Filter Chain.
RATIO float 0.25 Lower limit on the ratio of the error on the weighted mean to the standard deviation of the source ellipse parameters. See Filter Chain
FILTER_BRIGHT_SOURCES bool True | False Turn filter based on source distance to bright sources on or off. See Filter Chain.
DISTANCE float
Minimum distance of source to bright star in star catalogue in arcsecond. See Filter Chain.
MAG_LIMITS float -99,99 Minimum and maximum magnitudes of bright sources in the catalogue. See Filter Chain.
BRIGHT_SOURCES_CAT string REFCAT | path/to/cat Use SCAMP reference catalogue or provide path to one, e.g. HYG. See Filter Chain

CROSSMATCH_SKYBOT bool True | False Turn cross-matching with SkyBoT database on or off. See Optional Analyses.
Upper limit of distance between source candidate and SkyBoT source to be considered a match, in arcsecond. See Optional Analyses.
OBSERVATORY_CODE string 500 IAU Observatory Code
FOV_DIMENSIONS string 0x0 Dimensions of exposure field-of-view in degrees, see SkyBoT.
EXTRACT_CUTOUTS bool True | False Turn cutout extraction with SWARP on or off. See Optional Analyses.
CUTOUT_SIZE integer 256 Size of cutouts in pixel, each dimension, see Optional Analyses.
FIXED_APER_MAGS bool True | False Compute fixed aperture magnitudes for colours. See Optional Analyses.
REFERENCE_FILTER string gSDSS,uSDSS Filter to use as reference in SExtractor dual-image mode runs. Value has to correspond to FILTER keyword in FITS header. See Optional Analyses.
CHECKPLOTS string SKYBOT_RESIUDALS,SKYBOT_PM,False Checkplots to create. False for no checkplots, otherwise a comma-separated list of possible checkplots, given in Optional Analyses.

The configuration file can be formatted with tabs and spaces. Comments are marked with #. Lines beginning with # or newline characters are ignored.

Note

The pipeline script first checks if the -c flag is pointing to a configuration file. If not, it looks for a file called default.ssos in the current working directory. If no file is found, the hard-coded default values are used. Any parameter can be overwritten temporarily by using the appropriate flag, see Command-Line API.

## Step-by-step workflow¶

It is unlikely that the pipeline will give you the optimum result (clean and complete sample of SSOs) right out-of-the-box. Before running ssos on thousands of images from an observation campaign, it is helpful to pick test images taken close to the ecliptic to find the right settings. Outlined here is a typical workflow to set-up the pipeline for batch processing of images. The order of these steps can be varied in some cases. Further help can be found on the Tips & Tricks & Troubleshooting page.

1. Create a directory to contain the configuration files. In the directory, run
$ssos --default  to copy the default configurations files into this directory. Adjust the SEX_CONFIG, SEX_FILTER, SEX_PARAMS, SEX_NNW, SCAMP_CONFIG, SWARP_CONFIG parameters in the ssos config file to point to the just created files. 1. Open some images in DS9 and use the Analysis functionality to overplot known SkyBoT SSOs in the FoV. Make note of how many known SSOs are visible in how many images - these numbers will be a goal to aim for in the following fine-tuning steps 2. Adjust the image specific keywords in the ssos config file: RA, DEC, OBJECT, DATE-OBS, FILTER, EXPTIME. Insert the corresponding header keyword names. You may have to create or adjust this keywords for the pipeline using e.g. wcstools 3. Run the pipeline in default settings. Use topcat to inspect the output catalogues and find out why known SSOs were missed. You can load all catalogues into topcat and overplot them in RA-Dec space. 4. If an SSO is not in the full catalogue created by SCAMP, you have to adjust the SExtractor settings. To quickly test different settings, you can run the pipeline again using the debugging flag -l DEBUG and copy the necessary SExtractor command that is printed to the console. Using Aladin, you can overplot the SExtractor catalogues and the images and see why source detections are missing or were rejected. 5. If an SSO is in the full catalogue but not in the final sample, SCAMP may have mis-associated source detections. Open the full catalogue and click through the SSO detections in the graphical display in RA-Dec space. Do they share the same SOURCE_NUMBER? You may have to adjust the CROSSMATCH_RADIUS parameter. 6. If an SSO is correctly linked up by SCAMP, it is removed by the filter pipeline. Adjust the settings in the ssos config file. 7. Inspect the checkplots and output candidates: After running ssos with EXTRACT_CUTOUTS set to True, you can quickly inspect the output candidates sample by running $ ssos --inspect path/to/output/folder


where the output folder is the directory which contains the cats and cutouts directories. You are then shown videos of the recovered sources one after the other and can quickly classify them using the arrow keys: “left arrow” for artifact, “right arrow” for asteroid, “up arrow” for unknown/unclear. These classifications are saved in the output csv database. If a candidate was matched to a SkyBoT source, its designation is printed in the upper left part of the animation.

1. Once you are confident about the recovered candidates, the accuracy of the positions (check the SKYBOT_RESIDUALS checkplot), and included calibrated magnitudes as MAG_CALIB column to the database, you can run
$ssos --mpc path/to/output/csv  on the pipeline CSV output file to convert it to the MPC submission format. Apart from the default.ssos parameters listed above, you likely have to adjust the following files and parameters before running it the first time, mostly by setting them to the appropriate FITS header keywords of your images: ssos.sex • SATUR_KEY • GAIN_KEY • SEEING_FWHM • MAG_ZEROPOINT semp/ssos.scamp • ASTRINSTRU_KEY • ASTRACCURACY_KEY • PHOTINSTRU_KEY • MAGZERO_KEY • EXPOTIME_KEY • AIRMASS_KEY • EXTINCT_KEY • PHOTOMFLAG_KEY Special care has to be taken with the CROSSID_RADIUS. It defines the maximum distance in arcsec between two source detections. Therefore, the CROSSID_RADIUS divided by the time between two sub-sequent exposures sets an upper limit on the proper motion of sources that can be detected in both exposures. Increasing the CROSSID_RADIUS will therefore allow for the detection of fast SSOs, however, it also increases the amount of randomly associated detections, making artifact detections more likely. A rule of thumb: The CROSSID_RADIUS divided by the longest time between two-exposures (lowest upper proper motion limit) should be around 100”/h. semp/ssos.swarp • GAIN_KEYWORD After these initial changes, you should experiment with the different SExtractor, SCAMP, and pipeline settings, adjusting e.g. the filter chain parameters. A good way to fine-tune is to pick a test field with several SSOs and run the pipeline with different configurations. The cutout images will tell you what types of artifacts are remaining and whether you accidentally filtered out SSOs by restricting the candidate filters too much.  [1] The installation might fail if the pip tool is outdated, due to a change in the PyPI retrievals. If this is the case, run $[sudo] pip install --upgrade pip and repeat the install.
 [2] The implementation does not allow for empty strings (e.g. to point to the current working directory). Instead, put the absolute path.