Getting Started

Installing the Pipeline

The ssos pipeline is available from the Python Package Index via [1]

$ pip install ssos

Alternatively, you can clone the GitHub Repository or download the zip file.

If not installing with pip, the package dependencies can be installed by running the following line within the package directory

$ pip install -r requirements.txt

or by using the setup script:

$ [sudo] python setup.py install

Make sure that the astrOmatic binaries sex, scamp, and swarp are in your PATH shell variable. Try it with e.g.

$ sex --version
SExtractor version 2.25.0
$ scamp --version
SCAMP version 2.7.8
$ swarp --version
SWarp version 2.38.1

After installing, you can verify that the install worked by entering ssos from anywhere in your system. It should give you the version and some basic instructions.

Pipeline Setting Files

The default default.ssos file can be found here. It is a plain ASCII, designed very similar to the configuration files of SExtractor and SCAMP in order to make the user feel right at home. Short descriptions and default values of the parameters are below, for more detailed descriptions refer to the Pipeline page.

Note

The default configuration file default.ssos and the auxiliary SExtractor, SCAMP, and SWARP setup files are included in the python package and can be copied to the current working directory using the ssos -d or ssos --default syntax.

Parameter	Values	Examples	Description
SCI_EXTENSION	mixed	1 | 2 | 1,2 | all	Index of science extension of FITS images. For details, see SExtractor.
WEIGHT_IMAGES	mixed	False | /tmp/weights	Absolute path to weight images for SExtractor run. [2] If False, SExtractor runs with settings according to ssos.sex file.
DATE-OBS	string	DATE-OBS / DATE	FITS header keyword for observation date in ISOT or MJD format
FILTER	string	FILTER	FITS header keyword for observation filter/band
EXPTIME	string	EXPTIME / TEXP / EXP	FITS header keyword for exposure time in seconds
SEX_CONFIG	string	semp/sso.sex	SExtractor configuration file for source detection in the survey images. For details, see SExtractor.
SEX_PARAMS	string	semp/sso.param	SExtractor output parameter for source detection in the survey images. For details, see SExtractor.
SEX_FILTER	string	semp/gauss_2.5_5x5 .conv	SExtractor convolution filter file for source detection in the survey images. For details, see SExtractor and the Guide to SExtractor.
SEX_NNW	string	semp/sso.nnw	SExtractor neural network for galaxy-star differentiation. For details, see SExtractor and the Guide to SExtractor.
SCAMP_CONFIG	string	semp/sso.scamp	SCAMP configuration file to link source detections at different epochs, see SCAMP.
REMOVE_REF_SOURCES	bool	True | False	Remove source detections close to reference catalogue sources, see SCAMP.
SWARP_CONFIG	string	semp/sso.swarp	SWARP configuration file for creation of cutout images of SSO candidates, see Optional Analyses.
FILTER_DETEC	bool	True | False	Turn filter based on number of detections on or off. See Filter Chain.
DETECTIONS	integer	1,2 | 1,2,3,4 | 1,5	Sources with this number of detections are rejected. See Filter Chain.
FILTER_PM	bool	True | False	Turn filter based on proper motion values on or off. See Filter Chain.
PM_LOW	float		Lower limit on proper motion of sources. See Filter Chain.
PM_UP	float		Upper limit on proper motion of sources. See Filter Chain.
PM_SNR	float		Lower limit on signal-to-noise ratio of proper motion of sources. See Filter Chain.
FILTER_PIXEL	bool	True | False	Turn filter based on pixel positions on or off. See Filter Chain.
DELTA_PIXEL	float		Minimum number of pixel the centre position of the source has to shift by over all exposures in X and Y. See Filter Chain.
FILTER_MOTION	bool	True | False	Turn filter based on linearity of motion on or off. See Filter Chain.
IDENTIFY_OUTLIER	bool	True | False	Identify outliers in epoch-space and treat their motion separately. See Filter Chain.
OUTLIER_THRESHOLD	float		Threshold in Median Absolute Deviations for identification of outlier. See Filter Chain.
R_SQU_M	float	0.95	Lower limit of R-Squared goodness-of-fit parameter for linear motion fit. Must be between 0 and 1. See Filter Chain.
FILTER_TRAIL	bool	True | False	Turn filter based on constant trail parameters on or off. See Filter Chain.
RATIO	float	0.25	Lower limit on the ratio of the error on the weighted mean to the standard deviation of the source ellipse parameters. See Filter Chain
FILTER_BRIGHT_SOURCES	bool	True | False	Turn filter based on source distance to bright sources on or off. See Filter Chain.
DISTANCE	float		Minimum distance of source to bright star in star catalogue in arcsecond. See Filter Chain.
MAG_LIMITS	float	-99,99	Minimum and maximum magnitudes of bright sources in the catalogue. See Filter Chain.
BRIGHT_SOURCES_CAT	string	REFCAT | path/to/cat	Use SCAMP reference catalogue or provide path to one, e.g. HYG. See Filter Chain
CROSSMATCH_SKYBOT	bool	True | False	Turn cross-matching with SkyBoT database on or off. See Optional Analyses.
CROSSMATCH_RADIUS	float		Upper limit of distance between source candidate and SkyBoT source to be considered a match, in arcsecond. See Optional Analyses.
OBSERVATORY_CODE	string	500	IAU Observatory Code
FOV_DIMENSIONS	string	0x0	Dimensions of exposure field-of-view in degrees, see SkyBoT.
EXTRACT_CUTOUTS	bool	True | False	Turn cutout extraction with SWARP on or off. See Optional Analyses.
CUTOUT_SIZE	integer	256	Size of cutouts in pixel, each dimension, see Optional Analyses.
FIXED_APER_MAGS	bool	True | False	Compute fixed aperture magnitudes for colours. See Optional Analyses.
REFERENCE_FILTER	string	gSDSS,uSDSS	Filter to use as reference in SExtractor dual-image mode runs. Value has to correspond to FILTER keyword in FITS header. See Optional Analyses.
CHECKPLOTS	string	SKYBOT_RESIUDALS,SKYBOT_PM,False	Checkplots to create. False for no checkplots, otherwise a comma-separated list of possible checkplots, given in Optional Analyses.

The configuration file can be formatted with tabs and spaces. Comments are marked with #. Lines beginning with # or newline characters are ignored.

Note

The pipeline script first checks if the -c flag is pointing to a configuration file. If not, it looks for a file called default.ssos in the current working directory. If no file is found, the hard-coded default values are used. Any parameter can be overwritten temporarily by using the appropriate flag, see Command-Line API.

Step-by-step workflow

It is unlikely that the pipeline will give you the optimum result (clean and complete sample of SSOs) right out-of-the-box. Before running ssos on thousands of images from an observation campaign, it is helpful to pick test images taken close to the ecliptic to find the right settings. Outlined here is a typical workflow to set-up the pipeline for batch processing of images. The order of these steps can be varied in some cases. Further help can be found on the Tips & Tricks & Troubleshooting page.

1. Create a directory to contain the configuration files. In the directory, run

$ ssos --default

to copy the default configurations files into this directory. Adjust the SEX_CONFIG, SEX_FILTER, SEX_PARAMS, SEX_NNW, SCAMP_CONFIG, SWARP_CONFIG parameters in the ssos config file to point to the just created files.

2. Open some images in DS9 and use the Analysis functionality to overplot known SkyBoT SSOs in the FoV. Make note of how many known SSOs are visible in how many images - these numbers will be a goal to aim for in the following fine-tuning steps
3. Adjust the image specific keywords in the ssos config file: RA, DEC, OBJECT, DATE-OBS, FILTER, EXPTIME. Insert the corresponding header keyword names. You may have to create or adjust this keywords for the pipeline using e.g. wcstools
4. Run the pipeline in default settings. Use topcat to inspect the output catalogues and find out why known SSOs were missed. You can load all catalogues into topcat and overplot them in RA-Dec space.
5. If an SSO is not in the full catalogue created by SCAMP, you have to adjust the SExtractor settings. To quickly test different settings, you can run the pipeline again using the debugging flag -l DEBUG and copy the necessary SExtractor command that is printed to the console. Using Aladin, you can overplot the SExtractor catalogues and the images and see why source detections are missing or were rejected.
6. If an SSO is in the full catalogue but not in the final sample, SCAMP may have mis-associated source detections. Open the full catalogue and click through the SSO detections in the graphical display in RA-Dec space. Do they share the same SOURCE_NUMBER? You may have to adjust the CROSSMATCH_RADIUS parameter.
7. If an SSO is correctly linked up by SCAMP, it is removed by the filter pipeline. Adjust the settings in the ssos config file.
8. Inspect the checkplots and output candidates: After running ssos with EXTRACT_CUTOUTS set to True, you can quickly inspect the output candidates sample by running

$ ssos --inspect path/to/output/folder

where the output folder is the directory which contains the cats and cutouts directories. You are then shown videos of the recovered sources one after the other and can quickly classify them using the arrow keys: “left arrow” for artifact, “right arrow” for asteroid, “up arrow” for unknown/unclear. These classifications are saved in the output csv database. If a candidate was matched to a SkyBoT source, its designation is printed in the upper left part of the animation.

9. Once you are confident about the recovered candidates, the accuracy of the positions (check the SKYBOT_RESIDUALS checkplot), and included calibrated magnitudes as MAG_CALIB column to the database, you can run

$ ssos --mpc path/to/output/csv

on the pipeline CSV output file to convert it to the MPC submission format.

Apart from the default.ssos parameters listed above, you likely have to adjust the following files and parameters before running it the first time, mostly by setting them to the appropriate FITS header keywords of your images:

ssos.sex

• SATUR_KEY
• GAIN_KEY
• SEEING_FWHM
• MAG_ZEROPOINT

semp/ssos.scamp

• ASTRINSTRU_KEY
• ASTRACCURACY_KEY
• PHOTINSTRU_KEY
• MAGZERO_KEY
• EXPOTIME_KEY
• AIRMASS_KEY
• EXTINCT_KEY
• PHOTOMFLAG_KEY

Special care has to be taken with the CROSSID_RADIUS. It defines the maximum distance in arcsec between two source detections. Therefore, the CROSSID_RADIUS divided by the time between two sub-sequent exposures sets an upper limit on the proper motion of sources that can be detected in both exposures. Increasing the CROSSID_RADIUS will therefore allow for the detection of fast SSOs, however, it also increases the amount of randomly associated detections, making artifact detections more likely. A rule of thumb: The CROSSID_RADIUS divided by the longest time between two-exposures (lowest upper proper motion limit) should be around 100”/h.

semp/ssos.swarp

• GAIN_KEYWORD

After these initial changes, you should experiment with the different SExtractor, SCAMP, and pipeline settings, adjusting e.g. the filter chain parameters. A good way to fine-tune is to pick a test field with several SSOs and run the pipeline with different configurations. The cutout images will tell you what types of artifacts are remaining and whether you accidentally filtered out SSOs by restricting the candidate filters too much.

[1]	The installation might fail if the pip tool is outdated, due to a change in the PyPI retrievals. If this is the case, run $[sudo] pip install --upgrade pip and repeat the install.

[2]	The implementation does not allow for empty strings (e.g. to point to the current working directory). Instead, put the absolute path.