A Cyberinfrastructure platform to meet the needs of data intensive radio astronomy on route to the SKA

ADASS 2011 - Day 3

Knowledge discovery (KD) work flows in astronomy

KD is used in many different fields, social sciences, security, finance, life science, google...
Survey based and driven astronomy, lots of data and many parameters measures, so the data complexity is larger. (plot of number of sources versus data complexity) - this includes better hardware and better software.

Versatile algorithm Galaxy classification
(AGNs vs normal galaxies) - Self-Organizing Map


Automatic transient classifications, scientific throughput of transients depends critically on how quickly transients can be reliably detected and classified
Clustering at work - extraction of optical candidate quasars from the SDSS photometric dataset
The Weak Gated expert is a method for the determination of z_phot for galaxies and quasars
CLaSPS method - they use python as the connective tissue, passepartout for the tables STILTS and R is the algorithm and statistics
In astronomy we are missing a repository for code, worflows and template datasets


VisIVO - a libary and integrated tools

Its c/c++ libarary
Visualisation tool - allows upload of data


Spectro-Perfectionism in SDSS-III

SDSS-III Specifically talking about BOSS - 1.5million galaxy redshifts up to z -0.7; 2D PSF extraction

Application: Bayesian stacking

R - a statistical analysis tool

1980 - s was developed in c.
R is an open source version, it mimic S is an open source system - users can provided specialized packages using CRAN - this has been growing exponentially.
Use R! conferences, The R journal & J. stat. software
Can import R into Fortran, Python, C and Ruby. "don't write it, import it"
Only one astronomy CRAN package to date; FITSio (limited functionality)
Astrostatistics and Astroinformatics Protal

Data Mining Ice Cubes

IceCube completed in Dec 2010 - 5160 digital optical modules, instrument volume of 1 km**3
They use RapidMiner  - open source Java data mining environment.

VAST Survey Overview

Radio transients with ASAKP
Will survey the whole sky two nights
Trigger events to the community, e.g. VOEvent