Knowledge discovery (KD) work flows in astronomy

KD is used in many different fields, social sciences, security, finance, life science, google...
Survey based and driven astronomy, lots of data and many parameters measures, so the data complexity is larger. (plot of number of sources versus data complexity) - this includes better hardware and better software.

Versatile algorithm Galaxy classification (AGNs vs normal galaxies) - Self-Organizing Map

Automatic transient classifications, scientific throughput of transients depends critically on how quickly transients can be reliably detected and classified
Clustering at work - extraction of optical candidate quasars from the SDSS photometric dataset
The Weak Gated expert is a method for the determination of z_phot for galaxies and quasars
CLaSPS method - they use python as the connective tissue, passepartout for the tables STILTS and R is the algorithm and statistics
In astronomy we are missing a repository for code, worflows and template datasets

VisIVO - a libary and integrated tools

Its c/c++ libarary
Visualisation tool - allows upload of data

Spectro-Perfectionism in SDSS-III

SDSS-III Specifically talking about BOSS - 1.5million galaxy redshifts up to z -0.7; 2D PSF extraction

Application: Bayesian stacking

R - a statistical analysis tool

1980 - s was developed in c.
R is an open source version, it mimic S is an open source system - users can provided specialized packages using CRAN - this has been growing exponentially.
See www.r-project.org
Use R! conferences, The R journal & J. stat. software
Can import R into Fortran, Python, C and Ruby. "don't write it, import it"
Only one astronomy CRAN package to date; FITSio (limited functionality)
Astrostatistics and Astroinformatics Protal


Data Mining Ice Cubes

IceCube completed in Dec 2010 - 5160 digital optical modules, instrument volume of 1 km**3
They use RapidMiner  - open source Java data mining environment.


VAST Survey Overview

Radio transients with ASAKP
Will survey the whole sky two nights
Trigger events to the community, e.g. VOEvent