A Cyberinfrastructure platform to meet the needs of data intensive radio astronomy on route to the SKA

Cyber SKA

Cyber SKA

Collaborators on the CANARIE Cyber SKA Project

CyberSKA Portal Release Notes - v1.3.0 - June 30, 2011

  • Public
By Cameron Kiddle 4285 days ago

Below is a summary of the latest CyberSKA portal release (v1.3.0) on June 30, 2011. The release includes some major changes including a new site logo and look.  The centralized file management system has been replaced with a new Distributed Data Management System (v0.8.0) with support for uploading files larger than 200MB in size now. A prototype server side based FITS Viewer (v0.8.0) that can play movies of multi-dimensional images and is accessible on-line via Calgary Scientific's PureWeb technology is also included in this release.


Prominent in this release is the new site theme with a new CyberSKA logo and top banner as well as an overhaul to the home page. The home page now includes a slide show highlighting different radio telescopes and


A significant change this release is the switch to a distributed data management system. Previously all file uploads were stored on the CyberSKA Web server. The new distributed data management system is built on iRODS (Integrated Rule-Oriented Data System) which abstracts location of data, supports data replication and efficient transfer of data between storage sites. The new data management system will allow storage capacity to be easily expanded by adding new storage sites. With multiple storage sites uploads and downloads will also be able to be distributed across multiple sites, all transparent to the users knowledge.


All files on the site have been migrated to the new data management system. From a user interface perspective accessing files from the site remains the same.  Please note that with the new data management system the File widget used on group pages, profiles and dashboards needed to be replaced. As such if you are using the file widget currently it will be broken. To correct this you will need to remove the widget and add the File widget to the page again.


Affiliated with the new distributed data management system, are new Advanced Upload and Download tools. The current basic upload mechanism is restricted to a single file upload of no more that 200MB and can be continued to use for such purposes. The new Advanced Upload tool is a Java Applet (JUpload) that provides more reliability, enables upload of files greater than 200MB in size and also supports multiple file uploads at the same time. After the uploads are complete you will be prompted to enter information for each file and then press the Submit button. Please note that it may take some time to complete the submission after pressing the Submit button. There are currently no status bars to indicate that something is happening although we plan to add these in a future release. The new Advanced Download tool (CADC download manager) is a Java Web Start based application that will provide more reliability, will allow you to continue browsing while downloading, and is recommended for the download of large files.


A new server side based FITS Viewer prototype has also been included in this release that was developed in collaboration with Calgary Scientific. The previous version was client side based - meaning that the visualization tool ran in the users browsers and loads a full frame worth of data at a time. This approach is not scalable and does not allow movies of multi-dimensional images to be displayed. The previous version still has more functionality at this point and is available via the "View Image" button associated with a FITS file. The new prototype is available via the "View Movie" button associated with a FITS file and is accessible only to logged in users of the portal currently. With the new server side based FITS Viewer the visualization tool is actually run on the server side where the data resides. Users are able to interact with the tool on-line, just as though they were running it locally, via Calgary Scientific's Pure Web technology. Features currently supported by the new FITS Viewer include panning and zooming, histogram correction, color map adjustment, intensity profiles, movie player (for multi frame/channel images). If you want to try it out you could take a look at the following image from the Canadian Galactic Plane Survey - . Please note that this is only a prototype at this point.  If playing a movie it is recommend that you change the frame interval to be at least 200ms. We are working on improving performance and scalability. There is also a known bug in the mapping of the cursor position on the image. Corrections to this and more features will be part of future releases.


More details on the components released can be found below:

CyberSKA Portal v1.3.0

  • new site theme - new logo / banner/ colors/ home page layout - now features informational slide show
  • various site consistency issues were addressed
  • search by tag added for pages
  • comments added for events
  • various bug fixes
  • introduction of Distributed Data Management System v0.8.0 (all files migrated to new system)
  • new File widget (existing users of file widget should delete old one and add new one)
  • introduction of server side based FITS Viewer v0.8.0 (available via "View Movie" button for FITS files)

Distributed Data Management Service v0.8.0

  • enhancement and full integration of iRODS based distributed data management system with production portal (all files converted over to new system)
  • asynchronous generation of FITS cache files when uploaded
  • 2 legged OAuth between portal and data management service
  • addition of Advanced Upload Tool - uses JUpload v5.0.2 java applet
    • ftp used for uploads
    • multiple file uploads supported
    • allows for uploads of files > 200MB
  • addition of Advanced Download Tool - uses Java Webstart version of CADC download manager
    • http used for downloads
  • deployed UofC data management server - deployments at other participating sites in Canada to take place over the coming months

FITS Viewer v0.8.0 (Server side based FITS Viewer)

  • developed in collaboration with Calgary Scientific, using their PureWeb technology to provide Web-enabled access to a server side hosted visualization tool
  • supports interactive panning and zooming, histogram correction, color map adjustments, display pixel data value, intensity profiles, movie player, sharing of sessions
  • Flash, iPad and Android clients
  • known bugs in mapping of cursor location currently