A Cyberinfrastructure platform to meet the needs of data intensive radio astronomy on route to the SKA

The LOFAR Known Pulsar Data Pipeline

Transient radio phenomena and pulsars are one of six LOFAR Key Science Projects (KSPs). As part of the Transients KSP, the Pulsar Working Group (PWG) has been developing the LOFAR Pulsar Data Pipelines to both study known pulsars as well as search for new ones. The pipelines are being developed for the Blue Gene/P (BG/P) supercomputer and a large Linux cluster in order to utilize enormous amounts of computational capabilities (50Tflops) to process data streams of up to 23TB/hour. The LOFAR pipeline output will be using the Hierarchical Data Format 5 (HDF5) to efficiently store large amounts of numerical data, and to manage complex data encompassing a variety of data types, across distributed storage and processing architectures. We present the LOFAR Known Pulsar Data Pipeline overview, the pulsar beam-formed data format, the status of the pipeline processing as well as our future plans for developing the LOFAR Pulsar Search Pipeline. These LOFAR pipelines and software tools are being developed as the next generation toolset for pulsar processing in Radio Astronomy.


  • Shannon Jaeger 4229 days ago

    Ah ha!  Confirmation that LOFAR is using HDF5, luckily CASA (and Meqtrees) already support HDF5 as long as a bit if flipped when compiling.


  • Samuel George 4229 days ago

    oh no, not HDF! I had a heck of time with those files in IDL back when running Zeus to investigate the impact of the stellar wind on close-in exoplanets.