LOFAR ERIC has been officially set up by the European Commission on 20 December 2023.

LOFAR ERIC will soon be taking on all governance and operations responsibilities for the LOFAR Distributed Research Infrastructure. This website will be updated accordingly in due time.

Retirement of Early LOFAR Cycle data (Cycle 0 to Cycle 6)

LOFAR is preparing the upgrade to the LOFAR2.0 system, which will offer simultaneous low- and high-band observing, increased field-of-view, and various other improvements to the sensitivity and operation of the telescope. The new system will generate considerably more data than the current one. The need for processing and storage of this data represents a significant challenge, as it outstrips current affordable solutions. To realize the goals of the new Observatory, the ILT board has approved a data life cycle that optimises science data processing capacity and efficiency with respect to affordable data storage. This includes data retirement. The ILT anticipates that (final) science data products, generated through standard pipelines (such as image cubes) can be kept available indefinitely (under open access policies), but that raw/intermediate products may only be retained for further processing within a restricted dwell time.

To prepare the ILT infrastructure for the LOFAR2.0 data challenges, a first step needs to be taken now that involves the retirement of early Cycle data from the LOFAR Long-Term Archive (LTA). After careful analysis, it has been concluded that data from Cycle 0 up to and including Cycle 6 is either low in metadata quality or already valorized and published (https://science.astron.nl/telescopes/lofar/science-with-lofar/lofar-papers/) or, based on the download statistics, of little demonstrable interest to the current community. The ILT has therefore approved that mid-2024 this data will be deleted from all LTA sites.

Through this notification, we give the opportunity to any users (i.e. not only the projects’ PI’s) who may need to retrieve data from these early LOFAR Cycles, to signal this to the Observatory to coordinate the data download. The requests will be evaluated by the Observatory for technical feasibility and reviewed by the ILT-board for approval. We emphasize that all data concerned is public data.

The data selected for retirement

This excel overview lists all the data that will be retired mid-2024.
The spreadsheet contains two tabs:

  • ‘Projects’: in this tab you will find the general details of the projects whose data will be retired: Project code, title, abstract and size in TB of the project at each of the LTA sites (SURF – column D, Juelich – column E, and Poznan – column F) along with the total size of the project across all sites (column G);
  • ‘Observation and Pipeline ID’s’: in here, you will find two tables. In the table on the left you will find the list of all the projects whose data are selected for retirement. In this table, you can select the projects for which you want to display more information in the table on the right. A ‘multi-select’ option is available in the top right corner of the table. The table on the right provides the detailed list of observation and pipeline products to be retired (column E) and their size (in TB) at each LTA site (SURF – column F, Juelich – column G, Poznan – column H). The total is given in column I.

Note that data sizes in the spreadsheet have byte level accuracy but are shown with one decimal accuracy.
More information about the data products can be obtained by querying the LOFAR LTA here. The LTA portal provides documentation on how to use the query service.

If you encounter issues in opening the spreadsheet on your system, please get in contact with us through the SDC Helpdesk.

What we request from you

If you need to download some of the data targeted for retirement, we request you to adhere to the guidelines given below.

To avoid excessive load of the data access services at the LTA sites, the Observatory needs to be informed of your data download plans and needs to evaluate them in relation to data access requirements for other users and purposes. For this, we ask you to submit a formal request for data download by using the Google form linked below. You will be asked to answer questions about the data you wish to retrieve, their size at the various LTA sites, and your data management plan.

https://forms.gle/n8i25WfrDXAxq7U19

The deadline for submitting your request is Friday, 1 December, 12 UT (noon).

The process

All the requests will be evaluated by the Observatory for technical feasibility and will be presented to the ILT-board for their approval and to evaluate whether specific actions are needed. Observatory personnel will notify you of the outcome of your request as soon as possible after this, along with instructions on when and how to initiate the data transfers. Instructions may include recommendations on how efficient data transfers can be achieved given the size and location of the data in relation to the destination(s) indicated in the provided data access plan.

Questions

If you have any questions about the above, do not hesitate to contact us through the SDC Helpdesk.

LOFAR News

LOFAR ERIC: Distributed Research Infrastructure for European Astronomical Research Launched

LOFAR ERIC (European Research Infrastructure Consortium) has been officially launched at its first Council meeting today. The world-leading LOFAR (LOw Frequency ARray) Distributed Research Infrastructure has already revolutionised low-frequency radio astronomy research, resulting in an avalanche of scientific publications in the past decade. LOFAR ERIC is now a single legal entity across the European Union.

Published 2 hours ago by

LOFAR ERIC

defines and implements a common long-term strategy, joint fundraising, and consistent prioritisation of the development effort for LOFAR2.0, as well as optimising the availability of the collective partner and LOFAR ERIC owned facilities (including sensor, compute, and data storage resources) that form the LOFAR research infrastructure.

LOFAR ERIC

is the appropriate vehicle to attract and consolidate partners with a range of levels of involvement, and to establish and maintain clear long-term policies and funding stability for the organisation and its infrastructure as a whole.

LOFAR ERIC

cohered at the (multi-)national level, brings appropriate visibility and recognition at national and European levels, facilitating a dialogue with science policy makers and funders across its working domain.

LOFAR ERIC

with its unique and cutting-edge facilities, has a continuing role in assuring and stimulating the vitality of the European science community.
crossmenuchevron-down