SOLARNET Metadata Recommendations for Solar Observations

Stein Vidar Hagfors Haugan, Terje Fredvik

This specification is licensed under the Creative Commons Attribution 4.0 International Public License. For more information see the LICENSE.txt in our GitHub Repository or the Creative Commons website.

If you want to find the latest released or past version, see the releases. To provide comments or suggestions, please create an issue or a pull request, or contact prits-group@astro.uio.no.

Recent significant changes are highlighted in yellow.

Abstract

Prior to these SOLARNET [1] recommendations, metadata descriptions of solar observations have been standardized only for space-based observations, but the standards have been mostly within a single mission at a time - at times with significant differences between different missions. Ground-based solar observations have typically not been made freely available to the general community, resulting in an even greater lack of standards for metadata descriptions. This situation makes it difficult to construct multi-instrument archives/virtual observatories, and makes it difficult to write generic data analysis tools. The metadata recommendations in this document aim to remedy this, to foster more collaboration and data sharing by acting as a common reference for generic analysis software and for ingestion into solar virtual observatories. As part of this, we also introduce a set of new, flexible mechanisms to represent metadata in FITS files (see the list of Appendices).

These recommendations are currently being followed by at least nine data pipelines [2] . Please notify prits-group@astro.uio.no if you use these recommendations so your pipeline can be added to the list. We also highly recommend that you ask to be put on our mailing list for occasional discussions and announcements of changes (prior to implementation in the document itself). This is important to ensure we do not implement changes that have already been “locked in”.

[1]

The SOLARNET project was funded by the European Union’s Horizon 2020 and FP7 programmes under grant agreements No 312495 and 824135. Version 2.0 of this document was the final version produced under these grants, but it keeps evolving as further needs and issues arise.

[2]

Solar Orbiter SPICE, SST CHROMIS/CRISP (SSTRED), AISAS/Lomnicky Stit COMP-S/SCD, SAMNET, Gregor HiFI/GFPI, ROB USET, Alma pipeline for Solar data (SOAP), INAF IBIS-A (IBIS data Archive), PADRE/MeDDEA

Release History

• Release History
  • Latest
  • 3.2.4
  • 3.2.3
  • 3.2.2
  • 3.1.0 (2025-06-25)
  • 3.0.0 (2025-02-04)

Part A. Description of FITS Keywords

• 1. About file formats
• 2. Header and Data Units (HDUs) in FITS files
  • 2.1 Naming of HDUs in SOLARNET FITS files
  • 2.2 Fully and partially SOLARNET-compliant Obs-HDUs
  • 2.3 Other HDUs
  • 2.4 Comments
• 3 The World Coordinate System (WCS) and related keywords
  • 3.1 Fundamental WCS coordinate specification
  • 3.2 WCS positional keywords and relative radial velocity
• 4 Time-related WCS keywords
  • 4.1 Specifying WCS time coordinates
• 5 Description of data contents
  • 5.1 Data type/units (BTYPE/BNAME/BUNIT)
  • 5.2 Exposure time, binning factors
  • 5.3 Cadence
  • 5.4 Instrument/data characteristics etc
  • 5.4.1 Polarimetric data reference system
  • 5.5 Quality aspects
  • 5.6 Data statistics
  • 5.6.1 Missing and saturated pixels, spikes/cosmic rays, padding, etc
  • 5.6.2 Explicit listing of missing, saturated, spike/cosmic ray pixels etc
• 6 Metadata about affiliation, origin, acquisition, etc
• 7 Grouping
  • 7.1 Pointing ID separates observations into groups
  • 7.2 Further separating observations with identical pointing ID
  • 7.3 Group ID ties together logical groups of observations
  • 7.4 Mosaics
• 8 Pipeline processing applied to the data
  • 8.1 Basic description of processing software
  • 8.2 Detailed description of all processing steps
• 9 Integrity and administrative information
• 10 Reporting of events detected by the pipeline/spacecraft
• Appendices
  • Appendix I. Variable-keyword mechanism
    • Appendix I-a. Variable keywords using coordinate association
    • Appendix I-b. Variable keywords using pixel-to-pixel association
    • Appendix I-c. Array-valued keywords (no association)
    • Appendix I-d. Using image extensions instead of binary tables
  • Appendix II. Pixel list mechanism for flagging pixels
  • Appendix III. Meta-HDU mechanism
    • Appendix III-a. Extension to multiple split dimensions
  • Appendix IV. Adaptation to binary table extensions
  • Appendix V. Other recommendations or suggestions
    • Appendix V-a. File naming suggestions
    • Appendix V-b. Storing data in a single file or in separate files
    • Appendix V-c. Obs-HDU content guidelines
  • Appendix VI. Extended mechanism for distortion corrections
    • Appendix VI-a. Multiple distortion corrections to the same coordinate
  • Appendix VII. External Extension References
    • Placeholder extensions
    • Virtual external extensions
  • Appendix VIII. File list wildcard patterns and sorting
  • Appendix IX. Higher-level (Level P) data: parameterized components

Part B. Lists of Mandatory and Optional FITS Keywords with Example Values

• 12 Mandatory keyword for all HDUs (Section 2.1)
• 13 Mandatory keywords for all Obs-HDUs (Section 2.2)
• 14 Mandatory WCS keyword for all HDUs with a UTC (time) coordinate (Section 4.1)
• 15 Mandatory keywords for fully SOLARNET-compliant Obs-HDUs
  • 15.1 Mandatory general keywords (Sections 8, 9, and Appendix V-a)
  • 15.2 Fundamental WCS coordinate keywords (Section 3.1)
  • 15.3 Mandatory WCS positional keywords (Section 3.2)
  • 15.3.1 Mandatory for ground-based observatories (Section 3.2)
  • 15.3.2 Mandatory for Earth orbiting satellites (Section 3.2)
  • 15.3.3 Mandatory for deep space missions (not Earth orbiting satellites) (Section 3.2)
  • 15.4 Mandatory data description keywords (Sections 5.1, 5.2 and 5.6.2)
  • 15.5 Mandatory keywords identifying the origin of the observations (Section 6)
  • 15.6 Mandatory keywords for spectrographs and filter instruments (Sections 3.2 and 5.4)
  • 15.7 Mandatory keyword for spectrographs (Section 5.4)
  • 15.8 Mandatory keywords for polarimetric data (Section 5.4.1)
  • 15.9 Mandatory keyword for grouping (Sections 7 and Appendix V-b )
• 16 Mandatory keyword for SOLARNET HDUs that contain keywords with a definition in conflict with the specifications in this document (Section 2.2)
• 17 Mandatory keywords for all HDUs that uses any of the variable-keyword, pixel list or meta-observation mechanism (Sections 2.1, 2.2, 2.3, Appendix I, Appendix II, and Appendix III)
  • 17.1 Mandatory keyword for binary table extension value columns that use pixel-to-pixel association (Appendix I-b)
  • 17.2 Mandatory keywords for binary table extensions that use the SOLARNET pixel lists mechanism for flagging pixels (Appendix I-d)
• 18 Optional keywords for all Obs-HDUs
  • 18.1 Optional keyword for deep space missions (not Earth orbiting satellites) (Section 3.2)
  • 18.2 Optional date and time keywords (Section 4)
  • 18.3 Optional keywords describing cadence (Section 5.3)
  • 18.4 Optional keyword for spectrographs (Section 5.4)
  • 18.5 Optional data description keywords (Section 5.1)
  • 18.6 Optional keywords characterizing the instrument/data (Sections, 5.4 and 5.5)
  • 18.7 Optional quality aspects keywords (Sections 3.1 and 5.5)
  • 18.8 Optional data statistics keywords (Section 5.6)
  • 18.9 Optional keywords for missing and saturated pixels (Section 5.6.1)
  • 18.10 Optional pipeline processing keywords (Sections 8, 8.1 and 8.2)
  • 18.11 Optional keyword for administrative information (Section 9)
  • 18.12 Optional keywords for grouping (Section 7)
  • 18.13 Optional keyword for binary table extensions using the variable-keyword, pixel list or meta-observation mechanism (Appendix I, Appendix I-d and Appendix III)

Part C. Part C. Alphabetical listings of FITS keywords with section references

• 19 Alphabetical listing of all keywords with section references

User’s Guide

• User’s Guide
  • Installing SOLARNET Metadata Package
  • SOLARNET Metadata Templates
  • SOLARNET Metadata Validation
  • Customizing the SOLARNET Schema
  • Acknowledging the SOLARNET Metadata Project

Developer’s Guide

• Developer’s Guide
  • Coding Standards
  • Testing Guidelines
  • Documentation Rules
  • Workflow for Maintainers
  • Version Guidelines for SOLARNET Metadata Recommendations

API Guide

• API Reference
  • solarnet_metadata Package
  • solarnet_metadata.schema Module
  • solarnet_metadata.validation Module

Citation and Acknowledgements

• Acknowledging or Citing The SOLARNET Metadata Recommendations
  • Citing The SOLARNET Metadata Recommendations in Publications
    • Other SOLARNET publications

References

• Reference List

• Index

• Search Page