Differences

This shows you the differences between two versions of the page.

--- open:wp4:wp4techforum5:radiointhevo [2019/03/13 10:13]
bonnarel
+++ open:wp4:wp4techforum5:radiointhevo [2019/03/13 16:21] (current)
bonnarel
@@ Line 12: / Line 12: @@
 **Participants :** Mark Allen, Sarah Bertocco, Rosie Bolton (remotely), François Bonnarel, Françoise Genova, Ian Grange (remotely), Marco Iacobelli, Gilles Landais, Mireille Louys, Katharina Lutz, Zheng Meyer-Zhao,  Laurent Michel, Marco Molinaro,Carlos Rodrigo, Eric Slezak, Yelena Stein, Arpad Szomoru, Harro Verkouter, Bernd Vollmer, Alessandra Zanichelli
-**3 main trends in the discussion**
+**3 main trends in the discussion.** Summary by FB with help of notes taken by ML and FG
 I ) **status and feedback on Multi-D and existing standard protocols for radio data**
-      ObstPAP(Obscore)-SIAV2+DataLink+SODA+clients(Aladin) work for imaging cubes (science ready products)
+      ObstPAP(Obscore)-SIAV2+DataLink+SODA+clients(Aladin) work for imaging cubes (science ready
-      HiPS works for the same (full 3D HiPS or alternatively  2D continuum HiPS  + DataLink access to cubes, Other moment maps SODA). See CASDA (ASKAP) data + partially ALMA. Detailed analysis by INAF.
+      products)
-      MOC also useful for discovery.
+      HiPS works for the same (full 3D HiPS or alternatively  2D continuum HiPS  + DataLink access
-      Due to the nature of raw data, image sensitivity may change a lot from pixel to pixel .. --> need for sensitivity cubes?
+      to cubes, Other moment maps SODA). See CASDA (ASKAP) data + partially ALMA. Detailed
+      analysis by INAF.
+      MOC also useful for discovery. But MOC in velocity could help.
+      Due to the nature of raw data, image sensitivity may change a lot from pixel to pixel ..
+      --> need for sensitivity cubes?
       Some of the data are complex. ObsCore not really suited for this. Use of CAOM better?
-      Coarse grain discovery of raw data (or calibrated discovery) via ObstAP/SIAV2 (or HiPS + DataLink) probably possible, but not experimented yet.
+      Coarse grain discovery of raw data (or calibrated discovery) via ObstAP/SIAV2 (or HiPS +
-      It appears that currently most of the data in the radio archives are not science ready. So there is a big question mark: why and how integrating visibility/raw data in the VO?
+      DataLink) probably possible, but not experimented yet. Links to project native pages for
+      the data can be made available
+      It appears that currently most of the data in the radio archives are not science ready.
+      So there is a big question mark: why and how integrating visibility/raw data in the VO?
 II ) **Fine grain discovery:**
-         Seems to be needed before retrieving data (due to data volume). Need to define some "data filters" by characterizing data.
+      Seems to be needed before retrieving data (due to data volume). Need to define some "data
-        Visibility characterisation to answer the trustability question: uv coverage = uv ranges, uv power plots, uv distribution maps, amplitude versus spectral frequency, versus phase and versus time.
+      filters" by characterizing data.
-        sensitivity based discovery.  For a given target which are the best observations to provide some minimal sensitivity? It seems to require information on the beam shape and size.
+      Visibility characterisation to answer the trustability question: uv coverage = uv ranges,
-       Discovery based on configuration details. : number of antennas, array configuration, etc.. or others: filling factor, baseline length ...
+      uv power plots, uv distribution maps, amplitude versus spectral frequency, versus phase
-       Apparently some of this stuff can be provided statically, some other stuff can only be generated dynamically according to user needs.
+      and versus time.
+      Sensitivity or resolution based discovery.  For a given target which are the best
+      observations to provide some minimal sensitivity? some minimal resolution ? It seems to
+      require information on the beam shape and size.
+      Discovery based on configuration details. : number of antennas, array configuration, etc..
+      or others: filling factor, baseline length ... or PROPOSALS. ---> Provenance dM
+      Trustability by examining different calibration steps (provenance also ?)
+      Aparently some of this stuff can be provided statically, some other stuff can only be
+      generated dynamically according to user needs. This is more difficult to provide for large
+      field of views
       Open questions: what does the VO can do for this?
-                     - integrate additional characterisation feature in existing models (ObsCore) usage of provenance?
+                     - integrate additional characterisation feature in existing models (ObsCore)
-                     - Use DataLink to give access to various characterisation or sensitivity features?
+                       usage of provenance?
+                     - Use DataLink to give access to various characterisation or sensitivity
+                       features? Including graphically.
                      - provide dynamical tools: as web services? as Desktop applications?
-                             - they can probably be easily integrated in the VO as custom services using uws and service descriptor or applications using SAMP, registry and DAL communications.
+                             - they can probably be easily integrated in the VO as custom services
-                              - provide more may be cumbersome and outside of the scope of the VO.
+                               using UWS and DataLink "service descriptor" or applications using
-       Last but note least : VO is not in charge of verifying the quality of the data. (No data police !). VO Validators deal with compliance of services with the standard.
+                               SAMP, registry and DAL communications.
+                             - provide more may be cumbersome and outside of the scope of the VO.
+      Last but not least : VO is not in charge of verifying the quality of the data. (No data
+      police !). VO Validators deal with compliance of services with the standard.
@@ Line 43: / Line 66: @@
 Among other possibilities:
-           Web service creating on the fly science ready data (reduction details hidden behind the service interface)
+           Web service creating on the fly science ready data (reduction details hidden behind the
-           Go back to progenitors of science data using provenance and reprocess.
+           service interface)
+           Go back to progenitors of science data using provenance and reprocess. characterisation
+           and other metadata information needed to reprocess (see II above)
            Port code to the data and execute CASA or whatever reduction software remotely.
            Store and distribute science data produced by users. Reusability of the data.
-           Download data and reduce them with astropy tools in Jupyter notebooks.
+           Download data and reduce them with Python VO package tools in Jupyter notebooks.
-            "Measurement sets" seems to be the most complete and widely used data model for visibility data. Used by VLBI.
+           "Measurement sets" seem to be the most complete and widely used data model for
-              Do we need to tackle other formats (MBFITs, etc ???)
+           visibility data. Used by VLBI.
-              Do we need to map measurements and metadata to some extensions of VO data models ( Cube DM, TS DM, Provenance ?) For which purpose?
+           Do we need to tackle other formats (MBFITs, VLBI FITS, PSRFITS for pulsars etc ???)
+           Do we need to map measurements and metadata to some extensions of VO data models
+           (Cube DM, TS DM, Provenance ?) For which purpose? Exposing what is relevant for
+           other users from the internal model ?
 IV )    **Conclusion.**
-              We need use cases (JIVE,LOFAR) and experience reports (SKA) at all levels. ESCAPE will settle a "Radio astronomy data in the VO" page organised in 3 parts as above which will gather these use cases.
+              We need use cases (JIVE,LOFAR) and experience reports (SKA) at all levels. This will
+              allow to find some minimal common requirements. this will allow to go from simple
+              ones to more complex (what can be already done with little changes, what  does need
+              changes of larger extent, what is outside of VO Scope). ESCAPE will settle a "Radio
+              astronomy data in the VO" page organised in 3 parts as above which will gather these
+              use cases. JIVE, LOFAR, SKA may have very different use cases.
+|**Relevant VO standards** |
+|**ObsCore** http://www.ivoa.net/documents/ObsCore/20170509/index.html |
+|**SIAP 2.0** http://www.ivoa.net/documents/ObsCore/20170509/index.html |
+|**DataLink** http://www.ivoa.net/documents/DataLink/20150617/index.html |
+|**SODA** http://www.ivoa.net/documents/SODA/20170517/index.html |
+|**HiPS** http://www.ivoa.net/documents/SODA/20170517/index.html |
+|**MOC** http://www.ivoa.net/documents/MOC/20140602/index.html |
+|**Cube DM (and dependencies to Dataset  DM and STC 2.0 data models) ** http://www.ivoa.net/documents/NDimCubeDM/20150320/index.html |
+|**Provenance DM** http://www.ivoa.net/documents/ProvenanceDM/20181015/index.html |

ASTERICS Wiki pages

User Tools

Site Tools

Differences

Page Tools