User Tools

Site Tools


open:wp4:wp4techforum5:radiointhevo

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

open:wp4:wp4techforum5:radiointhevo [2019/03/08 22:51]
bonnarel
open:wp4:wp4techforum5:radiointhevo [2019/03/13 16:21] (current)
bonnarel
Line 10: Line 10:
 |**Discussion** |  |**Discussion** | 
  
-**3 main trends in the discussion**+**Participants :** Mark Allen, Sarah Bertocco, Rosie Bolton (remotely), François Bonnarel, Françoise Genova, Ian Grange (remotely), Marco Iacobelli, Gilles Landais, Mireille Louys, Katharina Lutz, Zheng Meyer-Zhao, ​ Laurent Michel, Marco Molinaro,​Carlos Rodrigo, Eric Slezak, Yelena Stein, Arpad Szomoru, Harro Verkouter, Bernd Vollmer, Alessandra Zanichelli 
 + 
 +**3 main trends in the discussion.** Summary by FB with help of notes taken by ML and FG 
  
 I ) **status and feedback on Multi-D and existing standard protocols for radio data** I ) **status and feedback on Multi-D and existing standard protocols for radio data**
-      ObstPAP(Obscore)-SIAV2+DataLink+SODA+clients(Aladin) work for imaging cubes (science ready products) +      ObstPAP(Obscore)-SIAV2+DataLink+SODA+clients(Aladin) work for imaging cubes (science ready 
-      HiPS works for the same (full 3D HiPS or alternatively ​ 2D continuum HiPS  + DataLink access to cubes, Other moment maps SODA). See CASDA (ASKAP) data + partially ALMA. Detailed analysis by INAF. +      ​products) 
-      MOC also useful for discovery. +      HiPS works for the same (full 3D HiPS or alternatively ​ 2D continuum HiPS  + DataLink access 
-      Due to the nature of raw data, image sensitivity may change a lot from pixel to pixel .. --> need for sensitivity cubes?+      ​to cubes, Other moment maps SODA). See CASDA (ASKAP) data + partially ALMA. Detailed ​ 
 +      ​analysis by INAF. 
 +      MOC also useful for discovery. But MOC in velocity could help
 +      Due to the nature of raw data, image sensitivity may change a lot from pixel to pixel .. 
 +      ​--> need for sensitivity cubes?
       Some of the data are complex. ObsCore not really suited for this. Use of CAOM better?  ​       Some of the data are complex. ObsCore not really suited for this. Use of CAOM better?  ​
-      Coarse grain discovery of raw data (or calibrated discovery) via ObstAP/​SIAV2 (or HiPS + DataLink) probably possible, but not experimented yet. +      Coarse grain discovery of raw data (or calibrated discovery) via ObstAP/​SIAV2 (or HiPS + 
-      It appears that currently most of the data in the radio archives are not science ready. So there is a big question mark: how and why integrating visibility/​raw data in the VO?+      ​DataLink) probably possible, but not experimented yet. Links to project native pages for 
 +      the data can be made available ​ 
 +      It appears that currently most of the data in the radio archives are not science ready. 
 +      ​So there is a big question mark: why and how integrating visibility/​raw data in the VO?
  
  
 II ) **Fine grain discovery:​** II ) **Fine grain discovery:​**
  
-         Seems to be needed before retrieving data (due to data volume). Need to define some filters by characterizing data. +      ​Seems to be needed before retrieving data (due to data volume). Need to define some "​data 
-        Visibility characterisation to answer the trustability question: uv coverage = uv ranges, power plots, uv distribution maps, amplitude versus spectral frequency, versus phase and versus time.  +      ​filters" ​by characterizing data. 
-        ​sensitivity ​based discovery. ​ For a given target which are the best observations to provide some minimal sensitivity?​ It seems to require information on the beam shape and size. +      Visibility characterisation to answer the trustability question: uv coverage = uv ranges, 
-       ​Discovery based on configuration details. : number of antennas, array configuration,​ etc.. or others: ​fill factor, baseline length ... +      uv power plots, uv distribution maps, amplitude versus spectral frequency, versus phase  
-       Apparently ​some of this stuff can be provided statically, some other stuff can only be generated dynamically according to user needs.+      ​and versus time.  
 +      ​Sensitivity or resolution ​based discovery. ​ For a given target which are the best 
 +      ​observations to provide some minimal sensitivity? some minimal resolution ​? It seems to 
 +      ​require information on the beam shape and size. 
 +      Discovery based on configuration details. : number of antennas, array configuration,​ etc..  
 +      ​or others: ​filling ​factor, baseline length ... or PROPOSALS. ---> Provenance dM 
 +      ​Trustability by examining different calibration steps (provenance also ?) 
 +      Aparently ​some of this stuff can be provided statically, some other stuff can only be 
 +      ​generated dynamically according to user needs. ​This is more difficult to provide for large 
 +      field of views
       Open questions: what does the VO can do for this?       Open questions: what does the VO can do for this?
-                     - integrate additional characterisation feature in existing models (ObsCore) usage of provenance?​ +                     - integrate additional characterisation feature in existing models (ObsCore) ​ 
-                     - Use DataLink to give access to various characterisation or sensitivity features?+                       usage of provenance?​ 
 +                     - Use DataLink to give access to various characterisation or sensitivity 
 +                       features? ​Including graphically.
                      - provide dynamical tools: as web services? as Desktop applications?​                      - provide dynamical tools: as web services? as Desktop applications?​
-                             - they can probably be easily integrated in the VO as custom services using uws and service descriptor or applications using SAMP, registry and DAL communications. +                             - they can probably be easily integrated in the VO as custom services 
-                              - provide more may be cumbersome and outside of the scope of the VO.+                               using UWS and DataLink "service descriptor" ​or applications using 
 +                               SAMP, registry and DAL communications. 
 +                             ​- provide more may be cumbersome and outside of the scope of the VO. 
 +                              
 +      Last but not least : VO is not in charge of verifying the quality of the data. (No data 
 +      police !). VO Validators deal with compliance of services with the standard.  ​
  
  
 III ) **Data Access for observations (non science ready data)** III ) **Data Access for observations (non science ready data)**
  
-           Web service creating on the fly science ready data (reduction details hidden behind the service interface) +Among other possibilities:​  
-           Go back to progenitors of science data using provenance and reprocess.+           Web service creating on the fly science ready data (reduction details hidden behind the 
 +           service interface) 
 +           Go back to progenitors of science data using provenance and reprocess. ​characterisation 
 +           and other metadata information needed to reprocess (see II above)
            Port code to the data and execute CASA or whatever reduction software remotely.  ​            Port code to the data and execute CASA or whatever reduction software remotely.  ​
-           Store and distribute science data produced by users. Reusability. +           Store and distribute science data produced by users. Reusability ​of the data
-           ​Download data and reduce them with astropy ​tools in Jupyter notebooks. +           ​Download data and reduce them with Python VO package ​tools in Jupyter notebooks. 
-            "​Measurement sets" ​seems to be the most complete and widely used data model for visibility data. Used by VLBI. +           ​"​Measurement sets" ​seem to be the most complete and widely used data model for 
-              Do we need to tackle other formats (MBFITs, etc ???) +           visibility data. Used by VLBI. 
-              Do we need to map measurements ​et contents ​to some extensions of VO data models ( Cube DM, TS DM, Provenance ?) For which purpose?+           ​Do we need to tackle other formats (MBFITs, ​VLBI FITS, PSRFITS for pulsars ​etc ???) 
 +           ​Do we need to map measurements ​and metadata ​to some extensions of VO data models ​ 
 +           (Cube DM, TS DM, Provenance ?) For which purpose? Exposing what is relevant for 
 +           other users from the internal model ?
  
  
 IV )    **Conclusion.** IV )    **Conclusion.**
  
-              We need use cases and experience reports at all levels. ​ESCAP will settle a "Radio astronomy data in the VO" page organised in 3 parts as above which will gather these use cases. ​+              We need use cases (JIVE,​LOFAR) ​and experience reports ​(SKA) at all levels. ​This will 
 +              allow to find some minimal common requirements. this will allow to go from simple 
 +              ones to more complex (what can be already done with little changes, what  does need 
 +              changes of larger extent, what is outside of VO Scope). ESCAPE ​will settle a "Radio 
 +              ​astronomy data in the VO" page organised in 3 parts as above which will gather these 
 +              use cases. JIVE, LOFAR, SKA may have very different ​use cases. ​
  
  
 +|**Relevant VO standards** |
 +|**ObsCore** http://​www.ivoa.net/​documents/​ObsCore/​20170509/​index.html |
 +|**SIAP 2.0** http://​www.ivoa.net/​documents/​ObsCore/​20170509/​index.html |
 +|**DataLink** http://​www.ivoa.net/​documents/​DataLink/​20150617/​index.html |
 +|**SODA** http://​www.ivoa.net/​documents/​SODA/​20170517/​index.html |
 +|**HiPS** http://​www.ivoa.net/​documents/​SODA/​20170517/​index.html |
 +|**MOC** http://​www.ivoa.net/​documents/​MOC/​20140602/​index.html |
 +|**Cube DM (and dependencies to Dataset ​ DM and STC 2.0 data models) ** http://​www.ivoa.net/​documents/​NDimCubeDM/​20150320/​index.html |
 +|**Provenance DM** http://​www.ivoa.net/​documents/​ProvenanceDM/​20181015/​index.html |
  
           ​           ​
open/wp4/wp4techforum5/radiointhevo.1552081868.txt.gz · Last modified: 2019/03/08 22:51 by bonnarel