Skip to main content
The search returned 7 results
Now showing 1 - 7 of 7
  • Some of the metrics are blocked by your 
    Item type:Publication, Access status: Open Access ,

    An EMSO data case study within the INDIGO-DC project

    We present our experience based on a case study within the INDIGO-DataCloud (INtegrating Distributed data In-frastructures for Global ExplOitation) project (www.indigo-datacloud.eu). The aim of INDIGO-DC is to develop a data and computing platform targeting scientific communities. Our case study is an example of activities performed by INGV using data from seafloor observatories that are nodes of the infrastructure EMSO (European Multidisciplinary Seafloor and water column Observatory)-ERIC (www.emso-eu.org). EMSO is composed of several deep-seafloor and water column observatories, deployed at key sites in the European waters, thus forming a widely distributed pan-European infrastructure. In our case study we consider data collected by the NEMO-SN1 observatory, one of the EMSO nodes used for geohazard monitoring, located in the Western Ionian Sea in proximity of Etna volcano. Starting from the case study, through an agile approach, we defined some requirements for INDIGO developers, and tested some of the proposed INDIGO solutions that are of interest for our research community. Given that EMSO is a distributed infrastructure, we are interested in INDIGO solutions that allow access to distributed data storage. Access should be both user-oriented and machine-oriented, and with the use of a common identity and access system. For this purpose, we have been testing: - ONEDATA (https://onedata.org), as global data management system. - INDIGO-IAM as Identity and Access Management system. Another aspect we are interested in is the efficient data processing, and we have focused on two types of INDIGO products: - Ophidia (http://ophidia.cmcc.it), a big data analytics framework for eScience for the analysis of multidimensional data. - A collection of INDIGO Services to run processes for scientific computing through the INDIGO Orchestra
      170  468
  • Some of the metrics are blocked by your 
    Item type:Publication, Access status: Open Access ,

    SeaConditions: a web and mobile service for safer professional and recreational activities in the Mediterranean Sea

    Reliable and timely information on the environmental conditions at sea is key to the safety of professional and recreational users as well as to the optimal execution of their activities. The possibility of users obtaining environmental information in due time and with adequate accuracy in the marine and coastal environment is defined as sea situational awareness (SSA). Without adequate information on the environmental meteorological and oceanographic conditions, users have a limited capacity to respond, which has led to loss of lives and to large environmental disasters with enormous consequent damage to the economy, society and ecosystems. Within the framework of the TESSA project, new SSA services for the Mediterranean Sea have been developed. In this paper we present SeaConditions, which is a web and mobile application for the provision of meteorological and oceanographic observation and forecasting products. Model forecasts and satellite products from operational services, such as ECMWF and CMEMS, can be visualized in SeaConditions. In addition, layers of information related to bathymetry, sea level and ocean-colour data (chl a and water transparency) are displayed. Ocean forecasts at high spatial resolutions are included in the version of SeaConditions presented here. SeaConditions provides a user-friendly experience with a fluid zoom capability, facilitating the appropriate display of data with different levels of detail. SeaConditions is a single point of access to interactive maps from different geophysical fields, providing high-quality information based on advanced oceanographic models. The SeaConditions services are available through both web and mobile applications. The web application is available at www.sea-conditions.com and is accessible and compatible with present-day browsers. Interoperability with GIS software is implemented. User feedback has been collected and taken into account in order to improve the service. The SeaConditions iOS and Android apps have been downloaded by more than 105 000 users to date (May 2016), and more than 100 000 users have visited the web version.
      115  158
  • Some of the metrics are blocked by your 
    Item type:Publication, Access status: Open Access ,

    A procedure to import seismological data into the Ophidia big data analytics framework

    The term Big Data identifies large data sets that cannot be managed with traditional processing software. Volume, Variety and Velocity are properties used to classify Big Data. Velocity, the most relevant property, refers to high speed of data acquisition and processing. Management of Big Data, and particularly Velocity, has become an open issue for scientific communities. The EC project INDIGO-Datacloud was conceived to tackle some of the Big Data challenges that scientific communities are facing. In the context of INDIGO-Datacloud we defined a case study in which one of the goals was the improvement of processing performance on seismological data. Starting with a practical case study, we identified, as possible software solution, Ophidia, a big data analytics framework. The first step to exploit the framework functionalities required the import of the seismological data in Ophidia. For this purpose, we designed and implemented a software module that enables the import of seismological data in SAC format. As a second step, we tested the capabilities of Ophidia through a performance test to evaluate the framework potential.
      200  73
  • Some of the metrics are blocked by your 
    Item type:Publication, Access status: Open Access ,

    Big Data Analytics on Large-Scale Scientific Datasets in the INDIGO-DataCloud Project

    In the context of the EU H2020 INDIGO-DataCloud project several use case on large scale scientfic data analysis regarding different research communities have been implemented. All of them require the availability of large amount of data related to either output of imulations or observed data from sensors and need scientic (big) data solutions to run data analysis experiments. More specically,the paper presents the case studies related to the following research communities: (i) the European Multidisciplinary Seaoor and water column Observatory (INGV-EMSO), (ii) the Large Binocular Tele-scope, (iii) LifeWatch, and (iv) the European Network for Earth System Modelling (ENES).
      108  339
  • Some of the metrics are blocked by your 
    Item type:Publication, Access status: Open Access ,

    INDIGO-DATA CLOUD EC project: A study case applied to one of the EMSO Research Infrastructure Deep sea Observatories

    Our case study is a pilot experience used to describe some of the activities performed by INGV in the frame of the European Research Infrastructure EMSO (European Multidisciplinary Seafloor and water column Observatory). EMSO is composed of several deep-seafloor and water column observatories, deployed at key sites in the European waters, thus forming a widely distributed pan-European infrastructure. We consider data collected by the NEMO-SN1 observatory, one of the EMSO nodes used for geohazard monitoring, located in the Western Ionian Sea in proximity of Etna volcano. In this poster we will focus on the Researcher and Data Manager user-types. The INGV EMSO community uses MOIST (Multidisciplinary Oceanic Information System) for storing and visualizing data and metadata produced by NEMO-SN1 Observatory. Data quality control and analysis often requires several steps that include the use of different scripts and software developed in-house, commercials tools (Matlab, R-Studio....), and proprietary tools available from sensor manufacturers. In this chain of events, some operations might require a relevant computing power. Data are retrieved from MOIST through remote mount (via samba or sshfs). Analysis might also be performed on datasets that are produced by other partners and remote access and sharing of these data is needed. At present, in the majority of cases, software is run on individual researchers’ Pcs. The first test for the implementation of our use-case within INDIGO-DataCloud included running an R script on a cloud environment and exploiting data sharing capabilities. The input to this script is data coming from the analysis of Short Duration seismic Events (SDE) automatically detected on the seismometer continuous time series. The script calculates a cumulate energy in the measurement period (8 months) and compares this cumulate curve to N random cumulates calculated by mixing the energy values at the fixed observed times where SDE are detected. Within the INDIGO project (WP2) we defined the users requirements and we identified some useful INDIGO solutions. In particular we are testing Ophidia, a software stack for big data analytics (Fiore et al., 2013), and the execution of R jobs in docker containers described by TOSCA templates through INDIGO Orchestrator and Apache Mesos. This poster illustrates our case study, the users’ requirements and the INDIGO solutions that we have been testing so far and would like to test in the near future.
      212  333
  • Some of the metrics are blocked by your 
    Item type:Publication, Access status: Open Access ,

    Enabling Dynamic and Intelligent Workflows for HPC, Data Analytics, and AI Convergence

    The evolution of High-Performance Computing (HPC) platforms enables the design and execution of progressively larger and more complex workflow applications in these systems. The complexity comes not only from the number of elements that compose the workflows but also from the type of computations they perform. While traditional HPC workflows target simulations and modelling of physical phenomena, current needs require in addition data analytics (DA) and artificial intelligence (AI) tasks. However, the development of these workflows is hampered by the lack of proper programming models and environments that support the integration of HPC, DA, and AI, as well as the lack of tools to easily deploy and execute the workflows in HPC systems. To progress in this direction, this paper presents use cases where complex workflows are required and investigates the main issues to be addressed for the HPC/DA/AI convergence. Based on this study, the paper identifies the challenges of a new workflow platform to manage complex workflows. Finally, it proposes a development approach for such a workflow platform addressing these challenges in two directions: first, by defining a software stack that provides the functionalities to manage these complex workflows; and second, by proposing the HPC Workflow as a Service (HPCWaaS) paradigm, which leverages the software stack to facilitate the reusability of complex workflows in federated HPC infrastructures. Proposals presented in this work are subject to study and development as part of the EuroHPC eFlows4HPC project.
      147  481
  • Some of the metrics are blocked by your 
    Item type:Publication, Access status: Open Access ,

    Mediterranean monitoring and forecasting operational system for Copernicus Marine Service

    The MEDiterranean Monitoring and Forecasting Center (Med-MFC) is part of the Copernicus Marine Environment Monitoring Service (CMEMS, http://marine.copernicus.eu/), provided on an operational mode by Mercator Ocean in agreement with the European Commission. Specifically, Med MFC system provides regular and systematic information about the physical state of the ocean and marine ecosystems for the Mediterranean Sea. The Med-MFC service started in May 2015 from the pre-operational system developed during the MyOcean projects, consolidating the understanding of regional Mediterranean Sea dynamics, from currents to biogeochemistry to waves, interfacing with local data collection networks and guaranteeing an efficient link with other Centers in Copernicus network. The Med-MFC products include analyses, 10 days forecasts and reanalysis, describing currents, temperature, salinity, sea level and pelagic biogeochemistry. Waves products will be available in MED-MFC version in 2017. The consortium, composed of INGV (Italy), HCMR (Greece) and OGS (Italy) and coordinated by the Euro-Mediterranean Centre on Climate Change (CMCC, Italy), performs advanced R&D activities and manages the service delivery. The Med-MFC infrastructure consists of 3 Production Units (PU), for Physics, Biogechemistry and Waves, a unique Dissemination Unit (DU) and Archiving Unit (AU) and Backup Units (BU) for all principal components, guaranteeing a resilient configuration of the service and providing and efficient and robust solution for the maintenance of the service and delivery. The Med-MFC includes also an evolution plan, both in terms of research and operational activities, oriented to increase the spatial resolution of products, to start wave products dissemination, to increase temporal extent of the reanalysis products and improving ocean physical modeling for delivering new products. The scientific activities carried out in 2015 concerned some improvements in the physical, biogeochemical and wave components of the system. Regarding the currents, new grid-point EOFs have been implemented in the Med-MFC assimilation system; the climatological CMAP precipitation was replaced by the ECMWF daily precipitation; reanalysis time-series have been increased by one year. Regarding the biogeochemistry, the main scientific achievement is related to the implementation of the carbon system in the Med-MFC biogeochemistry model system already available. The new model is able to reproduce the principal spatial patterns of the carbonate system variables in the Mediterranean Sea. Further, a key result consists of the calibration of the new variables (DIC and alkalinity), which serves to the estimation of the accuracy of the new products to be released in the next version of the system (i.e. pH and pCO2 at surface). Regarding the waves, the system has been validated against in-situ and satellite observations. For example, a very good agreement between model output and in-situ observations has been obtained at offshore and/or well-exposed wave buoys in the Mediterranean Sea.
      150  123