Changes between Version 40 and Version 41 of SatelliteBigData

Show
Ignore:
Timestamp:
2009/03/20 14:10:23 (15 years ago)
Author:
severin
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • SatelliteBigData

    v40 v41  
    6666 
    6767Several attendees are looking into new ways to store their data because they are hitting the ceiling of their storage capacity. Several technologies were mentioned, including [http://www.ogsadai.org.uk/ OGSADAI] which is a grid-based solution where you can setup several databases that then can be queried as one. Other technologies like the cloud might also provide part of a solution. [http://aws.amazon.com/s3/ Amazon S3] and [http://www.google.com/base/ GoogleBase] allow for storing very large amounts of data (large numbers of large objects) and are relatively cheap. The upload to these systems is however very slow which must be taken into account. In addition, these services are commercial and it might be dangerous to store non-public data there. A possible solution mentioned involves creating an encrypted data image to be uploaded instead of the original dataset. Another issue is that these companies might in the future decide to increase their prices or even stop their activities. Using the cloud for data storage therefore means that you still need a local backup in case this happens. 
     68 
     69In addition to the storage problem there is the issue of actually submitting our new data to public resources when publishing.  Our datasets are getting very big and FTP often is not good enough.  This forces many of us to have to submit data on DVD via FedEx for example.  This is not optimal. 
    6870 
    6971[Jessica: can you add your vertical red line to this as well?]