Changes between Version 80 and Version 81 of SatelliteBigData

Timestamp:

2009/03/20 16:48:29 (16 years ago)

Author:

jan.aerts

Comment:

--

SatelliteBigData

@@ -78 +78 @@
 ==== Querying ====
 
-Several approaches to make the data accessible to others were discussed. For smaller datasets regular SQL can be used to get to the data (example: the Ensembl MySQL server on ensembldb.ensembl.org). If the database schema has to be tweaked to allow larger datasets (e.g. using a minimal number of tables such as eeDB) an API becomes necessary. A webservice API where a text-representation of an object can be retrieved by URL was mentioned as useful in this case (e.g. http://www.example.com/genes/BRCA2;format=bed). For even larger datasets you want to try and limit the types of queries people can run. This way you can build a toolkit to ask this limited number of questions, but optimize that toolkit so that latency is reduced.
+Several approaches to make the data accessible to others were discussed. For smaller datasets regular SQL can be used to get to the data (example: the Ensembl MySQL server on ensembldb.ensembl.org). For complex data however a fully normalized database can become difficult to query because you might need a lot of joins. In this case an API becomes necessary or the database schema can be rewritten in a [@Arek: what's that called?] version. A webservice API where a text-representation of an object can be retrieved by URL was mentioned as useful (e.g. http://www.example.com/genes/BRCA2;format=bed). For even larger datasets you want to try and limit the types of queries people can run. This way you can build a toolkit to ask this limited number of questions, but optimize that toolkit so that latency is reduced.
 
 Streaming