My long-term goal is to eventually put together a large data ingest and aggregation service for academic research that pools together data from hundreds of sources and allows people to choose what data they are interested in so that they can have instant access to that data.
Conversation
This system would be able to stream that data into other systems (such as machine learning systems, AI systems, etc.). It would be completely open and give the end user complete control on how the data is formatted, delivered, filtered, etc.
1
One of my biggest frustrations with using data is that every source has their own different methods of presenting the data. Each API is different with parameters that change from system to system. I want to create a global repository of data on a massive scale (petabytes) and
Replying to
create a system that can search and filter the data quickly so that researchers could instantly know the temperature changes in an obscure place or instantly see all earthquakes between certain magnitudes, time-frames, etc. This is my ultimate end-goal.
1