Recently in the Data Category

"The National Science Foundation today awarded a $2.5-million grant to Rensselaer Polytechnic Institute to enable its participation in a new international organization that will accelerate research data sharing among scientists around the globe. The grant will be used to develop a Research Data Alliance (RDA) that will allow researchers the world over to collaboratively use scientific data to speed up innovation. To date, more than 120 U.S. and international participants are helping conceptualize the organization and populate its first efforts. Along with scientific and data leaders from the United States, members from Australia and the European Union are part of the new alliance's organizational steering committee. U.S. participation will be led by Rensselaer Computer Science Professor Francine Berman." More

Big Data Challenge Announced

NASA Tournament Lab to Launch Big Data Challenge Series for U.S. Government Agencies, NASA

"NASA, the National Science Foundation and the Department of Energy's Office of Science announced on Wednesday the launch of the Big Data Challenge, a series of competitions hosted through the NASA Tournament Lab (NTL). The Big Data Challenge series will apply the process of open innovation to conceptualizing new and novel approaches to using "big data" information sets from various U.S. government agencies. This data comes from the fields of health, energy and Earth science. Competitors will be tasked with imagining analytical techniques and software tools that use big data from discrete government information domains. They will need to describe how the data may be shared as universal, cross-agency solutions that transcend the limitations of individual agencies."

NASA Tournament Lab & TopCoder Launch Big Data Challenge Series for U.S. Government Agencies, TopCoder

"NASA and Harvard University have established the NASA Tournament Lab (NTL), which with the enabling capabilities of the TopCoder community allow for competitions to create the most innovative, most efficient, and most optimized solutions for specific, real-world challenges being faced by NASA researchers. The NTL provides an online virtual facility for NASA researchers with a computational or complex data processing challenge to "order" a solution, just like they would order laboratory tests or supplies."

Wyle Takes Part in TechAmerica Report on Big Data, Wyle

"TechAmerica Foundation's much anticipated report "Demystifying Big Data: A Practical Guide To Transforming The Business of Government," which was released today, gives the federal government a comprehensive roadmap to using "Big Data" to better serve Americans."

The Virtual Observatory and its Benefits for Amateur Astronomers

"The contemporary astronomical instruments have been producing the unprecedented amount of data. The largest part of this "data avalanche" is being produced by deep all-sky surveys yielding terabytes of raw data per night. Such a great data volumes can hardly even been reduced by automatic pipelines running on supercomputer grids but it is impossible to exploit fully their content by a small group of professional astronomers in the interested research teams. New tools for collaborative work with heterogeneous data sets spread over distant servers are being developed in the framework of the Virtual Observatory (VO)."

Data Mining The Cosmos

The DAME/VO-Neural Infrastructure: an Integrated Data Mining System Support for the Science Community

"The DAME/VONeural project, run jointly by the University Federico II, INAF (National Institute of Astrophysics) Astronomical Observatories of Napoli and the California Institute of Technology, aims at creating a single, sustainable, distributed e-infrastructure for data mining and exploration in massive data sets, to be offered to the astronomical (but not only) community as a web application. The framework makes use of distributed computing environments (e.g. S.Co.P.E.) and matches the international IVOA standards and requirements."

DAME: A Distributed Data Mining & Exploration Framework within the Virtual Observatory

"Originally designed to deal with astrophysical use cases, where first scientific application examples have demonstrated its effectiveness, the DAME Suite results as a multi-disciplinary platform-independent tool perfectly compliant with modern KDD (Knowledge Discovery in Databases) requirements and Information & Communication Technology trends."

How Will Astronomy Archives Survive The Data Tsunami?

"Astronomy is already awash with data: currently 1 PB (petabyte) of public data is electronically accessible, and this volume is growing at 0.5 PB per year. The availability of this data has already transformed research in astronomy, and the STScI (Space Telescope Science Institute) now reports that more papers are published with archived data sets than with newly acquired data. This growth in data size and anticipated usage will accelerate in the coming few years as new projects such as the LSST (Large Synoptic Survey Telescope), ALMA (Atacama Large Millimeter Array), and SKA (Square Kilometer Array) move into operation."

Finding New Planets in Old Data

Planets Found in Decade-Old Hubble Data

"In a painstaking re-analysis of Hubble Space Telescope images from 1998, astronomers have found visual evidence for two extrasolar planets that went undetected back then. Finding these hidden gems in the Hubble archive gives astronomers an invaluable time machine for comparing much earlier planet orbital motion data to more recent observations. It also demonstrates a novel approach for planet hunting in archival Hubble data."

To support upcoming robotic and human exploration needs, the National Aeronautics and Space Administration (NASA) anticipates that it and others will need to implement a unified architecture for internetworked communication and navigation services that span the solar system. Unlike the terrestrial internet, a future Solar System Internet (SSI) must be capable of accommodating intermittent connectivity, long or variable delays, asymmetric data rates, and high data loss rates. The underlying capability that enables the SSI is commonly referred to as "Disruption-Tolerant Networking" (DTN). The SSI will employ both opportunistic and scheduled communications paths to optimize routing among nodes of the SSI, while maintaining low communications overhead and data processing load.


"The Saratoga transfer protocol was developed by Surrey Satellite Technology Ltd (SSTL) for its Disaster Monitoring Constellation (DMC) satellites. In over seven years of operation, Saratoga has provided efficient delivery of remote-sensing Earth observation imagery, across private wireless links, from these seven low-orbit satellites to ground stations, using the Internet Protocol (IP). Saratoga is designed to cope with high bandwidth-delay products, constrained acknowledgement channels, and high loss while streaming or delivering extremely large files. An implementation of this protocol has now been developed at the Australian Commonwealth Scientific and Industrial Research Organisation (CSIRO) for wider use and testing. This is intended to prototype delivery of data across dedicated astronomy radio telescope networks on the ground, where networked sensors in Very Long Baseline Interferometer (VLBI) instruments generate large amounts of data for processing and can send that data across private IP- and Ethernet-based links at very high rates. We describe this new Saratoga implementation, its features and focus on high throughput and link utilization, and lessons learned in developing this protocol for sensor-network applications." More