Clearing big data hurdles.
For Big Data applications I/O needs to be efficient and scalable so that large data sets can be accessed quickly and fed to applications for processing. With this goal in mind, a team led by Oak Ridge National Laboratory has created ADIOS, a software framework designed to handle the I/O requirements of Big Data projects. ADIOS consists of an application programming interface (API) and an associated code library. The API is used to describe the application's I/O activities in a declarative manner. An extensible markup language (XML) configuration file, which describes user I/O characteristics, then describes the variables that comprise the I/O data. Attributes of the variable, such as name, type and dimensionality, are specified in the XML, along with the desired I/O plugin. A tool called GPP leverages the I/O description in the XML file to insert automatically generated ADIOS calls into the code, making integration faster and easier. The ADIOS library is then linked to the application at compile time, and I/O operations are performed when the application is running.
* Oak Ridge National Laboratory, http://www.ornl.gov
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||Information Technologies Mechanical Systems: R&D 100|
|Comment:||Clearing big data hurdles.(Information Technologies Mechanical Systems: R&D 100)|
|Publication:||R & D|
|Article Type:||Brief article|
|Date:||Oct 1, 2013|
|Previous Article:||Dual-tasking in the depths.|
|Next Article:||Overcoming the airgap.|