The DigiPolis project here at AG NBI is developing a Self-organized Semantic Storage Service (“S4″). A research paper written by NBI staff members Hannes Mühleisen, Tilman Walther and Prof. Robert Tolksdorf discussing self-optimization of the stored data was just accepted for the Third World Congress on Nature and Biologically Inspired Computing (NaBIC2011) in Salamanca, Spain.
The publication is titled “Data Location Optimization for a Self-Organized Distributed Storage System”, here the abstract:
Nature-inspired algorithms allow the creation of complex systems that are scalable in many dimensions, adaptable to changing conditions, and robust against failure. Our S4 system employs these algorithms to provide a distributed storage service based on swarm operations. Here, autonomous agents move on a virtual landscape of connected computers to store and retrieve data. However, these swarm-based approaches achieve their impressive performance by trading away correctness guarantees, occasionally leading to misplaced data items. In order to achieve consistent storage, there is a need for a constant optimization of the store’s data structure. In this paper, we describe a fully distributed and scalable heuristic for the optimization of the location of stored data items within a distributed storage system. We evaluate our heuristic using best- and worst-case test data sets to determine whether our location optimization method converges and whether it improves the location and organization of data inside a large-scale storage network.