java - key-value store suggestion

Question

I need a very basic key-value store for java. I started with a HashMap but it seems that HashMap is somewhat space inefficient (I'm storing ~20 million records, and seems to require ~6GB RAM).

The map is Map<Integer,String>, and so I'm considering using GNU Trove TIntObjectHashMap<byte[]>, and storing the map value as an ascii byte array rather than String.

As an alternative to that, is there a key-value store that only requires adding jar files, does not hold the entire map in RAM at once, and is still reasonably fast?

Answer 1

BabuDB

BabuDB is an embedded non-relational database system. Its lean and simple design allows it to persistently store large amounts of key-value pairs without the overhead and complexity of similar approaches such as BerkeleyDB.

License: New BSD license, Language: Java

JDBM2

JDBM2 provides HashMap and TreeMap which are backed by disk storage.

License: Apache License 2.0, Language: Java

Banana DB

Banana DB is a self-contained key/value pair database implemented in Java.

License: Apache License 2.0, Language: Java

---

I've tried BabuDB and JDBM2 and they work fine. BabuDB is a little bit more difficult to set up, but potentially delivers higher performance than JDBM2.

These all all databases, which allow to persist data on disk. There are also solutions to hold a large map in memory (ehcache, hazelcast, ...).