Java嵌入式NoSQL数据库之Berkeley DB Java Edition

一个通用的事务保护，100％纯Java编写的嵌入式NoSQL数据库，采取开源（免费）和商用（付费）的双License的授权模式。Berkeley DB是历史悠久的嵌入式数据库系统，06年被 Oracle 收购，而Berkeley DB Java Edition是Berkeley DB产品中的一部分。 

引用

Berkeley DB Java Edition is a open source, transactional storage solution for Java applications. The Direct Persistence Layer (DPL) API is faster and easier to develop, deploy, and manage than serialized object files or ORM-based Java persistence solutions.  The Collections API enhances the standard java.util.collections classes allowing them to be persisted to a local file system and accessed concurrently while protected by ACID transactions. Data is stored by serializing objects and managing class and instance data separately so as not to waste space. Berkeley DB Java Edition is the reliable drop-in solution for complex, fast, and scalable storage.

 

一、概述   

        Berkeley DB是历史悠久的嵌入式数据库系统,在06年被 Oracle 收购了。现在我们在 Oracle 网站上会看到： BerkeleyDB、BerkeleyDB XML 和 BerkeleyDB JAVA Edition 这个三个东东。简单的说最开始 BerkeleyDB 是只有 C 语言版本的，但是 JAVA 也可以使用，只不过需要通过 JNI 调用，效率可能有点影响。后来出了 JAVA Edition ，用纯 JAVA 实现了一遍，也就是我们看到的 BerkeleyDB JAVA Edition (简称 JE )。
       JE是一个通用的事务保护的，100％纯Java（JE不作任何JNI调用）编写的嵌入式数据库。因此，它为Java开发人员提供了安全高效的对任意数据的存储和管理。
       您可以通过一系列的Java API来使用JE，这些API能够读写数据，管理你的数据库，而且可以执行一些更高级的活动，如管理事务。
       和JE进行交互的Java API有两种基本形式。

• 第一个是高层次的API，可让您的Java类的持久性。
• 第二个是一个低级别的API，它提供了额外的灵活性与JE数据库。

二、特性

• 大型数据库的支持：它支持从1到数百万级的数据量，数据库的大小限制基本上受限于你的硬件支持。
• Database Environment：Database Environment提供了一个单元的封装和管理一个或多个数据库。Database Environment也是内部资源管理的一个单元，如内存中的缓存和后台线程。最后，使用它管理并发性和事务。请注意，所有应用程序使用JE都需要使用Database Environment。
• 多线程，多进程支持：JE是专为控制多线程设计的。读写操作都可以是多线程。JE使用记录级锁定为线程应用程序提供高并发性。此外，JE使用死锁超时检测的机制来确保不会有两个线程无限期的死锁。JE允许多个进程访问同一个DB，但在这种情况下， Berkeley 只允许一个线程进行写操作，读操作随意。 
• 事务：原子性，可恢复，隔离性。 
• 内存Cache：为了减少IO操作提高性能，将数据暂存在内存里面。 
• 索引。 
• 日志文件：JE数据是存储在主目录下一个或多个数字命名的日志文件。日志文件一次写成并且非常便于跨平台的移植。
• 后台线程：JE提供了几个线程为你管理内部资源。 checkpointer是负责数据库中的数据刷新到磁盘上的；compressor线程从数据库中删除由于删除活动产生的子树是空的数据；cleaner线程负责清理和删除不需要的日志文件，从而帮助您节省磁盘空间。 
• 备份和恢复：JE的备份就是将log文件copy到一个安全的存储介质， 它的灾后重建的做法就是把备份的日志文件拿出来，放到原位置, 重启JE即可。

三、maven 下载

 

<dependency>
            <groupId>com.sleepycat</groupId>
            <artifactId>je</artifactId>
            <version>6.4.25</version>
        </dependency>

四、应用       

（1）打开Database环境 

EnvironmentConfig envCfg = new EnvironmentConfig();
//当数据库环境不存在的时候，创建一个数据库环境，默认为false.
envCfg.setAllowCreate(true);
//以只读方式打开，默认为false.
envCfg.setReadOnly(false);
//事务支持，默认为false.
envCfg.setTransactional(true);

Environment mydbEnv = new Environment(new File(envHome), envCfg);

System.out.println("Env Config: " + mydbEnv.getConfig());

（2）打开Database 

DatabaseConfig dbCfg = new DatabaseConfig();
dbCfg.setAllowCreate(true);
dbCfg.setReadOnly(false);
dbCfg.setTransactional(true);

Database mytestdb = mydbEnv.openDatabase(null, "VendorDB", dbCfg);

System.out.println("Database Name: " + mytestdb.getDatabaseName());

（3）简单的CRUD 

//C
String key = "key-rensanning";
String value = "This is a test!(000)";

DatabaseEntry keyEntry = new DatabaseEntry(key.getBytes("utf-8"));
DatabaseEntry valEntry = new DatabaseEntry(value.getBytes("utf-8"));
OperationStatus status = mytestdb.put(null, keyEntry, valEntry);
System.out.println("Put Status: " + status);

//R
DatabaseEntry valGet = new DatabaseEntry();
status = mytestdb.get(null, keyEntry, valGet, LockMode.DEFAULT);
if (status == OperationStatus.SUCCESS) {
    value = new String(valGet.getData(), "utf-8");
    System.out.println("Read Value:" + value);
}

//U
value = "This is a test!(111)";
status = mytestdb.put(null, keyEntry, new DatabaseEntry(value.getBytes("utf-8")));

status = mytestdb.get(null, keyEntry, valGet, LockMode.DEFAULT);
if (status == OperationStatus.SUCCESS) {
    value = new String(valGet.getData(), "utf-8");
    System.out.println("Update Value:" + value);
}

//D
status = mytestdb.delete(null, keyEntry);
System.out.println("Delete Status: " + status);

（4）不同的数据写入方式 

String key = "key-rensanning";
DatabaseEntry keyEntry = new DatabaseEntry(key.getBytes("utf-8"));
DatabaseEntry valGet = new DatabaseEntry();

String value = "This is a test!(by 'put' method)";
//Database.put()： 向数据库写入数据，如果不支持重复记录，则会覆盖更新key对应的已有记录
OperationStatus status = mytestdb.put(null, keyEntry, new DatabaseEntry(value.getBytes("utf-8")));

status = mytestdb.get(null, keyEntry, valGet, LockMode.DEFAULT);
if (status == OperationStatus.SUCCESS) {
    value = new String(valGet.getData(), "utf-8");
    System.out.println("Put Value:" + value);
}

//Database.putNoOverwrite():向数据库写入数据，但是如果key已经存在，不会覆盖已有数据（即使数据库支持重复key）
value = "This is a test!(by 'putNoOverwrite' method)";
status = mytestdb.putNoOverwrite(null, keyEntry, new DatabaseEntry(value.getBytes("utf-8")));

status = mytestdb.get(null, keyEntry, valGet, LockMode.DEFAULT);
if (status == OperationStatus.SUCCESS) {
    value = new String(valGet.getData(), "utf-8");
    System.out.println("PutNoOverwrite Value:" + value);
}

//*****设置一个key是否允许存储多个值*****
DatabaseConfig dbCfg = new DatabaseConfig();
dbCfg.setAllowCreate(true);
dbCfg.setReadOnly(false);
dbCfg.setTransactional(true);
dbCfg.setSortedDuplicates(true);

Database mytestdb2 = mytestdb.getEnvironment().openDatabase(null, "DuplicatesDB", dbCfg);

//Database.putNoDupData():向数据库写入数据（该方法仅用于支持重复key的数据库），如果key和value对应的记录已经存在，那么操作结果是：OperationStatus.KEYEXIST
value = "This is a test!(by 'putNoDupData' method)";
status = mytestdb2.putNoDupData(null, keyEntry, new DatabaseEntry(value.getBytes("utf-8")));
if (status == OperationStatus.SUCCESS) {
    value = new String(valGet.getData(), "utf-8");
    System.out.println("PutNoOverwrite Value:" + value);
} else if (status == OperationStatus.KEYEXIST) {
    System.out.println("putNoDupData KEYEXIST:" + key);
}

status = mytestdb2.get(null, keyEntry, valGet, LockMode.DEFAULT);
if (status == OperationStatus.SUCCESS) {
    value = new String(valGet.getData(), "utf-8");
    System.out.println("putNoDupData Value:" + value);
}

if (null != mytestdb2) {
    mytestdb2.close();
    mytestdb2 = null;
}

（5）不同的数据读取方式 

String key = "key-rensanning";
String value = "This is a test!(000)";
DatabaseEntry keyEntry = new DatabaseEntry(key.getBytes("utf-8"));
DatabaseEntry valEntry = new DatabaseEntry(value.getBytes("utf-8"));
OperationStatus status = mytestdb.put(null, keyEntry, valEntry);
System.out.println("Put Status: " + status);

DatabaseEntry valGet = new DatabaseEntry();

//Database.get() ：检索key对应的记录
status = mytestdb.get(null, keyEntry, valGet, LockMode.DEFAULT);
if (status == OperationStatus.SUCCESS) {
    value = new String(valGet.getData(), "utf-8");
    System.out.println("Read Value(get):" + value);
}

//Database.getSearchBoth() ：根据key和value 检索数据库记录
status = mytestdb.getSearchBoth(null, keyEntry, valGet, LockMode.DEFAULT);
if (status == OperationStatus.SUCCESS) {
    value = new String(valGet.getData(), "utf-8");
    System.out.println("Read Value(getSearchBoth):" + value);
}

（6）对象的读写 

//C
String key = "key-rensanning-Object";
Person value = new Person(9527, "rensanning", true);

DatabaseEntry keyEntry = new DatabaseEntry(key.getBytes("utf-8"));
DatabaseEntry valEntry = new DatabaseEntry();
PersonTupleBinding personBinding = new PersonTupleBinding();
personBinding.objectToEntry(value, valEntry);

OperationStatus status = mytestdb.put(null, keyEntry, valEntry);
System.out.println("Put Person Status: " + status);

//R
DatabaseEntry valGet = new DatabaseEntry();
status = mytestdb.get(null, keyEntry, valGet, LockMode.DEFAULT);
if (status == OperationStatus.SUCCESS) {
    value = personBinding.entryToObject(valGet);
    System.out.println("Read Person Value:" + value.getId() + "\t" + value.getName() + "\t" + value.isSex());
}

class Person {
    int id;
    String name;
    boolean sex;

    public Person() {
    }

    public Person(int id, String name, boolean sex) {
        this.id = id;
        this.name = name;
        this.sex = sex;
    }

    public int getId() {
        return id;
    }
    public void setId(int id) {
        this.id = id;
    }
    public String getName() {
        return name;
    }
    public void setName(String name) {
        this.name = name;
    }
    public boolean isSex() {
        return sex;
    }
    public void setSex(boolean sex) {
        this.sex = sex;
    }
}

class PersonTupleBinding extends TupleBinding {

    @Override
    public Person entryToObject(TupleInput input) {
        Person p = new Person();
        p.setId(input.readInt());
        p.setName(input.readString());
        p.setSex(input.readBoolean());
        return p;
    }

    @Override
    public void objectToEntry(Person p, TupleOutput output) {
        output.writeInt(p.getId());
        output.writeString(p.getName());
        output.writeBoolean(p.isSex());
    }
}

（7）直接持久层应用（DPL：Direct Persistence Layer） 

Environment env = mytestdb.getEnvironment();

StoreConfig storeConfig = new StoreConfig();
storeConfig.setAllowCreate(true);
storeConfig.setTransactional(true);

EntityStore store = new EntityStore(env, "StoreDB", storeConfig);

PrimaryIndex pIndex = store.getPrimaryIndex(String.class, UserInfo.class);

//C
pIndex.put(new UserInfo("001", "user001"));
pIndex.put(new UserInfo("002", "user002"));
pIndex.put(new UserInfo("003", "user003"));
pIndex.put(new UserInfo("004", "user004"));
pIndex.put(new UserInfo("005", "user005"));

//R
String myKey = "001";
UserInfo getData = pIndex.get(myKey);
System.out.println("Read User 001:" + getData);

//U
pIndex.put(new UserInfo("002", "user002222"));

//Read ALL
EntityCursor cursor = pIndex.entities();
try {
    Iterator i = cursor.iterator();
    while (i.hasNext()) {
        System.out.println("Cursor data:" + i.next());
    }
} finally {
    cursor.close();
}

//D
String pkey = "003";
boolean flag = pIndex.delete(pkey);
System.out.println("delete object :" + pkey + " result:" + flag);

//关闭store
if (store != null) {
    store.close();
    store = null;
}

@Entity
@SuppressWarnings("serial")
class UserInfo implements Serializable {

    @PrimaryKey
    private String userId;
    private String userName;

    public UserInfo() {
    }

    public UserInfo(String userId, String userName) {
        this.userId = userId;
        this.userName = userName;
    }

    public String getUserId() {
        return userId;
    }
    public void setUserId(String userId) {
        this.userId = userId;
    }
    public String getUserName() {
        return userName;
    }
    public void setUserName(String userName) {
        this.userName = userName;
    }

    @Override
    public String toString() {
        return "UserInfo [userId=" + userId + ", userName=" + userName + "]";
    }
}

（8）事务处理 

Transaction txn = mytestdb.getEnvironment().beginTransaction(null, null);
try {
    for(int i = 0; i < 5; i++) {
        mytestdb.put(txn, new DatabaseEntry(("TXN-KEY"+(i+1)).getBytes("utf-8")), new DatabaseEntry(("TXN-VALUE"+(i+1)).getBytes("utf-8")));
    }
} catch (DatabaseException e) {
    if (txn != null) {
        txn.abort();
        txn = null;
    }
    throw e;
} finally {
    if (txn != null) {
        txn.commit();
    }
}

（9）游标操作 

//C
for(int i = 0; i < 5; i++) {
    mytestdb.put(null, new DatabaseEntry(("KEY"+(i+1)).getBytes("utf-8")), new DatabaseEntry(("VALUE"+(i+1)).getBytes("utf-8")));
}

DatabaseEntry key = new DatabaseEntry();
DatabaseEntry value = new DatabaseEntry();

//D (by Cursor)
Transaction txn = mytestdb.getEnvironment().beginTransaction(null, null);
Cursor cursor1 = mytestdb.openCursor(txn, null);
OperationStatus result1 = cursor1.getFirst(key, value, null);
while (result1 == OperationStatus.SUCCESS) {
    if ("VALUE3".equals(new String(value.getData(), "utf-8"))) {
        cursor1.delete();
    }
    result1 = cursor1.getNext(key, value, null);
}

if (cursor1 != null) {
    cursor1.close();
}
if (txn != null) {
    txn.commit();
}

//R (by Cursor)
Cursor cursor2 = mytestdb.openCursor(null, null);
OperationStatus result2 = cursor2.getFirst(key, value, null);

while (result2 == OperationStatus.SUCCESS) {
    System.out.println("Cursor Read Value:" + new String(value.getData(), "utf-8"));
    result2 = cursor2.getNext(key, value, null);
}

if (cursor2 != null) {
    cursor2.close();
}

（10）关闭database 

//关闭database
if (mytestdb != null) {
    mytestdb.close();
    mytestdb = null;
}

（11）关闭database环境 

//关闭database环境
if (mydbEnv != null) {
    mydbEnv.sync();//把数据同步到磁盘中去
    mydbEnv.cleanLog();//清理日志
    mydbEnv.close();
    mydbEnv = null;
}

•  (2.2 MB)
• 下载次数: 35

•  (4 MB)
• 下载次数: 32

•  (1.6 MB)
• 下载次数: 22